after a recent findbugs (FB) run it complains about a: Security - HTTP Response splitting vulnerability The following code triggers it:
String referrer = req.getParameter("referrer");
if (referrer != null) {
launchURL += "&referrer="+(referrer);
}
resp.sendRedirect(launchURL);
Basically the 'referrer' http parameter contains an url, to which, when clicking on a back button in our application the browser returns to. It is appended to the url as a parameter. After a bit research i know that i need to sanitize the referrer url. After a bit more research i found the esapi project which seem to offer this kind of functionality:
//1st canonicalize
import org.owasp.esapi.Encoder;
import org.owasp.esapi.Validator;
import org.owasp.esapi.reference.DefaultEncoder;
import org.owasp.esapi.reference.DefaultValidator;
[...]
Encoder encoder = new DefaultEncoder(new ArrayList<String>());
String cReferrer = encoder.canonicalize(referrer);
However I didn't figure out how to detect e.g. jscript code or other stuff which doesn't belong to a referrer url. So how can I achieve that with esapi?
I tried:
Validator validator = new DefaultValidator(encoder);
validator.isValidInput("Redirect URL",referrer,"HTTPParameterValue",512,false);
however this doesn't work. What I need is a function which results in:
http://www.google.com (ok)
http://www.google.com/login?dest=http://google.com/%0D%0ALocation: javascript:%0D%0A%0D%0Aalert(document.cookie) (not ok)
Or is it enough to call the following statement?
encoder.encodeForHTMLAttribute(referrer);
Any help appreciated.
Here's my final solution if anyone is interested. First I canonicalize and then URL decode the string. If a CR or LF exists (\n \r) I just cut of the rest of that potential 'attack' string starting with \n or \r.
String sanitize(String url) throws EncodingException{
Encoder encoder = new DefaultEncoder(new ArrayList<String>());
//first canonicalize
String clean = encoder.canonicalize(url).trim();
//then url decode
clean = encoder.decodeFromURL(clean);
//detect and remove any existent \r\n == %0D%0A == CRLF to prevent HTTP Response Splitting
int idxR = clean.indexOf('\r');
int idxN = clean.indexOf('\n');
if(idxN >= 0 || idxR>=0){
if(idxN<idxR){
//just cut off the part after the LF
clean = clean.substring(0,idxN);
}
else{
//just cut off the part after the CR
clean = clean.substring(0,idxR);
}
}
//re-encode again
return encoder.encodeForURL(clean);
}
Theoretically i could have later verified the value against 'HTTPParameterValue' regex which is defined in the ESAPI.properties however it didn't like colon in the http:// and I didn't investigated further.
And one more remark after testing it: Most modern browser nowadays (Firefox > 3.6, Chrome, IE10 etc.) detect this kind of vulnerability and do not execute the code...
I think you have the right idea, but are using an inappropriate encoder. The Referer [sic] header value is really a URL, not an HTML attribute, so you really want to use:
encoder.encodeForURL(referrer);
-kevin
I would suggest white-listing approach wherein you check the referrer string only for permissible characters. Regex would be a good option.
EDIT:
The class org.owasp.esapi.reference.DefaultEncoder being used by you is not really encoding anything. Look at the source code of the method encodeForHTMLAttribute(referrer) here at grepcode. A typical URL encoding (encoding carriage return and line feed) too wont help.
So the way forward would be device some validation logic which checks for valid set of characters. Here is another insightful article.
The accepted answer will not work if in case there is "\n\r" in the string.
Example:
If I have string: "This is str\n\rstr", it returns "This is str\nstr"
Rectified version of above accepted answer is:
String sanitizeCarriageReturns(String value) {
int idxR = value.indexOf('\r');
int idxN = value.indexOf('\n');
if (idxN >= 0 || idxR >= 0) {
if ((idxN > idxR && idxR<0) || (idxR > idxN && idxR>=0)) {
value = value.substring(0, idxN);
} else if (idxN < idxR){
value = value.substring(0, idxR);
}
}
return value;
}
Related
I'm trying to use Spring's webflux to create an http endpoint to stream github users using Github's api. I tried to do what is described here and here but it seems that the expand is not fetching the second page of results from github's api. What am I doing wrong?
Here's the code I currently have:
#RestController
#RequestMapping("/user")
public class GithubUserController {
private static final String GITHUB_API_URL = "https://api.github.com";
private final WebClient client = WebClient.create(GITHUB_API_URL);
#GetMapping(value = "/search/stream", produces = MediaType.APPLICATION_STREAM_JSON_VALUE)
public Flux<GithubUser> search(
#RequestParam String location,
#RequestParam String language,
#RequestParam String followers) {
return fetchUsers(
uriBuilder ->
uriBuilder
.path("/search/users")
.queryParam(
"q",
String.format(
"location:%s+language:%s+followers:%s", location, language, followers))
.build())
.expand(
response -> {
var links = response.headers().header("link");
Pattern p = Pattern.compile("<(.*)>; rel=\"next\".*");
for (String link : links) {
Matcher m = p.matcher(link);
if (m.matches()) {
return client.get().uri(m.group(1)).exchange();
}
}
return Flux.empty();
})
.flatMap(response -> response.bodyToFlux(GithubUsersResponse.class))
.flatMap(parsedResponse -> Flux.fromIterable(parsedResponse.getItems()))
.log();
}
private Mono<ClientResponse> fetchUsers(Function<UriBuilder, URI> url) {
return client.get().uri(url).exchange();
}
}
I can see that the regex for the second page works because if I add a print inside the if, it gets printed, however if I test this on the browser or on postman I only get the results for the first page of results returned by github's api:
{"login":"chrisbanes","id":"227486"}
{"login":"keyboardsurfer","id":"336005"}
{"login":"lucasr","id":"730395"}
{"login":"hitherejoe","id":"3879281"}
{"login":"StylingAndroid","id":"933874"}
{"login":"rstoyanchev","id":"401908"}
{"login":"RichardWarburton","id":"328174"}
{"login":"slightfoot","id":"906564"}
{"login":"tomwhite","id":"85085"}
{"login":"jstrachan","id":"30140"}
{"login":"wakaleo","id":"55986"}
{"login":"cesarferreira","id":"277426"}
{"login":"kevalpatel2106","id":"20060162"}
{"login":"jodastephen","id":"213212"}
{"login":"caveofprogramming","id":"19751656"}
{"login":"AlmasB","id":"3594742"}
{"login":"scottyab","id":"404105"}
{"login":"makovkastar","id":"1076309"}
{"login":"salaboy","id":"271966"}
{"login":"blundell","id":"655860"}
{"login":"PierfrancescoSoffritti","id":"7457011"}
{"login":"0xddr","id":"4354177"}
{"login":"irsdl","id":"1798313"}
{"login":"andreban","id":"1733592"}
{"login":"TWiStErRob","id":"2906988"}
{"login":"geometer","id":"344328"}
{"login":"neomatrix369","id":"1570917"}
{"login":"nebraslabs","id":"32421477"}
{"login":"lucko","id":"8352868"}
{"login":"isabelcosta","id":"11148726"}
The link header in the Github API provides the URI in an escaped format. The String you pass to client.get().uri() should be unescaped - so it escapes the escaped string, and you end up with a URL that returns nothing.
Instead, you probably want to use something similar to:
if (m.matches()) {
return client.get().uri(URI.create(m.group(1))).exchange();
}
Side note - your regular expression will probably want to account for any number of characters before the "next" link as well otherwise you'll be unable to go past the second page, so you probably want to prepend .* to that:
Pattern p = Pattern.compile(".*<(.*)>; rel=\"next\".*");
Second side note - Github's API is rate limited (heavily rate limited if you're unauthenticated), so you may well run into those rate limits. You'll probably want to handle that situation elegantly somehow, but that's a reasonably big topic that's beyond the scope of this question.
I need to pass ^ like a value of parameter in URL. For example:
http://localhost:8080/myapp/books?filter=^
But have an error:java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986. I've read, that I need to encode. Have something like this, but it still doesn't work. I also try to add
System.setProperty("tomcat.util.http.parser.HttpParser.requestTargetAllow" ^ ");
but for ^ it doen't help.
I have a controller:
#RequestMapping("/books")
public String getBooks(#RequestParam(value = "filter") String filter, Model model)
throws UnsupportedEncodingException {
String par = URLEncoder.encode(nameFilter,"UTF-8");
List<Books> books = (List<Books>) booksService.findAll(filter);
model.addAttribute("books", books);
return "getBooks";
}
}
Try encoding the URI before doing a request to your REST Api
For instance, when you're using JS read this:
https://www.w3schools.com/jsref/jsref_encodeURI.asp
On Java: Java URL encoding: URLEncoder vs. URI
Goodluck!
Try to follow this, it will help:
https://secure.n-able.com/webhelp/NC_9-1-0_SO_en/Content/SA_docs/API_Level_Integration/API_Integration_URLEncoding.html
#Mark’s comment is also correct.
For the below mentioned code, I'm getting Trust Boundary Violation in the CheckMarx report.
Error description -
Method 'getResponse' gets user input from element request. This element’s value flows through the code without being properly sanitized or validated and is eventually stored in the server-side Session object, in 'parseRequest' method.**
Code -
#Context
HttpHeaders httpHeader;
void parseRequest(SomeRequestType inputRequest) {
HashMap<String, Data> requestData = inputRequest.getRequestData(httpHeader);
if (requestData != null) {
if (Strings.isNullOrEmpty(inputRequest.getId())) {
Data data = requestData.get("data");
var dataID = data.getID();
if ((dataID != null) && Pattern.matches("[0-9]+", dataID)) {
inputRequest.setId(dataID);
ThreadContext.put("ID", dataID);
}
}
}
}
I am getting checkmarx vulnerability at below line for without being properly sanitized or validated
ThreadContext.put("ID", dataID);
Could some please help me, how to properly sanitize the above line.
If you know for sure that dataID is a number, convert it to integer/long right away, like this:
int dataIDasNumber = Integer.parseInt(dataID);
And use it like int/long here:
inputRequest.setId(dataIDasNumber);
ThreadContext.put("ID", dataIDasNumber);
Then you don't need to do this:
Pattern.matches...
And your checkmarx violation should go away.
I have a String, and I want to validate whether it is a valid CSS value or not. In the documentation of AntiSamy, I found that I might be able to use CSSValidator.isValidProperty (http://javadox.com/org.owasp/antisamy/1.4/org/owasp/validator/css/CssValidator) to do so. However, the type of the second param requires LexicalUnit.
Is there another way to validate a String with AnitSamy?
I think what you want is the CssScanner.
/****** pull out style tag from html *****/
Pattern p = Pattern.compile("<style>([\\s\\S]+?)</style>");
Matcher m = p.matcher(validHTML);
// if we find a match, get the group
if (m.find()) {
// get the matching group
codeGroup = m.group(1);
}
/****** block for checking all css for validity *****/
InternalPolicy policy = null;
try {
policy = (InternalPolicy) InternalPolicy.getInstance("antisamy-ebay.xml");
} catch (PolicyException e) {
e.printStackTrace();
}
ResourceBundle messages = ResourceBundle.getBundle("AntiSamy", Locale.getDefault());
CssScanner scanner = new CssScanner(policy, messages);
CleanResults results = scanner.scanStyleSheet(codeGroup, Integer.MAX_VALUE);
validCSS = results.getCleanHTML().toString();
That is the part of the code that worked for me. Let me know if any of this does not work for you, I have variables declared at the top of the code because I am also handling html validation in here too. So some variables are not in this code. But it should point you in the right direction. Also, you need a policy in place, I chose the ebay policy, this guides the whitelist of what the css will allow for the resulting output. I have not used the CssValidator, so I am not sure how they compare, but CssScanner does a great job of giving back clean css.
I am trying to do some stuff with replacing String containing some URL to a browser compatible linked URL.
My initial String looks like this :
"hello, i'm some text with an url like http://www.the-url.com/ and I need to have an hypertext link !"
What I want to get is a String looking like :
"hello, i'm some text with an url like http://www.the-url.com/ and I need to have an hypertext link !"
I can catch URL with this code line :
String withUrlString = myString.replaceAll(".*://[^<>[:space:]]+[[:alnum:]/]", "HereWasAnURL");
Maybe the regexp expression needs some correction, but it's working fine, need to test in further time.
So the question is how to keep the expression catched by the regexp and just add a what's needed to create the link : catched string
Thanks in advance for your interest and responses !
Try to use:
myString.replaceAll("(.*://[^<>[:space:]]+[[:alnum:]/])", "HereWasAnURL");
I didn't check your regex.
By using () you can create groups. The $1 indicates the group index.
$1 will replace the url.
I asked a simalir question: my question
Some exemples: Capturing Text in a Group in a regular expression
public static String textToHtmlConvertingURLsToLinks(String text) {
if (text == null) {
return text;
}
String escapedText = HtmlUtils.htmlEscape(text);
return escapedText.replaceAll("(\\A|\\s)((http|https|ftp|mailto):\\S+)(\\s|\\z)",
"$1$2$4");
}
There may be better REGEXs out there, but this does the trick as long as there is white space after the end of the URL or the URL is at the end of the text. This particular implementation also uses org.springframework.web.util.HtmlUtils to escape any other HTML that may have been entered.
For anybody who is searching a more robust solution I can suggest the Twitter Text Libraries.
Replacing the URLs with this library works like this:
new Autolink().autolink(plainText)
Belows code replaces links starting with "http" or "https", links starting just with "www." and finally replaces also email links.
Pattern httpLinkPattern = Pattern.compile("(http[s]?)://(www\\.)?([\\S&&[^.#]]+)(\\.[\\S&&[^#]]+)");
Pattern wwwLinkPattern = Pattern.compile("(?<!http[s]?://)(www\\.+)([\\S&&[^.#]]+)(\\.[\\S&&[^#]]+)");
Pattern mailAddressPattern = Pattern.compile("[\\S&&[^#]]+#([\\S&&[^.#]]+)(\\.[\\S&&[^#]]+)");
String textWithHttpLinksEnabled =
"ajdhkas www.dasda.pl/asdsad?asd=sd www.absda.pl maiandrze#asdsa.pl klajdld http://dsds.pl httpsda http://www.onet.pl https://www.onsdas.plad/dasda";
if (Objects.nonNull(textWithHttpLinksEnabled)) {
Matcher httpLinksMatcher = httpLinkPattern.matcher(textWithHttpLinksEnabled);
textWithHttpLinksEnabled = httpLinksMatcher.replaceAll("$0");
final Matcher wwwLinksMatcher = wwwLinkPattern.matcher(textWithHttpLinksEnabled);
textWithHttpLinksEnabled = wwwLinksMatcher.replaceAll("$0");
final Matcher mailLinksMatcher = mailAddressPattern.matcher(textWithHttpLinksEnabled);
textWithHttpLinksEnabled = mailLinksMatcher.replaceAll("$0");
System.out.println(textWithHttpLinksEnabled);
}
Prints:
ajdhkas www.dasda.pl/asdsad?asd=sd www.absda.pl maiandrze#asdsa.pl klajdld http://dsds.pl httpsda http://www.onet.pl https://www.onsdas.plad/dasda
Assuming your regex works to capture the correct info, you can use backreferences in your substitution. See the Java regexp tutorial.
In that case, you'd do
myString.replaceAll(....., "\1")
In case of multiline text you can use this:
text.replaceAll("(\\s|\\^|\\A)((http|https|ftp|mailto):\\S+)(\\s|\\$|\\z)",
"$1<a href='$2'>$2</a>$4");
And here is full example of my code where I need to show user's posts with urls in it:
private static final Pattern urlPattern = Pattern.compile(
"(\\s|\\^|\\A)((http|https|ftp|mailto):\\S+)(\\s|\\$|\\z)");
String userText = ""; // user content from db
String replacedValue = HtmlUtils.htmlEscape(userText);
replacedValue = urlPattern.matcher(replacedValue).replaceAll("$1$2$4");
replacedValue = StringUtils.replace(replacedValue, "\n", "<br>");
System.out.println(replacedValue);