We Keep Coding

Reflected XSS on BufferedReader readLine() method

I have scanned a java web project with the Checkmarx tool, and the analysis marks an XSS vulnerability in a method where a web service is executed that responds a JSON, the vulnerability is in the line while((output = Encode.forJava(br.readLine())) != null) {, specifically in br.readLine().
Checkmarx says:
The attacker would be able to alter the returned web page by simply
providing modified data in the user input readLine, which is read by
the NetClientPost method. This input then flows through the code
straight to the output web page, without sanitization.
This can enable a Reflected Cross-Site Scripting (XSS) attack.
I tried with OWASP for Java, implementing the method Encode.forJava(), but the vulnerability continues to appear in the analysis. This is the implementation of the method:
public String NetClientPost (String urlSer, String param){
String result ="";
try {
InetAddress ip = InetAddress.getLocalHost();
String host = ip.getHostAddress();
doTrustToCertificates();
URL url = new URL(urlSer);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setConnectTimeout(2000);
String input = param;
String output = "";
try(OutputStream os = conn.getOutputStream()) {
os.write(input.getBytes());
os.flush();
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP code : " + conn.getResponseCode());
}
try (BufferedReader br = new BufferedReader(new InputStreamReader((conn.getInputStream())))) {
while ((output = Encode.forJava(br.readLine())) != null) {//LINE OF CHECKMARX XSS
result += output;
}
}
}
conn.disconnect();
return result;
} catch (MalformedURLException e) {
return result;
} catch (IOException e) {
return result;
} catch (Exception e) {
return result;
}
}
Any have an idea of how to solve this?

Try parsing the incoming data as JSON and then serializing it back to a string before sending it on.
That way you can be sure that your method only returns JSON to the client. If for some reason, your incoming data isn't JSON, then your method would encounter an error parsing the JSON, which you can then handle appropriately.
Encode.forJava isn't a helpful method to use here: it is used to encode a string to be inserted into a Java string literal.

output = Encode.forHtmlAttribute(br.readLine() works for me

Symbols not copied correctly while getting the page source

I am trying to get the source of a webpage using the following code:
public static String getFile(String sUrl) throws ClientProtocolException, IOException {
DefaultHttpClient httpclient = new DefaultHttpClient();
StringBuilder b = new StringBuilder();
// Prepare a request object
HttpGet httpget = new HttpGet(sUrl);
// Execute the request
HttpResponse response = httpclient.execute(httpget);
// Examine the response status
System.out.println(response.getStatusLine());
//status code should be 200
if (response.getStatusLine().getStatusCode() != 200) {
return null;
}
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
InputStream instream = entity.getContent();
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(instream));
// do something useful with the response
String s = reader.readLine();
while (s != null) {
b.append(s);
b.append("\n");
s = reader.readLine();
}
} catch (IOException ex) {
// In case of an IOException the connection will be released
// back to the connection manager automatically
throw ex;
} catch (RuntimeException ex) {
// In case of an unexpected exception you may want to abort
// the HTTP request in order to shut down the underlying
// connection and release it back to the connection manager.
httpget.abort();
throw ex;
} finally {
// Closing the input stream will trigger connection release
instream.close();
}
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
return b.toString();
}
It works fine, but certain symbols like &nbsp, - , single quotes etc. are not getting copied correctly.
I try to save the page source as a text/html type into amazon s3 and display it by accessing the page saved in the s3 server.
The symbols that I mentioned above are displayed as � .
Is there any solution for this?

You need to make sure that you are reading the content with the encoding of the page, else your system default encoding would be used (which apparently is not the correct one as you have seen):
BufferedReader reader = new BufferedReader(
new InputStreamReader(instream, entity.getContentEncoding()));

First one need to specify the encoding that the InputStreamReader uses. Your version of the constructor takes the default encoding on your system.
The encoding could be delivered in the headers. It defaults to ISO-8859-1 but (Latin-1) but in reality is Windows-1252 (Windows Latin-1).
String charset = "Windows-1252"; // Can be used as default.
String enc = entity.getContentEncoding(); // Or from Content-Type.
if (enc != null) {
charset = enc;
}
BufferedReader reader = new BufferedReader(
new InputStreamReader(instream, charset));
For HTML entities, apache has:
String s = ...
s = StringEscapeUtils.unescapeHTML4(s);

Sending UTF-8 string using HttpURLConnection

till now I've used the following code snippet in order to send and recieve JSON strings:
static private String sendJson(String json,String url){
HttpClient httpClient = new DefaultHttpClient();
String responseString = "";
try {
HttpPost request = new HttpPost(url);
StringEntity params =new StringEntity(json, "UTF-8");
request.addHeader("content-type", "application/json");
request.setEntity(params);
HttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
responseString = EntityUtils.toString(entity, "UTF-8");
}catch (Exception ex) {
ex.printStackTrace();
// handle exception here
} finally {
httpClient.getConnectionManager().shutdown();
}
return responseString;
}
The code above worked perfect even if the json string contained UTF-8 chars, and everything worked fine.
For several reasons I had to change the way I send HTTP post requests and use HttpURLConnection instead apache's HttpClient. Here's my code:
static private String sendJson(String json,String url){
String responseString = "";
try {
URL m_url = new URL(url);
HttpURLConnection conn = (HttpURLConnection)m_url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("content-type", "application/json");
DataOutputStream outputStream = new DataOutputStream(conn.getOutputStream());
outputStream.writeBytes(json);
outputStream.flush();
outputStream.close();
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line+"\n");
}
br.close();
responseString = sb.toString();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return responseString;
}
This code works well for normal English characters, but doesn't seem to support UTF-8 characters in the json string, since it fails each time. (when sending json to server, server crushes saying that utf8 cant decode a certain byte, but when recieving utf8 json from server I think it does work since I manage to view the special characters).
Server didn't change at all and worked fine with previous code, so the problem is 100% on this new code snippet.
Any idea how to fix the json string sending so it would support UTF 8? Thanks

I think the problem is in this part:
DataOutputStream outputStream = new DataOutputStream(conn.getOutputStream());
outputStream.writeBytes(json);
outputStream.flush();
outputStream.close();
Instead of doing this you need to encode json as UTF-8
and send those bytes which represent the UTF-8 encoding.
Try using this:
Charset.forName("UTF-8").encode(json)
See:
Charset.encode
An even simpler approach is to use e.g. a BufferedWriter wrapping an
OutputStreamWriter. The OutputStreamWriter knows about its own encoding
and so it will do the work for you (the encoding work of the json String).
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(conn.getOutputStream(), "UTF-8"));
bw.write(json);
bw.flush();
bw.close();

When writing a String to an output stream (bytes), you need to specify the encoding to do the conversion.
One way to do is to wrap the output stream in a OutputStreamWriter that will use UTF-8 charset for the encoding.
conn.setRequestProperty("content-type", "application/json; charset=utf-8");
Writer writer = new BufferedWriter(new OutputStreamWriter(conn.getOutputStream(), "UTF-8"));
writer.write(json);
writer.close();
The flush() is also optional if you call close().
Another option, as mentionned by peter.petrov is to first convert your String to bytes (in memory) and then output the byte array to your output stream.
And to make it obvious on the server side, you can pass the charset used in the content-type header ("content-type", "application/json; charset=utf-8").

StringEntity uses a Charset to make sure the encoding is right. It does ~that:
byte[] content = s.getBytes(charset);
Without much change in your code, your write can be:
outputStream.write(json.getBytes("UTF-8"));
As for your read, there is no point in using a BufferedReader with readLine, except for normalizing the end of line. It is much slower than other methods, as it requires to read each byte individually.
EntityUtils does mostly that:
Reader reader = new InputStreamReader(conn.getInputStream(), "UTF-8");
StringBuilder buffer = new StringBuilder();
char[] tmp = new char[1024];
int l;
while((l = reader.read(tmp)) != -1) {
buffer.append(tmp, 0, l);
}
responseString = buffer.toString();

Grabbing JSON works from one link, not from another

I'm doing a simple JSON grab from two links with the same code. I'm doing it two separate times, so the cause of my issue isn't because they're running into each other or something.
Here is my code:
#Override
protected String doInBackground(Object... params) {
try {
URL weatherUrl = new URL("my url goes here");
HttpURLConnection connection = (HttpURLConnection) weatherUrl
.openConnection();
connection.connect();
responseCode = connection.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
InputStream inputStream = connection.getInputStream();
Reader reader = new InputStreamReader(inputStream);
int contentLength = connection.getContentLength();
char[] charArray = new char[contentLength];
reader.read(charArray);
String responseData = new String(charArray);
Log.v("test", responseData);
When I try this with:
http://www.google.com/calendar/feeds/developer-calendar#google.com/public/full?alt=json
I get an error of having an array lenth of -1
For this link:
http://api.openweathermap.org/data/2.5/weather?id=5815135
It returns fine and I get a log of all of the JSON. Does anyone have any idea why?
Note: I tried stepping through my code in debug mode, but I couldn't catch anything. I also downloaded a Google chrome extension for parsing json in the browser and both urls look completely valid. I'm out of ideas.

Log this: int contentLength = connection.getContentLength();
I don't see the google url returning a content-length header.
If you just want String output from a url, you can use Scanner and URL like so:
Scanner s = new Scanner(new URL("http://www.google.com").openStream(), "UTF-8").useDelimiter("\\A");
out = s.next();
s.close();
(don't forget try/finally block and exception handling)
The longer way (which allows for progress reporting and such):
String convertStreamToString(InputStream is) throws UnsupportedEncodingException {
BufferedReader reader = new BufferedReader(new
InputStreamReader(is, "UTF-8"));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null)
sb.append(line + "\n");
} catch (IOException e) {
// Handle exception
} finally {
try {
is.close();
} catch (IOException e) {
// Handle exception
}
}
return sb.toString();
}
}
and then call String response = convertStreamToString( inputStream );

What's the recommended way to get the HTTP response as a String when using Apache's HTTP Client? [duplicate]

This question already has answers here:
How to get HttpClient returning status code and response body?
(5 answers)
Closed 2 years ago.
I've just begun using Apache's HTTP Client library and noticed that there wasn't a built-in method of getting the HTTP response as a String. I'm just looking to get it as as String so that i can pass it to whatever parsing library I'm using.
What's the recommended way of getting the HTTP response as a String? Here's my code to make the request:
public String doGet(String strUrl, List<NameValuePair> lstParams) {
String strResponse = null;
try {
HttpGet htpGet = new HttpGet(strUrl);
htpGet.setEntity(new UrlEncodedFormEntity(lstParams));
DefaultHttpClient dhcClient = new DefaultHttpClient();
PersistentCookieStore pscStore = new PersistentCookieStore(this);
dhcClient.setCookieStore(pscStore);
HttpResponse resResponse = dhcClient.execute(htpGet);
//strResponse = getResponse(resResponse);
} catch (ClientProtocolException e) {
throw e;
} catch (IOException e) {
throw e;
}
return strResponse;
}

You can use EntityUtils#toString() for this.
// ...
HttpResponse response = client.execute(get);
String responseAsString = EntityUtils.toString(response.getEntity());
// ...

You need to consume the response body and get the response:
BufferedReader br = new BufferedReader(new InputStreamReader(httpresponse.getEntity().getContent()));
And then read it:
String readLine;
String responseBody = "";
while (((readLine = br.readLine()) != null)) {
responseBody += "\n" + readLine;
}
The responseBody now contains your response as string.
(Don't forget to close the BufferedReader in the end: br.close())

You can do something like:
Reader in = new BufferedReader(
new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
Using the reader you will be able to build your string. But if you are using SAX you can give the stream to the parser directly. This way you will not have to create the string and your memory footprint will be lower too.

In terms of conciseness of code it might be using the Fluent API like this:
import org.apache.http.client.fluent.Request;
[...]
String result = Request.Get(uri).execute().returnContent().asString();
The documentation warns though that this approach is not ideal in terms of memory consumption.