We Keep Coding

Code works in Xamarin Android but doestn't work in Java (HttpPost JSON)

I started developing in Xamarin, and then decided that license may be a bit expensive for playing around, so I'm transferring my code to java.
I have a small chunk that performs a POST with a JSON object, and it works in Xamarin and doest work in Java.
Xamarin:
var client = new HttpClient ();
var content = new FormUrlEncodedContent(new Dictionary<string, string>() {
{"action", "getEpisodeJSON"},
{"episode", "11813"}
});
client.DefaultRequestHeaders.Referrer = new Uri(link);
var resp = client.PostAsync("http://www.ts.kg/ajax", content).Result;
var repsStr = resp.Content.ReadAsStringAsync().Result;
dynamic res = JsonConvert.DeserializeObject (repsStr);
Android:
HttpClient httpclient = new DefaultHttpClient();
// 2. make POST request to the given URL
HttpPost httpPost = new HttpPost("http://www.ts.kg/ajax");
String json = "";
// 3. build jsonObject
JSONObject jsonObject = new JSONObject();
jsonObject.accumulate("action", "getEpisodeJSON");
jsonObject.accumulate("episode", "11813");
// 4. convert JSONObject to JSON to String
json = jsonObject.toString();
// 5. set json to StringEntity
StringEntity se = new StringEntity(json);
// 6. set httpPost Entity
httpPost.setEntity(se);
// 7. Set some headers to inform server about the type of the content
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
httpPost.addHeader("Referer", "http://www.ts.kg");
// 8. Execute POST request to the given URL
HttpResponse httpResponse = httpclient.execute(httpPost);
// 9. receive response as inputStream
InputStream inputStream = httpResponse.getEntity().getContent();
// 10. convert inputstream to string
String result;
if(inputStream != null)
result = convertInputStreamToString(inputStream);
What is a correct way to make such a POST in Android?
UPD
Current problem is that i'm getting an empty result string;
private static String convertInputStreamToString(InputStream inputStream) throws IOException{
BufferedReader bufferedReader = new BufferedReader( new InputStreamReader(inputStream));
String line = "";
String result = "";
while((line = bufferedReader.readLine()) != null)
result += line;
inputStream.close();
return result;
}

I ended up catching all requests of my device via Fiddle (good tutorial is here: http://tech.vg.no/2014/06/04/how-to-monitor-http-traffic-from-your-android-phone-through-fiddler/)
The difference was in cookie, so I used and HttpContex variable as described here:
Android HttpClient Cookie
And I also had a different Content-Type, so I set this header manually as this:
httpPost.setHeader("Content-Type", "application/x-www-form-urlencoded");

How can I forward HttpResponse to an open client socket? (Java Program acts as a proxy)

I am using httpclient lib from apache. I managed to get an HttpResponse by sending a GET request to the server. Now what I am trying to do is to send that response that I got to a clientSocket output stream.
So basically I want to send whatever I received from the server to the open client connection. Since I am using HttpClient I get the response in the form of an HttpResponse object. I tried the following:
private void forwardRequest(String header, String url){
try {
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet(url);
CloseableHttpResponse response;
//Adding the request headers to httpget
String lines[] = header.split("\\n");
for (String str : lines) {
String parts[] = str.split(":", 2);
httpget.addHeader(parts[0], parts[1]);
}
HttpResponse respone;
response = httpclient.execute(httpget);
//It works till here I can read from the response and print out the html page
//But after this I don't know how to send it to client
OutputStream bos = clientSocket.getOutputStream();
PrintWriter pw = new PrintWriter(bos);
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
while ((line = rd.readLine()) != null) {
pw.println(line);
//bos.write(line.getBytes()); //This also doesn't work
}
response.close();
}
Also clientSocket is a global variable which is associcated with a ServerSocket like:
clientSocket = serverSocket.accept();
I don't expect a full solution. Just point me in the right direction.. Thanks a ton!
EDIT:
I tried the following based on what EJP suggested.. It's still not working. I was wondering if it was correctly implemented?
int portNumber = 8012; // port on which the program listens
ServerSocket serverSocket =
new ServerSocket(portNumber); //the socket at which the program listens
Socket clientSocket = serverSocket.accept(); //clientSocket of the program
Socket toServer = new Socket("localhost", 8089); //proxy server to which program connects
PrintWriter out =
new PrintWriter(toServer.getOutputStream(), true);
PrintWriter outClient =
new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(clientSocket.getInputStream()));
BufferedReader inServer = new BufferedReader(
new InputStreamReader(toServer.getInputStream()));
) {
String inputLine;
while ((inputLine = in.readLine()) != null) {
out.println(inputLine); //Writing to proxy server
outClient.println(inServer.readLine()); //writing back to original request sender
System.out.println(inputLine);
}

The client made an HTTP request, so it will be expecting an HTTP response. If the global clientSocket is just a raw TCP socket and not an HttpClient, then you need to add the HTTP response protocol header yourself.
You have the content from the server, you'll want to first return an HTTP response 200 OK, then empty line with carriage return + linefeed (CR+LF), then Content-length: , then the document. If you are just proxying text documents, then you could convert to a string here, but otherwise, I would just pass the mime type, charset, and entity through as the raw bytes as the web server responded, that way you can proxy any document, including images or binary files.
It will look something like this:
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: length
<html> ...
</html>
To pass the http headers through from the server:
HttpEntity entity = response.getEntity();
// technically you should check the HTTP response rather than assume it is a 200
int statusCode = httpResp.getStatusLine().getStatusCode();
if(statusCode != 200)
... // do something with non 200 responses ?
clientSocket.write("HTTP/1.1 200 OK\r\n");
Header[] responseHeaders = response.getAllHeaders();
for(Header header : responseHeaders) {
clientSocket.write(header.toString() + "\r\n");
}
clientSocket.write("\r\n"); // empty line required
// Use BufferedInputStream to deal in bytes
BufferedInputStream input = new BufferedInputStream(entity.getContent());
byte[] buf = new byte[8192];
int bytesRead;
while ((bytesRead = input.read(buf, 8192)) > 0) {
clientSocket.write(buf, bytesRead);
}
I say "something like this", don't take this literal, I doubt it compiles. I don't have dev station in front of me, but this is the general idea.
NOTE: Since you are using the Apache client lib, you should be able to use the specific HTTP client instead of writing the raw protocol. This will abstract the HTTP protocol away somewhat. I'll update the answer later if nobody else provides a better one.

If you're just forwarding requests and responses you don't have any need to engage in the HTTP protocol at all beyond the first line of the request. If the client knows you're the proxy you will get either a GET request with the full URL or else a CONNECT request ditto. All you have to do is connect to the target and then just copy bytes in both directions simultaneously.

How to parse a large amount of data from a JSON webservice

I have a URL which returns a large amount of data in response, but I can't get a full response from the webservice URL. The webservice responds as follows:
\"},
{\"minute_usage_end_time\":\"11:59\",\"minute_usage_start_time\":\"11:00\",\"kwh_usage\":\"0\",\"meter_reading_date\":\"08-02-2011\"},{...
What should I do?

StringEntity entity = new StringEntity(jsonObject.toString(),HTTP.UTF_8);
httpPost.setEntity(entity);
HttpResponse response = httpClient.execute(httpPost);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
Log.v("mas\n", reader.readLine());
String jsonResultStr = reader.readLine();

HttpGet in Android/Java with GZip encoding

I am running MVC 4 on my server and to save a bit of data for my users I figured I would enable GZip encoding, to do this I simply used:
(C#)
Response.AddHeader("Content-Encoding", "gzip");
Response.Filter = new GZipStream(Response.Filter, CompressionMode.Compress);
In my android application I use:
(Java)
String response = "";
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
try {
HttpResponse execute = client.execute(httpGet);
InputStream content = execute.getEntity().getContent();
BufferedReader buffer = new BufferedReader(
new InputStreamReader(content));
String s = "";
while ((s = buffer.readLine()) != null) {
response += s;
}
} catch (Exception e) {
e.printStackTrace();
}
return response;
When I use GZip the Java code nuts out and causes GC to run, I was never patient enough to wait for it to return.
When I took off GZip from the server it ran perfectly fine. The function to get the response returns straight away with no problem.
I tried adding this to the java code:
httpGet.addHeader("Accept-Encoding", "gzip");
With no success.
Question is, is there something I'm not getting? Can I not put the response in a stream if it is using GZip? Am I meant to use the stream and uncompress it after?
What am I doing wrong?

Instead of using
DefaultHttpClient client = new DefaultHttpClient();
you can use
ContentEncodingHttpClient client = new ContentEncodingHttpClient();
which is a subclass of DefaultHttpClient and supports GZIP content.
You need Apache HttpClient 4.1 for this.
If you have Apache HttpClient 4.2, you should use
DecompressingHttpClient client = new DecompressingHttpClient();
if you have Apache HttpClient 4.3, you should use the HttpClientBuilder

Displaying NON-ASCII Characters using HttpClient

So, i am using this code to get the whole HTML of a website. But i dont seem to get non-ascii characters with me. all i get is diamonds with question mark.
characters like this: å, appears like this: �
I doubt its because of the charset, what could it then be?
Log.e("HTML", "henter htmlen..");
String url = "http://beep.tv2.dk";
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
client.getParams().setParameter(CoreProtocolPNames.HTTP_ELEMENT_CHARSET, "UTF-8");
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
Header h = HeaderValueFormatter
response.addHeader(header)
String html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while((line = reader.readLine()) != null)
{
str.append(line);
}
in.close();
//b = false;
html = str.toString();

Thank you. This worked (in case others have the issue):
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
client.getParams().setParameter(CoreProtocolPNames.HTTP_ELEMENT_CHARSET, "iso-8859-1");
HttpGet request = new HttpGet(url);
request.setHeader("Accept-Charset", "iso-8859-1, unicode-1-1;q=0.8");
HttpResponse response = client.execute(request);
String html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in,"iso-8859-1"));

use the new InputStreamReader(in, "UTF-8") constructor
Set the Accept-Charset request header to, say, Accept-Charset: iso-8859-5, unicode-1-1;q=0.8
Make sure the page opens properly in a browser. If it does not, then it might be a server-side issue.
If none of the above works, check other headers using firebug (or similar tool)

This really helped me get started, but I was having the same problem while reading a text file. It was fixed using the following command:
BufferedReader br = new BufferedReader(new InputStreamReader(new
FileInputStream(fileName), "iso-8859-1"));
...and of course, the HTTP Response needs to have the encoding set as well:
response.setCharacterEncoding("UTF-8");
Thanks for the help!