Gzip decompressing returns random characters - java

I'm trying to download a XML file that is gzip compressed from a server for that I use the following code:
HttpParams httpParameters = new BasicHttpParams();
int timeoutConnection = 3000;
HttpConnectionParams.setConnectionTimeout(httpParameters, timeoutConnection);
int timeoutSocket = 5000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutSocket);
DefaultHttpClient client = new DefaultHttpClient(httpParameters);
HttpGet response = new HttpGet(urlData);
client.addRequestInterceptor(new HttpRequestInterceptor() {
#Override
public void process(HttpRequest request, HttpContext context) {
// Add header to accept gzip content
if (!request.containsHeader("Accept-Encoding")) {
request.addHeader("Accept-Encoding", "gzip");
}
}
});
client.addResponseInterceptor(new HttpResponseInterceptor() {
#Override
public void process(HttpResponse response, HttpContext context) {
// Inflate any responses compressed with gzip
final HttpEntity entity = response.getEntity();
final Header encoding = entity.getContentEncoding();
if (encoding != null) {
for (HeaderElement element : encoding.getElements()) {
if (element.getName().equalsIgnoreCase("gzip")) {
response.setEntity(new InflatingEntity(response.getEntity()));
break;
}
}
}
}
});
ResponseHandler<String> responseHandler = new BasicResponseHandler();
return client.execute(response, responseHandler);
InflatingEntity method:
private static class InflatingEntity extends HttpEntityWrapper {
public InflatingEntity(HttpEntity wrapped) {
super(wrapped);
}
#Override
public InputStream getContent() throws IOException {
return new GZIPInputStream(wrappedEntity.getContent());
}
#Override
public long getContentLength() {
return -1;
}
}
If I remove everything related to Gzip compression and replace the compressed XML file from the server with a normal XML everything works fine, but after I implement the Gzip compression I get the compressed string:
Does anyone knows what is missing in my code to get the decompressed XML?

I have solved the problem, my response didn't have an entity so the code was not decompressing the response since that part of the code was not being reached, here is the modification in the responseinterceptor:
client.addResponseInterceptor(new HttpResponseInterceptor() {
#Override
public void process(HttpResponse response, HttpContext context) {
response.setEntity(new InflatingEntity(response.getEntity()));
}
});

Related

Java httppost file post print upload speed

With this java function would upload files to the server.
while the upload is in progress, upload speed and the size of the data sent How do I print?
public void PostFile() {
try {
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
File file = new File("D:/sqldata.zip");
HttpPost httppost = new HttpPost("http://localhost/upload.php");
HttpEntity httpEntity = MultipartEntityBuilder.create()
.addBinaryBody("userfile", file, ContentType.create("application/zip"), file.getName())
.build();
httppost.setEntity(httpEntity);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
System.out.println(response.getStatusLine());
if (resEntity != null) {
System.out.println(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
resEntity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
} catch (Exception e) {
System.out.println("hata: "+e);
}
}
Thank you to everyone. I solved the problem. In this way, you can print the data sent instantly sent.
main.java
public void post(String url, File sendFile) throws UnsupportedEncodingException, IOException {
long start = System.nanoTime();
HttpParams params = new BasicHttpParams();
params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, true);
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpClient client = new DefaultHttpClient(params);
HttpPost post = new HttpPost(url);
MultipartEntity reqEntity = new MultipartEntity(){
ProgressListener listener;
public void writeTo(final OutputStream outstream) throws IOException {
super.writeTo(new UploadStream(outstream,listener));
}
};
reqEntity.addPart("userfile", new FileBody(sendFile));
post.setEntity(reqEntity);
HttpResponse response = client.execute(post);
if (response != null) {
HttpEntity resEntity = response.getEntity();
System.out.println(response.getStatusLine());
if (resEntity != null) {
System.out.println(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
resEntity.consumeContent();
}
}
}
UploadStream.java
public class UploadStream extends FilterOutputStream {
final ProgressListener listener;
private long transferred;
public UploadStream(final OutputStream out, final ProgressListener listener)
{
super(out);
this.listener = listener;
this.transferred = 0;
}
public void write(byte[] b, int off, int len) throws IOException
{
out.write(b, off, len);
this.transferred += len;
System.out.println(this.transferred/1024+" KB");
}
public void write(int b) throws IOException
{
out.write(b);
this.transferred++;
}
}

DefaultClientConnection automatically gets shutdown

I am new to HttpClient. I am using DefaultHttpClient (which is said to be thread-safe).
In my app I have created two threads which have been scheduled to execute every 10 minutes simultaneously. Sometimes I found that the DefaultClientConnection automatically gets shutdown.
What could be the possible reasons?
Though DefaultHttpClient is thread-safe, shall I need to use PoolingClientConnectionManager in this scenario ?
You should use PoolingClientConnectionManager.
Also you must use IdleConnectionMonitorThread for monitor idle connection.
some code from my source :
private final PoolingClientConnectionManager connectionManager;
private final IdleConnectionMonitorThread connectionMonitorThread = null;
private final DefaultHttpClient httpclient;
initialization:
final HttpParams params = new BasicHttpParams();
final HttpProtocolParamBean paramsBean = new HttpProtocolParamBean(params);
paramsBean.setVersion(HttpVersion.HTTP_1_1);
paramsBean.setContentCharset("UTF-8");
paramsBean.setUseExpectContinue(false);
params.setBooleanParameter(HttpConnectionParams.STALE_CONNECTION_CHECK, false);
params.setIntParameter("http.socket.timeout", 20000);
params.setIntParameter("http.connection.timeout", 30000);
params.setBooleanParameter("http.protocol.handle-redirects", true);
params.setBooleanParameter(HttpConnectionParams.TCP_NODELAY, true);
params.setIntParameter(HttpConnectionParams.SOCKET_BUFFER_SIZE, 32 * 1024);
params.setParameter("http.protocol.cookie-policy", CookiePolicy.BROWSER_COMPATIBILITY);
// params.setParameter("http.useragent", "Crawler Airupt(http://www.airupt.com/)");
params.setParameter("http.useragent",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.79 Safari/537.1");
params.setParameter("http.language.Accept-Language", "en-us");
params.setParameter("http.protocol.content-charset", "UTF-8");
params.setParameter("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
params.setParameter("Cache-Control", "max-age=0");
final SSLSocketFactory socketFactory = SSLSocketFactory.getSocketFactory();
final SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme("https", 443, socketFactory/* SSLSocketFactory.getSocketFactory() */));
connectionManager = new PoolingClientConnectionManager(schemeRegistry);
connectionManager.setDefaultMaxPerRoute(500000);
connectionManager.setMaxTotal(2000000);
httpclient = new DefaultHttpClient(connectionManager, params);
httpclient.setKeepAliveStrategy(new ConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
final HeaderElementIterator it = new BasicHeaderElementIterator(response
.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
final HeaderElement he = it.nextElement();
final String param = he.getName();
final String value = he.getValue();
if (value != null && param.equalsIgnoreCase("timeout")) {
try {
return Long.parseLong(value) * 1000;
} catch (final NumberFormatException ignore) {
}
}
}
return 30 * 1000;
}
});
httpclient.setRedirectStrategy(new DefaultRedirectStrategy());
httpclient.addRequestInterceptor(new HttpRequestInterceptor() {
#Override
public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
if (!request.containsHeader("Accept-Encoding")) {
request.addHeader("Accept-Encoding", "gzip");
}
}
});
httpclient.addResponseInterceptor(new HttpResponseInterceptor() {
#Override
public void process(final HttpResponse response, final HttpContext context) throws HttpException, IOException {
response.setEntity(new BufferedHttpEntity(response.getEntity()));
final HttpEntity entity = response.getEntity();
final Header ceheader = entity.getContentEncoding();
if (ceheader != null) {
final HeaderElement[] codecs = ceheader.getElements();
for (int i = 0; i < codecs.length; i++) {
if (codecs[i].getName().equalsIgnoreCase("gzip")) {
response.setEntity(new GzipDecompressingEntity(response.getEntity()));
return;
}
}
}
}
});
startConnectionMonitorThread();
few addition methods:
private synchronized void startConnectionMonitorThread() {
if (connectionMonitorThread == null) {
connectionMonitorThread = new IdleConnectionMonitorThread(connectionManager);
}
connectionMonitorThread.start();
}
private synchronized void stopConnectionMonitorThread() {
if (connectionMonitorThread != null) {
connectionMonitorThread.shutdown();
connectionManager.shutdown();
}
}
public void shutdown() {
stopConnectionMonitorThread();
final ClientConnectionManager cm = httpclient.getConnectionManager();
if (cm != null) {
httpclient.getConnectionManager().shutdown();
}
}
Using:
final HttpGet httpGet = new HttpGet(url);
final HttpResponse response = httpclient.execute(httpGet);
final StatusLine statusLine = response.getStatusLine();
final int responseCode = statusLine.getStatusCode();
if (responseCode >= 300) {
logger.error(" {}. Received statusCode {}", url, responseCode);
httpGet.abort();
//throw some exception;
}
final HttpEntity entity = response.getEntity();
if (entity == null) {
//throw some exception or ignore;
}
responseBody = EntityUtils.toString(entity);
This code/params optimized for Crawler. to fast receive a lot of pages. with using gzip(if possible) and https if require without using cookies. For addition cookies you need to add cookieStore like httpclient.setCookieStore();

HttpRequestHandler : Sending a file in response

I m working with httpCore in order to create my own basic web server. Upon receiving a certain request from the user a file transfer operation has to be initiated. I m handling the request with HttpRequestHandler. My code looks like this
private HttpRequestHandler mRequestHandler = new HttpRequestHandler()
{
#Override
public void handle(HttpRequest request, HttpResponse response, HttpContext httpContext) throws HttpException, IOException
{
try{
HttpEntity entity=null;
String contentType="text/html";
entity = new EntityTemplate(new ContentProducer()
{
public void writeTo(final OutputStream outstream)throws IOException
{
OutputStreamWriter writer = new OutputStreamWriter(outstream, "UTF-8");
String resp = "Server is up and running";
writer.write(resp);
writer.flush();
}
});
((EntityTemplate)entity).setContentType(contentType);
response.setEntity(entity);
}catch(Exception e)
{
e.printStackTrace();
}
}
};
This is a very basic response , however What I m looking to transfer a file from the server to the client machine. How can I send a file in response ? Thanks
I figured it out myself. If someone stumbles across the same problem, here is how I got it working.
private HttpRequestHandler mRequestHandler = new HttpRequestHandler()
{
#Override
public void handle(HttpRequest request, HttpResponse response, HttpContext context)
throws HttpException, IOException
{
try{
File file = new File("/mnt/sdcard/Music/song.mp3");
FileEntity body = new FileEntity(file, "audio/mpeg");
response.setHeader("Content-Type", "application/force-download");
response.setHeader("Content-Disposition","attachment; filename=song.mp3");
response.setEntity(body);
}catch(Exception e)
{
e.printStackTrace();
}
}
};

Android: send post that has no response

In my app, I need to send all sorts of POST requests to a server. some of those requests have responses and others don't.
this is the code I'm using to send the requests:
private static final String TAG = "Server";
private static final String PATH = "http://10.0.0.2:8001/data_connection";
private static HttpResponse response = null;
private static StringEntity se = null;
private static HttpClient client;
private static HttpPost post = null;
public static String actionKey = null;
public static JSONObject sendRequest(JSONObject req) {
try {
client = new DefaultHttpClient();
actionKey = req.getString("actionKey");
se = new StringEntity(req.toString());
se.setContentEncoding(new BasicHeader(HTTP.CONTENT_ENCODING, "application/json"));
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
post = new HttpPost(PATH);
post.setEntity(se);
Log.d(TAG, "http request is being sent");
response = client.execute(post);
Log.d(TAG, "http request was sent");
if (response != null) {
InputStream in = response.getEntity().getContent();
String a = convertFromInputStream(in);
in.close();
return new JSONObject(a);
}
} catch (UnsupportedEncodingException e) {
Log.d(TAG, "encoding request to String entity faild!");
e.printStackTrace();
} catch (ClientProtocolException e) {
Log.d(TAG, "executing the http POST didn't work");
e.printStackTrace();
} catch (IOException e) {
Log.d(TAG, "executing the http POST didn't work");
e.printStackTrace();
} catch (JSONException e) {
Log.d(TAG, "no ActionKey");
e.printStackTrace();
}
return null;
}
private static String convertFromInputStream(InputStream in)
throws IOException {
BufferedReader br = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = br.readLine()) != null) {
sb.append(line);
}
return (sb.toString());
}
This is the code for the AsyncTask class that sends the request:
class ServerRequest extends AsyncTask<JSONObject, Void, JSONObject> {
#Override
protected JSONObject doInBackground(JSONObject... params) {
JSONObject req = params[0];
JSONObject response = Server.sendRequest(req);
return response;
}
#Override
protected void onPostExecute(JSONObject result) {
// HANDLE RESULT
super.onPostExecute(result);
}
}
my problem starts when the server doesn't return a response. the AsyncTask thread stays open even after the work is done because the HTTPClient never closes the connection.
Is there a way to not wait for a response? this is something that will definitely add a lot of overhead to the server since all the Android apps trying to connect to it will keep the connection alive, and will probably cause many problems on the app itself.
Basically, what I'm looking for is a method that will allow me to send to POST message and kill the connection right after the sending of the request since there is no response coming my way.
Just, Set ConnectionTimeOut with HttpClient Object, (Code is for your understanding in your case it may be different)
int TIMEOUT_MILLISEC = 30000;
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, TIMEOUT_MILLISEC);
HttpConnectionParams.setSoTimeout(httpParams, TIMEOUT_MILLISEC);
HttpClient httpclient = new DefaultHttpClient(httpParams);
HttpPost httppost = new HttpPost(url);
httppost.addHeader("Content-Type", "application/json");
Now, It will terminate the Connection after TimeoOut you defined. But be sure this will throw TimeOutException so You have to handle this exception in your HttpRequest.. (Use Try -catch)
EDIT: Or you can use HttpRequestExecutor class.
From class HttpRequestExecutor of package org.apache.http.protocol
protected boolean canResponseHaveBody (HttpRequest request, HttpResponse response)
Decide whether a response comes with an entity. The implementation in this class is based on RFC 2616. Unknown methods and response codes are supposed to indicate responses with an entity.
Derived executors can override this method to handle methods and response codes not specified in RFC 2616.

Httpclient 4, error 302. How to redirect?

I want to access one site that first requires an (tomcat server) authentication and then log in with a POST request and keep that user to see the site's pages. I use Httpclient 4.0.1
The first authentication works fine but not the logon that always complains about this error: "302 Moved Temporarily"
I keep cookies & I keep a context and yet nothing. Actually, it seems that the logon works, because if I write incorrect parameters or user||password, I see the login page. So I guess what doesn't work is the automatic redirection.
Following my code, which always throws the IOException, 302:
DefaultHttpClient httpclient = new DefaultHttpClient();
CookieStore cookieStore = new BasicCookieStore();
httpclient.getParams().setParameter(
ClientPNames.COOKIE_POLICY, CookiePolicy.BROWSER_COMPATIBILITY);
HttpContext context = new BasicHttpContext();
context.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
//ResponseHandler<String> responseHandler = new BasicResponseHandler();
Credentials testsystemCreds = new UsernamePasswordCredentials(TESTSYSTEM_USER, TESTSYSTEM_PASS);
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
testsystemCreds);
HttpPost postRequest = new HttpPost(cms + "/login");
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new BasicNameValuePair("pUserId", user));
formparams.add(new BasicNameValuePair("pPassword", pass));
postRequest.setEntity(new UrlEncodedFormEntity(formparams, "UTF-8"));
HttpResponse response = httpclient.execute(postRequest, context);
System.out.println(response);
if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK)
throw new IOException(response.getStatusLine().toString());
HttpUriRequest currentReq = (HttpUriRequest) context.getAttribute(
ExecutionContext.HTTP_REQUEST);
HttpHost currentHost = (HttpHost) context.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);
String currentUrl = currentHost.toURI() + currentReq.getURI();
System.out.println(currentUrl);
HttpEntity entity = response.getEntity();
if (entity != null) {
long len = entity.getContentLength();
if (len != -1 && len < 2048) {
System.out.println(EntityUtils.toString(entity));
} else {
// Stream content out
}
}
For 4.1 version:
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.setRedirectStrategy(new DefaultRedirectStrategy() {
public boolean isRedirected(HttpRequest request, HttpResponse response, HttpContext context) {
boolean isRedirect=false;
try {
isRedirect = super.isRedirected(request, response, context);
} catch (ProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (!isRedirect) {
int responseCode = response.getStatusLine().getStatusCode();
if (responseCode == 301 || responseCode == 302) {
return true;
}
}
return isRedirect;
}
});
For HttpClient 4.3.x :
HttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();
In later versions of HttpCLient (4.1+), you can just do this:
DefaultHttpClient client = new DefaultHttpClient()
client.setRedirectStrategy(new LaxRedirectStrategy())
LaxRedirectStrategy will automatically redirect HEAD, GET, and POST requests. For a stricter implementation, use DefaultRedirectStrategy.
You have to implement custom redirection handler that will indicate that response to POST is a redirection. This can be done by overriding isRedirectRequested() method as shown below.
DefaultHttpClient client = new DefaultHttpClient();
client.setRedirectHandler(new DefaultRedirectHandler() {
#Override
public boolean isRedirectRequested(HttpResponse response, HttpContext context) {
boolean isRedirect = super.isRedirectRequested(response, context);
if (!isRedirect) {
int responseCode = response.getStatusLine().getStatusCode();
if (responseCode == 301 || responseCode == 302) {
return true;
}
}
return isRedirect;
}
});
In later version of HttpClient, the class name is DefaultRedirectStrategy, but similar solution can be used there.
httpclient.setRedirectHandler(new DefaultRedirectHandler());
See HttpClient Javadoc
Redirects are not handled automatically by HttpClient 4.1 for other methods than GET and PUT.
Extend the DefaultRedirectStrategy class and override the methods.
#Override
protected URI createLocationURI(String arg0) throws ProtocolException {
// TODO Auto-generated method stub
return super.createLocationURI(arg0);
}
#Override
protected boolean isRedirectable(String arg0) {
// TODO Auto-generated method stub
return true;
}
#Override
public URI getLocationURI(HttpRequest arg0, HttpResponse arg1,
HttpContext arg2) throws ProtocolException {
// TODO Auto-generated method stub
return super.getLocationURI(arg0, arg1, arg2);
}
#Override
public HttpUriRequest getRedirect(HttpRequest request,
HttpResponse response, HttpContext context)
throws ProtocolException {
URI uri = getLocationURI(request, response, context);
String method = request.getRequestLine().getMethod();
if (method.equalsIgnoreCase(HttpHead.METHOD_NAME)) {
return new HttpHead(uri);
} else {
return new HttpPost(uri);
}
}
#Override
public boolean isRedirected(HttpRequest request, HttpResponse response,
HttpContext context) throws ProtocolException {
// TODO Auto-generated method stub
return super.isRedirected(request, response, context);
}
in this case isRedirectable method will always return true and getRedirect method will return post request in place of get request.

Categories