I'm currently trying to build an OSGi service that sends a POST request to a defined API. This API is used to virus-scan a file which is contained in the request body (JSON) as Base64 string.
For this, I am using Apache HttpClient contained in Adobe AEM uberjar v6.4.0
My current implementation works fine for smaller files (<2 MB), but as filesize gets bigger, the behaviour gets strange:
When I upload a 9 MB file, the request executes for ~1 minute, then gets a HTTP400 as response and afterwards retrys the request 7 times.
I tried to use a timeout with the request. If the timeout is below 60.000ms, a TimeoutException is thrown, if it's greater than 60.000ms, I get a HTTP400 Bad Request. I guess the latter is the APIs fault which I need to clarify.
However, in both cases after the exception is thrown, httpClient retries the request and I have not been able to prevent that since. I'm struggeling with many deprecated "HowTo's" on the web and now I'm here.
I have shortened the code a bit, as it's somehow big (mostly removing debug messages and some "if... return false" at the beginning). My Code:
public boolean isAttachmentClean(InputStream inputStream) throws IOException, JSONException, ServiceUnavailableException {
//prevent httpClient from retrying in case of an IOException
final HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(0, false);
HttpClient httpClient = HttpClients.custom().setRetryHandler(retryHandler).build();
HttpPost httpPost = new HttpPost(serviceUrl);
httpPost.setHeader("accept", "application/json");
//set some more headers...
//set timeout for POST from OSGi Config
RequestConfig timeoutConfig = RequestConfig.custom()
.setConnectionRequestTimeout(serviceTimeout)
.setConnectTimeout(serviceTimeout)
.setSocketTimeout(serviceTimeout)
.build();
httpPost.setConfig(timeoutConfig);
//create request body data
String requestBody;
try {
requestBody = buildDataJson(inputStream);
} finally {
inputStream.close();
}
HttpEntity requestBodyEntity = new ByteArrayEntity(requestBody.getBytes("UTF-8"));
httpPost.setEntity(requestBodyEntity);
//Execute and get the response.
HttpResponse response = httpClient.execute(httpPost);
if (response.getStatusLine().getStatusCode() != HttpServletResponse.SC_OK){
httpPost.abort();
throw new ServiceUnavailableException("API not available, Response Code was "+ response.getStatusLine().getStatusCode());
}
HttpEntity entity = response.getEntity();
boolean result = false;
if (entity != null) {
InputStream apiResult = entity.getContent();
try {
// check the response from the API (Virus yes or no)
result = evaluateResponse(apiResult);
} finally {
apiResult.close();
}
}
return result;
}
"buildDataJson()" simply reads the InputStream and creates a JSON needed for the API call.
"evaluateResponse()" also reads the InputStream, transforms it into a JSON and checks for a property named "Status:" "Clean".
I'd appreciate any tipps on why this request is retried over and over again.
/edit: So far, I found that Apache httpClient has some default mechanism that retries a request in case of an IOException - which is what I get here. Still, I have not found a solution on how to deactivate these retries.
I have a test server and a client application. Server is built with Vert.x 3.2.1 (users Netty 4.0.33.Final) and client with Apache httpclient 4.5.1.
Have configured server to accept maxHeaderSize as 8KB. As I send requests from client I see that requests bearing header sizes of around 3KB starts failing with below exception on server.
io.netty.handler.codec.TooLongFrameException: HTTP header is larger than 8192 bytes
Sharing the server and client code for reference.
Server:
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
HttpServerOptions options = new HttpServerOptions();
options.setMaxHeaderSize(1024*8);
HttpServer httpServer = (HttpServer) vertx.createHttpServer(options );
httpServer.requestHandler(req -> {
System.out.println(req.headers().names());
for(String h : req.headers().names()){
System.out.println(h+": "+req.headers().get(h).length()+": "+(h.length()+req.headers().get(h).length()));
}
req.response().setStatusCode(200).end();
});
httpServer.listen(6565);
}
Client:
public static void main(String[] args) {
try {
CloseableHttpClient httpclient = HttpClients.custom().build();
HttpGet get = new HttpGet("http://localhost:6565");
String headerValue = generateHeader(3045);
get.addHeader("myheader", headerValue);
HttpResponse response = httpclient.execute(get);
String resp = IOUtils.toString(response.getEntity().getContent());
System.out.println(response.getStatusLine());
httpclient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
generateHeader() is a method that generates a String of specified length - when input with 3, result is "aaa".
Apache http client also adds some more headers - Host, Connection, User-Agent and Accept-Encoding. Along with all these, The behavior I notice is that the server fails when I pass 3045 or above as input to generateHeader(), but succeeds for any lower value.
Though configured is 8k, I fail to understand why failures occur for such lower values.
This may be a bug in Netty. Please open a bug-report in Netty issue tracker and provide details how to reproduce it:
https://github.com/netty/netty/issues
I have this in my MainActivity.java file:
public static void getLatestVersion() {
try {
String myUri = "http://www.stonequest.de/version.php";
HttpClient httpClient = new DefaultHttpClient();
HttpGet get = new HttpGet(myUri);
HttpResponse response = httpClient.execute(get);
String Latest = EntityUtils.toString(response.getEntity());
System.out.println(Latest);
} catch (Exception e) {
e.printStackTrace();
}
}
It gets the text, but it also adds some special characters at the beginning of the text.
This is what I get:
0.1.572
I wish to retrieve the version without any characters as shown by going to the endpoint http://www.stonequest.de/version.php
0.1.572
So how can I fix it?
Querying this URL with Firefox, I can see the same characters in the response body (with Firebug). You should mention the characterset in the toString() method, and use the same as on the server side. Preferably set both to "UTF-8".
Authentication error: Unable to respond to any of these challenges: {} Android - 401 Unauthorized
I have taken reference from this link
Authentication Error when using HttpPost with DefaultHttpClient on Android
I am working on android app in that backed in Drupal. In that I am sending data from android app to drupal website - webservice in JSON format. Now I can read JSON data from Drupal webservice and writing it in my android application. But facing problem in writing on drupal from android, it generates response with status code
401 Unauthorized
From android native app it generates 401 , while from phonegap-from android when I initiate AJAX request it works perfectly & writes an article or page on drupal website. so that means webservice work perfectly &
my phonegap android app works perfectly there is problem with Android native JAVA application
I am running my android application on Android2.3.4 -> Samsung Galaxy
S Plus - Samsung GT-I9001
here is my code for java android.
==============================
String url = "XXX";
strResponse1 = makeWebForPostIdea(url,title,body);
public static String makeWebForPostIdea(String url, String title,String body)
{
JSONStringer jsonobject = null;
JSONObject json = null;
JSONObject jsonnode = null;
DefaultHttpClient client = new DefaultHttpClient();
Credentials creds = new UsernamePasswordCredentials("username", "password");
client.getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT), creds);
HttpPost post = new HttpPost(url);
System.out.println("value of the post =============> "+post);
try {
JSONObject jsonvalue = new JSONObject();
jsonvalue.put("value", body.toString());
JSONArray array = new JSONArray();
array.put(jsonvalue);
jsonnode = new JSONObject();
jsonnode.put("und", array);
System.out.println("######2 jsonnode=======>"+jsonnode.toString());
} catch (JSONException e3) {
// TODO Auto-generated catch block
e3.printStackTrace();
}
try {
jsonobject = new JSONStringer().array().object().key("und").object().key("0").object().key("value").value(body).endObject().endObject().endObject().endArray();
System.out.println("=============>"+jsonobject);
} catch (JSONException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("type","page"));
params.add(new BasicNameValuePair("title",title));
params.add(new BasicNameValuePair("language","und"));
params.add(new BasicNameValuePair("body",jsonobject.toString()));
System.out.println("value of the params =============> "+params);
UrlEncodedFormEntity formEntity = null;
try {
formEntity = new UrlEncodedFormEntity(params);
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
post.setEntity(formEntity);
try {
HttpResponse response = client.execute(post);
int statusCode = response.getStatusLine().getStatusCode();
System.out.println("=========> statusCode post idea=====> "+statusCode);
if (statusCode == HttpStatus.SC_OK)
{
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
return iStream_to_String(is);
}
else
{
return "Hello This is status ==> :"+String.valueOf(statusCode);
}
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
public static String iStream_to_String(InputStream is1) {
BufferedReader rd = new BufferedReader(new InputStreamReader(is1), 4096);
String line;
StringBuilder sb = new StringBuilder();
try {
while ((line = rd.readLine()) != null) {
sb.append(line);
}
rd.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String contentOfMyInputStream = sb.toString();
return contentOfMyInputStream;
}
}
}
here is the logcat that I am getting.
08-09 12:41:29.063: I/System.out(336): value of the post =============> org.apache.http.client.methods.HttpPost#4053c3c8
08-09 12:41:29.093: I/System.out(336): ######2 jsonnode=======>{"und": [{"value":"ddddddd"}]}
08-09 12:41:29.093: I/System.out(336): =============>[{"und":{"0":{"value":"ddddddd"}}}]
08-09 12:41:29.103: I/System.out(336): value of the params =============> [type=page, title=hhhh, language=und, body=[{"und":{"0":{"value":"ddddddd"}}}]]
08-09 12:41:30.913: W/DefaultRequestDirector(336): Authentication error: Unable to respond to any of these challenges: {}
08-09 12:41:30.913: I/System.out(336): =========> statusCode post idea=====> 401
08-09 12:41:30.924: I/System.out(336): =========> Response from post idea => Hello This is status ==> :401
Here is my PhoneGap Ajax request it works perfectly.
$('#page_node_create_submit').live('click',function(){
var title = $('#page_node_title').val();
//if (!title) { alert('Please enter a title.'); return false; }
var body = $('#page_node_body').val();
//if (!body) { alert('Please enter a body.'); return false; }
// BEGIN: drupal services node create login (warning: don't use https if you don't have ssl setup)
$.ajax({
url: "XXX",
type: 'post',
data: 'node[type]=page&node[title]=' + encodeURIComponent(title) + '&node[language]=und&node[body][und][0][value]=' + encodeURIComponent(body),
dataType: 'json',
error: function(XMLHttpRequest, textStatus, errorThrown) {
alert('page_node_create_submit - failed to login');
console.log(JSON.stringify(XMLHttpRequest));
console.log(JSON.stringify(textStatus));
console.log(JSON.stringify(errorThrown));
},
success: function (data) {
$.mobile.changePage("index.html", "slideup");
}
});
// END: drupal services node create
return false;
});
=================================================================================
Edit :
I have tried various methods for Apache httpclient for my error.During this time I have done some research and searched on google and found out some interesting stuff.
1st thing that I found it that Android-Google Officially does not recommend Apache HttpClient that I am using in my code. Check this link. In that Link message from Jesse Wilson from the Dalvik team. In that they suggest to use HttpURLConnection instead of DefaultHttpClient and also written that Android team will no longer develop Apache httpclient . so its the older version that I am using.
http://android-developers.blogspot.in/2011/09/androids-http-clients.html
2nd thing that I have found form this link. It suggests that Android is shipping with Apache's HttpClient 4.0 Beta2, which has a pitfall, when it comes to Basic Authentication. The Authentication method that I am using is of HttpClient 3.x , that I have found out from this link.
check the link.
http://hc.apache.org/httpclient-3.x/authentication.html#Preemptive_Authentication
So the version issue.
http://dlinsin.blogspot.in/2009/08/http-basic-authentication-with-android.html
I have also found some links with potential solution of this problem.
http://ogrelab.ikratko.com/using-newer-version-of-httpclient-like-4-1-x/
Apache HttpClient 4.1 on Android
What version of Apache HTTP Client is bundled in Android 1.6?
From these links , I made a conclusion that if we upgrade the Apache HttpClient to latest stable version , then this problem can be solved.
But this is directly no possible , as Android Team has officially stopped the support for the Apache httpclient.
With this link It could be possible to solve. I have not tried it but I am working on it.
It is the library that can help in upgrading httpclient version in Android.
http://code.google.com/p/httpclientandroidlib/
The other solution could be using HttpURLConnection .I am also working on it.
But most people here on stackoverflow and Internet seems to using DefaultHttpCLient with Android. And ofcourse it is also working with me throughout my application including login,registration,reading from server and session and other functionality.Just it is not working with directly post some article to my server-Drupal website.
It works perfectly with POST request during registration of user on server.
So friends , any suggestions regarding this ? why it is not working just with posting article ?
How come it works from the PhoneGap but not Java. PhoneGap runs the app in a web container and so already has been authenticated - and you have all the right cookies. AJAX will share the same session that and everything 'just works'.
However HTTPClient is a completely different - you are initiating a brand new HTTP session and everything has to be right.
A few comments on how HTTP Auth works:
There are several HTTP authentication methods - and it's the web server that chooses which. Before going any further, check your Drupal configuration to work out whether it is:
Basic Auth (username and password). Everyone and their dog supports this, but it's very insecure. See http://en.wikipedia.org/wiki/Basic_access_authentication for more details
Digest (username and challenge/response hash with MD5. This is more secure but much more complex. Note that MD5 is generally considered weak now. Many libraries support it, including Apache. See http://en.wikipedia.org/wiki/Digest_access_authentication for more details
NTLM (a variant of Kerberos/SPEGNO) which is implemented on IIS. This is not generally supported from Java, although HTTPClient does profess to - but using a different Credentials object. See http://hc.apache.org/httpclient-3.x/authentication.html#NTLM
(Note also that the web container has the 'smarts' to be able to try different authentication methods as requested by the server, all behind the scenes)
Also check the Drupal web logs. A few pointers:
Did you see the HTTPClient connect at all. And is the URL going to the correct resource. Always worth checking from the server's perspective...
Did it go to the right server? One example of what could go wrong: Are you using IP
addresses in the URL against a multi-homed web server, so the request goes to the wrong server?
Check that the authentication sent by the client pre-emptively is the correct type (basic, digest, NTLM)
Let me know if this helps. If not, and you can give more details as per this post, I can follow up with more advice.
You might try to check: How to do http post using apache httpclient with web authentication?
It uses a HttpInterceptor to inject the authentication data, when required
I'd suggest to test first the PHP side out the app. There are several ways to make your own calls including headers and auth. From curl to GraphicalHttpClient (http://itunes.apple.com/us/app/graphicalhttpclient/id433095876?mt=12 , I personally use that and it works decently). There some other options like REST client debugger (https://addons.mozilla.org/en-US/firefox/addon/restclient/)
This way you'll be able to test your call in so many ways which is pain doing directly in the client (sometimes it's just changing from http to https or adding the type of your token in the Authorization header and that's much easier to be madeo n the fly).
Once everything works as expected, reproduce the same call, headers and body in your client and you are ready to go.
I was running into the same "DefaultRequestDirector: Authentication error: Unable to respond to any of these challenges: {}" problem with Drupal services using the loopj android-async-http library (highly recommend it).
The key to the solution for me was in Jose L Ugia's comment in one of the answers regarding paying special attention to the JSON output from Drupal. I was trying to catch a JSONObject but the real message was in array format "["Wrong username or password."]". Switching to JSONArray caught the error properly and allowed me to handle it. In your case I believe it is because you are not posting the login credentials as drupal services expects it.
You should remember with Drupal Services you should do system/connect and grab the session, followed by user/login (and the user/password passed in as parameters) and grab the session and then all your subsequent requests should work. This is why I like using the loopj library because it makes all these requests more manageable. Here is a very basic example of connecting to drupal with loopj. All subsequent posts are easily done using the params.
public class Loopj {
private static final String TAG = "loopj";
private static AsyncHttpClient client = new AsyncHttpClient();
private final PersistentCookieStore myCookieStore;
public Loopj(Context context) {
myCookieStore = new PersistentCookieStore(context);
client.setCookieStore(myCookieStore);
client.addHeader("Content-type", "application/json");
}
public void systemConnect(String uri) throws JSONException {
client.post(uri + "/endpoint/system/connect", new JsonHttpResponseHandler() {
#Override
public void onSuccess(JSONObject json) {
Log.i("TAG", "Connect success =" + json.toString());
}
#Override
public void onFailure(Throwable e, String response) {
Log.e("TAG", "Connect failure");
}
});
}
public void userLogin(String uri) throws JSONException {
RequestParams params = new RequestParams();
params.put("username", username);
params.put("password", password);
client.post(uri + "/endpoint/user/login", params, new JsonHttpResponseHandler() {
#Override
public void onSuccess(JSONArray response) {
Log.i("TAG", "Login success =" + response.toString());
}
#Override
public void onFailure(Throwable e, JSONArray json) {
Log.e("TAG", "Login failure");
}
});
}
Greetings,
I am trying to setup a server connection from my BlackBerry Application . I was able to get a response code on the status of the server. Now i have a few values which i have to POST to the server
Its like a registration page values(username, password, age ) have to be sent to the server .
ConnectionFactory connFact = new ConnectionFactory();
ConnectionDescriptor connDesc;
connDesc = connFact.getConnection(url);
if (connDesc != null)
{
HttpConnection httpConn;
httpConn = (HttpConnection)connDesc.getConnection();
try
{
final int iResponseCode = httpConn.getResponseCode();
UiApplication.getUiApplication().invokeLater(new Runnable()
{
public void run()
{
Dialog.alert("Response code: " + Integer.toString(iResponseCode));
}
});
}
catch (IOException e)
{
System.err.println("Caught IOException: " + e.getMessage());
}
}
Thats the code i used to get the response code.
I would appreciate it if someone could help me how i can make a POST request to the server..
the server url for status was company.com/app/version/stats
when it for register it would be
company.com/app/register
Thank you
What type of a POST do you use? If you are just passing key-value pairs, then it should be a POST of a "application/x-www-form-urlencoded" content-type.
So, what lacks youe code is:
1). Set a proper content-type on your connection:
httpConn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
2). Prepare the content to be sent to the server via the POST:
URLEncodedPostData encPostData = new URLEncodedPostData("UTF-8", false);
encPostData.append("username", username);
encPostData.append("password", password);
encPostData.append("age", age);
byte[] postData = encPostData.toString().getBytes("UTF-8");
3). Set content-length for the connection (this step may be optional - try without this first, probably the BB OS is smart enough to set this automatically):
httpConn.setRequestProperty("Content-Length", String.valueOf(postData.length));
4). Open an OutputStream and write the content to it (the code is simplified):
OutputStream os = httpConn.openOutputStream();
os.write(postData);
os.flush();
...
httpConn = (HttpConnection)connDesc.getConnection();
httpConn.setRequestMethod(HttpConnection.POST);
httpConn.setRequestProperty("username",name);
httpConn.setRequestProperty("password",pass);
....