UrlEncodedFormEntity doesn't encode underscore - java

I want to use a remote API from my Android device, but for some reason, the UrlEncodedFormEntity class doesn't transform the _ with %5f like the remote API seems to expect. As a consequence, using this code:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(
new BasicNameValuePair("json",
"{\"params\":{\"player_name\":\"Toto\",
\"password\":\"clearPass\"},
\"class_name\":\"ApiMasterAuthentication\",
\"method_name\":\"login\"}")
);
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8));
ResponseHandler responseHandler = new BasicResponseHandler();
httpClient.execute(httpPost, responseHandler);
send a post request to the server, with this content:
json=%7B%22params%22%3A%7B%22player_name%22%3A%22Toto%22%2C%22password%22%3A%22clearPass%22%7D%2C%22class_name%22%3A%22ApiMasterAuthentication%22%2C%22method_name%22%3A%22login%22%7D
I would like it to be like this (replacing the preivous underscore by %5F):
json=%7B%22params%22%3A%7B%22player%5Fname%22%3A%22Toto%22%2C%22password%22%3A%22clearPass%22%7D%2C%22class%5Fname%22%3A%22ApiMasterAuthentication%22%2C%22method%5Fname%22%3A%22login%22%7D
I don't have control over the API, and the official client of the API behave like this. It seems to be the expected behaviour for an URL normalization
Am I missing something? I first thought it was an UTF-8 encoding issue, but adding HTTP.UTF-8 in the constructor of UrlEncodedFormEntity doesn't solve the problem.
Thanks for your help.
EDIT: Finally, the problem didn't come from this unescape underscore. Even if the other client I tried to reproduce the behaviour escaped it, I only had to set the proper header:
httpPost.addHeader("Content-Type","application/x-www-form-urlencoded");
And the request worked just fine. Thanks everyone, and especially singh.jagmohan for his help (even if the problem was finally elsewhere)!

"_" isn't a reserved symbol for urls.
setting : Content-Type: application/x-www-form-urlencoded'
should solve the problem. Otherwise you can try replacing it, if you really need this option:
String.Replace("_", "%5f");
See percent encodeing , replace

You can try the following code, it works for me.
try {
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(serviceUrl);
MultipartEntity multipartEntity = new MultipartEntity();
// Also, in place of building JSON string as below, you can build a **JSONObject**
// and then use jsonObject.toString() while building the **StringBody** object
String requestJsonStr = "{\"params\":{\"player_name\":\"Toto\",\"password\":\"clearPass\"},\"class_name\":\"ApiMasterAuthentication\",\"method_name\":\"login\"}";
multipartEntity.addPart("json", new StringBody(requestJsonStr));
httpPost.setEntity(multipartEntity);
HttpResponse response = httpClient.execute(httpPost);
} catch (Exception ex) {
// add specific exception catch block above
// I have used this one just for code snippet
}
PS: The code snippet requires two jar files apache-mime4j-0.6.jar and httpmime-4.0.1.jar.
Hope this helps.

Related

JAVA - How to make a purchases.products.acknowledge request

I have to make a request with the POST method as described in the guide: https://developers.google.com/android-publisher/api-ref/rest/v3/purchases.products/acknowledge
My Java code currently looks like this:
httpClient = HttpClientBuilder.create().build();
post = new HttpPost("https://androidpublisher.googleapis.com/androidpublisher/v3/applications");
ArrayList<BasicNameValuePair> nvps = new ArrayList<BasicNameValuePair>();
nvps.add(new BasicNameValuePair("packageName", "com.my.app"));
nvps.add(new BasicNameValuePair("productId", productID));
nvps.add(new BasicNameValuePair("token", token));
post.setEntity(new UrlEncodedFormEntity(nvps, StandardCharsets.UTF_8));
response = httpClient.execute(post);
I receive a 404 web page, page not found, what am I wrong with my request?
Thanks to everyone who will try to help me. I love you <3
Without having knowledge about your HTTPClient...
I think you are using the wrong URL.
That is from the docs:
https://androidpublisher.googleapis.com/androidpublisher/v3/applications/{packageName}/purchases/products/{productId}/tokens/{token}:acknowledge
and your URL simply is:
https://androidpublisher.googleapis.com/androidpublisher/v3/applications
Besides that, i think you are posting the params for the URL as an entity in the payload. Instead you have to fill the variables in the url with it.
The only allowed entity in the payload is in the form of:
{
"developerPayload": string
}

How can I be sure that an HttpClient PostMethod contains UTF-8 strings in the parameters?

In our webapp, we have to send a POST request via HttpClient to an endpoint on our network, which will receive this and do some work with it. We are having trouble with character encoding, and I am having difficulties finding an answer to my question.
We have used the postMethod.getParams().setContentCharset("UTF-8") method when sending the request, but on the receiving end, it seems like the characters are still encoded in ISO 8859-1. I have determined this because when I inspect the String on the receiving side, it has garbage characters in it that go away once I follow the steps found at https://stackoverflow.com/a/16549329/1130549. Is there any extra steps I need to take on the sending end to ensure that I am actually writing characters in UTF-8 as expected? All we are doing now is using postMethod.addParameter(paramKey, paramValue) with native String objects.
Edit: Here is a very simple example of how we're sending the POST request. For what it's worth, the values are being taken from an XMLBeans object.
PostMethod postMethod = new PostMethod(url);
postMethod.getParams().setContentCharset("UTF-8");
postMethod.addParameter("key1", "value1");
postMethod.addParameter("key2", "value2");
HttpClient httpClient = new HttpClient();
int status = httpClient.executeMethod(postMethod);
EDIT
Simpler solution is to encode the value
postMethod.addParameter("key1", URLEncoder.encode("value1","UTF-8"));
To encode properly UTF-8, you can execute differently, using StringEntity and NameValuePair, e.g.:
try (CloseableHttpClient httpClient = HttpClients.custom().build()) {
URIBuilder uriBuilder = new URIBuilder(url);
HttpHost target = new HttpHost(uriBuilder.getHost(), uriBuilder.getPort(), uriBuilder.getScheme());
List<NameValuePair> nameValuePairs = new ArrayList<>();
nameValuePairs.add(new BasicNameValuePair("key1", "value1"));
nameValuePairs.add(new BasicNameValuePair("key2", "value2"));
String entityValue = URLEncodedUtils.format(nameValuePairs, StandardCharsets.UTF_8.name());
StringEntity entity = new StringEntity(entityValue, StandardCharsets.UTF_8.name());
post.setEntity(entity);
httpClient.execute(target, post);
First of all, you do need to make sure that the string that you are actually writing is encoded in UTF-8. I realized that you already know that but still double-check that it is so, as it would be the prime suspect of your problem. Also, I would recommend trying a much simpler HTTP client. Apache HTTP client (I believe that's the library that you are using) is an excellent library. But due to covering a very wide range of options it tends to be a bit bulky. So, or simple requests I would suggest a lightweight HTTP client that maybe not that comprehensive as Apache library but offers simplicity as a trade-off. Here how your code may look like:
private static void testHttpClient() {
HttpClient client = new HttpClient();
// client.setContentType("text/html; charset=utf-8");
client.setContentType("application/json; charset=utf-8");
client.setConnectionUrl("http://www.my-url.com");
String content = null;
try {
String myMessage = getMyMessage() // get the string that you want to send
content = client.sendHttpRequest(HttpMethod.POST, myMessage);
} catch (IOException e) {
content = client.getLastResponseMessage() + TextUtils.getStacktrace(e, false);
}
System.out.println(content);
}
It looks much more simple, I think. Also in the same library, there is another utility that allows you to convert any string in any language into a sequence of unicodes and vice-versa. This helped me numerous times to diagnose encoding thorny issues. For instance, if you see some gibberish symbols that could be a wrong display of a valid character or actual character loss. Here is an example of how it works:
result = "Hello World";
result = StringUnicodeEncoderDecoder.encodeStringToUnicodeSequence(result);
System.out.println(result);
result = StringUnicodeEncoderDecoder.decodeUnicodeSequenceToString(result);
System.out.println(result);
The output of this code is:
\u0048\u0065\u006c\u006c\u006f\u0020\u0057\u006f\u0072\u006c\u0064
Hello World
That might help you to check if the string you passed is valid or not. The library is called MgntUtils and could be found at Maven Central or at Github It comes as maven artifact and with sources and Javadoc. Javadoc could be found separately here
Disclaimer: The MgntUtils library is written by me

Edit a value and post to http with HttpPost

The problem:I have a few forms in the html page which I want to edit, then submit the data.
I have read about entities in HttpClient, and I came across the UrlEncodedFormEntity, which as far as I understand you add parameters to it and then you can post them. I find this ok, but I thought is there a different way to post the changed attributes, since jsoup has a convenient method to set a value in an attribute. this is what I tried using a different entity, StringEntity:
HttpPost post = new HttpPost(url);
post.setHeader("User-Agent", USER_AGENT);
post.setHeader("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
post.setHeader("Accept-Charset", "UTF-8");
post.setHeader("Cookie", getCookies());
post.setHeader("Connection", "keep-alive");
post.setHeader("Content-Type", "application/x-www-form-urlencoded");
post.setEntity(new StringEntity(updatedHTML, ContentType.TEXT_HTML));
HttpResponse response = null;
response = client.execute(post);
where updatedHTML is the full html code with the changes I want to post.
but as you guessed, its not working.
edit: I don't think it's the problem, but I also have a sumbit button, which I ignored here, should it also be considered in the updatedHTML?
Thanks for help.
Two things are wrong in your approach.
you cannot pass an html in the StringEntity as it is not the usage of the class
The StringEntity, as well as its derived classes, is intended to carry messages.
The second error is that you seem to use the library to change the html.
You need to work on what you are posting. Here an example.
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new BasicNameValuePair("your parameter name","your parameter value"));
formparams.add(new BasicNameValuePair("another parameter name","another paramete value"));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, "UTF-8");
HttpPost httppost = new HttpPost("http://localhost/");
httppost.setEntity(entity);
HttpClient httpclient = new DefaultHttpClient();
HttpResponse response = httpclient.execute(httppost);
I made some assumptions:
you have in your hands all the parameters you are passing (simply you will change the approach not working on the html but on the url)
The exception handling is not considered in my snippet. The code is a simple example to show you how to deal with forms
Note also that the UrlEncodedFormEntity will handle the parameters for you. E.g. in our example>
your parameter name=your parameter value&another parameter name=another parameter value

Android Rest Client

I found so many samples for requesting a REST API, but all together are confusing, can some one please explain me a way to use http requests.
My Requirement is, I want to get data from a REST API by providing username, pwd and a key.
What I have Used was,
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("REST API url");
post.setHeader("Content-type", "application/json");
JSONObject obj = new JSONObject();
obj.put("username", "un");
obj.put("pwd", "password");
obj.put("key","123456");
post.setEntity(new StringEntity(obj.toString(), "UTF-8"));
HttpResponse response = client.execute(post);
But the response is always null and these working fine when tested with browser tool by posting the same data.Is some thing wrong with my approach? please suggest me the correct way. Thank you
(1) Google I/O video session for developing REST clients
(2) search in android developer blog
(3) https://github.com/darko1002001/android-rest-client
Please try after that post your question,
I can share code snippet from my rest client developed based on (1) & (2)
Do not use Cloud to Device Messaging, instead use the latest cloud approach with android application development.
There is new library called Volley, which looks better than AsyncTask. It should be useful in developing RESTful clients.
You probably forgot to add the internet permission to the manifest file.
Add the following line.
<uses-permission android:name="android.permission.INTERNET" />
I thing you should try this,
HttpContext localContext = new BasicHttpContext();
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("REST API url");
post.setHeader("Content-type", "application/json");
JSONObject obj = new JSONObject();
obj.put("username", "un");
obj.put("pwd", "password");
obj.put("key","123456");
post.setEntity(new StringEntity(obj.toString(), "UTF-8"));
HttpResponse response = client.execute(post,localContext);
Hope this will help.
By any chance, is the server expecting a GET request for this operation? If so, you may want to use HttpGet instead of HttpPost.

"Illegal Characters" in URL for HttpGet in Android get double-encoded

I am trying to find a solution to this the whole evening now...
I write an app which requests data from a web server. The Server answers in JSON format.
Everything works well except when I enter a umlaut like ä into my App.
In the following I assume the request URL is http://example.com/?q= and I am searching for "Jäger"
The correct call would then be h++p://example.com/?q=J%C3%A4ger
(Sorry for plus-signs but the spam protection doesnt let me post it correctly.)
So my problem is now:
When I give my URL String encoded or unencoded over to HttpGet it will always result in a doublee-encoded URL.
The Request to my Server is then http://example.com/?q=J%25C3%25A4ger (It encodes the percent signs)
which leads to the server searching in database for J%C3%A4ger what is obviously wrong.
So my question is how can I achive that if the user enters "Jäger" my app calls the correctly encoded URL?
Thanks for any help!
Here is the currently used code... Ist probably the worst possible idea I had...
URI url = new URI("http", "//example.com/?q=" + ((EditText)findViewById(R.id.input)).getText().toString(), null);
Log.v("MyLogTag", "API Request: " + url);
HttpGet httpGetRequest = new HttpGet(url);
// Execute the request in the client
HttpResponse httpResponse;
httpResponse = defaultClient.execute(httpGetRequest);
Update: Sorry, HttpParams isn't meant for request parameters but for configuring HttpClient.
On Android, you might want to use Uri.Builder, like suggested in this other SO answer:
Uri uri = new Uri.Builder()
.scheme("http")
.authority("example.com")
.path("someservlet")
.appendQueryParameter("param1", foo)
.appendQueryParameter("param2", bar)
.build();
HttpGet request = new HttpGet(uri.toString());
// This looks very tempting but does NOT set request parameters
// but just HttpClient configuration parameters:
// HttpParams params = new BasicHttpParams();
// params.setParameter("q", query);
// request.setParams(params);
HttpResponse response = defaultClient.execute(request);
String json = EntityUtils.toString(response.getEntity());
Outside of Android, your best bet is building the query string manually (with all the encoding hassles) or finding something similar to Android's Uri.Builder.

Categories