java.net.URISyntaxException: Illegal character in query at index 177 - java

I tried to get Azure Usage details via nextLink which is shared by Azure. while i tried to make http request URISyntaxException is occured.
HttpClient httpclient = getHttpClient();
URIBuilder uriBuilder=new URIBuilder(url);
HttpGet httpGet = new HttpGet(uriBuilder.build());
HttpResponse httpResponse = httpclient.execute(httpGet);
This is the nextLink url:
"https://management.azure.com/subscriptions/78c50b17-61fd-40cc-819c-4953586c7850/providers/Microsoft.Consumption/usageDetails?api-version=2019-11-01&$filter=properties/usageStart eq '2020-07-1' and properties/usageEnd eq '2020-07-30' &metric=actualcost&$expand=properties/meterDetails,properties/additionalInfo&sessiontoken=15:785628&$skiptoken=827CDTHDWI07C46616C7365730&skiptokenver=v1&id=2d790-d675-45d-89j56-3989w06cca"
I think this is because of characters such as ?, & and ! in my URL. so I tried using:
URLEncoder.encode(myUrl, "UTF-8");
but after this, I faced protocol exception.
Am I missing something here?

Your URL contains spaces and single quotes, these should be URL encoded like you tried. However, because you tried to URL-encode the entire URL, you end up with this:
https%3A%2F%2Fmanagement.azure.com%2Fsubscriptions%2F78c50b17-61fd-40cc-819c-4953586c7850%2Fproviders%2FMicrosoft.Consumption%2FusageDetails%3Fapi-version%3D2019-11-01%26%24filter%3Dproperties%2FusageStart+eq+%272020-07-1%27+and+properties%2FusageEnd+eq+%272020-07-30%27+%26metric%3Dactualcost%26%24expand%3Dproperties%2FmeterDetails%2Cproperties%2FadditionalInfo%26sessiontoken%3D15%3A785628%26%24skiptoken%3D827CDTHDWI07C46616C7365730%26skiptokenver%3Dv1%26id%3D2d790-d675-45d-89j56-3989w06cca
Which is not a valid URL. You could simply try using a naive form of String replacement:
myUrl = myUrl.replace(" ", "%20").replace("'", "%27");
If that is not sufficient, you'll need to reconstruct the URL yourself, and only apply URL-encoding on the query parameter values.

Related

PostMethod url parameters

Is there any alternative to pass URL parameters using PostMethod? After this an XML needs to be posted along with the URL.
Since it is a post request, the URL parameters should be passed in the Request body and should not be visible.
Can addParameter method be used?
URL- http://mytest.com?abc=xyz&token=aisk%2s
1)
//this works ( no utf-8 encoding)
PostMethod pm =new PostMethod("http://mytest.com");
pm.setQueryString("abc=xyz");
pm.setQueryString("token=aisk%2s");
2)
// it encodes utf-8 and fails
PostMethod pm =new PostMethod("http://mytest.com");
NameValuePair [] nvp= new NameValuePair[2];
nvp[0]=new NameValuePair("abc","xyz");
nvp[1]=new NameValuePair("token","aisk%2s");
//encodes the token value as aisk%252s
pm.setQueryString(nvp);
An XML needs to be posted after setting the above URL parameters.
pm.setRequestEntity(new StringRequestEntity(xml, "application/xml", "UTF-8"));
What about using parts instead of adding your parameter to the url?
You can add various datatypes as well:
// using **MultipartEntity**
multipartContent.addPart("user", new StringBody("admin", ContentType.TEXT_PLAIN));
multipartContent.addPart("content", new InputStreamBody(new FileInputStream(file), ContentType.DEFAULT_BINARY));
and of course you can get a page, as the status of your request.
On your server, thread them like they are variables in your $_REQUEST / $_POST array.
use
var_dump($_REQUEST)
to see the content.

UrlEncodedFormEntity doesn't encode underscore

I want to use a remote API from my Android device, but for some reason, the UrlEncodedFormEntity class doesn't transform the _ with %5f like the remote API seems to expect. As a consequence, using this code:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(
new BasicNameValuePair("json",
"{\"params\":{\"player_name\":\"Toto\",
\"password\":\"clearPass\"},
\"class_name\":\"ApiMasterAuthentication\",
\"method_name\":\"login\"}")
);
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8));
ResponseHandler responseHandler = new BasicResponseHandler();
httpClient.execute(httpPost, responseHandler);
send a post request to the server, with this content:
json=%7B%22params%22%3A%7B%22player_name%22%3A%22Toto%22%2C%22password%22%3A%22clearPass%22%7D%2C%22class_name%22%3A%22ApiMasterAuthentication%22%2C%22method_name%22%3A%22login%22%7D
I would like it to be like this (replacing the preivous underscore by %5F):
json=%7B%22params%22%3A%7B%22player%5Fname%22%3A%22Toto%22%2C%22password%22%3A%22clearPass%22%7D%2C%22class%5Fname%22%3A%22ApiMasterAuthentication%22%2C%22method%5Fname%22%3A%22login%22%7D
I don't have control over the API, and the official client of the API behave like this. It seems to be the expected behaviour for an URL normalization
Am I missing something? I first thought it was an UTF-8 encoding issue, but adding HTTP.UTF-8 in the constructor of UrlEncodedFormEntity doesn't solve the problem.
Thanks for your help.
EDIT: Finally, the problem didn't come from this unescape underscore. Even if the other client I tried to reproduce the behaviour escaped it, I only had to set the proper header:
httpPost.addHeader("Content-Type","application/x-www-form-urlencoded");
And the request worked just fine. Thanks everyone, and especially singh.jagmohan for his help (even if the problem was finally elsewhere)!
"_" isn't a reserved symbol for urls.
setting : Content-Type: application/x-www-form-urlencoded'
should solve the problem. Otherwise you can try replacing it, if you really need this option:
String.Replace("_", "%5f");
See percent encodeing , replace
You can try the following code, it works for me.
try {
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(serviceUrl);
MultipartEntity multipartEntity = new MultipartEntity();
// Also, in place of building JSON string as below, you can build a **JSONObject**
// and then use jsonObject.toString() while building the **StringBody** object
String requestJsonStr = "{\"params\":{\"player_name\":\"Toto\",\"password\":\"clearPass\"},\"class_name\":\"ApiMasterAuthentication\",\"method_name\":\"login\"}";
multipartEntity.addPart("json", new StringBody(requestJsonStr));
httpPost.setEntity(multipartEntity);
HttpResponse response = httpClient.execute(httpPost);
} catch (Exception ex) {
// add specific exception catch block above
// I have used this one just for code snippet
}
PS: The code snippet requires two jar files apache-mime4j-0.6.jar and httpmime-4.0.1.jar.
Hope this helps.

android HTTP POST fail

I'm trying to connect and post to a simple java webservice, running the post's URL from chrome succeeded, but android code skip the following lines (without throwing errors), but the webservice doesn't accept the post
HttpPost post = new HttpPost(setFacebookEventsAddress+userId+"/"+accesstoken);
post.setHeader("Accept", "application/json");
post.setHeader("Content-type", "application/json");
HttpResponse response = client.execute(post);
the webservice method signature handling the above request:
#GET
#Path("setData/{user_id}/{accessToken}")
#Produces(MediaType.APPLICATION_JSON+ ";charset=utf-8")
public String setData(#PathParam("user_id") String user_id,
#PathParam("accessToken") String accessToken) {
since I manage to post throw my browser, anyone can help with what's wrong with my android code?
URL url = new URL(setFacebookEventsAddress+userId+"/"+accesstoken);
HttpURLConnection con = (HttpURLConnection) url
.openConnection();
ja = readStream(con.getInputStream());
Using HttpURLConnection instead of HttpPost did the trick for me, thanks for all the helpers!
It is not possible to say with any certainty (given the evidence), but my guess would be that the expression
setFacebookEventsAddress + userId + "/" + accesstoken
is evaluating to a different URL to the one you are using from the web browser.
I suggest that you try the following:
Turn on request logging on your server, and compare the URLs in the requests being sent.
Modify your client to print out the response status code and the response body. The latter is likely to be an error page that will give you more clues.
Another possible problem is that your code doesn't appear to be sending any body with the POST request.
On revisiting this, the problem was that you were using / trying to do a POST to a web service that you had configured to support GET only. I expect that if you had looked at the status code you would have found that the response code was "Method not supported".

"Illegal Characters" in URL for HttpGet in Android get double-encoded

I am trying to find a solution to this the whole evening now...
I write an app which requests data from a web server. The Server answers in JSON format.
Everything works well except when I enter a umlaut like ä into my App.
In the following I assume the request URL is http://example.com/?q= and I am searching for "Jäger"
The correct call would then be h++p://example.com/?q=J%C3%A4ger
(Sorry for plus-signs but the spam protection doesnt let me post it correctly.)
So my problem is now:
When I give my URL String encoded or unencoded over to HttpGet it will always result in a doublee-encoded URL.
The Request to my Server is then http://example.com/?q=J%25C3%25A4ger (It encodes the percent signs)
which leads to the server searching in database for J%C3%A4ger what is obviously wrong.
So my question is how can I achive that if the user enters "Jäger" my app calls the correctly encoded URL?
Thanks for any help!
Here is the currently used code... Ist probably the worst possible idea I had...
URI url = new URI("http", "//example.com/?q=" + ((EditText)findViewById(R.id.input)).getText().toString(), null);
Log.v("MyLogTag", "API Request: " + url);
HttpGet httpGetRequest = new HttpGet(url);
// Execute the request in the client
HttpResponse httpResponse;
httpResponse = defaultClient.execute(httpGetRequest);
Update: Sorry, HttpParams isn't meant for request parameters but for configuring HttpClient.
On Android, you might want to use Uri.Builder, like suggested in this other SO answer:
Uri uri = new Uri.Builder()
.scheme("http")
.authority("example.com")
.path("someservlet")
.appendQueryParameter("param1", foo)
.appendQueryParameter("param2", bar)
.build();
HttpGet request = new HttpGet(uri.toString());
// This looks very tempting but does NOT set request parameters
// but just HttpClient configuration parameters:
// HttpParams params = new BasicHttpParams();
// params.setParameter("q", query);
// request.setParams(params);
HttpResponse response = defaultClient.execute(request);
String json = EntityUtils.toString(response.getEntity());
Outside of Android, your best bet is building the query string manually (with all the encoding hassles) or finding something similar to Android's Uri.Builder.

Apache HttpGet url encoding problem (plus sign, +)

I'm sending a GET request with HttpClient but the + is not encoded.
1.
If I pass the query parameter string unencoded like this
URI uri = new URI(scheme, host, path, query, null);
HttpGet get = new HttpGet(uri);
Then the + sign is not encoded and it is received as a space on the server. The rest of the url is encoded fine.
2.If I encode the parameters in the query string like this
param = URLEncoder.encode(param,"UTF-8");
Then I get a bunch of weird symbols on the server, probably because the url has been encoded twice.
3.If I only replace the + with %B2 like this
query = query.replaceAll("\\+","%B2");
Then %B2 is encoded when the GET is executed by HttpClient
How can I properly encode Get parameters with Apache HttpClient and make sure the + is encoded as well?
Ok, the solution was that instead of creating the URI like this
URI uri = new URI(scheme, host, path, query, null);
One should create it like this
URIUtils.createURI(scheme, host, -1, path, query, null);
The purpose of the URIUtils class is
A collection of utilities for URIs, to workaround bugs within the
class
no comment........
When you build the query string, use URLEncoder.encode(paramValue, "UTF-8") for each parameter value. Then when you send the request, use URLDecoder.decode(paramValue, "UTF-8") and the "weird symbols" will be decoded.

Categories