Our code uses Asyncresttemplate as follows
String uri = http://api.host.com/version/test?address=%23&language=en-US&format=json
getAysncRestTemplate().getForEntity(uri, String.class);
But %23 is double encoded in Rest template as %2523 and the url becomes
http://api.host.com/version/test?address=%2523&language=en-US&format=json,
But I need to pass encoded string, It doesn't encode if I pass decoded data '#'
How can I send this request without double encoding the URL?
Already tried using UriComponentsBuilder
Avoid Double Encoding of URL query param with Spring's RestTemplate
The uri argument (of type String) passed to the RestTemplate is actually a URI template as per the JavaDoc. The way to use the rest template without the double encoding would be as follows:
getAysncRestTemplate().getForEntity(
"http://api.host.com/version/test?address={address}&language=en-US&format=json",
String.class,
"#"); // (%23 decoded)
If you know you already have a properly encoded URL you can use the method that has a URI as the first parameter instead:
restTemplate.getForEntity(
new URI("http://api.host.com/version/test?address=%23&language=en-US&format=json"),
String.class);
You can avoid this by not encoding any part of it yourself, e.g use # rather than %23
Related
I am trying to make a HTTP request to a API with a route like this:
https://example.test/{ID}/get
The problem is, the ID starts with %, something like "%123123123", after the "excute" method, locate in RestTemplate class, this ID's becomes %25123123123, % changes into %25.
More specific, in this line of the method:
URI expanded = getUriTemplateHandler().expand(url, uriVariables);
Here my uriVariables is just a empty Object, and when i manually change the value of expanded variable removing "25", the request works just fine.
Anyone knows how to fix it ?
After Spring 5, the DefaultUriBuilderFactory, used by RestTemplate, defaults is EncodingMode.URI_COMPONENT, as described below:
Expand URI variables first, and then encode the resulting URI component values, replacing only non-ASCII and illegal (within a given URI component type) characters, but not characters with reserved meaning.
If you want your URI to be unencoded like I do, just create a new DefaultUriBuilderFactory with EncodingMode.NONE and add it to your RestTemplate:
DefaultUriBuilderFactory uriBuilderFactory = new DefaultUriBuilderFactory();
uriBuilderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
RestTemplate restTemplate = new RestTemplate();
restTemplate.setUriTemplateHandler(uriBuilderFactory);
You can check here for other EncondingModes.
I have following code:
String convertedBuilder = builder.toUriString();
convertedBuilder = convertedBuilder.replace(",", "\\,");
URI uri = UriComponentsBuilder.fromUriString(convertedBuilder)
.build().toUri();
The idea is to replace a single comma ',' by a '\,' (slash and comma).
Originated URL should be something like
'server-url'?name=te%5C%2Cst for parameter value 'te\,st.
However Spring generates this one:
'server-url'?name=te%5C%252Cst
What am I doing wrong?
Regards,
I had a similar requirement where I had to call a service with a GPS coordinate like this:
http://example.com/path?param1=40.678806%2C-73.943244
UriComponentsBuilder alone would either:
not encode the , (comma) within the parameter value
encode the encoded comma if I already had encoded the parameter with URLEncoder, thus producing the value 40.678806%252C-73.943244
I resolved by explicitly encoding the query parameter with URLEncoder and telling UriComponentsBuilder that the URL was already encoded:
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl("http://example.com/path")
.queryParam("param1" , URLEncoder.encode("40.678806,-73.943244", "UTF-8"));
ResponseEntity<String> resp = template.exchange(
builder.build(true).toUri(),
HttpMethod.GET,
null, new ParameterizedTypeReference<String> () {}
);
How to encode comma in Spring UriComponentsBuilder?
UriComponentsBuilder doesn't encode the comma since RFC doesn't require it to be encoded. You will have to encode it manually as suggested by Andreas in the comments.
However Spring generates this one:
'server-url'?name=te%5C%252Cst
No, it doesn't, as also mentioned by Andreas. It encodes the backslash but not the comma. Please fix the question if you need any more information.
I created the following simple test to query iTunes:
#Test
fun loadArtist()
{
val restTemplate = RestTemplate()
val builder = UriComponentsBuilder.fromHttpUrl("https://itunes.apple.com/search")
builder.queryParam("term", "howling wolf")
builder.queryParam("entity", "allArtist")
builder.queryParam("limit", 1)
println("\n\nURL ${builder.toUriString()}")
val result = restTemplate.getForObject(builder.toUriString(), String::class.java);
println("Got artist: $result")
}
And the output was unexpected:
URL https://itunes.apple.com/search?term=howling%20wolf&entity=allArtist&limit=1
Got artist:
{
"resultCount":0,
"results": []
}
Pasting the generated URL into a browser does give expected results - artist returned.
https://itunes.apple.com/search?term=howling%20wolf&entity=allArtist&limit=1
Also, hard-coding the query works:
val result = restTemplate.getForObject("https://itunes.apple.com/search?term=howling%20wolf&entity=allArtist&limit=1", String::class.java);
. . the problem only seems to occur for term queries that include spaces.
What went wrong? Other than assemble the URL by hand, how to fix?
Seems like a case of double encoding the whitespace. From the RestTemplate Javadoc:
For each HTTP method there are three variants: two accept a URI
template string and URI variables (array or map) while a third accepts
a URI. Note that for URI templates it is assumed encoding is
necessary, e.g. restTemplate.getForObject("http://example.com/hotel
list") becomes "http://example.com/hotel%20list". This also means if
the URI template or URI variables are already encoded, double encoding
will occur, e.g. http://example.com/hotel%20list becomes
http://example.com/hotel%2520list). To avoid that use a URI method
variant to provide (or re-use) a previously encoded URI. To prepare
such an URI with full control over encoding, consider using
UriComponentsBuilder.
So it looks like getForObject will actually query for https://itunes.apple.com/search?term=howling%2520wolf&entity=allArtist&limit=1 and thus result in an empty result. You can always just replace whitespaces with a "+" in your term or try to make one of those classes skip the encoding process.
Hi I'm trying to do a simple http get query via Retrofit.
My parameter has some special characters and it seems that the url encoding fails.
Original:
data=[out:json];node["name"~"Karlsruhe"]["place"~"city|village|town"];out body;
correct encoding should look like this:
data=%5Bout%3Ajson%5D%3Bnode%5B%22name%22~%22Karlsruhe%22%5D%5B%22place%22~%22city%7Cvillage%7Ctown%22%5D%3Bout%20body%3B
but Retrofit creates this:
data=[out:json];node[%22name%22~%22Karlsruhe%22][%22place%22~%22city|village|town%22];out%20body;
and this will fail with:
java.lang.IllegalStateException: not valid as a java.net.URI:
http://overpass.osm.rambler.ru/cgi/interpreter?data=[out:json];node[%22name%22~%22Karlsruhe%22][%22place%22~%22city|village|town%22];out%20body;
at com.squareup.okhttp.HttpUrl.uri(HttpUrl.java:336) at
com.squareup.okhttp.internal.http.RouteSelector.resetNextProxy(RouteSelector.java:135)
at
com.squareup.okhttp.internal.http.RouteSelector.(RouteSelector.java:71)
at
com.squareup.okhttp.internal.http.RouteSelector.get(RouteSelector.java:76)
at
com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:321)
at
com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:245)
at com.squareup.okhttp.Call.getResponse(Call.java:267) at
com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:224)
at
com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:195)
at com.squareup.okhttp.Call.execute(Call.java:79) at
retrofit.OkHttpCall.execute(OkHttpCall.java:112)
What can be done here to fix this encoding issue?
Thanks
I am not sure about what the root cause of the encoding error is, but you can work around it with the encoded parameter to the Query notation. Setting the parameter to true tells retrofit the parameter is already encoded, so do not encode again.
In your service interface, add encoded=true to your #Query annotation. Something like --
Call<ResponseBody> getResponse(#Query(value = "data", encoded = true) String data);
Then, encode the parameter yourself before sending to retrofit.
final String encodedData = URLEncoder.encode(data, "UTF-8");
Call<ResponseBody> result = service.getResponse(encodedData);
How can I encode URL query parameter values? I need to replace spaces with %20, accents, non-ASCII characters etc.
I tried to use URLEncoder but it also encodes / character and if I give a string encoded with URLEncoder to the URL constructor I get a MalformedURLException (no protocol).
URLEncoder has a very misleading name. It is according to the Javadocs used encode form parameters using MIME type application/x-www-form-urlencoded.
With this said it can be used to encode e.g., query parameters. For instance if a parameter looks like &/?# its encoded equivalent can be used as:
String url = "http://host.com/?key=" + URLEncoder.encode("&/?#");
Unless you have those special needs the URL javadocs suggests using new URI(..).toURL which performs URI encoding according to RFC2396.
The recommended way to manage the encoding and decoding of URLs is to use URI
The following sample
new URI("http", "host.com", "/path/", "key=| ?/#ä", "fragment").toURL();
produces the result http://host.com/path/?key=%7C%20?/%23ä#fragment. Note how characters such as ?&/ are not encoded.
For further information, see the posts HTTP URL Address Encoding in Java or how to encode URL to avoid special characters in java.
EDIT
Since your input is a string URL, using one of the parameterized constructor of URI will not help you. Neither can you use new URI(strUrl) directly since it doesn't quote URL parameters.
So at this stage we must use a trick to get what you want:
public URL parseUrl(String s) throws Exception {
URL u = new URL(s);
return new URI(
u.getProtocol(),
u.getAuthority(),
u.getPath(),
u.getQuery(),
u.getRef()).
toURL();
}
Before you can use this routine you have to sanitize your string to ensure it represents an absolute URL. I see two approaches to this:
Guessing. Prepend http:// to the string unless it's already present.
Construct the URI from a context using new URL(URL context, String spec)
So what you're saying is that you want to encode part of your URL but not the whole thing. Sounds to me like you'll have to break it up into parts, pass the ones that you want encoded through the encoder, and re-assemble it to get your whole URL.