I am trying to receive name and value (query details) from a URL but I have a restriction that the URL shouldn't be parsed to find out query details from the URL. I tried to find the index of '?' and then add the substring on a list till index of '='-1 from the query which I found using the statement:
String query = uri.getQuery() == null || uri.getQuery().trim().isEmpty() ? "" : uri.getQuery();
The issue that comes here is I am getting an error which states:
java.net.URISyntaxException: Illegal character in query at index
Can anyone point where am I going wrong ?
The URL which I am using is:
https://xyzz.com.in/collections/two-tone-wedding-rings?ugg_9ty554_tags=ROSE|WHITE&ugg_yuiolz_price=%3A700%201000%20%7B0%7D%20-%20%7B1%7D
(The | symbol is creating issues)
I think you're using the wrong class for this
I tried doing it with URL class instead
String input = "https://xyzz.com.in/collections/two-tone-wedding-rings?ugg_9ty554_tags=ROSE|WHITE&ugg_yuiolz_price=%3A700%201000%20%7B0%7D%20-%20%7B1%7D\n"
URL uri = new URL(input);
System.out.println(uri.getQuery());
Output:
ugg_9ty554_tags=ROSE|WHITE&ugg_yuiolz_price=%3A700%201000%20%7B0%7D%20-%20%7B1%7D
Getting exception while inserting field value with '/' in elastic search
Code:
String id = "/EACVBSDSFASFA";
IndexRequest indexRequest = new IndexRequest(INDEX, TYPE, id).source(objectsMap);
IndexResponse indexResponse = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
Exception:
Path part [/EACVBSDSFASFA] couldn't be encoded: java.lang.IllegalArgumentException
java.lang.IllegalArgumentException: Path part [/EACVBSDSFASFA] couldn't be encoded
at org.elasticsearch.client.RequestConverters$EndpointBuilder.encodePart(RequestConverters.java:1142)
at org.elasticsearch.client.RequestConverters$EndpointBuilder.addPathPart(RequestConverters.java:1104)
at org.elasticsearch.client.RequestConverters.endpoint(RequestConverters.java:704)
at org.elasticsearch.client.RequestConverters.index(RequestConverters.java:309)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1761)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1735)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1697)
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:929)
at
.....
.....
Caused by: java.net.URISyntaxException: Illegal character in port number at index 30:
The issue is not because of the field, but because you're using / as part of the id.
I'd guess this is a restriction to prevent resources with where IDs would have to be encoded, e.g. http://localhost:9200/index/_type/%2FEACVBSDSFASFA, note / -> %2F conversion.
In any case - prefer url-safe IDs, e.g:
Incremental IDs
GUIDs
If you want to use base64 IDs, use base64url version of it - https://en.wikipedia.org/wiki/Base64#URL_applications where +, =, / are replaced with URI-safe characters.
I am trying to write a rest api in which I am passing date as a URL parameter.
Date formate is dd/MM/yyyy HH:mm;
REST API URL Is
public static final String GET_TestDate = "/stay/datecheck?dateCheckIn={dateCheckIn}";
and Rest Method is
#RequestMapping(value = HotelRestURIConstants.GET_TestDate, method = RequestMethod.GET)
public #ResponseBody String getDate(#PathVariable("dateCheckIn") #DateTimeFormat(iso= DateTimeFormat.ISO.DATE) String dateCheckIn) {
logger.info("passing date as a param");
String str="date"+dateCheckIn;
return str;
}
but when am calling this api using REST client I am getting 404 error.
Here is REST URL
http://localhost:8089/stay/datecheck?dateCheckIn="28/01/2016 19:00"
Instead of space, use %20. Instead of slash, you can use %2F. But, you have to decode (transform %20 to space and %2F to slash) after you get the value. Instead of colon, use %3A. You have an URL enconding table here: http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
The last hint: don't use quotes.
Try something like:
http://localhost:8089/stay/datecheck?dateCheckIn=28%2F01%2F2016%2019%3A00
Remember to decode it.
Something like: String result = java.net.URLDecoder.decode(url, "UTF-8");
The main problem is here: #PathVariable("dateCheckIn") #DateTimeFormat(iso= DateTimeFormat.ISO.DATE) String dateCheckIn
dateCheckIn should not be #PathVariable but #RequestParam
Let's see the difference:
http://localhost:8089/stay/{path_var}/datecheck?{query_param}=some_value
Path variable is a part of the path, it must be there for the path to map correctly to your method. In the actual call, you never actually specify any name for the variable. Query parameter (or request parameter) is a parameter that occurs after "?" which appears after the path. There you always write the name of a parameter followed by the "=" sign and the value. It might or may not be required. See following example:
Path String:
String GET_TestDate = "/stay/{path_var}/datecheck";
Parameter annotations:
#PathVariable("path_var") Integer var1, #RequestParam("query_param") String
Actual call:
http://localhost:8089/stay/1/datecheck?query_param=abc
Values populated:
var1 = 1
var2 = "abc"
There might be other problems (such as the date format you used in your URL - you shouldn't use quotes and spaces and should URL encode it or change the format to use dashes for example or send time and date in Epoch (unix time) format), but I believe the 404 is because of the wrong Path String and annotations on your method.
More on the topic: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/mvc.html#mvc-ann-requestparam
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/mvc.html#mvc-ann-requestmapping-uri-templates
You actually have 2 problems.
Your 404 is because your URL doesn't match any patterns. This is almost certainly because you didn't MIME encode your date parameter. An actual browser will do this for you but code/REST clients probably won't as they wisely should never mess with your input.
Your next problem is that your date is a #QueryParam and not #PathParam. Once you fix the encoding issue you would then discover that your date would be null since there is not PathParam by that name
this is my code
// assume var data has japanese characters
xmlhttp.open("POST","adminUpdate?&value="+data,true); // tried GET as well
xmlhttp.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");
xmlhttp.send();
if I insert alert(data) then i can see japanese characters perfectly fine.
But on the server side (servlet class) when I add this code :
String query = request.getParameter("value");
system.out.println(query)
Now I see garbage value ??????
Ok so I added this line server side :
System.out.println("content type : "+ request.getContentType());
and I got this : text/plain;charset=UTF-8
So now my question is if the encoding is set correctly then why I cant see Japanese characters
One option is to send the query parameters as part of the request body and have the content type set to application/x-www-form-urlencoded.
Then, before getting the parameter, set the request's content character encoding
request.setCharacterEncoding("UTF-8");
String query = request.getParameter("value");
Note that wherever you're printing the query value has to be able to display UTF-8 encoded characters.
System.out.println(
new URI("http", "example.com", "/servlet", "a=x%20y", null));
The result is http://example.com/servlet?a=x%2520y, where the query parameter value differs from the supplied one. Strange, but this does follow the Javadoc:
"The percent character ('%') is always quoted by these constructors."
We can pass the decoded string, a=x y and then we get a reasonable(?) result a=x%20y.
But what if the query parameter value contains an "&" character? This happens for example if the value is an URL itself with query parameters. Look at this (wrong) query string:
a=b&c. The ampersand must be escaped here (a=b%26c), otherwise this can be considered as a query parameter a=b and some garbage (c). If I pass this to an URI constructor, it encodes it, and returns a wrong URL: ...?a=b%2526c
This issue seems to render java.util.URI useless. Am I missing something here?
Summary of answers
java.net.URI does know about the existence of the query part of an URI, but it does not understand the internals of the query part, which can differ for each scheme. For example java.net.URI does not understand the internal structure of the HTTP query part. This would not be a problem, if java.net.URI considered query as an opaque string, and did not alter it. But it tries to apply some generic percent-encoding algorithm, which breaks HTTP URLs.
Therefore I cannot use the URI class to reliably assemble an URL from its parts, despite there are constructors for it. I would also mention that as of Java 7, the implementation of the relativize operation is quite limited, only works if one URL is the prefix of another one. These two functionality (and its leaner interface for these purposes) were the reason why I was interested in java.net.URI, but neither of them works for me.
At the end I used java.net.URL for parsing, and wrote code to assemble an URL from parts and to relativize two URLs. I also checked the Apache HttpClient URIBuilder class, and although it does understand the internals of an HTTP query string, but as of 4.3, it has the same problem with encoding like java.net.URI when dealing with the query part as a whole.
The query string
a=b&c
is not wrong in a URI. The RFC on URI Generic Syntax states
The query component is a string of information to be interpreted by
the resource.
query = *uric
Within a query component, the characters ";", "/", "?", ":", "#",
"&", "=", "+", ",", and "$" are reserved.
The character & in the query string is very much valid (uric represents reserved, mark, and alphanumeric characters). The RFC also states
Many URI include components consisting of or delimited by, certain
special characters. These characters are called "reserved", since
their usage within the URI component is limited to their reserved
purpose. If the data for a URI component would conflict with the
reserved purpose, then the conflicting data must be escaped before
forming the URI.
Because the & is valid but reserved, it is up to the user to determine if it is meant to be encoded or not.
What you call a query parameter is not a feature of a URI and therefore the URI class has no reason to (and shouldn't) support it.
Related:
Which characters make a URL invalid?
The only workaround I found was to use the single-argument constructors and methods. Note that you must use URI#getRawQuery() to avoid decoding %26. For example:
URI uri = new URI("http://a/?b=c%26d&e");
// uri.getRawQuery() equals "b=c%26d&e"
uri = new URI(new URI(uri.getScheme(), uri.getAuthority(),
uri.getPath(), null, null) + "?f=g%26h&i");
// uri.getRawQuery() equals "f=g%26h&i"
uri = uri.resolve("?j=k%26l&m");
// uri.getRawQuery() equals "j=k%26l&m"
// uri.toString() equals "http://a/?j=k%26l&m"
Single working solution known for me is reflection (see https://blog.stackhunter.com/2014/03/31/encode-special-characters-java-net-uri/)
URI uri = new URI("http", null, "example.com", -1, "/accounts", null, null);
Field field = URI.class.getDeclaredField("query");
field.setAccessible(true);
field.set(uri, encodedQueryString);
//clear cached string representation
field = URI.class.getDeclaredField("string");
field.setAccessible(true);
field.set(uri, null);
Use URLEncoder.encode() method, in your case for example:
URLEncoder.encode("a=x%20y", "ISO-8859-1");