Related
The Situation:
I decremented a Uri
First, I converted the Uri into a string and in turn into an int
Afterwhich, I did a -1, and then I got the string that looks exactly like a decremented string
However, when I parse the uri and try to setImageURI() on it,
it is showing "File error accessing recents directory (directory
doesn't exist?)."
Here is the code that I have used:
Uri ImageUri = data.getData();
String uri1 = ImageUri.toString();
//region uri2
String substr1 = uri1.substring(uri1.length()-3);
int substr1int = parseInt(substr1)-1;
String decrementedstr1 = new Integer(substr1int).toString();
int numberofchars1 = uri1.length()-3;
String firstcomponent1 = uri1.substring(0, numberofchars1);
String uri2 = firstcomponent1 + decrementedstr1;
//endregion
Uri test = Uri.parse(uri2);
animateobject.setImageURI(test);
Got this Error:
File error accessing recents directory (directory doesn't exist?).
After I used 'Debug App', it showed the error in more details:
java.lang.SecurityException: Permission Denial: reading
com.android.providers.media.MediaDocumentsProvider uri
content://com.android.providers.media.documents/document/image%3A1000002538
from pid=1309, uid=10925 requires that you obtain access using
ACTION_OPEN_DOCUMENT or related APIs
Note: This is in java and I'm using Android Studio to code.
Let's first get something straight. What is the meaning of that % character?
Well ... if you look at the URI Specification (RFC ....) the % is a percent encoding marker. The two characters after the % are hex digits, and the whole thing represents an ASCII character. In fact, %3A represents the colon character (:). So the unencoded "opaque" component of that URI is actually
//com.android.providers.media.documents/document/image:1000002538
Thus, the image (document) number is really 1000002538 and decrementing it should give 1000002537 as the image number.
I'm not entirely sure why your "string bashing" approach is failing, but you are decrementing just the last 3 digits of the image numbers ... and your example has 4 significant digits on the right end.
So here's how you should code it:
Uri imageUri = data.getData();
String[] pathSegments = imageUri.getSchemeSpecificPart().split("/");
String lastSegment = pathSegments[pathSegmentslength - 1);
String[] parts = lastSegment.split(":");
assert parts.length == 1 && "image".equals(parts[0]);
long imageNo = Long.parseLong(parts[1]);
imageNo--;
lastSegment = "image:" + imageNo;
pathSegments[pathSegments.length - 1] = lastSegment;
String path = String.join("/", pathSegments);
imageUri = Uri.Builder().scheme("content").opaquePart(path).build();
By calling getSchemeSpecificPart() we are getting the relevant part of the URI with the percent encoding decoded. Likewise, the Builder is going to re-apply encoding as required.
CAVEATS
This code is not compiled or tested. I don't have an Android dev platform.
For non-Android folks, this is using the Android Uri class not the Java SE URI class!
Say I have a URL
http://example.com/query?q=
and I have a query entered by the user such as:
random word £500 bank $
I want the result to be a properly encoded URL:
http://example.com/query?q=random%20word%20%A3500%20bank%20%24
What's the best way to achieve this? I tried URLEncoder and creating URI/URL objects but none of them come out quite right.
URLEncoder is the way to go. You only need to keep in mind to encode only the individual query string parameter name and/or value, not the entire URL, for sure not the query string parameter separator character & nor the parameter name-value separator character =.
String q = "random word £500 bank $";
String url = "https://example.com?q=" + URLEncoder.encode(q, StandardCharsets.UTF_8);
When you're still not on Java 10 or newer, then use StandardCharsets.UTF_8.toString() as charset argument, or when you're still not on Java 7 or newer, then use "UTF-8".
Note that spaces in query parameters are represented by +, not %20, which is legitimately valid. The %20 is usually to be used to represent spaces in URI itself (the part before the URI-query string separator character ?), not in query string (the part after ?).
Also note that there are three encode() methods. One without Charset as second argument and another with String as second argument which throws a checked exception. The one without Charset argument is deprecated. Never use it and always specify the Charset argument. The javadoc even explicitly recommends to use the UTF-8 encoding, as mandated by RFC3986 and W3C.
All other characters are unsafe and are first converted into one or more bytes using some encoding scheme. Then each byte is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. The recommended encoding scheme to use is UTF-8. However, for compatibility reasons, if an encoding is not specified, then the default encoding of the platform is used.
See also:
What every web developer must know about URL encoding
I would not use URLEncoder. Besides being incorrectly named (URLEncoder has nothing to do with URLs), inefficient (it uses a StringBuffer instead of Builder and does a couple of other things that are slow) Its also way too easy to screw it up.
Instead I would use URIBuilder or Spring's org.springframework.web.util.UriUtils.encodeQuery or Commons Apache HttpClient.
The reason being you have to escape the query parameters name (ie BalusC's answer q) differently than the parameter value.
The only downside to the above (that I found out painfully) is that URL's are not a true subset of URI's.
Sample code:
import org.apache.http.client.utils.URIBuilder;
URIBuilder ub = new URIBuilder("http://example.com/query");
ub.addParameter("q", "random word £500 bank \$");
String url = ub.toString();
// Result: http://example.com/query?q=random+word+%C2%A3500+bank+%24
You need to first create a URI like:
String urlStr = "http://www.example.com/CEREC® Materials & Accessories/IPS Empress® CAD.pdf"
URL url = new URL(urlStr);
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
Then convert that URI to an ASCII string:
urlStr = uri.toASCIIString();
Now your URL string is completely encoded. First we did simple URL encoding and then we converted it to an ASCII string to make sure no character outside US-ASCII remained in the string. This is exactly how browsers do it.
Guava 15 has now added a set of straightforward URL escapers.
The code
URL url = new URL("http://example.com/query?q=random word £500 bank $");
URI uri = new URI(url.getProtocol(), url.getUserInfo(), IDN.toASCII(url.getHost()), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
String correctEncodedURL = uri.toASCIIString();
System.out.println(correctEncodedURL);
Prints
http://example.com/query?q=random%20word%20%C2%A3500%20bank%20$
What is happening here?
1. Split URL into structural parts. Use java.net.URL for it.
2. Encode each structural part properly!
3. Use IDN.toASCII(putDomainNameHere) to Punycode encode the hostname!
4. Use java.net.URI.toASCIIString() to percent-encode, NFC encoded Unicode - (better would be NFKC!). For more information, see: How to encode properly this URL
In some cases it is advisable to check if the URL is already encoded. Also replace '+' encoded spaces with '%20' encoded spaces.
Here are some examples that will also work properly
{
"in" : "http://نامهای.com/",
"out" : "http://xn--mgba3gch31f.com/"
},{
"in" : "http://www.example.com/‥/foo",
"out" : "http://www.example.com/%E2%80%A5/foo"
},{
"in" : "http://search.barnesandnoble.com/booksearch/first book.pdf",
"out" : "http://search.barnesandnoble.com/booksearch/first%20book.pdf"
}, {
"in" : "http://example.com/query?q=random word £500 bank $",
"out" : "http://example.com/query?q=random%20word%20%C2%A3500%20bank%20$"
}
The solution passes around 100 of the test cases provided by Web Platform Tests.
Using Spring's UriComponentsBuilder:
UriComponentsBuilder
.fromUriString(url)
.build()
.encode()
.toUri()
The Apache HttpComponents library provides a neat option for building and encoding query parameters.
With HttpComponents 4.x use:
URLEncodedUtils
For HttpClient 3.x use:
EncodingUtil
Here's a method you can use in your code to convert a URL string and map of parameters to a valid encoded URL string containing the query parameters.
String addQueryStringToUrlString(String url, final Map<Object, Object> parameters) throws UnsupportedEncodingException {
if (parameters == null) {
return url;
}
for (Map.Entry<Object, Object> parameter : parameters.entrySet()) {
final String encodedKey = URLEncoder.encode(parameter.getKey().toString(), "UTF-8");
final String encodedValue = URLEncoder.encode(parameter.getValue().toString(), "UTF-8");
if (!url.contains("?")) {
url += "?" + encodedKey + "=" + encodedValue;
} else {
url += "&" + encodedKey + "=" + encodedValue;
}
}
return url;
}
In Android, I would use this code:
Uri myUI = Uri.parse("http://example.com/query").buildUpon().appendQueryParameter("q", "random word A3500 bank 24").build();
Where Uri is a android.net.Uri
In my case I just needed to pass the whole URL and encode only the value of each parameters.
I didn't find common code to do that, so (!!) so I created this small method to do the job:
public static String encodeUrl(String url) throws Exception {
if (url == null || !url.contains("?")) {
return url;
}
List<String> list = new ArrayList<>();
String rootUrl = url.split("\\?")[0] + "?";
String paramsUrl = url.replace(rootUrl, "");
List<String> paramsUrlList = Arrays.asList(paramsUrl.split("&"));
for (String param : paramsUrlList) {
if (param.contains("=")) {
String key = param.split("=")[0];
String value = param.replace(key + "=", "");
list.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
else {
list.add(param);
}
}
return rootUrl + StringUtils.join(list, "&");
}
public static String decodeUrl(String url) throws Exception {
return URLDecoder.decode(url, "UTF-8");
}
It uses Apache Commons' org.apache.commons.lang3.StringUtils.
Use this：
URLEncoder.encode(query, StandardCharsets.UTF_8.displayName());
or this:
URLEncoder.encode(query, "UTF-8");
You can use the following code.
String encodedUrl1 = UriUtils.encodeQuery(query, "UTF-8"); // No change
String encodedUrl2 = URLEncoder.encode(query, "UTF-8"); // Changed
String encodedUrl3 = URLEncoder.encode(query, StandardCharsets.UTF_8.displayName()); // Changed
System.out.println("url1 " + encodedUrl1 + "\n" + "url2=" + encodedUrl2 + "\n" + "url3=" + encodedUrl3);
Say I have a URL
http://example.com/query?q=
and I have a query entered by the user such as:
random word £500 bank $
I want the result to be a properly encoded URL:
http://example.com/query?q=random%20word%20%A3500%20bank%20%24
What's the best way to achieve this? I tried URLEncoder and creating URI/URL objects but none of them come out quite right.
URLEncoder is the way to go. You only need to keep in mind to encode only the individual query string parameter name and/or value, not the entire URL, for sure not the query string parameter separator character & nor the parameter name-value separator character =.
String q = "random word £500 bank $";
String url = "https://example.com?q=" + URLEncoder.encode(q, StandardCharsets.UTF_8);
When you're still not on Java 10 or newer, then use StandardCharsets.UTF_8.toString() as charset argument, or when you're still not on Java 7 or newer, then use "UTF-8".
Note that spaces in query parameters are represented by +, not %20, which is legitimately valid. The %20 is usually to be used to represent spaces in URI itself (the part before the URI-query string separator character ?), not in query string (the part after ?).
Also note that there are three encode() methods. One without Charset as second argument and another with String as second argument which throws a checked exception. The one without Charset argument is deprecated. Never use it and always specify the Charset argument. The javadoc even explicitly recommends to use the UTF-8 encoding, as mandated by RFC3986 and W3C.
All other characters are unsafe and are first converted into one or more bytes using some encoding scheme. Then each byte is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. The recommended encoding scheme to use is UTF-8. However, for compatibility reasons, if an encoding is not specified, then the default encoding of the platform is used.
See also:
What every web developer must know about URL encoding
I would not use URLEncoder. Besides being incorrectly named (URLEncoder has nothing to do with URLs), inefficient (it uses a StringBuffer instead of Builder and does a couple of other things that are slow) Its also way too easy to screw it up.
Instead I would use URIBuilder or Spring's org.springframework.web.util.UriUtils.encodeQuery or Commons Apache HttpClient.
The reason being you have to escape the query parameters name (ie BalusC's answer q) differently than the parameter value.
The only downside to the above (that I found out painfully) is that URL's are not a true subset of URI's.
Sample code:
import org.apache.http.client.utils.URIBuilder;
URIBuilder ub = new URIBuilder("http://example.com/query");
ub.addParameter("q", "random word £500 bank \$");
String url = ub.toString();
// Result: http://example.com/query?q=random+word+%C2%A3500+bank+%24
You need to first create a URI like:
String urlStr = "http://www.example.com/CEREC® Materials & Accessories/IPS Empress® CAD.pdf"
URL url = new URL(urlStr);
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
Then convert that URI to an ASCII string:
urlStr = uri.toASCIIString();
Now your URL string is completely encoded. First we did simple URL encoding and then we converted it to an ASCII string to make sure no character outside US-ASCII remained in the string. This is exactly how browsers do it.
Guava 15 has now added a set of straightforward URL escapers.
The code
URL url = new URL("http://example.com/query?q=random word £500 bank $");
URI uri = new URI(url.getProtocol(), url.getUserInfo(), IDN.toASCII(url.getHost()), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
String correctEncodedURL = uri.toASCIIString();
System.out.println(correctEncodedURL);
Prints
http://example.com/query?q=random%20word%20%C2%A3500%20bank%20$
What is happening here?
1. Split URL into structural parts. Use java.net.URL for it.
2. Encode each structural part properly!
3. Use IDN.toASCII(putDomainNameHere) to Punycode encode the hostname!
4. Use java.net.URI.toASCIIString() to percent-encode, NFC encoded Unicode - (better would be NFKC!). For more information, see: How to encode properly this URL
In some cases it is advisable to check if the URL is already encoded. Also replace '+' encoded spaces with '%20' encoded spaces.
Here are some examples that will also work properly
{
"in" : "http://نامهای.com/",
"out" : "http://xn--mgba3gch31f.com/"
},{
"in" : "http://www.example.com/‥/foo",
"out" : "http://www.example.com/%E2%80%A5/foo"
},{
"in" : "http://search.barnesandnoble.com/booksearch/first book.pdf",
"out" : "http://search.barnesandnoble.com/booksearch/first%20book.pdf"
}, {
"in" : "http://example.com/query?q=random word £500 bank $",
"out" : "http://example.com/query?q=random%20word%20%C2%A3500%20bank%20$"
}
The solution passes around 100 of the test cases provided by Web Platform Tests.
Using Spring's UriComponentsBuilder:
UriComponentsBuilder
.fromUriString(url)
.build()
.encode()
.toUri()
The Apache HttpComponents library provides a neat option for building and encoding query parameters.
With HttpComponents 4.x use:
URLEncodedUtils
For HttpClient 3.x use:
EncodingUtil
Here's a method you can use in your code to convert a URL string and map of parameters to a valid encoded URL string containing the query parameters.
String addQueryStringToUrlString(String url, final Map<Object, Object> parameters) throws UnsupportedEncodingException {
if (parameters == null) {
return url;
}
for (Map.Entry<Object, Object> parameter : parameters.entrySet()) {
final String encodedKey = URLEncoder.encode(parameter.getKey().toString(), "UTF-8");
final String encodedValue = URLEncoder.encode(parameter.getValue().toString(), "UTF-8");
if (!url.contains("?")) {
url += "?" + encodedKey + "=" + encodedValue;
} else {
url += "&" + encodedKey + "=" + encodedValue;
}
}
return url;
}
In Android, I would use this code:
Uri myUI = Uri.parse("http://example.com/query").buildUpon().appendQueryParameter("q", "random word A3500 bank 24").build();
Where Uri is a android.net.Uri
In my case I just needed to pass the whole URL and encode only the value of each parameters.
I didn't find common code to do that, so (!!) so I created this small method to do the job:
public static String encodeUrl(String url) throws Exception {
if (url == null || !url.contains("?")) {
return url;
}
List<String> list = new ArrayList<>();
String rootUrl = url.split("\\?")[0] + "?";
String paramsUrl = url.replace(rootUrl, "");
List<String> paramsUrlList = Arrays.asList(paramsUrl.split("&"));
for (String param : paramsUrlList) {
if (param.contains("=")) {
String key = param.split("=")[0];
String value = param.replace(key + "=", "");
list.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
else {
list.add(param);
}
}
return rootUrl + StringUtils.join(list, "&");
}
public static String decodeUrl(String url) throws Exception {
return URLDecoder.decode(url, "UTF-8");
}
It uses Apache Commons' org.apache.commons.lang3.StringUtils.
Use this：
URLEncoder.encode(query, StandardCharsets.UTF_8.displayName());
or this:
URLEncoder.encode(query, "UTF-8");
You can use the following code.
String encodedUrl1 = UriUtils.encodeQuery(query, "UTF-8"); // No change
String encodedUrl2 = URLEncoder.encode(query, "UTF-8"); // Changed
String encodedUrl3 = URLEncoder.encode(query, StandardCharsets.UTF_8.displayName()); // Changed
System.out.println("url1 " + encodedUrl1 + "\n" + "url2=" + encodedUrl2 + "\n" + "url3=" + encodedUrl3);
I'm having an issue with this line of code:
try (OutputStreamWriter fileout = new OutputStreamWriter(new FileOutputStream(Paths.get(path.toString(), TAGS_FILE.toString()).toString()), "UTF-16")) {
fileout.write(gson.toJson(imageList, listType));
fileout.flush();
fileout.close();
}
I was using UTF-8 originally and it was working fine, loaded fine and everything, but had to change to UTF-16 to preserve some special characters. It still writes out the file correctly, exact same as with UTF-8 (except with the special characters in-tact), but when it tries to load the file into another session I get "Expected BEGIN_ARRAY but was STRING..."
Is there a way around this?
Also, if this helps:
private final Type listType = new TypeToken<TreeSet<MyClass>>(){}.getType();
TreeSet<MyClass> imageList;
UPDATE:
private void move(File file, Path destination, boolean autoTag) {
String fileName = file.getName();
Matcher numberMatcher = leadingNumbersPattern.matcher(fileName);
// remove leading numbers
while (numberMatcher.find()) {
fileName = clean(fileName, leadingNumbersPattern);
}
Matcher artistMatcher = artistPattern.matcher(fileName);
Matcher newFileNameMatcher = newFileNamePattern.matcher(fileName);
if (artistMatcher.find() && newFileNameMatcher.find()) {
// set artist name
String artist = artistMatcher.group().substring(0, artistMatcher.group().length() - 1);
// set new picture name
String newFileName = newFileNameMatcher.group().substring(1);
Path newPath = Paths.get(destination.toString(), artist); // path to artist folder
new File(newPath.toString()).mkdirs(); // make artist folder
newPath = Paths.get(destination.toString(), artist, newFileName); // make path to new file location
try {
Files.move(file.toPath(), newPath, StandardCopyOption.REPLACE_EXISTING); // move file to new location
MyImage newImage = new MyImage(newPath.toString(), artist, newFileName);
Changing back to UTF-8 fixed the issue. I had to remake the json file; I guess the Thai characters had somehow slipped through when I converted everything to UTF-8 originally.
EDIT:
Found the cause! The load() method I was using to deserialize the file wasn't set to use UTF-8 on the FileInputStream. Adding this fixed the issue completely.
I have a string representing an URL containing spaces and want to convert it to an URI object. If I simply try to create it via
String myString = "http://myhost.com/media/File Name that has spaces inside.mp3";
URI myUri = new URI(myString);
it gives me
java.net.URISyntaxException: Illegal character in path at index X
where index X is the position of the first space in the URL string.
How can i parse myString into a URI object?
You should in fact URI-encode the "invalid" characters. Since the string actually contains the complete URL, it's hard to properly URI-encode it. You don't know which slashes / should be taken into account and which not. You cannot predict that on a raw String beforehand. The problem really needs to be solved at a higher level. Where does that String come from? Is it hardcoded? Then just change it yourself accordingly. Does it come in as user input? Validate it and show error, let the user solve itself.
At any way, if you can ensure that it are only the spaces in URLs which makes it invalid, then you can also just do a string-by-string replace with %20:
URI uri = new URI(string.replace(" ", "%20"));
Or if you can ensure that it's only the part after the last slash which needs to be URI-encoded, then you can also just do so with help of android.net.Uri utility class:
int pos = string.lastIndexOf('/') + 1;
URI uri = new URI(string.substring(0, pos) + Uri.encode(string.substring(pos)));
Do note that URLEncoder is insuitable for the task as it's designed to encode query string parameter names/values as per application/x-www-form-urlencoded rules (as used in HTML forms). See also Java URL encoding of query string parameters.
java.net.URLEncoder.encode(finalPartOfString, "utf-8");
This will URL-encode the string.
finalPartOfString is the part after the last slash - in your case, the name of the song, as it seems.
To handle spaces, #, and other unsafe characters in arbitrary locations in the url path, Use Uri.Builder in combination with a local instance of URL as I have described here:
private Uri.Builder builder;
public Uri getUriFromUrl(String thisUrl) {
URL url = new URL(thisUrl);
builder = new Uri.Builder()
.scheme(url.getProtocol())
.authority(url.getAuthority())
.appendPath(url.getPath());
return builder.build();
}
URL url = Test.class.getResource(args[0]); // reading demo file path from
// same location where class
File input=null;
try {
input = new File(url.toURI());
} catch (URISyntaxException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
I wrote this function:
public static String encode(#NonNull String uriString) {
if (TextUtils.isEmpty(uriString)) {
Assert.fail("Uri string cannot be empty!");
return uriString;
}
// getQueryParameterNames is not exist then cannot iterate on queries
if (Build.VERSION.SDK_INT < 11) {
return uriString;
}
// Check if uri has valid characters
// See https://tools.ietf.org/html/rfc3986
Pattern allowedUrlCharacters = Pattern.compile("([A-Za-z0-9_.~:/?\\#\\[\\]#!$&'()*+,;" +
"=-]|%[0-9a-fA-F]{2})+");
Matcher matcher = allowedUrlCharacters.matcher(uriString);
String validUri = null;
if (matcher.find()) {
validUri = matcher.group();
}
if (TextUtils.isEmpty(validUri) || uriString.length() == validUri.length()) {
return uriString;
}
// The uriString is not encoded. Then recreate the uri and encode it this time
Uri uri = Uri.parse(uriString);
Uri.Builder uriBuilder = new Uri.Builder()
.scheme(uri.getScheme())
.authority(uri.getAuthority());
for (String path : uri.getPathSegments()) {
uriBuilder.appendPath(path);
}
for (String key : uri.getQueryParameterNames()) {
uriBuilder.appendQueryParameter(key, uri.getQueryParameter(key));
}
String correctUrl = uriBuilder.build().toString();
return correctUrl;
}