We Keep Coding

How to handle  (object replacement character) in URL

Using Jsoup to scrape URLS and one of the URLS I keep getting has this  symbol in it. I have tried decoding the URL:
url = URLDecoder.decode(url, "UTF-8" );
but it still remains in the code looking like this:
I cant find much online about this other than it is "The object replacement character, sometimes used to represent an embedded object in a document when it is converted to plain text."
But if this is the case I should be able to print the symbol if it is plain text but when I run
System.out.println("");
I get the following complication error:
and it reverts back to the last save.
Sample URL: https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles%ef%bf%bc/
NOTE: If you decode the url then compare it to the decoded url it comes back as not the same e.g.:
String url = URLDecoder.decode("https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles%ef%bf%bc/", "UTF-8");
if(url.contains("https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles?/")){
System.out.println("The same");
}else {
System.out.println("Not the same");
}

That's not a compilation error. That's the eclipse code editor telling you it can't save the source code to a file, because you have told it to save the file in a cp1252 encoding, but that encoding can't express a .
Put differently, your development environment is currently configured to store source code in the cp1252 encoding, which doesn't support the character you want, so you either configure your development environment to store source code using a more flexible encoding (such as UTF-8 the error message suggests), or avoid having that character in your source code, for instance by using its unicode escape sequence instead:
System.out.println("\ufffc");
Note that as far as the Java language and runtime are concerned,  is a character like any other, so there may not be a particular need to "handle" it. Also, I am unsure why you'd expect URLDecoder to do anything if the URL hasn't been URL-encoded to begin with.

"ef bf bc" is a 3 bytes UTF-8 character so as the error says, there's no representation for that character in "CP1252" Windows page encoding.
An option could be to replace that percent encoding sequence with an ascii representation to make the filename for saving:
String url = URLDecoder.decode("https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles%ef%bf%bc/".replace("%ef%bf%bc", "-xEFxBFxBC"), "UTF-8");
url ==> "https://www.breightgroup.com/job/hse-advisor-emb ... contract-roles-xEFxBFxBC/"
Another option using CharsetDecoder
String urlDec = URLDecoder.decode("https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles%ef%bf%bc/", "UTF-8");
CharsetDecoder decoder = Charset.forName("CP1252").newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE);
String urlDec = URLDecoder.decode("https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles%ef%bf%bc/", "UTF-8");
ByteBuffer buffer = ByteBuffer.wrap(urlDec.getBytes(Charset.forName("UTF-8")));
decoder.decode(buffer).toString();
Result
"https://www.breightgroup.com/job/hse-advisor-embedded-contract-roles/"

I found the issue resolved by just replacing URLs with this symbol because there are other URLs with Unicode symbols that were invisible that couldnt be converted ect..
So I just compared the urls to the following regex if it returns false then I just bypass it. Hope this helps someone out:
boolean newURL = url.matches("^[a-zA-Z0-9_:;/.&|%!+=#?-]*$");

How do I manipulate strings with regex?

I'm fairly new to java and I'm trying to get a part of a string:
Say I have a URL and I want a specific part of it, such as a filename:
String url = "http://example.com/filename02563.zip";
The 02563 will be generated at random every time and it's now always 5 characters long.
I want to have java find what's between "m/" (from .com/) to the end of the line to get the filename alone.
Now consider this example:
Say I have an html file that I want a snippet extracted from. Below would be the extracted example:
<applet name=someApplet id=game width="100%" height="100%" archive=someJarFile0456799.jar code=classInsideAJarFile.class mayscript>
I want to extract the jar filename, so I want to get the text between "ve=" and ".jar". The extension will always be ".jar", so including this is not important.
How would I do this? If possible, could you comment the code so I understand what's happening?

Use the Java URI class where you can access the individual elements.
URI uri = new URI("http://example.com/filename02563.zip");
String filename = uri.getPath();
Granted, this will need a little more work if the resource no longer resides in the root path.

You can use the lastIndexOf() and substring() methods from the String class to extract a specific piece of a String:
String url = "http://example.com/filename02563.zip";
String filename = url.substring(url.lastIndexOf("/") + 1); //+1 skips ahead of the '/'

You have answers for your first question so this is for second one. Normally I would use some XML parser but your example is not valid XML file so this will be solved with regex (as you wanted).
String url = "<applet name=someApplet id=game width=\"100%\" height=\"100%\" archive=someJarFile0456799.jar code=classInsideAJarFile.class mayscript>";
Pattern pattern= Pattern.compile("(?<=archive=).*?(?= )");
Matcher m=pattern.matcher(url);
if(m.find())
System.out.println(m.group());
output:
someJarFile0456799.jar

Converting from Java String to Windows-1252 Format

I want to send a URL request, but the parameter values in the URL can have french characters (eg. è). How do I convert from a Java String to Windows-1252 format (which supports the French characters)?
I am currently doing this:
String encodedURL = new String (unencodedUrl.getBytes("UTF-8"), "Windows-1252");
However, it makes:
param=Stationnement extèrieur into param=Stationnement extérieur .
How do I fix this? Any suggestions?
Edit for further clarification:
The user chooses values from a drop down. When the language is French, the values from the drop down sometimes include French characters, like 'è'. When I send this request to the server, it fails, saying it is unable to decipher the request. I have to figure out how to send the 'è' as a different format (preferably Windows-1252) that supports French characters. I have chosen to send as Windows-1252. The server will accept this format. I don't want to replace each character, because I could miss a special character, and then the server will throw an exception.

Use URLEncoder to encode parameter values as application/x-www-form-urlencoded data:
String param = "param="
+ URLEncoder.encode("Stationnement extr\u00e8ieur", "cp1252");
See here for an expanded explanation.

Try using
String encodedURL = new String (unencodedUrl.getBytes("UTF-8"), Charset.forName("Windows-1252"));

As per McDowell's suggestion, I tried encoding doing:
URLEncoder.encode("stringValueWithFrechCharacters", "cp1252") but it didn't work perfectly. I replayced "cp1252" with HTTP.ISO_8859_1 because I believe Android does not have the support for Windows-1252 yet. It does allow for ISO_8859_1, and after reading here, this supports MOST of the French characters, with the exception of 'Œ', 'œ', and 'Ÿ'.
So doing this made it work:
URLEncoder.encode(frenchString, HTTP.ISO_8859_1);
Works perfectly!

How do I correctly set the HTTP Content-Disposition for large file names in Java?

I'm working on some requirements that will lead to arbitrary PDF files being downloaded from a J2EE web server. The names may look like this:
Xxxxxxxxxxxxxxxxxx - Yyyyyyyyyy - Aaaaaaaaaaa - Bbbbbbbb ccc Dddddddddddddd - abc1234560 - 2009-03-26 – 235959.pdf
Now I've read a couple of sections in RFC2183:
http://www.ietf.org/rfc/rfc2183.txt
For instance
A short (length <= 78 characters) parameter value containing
only non-tspecials' characters SHOULD be represented as a single
token'. A short parameter value containing only ASCII characters,
but including tspecials' characters, SHOULD be represented as
quoted-string'. Parameter values longer than 78 characters, or
which contain non-ASCII characters, MUST be encoded as specified in
[RFC 2184].
Etc etc. Now there are millions of things that can go wrong, if I don't read through all of those RFC's... Or I choose a library which handles such RFC specifications. Is there any such thing for Java? Or am I paranoid, and actually it's sufficient to just write this header to the out stream:
String filename = "\"" + filename.replace("\"", "\\\"") + "\"";
addHeader("Content-Disposition", "attachment; filename=" + filename);

I had similar problem in past and found the following solution.
The first URL looks like http://myhost.com/file/1234
where 1234 is the file ID. Let's say that the file name should be my-very-long-file-name.pdf. So, instead of setting HTTP header redirect the call to URL like
http://myhost.com/download/1234/my-very-long-file-name.pdf
The sevlet mapped to /download/ will take ID from URL and print the file to its output stream. But browser will extract the file name from URL and offer you to download and save the file because its name is into the URL. I hope this will work for you also for long file names.

RFC 2183 isn't relevant, RFC 6266 is.
Also, the 78 character limit only applies to email, not http, so you don't have to worry about that.

Java : How to accommodate special characters in the filename while uploading and downloading the file?

Background:
I have a file which I upload, during this process the link of the file is stored in the database and not the actual file, acutal file is stored in the File System, currently am storing it in my local machine.
Goal:
My goal is to upload a file and download a file properly which has special characters in it - #,$,%,# etc.
Issue:
I am able to upload the file with special character but am not able to download file with special characters. Also I cannot do any changes in the Download Servlet as it is part of the Framework, so all I can work with is the Upload Servlet, so my focus is to upload file with special characters in such a way so that I can download them.
I have tried creating an alias for the filename where in am replacing the special characters with '_' symbol, this approach works fine and am able to download the file but actual name of file is not maintained in here, all special characters in the filename are replaced by '_' symbol and this is not acceptable as user should actual name of the file.
Any suggestions or approach:
Code:
public ModelAndView save(HttpServletRequest request, HttpServletResponse response, Object command,
ModelAndView modelView, BindException errors) throws Exception {
String newFileName = checkForSpecialCharsAndGetNewFileName(file.getOriginalFilename());
System.out.println("alias filename="+ newFileName);
String url = "f" + (String.valueOf(System.currentTimeMillis())) + "_" + newFileName;
String fileName = file.getOriginalFilename();
System.out.println("FileName "+ fileName);
}
//Code to replace all special characters in the incoming file with '_' symbol.
private String checkForSpecialCharsAndGetNewFileName (String originalFileName) {
final String[] splChars = {"#", "+", "$"};
String newString = originalFileName;
for (int i=0; i<splChars.length; i++)
newString = StringUtils.replace(newString, splChars[i], "_");
return newString;
}
Hope am making some sense here.
Thanks.

If I am understanding you correctly, you want to encode the filename such that when you upload it, and later download it, you want to be able to find the same file from the file name.
To do this, you can use URLEncoder and URLDecoder classes.
You can do this doing something like the following:
String fileName;
fileName = URLEncoder.encode("My ! String #", "UTF-8");
That will encode it. To get the original file name:
String originalFileName = URLDecoder.decode(fileName, "UTF-8");
You can use the encoded file name to download the file from the service. You can then decode the file name to store it appropriately.
Hope that helps.

I have a problem where the java.io.File class has started auto encoding tildes that are contained in the filename thats passed in the constructor.
So for example if you instantiate a file using "~filename" it will internally interpret it as "%7Efilename" so that if you need to read or write to a file named "~filename" there is no way to do it.
The issue was introduced when I endorsed a newer set of Xalan / Xerces jars (the full set of 5) on the tomcat server. If you remove the endorsed jars the issue immediately goes away (go figure).
If that is similar to your issue, you may need to look to see if your server is using any endorsed XML parsing jars and consider removing them. I haven't figured out a way to make the newer xerces jars play well with java.io.File or even understand why there is an impact here to begin with.