We Keep Coding

How to resolve UTF-8 enconding in JSP on tomcat server? [duplicate]

I have used Java Mail API, for sending emails. I am using a contact formular to send the input, which has to be send to a specific email.
The email is send without problems, though I am a danish guy, and I am therefore in need of three danish characters which is 'æ', 'ø' and 'å', in the subject and the email text.
I have therefore seen that I can use UTF-8 character encoding, to provide these characters, but when my mail is send I only see some strange letters - 'ã¦', 'ã¸' and 'ã¥' - instead of the danish letters - 'æ', 'ø' and 'å'.
My method to send the email is looking like this:
public void sendEmail(String name, String fromEmail, String subject, String message) throws AddressException, MessagingException, UnsupportedEncodingException, SendFailedException
{
//Set Mail properties
Properties props = System.getProperties();
props.setProperty("mail.smtp.starttls.enable", "true");
props.setProperty("mail.smtp.host", "smtp.gmail.com");
props.setProperty("mail.smtp.socketFactory.port", "465");
props.setProperty("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
props.setProperty("mail.smtp.auth", "true");
props.setProperty("mail.smtp.port", "465");
Session session = Session.getDefaultInstance(props, new javax.mail.Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("my_username", "my_password");
}
});
//Create the email with variable input
MimeMessage mimeMessage = new MimeMessage(session);
mimeMessage.setHeader("Content-Type", "text/plain; charset=UTF-8");
mimeMessage.setFrom(new InternetAddress(fromEmail, name));
mimeMessage.setRecipient(Message.RecipientType.TO, new InternetAddress("my_email"));
mimeMessage.setSubject(subject, "utf-8");
mimeMessage.setContent(message, "text/plain");
//Send the email
Transport.send(mimeMessage);
}
Please help me find out how I can correct this 'error'.

For all e-mails
There are a couple of system properties related to mailing, that can probably simplify your code. I am talking about this specific property actually: "mail.mime.charset".
The mail.mime.charset System property can be used to specify the default MIME charset to use for encoded words and text parts that don't otherwise specify a charset. Normally, the default MIME charset is derived from the default Java charset, as specified in the file.encoding System property. Most applications will have no need to explicitly set the default MIME charset. In cases where the default MIME charset to be used for mail messages is different than the charset used for files stored on the system, this property should be set.
As you can read above, by default there is no value for the mail.mime.charset and the file encoding (file.encoding property) is used.
For a specific e-mail
However, if you want to specify a specific encoding for a specific e-mail, then you should probably use the 2 parameter setSubject(subject,charset) and setText(text,charset) methods.
If that doesn't work, then probably your input is already corrupted before it reached this point. In other words, you probably used the wrong encoding to collect your data.
Mime types are complicated
The setContent(content, "UTF-8") (as other sources claim) will just not work. Just look at the signature of this method: setContent(Object content, String mimetype). Mime type and charset are 2 totally different things. Imho, you should really be using one of the setText(...) methods with a charset parameter.
But if you persist in using a mimetype to set the charset setContent(content,mimetype), then use the correct format. (not just "UTF-8", but something like "text/plain; charset=UTF-8"). But more importantly, be aware that every mime-type has its own way of handling charsets.
As specified in RFC-2046 the default charset for text/plain is US-ASCII, but can be overruled with an additional charset parameter.
However, in RFC-6657 makes clear that the text/xml type determines the charset using the content of the message. The charset parameter will just be ignored here.
And in RFC-2854 is stated that text/html should really always specify a charset. But if you don't, then it will use ISO-8859-1 (=Latin-1).

Maybe You should provide also UTF-8 here
mimeMessage.setContent(message, "text/plain; charset=UTF-8");
You have to look at http://www.coderanch.com/t/274480/java/java/JavaMail-set-content-utf

After spending a lot of time on debugging, and searching the internet for a clue, I have found a solution to my problem.
It seems that whenever I sended data through a web request, my application didn't encode the characters with UTF-8 encoding. This meant that the data which was send from my contact form, which contained æ, ø and å characters, couldn't be handled correct by the character encoding.
The solution seemed to setup a Character Encoding Filter, in my Deployment Descriptor, which would encode all incoming request from the web to be with the character encoding UTF-8.
private void registerCharacterEncodingFilter(ServletContext servletContext) {
CharacterEncodingFilter encodingFilter = new CharacterEncodingFilter();
encodingFilter.setEncoding("UTF-8");
encodingFilter.setForceEncoding(true);
FilterRegistration.Dynamic characterEncodingFilter = servletContext.addFilter("characterEncodingFilter", encodingFilter);
characterEncodingFilter.addMappingForUrlPatterns(null, false, "/*");
}
This filter sets the encoding to be UTF-8 and force the encoding to all requests comming at the url ' /* '.

It's easy,
run your project with parameter -Dfile.encoding=UTF-8
ex:
java -Dfile.encoding=UTF-8 -jar MyProject.jar
//Fix a typo

Before sending your String to the send method, you must convert the String into UTF-8
If you are receiving a "request" parameter, you can use "setCharacterEncoding":
request.setCharacterEncoding("utf-8");
String subject = request.getParameter("subject");
String content = request.getParameter("content");
...
MimeMessage mineMessage = new MimeMessage(session);
mineMessage.setFrom(new InternetAddress(myAccountEmail));
mineMessage.setRecipient(Message.RecipientType.TO, new InternetAddress(recepient));
mineMessage.setSubject(subject, "UTF-8");
mineMessage.setContent(content, "text/plain;charset=UTF-8");
Otherwise, convert your String into UTF-8 format with the following method:
String subject = new String(subject.getBytes(Charset.forName("ISO-8859-1")), Charset.forName("UTF-8"));
String content = new String(content.getBytes(Charset.forName("ISO-8859-1")), Charset.forName("UTF-8"));
...
MimeMessage mineMessage = new MimeMessage(session);
mineMessage.setFrom(new InternetAddress(myAccountEmail));
mineMessage.setRecipient(Message.RecipientType.TO, new InternetAddress(recepient));
mineMessage.setSubject(subject, "UTF-8");
mineMessage.setContent(content, "plain/plain;charset=UTF-8");
This is the result in Spanish.

mimeMessage.setContent(mail.getBody(), "text/html; charset=UTF-8");
maybe iam wrong, but this work for me. :) any ööö, äää, üüü character will shown correctly in my outlook.
outlook screenshot

I know I'm late to this question, but I had a similar problem just now.
It may be worth it to check your source encodings too! I was using a test class, with hardcoded subject/text containing some special characters, which kept coming garbled when sending the email. Even though I had set the charset UTF-8 wherever applicable (mimeMessage.setSubject(subject, charset), mimeMessage.setContent(content, "text/plain; charset=UTF-8")).
Then I noted that the source encoding of this class was windows-1252. From my understanding, when a java file is compiled, any source texts are converted to UTF-8. But in this case, in the maven pom.xml for this project, the project.build.sourceEncoding property was missing - so I'm actually not sure which encoding maven was using during compilation (by default) since none was specified.
Changing the source encoding was not possible here, but as soon as I changed the special characters to Unicode code literals (e.g. "ü" to "\u00fc"), the whole thing worked fine.

Maybe is too later, but there is a very simple method to fix this problem.
Just call this constructor to create a MimeMessageHelper that encode UTF-8 as we escpect:
MimeMessage **mimeMessage** = mailSender.createMimeMessage();
MimeMessageHelper **helper** = new MimeMessageHelper(mimeMessage, false(or true if you want include Multipart), "UTF-8");
No more actions are needed, continue the mail sending flow as you wish.

sending an Arabic text in email

I have a problem sending an Arabic text in email with java language.
this is my message in properties file:
mail.send=تجربة
I use this syntax in jave :
ResourceBundle dq_resource = ResourceBundle.getBundle("nls.myfile_ar");
String text= dq_resource.getString("mail.send")
but when I received email U have this text : اÙ?Ù?Ù?ضÙ?ع
I try also in java with this code :
String text= new String(dq_resource.getString("mail.send").getBytes(),Charset.forName("UTF-8"));
but I have this text in mail :
ا�?�?�?ض�?ع

You need to set a header for the mail, something like
message.setHeader("Content-Type", "text/plain; charset=UTF-8");
setHeader is a method of the Message class that allows you to set a header.

The easiest solution could be using strings normally ( without getBytes method ) changing the default encoding in your workspace for example eclipse.
Windows-->Preferences-->General-->workspace-->Text file encoding
Also you can try to convert UTF-8 to UTF-16 .

JavaMail: msg.setSubject() won't accept second argument (encoding) in version 1.4.7, throws compile time error

I have a Java SE 7 project that uses maven with dependency:
<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.4.7</version>
</dependency>
Recently I noticed problems with sending mails - subjects that contained some special chars (specific to Polish language) weren't properly displayed.
I found few questions on SO with solution: add second argument to your setSubject() method, like this:
Message msg = new MimeMessage(session);
String subject = "ĄŻĄŻŚśążćół";
msg.setSubject(subject, "utf-8");
Looks nice, but NetBeans warned me that there is no overloaded method for setSubject() that accept two arguments. I ignored that, thought that "it's just NetBeans, probably it's some kind of internal error", but I was wrong.
I ran mvn clean install exec:java from terminal, and I got:
error: method setSubject in class Message cannot be applied to given types;
What's wrong? Documentation says that this version (1.4.7) supports specifing encoding in subject, every answer I found relies on that...

Note that setSubject(String subject, String charset) is added to the MimeMessage class. I.e. it is not part of the Message class.
So, changing:
Message msg = new MimeMessage(session);
to
MimeMessage msg = new MimeMessage(session);
should fix your issue.

Two things:
According to the API documentation online setSubjects only
takes a String as parameter.
There would be no point in passing an
additional encoding, because you already have a String.
Specifying encoding makes sense when converting between bytes and Strings. E.g. this String constructor (quoted from Oracle's Java API documentation):
String(byte[] bytes, int offset, int length, Charset charset)
Constructs a new String by decoding the specified subarray of bytes
using the specified charset.
Here you already deliver a String (with non-ascii charaters), so having an encoding specified makes no sense.

UTF-8 charset doesn't work with javax.mail

I have used Java Mail API, for sending emails. I am using a contact formular to send the input, which has to be send to a specific email.
The email is send without problems, though I am a danish guy, and I am therefore in need of three danish characters which is 'æ', 'ø' and 'å', in the subject and the email text.
I have therefore seen that I can use UTF-8 character encoding, to provide these characters, but when my mail is send I only see some strange letters - 'ã¦', 'ã¸' and 'ã¥' - instead of the danish letters - 'æ', 'ø' and 'å'.
My method to send the email is looking like this:
public void sendEmail(String name, String fromEmail, String subject, String message) throws AddressException, MessagingException, UnsupportedEncodingException, SendFailedException
{
//Set Mail properties
Properties props = System.getProperties();
props.setProperty("mail.smtp.starttls.enable", "true");
props.setProperty("mail.smtp.host", "smtp.gmail.com");
props.setProperty("mail.smtp.socketFactory.port", "465");
props.setProperty("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
props.setProperty("mail.smtp.auth", "true");
props.setProperty("mail.smtp.port", "465");
Session session = Session.getDefaultInstance(props, new javax.mail.Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("my_username", "my_password");
}
});
//Create the email with variable input
MimeMessage mimeMessage = new MimeMessage(session);
mimeMessage.setHeader("Content-Type", "text/plain; charset=UTF-8");
mimeMessage.setFrom(new InternetAddress(fromEmail, name));
mimeMessage.setRecipient(Message.RecipientType.TO, new InternetAddress("my_email"));
mimeMessage.setSubject(subject, "utf-8");
mimeMessage.setContent(message, "text/plain");
//Send the email
Transport.send(mimeMessage);
}
Please help me find out how I can correct this 'error'.

For all e-mails
There are a couple of system properties related to mailing, that can probably simplify your code. I am talking about this specific property actually: "mail.mime.charset".
The mail.mime.charset System property can be used to specify the default MIME charset to use for encoded words and text parts that don't otherwise specify a charset. Normally, the default MIME charset is derived from the default Java charset, as specified in the file.encoding System property. Most applications will have no need to explicitly set the default MIME charset. In cases where the default MIME charset to be used for mail messages is different than the charset used for files stored on the system, this property should be set.
As you can read above, by default there is no value for the mail.mime.charset and the file encoding (file.encoding property) is used.
For a specific e-mail
However, if you want to specify a specific encoding for a specific e-mail, then you should probably use the 2 parameter setSubject(subject,charset) and setText(text,charset) methods.
If that doesn't work, then probably your input is already corrupted before it reached this point. In other words, you probably used the wrong encoding to collect your data.
Mime types are complicated
The setContent(content, "UTF-8") (as other sources claim) will just not work. Just look at the signature of this method: setContent(Object content, String mimetype). Mime type and charset are 2 totally different things. Imho, you should really be using one of the setText(...) methods with a charset parameter.
But if you persist in using a mimetype to set the charset setContent(content,mimetype), then use the correct format. (not just "UTF-8", but something like "text/plain; charset=UTF-8"). But more importantly, be aware that every mime-type has its own way of handling charsets.
As specified in RFC-2046 the default charset for text/plain is US-ASCII, but can be overruled with an additional charset parameter.
However, in RFC-6657 makes clear that the text/xml type determines the charset using the content of the message. The charset parameter will just be ignored here.
And in RFC-2854 is stated that text/html should really always specify a charset. But if you don't, then it will use ISO-8859-1 (=Latin-1).

Maybe You should provide also UTF-8 here
mimeMessage.setContent(message, "text/plain; charset=UTF-8");
You have to look at http://www.coderanch.com/t/274480/java/java/JavaMail-set-content-utf

After spending a lot of time on debugging, and searching the internet for a clue, I have found a solution to my problem.
It seems that whenever I sended data through a web request, my application didn't encode the characters with UTF-8 encoding. This meant that the data which was send from my contact form, which contained æ, ø and å characters, couldn't be handled correct by the character encoding.
The solution seemed to setup a Character Encoding Filter, in my Deployment Descriptor, which would encode all incoming request from the web to be with the character encoding UTF-8.
private void registerCharacterEncodingFilter(ServletContext servletContext) {
CharacterEncodingFilter encodingFilter = new CharacterEncodingFilter();
encodingFilter.setEncoding("UTF-8");
encodingFilter.setForceEncoding(true);
FilterRegistration.Dynamic characterEncodingFilter = servletContext.addFilter("characterEncodingFilter", encodingFilter);
characterEncodingFilter.addMappingForUrlPatterns(null, false, "/*");
}
This filter sets the encoding to be UTF-8 and force the encoding to all requests comming at the url ' /* '.

It's easy,
run your project with parameter -Dfile.encoding=UTF-8
ex:
java -Dfile.encoding=UTF-8 -jar MyProject.jar
//Fix a typo

Before sending your String to the send method, you must convert the String into UTF-8
If you are receiving a "request" parameter, you can use "setCharacterEncoding":
request.setCharacterEncoding("utf-8");
String subject = request.getParameter("subject");
String content = request.getParameter("content");
...
MimeMessage mineMessage = new MimeMessage(session);
mineMessage.setFrom(new InternetAddress(myAccountEmail));
mineMessage.setRecipient(Message.RecipientType.TO, new InternetAddress(recepient));
mineMessage.setSubject(subject, "UTF-8");
mineMessage.setContent(content, "text/plain;charset=UTF-8");
Otherwise, convert your String into UTF-8 format with the following method:
String subject = new String(subject.getBytes(Charset.forName("ISO-8859-1")), Charset.forName("UTF-8"));
String content = new String(content.getBytes(Charset.forName("ISO-8859-1")), Charset.forName("UTF-8"));
...
MimeMessage mineMessage = new MimeMessage(session);
mineMessage.setFrom(new InternetAddress(myAccountEmail));
mineMessage.setRecipient(Message.RecipientType.TO, new InternetAddress(recepient));
mineMessage.setSubject(subject, "UTF-8");
mineMessage.setContent(content, "plain/plain;charset=UTF-8");
This is the result in Spanish.

mimeMessage.setContent(mail.getBody(), "text/html; charset=UTF-8");
maybe iam wrong, but this work for me. :) any ööö, äää, üüü character will shown correctly in my outlook.
outlook screenshot

I know I'm late to this question, but I had a similar problem just now.
It may be worth it to check your source encodings too! I was using a test class, with hardcoded subject/text containing some special characters, which kept coming garbled when sending the email. Even though I had set the charset UTF-8 wherever applicable (mimeMessage.setSubject(subject, charset), mimeMessage.setContent(content, "text/plain; charset=UTF-8")).
Then I noted that the source encoding of this class was windows-1252. From my understanding, when a java file is compiled, any source texts are converted to UTF-8. But in this case, in the maven pom.xml for this project, the project.build.sourceEncoding property was missing - so I'm actually not sure which encoding maven was using during compilation (by default) since none was specified.
Changing the source encoding was not possible here, but as soon as I changed the special characters to Unicode code literals (e.g. "ü" to "\u00fc"), the whole thing worked fine.

Maybe is too later, but there is a very simple method to fix this problem.
Just call this constructor to create a MimeMessageHelper that encode UTF-8 as we escpect:
MimeMessage **mimeMessage** = mailSender.createMimeMessage();
MimeMessageHelper **helper** = new MimeMessageHelper(mimeMessage, false(or true if you want include Multipart), "UTF-8");
No more actions are needed, continue the mail sending flow as you wish.

Encoding Problems

Having an issue with a java string used for emails in a java source file. The string contains "Protégé". Our server environment from what I have been able to determine uses UTF-8.
So I converted it to "Protégé" for UTF-8. It works great on our server, but when I run it locally it doesn't translate it properly. So I changed eclipse to use UTF-8 under preferences but it doesn't translate it locally. Still shows "Protégé". Any ideas?
From the comments:
I ran this locally and on our server:
OutputStreamWriter out = new OutputStreamWriter(new ByteArrayOutputStream());
System.out.println(out.getEncoding());
And it displays Cp1252 locally and UTF-8 on our JBoss server. We originally had the string with "Protégé" but on JBoss it only
shows "Prot".
When I use "Prot\u00e9g\u00e9" it works fine locally but when ran on our server it shows "Protg".

If the string contains "Prot\u00e9g\u00e9", this precludes a compiler encoding problem (like alluded by SyntaxT3rr0r), since it is now right in the Java String (unless there is a compiler bug, which I would not assume).
Thus we have an problem between output, transfer and display. How do you look at the output from your server? It could be that there somewhere is some recoding which destroys your strings. Or that somewhere some output is mis-declared.
If you are using a Terminal/command window to look at the output, consider setting it to UTF-8 before connecting to the server.
And yes, Java uses internally UTF-16 for the strings, but some system dependent encoding as both compiler default and default encoding of OutputStreamWriter/InputStreamReader and several other APIs which convert between strings and bytes. Looks like this is UTF-8 on the server and Windows-1252 on your client system. This should not really matter here.

Try this:
MimeMessage msg = new MimeMessage(session);
MimeBodyPart mbp1 = new MimeBodyPart();
mbp1.setDataHandler(new DataHandler(new ByteArrayDataSource(message.toString, "text/html")));
mbp1.setContent(new String(message.getBytes("UTF-8"),"ISO-8859-1"), "text/html");
Multipart mp = new MimeMultipart();
mp.addBodyPart(mbp1);
msg.setContent(mp, "text/html");
put your language char set instead of "ISO-8859-1"