We Keep Coding

How to encode Japanese characters javamail

So basically I'm trying to send an email with Japanese characters, something like "𥹖𥹖𥹖" and then I got "???" what should I do to encode this? I have looked over a bunch of solutions but none of them have helped me solve this.
here's the method I've been trying to do the encode:
public String encoding(String str) throws UnsupportedEncodingException{
String Encoding = "Shift_JIS";
return this.changeCharset(str, Encoding);
}
public String changeCharset(String str, String newCharset) throws UnsupportedEncodingException {
if (str != null) {
byte[] jis = str.getBytes("Shift_JIS");
return new String(bs, newCharset);
}
return null;
}

You're making this too complicated...
First, make sure you have the Japanese text in a proper Java String object, using proper Unicode characters.
Then, set the content of the body part using this method:
htmlPart.setText(japaneseString, "Shift_JIS", "html");

How to convert a string to another so that specific characters aren't allowed in the output?

I have a constraint: I cannot save some chars (like & and =) in a some special storage.
The problem is that I have strings (user input) that contain these not allowed special chars, which I'd like to save to that storage .
I'd like to convert such string to another string that wouldn't contain these special characters.
I'd like to still be able to convert back to the original string without creating ambiguity.
Any idea how to implement the de/convert? Thanks.

Convert the user input to Hex and save. And convert the hex value back to string. Use these methods.
public static String stringToHex(String arg) {
return String.format("%x", new BigInteger(1, arg.getBytes(Charset.forName("UTF-8"))));
}
public static String hexToString(String arg) {
byte[] bytes = DatatypeConverter.parseHexBinary(arg);
return new String(bytes, Charset.forName("UTF-8"));
}
Usage:
String h = stringToHex("Perera & Sons");
System.out.println(h);
System.out.println(hexToString(h));
OUTPUT
506572657261202620536f6e73
Perera & Sons

Already pointed out in the comments but URL Encoding looks like the way to go.
In Java done simply URLEncoder and URLDecoder
String encoded = URLEncoder.encode("My string &with& illegal = characters ", "UTF-8");
System.out.println("Encoded String:" + encoded);
String decoded = URLDecoder.decode(encoded, "UTF-8");
System.out.println("Decoded String:" + decoded);
URLEncoder
URLDecoder

How to decode Unicode characters in string [duplicate]

I have the following value in a string variable in Java which has UTF-8 characters encoded like below
Dodd\u2013Frank
instead of
Dodd–Frank
(Assume that I don't have control over how this value is assigned to this string variable)
Now how do I convert (encode) it properly and store it back in a String variable?
I found the following code
Charset.forName("UTF-8").encode(str);
But this returns a ByteBuffer, but I want a String back.
Edit:
Some more additional information.
When I use System.out.println(str); I get
Dodd\u2013Frank
I am not sure what is the correct terminology (UTF-8 or unicode). Pardon me for that.

try
str = org.apache.commons.lang3.StringEscapeUtils.unescapeJava(str);
from Apache Commons Lang

java.util.Properties
You can take advantage of the fact that java.util.Properties supports strings with '\uXXXX' escape sequences and do something like this:
Properties p = new Properties();
p.load(new StringReader("key="+yourInputString));
System.out.println("Escaped value: " + p.getProperty("key"));
Inelegant, but functional.
To handle the possible IOExeception, you may want a try-catch.
Properties p = new Properties();
try { p.load( new StringReader( "key=" + input ) ) ; } catch ( IOException e ) { e.printStackTrace(); }
System.out.println( "Escaped value: " + p.getProperty( "key" ) );

try
str = org.apache.commons.text.StringEscapeUtils.unescapeJava(str);
as org.apache.commons.lang3.StringEscapeUtils is deprecated.

Suppose you have a Unicode value, such as 00B0 (degree symbol, or superscript 'o', as in Spanish abbreviation for 'primero')
Here is a function that does just what you want:
public static String unicodeToString( char charValue )
{
Character ch = new Character( charValue );
return ch.toString();
}

I used StringEscapeUtils.unescapeXml to unescape the string loaded from an API that gives XML result.

UnicodeUnescaper from org.apache.commons:commons-text is also acceptable.
new UnicodeUnescaper().translate("Dodd\u2013Frank")

Perhaps the following solution which decodes the string correctly without any additional dependencies.
This works in a scala repl, though should work just as good in Java only solution.
import java.nio.charset.StandardCharsets
import java.nio.charset.Charset
> StandardCharsets.UTF_8.decode(Charset.forName("UTF-8").encode("Dodd\u2013Frank"))
res: java.nio.CharBuffer = Dodd–Frank

You can convert that byte buffer to String like this :
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.ByteBuffer
public static CharsetDecoder decoder = CharsetDecoder.newDecoder();
public static String byteBufferToString(ByteBuffer buffer)
{
String data = "";
try
{
// EDITOR'S NOTE -- There is no 'position' method for ByteBuffer.
// As such, this is pseudocode.
int old_position = buffer.position();
data = decoder.decode(buffer).toString();
// reset buffer's position to its original so it is not altered:
buffer.position(old_position);
}
catch (Exception e)
{
e.printStackTrace();
return "";
}
return data;
}

new line appending on my encrypted string

In Main:
public static void main(String[] args) throws NoSuchAlgorithmException {
System.out.println("encrypt:" + encryptPassword("superuser")+":" );
}
public static String encryptPassword(final String password) throws NoSuchAlgorithmException {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] hashPassword = md.digest(password.getBytes());
String encryPass = Base64.encodeBase64String(hashPassword);
return encryPass;
}
I'm getting this output:
encrypt:C66i8K4gFQ23j1jN2sRCqQ==:
But when I implemented the same thing in my application I'm getting the output below:
encrypt:C66i8K4gFQ23j1jN2sRCqQ==
:
Note: new line appending on my encrypted string.
application code:
public boolean authenticateUsernamePasswordInternal(UsernamePasswordCredentials credentials) {
try {
System.out.println("encrypt:" + getHash("superuser")+":" );
} catch (Exception e) {
logger.error(e.getMessage(), e);
throw new BadCredentialsAuthenticationException(ErrorConstants.CONNECTION_FAILED);
}
}
private String getHash(String password) throws NoSuchAlgorithmException, UnsupportedEncodingException{
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] hashPassword = md.digest(password.getBytes());
String encryPass = Base64.encodeBase64String(hashPassword);
return encryPass;
}
How I can remove that extra new line.
why this is happened, please help me what is the reason?

I may be late in answering this, but came across with same problem. Actually problem lies here
Base64.encodeBase64String(hashPassword)
Change that line to look like this it should work:
Base64.encodeBase64String(hashPassword,Base64.NO_WRAP)
By default the Android Base64 util adds a newline character to the end of the encoded string.
The Base64.NO_WRAP flag tells the util to create the encoded string without the newline character.
Check here

In case anyone needs this for any libraries using OkHttp, there's a Credentials class you can use for Base64 encoding your username/pass
String credentials = Credentials.basic("username", "password");
request.header(HttpHeaders.AUTHORIZATION, credentials);

Use:
String encryPass = Base64.encodeBase64String(hashPassword).trim();

A cleaner option without trimming:
String encryPass = BaseEncoding.base64().encode(hashPassword);

You just need to Use Base64 encoding in following way
Base64.encodeBase64String("Your data to encrypt in base64", Base64.DEFAULT)
Change above line with the followings
Base64.encodeBase64String("Your data to encrypt in base64",Base64.NO_WRAP)
It worked for me.

It depends on the implementation of Base64.encodeBase64String(). What is that method?
If it's from Apache commons, be aware that there are a few different classes that handle whitespace differently.
For example, org.apache.commons.net.util.Base64 chunks output, and it probably adds a CR-LF sequence to the final chunk.
The more common version, org.apache.commons.codec.binary.Base64, does not add whitespace.

JAVA Md5 returning non-deterministic results

I have written following function to compute Md5 checksum in Java.
class Utils {
public static String md5Hash(String input) {
String result = "";
try {
System.out.println("Input=" + input);
final MessageDigest md = MessageDigest.getInstance("MD5");
md.reset();
md.update(input.getBytes());
result = md.digest().toString();
} catch (Exception ee) {
System.err.println("Error computing MD5 Hash");
}
return result;
}
};
Calling Utils.md5Hash("abcde") multiple times gives different results. My understanding says md5 returns a deterministic and unique checksum for a string. Is that wrong? Else please let me know the bug in my implementation. Thanks

The toString() method of a byte array doesn't return a meaningful string. It returns the type of the array object, followed by the hashCode of the array.
Transform the byte array to a String using Hex or Base64 encoding if you want it printed. Apache commons-codec has methods to do that.
Also, make sure to specify en encoding which supports any kind of character to transform your string to a byte array. The method you're using uses the platform default encoding, which could fail if, for example, it's latin-1 and you're transforming non-latin-1 characters. UTF-8 is a good choice.

I have done using the following way :
public static String encryptedLoginPassword( String password )
{
String encryptedData="";
try{
MessageDigest algorithm = MessageDigest.getInstance("MD5");
byte[] defaultBytes = password.getBytes();
algorithm.reset();
algorithm.update(defaultBytes);
byte messageDigest[] = algorithm.digest();
StringBuffer hexString = new StringBuffer();
for (int i=0;i<messageDigest.length;i++) {
hexString.append(Integer.toHexString(0xFF & messageDigest[i]));
}
encryptedData=hexString.toString();
}catch(NoSuchAlgorithmException nsae){
}
return encryptedData;
}

int the code given by Dinup Kandel, I had to change this:
for (int i=0;i<messageDigest.length;i++) {
hexString.append(Integer.toHexString(0xFF & messageDigest[i]));
}
in to
if ((0xff & messageDigest[i]) < 0x10) {
hexString.append("0"
+ Integer.toHexString((0xFF & messageDigest[i])));
} else {
hexString.append(Integer.toHexString(0xFF & messageDigest[i]));
}
to get my unit tests working.
note: i used this to verify the correct answer:
echo -n MyTestString | md5sum