We Keep Coding

Correctly compare code_verifier with code_challenge in Java

I'm using passport-oauth2 (passportjs.org and https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js) for OAuth2+PKCE integration in a nodejs application.
The backend it's authenticating against is written in Java.
The problem is that I can't seem to decode->hash the code_verifier to correctly match the code_challenge that comes from passport-oauth2.
I know that the Base64 encoding that comes from passport has been generated to be URL safe (no padding, no wrapping, replacements for + or /), so I'm using a Url Decoder:
Base64.getUrlDecoder().decode(...)
Then I'm using commons DigestUtils to generate a SHA256 of the decoded verifier and comparing it with the challenge. So the whole thing looks something like this:
java.util.Base64.Decoder decoder = java.util.Base64.getUrlDecoder();
String codeChallenge = // get the code challenge from my cache
byte[] decodedCodeChallenge = decoder.decode(codeChallenge);
byte[] decodedCodeVerifier = decoder.decode(codeVerifier);
if (!Arrays.equals(sha256(decodedCodeVerifier), decodedCodeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}
Example:
This code verifier: 5CFCAiZC0g0OA-jmBmmjTBZiyPCQsnq_2q5k9fD-aAY
should match this code challenge: Fw7s3XHRVb2m1nT7s646UrYiYLMJ54as0ZIU_injyqw once both have been Base64-url-decoded and the verifier has been SHA256 hashed, but it doesn't.
What am I doing wrong?

Just 5 minutes later I figured it out.
In passport-oauth2, the code verifier is Base64-url-encoded(random bytes):
verifier = base64url(crypto.pseudoRandomBytes(32))
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L236
The challenge is then Base64-url-encoded(sha256(verifier)), which expands to Base64-url-encoded(sha256(Base64-url-encoded(random bytes))):
challenge = base64url(crypto.createHash('sha256').update(verifier).digest());
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L242
So to do the verification, I don't need to decode anything. It was sha256-d in it's encoded state.
This worked in the end:
java.util.Base64.Encoder encoder = java.util.Base64.getUrlEncoder();
String codeChallenge = // get code challenge from my cache;
String encodedVerifier = new String(encoder.encode(sha256(codeVerifier))).split("=")[0]; // Remember to remove padding
if (!encodedVerifier.equals(codeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}

encode a groovy variable using base64

I have a string like this and am trying to encode this string using base64
{"htmlBrowserType":"Default","mimeType":"text/html","url":"https://github.comcast.com"}
String base64Config = {"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com"}
Actually it is a groovy code
def encoded = base64Config.bytes.encodeBase64().toString()
While encoding using the tool am getting
eyJodG1sQnJvd3NlclR5cGUiOiJEZWZhdWx0IiwibWltZVR5cGUiOiJ0ZXh0L2h0bWwiLCJ1cmwiOiJodHRwczovL2dpdGh1Yi5jb21jYXN0LmNvbSJ9
But it is not working using java code while am decoding the string getting as the result of the above java code is not
{"htmlBrowserType":"Default","mimeType":"text/html","url":"https://github.comcast.com"}

Your base64Config is not a String, but Closure:
String base64Config = {"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com"}
It should be:
String base64Config = "{\"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com\"}"

Unable to convert bytes from charset 65535 in to Japanese (5035)

I have a character set conversion issue:
I am updating Japanese Kanji characters in DB2 in iSeries system with the following conversion method:
AS400 sys = new AS400("<host>","username","password");
CharConverter charConv = new CharConverter(5035, sys);
byte[] b = charConv.stringToByteArray(5035, sys, "試験");
AS400Text textConverter = new AS400Text(b.length, 65535,sys);
While retrieving, I use the following code to convert & display:
CharConverter charConv = new CharConverter(5035, sys);
byte[] bytes = charConv.stringToByteArray(5035, sys, dbRemarks);
String s = new String(bytes);
System.out.println("Remarks after conversion to AS400Text :"+s);
But, the system is displaying garbled characters while displaying. Can anybody help me to decode Japanese characters from binary storage?

Well I don't know anything about CharConverter or AS400Text, but code like this is almost always a mistake:
String s = new String(bytes);
That uses the platform default encoding to convert the binary data to text.
Usually storage and retrieval should go through opposite processes - so while you've started with a string and then converted it to bytes, and converted that to an AS400Text object when storing it, I'd expect you to start with an AS400Text object, convert that to a byte array, and then convert that to a String using CharConverter when fetching. The fact that you're calling stringToByteArray in both cases suggests there's something amiss.
(It would also help if you'd tell us what dbRemarks is, and how you've fetched it.)
I do note that having checked some documentation for AS400Text, I've seen this:
Due to recent changes in the behavior of the character conversion routines, this system object is no longer necessary, except when the AS400Text object is to be passed as a parameter on a Toolbox Proxy connection.
There's similar documentation for CharConverter. Are you sure you actually need to go through this at all? Have you tried just storing the string directly and retrieving it directly, without going through intermediate steps?

Thank you Jon Skeet!
Yes. I have committed a mistake, not encoding the string while declaration.
My issue is to get the data stored in DB2, convert it into Japanese and provide for editing in web page. I am getting dbRemarks from the result set. I have missed another thing in my post:
While inserting, I am converting to text like:
String text = (String) textConverter.toObject(b);
PreparedStatement prepareStatementUpdate = connection.prepareStatement(updateSql);
prepareStatementUpdate.setString(1, text);
int count = prepareStatementUpdate.executeUpdate();
I am able to retrieve and display clearly with this code:
String selectSQL = "SELECT remarks FROM empTable WHERE emp_id = ? AND dep_id=? AND join_date='2013-11-15' ";
prepareStatement = connection.prepareStatement(selectSQL);
prepareStatement.setInt(1, 1);
prepareStatement.setString(2, 1);
ResultSet resultSet = prepareStatement.executeQuery();
while ( resultSet.next() ) {
byte[] bytedata = resultSet.getBytes( "remarks" );
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);
}
It is working fine with JDBC, but for working with JPA, I need to convert to string for editing in web page or store in table. So, I have tried this way, but could not succeed:
String remarks = resultSet.getString( "remarks" );
byte[] bytedata = remarks.getBytes();
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);

Thanks a lot Jon and Buck Calabro !
With your clues, I have succeeded with the following approach:
String remarks = new String(resultSet.getBytes("remarks"),"SJIS");
byte[] byteData = remarks.getBytes("SJIS");
CharConverter charConv = new CharConverter(5035, sys);
String convertedStr = charConv.byteArrayToString(5035, sys, byteData);
I am able to convert from string. I am planning to implement the same with JPA, and started coding.

What's the difference between new String(byte[]) and DatatypeConverter.printBase64Binary(byte[])?

I need to pass base64 encoded data into xml as a string value. I noticed that code below prints different string representation. Which one is correct and why?
String example = "Hello universe!";
byte[] base64data = Base64.encodeBase64(example.getBytes());
System.out.println(new String(base64data));
System.out.println(DatatypeConverter.printBase64Binary(base64data));
System.out.println(new String(Base64.decodeBase64(base64data), "UTF-8"));
And what I get as a result:
SGVsbG8gdW5pdmVyc2Uh
U0dWc2JHOGdkVzVwZG1WeWMyVWg=
Hello universe!

U0dWc2JHOGdkVzVwZG1WeWMyVWg= decoded is SGVsbG8gdW5pdmVyc2Uh which is Hello universe! encoded. So you did the encoding twice.
There is no difference. You are using the API the wrong way. Don't encode the already encoded data again.

MD5 Hash of ISO-8859-1 string in Java

I'm implementing an interface for digital payment service called Suomen Verkkomaksut. The information about the payment is sent to them via HTML form. To ensure that no one messes with the information during the transfer a MD5 hash is calculated at both ends with a special key that is not sent to them.
My problem is that for some reason they seem to decide that the incoming data is encoded with ISO-8859-1 and not UTF-8. The hash that I sent to them is calculated with UTF-8 strings so it differs from the hash that they calculate.
I tried this with following code:
String prehash = "6pKF4jkv97zmqBJ3ZL8gUw5DfT2NMQ|13466|123456||Testitilaus|EUR|http://www.esimerkki.fi/success|http://www.esimerkki.fi/cancel|http://www.esimerkki.fi/notify|5.1|fi_FI|0412345678|0412345678|esimerkki#esimerkki.fi|Matti|Meikäläinen||Testikatu 1|40500|Jyväskylä|FI|1|2|Tuote #101|101|1|10.00|22.00|0|1|Tuote #202|202|2|8.50|22.00|0|1";
String prehashIso = new String(prehash.getBytes("ISO-8859-1"), "ISO-8859-1");
String hash = Crypt.md5sum(prehash).toUpperCase();
String hashIso = Crypt.md5sum(prehashIso).toUpperCase();
Unfortunately both hashes are identical with value C83CF67455AF10913D54252737F30E21. The correct value for this example case is 975816A41B9EB79B18B3B4526569640E according to Suomen Verkkomaksut's documentation.
Is there a way to calculate MD5 hash in Java with ISO-8859-1 strings?
UPDATE: While waiting answer from Suomen Verkkomaksut, I found an alternative way to make the hash. Michael Borgwardt corrected my understanding of String and encodings and I looked for a way to make the hash from byte[].
Apache Commons is an excellent source of libraries and I found their DigestUtils class which has a md5hex function which takes byte[] input and returns a 32 character hex string.
For some reason this still doesn't work. Both of these return the same value:
DigestUtils.md5Hex(prehash.getBytes());
DigestUtils.md5Hex(prehash.getBytes("ISO-8859-1"));

You seem to misunderstand how string encoding works, and your Crypt class's API is suspect.
Strings don't really "have an encoding" - an encoding is what you use to convert between Strings and bytes.
Java Strings are internally stored as UTF-16, but that does not really matter, as MD5 works on bytes, not Strings. Your Crypt.md5sum() method has to convert the Strings it's passed to bytes first - what encoding does it use to do that? That's probably the source of your problem.
Your example code is pretty nonsensical as the only effect this line has:
String prehashIso = new String(prehash.getBytes("ISO-8859-1"), "ISO-8859-1");
is to replace characters that cannot be represented in ISO-8859-1 with question marks.

Java has a standard java.security.MessageDigest class, for calculating different hashes.
Here is the sample code
include java.security.MessageDigest;
// Exception handling not shown
String prehash = ...
final byte[] prehashBytes= prehash.getBytes( "iso-8859-1" );
System.out.println( prehash.length( ) );
System.out.println( prehashBytes.length );
final MessageDigest digester = MessageDigest.getInstance( "MD5" );
digester.update( prehashBytes );
final byte[] digest = digester.digest( );
final StringBuffer hexString = new StringBuffer();
for ( final byte b : digest ) {
final int intByte = 0xFF & b;
if ( intByte < 10 )
{
hexString.append( "0" );
}
hexString.append(
Integer.toHexString( intByte )
);
}
System.out.println( hexString.toString( ).toUpperCase( ) );
Unfortunately for you it produces the same "C83CF67455AF10913D54252737F30E21" hash. So, I guess your Crypto class is exonerated. I specifically added the prehash and prehashBytes length printouts to verify that indeed 'ISO-8859-1' is used. In this case both are 328.
When I did presash.getBytes( "utf-8" ) it produced "9CC2E0D1D41E67BE9C2AB4AABDB6FD3" (and the length of the byte array became 332). Again, not the result you are looking for.
So, I guess Suomen Verkkomaksut does some massaging of the prehash string that they did not document, or you have overlooked.

Not sure if you solved your problem, but I had a similar problem with ISO-8859-1 encoded strings with nordic ä & ö characters and calculating a SHA-256 hash to compare with stuff in documentation. The following snippet worked for me:
import java.security.MessageDigest;
//imports omitted
#Test
public void test() throws ProcessingException{
String test = "iamastringwithäöchars";
System.out.println(this.digest(test));
}
public String digest(String data) throws ProcessingException {
MessageDigest hash = null;
try{
hash = MessageDigest.getInstance("SHA-256");
}
catch(Throwable throwable){
throw new ProcessingException(throwable);
}
byte[] digested = null;
try {
digested = hash.digest(data.getBytes("ISO-8859-1"));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
String ret = BinaryUtils.BinToHexString(digested);
return ret;
}
To transform bytes to hex string there are many options, including the apache commons codec Hex class mentioned in this thread.

If you send UTF-8 encoded data that they treat as ISO-8859-1 then that could be the source of your problem. I suggest you either send the data in ISO-8859-1 or try to communicate to Suomen Verkkomaksut that you're sending UTF-8. In a http-based protocol you do this by adding charset=utf-8 to Content-Type in the HTTP header.
A way to rule out some issues would be to try a prehash String that only contains characters that are encoded the same in UTF-8 and ISO-8859-1. From what I can see you can achieve this by removing all "ä" characters in the string you'e used.