We Keep Coding

Decoding String (from header) encoded by Base64 and RFC2047 in Java

I'm working on a function to decode a string (from a header) that is encoded in both Base64 and RFC2047 in Java.
Given this header:
SGVhZGVyOiBoZWFkZXJ2YWx1ZQ0KQmFkOiBOYW1lOiBiYWRuYW1ldmFsdWUNClVuaWNvZGU6ID0/VVRGLTg/Qj81YmV4NXF5eTU2dUw2SUNNNTZ1TDVMcTY3N3lNNWJleDVxeXk2WUdVNklDTTZZR1U/PSA9P1VURi04P0I/NUxxNjc3eU01YmV4NW9tQTVMaU41cXl5Nzd5TTVZdS81cGE5NXBhODVMcTY0NENDPz0NCg0K
My expected output is:
Header: headervalue Bad: Name: badnamevalue Unicode:
己欲立而立人，己欲達而達人，己所不欲，勿施於人。
The only relevant function that I have found and tried was Base64.decodeBase64(headers), which produces this when printed out:
Header: headervalue Bad: Name: badnamevalue Unicode:
=?UTF-8?B?5bex5qyy56uL6ICM56uL5Lq677yM5bex5qyy6YGU6ICM6YGU?= =?UTF-8?B?5Lq677yM5bex5omA5LiN5qyy77yM5Yu/5pa95pa85Lq644CC?=
To solve this, I've been trying MimeUtility.decode() by converting the byte array returned from Base64.decodeBase64(headers) to InputStream, but the result was identical as above.
InputStream headerStream = new ByteArrayInputStream(Base64.decodeBase64(headers));
InputStream result = MimeUtility.decode(headerStream, "quoted-printable");
Have been searching around the internet but have yet found a solution, wondering if anyone knows ways to decode MIME headers from resulted byte arrays?
Any help is appreciated! It's also my first stack overflow post, apologies if I'm missing anything but please let me know if there's more information that I can provide!

The base64 you have there actually is what you pasted. Including the bizarre =?UTF-8?B? weirdness.
The stuff that follows is again base64.
There's base64-encoded data inside your base-64 encoded data. As Xzibit would say: I put some Base64 in your base64 so you can base64 while you base64. Why do I feel old all of a sudden?
In other words, the base64 input you get is a crazy, extremely inefficient format invented by a crazy person.
My advice is that you tell them to come up with something less insane.
Failing that:
Search the resulting string for the regex pattern and then again apply base64 decode to the stuff in the middle.
Also, you're using some third party base64 decoder, probably apache. Apache libraries tend to suck. Base64 is baked into java, there is no reason to use worse libraries here. I've fixed that; the Base64 in this snippet is java.util.Base64. Its API is slightly different.
String sourceB64 = "SGV..."; // that input base64 you have.
byte[] sourceBytes = Base64.decodeBase64(sourceB64);
String source = new String(sourceBytes, StandardCharsets.UTF_8);
Pattern p = Pattern.compile("=\\?UTF-8\\?B\\?(.*?)\\?=");
Matcher m = p.matcher(source);
StringBuilder out = new StringBuilder();
int curPos = 0;
while (m.find()) {
out.append(source.substring(curPos, m.start()));
curPos = m.end();
String content = new String(Base64.getDecoder().decode(m.group(1)), StandardCharsets.UTF_8);
out.append(content);
}
out.append(source.substring(curPos));
System.out.println(out.toString());
If I run that, I get:
Header: headervalue
Bad: Name: badnamevalue
Unicode: 己欲立而立人，己欲達而達 人，己所不欲，勿施於人。
Which looks exactly like what you want.
Explanation of that code:
It first base64-decodes the input, and turns that into a string. (Your idea of using InputStream is a red herring. That doesn't help at all here. You just want to turn bytes into a string, you do it as per line 3 of that snippet. Pass the byte array and the encoding those bytes are in, that's all you need to do).
It then goes on the hunt for =?UTF-8?B?--base64here--?= inside your base64. The base64-in-the-base64.
It then decoder that base64, turns it into a string in the same fashion, and replaces it.
It just adds everything besides those =?UTF-8?B?...?= segments verbatim.

Response has 2 bytes per character in Netty

In Netty, I create a response by feeding a String in body:
DefaultFullHttpResponse res = new DefaultFullHttpResponse(HTTP_1_1, httpResponse.getHttpResponseStatus());
if (body != null) {
ByteBuf buf = Unpooled.copiedBuffer(body, CharsetUtil.UTF_8);
res.content().writeBytes(buf);
buf.release();
res.headers().set(HttpHeaderNames.CONTENT_LENGTH, res.content().readableBytes());
}
When I look at the response, I see content-length being twice the length of the characters in the String. I understand the Java String contains 2 bytes per character, but I can't figure out how to prevent this in Netty when returning the request.
When I look at Cloudflare responses, these contain one byte per character. So there must be a way to change this. Ideas?

As #Chris O'Toole shows in How to convert a netty ByteBuf to a String and vice versa we must
first convert the String to Byte Array using the desired charset (UTF-8 works fine) String.getBytes(Charset),
then Unpooled.wrappedBuffer(byte[]) using the Byte Array instead the
String.
One byte per character for most characters, as #rossum stated.

Use US_ASCII charset instead of UTF-8. Haven't tested, try.

Input byte array has incorrect ending byte at 40

I have a string that is base64 encoded. It looks like this:
eyJibGExIjoiYmxhMSIsImJsYTIiOiJibGEyIn0=
Any online tool can decode this to the proper string which is {"bla1":"bla1","bla2":"bla2"}. However, my Java implementation fails:
import java.util.Base64;
System.out.println("payload = " + payload);
String json = new String(Base64.getDecoder().decode(payload));
I'm getting the following error:
payload = eyJibGExIjoiYmxhMSIsImJsYTIiOiJibGEyIn0=
java.lang.IllegalArgumentException: Input byte array has incorrect ending byte at 40
What is wrong with my code?

Okay, I found out. The original String is encoded on an Android device using android.util.Base64 by Base64.encodeToString(json.getBytes("UTF-8"), Base64.DEFAULT);. It uses android.util.Base64.DEFAULT encoding scheme.
Then on the server side when using java.util.Base64 this has to be decoded with Base64.getMimeDecoder().decode(payload) not with Base64.getDecoder().decode(payload)

I was trying to use the strings from the args. I found that if I use arg[0].trim() that it made it work. eg
Base64.getDecoder().decode(arg[0].trim());
I guess there's some sort of whitespace that gets it messed up.

Maybe too late, but I also had this problem.
By default, the Android Base64 util adds a newline character to the end of the encoded string.
The Base64.NO_WRAP flag tells the util to create the encoded string without the newline character.
Your android app should encode src something like this:
String encode = Base64.encodeToString(src.getBytes(), Base64.NO_WRAP);

Reading post data from servlet

I'm trying to read post data from a servelet doPost method the following way.
using httpservletrequest.getInputStream();
Creating an instance of bytearrayoutputstream.
writing the post data to bytearrayoutputstream from httpservletrequest.getInputStream();
Finally, output I'm getting from bytearrayoutputstream.toByteArray().
The problem with that is, when I enter 150/10 in the textfield, toByteArray gives me 150%2F10.
toByteArray seems to be encoding the special characters in the output for the / character.
What will be the elegant way to read post data from servlet doPost() method?

before you pass data to the servlet, try to encode it in client side. or else when you retrieve it, in the servlet, do a Url encode there.
try this way:
String abc = URLEncoder.encode("convert my string 150/10", "UTF-8");
byte[] barr = abc.getBytes(Charset.forName("UTF-8"));
use above with the
request.getParameter();
then you don't have to play with all these fancy codes. I guess when you convert into a byte array, you will have to specify the encoding mechanism to avoid char set ambiguity.
I did not try the above code, but would like you to have a try!

Url Routing with Encoding in Java Web API

I'm new to java and I'm trying to write a utility api to encrypt/decrypt a string. I have my controller and request mapping working, and I have the encrypt/decrypt methods working.
The problem I'm running into is decrypting a string that contains a forward slash ('/'). If I leave it unencoded in the url (such as http://localhost:8080/api/package/util/decrypt/oJfTtchpM9WC/4Oqpu7FZQ==) then url routing breaks (which is obvious why).
If I url encode it (such as http://localhost:8080/api/package/util/decrypt/oJfTtchpM9WC%2F4Oqpu7FZQ==) then the value in the path variable is empty.
Also note that http://localhost:8080/api/package/util/decrypt/oJfTtchpM9WC (no slash) works fine.
It seems whenever I put in a '%' into the path variable, I just get a blank page in response. Doesn't hit the method. Doesn't throw an exception (that I can find - again, I'm learning the environment still)
I've simplified my decrypt method to only output the path variable denoted as data
#RequestMapping(value = "/util/decrypt/{data}", method = RequestMethod.GET)
#ResponseBody
public String decrypt(HttpServletResponse httpResponse,
#PathVariable String data) throws Exception
{
return data;
// return URLDecoder.decode(data, "UTF-8");
}
Can anyone explain to me what is happening and how I can get this working?
Many thanks!!

I updated my encrypt method to use the function encodeBase64URLSafeString (vs just encodeBase64String) and also changed it to pass the data as a standard request param (ie ?data=string). This removed the problem.
Encodes binary data using a URL-safe variation of the base64 algorithm
but does not chunk the output. The url-safe variation emits - and _
instead of + and / characters. Note: no padding is added.