We Keep Coding

How do I maintain the backslash when converting to json String using Json Format of Protobuf?

I have to use gRPC.
I was converting the object I received into json string, and the following problem occurred
example proto
hash: "v\016\177\350\207y\225wM\335]1(Z\266\305\376\027\310_v\321\016Q\v\332\030\303^\032|\375"
but, However, if I convert using Protobuf's util JsonFormat, I get the following results
"hash": "dg5/6Id5lXdN3V0xKFq2xf4XyF920Q5RC9oYw14afP0="
I want to get this back to its original form, is there a way to write another library or decode it in reverse?

Forget about the format, basically; these are just two ways of representing the same data. The second version is base-64, and decodes to the bytes:
76-0E-7F-E8-87-79-95-77-4D-DD-5D-31-28-5A-B6-C5-FE-17-C8-5F-76-D1-0E-51-0B-DA-18-C3-5E-1A-7C-FD
The first version is C-literal style with octal escapes; v is ASCII 118, aka hex 0x76; \016 is escaped octal for decimal 14, aka hex 0x0E; \177 is escaped octal for decimal 127, aka hex 0x7F - and so on. Most languages have a base-64 encode/decode; the C-literal style with octal escape sequences is ... more niche, and you might need to write your own decoder for that. Depending on where the first string came from, it is worth noting that protobuf (at least the schema variant) also allows fixed-width unicode escapes, via \uNNNN and \UNNNNNNNN, IIRC. And note: the octal in .proto schemas can short-circuit: \12n means the same as \012n - at most 3 digits are taken, but if a non-digit character is encountered, it is still valid as a shorter form.

Differences between Crypt.crypt() and DigestUtils.md5() in apache.commons.Codec

I am writing a basic password cracker for the MD5 hashing scheme against a Linux /etc/shadow file. When I use commons.codec's DigestUtils or Crypt libraries, the hash length for them are different (among other things).
When I use the Crypt.crypt(passwordToHash, "$1$Jhe937$") the output is a 22-character string. When I use the DigestUtils.md5[Hex](passwordToHash + "Jhe937")(or the Java MessageDigest class) the output is a 32-character string (after converted). This makes no sense to me.
aside: is there no easy way to convert the DigestUtils.md5(passwordToHash)'s byte[] to a String. I've tried all* the ways and I get all non-valid output: Nz_èJÓ_µù[î¬y
*all being: new String(byte[], "UTF-8") and convert to char then to String

The executive summary is that while they'll perform the same hashing, the output format is different between the two so the lengths will be different. Read on for details.
MD5 is a message digesting algorithm that produces a 16 byte hash value, always (assuming valid input, etc.) Those bytes aren't all printable characters, they can take any value from 0-255 for any of the bytes, while the printable characters in ASCII are in the range 32-126.
DigestUtils.md5(String) generates the MD5 of the string and returns a 16 element byte array. DigestUtils.md5Hex(String) is a convenience wrapper (I'm assuming, I haven't looked at the source, but that's how I'd write it :-) ) around DigestUtils.md5 that takes the 16 element byte array md5 produces and base16 encodes it (also known as hex encoding). That replaces each byte with the equivalent two hex characters, which is why you get a 32 character String out of it.
Crypt.crypt uses a special format that goes back to the original Unix method of storing passwords. It's been extended over the years to use different hash/encryption algorithms, longer salts, and additional features. It also encodes it's output to be printable text, which is where the length difference is coming from. By using a salt of "$1$...", you're saying to use MD5, so the password plus the salt will be hashed using MD5, resulting in 16 bytes as expected, but because those bytes aren't necessarily printable, the hash is base64 encoded (using a slightly different alphabet than the standard base64 encoding), which replaces 3 bytes with 4 printable characters. So 16 bytes becomes 16 / 3 * 4 = 21-1/3 characters, rounded up to 22.
On your aside, DigestUtils.md5 produces 16 bytes, but those bytes can have any value from 0 to 255 and are (effectively) random. new String(byte[], "UTF-8") says the bytes in the byte array are a UTF-8 encoding, which is a very specific format. new String does it's best to treat the bytes as a UTF-8 encoded string, but because they're really not, you generally get gibberish out. If you want something printable, you'll have to use something that takes random bytes, not bytes in a specific format (like UTF-8). Two popular options are base16/hex encoding, which you can get with DigestUtils.md5Hex, or base64, which you can get with Base64.encodeBase64String(DigestUtils.md5(pwd + salt)).

Converting from Unicode to CP1252

I am reading in an encrypted file in JAVA, with many irregular characters. The problem is that I am trying to decode it, however the decryption algorithms are implemented using CP1252 encoding, and not the native Java Unicode.
What I am doing essentially, is reading in the bytes from this file and saving them into a char[]. The reason I am not using a byte array is because their values can be unsigned, and so I wanted to make sure there was no loss of data because of it.
Anyway my question is that, is there a way to convert from Unicode encoding to CP1252 encoding, without converting the data into bytes?
An example of what I need is this.
In CP1252: The value of € = 128 (I want this)
whereas in UNICODE: The value of € = 8364 (I read in this)
If I were to convert it using bytes, the value would not hold constant, this is why I am using char[].

Does an array of bytes with negative values lose information when converted to String?

I've got a code like this where in the encoding i convert the letters to bytes and then flip them with unary bitwise complement ~ at the end convert it to String.
After that i want to decrypt it with a similar method. The problem is that for two similar input Strings (but not the same) i get the same encoded String with the same hashcode.
Does the String(bytes) method lose the information because the bytes are negative or can i retrieve it somehow without changing my encryption part?
thanx
static String encrypt(String s){
byte[] bytes=s.getBytes();
byte[] enc=new byte[bytes.length];
for (int i=0;i<bytes.length;i++){
enc[i]=(byte) ~bytes[i];
}
return new String(enc);
}
static String decrypt(String s){
...

You should never use new String(...) to encode arbitrary binary data. That's not what it's there for.
Additionally, you should only very rarely use the default platform encoding, which is what you get when you call String.getBytes() and new String(byte[]) without specifying an encoding.
In general, encryption converts binary data to binary data. The normal process of encrypting a string to a string is therefore:
Convert the string into bytes with a known encoding (e.g. UTF-8)
Encrypt the binary data
Convert the encrypted binary data back into a string using base64.
Base64 is used to encode arbitrary binary data as ASCII data in a lossless fashion. Decryption is just a matter of reversing the steps:
Convert the base64 text back to a byte array
Decrypt the byte array
Decode the decrypted byte array as a string using UTF-8
(Note that what you've got currently is not really encryption - it's obfuscation at best.)

Your effectively converting arbitrary byte data into a String.
That's not what that constructor is for.
The String constructor that takes a byte[] is meant to convert text in the platform default encoding into a String. Since what you have is not text, the behaviour will be "bad".
If, for example, your platform default encoding is a 8-bit encoding (such as ISO-8859-*), then you'll "only" get random characters.
If your platform default encoding is UTF-8 you'll probably get random characters and some replacement characters for malformed byte sequences.
To summarize: don't do that. I can't tell you what to do instead, since it's not obvious what you're trying to achieve.

Why does the Blowfish output in Java and PHP differ by only 2 chars?

I have a blowfish encryption script in PHP and JAVA vice versa that was working fine until today when I came across a problem.
The same content is encrypted differently in Java vs PHP by only 2 chars, which is really weird.
PHP
wTHzxfxLHdMm/JMFnoh0hciS/JADvFFg
Java
wTHzxfxLHdMm/JMFnoh0hciS/D8DvFFg
-------------------------^^
As you see those two positions do not match. Unfortunately the value is a real email address and I can't share it. Also I was not able to reproduce the problem with other few values I've tested. I've tried changing Base64 encode classes on Java, and that neither helped.
The source code for PHP is here, and for Java is here.
What could I do to resolve this problem?

Let's have a look at your Java code:
String c = new String(Test.encrypt((new String("thevalue")).getBytes(),
(new String("mykey")).getBytes()));
...
System.out.println("Base64 encoded String:" +
new sun.misc.BASE64Encoder().encode(c.getBytes()));
What you are doing here is:
Convert the plaintext string to bytes, using the system's default encoding
convert the key to bytes, using the system's default encoding
encrypt the bytes
convert the encrypted bytes back to a string, using the system's default encoding
convert the encrypted string back to bytes, using the system's default encoding
encode these encrypted bytes using Base64.
The problem is in step 4. It assumes that an arbitrary byte array represents a string in your system's default encoding, and encoding this string back gives the same byte[]. This is valid for some encodings (the ISO-8859 series, for example), but not for others. In Java, when some byte (or byte sequence) is not representable in the given encoding, it will be replaced by some other character, which later for reconverting will be mapped to byte 63 (ASCII ?). Actually, the documentation even says:
The behavior of this constructor when the given bytes are not valid in the default charset is unspecified.
In your case, there is no reason to do this at all - simply use the bytes which your encrypt method outputs directly to convert them to Base64.
byte[] encrypted = Test.encrypt("thevalue".getBytes(),
"mykey".getBytes());
System.out.println("Base64 encoded String:"+ new sun.misc.BASE64Encoder().encode(encrypted));
(Also note that I removed the superfluous new String("...") constructor calls here, though this does not relate to your problem.)
The point to remember: Never ever convert an arbitrary byte[], which did not come from encoding a string, to a string. Output of an encryption algorithm (and most other cryptographic algorithms, except decryption) certainly belongs to the category of data which should not be converted to a string.
And never ever use the System's default encoding, if you want portable programs.

Your code seems right to me.
It looks like you have a trailing white space in the input to one of these programs, and it is only one. I'll tell you why:
Each of these 4-char blocks represent 3 characters in the encrypted string. Th different part (JA and D8 in the 7th block) actually come from a single different character.
wTHz xfxL HdMm /JMF noh0 hciS /JAD vFFg
wTHz xfxL HdMm /JMF noh0 hciS /D8D vFFg
If I have got it right your email address is 19 characters long. The 20th character in one of your input strings is a white space.

Question: Have you tried the associated PHP decryption library to decrypt the PHP generated encrypted text? Have you tried the associated JAVA decryption library to decrypt the JAVA encrypted text?
If both produce differing outputs, then one MUST fail decrypting.
Is that one PHP, or Java?
Whichever one it is -- I would try to duplicate another such failure with a publicly shareable string... give that string as a unit test -- to the developer or developers that created the encrypt/decrypt code in the language that the round-trip encrypt/decrypt fails in.
Then... wait for them to fix it.
Not sure of any faster solutions -- except maybe change encryption/decryption library providers... or roll your own...