I have many hexcodes here and I want to get them into Java without appending 0x to every entity. Like:
0102FFAB and I have to do the following:
byte[] test = {0x01, 0x02, 0xFF, 0xAB};
And I have many hexcodes which are pretty long. Is there any way to make this automatically?
You could try and put the hex codes into a string and then iterate over the string, similar to this:
String input = "0102FFAB";
byte[] bytes = new byte[input.length() / 2];
for( int i = 0; i < input.length(); i+=2)
{
bytes[i/2] = Integer.decode( "0x" + input.substring( i, i + 2 ) ).byteValue();
}
Note that this requires even length strings and it is quite a quick and dirty solution. However, it should still get you started.
You can use BigInteger to load a long hex string.
public static void main(String[] args) {
String hex = "c33b2cfca154c3a3362acfbde34782af31afb606f6806313cc0df40928662edd3ef1d630ab1b75639154d71ed490a36e5f51f6c9d270c4062e8266ad1608bdc496a70f6696fa6e7cd7078c6674188e8a49ecba71fad049a3d483ccac45d27aedfbb31d82adb8135238b858143492b1cbda2e854e735909256365a270095fc";
byte[] bytes2 = hexToBytes(hex);
for(byte b: bytes2)
System.out.printf("%02x", b & 0xFF);
}
public static byte[] hexToBytes(String hex) {
// add a 10 to the start to avoid sign issues, or an odd number of characters.
BigInteger bi2 = new BigInteger("10" +hex, 16);
byte[] bytes2 = bi2.toByteArray();
byte[] bytes = new byte[bytes2.length-1];
System.arraycopy(bytes2, 1, bytes, 0, bytes.length);
return bytes;
}
prints
0c33b2cfca154c3a3362acfbde34782af31afb606f6806313cc0df40928662edd3ef1d630ab1b75639154d71ed490a36e5f51f6c9d270c4062e8266ad1608bdc496a70f6696fa6e7cd7078c6674188e8a49ecba71fad049a3d483ccac45d27aedfbb31d82adb8135238b858143492b1cbda2e854e735909256365a270095fc
note: it handles the possibility that there is one hex value short at the start.
Related
I'm looking for a way, how to reversibly convert a byte[] of an arbitrary length to positive number (String representation in numbers).
BigInteger offers a solution:
byte[] originalBytes = ...
String string = new BigInteger(originalBytes).toString();
...
byte[] decodedBytes = new BigInteger(string).toByteArray();
However, I'm not sure how to get gracefully rid of negative values (or where to store the sign) and keep the process reversible.
Edit: just replace
String string = new BigInteger(originalBytes).toString();
with
String string = new BigInteger(1, originalBytes).toString();
The 1, signals that the passed array represents a positive number (signum = 1)
Original:
You can just prefix the array with a zero byte:
byte[] original = new byte[] { (byte) 255 };
System.out.println(new BigInteger(original).toString()); // prints "-1"
byte[] paddedCopy = new byte[original.length + 1];
for (int i = 0; i < original.length; i++) {
paddedCopy[i + 1] = original[i];
}
System.out.println(new BigInteger(paddedCopy).toString()); // prints "255"
This will essentially nullify the sign bit, making the number unsigned.
I have a problem with CharsetDecoder class.
First example of code (which works):
final CharsetDecoder dec = Charset.forName("UTF-8").newDecoder();
final ByteBuffer b = ByteBuffer.allocate(3);
final byte[] tab = new byte[]{(byte)-30, (byte)-126, (byte)-84}; //char €
for (int i=0; i<tab.length; i++){
b.put(tab, i, 1);
}
try {
b.flip();
System.out.println("a" + dec.decode(b).toString() + "a");
} catch (CharacterCodingException e1) {
e1.printStackTrace();
}
The result is a€a
But when i execute this code:
final CharsetDecoder dec = Charset.forName("UTF-8").newDecoder();
final CharBuffer chars = CharBuffer.allocate(3);
final byte[] tab = new byte[]{(byte)-30, (byte)-126, (byte)-84}; //char €
for (int i=0; i<tab.length; i++){
ByteBuffer buffer = ByteBuffer.wrap(tab, i, 1);
dec.decode(buffer, chars, i == 2);
}
dec.flush(chars);
System.out.println("a" + chars.toString() + "a");
The result is a
Why is not the same result?
How to use the method decode(ByteBuffer, CharBuffer, endOfInput) of class CharsetDecoder in order to retrieve the result a€a ?
-- EDIT --
So with code of Jesper I do that. It's no perfect but works with a step = 1, 2 and 3
final CharsetDecoder dec = Charset.forName("UTF-8").newDecoder();
final CharBuffer chars = CharBuffer.allocate(6);
final byte[] tab = new byte[]{(byte)97, (byte)-30, (byte)-126, (byte)-84, (byte)97, (byte)97}; //char €
final ByteBuffer buffer = ByteBuffer.allocate(10);
final int step = 3;
for (int i = 0; i < tab.length; i++) {
// Add the next byte to the buffer
buffer.put(tab, i, step);
i+=step-1;
// Remember the current position
final int pos = buffer.position();
int l=chars.position();
// Try to decode
buffer.flip();
final CoderResult result = dec.decode(buffer, chars, i >= tab.length -1);
System.out.println(result);
if (result.isUnderflow() && chars.position() == l) {
// Underflow, prepare the buffer for more writing
buffer.position(pos);
}else{
if (buffer.position() == buffer.limit()){
//ByteBuffer decoded
buffer.clear();
buffer.position(0);
}else{
//a part of ByteBuffer is decoded. We keep only bytes which are not decoded
final byte[] b = buffer.array();
final int f = buffer.position();
final int g = buffer.limit() - buffer.position();
buffer.clear();
buffer.position(0);
buffer.put(b, f, g);
}
}
buffer.limit(buffer.capacity());
}
dec.flush(chars);
chars.flip();
System.out.println(chars.toString());
The method decode(ByteBuffer, CharBuffer, boolean) returns a result, but you are ignoring the result. If print the result in your second code fragment:
for (int i = 0; i < tab.length; i++) {
ByteBuffer buffer = ByteBuffer.wrap(tab, i, 1);
System.out.println(dec.decode(buffer, chars, i == 2));
}
you'll see this output:
UNDERFLOW
MALFORMED[1]
MALFORMED[1]
a a
Apparently it does not work correctly if you start decoding in the middle of a character. The decoder expects that the first thing it reads is the start of a valid UTF-8 sequence.
edit - When the decoder reports UNDERFLOW, it expects you to add more data to the input buffer and then try to call decode() again, but you must re-offer it the data from the start of the UTF-8 sequence that you are trying to decode. You can't continue in the middle of an UTF-8 sequence.
Here is a version that works, adding one byte from tab in every iteration of the loop:
final CharsetDecoder dec = Charset.forName("UTF-8").newDecoder();
final CharBuffer chars = CharBuffer.allocate(3);
final byte[] tab = new byte[]{(byte) -30, (byte) -126, (byte) -84}; //char €
final ByteBuffer buffer = ByteBuffer.allocate(10);
for (int i = 0; i < tab.length; i++) {
// Add the next byte to the buffer
buffer.put(tab[i]);
// Remember the current position
final int pos = buffer.position();
// Try to decode
buffer.flip();
final CoderResult result = dec.decode(buffer, chars, i == 2);
System.out.println(result);
if (result.isUnderflow()) {
// Underflow, prepare the buffer for more writing
buffer.limit(buffer.capacity());
buffer.position(pos);
}
}
dec.flush(chars);
chars.flip();
System.out.println("a" + chars.toString() + "a");
The decoder does not internally cache the data from partial characters, but this does not mean that you have to do complicated things to figure out what data to re-feed the decoder. You gave it a clear way to represent what data it actually consumed, i.e. the input ByteBuffer and its position. In the second example, by giving it a new ByteBuffer every time, the OP failed to pass the decoder back what it reported it had not yet consumed.
The standard pattern for using NIO Buffers is input, flip, output, compact, loop. Short of optimization (which may be premature), there is no reason to re-implement compact yourself. You might just get it wrong, like #Jesper and #lecogiteur did (if more than a single character was ever presented). You should NOT be resetting to the position from before the decode call.
The second example should have read something like:
final CharsetDecoder dec = Charset.forName("UTF-8").newDecoder();
final CharBuffer chars = CharBuffer.allocate(3);
final byte[] tab = new byte[]{(byte)-30, (byte)-126, (byte)-84}; //char €
final ByteBuffer buffer = ByteBuffer.wrap(new byte[3]);
for (int i=0; i<tab.length; i++){
b.put(tab, i, 1); // In actual usage some type of IO read/transfer would occur here
b.flip();
dec.decode(buffer, chars, i == 2);
b.compact();
}
dec.flush(chars);
System.out.println("a" + chars.toString() + "a");
NOTE: The above does not check the return value to detect malformed input or other error handling for running safely on arbitrary input/IO conditions.
Code for the same:
public byte[] stringToBytesUTFCustom(String str) {
char[] buffer1 = str.toCharArray();
byte[] b = new byte[buffer1.length << 1];
for(int i = 0; i < buffer1.length; i++) {
int bpos = i << 1;
b[bpos] = (byte) ((buffer1[i]&0xFF00)>>8);
b[bpos + 1] = (byte) (buffer1[i]&0x00FF);
}
return b;
}
public String bytesToStringUTFCustom(byte[] bytes) {
char[] buffer = new char[bytes.length >> 1];
for(int i = 0; i < buffer.length; i++) {
int bpos = i << 1;
char c = (char)(((bytes[bpos]&0x00FF)<<8) + (bytes[bpos+1]&0x00FF));
buffer[i] = c;
}
String txt = String.valueOf(buffer);
//return new String(buffer);
return txt;
}
First, I implement a SMS encryption app (Client to Client) and then want to encode cipher(format "Byte[]") to string, Base64 it's work but can't send because more than 160 character.
I'm want to convert byte array to string ,when use function above it's work for same function, but when I use bytesToStringUTFCustom and then send this text(SMS) can't work.
Receiver cannot read a text to decode from.
Cipher is a result of bytesToStringUTFCustom function, so anyone can help me?
Thanks.
Did you know these:
String.getBytes(Charset encoding)
new String(byte[] byteArray, Charset encoding)
You can use Charset.forName(String) to get the Charset.
Charset UTF8 = Charset.forName("UTF-8");
byte[] bytes = str.getBytes(UTF8);
String reverted = new String(bytes, UTF8);
as the title says, how do I do it? Its easy to convert from string -> byte -> string binary, But how do I convert back? Below is a example.
The output is :
'f' to binary: 01100110
294984
I read somewhere that I could use the Integer.parseInt but clearly that is not the case :( Or am I doing something wrong?
Thanks,
:)
public class main{
public static void main(String[] args) {
String s = "f";
byte[] bytes = s.getBytes();
StringBuilder binary = new StringBuilder();
for (byte b : bytes)
{
int val = b;
for (int i = 0; i < 8; i++)
{
binary.append((val & 128) == 0 ? 0 : 1);
val <<= 1;
}
binary.append(' ');
}
System.out.println("'" + s + "' to binary: " + binary);
System.out.println(Integer.parseInt("01100110", 2));
}
}
You can use Byte.parseByte() with a radix of 2:
byte b = Byte.parseByte(str, 2);
Using your example:
System.out.println(Byte.parseByte("01100110", 2));
102
You can parse it to an integer in base 2, and convert to a byte array.
In your example you've got 16 bits you can also use short.
short a = Short.parseShort(b, 2);
ByteBuffer bytes = ByteBuffer.allocate(2).putShort(a);
byte[] array = bytes.array();
Just in case if you need it for a Very Big String.
String b = "0110100001101001";
byte[] bval = new BigInteger(b, 2).toByteArray();
I made like this, converted a string s -> byte[] and then used Integer.toBinaryString to get binaryStringRep. I converted bianryStringRep by using Byte.parseByte to get the bianryStringRep into byte and the String(newByte[]) to get the byte[] into a String! Hope it helps others then me aswell! ^^
public class main{
public static void main(String[] args) throws UnsupportedEncodingException {
String s = "foo";
byte[] bytes = s.getBytes();
byte[] newBytes = new byte[s.getBytes().length];
for(int i = 0; i < bytes.length; i++){
String binaryStringRep = String.format("%8s", Integer.toBinaryString(bytes[i] & 0xFF)).replace(' ', '0');
byte newByte = Byte.parseByte(binaryStringRep, 2);
newBytes[i] = newByte;
}
String str = new String(newBytes, "UTF-8");
System.out.println(str);
}
}
I would like to know if there is a way to use PBEKeySpec with a byte array argument.
Please find a link to the documentation below:
http://docs.oracle.com/javase/1.7/docs/api/javax/crypto/spec/PBEKeySpec.html)
Here below is my solution: I got it googling around. Please consider I have to internally copy the password and the salt since they have another format when they come from the outside, but the result is the same. It seems it works and solves the problem of having a password as byte[] and not as char[] (it was driving me crazy)
I hope it helps!
Cheers, Soosta
public class Pbkdf2 {
public Pbkdf2() {
}
public void GenerateKey(final byte[] masterPassword, int masterPasswordLen,
final byte[] salt, int saltLen,
int iterationCount, int requestedKeyLen,
byte[] generatedKey) {
byte[] masterPasswordInternal = new byte[masterPasswordLen];
System.arraycopy(masterPassword, 0, masterPasswordInternal, 0, masterPasswordLen);
byte[] saltInternal = new byte[saltLen];
System.arraycopy(salt, 0, saltInternal, 0, saltLen);
SecretKeySpec keyspec = new SecretKeySpec(masterPasswordInternal, "HmacSHA1");
Mac prf = null;
try {
prf = Mac.getInstance("HmacSHA1");
prf.init(keyspec);
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (InvalidKeyException e) {
e.printStackTrace();
}
int hLen = prf.getMacLength(); // 20 for SHA1
int l = Math.max(requestedKeyLen, hLen); // 1 for 128bit (16-byte) keys
int r = requestedKeyLen - (l - 1) * hLen; // 16 for 128bit (16-byte) keys
byte T[] = new byte[l * hLen];
int ti_offset = 0;
for (int i = 1; i <= l; i++) {
F(T, ti_offset, prf, saltInternal, iterationCount, i);
ti_offset += hLen;
}
System.arraycopy(T, 0, generatedKey, 0, requestedKeyLen);
}
private static void F(byte[] dest, int offset, Mac prf, byte[] S, int c, int blockIndex) {
final int hLen = prf.getMacLength();
byte U_r[] = new byte[hLen];
// U0 = S || INT (i);
byte U_i[] = new byte[S.length + 4];
System.arraycopy(S, 0, U_i, 0, S.length);
INT(U_i, S.length, blockIndex);
for (int i = 0; i < c; i++) {
U_i = prf.doFinal(U_i);
xor(U_r, U_i);
}
System.arraycopy(U_r, 0, dest, offset, hLen);
}
private static void xor(byte[] dest, byte[] src) {
for (int i = 0; i < dest.length; i++) {
dest[i] ^= src[i];
}
}
private static void INT(byte[] dest, int offset, int i) {
dest[offset + 0] = (byte) (i / (256 * 256 * 256));
dest[offset + 1] = (byte) (i / (256 * 256));
dest[offset + 2] = (byte) (i / (256));
dest[offset + 3] = (byte) (i);
}
}
I had to implement a two-phase pbkdf2 derivation (so the second pbkdf2 had bytes from the first as input). I ended up using BouncyCastle because I just couldn't get the byte array to char array gymnastics to work. Credit to Pasi from this other question: Reliable implementation of PBKDF2-HMAC-SHA256 for JAVA
import org.bouncycastle.crypto.generators.PKCS5S2ParametersGenerator;
import org.bouncycastle.crypto.digests.SHA256Digest;
import org.bouncycastle.crypto.digests.GeneralDigest;
import org.bouncycastle.crypto.params.KeyParameter;
GeneraDigest algorithm = new SHA256Digest();
PKCS5S2ParametersGenerator gen = new PKCS5S2ParametersGenerator(algorithm);
gen.init(passwordBytes, salt, iterations);
byte[] dk = ((KeyParameter) gen.generateDerivedParameters(256)).getKey();
As the Java PKCS#5 KeyFactory has been specified to only use the lower 8 bits of the characters in the PBEKeySpec, you should be able to convert your byte array into a (16 bit) character array without issue. Just copy the value of each byte into the character array and you should be set.
Just to be sure, I would perform charArray[i] = byteArray[i] & 0xFF as assignment statement, otherwise you would get very high valued characters.
It's an ugly workaround, but I don't see any reason why it should not work.
Note that the above assumes Latin / Windows 1252 compatible encoding for values 0x80 and over. If you allow code points of 0x80 to 0xFF then you cannot use UTF-8 (or UTF-16 of course) as encoding.
I was able to do this using a 3rd party library and extending one of their classes.
Here is the RFC 2898 implementation library that I used:
http://www.rtner.de/software/PBKDF2.html
My code:
import de.rtner.security.auth.spi.PBKDF2Engine;
import de.rtner.security.auth.spi.PBKDF2Parameters;
public class PBKDF2Utils {
private static class PBKDF2EngineWithBinaryPassword extends PBKDF2Engine {
private PBKDF2EngineWithBinaryPassword(PBKDF2Parameters parameters) {
super(parameters);
}
public byte[] deriveKey(byte[] inputPassword, int dkLen) {
this.assertPRF(inputPassword);
return this.PBKDF2(prf, parameters.getSalt(), parameters.getIterationCount(), dkLen);
}
}
public static byte[] deriveKey(
byte[] password,
byte[] salt,
int iterationCount,
int dkLen) {
PBKDF2Parameters parameters = new PBKDF2Parameters("HmacSHA1", null, salt, iterationCount);
PBKDF2EngineWithBinaryPassword engine = new PBKDF2EngineWithBinaryPassword(parameters);
return engine.deriveKey(password, dkLen);
}
}