How store hexadecimal strings in a Byte array? [duplicate] - java

I am looking for a way to convert a long string (from a dump), that represents hex values into a byte array.
I couldn't have phrased it better than the person that posted the same question here.
But to keep it original, I'll phrase it my own way: suppose I have a string "00A0BF" that I would like interpreted as the
byte[] {0x00,0xA0,0xBf}
what should I do?
I am a Java novice and ended up using BigInteger and watching out for leading hex zeros. But I think it is ugly and I am sure I am missing something simple.

Update (2021) - Java 17 now includes java.util.HexFormat (only took 25 years):
HexFormat.of().parseHex(s)
For older versions of Java:
Here's a solution that I think is better than any posted so far:
/* s must be an even-length string. */
public static byte[] hexStringToByteArray(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
Reasons why it is an improvement:
Safe with leading zeros (unlike BigInteger) and with negative byte values (unlike Byte.parseByte)
Doesn't convert the String into a char[], or create StringBuilder and String objects for every single byte.
No library dependencies that may not be available
Feel free to add argument checking via assert or exceptions if the argument is not known to be safe.

One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
Warnings:
in Java 9 Jigsaw this is no longer part of the (default) java.se root
set so it will result in a ClassNotFoundException unless you specify
--add-modules java.se.ee (thanks to #eckes)
Not available on Android (thanks to Fabian for noting that), but you can just take the source code if your system lacks javax.xml for some reason. Thanks to #Bert Regelink for extracting the source.

The Hex class in commons-codec should do that for you.
http://commons.apache.org/codec/
import org.apache.commons.codec.binary.Hex;
...
byte[] decoded = Hex.decodeHex("00A0BF");
// 0x00 0xA0 0xBF

You can now use BaseEncoding in guava to accomplish this.
BaseEncoding.base16().decode(string);
To reverse it use
BaseEncoding.base16().encode(bytes);

Actually, I think the BigInteger is solution is very nice:
new BigInteger("00A0BF", 16).toByteArray();
Edit: Not safe for leading zeros, as noted by the poster.

One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
For those of you interested in the actual code behind the One-liners from FractalizeR (I needed that since javax.xml.bind is not available for Android (by default)), this comes from com.sun.xml.internal.bind.DatatypeConverterImpl.java :
public byte[] parseHexBinary(String s) {
final int len = s.length();
// "111" is not a valid hex encoding.
if( len%2 != 0 )
throw new IllegalArgumentException("hexBinary needs to be even-length: "+s);
byte[] out = new byte[len/2];
for( int i=0; i<len; i+=2 ) {
int h = hexToBin(s.charAt(i ));
int l = hexToBin(s.charAt(i+1));
if( h==-1 || l==-1 )
throw new IllegalArgumentException("contains illegal character for hexBinary: "+s);
out[i/2] = (byte)(h*16+l);
}
return out;
}
private static int hexToBin( char ch ) {
if( '0'<=ch && ch<='9' ) return ch-'0';
if( 'A'<=ch && ch<='F' ) return ch-'A'+10;
if( 'a'<=ch && ch<='f' ) return ch-'a'+10;
return -1;
}
private static final char[] hexCode = "0123456789ABCDEF".toCharArray();
public String printHexBinary(byte[] data) {
StringBuilder r = new StringBuilder(data.length*2);
for ( byte b : data) {
r.append(hexCode[(b >> 4) & 0xF]);
r.append(hexCode[(b & 0xF)]);
}
return r.toString();
}

The HexBinaryAdapter provides the ability to marshal and unmarshal between String and byte[].
import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
public byte[] hexToBytes(String hexString) {
HexBinaryAdapter adapter = new HexBinaryAdapter();
byte[] bytes = adapter.unmarshal(hexString);
return bytes;
}
That's just an example I typed in...I actually just use it as is and don't need to make a separate method for using it.

Here is a method that actually works (based on several previous semi-correct answers):
private static byte[] fromHexString(final String encoded) {
if ((encoded.length() % 2) != 0)
throw new IllegalArgumentException("Input string must contain an even number of characters");
final byte result[] = new byte[encoded.length()/2];
final char enc[] = encoded.toCharArray();
for (int i = 0; i < enc.length; i += 2) {
StringBuilder curr = new StringBuilder(2);
curr.append(enc[i]).append(enc[i + 1]);
result[i/2] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
The only possible issue that I can see is if the input string is extremely long; calling toCharArray() makes a copy of the string's internal array.
EDIT: Oh, and by the way, bytes are signed in Java, so your input string converts to [0, -96, -65] instead of [0, 160, 191]. But you probably knew that already.

In android ,if you are working with hex, you can try okio.
simple usage:
byte[] bytes = ByteString.decodeHex("c000060000").toByteArray();
and result will be
[-64, 0, 6, 0, 0]

The BigInteger() Method from java.math is very Slow and not recommandable.
Integer.parseInt(HEXString, 16)
can cause problems with some characters without
converting to Digit / Integer
a Well Working method:
Integer.decode("0xXX") .byteValue()
Function:
public static byte[] HexStringToByteArray(String s) {
byte data[] = new byte[s.length()/2];
for(int i=0;i < s.length();i+=2) {
data[i/2] = (Integer.decode("0x"+s.charAt(i)+s.charAt(i+1))).byteValue();
}
return data;
}
Have Fun, Good Luck

EDIT: as pointed out by #mmyers, this method doesn't work on input that contains substrings corresponding to bytes with the high bit set ("80" - "FF"). The explanation is at Bug ID: 6259307 Byte.parseByte not working as advertised in the SDK Documentation.
public static final byte[] fromHexString(final String s) {
byte[] arr = new byte[s.length()/2];
for ( int start = 0; start < s.length(); start += 2 )
{
String thisByte = s.substring(start, start+2);
arr[start/2] = Byte.parseByte(thisByte, 16);
}
return arr;
}

For what it's worth, here's another version which supports odd length strings, without resorting to string concatenation.
public static byte[] hexStringToByteArray(String input) {
int len = input.length();
if (len == 0) {
return new byte[] {};
}
byte[] data;
int startIdx;
if (len % 2 != 0) {
data = new byte[(len / 2) + 1];
data[0] = (byte) Character.digit(input.charAt(0), 16);
startIdx = 1;
} else {
data = new byte[len / 2];
startIdx = 0;
}
for (int i = startIdx; i < len; i += 2) {
data[(i + 1) / 2] = (byte) ((Character.digit(input.charAt(i), 16) << 4)
+ Character.digit(input.charAt(i+1), 16));
}
return data;
}

I like the Character.digit solution, but here is how I solved it
public byte[] hex2ByteArray( String hexString ) {
String hexVal = "0123456789ABCDEF";
byte[] out = new byte[hexString.length() / 2];
int n = hexString.length();
for( int i = 0; i < n; i += 2 ) {
//make a bit representation in an int of the hex value
int hn = hexVal.indexOf( hexString.charAt( i ) );
int ln = hexVal.indexOf( hexString.charAt( i + 1 ) );
//now just shift the high order nibble and add them together
out[i/2] = (byte)( ( hn << 4 ) | ln );
}
return out;
}

I've always used a method like
public static final byte[] fromHexString(final String s) {
String[] v = s.split(" ");
byte[] arr = new byte[v.length];
int i = 0;
for(String val: v) {
arr[i++] = Integer.decode("0x" + val).byteValue();
}
return arr;
}
this method splits on space delimited hex values but it wouldn't be hard to make it split the string on any other criteria such as into groupings of two characters.

The Code presented by Bert Regelink simply does not work.
Try the following:
import javax.xml.bind.DatatypeConverter;
import java.io.*;
public class Test
{
#Test
public void testObjectStreams( ) throws IOException, ClassNotFoundException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
String stringTest = "TEST";
oos.writeObject( stringTest );
oos.close();
baos.close();
byte[] bytes = baos.toByteArray();
String hexString = DatatypeConverter.printHexBinary( bytes);
byte[] reconvertedBytes = DatatypeConverter.parseHexBinary(hexString);
assertArrayEquals( bytes, reconvertedBytes );
ByteArrayInputStream bais = new ByteArrayInputStream(reconvertedBytes);
ObjectInputStream ois = new ObjectInputStream(bais);
String readString = (String) ois.readObject();
assertEquals( stringTest, readString);
}
}

I found Kernel Panic to have the solution most useful to me, but ran into problems if the hex string was an odd number. solved it this way:
boolean isOdd(int value)
{
return (value & 0x01) !=0;
}
private int hexToByte(byte[] out, int value)
{
String hexVal = "0123456789ABCDEF";
String hexValL = "0123456789abcdef";
String st = Integer.toHexString(value);
int len = st.length();
if (isOdd(len))
{
len+=1; // need length to be an even number.
st = ("0" + st); // make it an even number of chars
}
out[0]=(byte)(len/2);
for (int i =0;i<len;i+=2)
{
int hh = hexVal.indexOf(st.charAt(i));
if (hh == -1) hh = hexValL.indexOf(st.charAt(i));
int lh = hexVal.indexOf(st.charAt(i+1));
if (lh == -1) lh = hexValL.indexOf(st.charAt(i+1));
out[(i/2)+1] = (byte)((hh << 4)|lh);
}
return (len/2)+1;
}
I am adding a number of hex numbers to an array, so i pass the reference to the array I am using, and the int I need converted and returning the relative position of the next hex number. So the final byte array has [0] number of hex pairs, [1...] hex pairs, then the number of pairs...

Based on the op voted solution, the following should be a bit more efficient:
public static byte [] hexStringToByteArray (final String s) {
if (s == null || (s.length () % 2) == 1)
throw new IllegalArgumentException ();
final char [] chars = s.toCharArray ();
final int len = chars.length;
final byte [] data = new byte [len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit (chars[i], 16) << 4) + Character.digit (chars[i + 1], 16));
}
return data;
}
Because: the initial conversion to a char array spares the length checks in charAt

If you have a preference for Java 8 streams as your coding style then this can be achieved using just JDK primitives.
String hex = "0001027f80fdfeff";
byte[] converted = IntStream.range(0, hex.length() / 2)
.map(i -> Character.digit(hex.charAt(i * 2), 16) << 4 | Character.digit(hex.charAt((i * 2) + 1), 16))
.collect(ByteArrayOutputStream::new,
ByteArrayOutputStream::write,
(s1, s2) -> s1.write(s2.toByteArray(), 0, s2.size()))
.toByteArray();
The , 0, s2.size() parameters in the collector concatenate function can be omitted if you don't mind catching IOException.

If your needs are more than just the occasional conversion then you can use HexUtils.
Example:
byte[] byteArray = Hex.hexStrToBytes("00A0BF");
This is the most simple case. Your input may contain delimiters (think MAC addresses, certificate thumbprints, etc), your input may be streaming, etc. In such cases it gets easier to justify to pull in an external library like HexUtils, however small.
With JDK 17 the HexFormat class will fulfill most needs and the need for something like HexUtils is greatly diminished. However, HexUtils can still be used for things like converting very large amounts to/from hex (streaming) or pretty printing hex (think wire dumps) which the JDK HexFormat class cannot do.
(full disclosure: I'm the author of HexUtils)

public static byte[] hex2ba(String sHex) throws Hex2baException {
if (1==sHex.length()%2) {
throw(new Hex2baException("Hex string need even number of chars"));
}
byte[] ba = new byte[sHex.length()/2];
for (int i=0;i<sHex.length()/2;i++) {
ba[i] = (Integer.decode(
"0x"+sHex.substring(i*2, (i+1)*2))).byteValue();
}
return ba;
}

My formal solution:
/**
* Decodes a hexadecimally encoded binary string.
* <p>
* Note that this function does <em>NOT</em> convert a hexadecimal number to a
* binary number.
*
* #param hex Hexadecimal representation of data.
* #return The byte[] representation of the given data.
* #throws NumberFormatException If the hexadecimal input string is of odd
* length or invalid hexadecimal string.
*/
public static byte[] hex2bin(String hex) throws NumberFormatException {
if (hex.length() % 2 > 0) {
throw new NumberFormatException("Hexadecimal input string must have an even length.");
}
byte[] r = new byte[hex.length() / 2];
for (int i = hex.length(); i > 0;) {
r[i / 2 - 1] = (byte) (digit(hex.charAt(--i)) | (digit(hex.charAt(--i)) << 4));
}
return r;
}
private static int digit(char ch) {
int r = Character.digit(ch, 16);
if (r < 0) {
throw new NumberFormatException("Invalid hexadecimal string: " + ch);
}
return r;
}
Is like the PHP hex2bin() Function but in Java style.
Example:
String data = new String(hex2bin("6578616d706c65206865782064617461"));
// data value: "example hex data"

Late to the party, but I have amalgamated the answer above by DaveL into a class with the reverse action - just in case it helps.
public final class HexString {
private static final char[] digits = "0123456789ABCDEF".toCharArray();
private HexString() {}
public static final String fromBytes(final byte[] bytes) {
final StringBuilder buf = new StringBuilder();
for (int i = 0; i < bytes.length; i++) {
buf.append(HexString.digits[(bytes[i] >> 4) & 0x0f]);
buf.append(HexString.digits[bytes[i] & 0x0f]);
}
return buf.toString();
}
public static final byte[] toByteArray(final String hexString) {
if ((hexString.length() % 2) != 0) {
throw new IllegalArgumentException("Input string must contain an even number of characters");
}
final int len = hexString.length();
final byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4)
+ Character.digit(hexString.charAt(i + 1), 16));
}
return data;
}
}
And JUnit test class:
public class TestHexString {
#Test
public void test() {
String[] tests = {"0FA1056D73", "", "00", "0123456789ABCDEF", "FFFFFFFF"};
for (int i = 0; i < tests.length; i++) {
String in = tests[i];
byte[] bytes = HexString.toByteArray(in);
String out = HexString.fromBytes(bytes);
System.out.println(in); //DEBUG
System.out.println(out); //DEBUG
Assert.assertEquals(in, out);
}
}
}

I know this is a very old thread, but still like to add my penny worth.
If I really need to code up a simple hex string to binary converter, I'd like to do it as follows.
public static byte[] hexToBinary(String s){
/*
* skipped any input validation code
*/
byte[] data = new byte[s.length()/2];
for( int i=0, j=0;
i<s.length() && j<data.length;
i+=2, j++)
{
data[j] = (byte)Integer.parseInt(s.substring(i, i+2), 16);
}
return data;
}

I think will do it for you. I cobbled it together from a similar function that returned the data as a string:
private static byte[] decode(String encoded) {
byte result[] = new byte[encoded/2];
char enc[] = encoded.toUpperCase().toCharArray();
StringBuffer curr;
for (int i = 0; i < enc.length; i += 2) {
curr = new StringBuffer("");
curr.append(String.valueOf(enc[i]));
curr.append(String.valueOf(enc[i + 1]));
result[i] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}

For Me this was the solution, HEX="FF01" then split to FF(255) and 01(01)
private static byte[] BytesEncode(String encoded) {
//System.out.println(encoded.length());
byte result[] = new byte[encoded.length() / 2];
char enc[] = encoded.toUpperCase().toCharArray();
String curr = "";
for (int i = 0; i < encoded.length(); i=i+2) {
curr = encoded.substring(i,i+2);
System.out.println(curr);
if(i==0){
result[i]=((byte) Integer.parseInt(curr, 16));
}else{
result[i/2]=((byte) Integer.parseInt(curr, 16));
}
}
return result;
}

Related

write raw binary to file java

Say I have a binary string ("0 and 1s") and I want to write this string to a binary file, how can this be done in java? I tried converting to ASCII values string, then create a ByteArrayInputStream from that but values over 127 do not display correctly. Can anyone help me with this?
My binaryToAscii method:
public static String BinaryToAscii(String bin){
int num_of_bytes = bin.length()/8;
StringBuilder sb = new StringBuilder();
int index = 0;
String byte_code;
Character char_code;
for (int i =0; i<num_of_bytes;i++){
index = i*8;
byte_code = bin.substring(index,index+8);
int charCode = Integer.parseInt(byte_code, 2);
char_code = new Character((char)charCode);
sb.append(char_code);
}
return sb.toString();
}
Then I convert the returned string to a ByteArrayInputStream using
InputStream is = new ByteArrayInputStream(ascii.toString().getBytes());
you first would transform the 0 / 1 string to a byte[].
then write out using
DataOutputStream.writeByte().
read in with
DataInputStream.readUnsignedByte() // to get 0 - 255
To convert string to binary use this:
You will first need to break the string into its individual letters then run them through this one at a time.
char letter = c;
byte[] bytes = letter.getBytes();
StringBuilder binary = new StringBuilder();
for (byte b : bytes)
{
int val = b;
for (int i = 0; i < 8; i++)
{
binary.append((val & 128) == 0 ? 0 : 1);
val <<= 1;
}
binary.append(' ');
}
System.out.println(binary);

Java How to store/represent a String into long. Then from long to String

How would you store/represent a String into a long? Then jam it into an 8 byte array?
Things I've tried/working with
String eidString = "Awesome!";
ByteBuffer buf = ByteBuffer.allocate(8);
CharBuffer cbuf = buf.asCharBuffer();
cbuf.put(eidString);
byte[] eid = ByteBuffer.allocate(8).putLong(cbuf ??);
Attempt 2
Long l = Long.valueOf("Awesome!");
byte[] eid = ByteBuffer.allocate(8).putLong(l).array();
long p = ByteBuffer.wrap(eid).getLong();
System.out.println(p);
Attemp 3
String input = "hello long world";
byte[] bytes = input.getBytes();
LongBuffer tmpBuf = ByteBuffer.wrap(bytes).asLongBuffer();
long[] lArr = new long[tmpBuf.remaining()];
for (int i = 0; i < lArr.length; i++)
lArr[i] = tmpBuf.get();
System.out.println(input);
System.out.println(Arrays.toString(lArr));
// store longs...
// ...load longs
long[] longs = { 7522537965568945263L, 7955362964116237412L };
byte[] inputBytes = new byte[longs.length * 8];
ByteBuffer bbuf = ByteBuffer.wrap(inputBytes);
for (long l : longs)
bbuf.putLong(l);
System.out.println(new String(inputBytes));
You need to encode your string as a number and reverse it.
you have to determine the number of symbols you will need. e.g. 64 symbols need 6 bits. 32 symbols need 5 bits.
this will determine maximum length of a string. e.g. for 6 bits => 64/6 = 10 symbols, for 8 bits => 64/8 = 8 symbols. e.g. "hello long world" will not fit unless you assume not all a-z is available.
Once you have done this you can encode the symbols in the same way you would parse a 10 or 36 base number. To turn back into a String you can do the reverse (like printing a base 10 or 36 number)
What is the range of possible characters/symbols?
(you need to include a terminating symbol as the Strings can vary in length)
If you accept the limited character set of:
a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,
A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,
0,1,2,3,4,5,6,7,8,9, , <-- space character
then you will have 63 symbols in your reduced alphabet R. Each symbol can be mapped to a 6 bit representation (64 combinations unique combination of bits). There is an implicit 64th symbol which is the empty symbol used to mark the termination of the string, which should be represented by 0x00. This leaves us with an alphabet R that maps to 6 bits as a bijection.
A long has 64 bits of information. Since 64/6 = 10, this means that we can store a string with a length of up to 10 characters from alphabet R in a Java long variable.
It is worth noting that even though R is a reduced alphabet and we have a string length limit of 10, we can still express meaningful English phrases, converted them to a long, and back again!
Java Code (64 bit mapping):
public static long stringToLong(String s) {
if(s.length() >10) { throw new IllegalArgumentException("String is too long: "+s); }
long out = 0L;
for(int i=0; i<s.length(); ++i) {
long m = reducedMapping(s.codePointAt(i));
if (m==-1) { throw new IllegalArgumentException("Unmapped Character in String: "+s); }
m <<= ((9-i)*6)+4;
out |= m;
}
return out;
}
public static String longToString(long l) {
String out = "";
long m = 0xFC00000000000000L;
for(int i=0; i<10; ++i,m>>>=6) {
int x =(int)( (l&m) >>> (((9-i)*6)+4));
if(x==0) { break; }
out += mapping[x];
}
return out;
}
public static long reducedMapping(int x) {
long out=-1;
if(x >= 97 && x <= 122) { out = (long)(x-96); } // 'a' => 1 : 0x01
else if(x >= 65 && x <= 90) { out = (long)(x-37); } // 'A' => 27 : 0x1B
else if(x >= 48 && x <= 57) { out = (long)(x-+5); } // '0' => 53 : 0x35
else if(x == 32 ) { out = 63L; } // ' ' => 63 : 0x3F
return out;
}
public static char[] mapping = {
'\n', //<-- unused/empty character
'a','b','c','d','e','f','g','h','i','j','k','l','m',
'n','o','p','q','r','s','t','u','v','w','x','y','z',
'A','B','C','D','E','F','G','H','I','J','K','L','M',
'N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
'0','1','2','3','4','5','6','7','8','9',' '
};
Java Code (32 bit mapping):
public static long stringToLong32(String s) {
if(s.length() >12) { throw new IllegalArgumentException("String is too long: "+s); }
long out = 0L;
for(int i=0; i<s.length(); ++i) {
long m = reducedMapping32(s.codePointAt(i));
if (m==-1) { throw new IllegalArgumentException("Unmapped Character in String: "+s); }
m <<= ((11-i)*5)+4;
out |= m;
}
return out;
}
public static String longToString32(long l) {
String out = "";
long m = 0xF800000000000000L;
for(int i=0; i<12; ++i,m>>>=5) {
int x =(int)( (l&m) >>> (((11-i)*5)+4));
if(x==0) { break; }
out += mapping32[x];
}
return out;
}
public static long reducedMapping32(int x) {
long out=-1;
if(x >= 97 && x <= 122) { out = (long)(x-96); } // 'a' => 1 : 0x01
else if(x >= 65 && x <= 90) { out = (long)(x-64); } // 'A' => 1 : 0x01
else if(x >= 32 && x <= 34) { out = (long)(x-5); } // ' ','!','"' => 27,28,29
else if(x == 44 ) { out = 30L; } // ',' => 30 : 0x1E
else if(x == 46 ) { out = 31L; } // '.' => 31 : 0x1F
return out;
}
public static char[] mapping32 = {
'\n', //<-- unused/empty character
'a','b','c','d','e','f','g','h','i','j','k','l','m',
'n','o','p','q','r','s','t','u','v','w','x','y','z',
' ','!','"',',','.'
};
}
EDIT:
Use this class for a more generalized conversions. It allows Strings of any non empty set of characters to be uniquely mapped uniquely to a long and converted back to the original String again. Simply define any character set (char[]) through the constructor and use the resulting object to convert back and forth using str2long & long2str.
Java StringAndLongConverter Class:
import java.util.Arrays;
import java.util.regex.Pattern;
public class StringAndLongConveter {
/* .,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,., */
/* String <--> long ID Conversion */
/* `'`'`'`'`'`'`'`'`'`'`'`'`'`'`'`'`' */
/* --<[ Class Members ]>-- */
// Don't re-arrange, order-dependent initializations
private final char[] CHAR_MAP;
public final int NUM_ACTUAL_CHARS;
public final int NUM_MAPPED_CHARS;
public final int BIT_COUNT;
public final int MAX_NUM_CHARS;
public final long ROLLING_MASK;
public final long FORMAT_MASK;
public final long MIN_VALUE;
public final long MAX_VALUE;
public final Pattern REGEX_CHAR_VALIDATOR;
public StringAndLongConveter(char[] chars) {
if(chars == null ) { throw new IllegalArgumentException("Cannot Pass in null reference"); }
if(chars.length==0) { throw new IllegalArgumentException("Cannot Pass in empty set" ); }
CHAR_MAP = setCharMap(chars);
NUM_ACTUAL_CHARS = CHAR_MAP.length;
NUM_MAPPED_CHARS = NUM_ACTUAL_CHARS+1;
BIT_COUNT = calcMinBitsNeeded();
MAX_NUM_CHARS = calcMaxPossibleChars();
ROLLING_MASK = calcRollingMask();
FORMAT_MASK = calcFormatMask();
MIN_VALUE = calcIDMinVal();
MAX_VALUE = calcIDMaxVal();
REGEX_CHAR_VALIDATOR = createRegExValidator();
}
/* --<[ Dynamic Initialization Calculation Helper Methods ]>-- */
//Remove duplicates
private final char[] setCharMap(final char[] chars) {
char[] tmp = new char[chars.length];
int dupes = 0;
for(int i=0; i<chars.length; ++i) {
boolean dupeFound = false;
for(int j=0; !dupeFound && j<i; ++j) {
if(chars[i]==chars[j]) {
++dupes;
dupeFound = true;
}
}
if(!dupeFound) { tmp[i-dupes] = chars[i]; }
}
char[] out = new char[chars.length-dupes];
if(dupes==0) { out = chars; }
else {
for(int i=0; i<out.length; ++i) out[i] = tmp[i];
}
return out;
}
// calculate minimum bits necessary to encode characters uniquely
private final int calcMinBitsNeeded() {
if(NUM_MAPPED_CHARS==0) { return 0; }
int val,tmp,log;
val = NUM_MAPPED_CHARS;
tmp = Integer.highestOneBit(val); // returns only the highest set bit
tmp = tmp | (tmp-1); // propagate left bits
log = Integer.bitCount(tmp); // count bits (logarithm base 2)
return ((val&(val-1))==0) ? log-1 : log;
//return one less then bit count if even power of two
}
//Calculate maximum number of characters that can be encoded in long
private final int calcMaxPossibleChars() {
return Long.SIZE/BIT_COUNT;
}
//Calculate rolling mask for str <--> long conversion loops
private final long calcRollingMask() {
long mask = 0x0000000000000001L;
for(int i=1; i<BIT_COUNT; ++i) { mask |= mask << 1; }
for(int i=1; i<MAX_NUM_CHARS; ++i) { mask <<= BIT_COUNT; }
return mask;
}
//Calculate format mask for long input format validation
private final long calcFormatMask() {
//propagate lest significant set bit in rolling mask & negate resulting value
return ~(ROLLING_MASK | (ROLLING_MASK-1));
}
//Calculate min value of long encoding
//doubles as format specification for unused bits
private final long calcIDMinVal() {
return 0xAAAAAAAAAAAAAAAAL & FORMAT_MASK;
}
//Calculate max value of long encoding
private final long calcIDMaxVal(){
char maxChar = CHAR_MAP[CHAR_MAP.length-1];
char[] maxCharArr = new char[MAX_NUM_CHARS];
Arrays.fill(maxCharArr, maxChar);
return str2long(new String(maxCharArr));
}
//Dynamically create RegEx validation string for invalid characters
private final Pattern createRegExValidator() {
return Pattern.compile("^["+Pattern.quote(new String(CHAR_MAP))+"]+?$");
}
/* --<[ Internal Helper Methods ]>-- */
private static boolean ulongLessThen(long lh, long rh) {
return (((lh ^ rh) >> 63) == 0) ? lh < rh : (0x8000000000000000L & lh)==0;
}
private long charMapping(final char c) {
for(int i=0; i<CHAR_MAP.length; ++i)
if(CHAR_MAP[i]==c)
return i+1;
return -1;
}
/* --<[ String <--> long Conversion Methods ]>-- */
public final String long2str(final long n) {
String out = "";
if (ulongLessThen(n,MIN_VALUE) || ulongLessThen(MAX_VALUE,n)) { throw new IllegalArgumentException("Long Outside of Formatted Range: "+Long.toHexString(n)); }
if ((FORMAT_MASK & n) != MIN_VALUE) { throw new IllegalArgumentException("Improperly Formatted long"); }
long m = ROLLING_MASK;
for(int i=0; i<MAX_NUM_CHARS; ++i,m>>>=BIT_COUNT) {
int x =(int)( (n&m) >>> ((MAX_NUM_CHARS-i-1)*BIT_COUNT));//10|10 0111
if(x >= NUM_MAPPED_CHARS) { throw new IllegalArgumentException("Invalid Formatted bit mapping: \nlong="+Long.toHexString(n)+"\n masked="+Long.toHexString(n&m)+"\n i="+i+" x="+x); }
if(x==0) { break; }
out += CHAR_MAP[x-1];
}
return out;
}
public final long str2long(String str) {
if(str.length() > MAX_NUM_CHARS) { throw new IllegalArgumentException("String is too long: "+str); }
long out = MIN_VALUE;
for(int i=0; i<str.length(); ++i) {
long m = charMapping(str.charAt(i));
if (m==-1) { throw new IllegalArgumentException("Unmapped Character in String: "+str); }
m <<= ((MAX_NUM_CHARS-i-1)*BIT_COUNT);
out += m; // += is more destructive then |= allowing errors to be more readily detected
}
return out;
}
public final boolean isValidString(String str) {
return str != null && !str.equals("") //null or empty String
&& str.length() <= MAX_NUM_CHARS //too long
&& REGEX_CHAR_VALIDATOR.matcher(str).matches(); //only valid chars in string
}
public final char[] getMappedChars() { return Arrays.copyOf(CHAR_MAP,CHAR_MAP.length); }
}
Have fun encoding & decoding Strings & longs
To parse a String into a Long, can use the Long wrapper class.
String myString = "1500000";
Long myLong = Long.parseLong(myString);
To stuff it into an 8-byte array...
long value = myLong.longValue();
byte[] bytes = new byte[8];
for (int i = 0; i < bytes.length; i++) {
long mask = 0xFF00000000000000 >> (i * 8);
bytes[i] = (byte) (value & mask);
}
This example is big endian.
If you're encoding a String into a long, then you can do something like:
String myString = "HELLO";
long toLong = 0;
for (int i = 0; i < myString.length(); i++) {
long c = (long) myString.charAt(i);
int shift = (myString.length() - 1 - i) * 8;
toLong += c << shift;
}
This hasn't been tested. There might be a few things wrong with it.
You don't need to do that. String has a method called getBytes(). It do the convert for you directly. Call the following method with parameter "Hallelujah"
public static void strToLong(String s) throws IOException
{
byte[] bArr = s.getBytes();
for( byte b : bArr)
{
System.out.print(" " + b);
}
System.out.println();
System.out.write(bArr);
}
The result is
72 97 108 108 101 108 117 106 97 104
Hallelujah

Convert hexadecimal string (hex) to a binary string

I found the following way hex to binary conversion:
String binAddr = Integer.toBinaryString(Integer.parseInt(hexAddr, 16));
While this approach works for small hex numbers, a hex number such as the following
A14AA1DBDB818F9759
Throws a NumberFormatException.
I therefore wrote the following method that seems to work:
private String hexToBin(String hex){
String bin = "";
String binFragment = "";
int iHex;
hex = hex.trim();
hex = hex.replaceFirst("0x", "");
for(int i = 0; i < hex.length(); i++){
iHex = Integer.parseInt(""+hex.charAt(i),16);
binFragment = Integer.toBinaryString(iHex);
while(binFragment.length() < 4){
binFragment = "0" + binFragment;
}
bin += binFragment;
}
return bin;
}
The above method basically takes each character in the Hex string and converts it to its binary equivalent pads it with zeros if necessary then joins it to the return value.
Is this a proper way of performing a conversion? Or am I overlooking something that may cause my approach to fail?
Thanks in advance for any assistance.
BigInteger.toString(radix) will do what you want. Just pass in a radix of 2.
static String hexToBin(String s) {
return new BigInteger(s, 16).toString(2);
}
Fast, and works for large strings:
private String hexToBin(String hex){
hex = hex.replaceAll("0", "0000");
hex = hex.replaceAll("1", "0001");
hex = hex.replaceAll("2", "0010");
hex = hex.replaceAll("3", "0011");
hex = hex.replaceAll("4", "0100");
hex = hex.replaceAll("5", "0101");
hex = hex.replaceAll("6", "0110");
hex = hex.replaceAll("7", "0111");
hex = hex.replaceAll("8", "1000");
hex = hex.replaceAll("9", "1001");
hex = hex.replaceAll("A", "1010");
hex = hex.replaceAll("B", "1011");
hex = hex.replaceAll("C", "1100");
hex = hex.replaceAll("D", "1101");
hex = hex.replaceAll("E", "1110");
hex = hex.replaceAll("F", "1111");
return hex;
}
Integer.parseInt(hex,16);
System.out.print(Integer.toBinaryString(hex));
Parse hex(String) to integer with base 16 then convert it to Binary String using toBinaryString(int) method
example
int num = (Integer.parseInt("A2B", 16));
System.out.print(Integer.toBinaryString(num));
Will Print
101000101011
Max Hex vakue Handled by int is FFFFFFF
i.e. if FFFFFFF0 is passed ti will give error
With all zeroes:
static String hexToBin(String s) {
String preBin = new BigInteger(s, 16).toString(2);
Integer length = preBin.length();
if (length < 8) {
for (int i = 0; i < 8 - length; i++) {
preBin = "0" + preBin;
}
}
return preBin;
}
public static byte[] hexToBin(String str)
{
int len = str.length();
byte[] out = new byte[len / 2];
int endIndx;
for (int i = 0; i < len; i = i + 2)
{
endIndx = i + 2;
if (endIndx > len)
endIndx = len - 1;
out[i / 2] = (byte) Integer.parseInt(str.substring(i, endIndx), 16);
}
return out;
}
import java.util.*;
public class HexadeciamlToBinary
{
public static void main()
{
Scanner sc=new Scanner(System.in);
System.out.println("enter the hexadecimal number");
String s=sc.nextLine();
String p="";
long n=0;
int c=0;
for(int i=s.length()-1;i>=0;i--)
{
if(s.charAt(i)=='A')
{
n=n+(long)(Math.pow(16,c)*10);
c++;
}
else if(s.charAt(i)=='B')
{
n=n+(long)(Math.pow(16,c)*11);
c++;
}
else if(s.charAt(i)=='C')
{
n=n+(long)(Math.pow(16,c)*12);
c++;
}
else if(s.charAt(i)=='D')
{
n=n+(long)(Math.pow(16,c)*13);
c++;
}
else if(s.charAt(i)=='E')
{
n=n+(long)(Math.pow(16,c)*14);
c++;
}
else if(s.charAt(i)=='F')
{
n=n+(long)(Math.pow(16,c)*15);
c++;
}
else
{
n=n+(long)Math.pow(16,c)*(long)s.charAt(i);
c++;
}
}
String s1="",k="";
if(n>1)
{
while(n>0)
{
if(n%2==0)
{
k=k+"0";
n=n/2;
}
else
{
k=k+"1";
n=n/2;
}
}
for(int i=0;i<k.length();i++)
{
s1=k.charAt(i)+s1;
}
System.out.println("The respective binary number is : "+s1);
}
else
{
System.out.println("The respective binary number is : "+n);
}
}
}
public static byte[] hexToBytes(String string) {
int length = string.length();
byte[] data = new byte[length / 2];
for (int i = 0; i < length; i += 2) {
data[i / 2] = (byte)((Character.digit(string.charAt(i), 16) << 4) + Character.digit(string.charAt(i + 1), 16));
}
return data;
}

Simple caesar cipher in java

Hey I'm making a simple caesar cipher in Java using the formula [x-> (x+shift-1) mod 127 + 1] I want to have my encrypted text to have the ASCII characters except the control characters(i.e from 32-127). How can I avoid the control characters from 0-31 applying in the encrypted text. Thank you.
How about something like this:
public String applyCaesar(String text, int shift)
{
char[] chars = text.toCharArray();
for (int i=0; i < text.length(); i++)
{
char c = chars[i];
if (c >= 32 && c <= 127)
{
// Change base to make life easier, and use an
// int explicitly to avoid worrying... cast later
int x = c - 32;
x = (x + shift) % 96;
if (x < 0)
x += 96; //java modulo can lead to negative values!
chars[i] = (char) (x + 32);
}
}
return new String(chars);
}
Admittedly this treats 127 as a non-control character, which it isn't... you may wish to tweak it to keep the range as [32, 126].
Map your characters from [32..127] to [0..95], do a mod 95+1 and map the result back to [32..127].
Usually cipher text is base64 encoded, base16 (hex) also works well. Base64 is used most often for cipher text because it takes up less space than hex, hex is most commonly used for message digests. In the java.util.prefs.Base64 library you will find byteArrayToBase64() and base64ToByteArray().
On a side note you should NEVER write your own encryption algorithm for security reasons, you should be using a block cipher or stream cipher. I hope this is for fun!
there! Is there any way to consider the whole range of characters? For example, "á", "é", "ö", "ñ", and not consider " " (the [Space])? (For example, my String is "Hello World", and the standard result is "Khoor#Zruog"; I want to erase that "#", so the result would be "KhoorZruog")
I'm sure my answer is in this piece of code:
if (c >= 32 && c <= 127)
{
// Change base to make life easier, and use an
// int explicitly to avoid worrying... cast later
int x = c - 32;
x = (x + shift) % 96;
chars[i] = (char) (x + 32);
}
... But I've tried some things, and the didn't work :S So, I'll wait for your answers :D See you!
Why not try
for(int i = 0; i < length; i++)
{
char c = chars[i]
if(Character.isLetter(c))
{
int x = c - 32;
x = (x + shift) % 96;
chars[i] = (char) (x+32);
}
}
Copy paste this in NetBeans with name "caesar":
//package caesar;
import java.io.*;
public class caesar {
int offset=3;
public String encrypt(String s) throws IOException
{
StringBuilder sb=new StringBuilder();
for(int i=0;i<s.length();i++)
{
char t=s.charAt(i);
if(t>='A' && t<='Z')
{
int t1=t-'A'+offset;
t1=t1%26;
sb.append((char)(t1+'A'));
}
else if(t>='a' && t<='z')
{
int t1=t-'a'+offset;
t1=t1%26;
sb.append((char)(t1+'a'));
}
}
return sb.toString();
}
public String decrypt(String s) throws IOException
{
StringBuilder sb=new StringBuilder();
for(int i=0;i<s.length();i++)
{
char t=s.charAt(i);
if(t>='A' && t<='Z')
{
int t1=t-'A'-offset;
if(t1<0)t1=26+t1;
sb.append((char)(t1+'A'));
}
else if(t>='a' && t<='z')
{
int t1=t-'a'-offset;
if(t1<0)t1=26+t1;
sb.append((char)(t1+'a'));
}
}
return sb.toString();
}
public static void main(String[] args) {
try
{
System.out.println("Caesar encrypion technique");
BufferedReader b;
String oriTxt,encTxt,decTxt;
System.out.println("Enter string to encrypt:");
b=new BufferedReader(new InputStreamReader(System.in));
oriTxt=b.readLine();
caesar c=new caesar();
encTxt=c.encrypt(oriTxt);
System.out.println("Encrypted text :"+encTxt);
decTxt=c.decrypt(encTxt);
System.out.println("Derypted text :"+decTxt);
}
catch(Exception e)
{
System.out.println(e.toString());
}
}
}
import java.util.Scanner;
//caeser
public class Major_Assingment {
public static final String ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZhh";
public static String encrypt(String plainText,int shiftKey)
{
plainText = plainText.toUpperCase();
String cipherText= " ";
for(int i=0; i<plainText.length(); i++)
{
int charPosition = ALPHABET.indexOf(plainText.charAt(i));
int keyVal = (shiftKey + charPosition)% 26 ;
char replaceVal = ALPHABET.charAt(keyVal);
cipherText += replaceVal;
}
return cipherText;
}
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
System.out.println("Enter the string for Encryption:");
String message = new String();
message = sc.next();
System.out.println(encrypt(message,3));
sc.close();
}
}

How to create a string representing a Java long as though it were unsigned 64-bit value

My component is handed a long value that I later use as a key into a cache. The key itself is a string representation of the long value as if it were unsigned 64-bit value. That is, when my component is handed -2944827264075010823L, I need to convert that into the string key "15501916809634540793".
I have a solution, but it seems brute force and it makes me a bit queasy. Essentially, I convert the long into a hexadecimal string representation (so -2944827264075010823L becomes "d721df34a7ec6cf9") and convert the hexadecimal string into a BigInteger:
String longValueAsHexString = convertLongToHexString(longValue);
BigInteger bi = new BigInteger(longValueAsHexString, 16);
String longValueString = bi.toString();
I then use longValueString as the key into the cache.
I cannot use Long.toString(longValue,16), because it returns the hex string for the absolute value, prefixed by a "-".
So my convertLongToHexString looks like this:
long mask = 0x00000000ffffffffL;
long bottomHalf = number & mask;
long upperHalf = (number >> 32) & mask;
String bottomHalfString = Long.toString(bottomHalf, 16);
if (bottomHalfString.length() != 8) {
String zeroes = "0000000000000000";
bottomHalfString = zeroes.substring(16-bottomHalfString.length()) + bottomHalfString;
}
return Long.toString(upperHalf,16)+bottomHalfString;
There must be a more elegant way of doing this. Any suggestions?
Here's my implementation. I've refactored it to have a function taking a long and returning a string. :-)
import java.math.BigInteger;
class UInt64Test {
public static void main(String[] args) {
for (String arg : args)
System.out.println(toUnsignedString(Long.parseLong(arg)));
}
private static final BigInteger B64 = BigInteger.ZERO.setBit(64);
public static String toUnsignedString(long num) {
if (num >= 0)
return String.valueOf(num);
return BigInteger.valueOf(num).add(B64).toString();
}
}
I think Chris's answer is better, but here's another just for fun.
public static String longUnsignedString(long l) {
byte[] bytes = new byte[9];
for (int i = 1; i < 9; i++) {
bytes[i] = (byte) ((l >> ((8 - i) * 8)) & 255);
}
return (new BigInteger(bytes)).toString();
}
Five years late, but here is an implementation that doesn't use BigInteger or byte arrays.
Instead, it emulates unsigned division for one step and offloads the rest to the standard library function:
public static String unsignedToString(long n) {
long temp = (n >>> 1) / 5; // Unsigned divide by 10 and floor
if (temp == 0)
return Integer.toString((int)n); // Single digit
else
return Long.toString(temp) + (n - temp * 10); // Multiple digits
}
Bitless implementations:
byte[] bytes = ByteBuffer.allocate(8).putLong(1023L).array();
System.out.println(new BigInteger(bytes).toString(2));
regards, Alex

Categories