We Keep Coding

How can I convert a big primitive byte array to an Object Byte array?

I'm having problems downloading files with my company's application, it crashes when I download files over 50mb.
I'm getting the file as an array of native bytes from a WS and then I convert it into an array of Object Byte to serialize it before converting these bytes into the file to download.
This works perfectly with small files but when you are downloading files over 50mb the conversion from bytes[] to Byte[] fails.
This was the code I had:
private RespuestaDescargarDocumento rellenarRespuestaDescarga(byte[] contenido, String nombreDocumento){
Byte[] bytes = ArrayUtils.toObject(contenido);
RespuestaDescargarDocumento respuesta = new RespuestaDescargarDocumento();
respuesta.setContenido(bytes);
respuesta.setNombreDocumento(nombreDocumento);
return respuesta;
}
I take a look what toObject() does and I realised that it is doing a new Byte() every iteration of this loop so if I'm downloading a file of for example 52mb it does 53167398 iterations and for every iteration is creating a new Byte.
public static Byte[] toObject(byte[] array) {
if (array == null) {
return null;
} else if (array.length == 0) {
return EMPTY_BYTE_OBJECT_ARRAY;
}
final Byte[] result = new Byte[array.length];
for (int i = 0; i < array.length; i++) {
result[i] = new Byte(array[i]);
}
return result;
}
I think this is ridiculous so I looked the way of doing this conversion without this method and found this thread:
how to convert byte[] to Byte[], and the other way around?
And I tried the best answer way like this:
private RespuestaDescargarDocumento rellenarRespuestaDescarga(byte[] contenido, String nombreDocumento) {
//Byte[] bytes = ArrayUtils.toObject(contenido);
Byte[] bytes = byteToByte(contenido);
RespuestaDescargarDocumento respuesta = new RespuestaDescargarDocumento();
respuesta.setContenido(bytes);
respuesta.setNombreDocumento(nombreDocumento);
return respuesta;
}
private Byte[] byteToByte(byte[] array) {
if (array == null || array.length == 0) {
return ArrayUtils.EMPTY_BYTE_OBJECT_ARRAY;
}
Byte[] bytes;
bytes = new Byte[array.length];
for (int i = 0; i < array.length; i++) {
bytes[i] = array[i];
}
return bytes;
}
And now I can download perfectly that file of 52mb so that worked, or that was I thought, when I tried to download a bigger file of 150mb it crashed again. This time the loop is doing 159551619 iterations.
I'm getting the following error:
java.lang.OutOfMemoryError: Java heap space
In this line:
bytes = new Byte[array.length];
So it doesn't even get the for loop of my function, is like Java can't create an array of this size (159551619). But I found that the maximum size for an array in Java is 2147483647, so it should still work with this.
Am I doing something wrong?

Save bits in a stream of bytes in Android Java?

I need to save a binary stream, that I will later convert to text. Since binary streams don't exist in Java, I just saved my 'bits' in a stream of 'bytes' just to test my code. Now I have a stream of bytes where 1 bit is encoded on 1 byte.
byte [] stream = new byte [1500];
int str = 0;
byte [] data = new byte [1];
for (int i = 0; i<original.cols(); i++)
{
for (int j= 0; j<original.rows(); j++)
{
original.get(j,i, data);
if ((data[0]==0))
{
stream [str]=0;
str = str+1;
}
else
{
stream [str]=1;
str = str+1;
}
}
}
Can anyone help me to properly save my bits encoded in a stream of bytes, where 1 byte would represent 8 bits ?

A java.util.BitSet contains helper methods for dealing with raw bits, and conversions to and from byte arrays. In the following example, bytes will contain a single byte:
int numberOfBits = 8;
BitSet bitSet = new BitSet(numberOfBits);
bitSet.set(3, true);
bitSet.set(7, true);
byte[] bytes = bitSet.toByteArray();

Java convert Hexadecimal to Base64

I am working on the Matasano CryptoChallenge, and the first one is to create a Hex to Base 64 converter. I honestly don't know how to continue from here. My code:
public class HexToBase64 {
public static void main(String[] args) {
// String hex = "49276d206b696c6c696e6720796f757220627261696e206c696b65206120706f69736f6e6f7573206d757368726f6f6d";
String hex = "DA65A";
convertHexTo64(hex);
}
public static String convertHexTo64(String hex) {
//convert each letter in the hex string to a 4-digit binary string to create a binary representation of the hex string
StringBuilder binary = new StringBuilder();
for (int i = 0; i < hex.length(); i++) {
int dec = Integer.parseInt(hex.charAt(i) + "", 16);
StringBuilder bin = new StringBuilder(Integer.toBinaryString(dec));
while(bin.length() < 4){
bin.insert(0,'0');
}
binary.append(bin);
}
//now take 6 bits at a time and convert to a single b64 digit to create the final b64 representation
StringBuilder b64 = new StringBuilder();
for (int i = 0; i < binary.length(); i++) {
String temp = binary.substring(i, i+5);
int dec = Integer.parseInt(temp, 10);
//convert dec to b64 with the lookup table here then append to b64
}
return b64.toString();
}
}
So after I separate the binary 6 bits at a time and convert to decimal, how do I map the decimal number to the corresponding digit in b64? Would a Hashmap/Hashtable implementation be efficient?
Also, this algorithm displays how I would go about doing the conversion by hand. Is there a better way of doing this? I am looking for a way to convert that will take a reasonable amount of time, so time, and implicitly efficiency, is relevant.
Thank you for your time
EDIT: And the page also mentions that "Always operate on raw bytes, never on encoded strings. Only use hex and base64 for pretty-printing." What does that mean exactly?

Extracted from this Stack Overflow post, which references Apache Commons Codec
byte[] decodedHex = Hex.decodeHex(hex);
byte[] encodedHexB64 = Base64.codeBase64(decodedHex);

String hex = "00bc9d2a05ef06c79a6e972f8a36737e";
byte[] decodedHex = org.apache.commons.codec.binary.Hex.decodeHex(hex.toCharArray());
String result = Base64.encodeBase64String(decodedHex);
System.out.println("==> " + result);

Write input to binaryfile and read it back as int (getting back decimal values)

When writing to a binary file like this:
byte[] content = {1,2,3};
FileOutputStream output = new FileOutputStream("data.bin");
DataOutputStream fileName = new DataOutputStream(output);
fileName.writeInt(content.length);
for (int i = 0; i < content.length; i++)
{
fileName.writeInt(content[i]);
System.out.println(content[i]);
}
fileName.close();
When reading it back using FileInputStream/DataInputStream and .readInt() everything is ok.
(If i use .skip(4); because the first one seems to contain a value wich is the number of digits written)
However, if the byte[] content is replaced with input using scanner.
java.util.Scanner in = new java.util.Scanner(System.in);
String inputAsString = in.nextLine();
byte[] content = inputAsString.getBytes();
I noticed it is written to the binaryfile in decimal. 1 becomes 49, 2 is 50, 3 is 51 ...
My question is how can i read it back to 1, 2, 3 just like the first example with the hardcoded byte array.

When you read in data from input it's a string. So you're getting ascii/utf-x bytes. So the value you're writing is the byte. You want to turn the input you read into an int:
int toWrite = parseInt(inputAsString);
fileName.writeInt(toWrite);
Note that 51 is ascii value assigned to the character '3'
If you write it out as an int you should be able to read it in as such as well.

Use in.nextByte() in a loop to read the the entries as byte.
byte[] bytes = new byte[SIZE];
int indx = 0;
while(in.hasNextByte()){
bytes[indx++] = in.nextByte();
}
EDIT: If you don't know the size then:
List<Byte> byteList = new ArrayList<Byte>();
while(in.hasNextByte()){
byteList.add(in.nextByte());
}
Byte[] bytes = byteList.toArray(new Byte[]{});

How to Convert an int ByteStream to a String?

Hi I am trying to implement the interface SourceStream in my application and overrides the method read(byte[],off,len) and I read the bytes from a server.But I want to convert those byte stream into String for that I used a string object by new String(byte[]) but it asks the initial byte in off and length of the bytes ie len as parameters..Why it is asking like that, as we contain only Strring(bye[])only. can any one help me...Thanks

If you just have a byte[] then you can create a new String via the String(byte[],int,int) constructor provided by the API.
In your case you would do
byte[] myBytes = ("Hello, World!").getBytes();
String myString = new String(myBytes, 0, myBytes.length);
System.out.println(myString);
EDIT:
Try something like this:
int readLength = (len > bufSize ? bufSize : len);
for (int i = 0; i < readLength; i++) {
b[off + i] = buffers[PBuf][PByte];
}
String metaSt = new String(b, 0, readLength);

Just provide 0 as the initial offset and yourArray.Length as the length and you're done. Quite why a method that takes just a byte array isn't provided is anyones guess - probably just to avoid 101 variations of the method.