I need to read a binary file consisting of 4 byte integers (little endian) into a 2D array for my Android application. My current solution is the following:
DataInputStream inp = null;
try {
inp = new DataInputStream(new BufferedInputStream(new FileInputStream(procData), 32768));
}
catch (FileNotFoundException e) {
Log.e(TAG, "File not found");
}
int[][] test_data = new int[SIZE_X][SIZE_Y];
byte[] buffer = new byte[4];
ByteBuffer byteBuffer = ByteBuffer.allocate(4);
for (int i=0; i < SIZE_Y; i++) {
for (int j=0; j < SIZE_X; j++) {
inp.read(buffer);
byteBuffer = ByteBuffer.wrap(buffer);
test_data[j][SIZE_Y - i - 1] = byteBuffer.order(ByteOrder.LITTLE_ENDIAN).getInt();
}
}
This is pretty slow for a 2k*2k array, it takes about 25 seconds. I can see in the DDMS that the garbage collector is working overtime, so that is probably one reason for the slowness.
There has to be a more efficient way of using the ByteBuffer to read that file into the array, but I'm not seeing it at the moment. Any idea on how to speed this up?
Why not read into a 4-byte buffer and then rearrange the bytes manually? It will look like this:
for (int i=0; i < SIZE_Y; i++) {
for (int j=0; j < SIZE_X; j++) {
inp.read(buffer);
int nextInt = (buffer[0] & 0xFF) | (buffer[1] & 0xFF) << 8 | (buffer[2] & 0xFF) << 16 | (buffer[3] & 0xFF) << 24;
test_data[j][SIZE_Y - i - 1] = nextInt;
}
}
Of course, it is assumed that read reads all four bytes, but you should check for the situation when it's not. This way you won't create any objects during reading (so no strain on the garbage collector), you don't call anything, you just use bitwise operations.
If you are on a platform that supports memory-mapped files, consider the MappedByteBuffer and friends from java.nio
FileChannel channel = new RandomAccessFile(procData, "r").getChannel();
MappedByteBuffer map = channel.map(FileChannel.MapMode.READ_ONLY, 0, 4 * SIZE_X * SIZE_Y);
map.order(ByteOrder.LITTLE_ENDIAN);
IntBuffer buffer = map.asIntBuffer();
int[][] test_data = new int[SIZE_X][SIZE_Y];
for (int i=0; i < SIZE_Y; i++) {
for (int j=0; j < SIZE_X; j++) {
test_data[j][SIZE_Y - i - 1] = buffer.get();
}
}
If you need cross-platform support or your platform lacks memory-mapped buffers, you may still want to avoid performing the conversions yourself using an IntBuffer. Consider dropping the BufferedInputStream, allocating a larger ByteBuffer yourself and obtaining a little-endian IntBuffer view on the data. Then in a loop reset the buffer positions to 0, use DataInputStream.readFully to read the large regions at once into the ByteBuffer, and pull int values out of the IntBuffer.
First of all, your 'inp.read(buffer)' is unsafe, as read contract does not guarantee that it will read all 4 bytes.
That aside, for quick transformation use the algorithm from DataInputStream.readInt
I've adapted for you case of byte array of 4 bytes:
int little2big(byte[ ] b) {
return (b[3]&0xff)<<24)+((b[2]&0xff)<<16)+((b[1]&0xff)<<8)+(b[0]&0xff);
}
I don't think it is necessary to reinvent the wheel and perform the byte reordering for endianness again. This is error prone and there is a reason a class like ByteBuffer exists.
Your code can be optimized in the sense that it wastes objects. When a byte[] is wrapped by a ByteBuffer the buffer adds a view, but the original array remains the same. It does not matter wheather the original array is modified/read from directly or the ByteBuffer instance is used.
Therefore, you only need to initialize one instance of ByteBuffer and also have to set the ByteOrder once.
To start again, just use rewind() to set the counter again to the beginning of the buffer.
I have taken your code and modified it as desribed. Be aware that it does not check for errors if there are not enough bytes in the input left. I would suggest to use inp.readFully, as this will throw EOFException if not enough bytes to fill the buffer are found.
int[][] test_data = new int[SIZE_X][SIZE_Y];
ByteBuffer byteBuffer = ByteBuffer.wrap(new byte[4]).order(ByteOrder.LITTLE_ENDIAN);
for (int i=0; i < SIZE_Y; i++) {
for (int j=0; j < SIZE_X; j++) {
inp.read(byteBuffer.array());
byteBuffer.rewind();
test_data[j][SIZE_Y - i - 1] = byteBuffer.getInt();
}
}
Related
I'm trying to implement the tail program and want to print the last n bytes of a file. I've used a RandomAccessFile variable to store the data from the text file. When I try to retrieve the data and print it to the console, I'm getting something like this:
-n1
65109710979710979710810979710810510510979710910659711010510979711410011897114107109797114100119111108102106597114111110
How does on properly retrieve the data from the byte array?
This is my code:
RandomAccessFile raf = new RandomAccessFile(file, "r");
byte[] b = new byte[n];
raf.readFully(b, 0, n);
for (int i = 0; i < n; i++) {
System.out.print(b[i]);
}
You are printing the byte value. To convert e.g. an array of bytes to a String that you can print on System.out.println try the following:
System.out.println(new String(b));
If you wish to convert each byte (as in your loop) to a printable charyou can do the following conversion:
for (int i = 0; i < 10; i++) {
char c = (char) (b[i] & 0xFF);
System.out.print(c);
}
A byte is simply one byte in size (8 bits) whereas a char in Java i 16 bits (2 bytes). Therefore the printed bytes does not make sense, it is not an entire character.
See this link for more info.
I am trying to read and parse a few pieces of information from a file that will be stored in sequential order. ,a char[] of size 8 ,an int ,an int[] of size 8 and finally ,an int.
So, I am reading 56 bytes of information. I am using RandomAccessFile and was wondering if I needed to seek() after preforming each operation of readChar() and readInt() or if I can just call these methods one after the other. I guess this is more of a question about whether the file pointer will reset after each operation completes or if it's safe so assume that the fp will follows it's last location until the file is closed.
Here is what I've written:
int currentOffset = 128;
for(int i = 0; i<16; i++){
//Initialize nodes.
readDisk.seek(currentOffset);
char[] name = new char[8];
for(int j = 0; j<8; j++){
name[i] = readDisk.readChar();
}
int size = readDisk.readInt();
int[] blockPointers = new int[8];
for(int j = 0; j<8; j++){
blockPointers[i] = readDisk.readInt();
}
int used = readDisk.readInt();
Will the fp be at 156 after these operations? Thank you! Sorry if this is a silly question.
Yes, read moves file pointer by number of bytes read, try this
RandomAccessFile f = new RandomAccessFile("1.txt", "r");
f.readChar();
System.out.println(f.getFilePointer());
output
2
How is it possible to store data (bytes) input like this:
input data (bytes) : 01 23 45 and 67 89 10
At the end Arraylist row : [[01,23,45],[67,89,10]]
// declaration
List<List<Byte>> row = new ArrayList<List<Byte>>();
List<Byte> myBytes = new ArrayList<Byte>();
int j=0;
// code before this defines the arraylengt to cut the buffer in pieces of 3 bytes
// and clears the Mybytes arraylist to be able to fill the buffer with new values
if(arrayLength > 0)
{
myBytes.add(value); // fill first arraylist (buffer) with byte value (for example: 01)
if(arrayLength == 1) // when buffer myBytes is full (with values 01 23 45) write value to position j in the new arraylist
{
row.add(j,new ArrayList<Byte>());
row.set(j,myBytes);
j +=j;
}
arrayLength -= 1; // for cutting the buffer in pieces of 3 bytes
}
Thank you for helping me !!
Just use an Arraylist with an Array of Bytes new ArrayList<Byte[]>();. It's a good way to produce a [n][3]-Matrix.
I'd suggest just using a matrix of size [n][3]. You can make sure that it stays as a byte by adding ifs, and must make it an int matrix that way it can have as many rows as you want.
I found the answer, use a arraylist to buffer the values convert to byte array and place them in a other arraylist. Thanks guys !
Note: Make shure u use myBytes.clear();
Here is my code:
if(bool == true)
{
myBytes.add(value); // buffer
if(arrayLength == 1)
{
byte[] data = new byte[myBytes.size()];
for (int i = 0; i < data.length; i++)
{
data[i] = (byte) myBytes.get(i);
}
row.add(data);
}
arrayLength -= 1;
}
I'm trying to store a 32 x 32 Boolean array in a 32 x 32 black and white image (either bitmap or PNG), to then be mapped to a Boolean[32][32] array with black pixels being true and white being false.
This is to store frames of animation to display on a virtual 32 x 32 display. Here's what I have so far below.
Bitmap bmp = BitmapFactory.decodeResource(context.getResources(), R.raw.f1);
bmp.compress(Bitmap.CompressFormat.PNG, 100, o_stream);
byte[] byteArray = o_stream.toByteArray();
What do I do with byteArray to make it a Boolean[32][32] array or am I going about this all wrong in the first place?
While I never did anything with images (so I don't know if this is anything close to what one should do to get the black-and-white of a pic), I suppose you need a rule to decide whether a pixel is closer to black or closer to white. But I'm curious, how can a byte represent a color? Even if it's RGB, you need at least three bytes, don't you?
if(src!= null){
ByteArrayOutputStream os=new ByteArrayOutputStream();
src.compress(android.graphics.Bitmap.CompressFormat.PNG, 100,(OutputStream) os);
byte[] byteArray = os.toByteArray();
//Log.d("byte=",""+byteArray.length);//returns length.
//str = Base64.encodeToString(byteArray,Base64.DEFAULT);//returns string
}
where src is the bitmap....
If you just want to encode an array of Booleans into a bitmap to save storage space, why use an image? That's a lot of extra overhead. Why not just create a bitmap yourself, like this:
Boolean[][] booleanArray = ... // this is your Boolean array
int[] bits = new int[32]; // One int holds 32 bits
for (int i = 0; i < 32; i++) {
for (int j = 0; j < 32; j++) {
if (booleanArray[i][j]) {
// Set bit at the corresponding position in the bits array
bits[i] |= 1 << j;
}
}
}
// Now you have the data in a int array which you can write to a file
// using DataOutputStream. The file would contain 128 bytes.
// To recreate the Boolean[32][32] from the int array, do this:
Boolean[][] booleanArray = new Boolean[32][32];
int[] bits = ... // This is the data you read from the file using DataInputStream
for (int i = 0; i < 32; i++) {
for (int j = 0; j < 32; j++) {
if ((bits[i] & (1 << j)) != 0) {
// Set corresponding Boolean
booleanArray[i][j] = true;
}
}
}
I have two-dimensional array of integers. First index indicates the number of channels. The second one indicates the number of sample in the channel. How can I save this array into the audio file? I know, I have to convert it to byte array, but I have no idea how to do that.
// edit
More info. I already have a class for drawing a waveform. It is here:
http://javafaq.nu/java-example-code-716.html
now I want to cut part of this wave and save it to the new file. So I have to cut part of int[][] samplesContainer, convert it to byte array (I don't know how) and then save it to file with the same format as audioInputStream.
// edit
OK. So the biggest problem is to write inverted function to this one:
protected int[][] getSampleArray(byte[] eightBitByteArray) {
int[][] toReturn = new int[getNumberOfChannels()][eightBitByteArray.length / (2 * getNumberOfChannels())];
int index = 0;
//loop through the byte[]
for (int t = 0; t < eightBitByteArray.length;) {
//for each iteration, loop through the channels
for (int a = 0; a < getNumberOfChannels(); a++) {
//do the byte to sample conversion
//see AmplitudeEditor for more info
int low = (int) eightBitByteArray[t];
t++;
int high = (int) eightBitByteArray[t];
t++;
int sample = (high << 8) + (low & 0x00ff);
if (sample < sampleMin) {
sampleMin = sample;
} else if (sample > sampleMax) {
sampleMax = sample;
}
//set the value.
toReturn[a][index] = sample;
}
index++;
}
return toReturn;
}
I don't understand why there is second incrementation of t, after high. I also have no idea how can i get high and low from sample.
The code you posted reads a sample stream, byte by byte, into the samples array. The code assumes that, in the stream, every two 8-bit bytes form a 16-bit sample, and that there is one sample for each of the NumOfChannels channels.
So, given an array of samples like the one returned by that code,
int[][] samples;
and a byte array for streaming,
byte[] stream;
you might build the converse stream of bytes this way
for (int i=0; i<NumOfSamples; i++) {
for (int j=0; j<NumOfChannels; j++) {
int sample=samples[i][j];
byte low = (byte) (sample & 0xff) ;
byte high = (byte) ((sample & 0xff00 ) >> 8);
stream[((i*NumOfChannels)+j)*2] = low;
stream[(((i*NumOfChannels)+j)*2)+1] = high;
}
}