We Keep Coding

ByteBuffer allocate direct example

I am writing a Java program for fun which stores sensitive information from users.
For this reason I want to ensure that the garbage collection does not touch it, so that in the future when I am finished I can wipe it from memory.
So far I have this line of code creating 2048 bytes which is more than enough to store any user's passwords.
My question is how do I store a String such as "secret123", and after delete it? This is a very basic question I know but I could not see it in the documentation. I am probably making this more difficult than it is in my head, but better safe than sorry.
ByteBuffer pass = ByteBuffer.allocateDirect(2048);
I am aware of other risks such as swap page files, the computer being coldboot attacked etc...
Thanks!
EDIT:
In response to first answer - I mean to fill memory with '0' characters afterwards, not to free it.

You can't explicitly free the allocated memory, but you can clear the buffer and then write zeros (or random bytes) to the buffer when you are done. This will destroy any data that was previously stored in the buffer, reducing the window of attack.
pass.clear();
while (pass.hasRemaining())
pass.put((byte) 0);

As an alternative to #erickson's approach, if you allocate the byte array yourself and create the ByteBuffer by wrapping, then you can clear the array with a call to Arrays.fill().
byte[] byteArray = new byte[2048];
ByteBuffer bb = ByteBuffer.wrap(byteArray);
//... do your thing here
Arrays.fill(byteArray, (byte)0);
As long as you maintain a reference to either the byteArray or the ByteBuffer, garbage collection won't touch the byte array. You can also get the array back later by calling ByteBuffer.array() and then zeroing it out. (NB: You are not guaranteed an actual array if you try this with a ByteBuffer created by allocateDirect().)

Cannot read in Java data serialized in C++

I have a Java client connected via socket to a C++ server.
The C++ server sends back to the client serialized objects.
However serialization works differently for Java and C++, so I cannot read the objects in that way:
objectInputStream.readObject();
This forces me to read each single value of the object manually:
byte[] buffer = read(FOUR_BYTES);
int flag = convertBufferToInt(buffer);
byte[] buffer = read(FOUR_BYTES);
float price = convertBufferToFloat(buffer);
// More stuff
myObject.setFlag(flag);
myObject.setPrice(price);
// More stuff
That's very hard to maintain. Isn't there an easier way to fill in my object with data?

To solve this in general you would need to write a C++ parser for objects serialized in Java. This is no small task.
Rather, I would recommend that you find some serialization format that is easy to parse and share between your Java and C++ programs. Preferably a format where there exists Java as well as C++ libraries for the serialization/deserialization. JSON or Google Protocol Buffers are obvious candidates.

Yes there is (are). You have 2 options using only the standard library:
Using the DataInputStream class
Check out the DataInputStream class. It has methods to read values of primitive types like readByte(), readInt(), readLong(), readFloat(), readChar(), readUTF() (for reading UTF-8 encoded String) etc.
So your code becomes as simple as:
// Obtain InputStream from Socket:
InputStream is = ...;
// Create DataInputStream:
DataInputStream dis = new DataInputStream(is);
myObject.setFlag(dis.readInt());
myObject.setPrice(dis.readFloat());
Using the ByteBuffer class
For this you have to read first the whole data into a byte array. Once you've done that, you can create a ByteBuffer using the ByteBuffer.wrap(byte[] array) method. The ByteBuffer class also supports reading primitive types just like the DataInputStream class.
The good thing about ByteBuffer that it supports changing the byte order (the order how the low and high bytes of a multi-byte value like int are read/written): ByteBuffer.order(ByteOrder bo). This is very useful if you're communicating with systems which use a differnet byte order (which might apply in your case).
Example using ByteBuffer:
// Read all your input data:
byte[] data = ...;
// Create ByteBuffer:
ByteBuffer bb = ByteBuffer.wrap(data);
myObject.setFlag(bb.getInt());
myObject.setPrice(bb.getFloat());

What is the best way to write an int array (image data) to file

I have a large int array containing image data in the format ARGB (alpha, R, G and B channels, 1 byte each). I want to save it to file in onPause() to be able to reload it when the app is restarted. What do you think is the best way to do that?
I found the following methods:
Convert the int array to a byte array manually (see here) and then use a FileOutputStream to output the byte array.
Wrap the array into a java.nio.IntBuffer and then write the object to file using java.io.ObjectOutputStream.writeObject().
Write each element one at a time using java.io.ObjectOutputStream.writeInt().
All these methods seem quite wasteful so there is probably another, better way. Possibly even a way to use image compression to reduce the size of the file?

From my point of view you can also use android specific storages
Use database/content provider for storing image data
Use out Bundle in onSaveInstance method
If your still want to write to a file you can use following NIO based code:
static void writeIntArray(int[] array) throws IOException {
FileOutputStream fos = new FileOutputStream("out.file");
try {
ByteBuffer byteBuff = ByteBuffer.allocate((Integer.SIZE / Byte.SIZE) * array.length);
IntBuffer intBuff = byteBuff.asIntBuffer();
intBuff.put(array);
intBuff.flip();
FileChannel fc = fos.getChannel();
fc.write(byteBuff);
} finally {
fos.close();
}
}

None of those. Some of them don't even make sense.
DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(file)));
then call dos.writeInt() as many times as necessary, then close dos. The buffer will take away most of the pain.
Or else create an IntBuffer and use FileChannel.write(), but I've never been able to figure out how that works in the absence of an IntBuffer.asByteBuffer() method. Or else create a ByteBuffer, take it as an IntBuffer via asIntBuffer(), put the data in, then adjust the ByteBuffer's length, which is another missing piece of the API, and again use FileChannel.write(ByteBuffer).

Growing ByteBuffer

Has anyone has ever seen an implementation of java.nio.ByteBuffer that will grow dynamically if a putX() call overruns the capacity?
The reason I want to do it this way is twofold:
I don't know how much space I need ahead of time.
I'd rather not do a new ByteBuffer.allocate() then a bulk put() every time I run out of space.

In order for asynchronous I/O to work, you must have continuous memory. In C you can attempt to re-alloc an array, but in Java you must allocate new memory. You could write to a ByteArrayOutputStream, and then convert it to a ByteBuffer at the time you are ready to send it. The downside is you are copying memory, and one of the keys to efficient IO is reducing the number of times memory is copied.

A ByteBuffer cannot really work this way, as its design concept is to be just a view of a specific array, which you may also have a direct reference to. It could not try to swap that array for a larger array without weirdness happening.
What you want to use is a DataOutput. The most convenient way is to use the (pre-release) Guava library:
ByteArrayDataOutput out = ByteStreams.newDataOutput();
out.write(someBytes);
out.writeInt(someInt);
// ...
return out.toByteArray();
But you could also create a DataOutputStream from a ByteArrayOutputStream manually, and just deal with the spurious IOExceptions by chaining them into AssertionErrors.

Another option is to use direct memory with a large buffer. This consumes virtual memory but only uses as much physical memory as you use (by page which is typically 4K)
So if you allocate a buffer of 1 MB, it comsumes 1 MB of virtual memory, but the only OS gives physical pages to the application which is actually uses.
The effect is you see your application using alot of virtual memory but a relatively small amount of resident memory.

Have a look at Mina IOBuffer https://mina.apache.org/mina-project/userguide/ch8-iobuffer/ch8-iobuffer.html which is a drop in replacement (it wraps the ByteBuffer)
However , I suggest you allocate more than you need and don't worry about it too much. If you allocate a buffer (esp a direct buffer) the OS gives it virtual memory but it only uses physical memory when its actually used. Virtual memory should be very cheap.

It may be also worth to have a look at Netty's DynamicChannelBuffer. Things that I find handy are:
slice(int index, int length)
unsigned operations
separated writer and reader indexes

Indeed, auto-extending buffers are so much more intuitive to work with. If you can afford the performance luxury of reallocation, why wouldn't you!?
Netty's ByteBuf gives you exactly this. It's like they've taken java.nio's ByteBuffer and scraped away the edges, making it much easier to use.
Furthermore, it's on Maven in an independent netty-buffer package so you don't need to include the full Netty suite to use.

I'd suggest using an input stream to receive data from a file (with a sperate thread if you need non-blocking) then read bytes into a ByteArrayOutstream which gives you the ability to get it as a byte array. Heres a simple example without adding too many workarounds.
try (InputStream inputStream = Files.newInputStream(
Paths.get("filepath"), StandardOpenOption.READ)){
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int byteRead = 0;
while(byteRead != -1){
byteRead = inputStream.read();
baos.write(byteRead);
}
ByteBuffer byteBuffer = ByteBuffer.allocate(baos.size())
byteBuffer.put(baos.toByteArray());
//. . . . use the buffer however you want
}catch(InvalidPathException pathException){
System.out.println("Path exception: " + pathException);
}
catch (IOException exception){
System.out.println("I/O exception: " + exception);
}

Another solution for this would be to allocate more than enough memory, fill the ByteBuffer and then only return the occupied byte array:
Initialize a big ByteBuffer:
ByteBuffer byteBuffer = ByteBuffer.allocate(1000);
After you're done putting things into it:
private static byte[] getOccupiedArray(ByteBuffer byteBuffer)
{
int position = byteBuffer.position();
return Arrays.copyOfRange(byteBuffer.array(), 0, position);
}
However, using a org.apache.commons.io.output.ByteArrayOutputStream from the start would probably be the best solution.

Netty ByteBuf is pretty good on that.

A Vector allows for continuous growth
Vector<Byte> bFOO = new Vector<Byte>();
bFOO.add((byte) 0x00);`

To serialize somethiing you will need object in entry. What you can do is put your object in collection of objects, and after that make loop to get iterator and put them in byte array. Then, call ByteBuffer.allocate(byte[].length). That is what I did and it worked for me.

How to get data out of network packet data in Java

In C if you have a certain type of packet, what you generally do is define some struct and cast the char * into a pointer to the struct. After this you have direct programmatic access to all data fields in the network packet. Like so :
struct rdp_header {
int version;
char serverId[20];
};
When you get a network packet you can do the following quickly :
char * packet;
// receive packet
rdp_header * pckt = (rdp_header * packet);
printf("Servername : %20.20s\n", pckt.serverId);
This technique works really great for UDP based protocols, and allows for very quick and very efficient packet parsing and sending using very little code, and trivial error handling (just check the length of the packet). Is there an equivalent, just as quick way in java to do the same ? Or are you forced to use stream based techniques ?

Read your packet into a byte array, and then extract the bits and bytes you want from that.
Here's a sample, sans exception handling:
DatagramSocket s = new DatagramSocket(port);
DatagramPacket p;
byte buffer[] = new byte[4096];
while (true) {
p = new DatagramPacket(buffer, buffer.length);
s.receive(p);
// your packet is now in buffer[];
int version = buffer[0] << 24 + buffer[1] << 16 + buffer[2] < 8 + buffer[3];
byte[] serverId = new byte[20];
System.arraycopy(buffer, 4, serverId, 0, 20);
// and process the rest
}
In practise you'll probably end up with helper functions to extract data fields in network order from the byte array, or as Tom points out in the comments, you can use a ByteArrayInputStream(), from which you can construct a DataInputStream() which has methods to read structured data from the stream:
...
while (true) {
p = new DatagramPacket(buffer, buffer.length);
s.receive(p);
ByteArrayInputStream bais = new ByteArrayInputStream(buffer);
DataInput di = new DataInputStream(bais);
int version = di.readInt();
byte[] serverId = new byte[20];
di.readFully(serverId);
...
}

I don't believe this technique can be done in Java, short of using JNI and actually writing the protocol handler in C. The other way to do the technique you describe is variant records and unions, which Java doesn't have either.
If you had control of the protocol (it's your server and client) you could use serialized objects (inc. xml), to get the automagic (but not so runtime efficient) parsing of the data, but that's about it.
Otherwise you're stuck with parsing Streams or byte arrays (which can be treated as Streams).
Mind you the technique you describe is tremendously error prone and a source of security vulnerabilities for any protocol that is reasonably interesting, so it's not that great a loss.

I wrote something to simplify this kind of work. Like most tasks, it was much easier to write a tool than to try to do everything by hand.
It consisted of two classes, Here's an example of how it was used:
// Resulting byte array is 9 bytes long.
byte[] ba = new ByteArrayBuilder()
.writeInt(0xaaaa5555) // 4 bytes
.writeByte(0x55) // 1 byte
.writeShort(0x5A5A) // 2 bytes
.write( (new BitBuilder()) // 2 bytes---0xBA12
.write(3, 5) // 101 (3 bits value of 5)
.write(2, 3) // 11 (2 bits value of 3)
.write(3, 2) // 010 (...)
.write(2, 0) // 00
.write(2, 1) // 01
.write(4, 2) // 0002
).getBytes();
I wrote the ByteArrayBuilder to simply accumulate bits. I used a method chaining pattern (Just returning "this" from all methods) to make it easier to write a bunch of statements together.
All the methods in the ByteArrayBuilder were trivial, just like 1 or 2 lines of code (I just wrote everything to a data output stream)
This is to build a packet, but tearing one apart shouldn't be any harder.
The only interesting method in BitBuilder is this one:
public BitBuilder write(int bitCount, int value) {
int bitMask=0xffffffff;
bitMask <<= bitCount; // If bitcount is 4, bitmask is now ffffff00
bitMask = ~bitMask; // and now it's 000000ff, a great mask
bitRegister <<= bitCount; // make room
bitRegister |= (value & bitMask); // or in the value (masked for safety)
bitsWritten += bitCount;
return this;
}
Again, the logic could be inverted very easily to read a packet instead of build one.
edit: I had proposed a different approach in this answer, I'm going to post it as a separate answer because it's completely different.

Look at the Javolution library and its struct classes, they will do just what you are asking for. In fact, the author has this exact example, using the Javolution Struct classes to manipulate UDP packets.

This is an alternate proposal for an answer I left above. I suggest you consider implementing it because it would act pretty much the same as a C solution where you could pick fields out of a packet by name.
You might start it out with an external text file something like this:
OneByte, 1
OneBit, .1
TenBits, .10
AlsoTenBits, 1.2
SignedInt, +4
It could specify the entire structure of a packet, including fields that may repeat. The language could be as simple or complicated as you need--
You'd create an object like this:
new PacketReader packetReader("PacketStructure.txt", byte[] packet);
Your constructor would iterate over the PacketStructure.txt file and store each string as the key of a hashtable, and the exact location of it's data (both bit offset and size) as the data.
Once you created an object, passing in the bitStructure and a packet, you could randomly access the data with statements as straight-forward as:
int x=packetReader.getInt("AlsoTenBits");
Also note, this stuff would be much less efficient than a C struct, but not as much as you might think--it's still probably many times more efficient than you'll need. If done right, the specification file would only be parsed once, so you would only take the minor hit of a single hash lookup and a few binary operations for each value you read from the packet--not bad at all.
The exception is if you are parsing packets from a high-speed continuous stream, and even then I doubt a fast network could flood even a slowish CPU.

Short answer, no you can't do it that easily.
Longer answer, if you can use Serializable objects, you can hook your InputStream up to an ObjectInputStream and use that to deserialize your objects. However, this requires you have some control over the protocol. It also works easier if you use a TCP Socket. If you use a UDP DatagramSocket, you will need to get the data from the packet and then feed that into a ByteArrayInputStream.
If you don't have control over the protocol, you may be able to still use the above deserialization method, but you're probably going to have to implement the readObject() and writeObject() methods rather than using the default implementation given to you. If you need to use someone else's protocol (say because you need to interop with a native program), this is likely the easiest solution you are going to find.
Also, remember that Java uses UTF-16 internally for strings, but I'm not certain that it serializes them that way. Either way, you need to be very careful when passing strings back and forth to non-Java programs.