I have been struggling for finding the exact size of java objects. I have tried different options and non of them are working correctly. Then I tried to serialize the object and find the size of the serialized object. The size of the serialized object is very small, like in few kilo byte. So I doubted my method. I was wondering is it the correct way? Is there any problem you people see in this procedure? Please help...
You can convert your object into a byte array using ObjectOutputStream and ByteArrayOutputStream:
public static int sizeof(Object obj) throws IOException {
ByteArrayOutputStream byteOutputStream = new ByteArrayOutputStream();
ObjectOutputStream objectOutputStream = new ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(obj);
objectOutputStream.flush();
objectOutputStream.close();
return byteOutputStream.toByteArray().length;
}
Related
I have a serializable class with custom writeObject() and readObject() methods.
When an object serializes, it needs to write two byte arrays, one after another. When something deserializes it, it needs to read those two arrays.
This is my code:
private void writeObject (final ObjectOutputStream out) throws IOException {
..
out.writeByte(this.signature.getV()); //one byte
out.writeObject(this.signature.getR()); //an array of bytes
out.writeObject(this.signature.getS()); //an array of bytes
out.close();
}
private void readObject (final ObjectInputStream in) throws IOException, ClassNotFoundException {
..
v = in.readByte();
r = (byte[])in.readObject();
s = (byte[])in.readObject();
this.signature = new Sign.SignatureData(v, r, s); //creating a new object because
//sign.signaturedata
// is not serializable
in.close();
}
When the object is being deserialized (readObject method) it throws an EOFException and all three variables are null/undefined.
Relating to question title, I saw a class called ByteArrayOutputStream, but to use it, it has to be enclosed in a ObjectOutputStream, which I cannot do, ad I have an OutputStream given and must write with it.
1. How do one properly write a byte array using objectOutputStream and properly reads it using ObjectInputStream?
2. Why the code above throws an EOFException without reading even one variable?
EDIT: I need to clarify: the readObject() and writeObject() are called by jvm itself while deserializing and serializing the object.
The second thing is, the SignatureData is a subclass to Sign, that comes from a third-party library - and that's why it's not serializable.
The third thing is, the problem probably lies in the reading and writing byte arrays by ObjectInput/ObjectOutput streams, not in the Sign.SignatureData class.
As far I know, Kryo serialization / deserialization happens per object. Is it possible to serialize multiple objects into a single file?. One of workaround suggested in another similar SO question was to use an array of objects. Considering a huge amount of data that needs to be serialized, I feel it would not be as efficient as it should be. Is it right assumption?
Does Kryo API take an OutputStream? If so, just feed it the same OutputStream to serialize multiple files. Do the same with InputStream when reading. A good serialization format will have length encodings or termination symbols and would not rely on EOF for anything.
The array approach would also work with minimal overhead as long as all of these objects are already in memory. You are talking about adding just a few bytes per object to create an array to hold them. If they aren't all in memory, you would have to load them all into memory first to create an array around them. That could definitely become a problem given large enough data set.
As Kryo supports streaming there is nothing to stop you writing/reading more than one object to kryo "at the top level". For example the following program writes two unrelated objects to a file and then deserializes them again
public class TestClass{
public static void main(String[] args) throws FileNotFoundException{
serialize();
deSerialize();
}
public static void serialize() throws FileNotFoundException{
Collection<String>collection=new ArrayList<>();
int otherData=12;
collection.add("This is a serialized collection of strings");
Kryo kryo = new Kryo();
Output output = new Output(new FileOutputStream("testfile"));
kryo.writeClassAndObject(output, collection);
kryo.writeClassAndObject(output, otherData); //we could add as many of these as we like
output.close();
}
public static void deSerialize() throws FileNotFoundException{
Collection<String>collection;
int otherData;
Kryo kryo = new Kryo();
Input input = new Input(new FileInputStream("testfile"));
collection=(Collection<String>)kryo.readClassAndObject(input);
otherData=(Integer)kryo.readClassAndObject(input);
input.close();
for(String string: collection){
System.out.println(string);
}
System.out.println("There are other things too! like; " + otherData);
}
}
Is there some subtle reason why java.nio.ByteBuffer does not implement java.io.DataOutput or java.io.DataInput, or did the authors just not choose to do this? It would seem straightforward to map the calls (e.g. putInt() -> writeInt()).
The basic problem I (and some others, apparently) have is older classes which know how to serialize/serialize themselves using the generic interfaces: DataInput/DataOutput. I would like to reuse my custom serialization without writing a custom proxy for ByteBuffer.
Just wrap the buffer in ByteArrayInputStream or ByteArrayOutputStream using the put() or wrap() methods. The problem with having a ByteBuffer directly emulate a datainput/output stream has to do with not knowing the sizes in advance. What if there's an overrun?
What is needed is a ByteBufferOutputStream in which you can wrap / expose the required behaviors. Examples of this exist; the Apache avro serialization scheme has such a thing. It's not too hard to roll your own. Why is there not one by default? Well, it's not a perfect world...
ByteArrayOutputStream backing = new ByteArrayOutputStream();
DataOutput foo = new DataOutputStream(backing);
// do your serialization out to foo
foo.close();
ByteBuffer buffer = ByteBuffer.wrap(backing.toByteArray());
// now you've got a bytebuffer...
A better way that works with direct buffers too:
class ByteBufferOutputStream extends OutputStream
{
private final ByteBuffer buffer;
public ByteBufferOutputStream(ByteBuffer buffer)
{
this.buffer = buffer;
}
public void write(int b) throws IOException
{
buffer.put((byte) b);
}
}
Note that this requires calling buffer.flip() after you are done writing to it, before you can read from it.
I understand the theory behind incompatible serialVersionUIDs (i.e. you can discriminate different compilation versions of the same class) but I am seeing an issue that I don't understand and doesn't fall into the obvious error causes (different compiled version of the same class).
I am testing a serialization/deserialization process. All code is running on one machine, in the same VM, and both serialization and deserialization methods are using the same version of the compiled class. Serialization works fine. The class being serialized is quite complex, contains a number of other classes (java types and UDTs), and contains reference cycles. I haven't declared my own UID in any class.
This is the code:
public class Test {
public static void main(String[] args) throws Exception {
ContextNode context = WorkflowBuilder.getSimpleSequentialContextNode();
String contextString = BinarySerialization.serializeToString(context);
ContextNode contextD = BinarySerialization.deserializeFromString(ContextNode.class, contextString);
}
}
public class BinarySerialization {
public static synchronized String serializeToString(Object obj) throws Exception {
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(byteStream);
oos.writeObject(obj);
oos.close();
return byteStream.toString();
}
public static synchronized <T> T deserializeFromString(Class<T> type, String byteString) throws Exception {
T object = null;
ByteArrayInputStream byteStream = new ByteArrayInputStream(byteString.getBytes());
ObjectInputStream in = new ObjectInputStream(byteStream);
object = (T)in.readObject();
in.close();
return object;
}
}
I am getting an InvalidClassException (local class incompatible: stream classdesc serialVersionUID = -7189235121689378989, local class serialVersionUID = -7189235121689362093) when deserializing.
What is the underlying issue? And how should I fix it?
Thanks
Edit
I should state the purpose of this. The serialized data will both need to be stored in a sqlite database and sent across the wire to other clients. If String is the wrong format for passing around the serialized data, what should I be using instead that will let me store and pass the data about? Thanks again.
First rule: never use String or char[] or Reader or Writer when handling binary data.
You're handling binary data and try to put it into a String. Don't do that, that's an inherently broken operation.
Next: the return value of byteStream.toString() doesn't in any way represent the actual content of the ByteArrayOutputStream. You'll want to use .getBytes() and pass the byte[] around (remember: treat binary data as binary data and not as a String).
I'm trying to use javax.crypto.Cipher.doFinal(byte[]) method to encrypt an object. But, for security reasons, the object cannot be serializable.
So, how to convert the object to byte array without serialization?
--update
is using serialization the only way to use this Cipher method? Because as I know important data should not be serializable.
I used com.fasterxml.jackson.databind.ObjectMapper.
private static byte[] serialize(Object obj) throws IOException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
mapper.writeValue(os, obj);
return os.toByteArray();
}
You just serialize each of it's components. Recurse. Eventually you end up with native objects that you can serialize.
If you implement this by implementing java's serialization methods, java will ensure that you do not serialize any object twice and will take care of references for you.
In short, make the object serializable.
Solved,
instead of use a getByteArray() to call Cipher.doFinal(), I'll use Cipher.doFinal() inside the class, with a getEncryptedByteArray() method; so I serialize the data inside the class without making the class itself serializable, and the return result will be encrypted.
Any objection to this approach will be considered.. :)
Here is a simple example of serializing a class to a byte array.
public Class Foo {
private boolean isHappy;
private short happyCount;
private Bar bar;
public byte[] serializeData () throws IOException
{
ByteArrayOutputStream stream = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream( stream );
out.writeBoolean(isHappy);
out.writeShort( slope );
// Serialize bar which will just append to this byte stream
bar.doSerializeData(out);
// Return the serialized object.
byte[] data = stream.toByteArray();
// Clean up.
stream.close();
return data;
}
}
Of course, a lot of the details in your case depend on your class structure but hopefully this gets you pointed in the right direction.
To deserialize you just need to reverse the above.
java.beans.XMLEncoder/Decoder.