I want to serialize a big object structure to store it to a sql database.
Object tree = getTree();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(tree);
String objectString = Base64.getEncoder().encodeToString(baos.toByteArray());
The problem is that the generated byte array by baos.toByteArray() is to large. It throws java.lang.OutOfMemoryError and it is to big to transfer it to the database at once. I'm looking for an option to get the generated byte array block by block from outputstream to work it off in a loop step by step.
Write the initial OutputStream to a temporary file (using FileOutputStream). also, make sure you close the ObjectOutputStream or it will be a malformed stream. Lastly, open your temp file as a FileInputStream and use that to stream into your database.
Forget about the base64-encoding and write the object directly to the Blob's output stream.
Related
if I Do like this:
FileOutputStream fos=new FileOutputStream("cse.txt");
ObjectOutputStream oos=new ObjectOutputStream(fos);
oos.writeObject(Dictionary);
and Again if I write the same file using the another object of Same type Dictionary on the same file like
FileOutputStream fos=new FileOutputStream("cse.txt");
ObjectOutputStream oos=new ObjectOutputStream(fos);
oos.writeObject(List);
Will that file get overwritten ?
If you write simply:
FileOutputStream fos=new FileOutputStream("cse.txt");
Then the old data will be lost and will get overridden by the new data.
However, if you write:
FileOutputStream fos=new FileOutputStream("cse.txt", true);
Then the old data will NOT be lost and the new data will be appended to the old one.
Here, the second argument true indicates that the bytes will be appended to the end of the file rather than overwriting the complete file.
See the documentation
In my case the out put stream is basically FileOutputStream. Hence for this code:
ByteArrayOutputStream bos = (ByteArrayOutputStream) streamToEncrypt;
(Where streamToEncrypt is OutputStream) is getting this exception:
java.lang.ClassCastException: java.io.FileOutputStream cannot be cast to java.io.ByteArrayOutputStream
All I need to do is to get the byte array out of this outputstream.
I do NOT have access to the file at this level. All I have is this output stream that I have to encrypt before pushing it to the file
That simply doesn't make any sense.
A ByteArrayOutputStream takes the data you push into it ... and stores them within a byte array that you can retrieve later on.
A FileOutputStream takes data ... and pushes it into a file.
Those are two completely different classes; you simply can't cast one into the other! Thats like you trying to cast a String into an Integer; or turning an Apple into a Banana by saying "you apple, now be a banana". Simply wont work!
So, the real "answer" here: you should step back and clarify what exactly you intend to do with your data; and then you use that stream(s) matching that requirement(s).
You don't get bytes out of an OutputStream. You get them out of an InputStream. You put bytes into an OutputStream.
If bytes have already been written to the OutputStream, it's too late. Opening the file again and reading from it is the only way for you to access those data.
If you want to encrypt an output stream, you should construct the stream and pass it to the code that writes to the stream.
Cipher enc = Cipher.getInstance("...");
enc.init(...);
try (OutputStream fos = Files.newOutputStream(path);
OutputStream os = new CipherOutputStream(fos, enc)) {
writingObject.write(os);
}
I was trying to read from file and then write to other file. I use code bellow to do so.
byte[] bytes = Files.readAllBytes(file1);
Writer Writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file2), "UTF-8"));
for(int i=0;i<bytes.length;i++)
Writer.write(bytes[i]);
Writer.close();
But when I change file1 to picture.png and file2 to picture2.png, this method doesn't work and I can't open picture2.png using image viewer.
What have I done wrong?
Writers are for writing text, possibly in different formats (ie utf-8 / 16, etc). For writing raw bytes, don't use writers. Just use (File)OutputStreams.
It is truly as simple as
byte[] bytes = ...;
FileOutputStream fos = ...;
fos.write(bytes);
The other answers explain why what you have potentially fails.
I'm curious why you're already using one Java NIO method, but not others? The library already has methods to do this for you.
byte[] bytes = Files.readAllBytes(file1);
Files.write(file2, bytes, StandardOpenOption.CREATE_NEW); // or relevant OpenOptions
or
FileOutputStream out = new FileOutputStream(file2); // or buffered
Files.copy(file1, out);
out.close();
or
Files.copy(file1, file2, options);
The problem is that Writer.write() doesn't take a byte. It takes a char, which is variable size, and often bigger than one byte.
But once you've got the whole thing read in as a byte[], you can just use Files.write() to send the whole array to a file in much the same way that you read it in:
Files.write(filename, bytes);
This is the more modern NIO idiom, rather than using an OutputStream.
It's worth reading the tutorial.
Consider these two functions:
Function A takes inputStream as parameter.
public void processStream(InputStream stream)
{
//Do process routine
}
Function B loads a file content to pass it to Function A as InputStream.
pulic void loadFile()
{
File file =new File("c:\\file.txt");
//Pass file as InputStream
}
How can I pass file from Function B to Function A as InputStream without reading it on first hand?
I did something like this:
File file = new File("c:\\file.txt");
DataInputStream stream= new DataInputStream(new FileInputStream(file));
This generated the exception below:
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: java.io.DataInputStream
EDIT:
loadFile() is passing the InputStream as RMI response.
The following should work just fine
processStream(new FileInputStream(file));
You should only not attempt to serialize an InputStream instance by ObjectOutputStream like as
objectOutputStream.writeObject(inputStream);
which you're apparently doing in processStream() method. That's namely exactly what the exception is trying to tell you. How to solve it properly depends on the sole functional requirement which you omitted from the question.
Update as per the comment
I am passing the InputStream as an RMI response.
There's the problem. You cannot pass non-serializable objects around as RMI response, let alone unread streams. You need to read the InputStream into a ByteArrayOutputStream the usual IO way and then use its toByteArray() to get a byte[] out of it and pass that instead. Something like:
InputStream input = new FileInputStream(file);
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
byte[] bytes = output.toByteArray(); // Pass that instead to RMI response.
Be careful with large files though. Every byte of a byte[] eats one byte of JVM's memory.
That exception seems to indicate that you are calling the processStream method on a remote object using something like RMI? if that is the case, you will need to re-visit what you are doing. sending streams of data over RMI is not an easy thing to do. if you are guaranteed to be using small files, you could copy the file data to a byte[] and pass that to the remote method call. if you need to process larger files, however, that will most likely cause memory issues on the client and/or server. in that case, you should use something like rmiio, which provides utilities for streaming data over RMI.
You could just pass the FileInputStream ?
processStream(new FileInputStream(yourFile));
The reason you are getting the exception is because DataInputStream is intended to read primitive Java types
I have some data values(of type TimePrimitive) which i need to write out to a file , but the method out.write() takes only the parameter as int so i need to find a way to convert my values to int
I think it is the wrong approach. If you want to write objects to a file, then you need to use an ObjectOutputStream to write to the file:
FileOutputStream fos = new FileOutputStream("t.tmp");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeInt(12345);
oos.writeObject("Today");
oos.writeObject(new Date());
oos.writeObject(myTimePrimitive);
oos.close();
The normal out.write(int) is used to write a simple byte to a stream, and it would be implicitely be used by the ObjectOutputStream class.
You can use an ObjectInputStream to read your object back.
Have you tried the getValue() method?
BTW, What platform are you using, Javascript itself does not have a TimePrimitive type, are you sure this isn't Java?