Java DataOutputStream object flushes after 8192 bytes [closed] - java

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I'm having a Java variable declared as such:
RandomAccessFile file = new RandomAccessFile("path-to-file");
DataOutputStream output = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(file.getFD())));
The problem is that after 2048 introduced integers (or 8192 bytes), if I try to write another one, the output flushes to disk without any warning.
After some research I found out that DataOutputStream has a 64k limit, and I couldn't find anything official about BufferedOutputStream or FileOutputStream.
Can anybody please tell me which structure has this limitation and how can I increase it?
It would also be nice to find out why does the structure flush without being told to instead of just raising an exception?

It's the BufferedOutputStream who flushes, whose default size is 8192:
public BufferedOutputStream(OutputStream out) {
this(out, 8192);
}
Just create the BufferedOutputStream with a seconds parameter describing the wanted buffer size!
int BUFFER_SIZE= <some value>;
DataOutputStream output = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(file.getFD()), BUFFER_SIZE));
However, this might not be what you want!
Queuing writes and writing them in one big chunk (at flush) is the very purpose of the BufferedOutputStream and it increases performance considerably. It's not at all a problem when it flushes. Just make sure that it flushes after you made your last write.
Additionally, I'm pretty sure you got the 64K limit of the DataOutputStream wrong!
Itself will write unlimited bytes, however it will only write strings with a maximum length of 64K when encoded as UTF-8.
It also contains a counter how many bytes have been written. Obviously this one will overflow after Integer.MAX_VALUE written bytes, but that shouldn't matter!

Related

Is there a convenient class library in Java for manipulating byte[], such as writing other types of data by subscript? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 days ago.
Improve this question
I'm rewriting a piece of Go code in Java:
//instruction is a byte[], o is an int.
//Start writing an int type data from the offset subscript of the instruction
binary.BigEndian.PutUint16(instruction[offset:], uint16(o))
I need to achieve a similar effect to a byte[] in Java, but I can't find a class library in Java that can easily achieve this kind of operation.
When I use copilot, it prompts me to use bit operations to do the job, but it's too tedious and unreadable, I don't want to do that, so I need a class library.
One way would be to wrap the byte array in a ByteBuffer and use its various putXXX methods.
e.g.
// "wrap" makes a big endian byte buffer by default
var buffer = ByteBuffer.wrap(yourByteArray);
// puts 300 as a 16 bit signed integer (short) at index 1
buffer.putShort(1, (short)300);
There are also put (for byte), putChar, putDouble, putFloat, putLong, putInt.
You could use MethodHandles::byteArrayViewVarHandle:
var handle = MethodHandles.byteArrayViewVarHandle(short[].class, ByteOrder.nativeOrder());
byte[] arr = new byte[2];
handle.set(arr, 0, (short) 42);
// arr ==> byte[2] { 42, 0 }
If you access the data sequentially, the time-honored DataOutputStream would fit, too:
var baos = new ByteArrayOutputStream();
try (var out = new DataOutputStream(baos)) {
out.writeShort(0xfffe);
}
var bytes = baos.toByteArray();
System.out.println(Arrays.toString(bytes));
try (var in = new DataInputStream(new ByteArrayInputStream(bytes))) {
var us = in.readUnsignedShort();
System.out.println(us);
}
(if you don't need the byte[], you can of course stream directly to/from a file, socket, ...)

How to know file size from bytes [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have byte array, how to know file size ? (in JAVA)
File file = new File("patch");
file.length() // <--- it's good, but I haven't original file... (( I get file in bytes from DataBase !
Thanks
You have an array with you and every array has a length. That's it.
byteArray.length;
And
1kb = 1024 bytes

What is a buffer on java? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have a homework to do on Java and is asks me to create a buffer method and the constructor should make an empty buffer structure.
There is no details about what that buffer is. It also wants to insert chars inside the buffer, delete the char that buffer shows, go buffer X positions left or right and tell the number of buffers chars. All these with different methods.
The problem is WHAT IS THAT BUFFER??? Is this something specific?
I would just use a StringBuilder. There are many other possible solutions but StringBuilder is the most widely used buffer for chars these days.
StringBuilder sb = new StringBuidler();
It also wants to insert chars inside the buffer,
sb.append('!');
sb.append("Hello");
sb.insert(5, "bye");
delete the char that buffer shows,
sb.delete(3, 6);
go buffer X positions left or right and tell the number of buffers chars.
sb.charAt(5); // character at 5
int len = sb.length(); // number of characters.

Java: Need optimzed (fast) method for writing integer array to FileOutputStream [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Searching yields many questions about how to convert an int to byte[]. I have a project with a critical loop that writes a long int[] to a FileOutputStream. FileOutputStream requires a byte[] for writing. I can brute-force different methods; I'm looking for a way to send an int[] directly to a FileOutputStream or the fastest method to convert int[] to byte[] - something like wrapping a buffer. I see ways to wrap a byte[] to convert to int[] and float[]... but none the other way (from int[] to byte[]). Thanks.
Update: still hoping to avoid the complexity (or experimenting - for now) of memory mapped I/O until the need is proven. The comments below prompted me to look at creating a ByteBuffer, wrapping it in an IntBuffer, writing ints to the IntBuffer, then extracting a byte[] from the ByteBuffer to send to the FileOutputStream. The obvious alternative is just to use byte[] directly, which requires that I manipulate my data as bytes rather than ints, which I can do - but how much more efficient (if at all) is it compared to the byte[]/ByteBuffer/IntBuffer wrapping scheme?
Your bottleneck is most likely to be your disk IO so what you do in CPU doesn't matter. I would make sure you trying to solve a problem which will make a difference to your application.
If you have a fast disk sub system and you have short bursts of data, your CPU can matter and the fastest way to do the conversion is to avoid performing the conversion in the first place, ie don't use a byte[] at all. An example if OpenHFT/Java Chronicle this takes an int value and writes it direct to a memory mapped file memory region as a 32-bit value. This means each write consists of a single machine code instruction and takes about 1.5 ns on average.
Try ObjectOutputStream.writeObject(intArray). You can later read it with ObjectIntputStream.readObject

Sample Java code to simulate out of memory situation [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I am basically a Weblogic admin and want to simulate out of memory situation through deploying a very simple Java code(war/ear file) in my Weblogic instance.
I have a very little knowledge about Java coding so can someone please provide me a sample code which I can easily pack as war and deploy?
This should be enough:
long[][] ary = new long[Integer.MAX_VALUE][Integer.MAX_VALUE];
This will try to allocate 2^31 + 1 memory blocks, each of size 2^34 bytes.
You can do final long[] l = new long[Integer.MAX_VALUE]; It will allocate 16Gb - 8 bytes.
Or you can just throw new OutOfMemoryError();
To simulate the memory being consumed over time try:
List<long[]> list = new LinkedList<long[]>();
while (true) {
list.add(new long[65536]); // an arbitrary number
// sleep(1) perhaps?
}

Categories