Since JDK 1.4, Direct Buffer was introduced along with Java NIO. One reason of it is Java GC may move the memory. Therefore the buffer data must be put off heap.
I'm wondering why traditional Java blocking IO api (BIO) doesn't need a direct buffer? Does BIO use something like direct buffer internally, or are there some other mechanisms to avoid the "memory movement" problem?
The simple answer is: It doesn't matter. Java has a clear, and public, spec. The JLS, the JVMS, and the javadoc of the core library. Java implementations do exactly what those 3 documents state, and you may trust that somehow it 'works'. This isn't as trite as it sounds, for example, the JMM (Java Memory Model, part of the JVMS if memory serves) lays out all sorts of things a JVM 'may' do in regards to re-ordering instructions and caching local writes, which is tricky, because it means if you mess that up, given that it is a 'may', a JVM may not actually bug out, even though your code is buggy, in that a JVM may do X, and if it does that, your code breaks; just that on your machine, at this time, with this song playing on your music player, the JVM chose never to do X, so you can't observe the problem.
Fortunately, the BIO stuff mostly has no may in it.
Here is the basic outlay of BIO in java:
You call .read() .read(byte[]), or .read(byte[], off, len).
(This is no guarantee; an implementation detail; a JVM is not required to do it this way): The JVM will read 'as much as is currently available' (hence, .read(some100SizedByteArr) may read only 50 bytes, even though if you call read again it'll read more bytes: 50 so happened to be 'ready' in the network buffer. Lots of folks get that wrong and think .read(byte[]) will fill the byte array if it can. Nope. That would make it impossible to write code that processes data as it comes in!
(Again, no guarantee): Given that byte arrays can be shoved around in memory, you'd think that's a problem, but it really isn't: That byte[] is guaranteed not to magically grow new bytes in it, there is no way with the BIO API to say: Just fill this array as the bytes fly in over the wire. The only way to fill that array is to call .read() on your inputstream. That is a blocking operation, and the JVM can therefore 'deal with it' as it pleases. Perhaps the native layer simply locks out the garbage collector until data is returned (this isn't as pricey as it sounds; the .read() method, once at least 1 byte can be returned, returns quickly, it doesn't wait for more data beyond the first byte, at least, that's how most JVMs do it). Perhaps it will read the data into a cloned buffer that lives out of heap and blits it over into your array later (sounds inefficient, perhaps, but a JVM is free to do it this way). Possibly the JVM marks that byte array specifically as off-limits for movement of any sort but the GC just collects 'around' it. It doesn't matter - a JVM can do whatever it wants. As long as it guarantees that `.read(byte[]):
Blocks until EOF is reached (in which case it returns -1), or at least 1 byte is available.
Fills the byte array with the bytes so returned.
Marks the inputstream as having 'consumed' all that you just got.
Returns a value representing how many bytes have been filled.
That's sort of the point of java: The how is irrelevant. Had the how not been irrelevant, writing a JVM for a new platform could be either impossible or require full virtualization, making it incredibly slow. The docs give themselves some 'may' clauses exactly so that this can be avoided.
One place where may does show up in BIO: When you .interrupt() a thread that is currently locked in a BIO .write() call (and the bytes haven't all been sent yet, let's say the network is slow and you sent a big array), o a BIO .read() call (it blocks until at least 1 byte is available; let's say the other side isn't sending anything) - then what happens? The docs leave it out. It 'may' result in an IOException being thrown, thus ending the read/write call, with a message indicating you interrupted it. Or, .interrupt() does nothing, and it is in fact impossible to interrupt a thread frozen on a BIO call. Most JVMs do the exception thing (fortunately), but the docs leave room - if for whatever reason the underlying OS/arch don't make that feasible, then a JVM is free not to do anything if you attempt to interrupt(). Conclusion: If you want to write proper 'write once run anywhere' code you can't rely on the idea that you can .interrupt() BIO freezes.
Related
I need to load a file into memory. Before I do that I want to make sure that there is enough memory in my VM left. If not I would like to show an error message. I want to avoid the OutOfMemory exception.
Approach:
Get filesize of my file
Use Runtime.getRuntime().freeMemory()
Check if it fits
Would this work or do you have any other suggestions?
The problem with any "check first then do" strategy is that there may be changes between the "check" and the "do" that render the entire thing useless.
A "try then recover" strategy is almost always a better idea and, unfortunately, that means trying to allocate the memory and handling the exception. Even if you do the "check first" option, you should still code for the possibility that the allocation may fail.
A classic example of that would be checking that a file exists before opening it. If someone were to delete the file between your check and open, you'll get an exception regardless of the fact the file was there very recently.
Now I don't know why you have an aversion to catching the exception but I'd urge you to rethink it. Since Java relies heavily on them, they're generally accepted as a good way to do things, if you don't actually control what it is you're trying (such as opening files or allocating memory).
If, as it seems from your comments, you're worried about the out-of-memory affecting other threads, that shouldn't be the case if you try to allocate one big area for the file. If you only have 400M left and you ask for 600, your request will fail but you should still have that 400M left.
It's only if you nickle-and-dime your way up to the limit (say trying 600 separate 1M allocations) would other threads start to feel the pinch after you'd done about 400. And that would only happen if you dodn't release those 400 in a hurry.
So perhaps a possibility would be to work out how much space you need and make sure you allocate it in one hit. Either it'll work or it won't. If it doesn't, your other threads will be no worse off.
I suppose you could use your suggested method to try and make sure the allocation left a bit of space for the other threads (say 100M or 10% or something like that), if you're really concerned. But I'd just go ahead and try anyway. If your other threads can't do their work because of low memory, there's ample precedent to tell the user to provide more memory for the VM.
Personally I would advice against loading a massive file directly into memory, rather try to load it in chunks or use some sort of temp file to store intermediate data.
You may want to look at the FileChannel.map(FileChannel.MapMode, long, long) method. This allows mapping a file (think POSIX mmap) without filling the heap. The operating system will (hopefully successfully) take care of the memory for you.
This is more like a matter of conscience than a technological issue :p
I'm writing some java code to dowload files from a server...For that, i'm using the BufferedOutputStream method write(), and BufferedInputStream method read().
So my question is, if i use a buffer to hold the bytes, what should be the number of bytes to read? Sure i can read byte to byte using just int byte = read() and then write(byte), or i could use a buffer. If i take the second approach, is there any aspects that i must pay attention when defining the number of bytes to read\write each time? What will this number affect in my program?
Thks
Unless you have a really fast network connection, the size of the buffer will make little difference. I'd say that 4k buffers would be fine, though there's no harm in using buffers a bit bigger.
The same probably applies to using read() versus read(byte[]) ... assuming that you are using a BufferedInputStream.
Unless you have an extraordinarily fast / low-latency network connection, the bottleneck is going to be the data rate that the network and your computers' network interfaces can sustain. For a typical internet connection, the application can move the data two or more orders of magnitude of times faster than the network can. So unless you do something silly (like doing 1 byte reads on an unbuffered stream), your Java code won't be the bottleneck.
BufferedInputStream and BufferedOutputStream typically rely on System.arraycopy for their implementations. System.arraycopy has a native implementation, which likely relies on memmove or bcopy. The amount of memory that is copied will depend on the available space in your buffer, but regardless, the implementation down to the native code is pretty efficient, unlikely to affect the performance of your application regardless of how many bytes you are reading/writing.
However, with respect to BufferedInputStream, if you set a mark with a high limit, a new internal buffer may need to be created. If you do use a mark, reading more bytes than are available in the old buffer may cause a temporary performance hit, though the amortized performance is still linear.
As Stephen C mentioned, you are more likely to see performance issues due to the network.
What is the MTU(maximum traffic unit) in your network connection? If you using UDP for example, you can check this value and use smaller array of bytes. If this is no metter, you need to check how memory eats your program. I think 1024 - 4096 will be good variant to save this data and continue to receive
If you pump data you normally do not need to use any Buffered streams. Just make sure you use a decently sized (8-64k) temporary byte[] buffer passed to the read method (or use a pump method which does it). The default buffer size is too small for most usages (and if you use a larger temp array it will be ignored anyway)
I have two scenarios in Netty where I am trying to minimize memory copies and optimize memory usage:
(1) Reading a very large frame (20 Megabites).
(2) Reading lots of very little frames (20 megabites at 50 bites per frame) to rebuild into one message at a higher level in the pipeline.
For the first scenario, as I get a length at the beginning of the frame, I extended FrameDecoder. Unfortunately as I don't see how to return the length to Netty (I only indicate whether the frame is complete or not), I believe Netty is going through multiple fill buffer, copy and realloc cycles thus using for more memory than is required. Is there something I am missing here? Or should I be avoiding the FrameDecoder entirely if I expect this scenario?
In the second scenario, I am currently creating a linked list of all the little frames which I wrap using ChannelBuffers.wrappedBuffer (which I can then wrap in a ChannelBufferInputStream), but I am again using far more memory than I expected to use (perhaps because the allocated ChannelBuffers have spare space?). Is this the right way to use Netty ChannelBuffers?
There is a specialized version of frame decoder called, LengthFieldBasedFrameDecoder. Its handy, when you have a header with message length. It can even extract the message length from header by giving an offset.
Actually, ChannelBuffers.wrappedBuffer does not creates copies of received data, it creates a composite buffer from given buffers, so your received frame data will not be copied. If you are holding the composite buffers/ your custom wrapper in the code and forgot to nullify, memory leaks can happen.
These are practices I follow,
Allocate direct buffers for long lived objects, slice it on use.
when I want to join/encode multiple buffers into one big buffer. I Use ChannelBuffers.wrappedBuffer
If I have a buffer and want to do something with it/portion of it, I make a slice of it by calling slice or slice(0,..) on channel buffer instance
If I have a channel buffer and know the position of data which is small, I always use getXXX methods
If I have a channel buffer, which is used in many places for make something out of it, always make it modifiable, slice it on use.
Note: channelbuffer.slice does not make a copy of the data, it creates a channel buffer with new reader & write index.
In the end, it appeared the best way to handle my FrameDecoder issue was to write my own on top of the SimpleChannelUpstreamHandler. As soon as I determined the length from the header, I created the ChannelBuffer with size exactly matching the length. This (along with other changes) significantly improved the memory performance of my application.
There's a common pattern, when each layer of application, dealing with data from a stream tends to wrap it into a BufferedInputStream, so that at a whole, there's a lot of buffers, filled from buffers, filled from buffers and so on.
I think this is bad practice and want to question: how does it impact the performance? Can this cause bugs?
This is a very general question, but I'd say there are a number of problems with having lots of layers of buffered input streams (in any language).
Each buffer takes up memory, even when it's not filled. So, even if the data gets sucked right up to the top "layer" straight away, memory is still being needlessly used. (Note: I'm assuming that Java doesn't resize its buffers automatically or anything — and I'm no Java expert.)
Whenever you read from the top-level buffer, you'll be setting off a big chain of method calls. Method calls involve indirection (i.e. pointer-following), passing-around of data (which could lead to poor caching performance), and so on.
It probably means that the design isn't very well-thought-out, since buffered streams should generally be for reading from sources that actually need buffering, like the disk or the network.
Just a few thoughts on the matter. I'm sure someone with better Java knowledge could contribute a more detailed analysis.
It will increase memory footprint due to the extra buffers, but I suspect its rare given the sizes likely involved that it will actually have a significant effect on a given program. Theres the standard rule of not trying to optimise before you need to.
Theres also bound to be a slight processor overhead, but this will be even less significant.
It all depends just how much it is used, if there are many large chains it could be a problem, but I think it unlikely to be a problem.
As David said it is likely an indication of poor design It would probably be more efficient for components to be able to share more complex objects directly, but its all down to specific uses (and I'm having trouble thinking of a reason that you would use multiple buffered streams in such a way).
It is indeed very bad practice and can indeed cause bugs. If method A does some reading and then passes the stream to method B which attaches a BufferedInputStream and does some more reading, the BufferedInputStream will fill its buffer, which may consume data that method A is expecting to be still there when method B returns. Data can be lost by method B's BufferedInputStream reading ahead.
As regards overheads, in practice, if the reads/writes are large enough, the intermediate buffers are bypassed anyway, so there isn't nearly as much extra copying as you might think: the performance impact is mostly the extra memory space plus the extra method calls.
I'm using ByteBuffers and FileChannels to write binary data to a file. When doing that for big files or successively for multiple files, I get an OutOfMemoryError exception.
I've read elsewhere that using Bytebuffers with NIO is broken and should be avoided. Does any of you already faced this kind of problem and found a solution to efficiently save large amounts of binary data in a file in java?
Is the jvm option -XX:MaxDirectMemorySize the way to go?
I would say don't create a huge ByteBuffer that contains ALL of the data at once. Create a much smaller ByteBuffer, fill it with data, then write this data to the FileChannel. Then reset the ByteBuffer and continue until all the data is written.
Check out Java's Mapped Byte Buffers, also known as 'direct buffers'. Basically, this mechanism uses the OS's virtual memory paging system to 'map' your buffer directly to disk. The OS will manage moving the bytes to/from disk and memory auto-magically, very quickly, and you won't have to worry about changing your virtual machine options. This will also allow you to take advantage of NIO's improved performance over traditional java stream-based i/o, without any weird hacks.
The only two catches that I can think of are:
On 32-bit system, you are limited to just under 4GB total for all mapped byte buffers. (That is actually a limit for my application, and I now run on 64-bit architectures.)
Implementation is JVM specific and not a requirement. I use Sun's JVM and there are no problems, but YMMV.
Kirk Pepperdine (a somewhat famous Java performance guru) is involved with a website, www.JavaPerformanceTuning.com, that has some more MBB details: NIO Performance Tips
If you access files in a random fashion (read here, skip, write there, move back) then you have a problem ;-)
But if you only write big files, you should seriously consider using streams. java.io.FileOutputStream can be used directly to write file byte after byte or wrapped in any other stream (i.e. DataOutputStream, ObjectOutputStream) for convenience of writing floats, ints, Strings or even serializeable objects. Similar classes exist for reading files.
Streams offer you convenience of manipulating arbitrarily large files in (almost) arbitrarily small memory. They are preferred way of accessing file system in vast majority of cases.
Using the transferFrom method should help with this, assuming you write to the channel incrementally and not all at once as previous answers also point out.
This can depend on the particular JDK vendor and version.
There is a bug in GC in some Sun JVMs. Shortages of direct memory will not trigger a GC in the main heap, but the direct memory is pinned down by garbage direct ByteBuffers in the main heap. If the main heap is mostly empty they many not be collected for a long time.
This can burn you even if you aren't using direct buffers on your own, because the JVM may be creating direct buffers on your behalf. For instance, writing a non-direct ByteBuffer to a SocketChannel creates a direct buffer under the covers to use for the actual I/O operation.
The workaround is to use a small number of direct buffers yourself, and keep them around for reuse.
The previous two responses seem pretty reasonable. As for whether the command line switch will work, it depends how quickly your memory usage hits the limit. If you don't have enough ram and virtual memory available to at least triple the memory available, then you will need to use one of the alternate suggestions given.