Is there any way to do asynchronous IO in Java without blocking any threads (including background threads)? Coming from C#, my understanding of async IO is that it when you call
await ReadAsync()
The calling thread (part of a threadpool) steps into the ReadAsync function, at some point calls an asynchronous read function from the OS kernel, and then adds itself back to the threadpool to pick up other Tasks. Once the read is completed, the threadpool is notified and another thread picks up the rest of the Task.
In Java, on the other hand, the documentation and this answer seem to suggest that asynchronous IO functions are simply called by a background thread that then blocks. This seems less performant. Is there any way to achieve true, non-blocking IO in Java?
The AsynchronousFileChannel.open() returns instances of different implementations according to the running environment. On Windows it should return an instance of WindowsAsynchronousFileChannelImpl which uses I/O completion port and avoids blocking threads on IO operations. Threads of thread pool are only used to dispatch results and do not block, unless the end user programmer blocks that thread.
The RxIo is built on top of AFC and provides the AsyncFiles equivalent to the synchronous Files class but with an asynchronous API. Taking advantage of the continuation-passing style of CompletableFuture (equivalent to .net Task) you may read a file content without blocking:
AsyncFiles
.readAll(path)
.thenAccept(body -> /* invoked on completion */)
.exceptionally(excep -> /* invoked on error*/
You may run the unit tests of RxIo and place a breakpoint at open() and inspect the implementation of WindowsAsynchronousFileChannelImpl.
Until some time ago there were problems with asynchronous file I/O on Linux. There was an aio interface, but it was only asynchronous for O_DIRECT, which is quite inconvenient for standard use cases. So the standard JDK implementation of AsynchronousFileChannel for Linux internally uses thread pooling and simple blocking I/O which is not really asynchronous I/O.
Things have changed a bit since Linux introduced the io_uring interface. It is now possible to use real non-blocking file I/O not just for O_DIRECT but for buffered I/O too. And a lot more, to reduce overhead of syscall and increase performance. Read more about io_uring.
At the moment there is no built-in support for io_uring in Java. There have been rumors that support may appear for better project Loom support, but that's just a rumors.
There are third party libraries that add asynchronous file I/O support via io_uring for Java - jasyncfio.
Related
People always show the same example when I read articles about concurrency in kotlin coroutine or golang goroutine.
Create 100_000 Threads in Java or C#, ooopps Stackoverflow.
Yes. but anyone Who uses directly Thread classes in Java or C#?
In java and C#, There are thread pools for CompletableFuture and Task.
When We try to create 100_000 Task or CompletableFuture, We can do that easily with ExecuterService/ForkJoinPool or dotnet DefaultThread Pool. They will reuse the threads. If there is no available thread. Tasks will wait in the queue.
My Questions;
yes structured concurrency is good for cancellations. But Kotlin uses the Thread Pool like CompletableFuture. But unlike Java Callbacks, It provides natural code syntax. The only Difference is Syntax for Kotlin coroutine between c# Task or Java CompletableFuture?
Kotlin runs on JVM. as far as I know, JVM doesn't support green Threads. But people talk like kotlin uses Green Threads. How is that possible with JVM? And Why Coroutines are called Lightweight Threads. Then We can say CompletableFuture and Task are Lightweight Thread too. Right?
Yes, golang has a scheduler. goroutines are user-level threads. When we create a goroutine it goes to localrunqueue. And a dedicated OS thread gets goroutines one by one from that queue and executes. There are no context switch operations. All of them run on the same OS Thread until blocking. Goroutines are cheap and We can say that YES goroutines are Lightweight Threads.
Maybe I'm completely wrong about coroutines. please correct me.
Making things simple:
Thread - easy to use, because it is sequential. Reading from the network and writing to a file is as simple as: writeToDisk(readFromNetwork()). On the other hand, it is expensive to run each task in a separate thread.
Executors/CompletableFuture - more performant, it makes better use of both CPU and memory. On the other hand, it requires using callbacks and the code quickly becomes hard to read and maintain.
Coroutines - both sequential and performant.
I ignore other features of coroutines like structured concurrency, because this was not your main concern.
I working on a service that will process large number of requests, each in a different file. The challenge is that processing requires both local processing (cpu), and remote (database) processing. Database has Hugh capacity. Database work is 30-80% of processing (Dynamic, can not be calculated up front)
The default commonPool used for completionService.completeAsync uses a pool with (processors-1) threads. Given that large portion of the processing is waiting for database work, the default commonPool underutilize local Machine resources.
I am believe using a custom executor that will conditionally pause if local load on the machine is high can improve situation. Now sure how to build such executor. Any advice ? Any existing library providing such code exists ?
For readers familiar with gnu make - equivalent to make ability to limit concurrent processing based on load.
I'm not sure if I understand your question correctly, but I'll take a shot at answering anyway. So what I understand is you run a large number of requests on a fixed size thread pool, and you find your CPU is underutilized because often times these threads are blocked waiting for a response from the database.
So generally speaking, I think what you want is to prevent your worker threads from being blocked by I/O. Instead of making your thread pool bigger to compensate for blocking I/O, you should use a non-blocking database driver and eliminate blocking I/O altogether.
Different approaches exist for different databases. Some support async I/O natively, some provide the illusion by maintaining a separate thread pool for DB I/O. Some integrate with higher-level abstractions such as Reactive Streams.
For Redis, for example, there is an alternative Java driver called Lettuce, that provides an asynchronous API and a reactive API. (Disclaimer: I have not used Lettuce myself.)
I would like to spawn an external process from Java and read its output without blocking the reader thread ("Non-blocking IO").
I know the (Socket) Java NIO Selector concept and the brilliant NuProcess library, but I am wondering, isn't there any built-in solution for that in Java8?
If you want non-blocking IO for consuming the output of an external process and you don't want to include a third party dependency, you you still have to write additional threading code and e.g. use BlockingQueue to pass around data read -- is that really all we have got?
As I know, one of the most common JVM concurrency API: futures - at least as implemented in scala - rely on user code to relinquish a thread when it is potentially going to be waiting idle. In scala it's commonly referred to as "avoiding blocking", and the developer has to implement it everywhere it makes sense.
Not quite efficient.
Is there something very entirely inherent to the JVM, that prevents the JVM switching the context of a thread to new tasks - when the thread is idle - as implemented by operating system process schedulers?
Is there something very entirely inherent to the JVM, that prevents the JVM switching the context of a thread to new tasks - when the thread is idle - as implemented by operating system process schedulers?
Mostly the need that such switch has to be done cooperatively. Every single blocking method must be wrapped or re-implemented in a way that allows the task to be resumed once it is done, after all, there is no native thread waiting for completion of the blocking action anymore.
While this can be done in principle for JVM-internal blocking methods, consider arbitrary native code executed via JNI, the JVM wouldn't know how to stack-switch those native threads, they're stuck in native code after all.
You might want to have a look at quasar, as I understand it they implemented such wrappers or equivalents for some JDK-internal methods, such as sleep, park/unpark, channel-based-IO and a bunch of others which allows their fibers (and thus futures running on those fibers) to perform exactly that kind of user-mode context switching while they wait for completion.
Edit: JNI alone already is sufficient to limit user-mode task switching to being an opportunistic optimization that may have to fall back to spinning up additional native threads when native code blocks a thread.
But it is not the only issue, for example on linux truly asynchronous file IO operations need filesystem and kernel support (see this SO question on AIO), which not all of them provide. Where it is not provided it has to be emulated using additional blocking IO threads, thus re-introducing all the overhead we wanted to avoid in the first place. Might as well just block on the thread pool itself and spin up additional threads, at least we'll avoid inter-thread-communication that way.
Memory-mapped files can also block a thread and force the OS-scheduler to suspend the thread due to page faults and I'm not aware of means to cooperate with the virtual memory system to avoid that.
Not to mention that all blocking calls on the VM would have to re-implemented using asynchronous equivalents provided by the OS. Miss even one and you'll have a blocked thread. If you have a blocked thread your thread pools will need an auto-grow feature and we're back to square one.
Last but not least, there may be cases where blocking, one-thread-per-filedescriptor IO may be desirable. The pervasive changes required to guarantee user-mode switching might break those.
So all in all, user mode switching is possible, sometimes. But the JVM cannot make hard guarantees about it so it has to implement all the native thread handling anyway and the programmer will have code at least somewhat cooperatively with the assumptions of the thread pools executing those futures in mind. Some of the cases could be eliminated, but not all of them.
The ForkJoinTask
explicitly calls out "Subdividable tasks should also not perform blocking I/O". It's primary aim is "computational tasks calculating pure functions or operating on purely isolated objects". My question is :-
Why design the ForkJoinTask to restrict blocking IO tasks?
What are the gotchas if i do implement a blocking IO task?
How come both spring and play frameworks, are full of examples using fork-join executors for DB calls?
In my scenario, a single request does two types of works, one of which is encryption, which pushes CPU core to 100% for 200 ms and second, few database calls. Any kind of static partitioning such as 6 threads for encryption and 2 threads for blocking IO, will not provide optimal usage of the CPU. Hence, having a fork-join executor, with certain level of over provisioning in number of threads over total CPU count, coupled with work stealing, would ensure better usage of CPU resources.
Is my above assumption and understanding around forkjoin executor correct and if not, please point me towards the gap.
Why design the ForkJoinTask to restrict blocking IO tasks?
underlying the fork join pool is shared amount of threads, if there's some IO work blocking on those threads, then less threads for CPU intensive work. other none blocking work will starve.
What are the gotchas if i do implement a blocking IO task?
typically, FJPool allocated thread about the number of processors. so if you do have to use IO blocking on threads, make sure you allocate enough threads for your other tasks.
you can also iso late your IO work on dedicated threads that is not shared with FJ pool. but you call blocking IO, your thread blocks and get scheduled for other task until unblocked
How come both spring and play frameworks, are full of examples using fork-join executors for DB calls?
play is no different. they use dedicated pools for IO task, so other task won't suffer.
The Framework does not restrict any type of processing. It is not recommended to do blocking, etc. I wrote a critique about this framework years ago, here is the point on the recommendations. This was for the Java7 version but it is still applicable for Java8.
Blocking is not fatal, sprint and play block and they work just fine. You need to be careful when using Java8 since there is a default common-fork/join pool and tying up threads there may have consequences for other users. You could always define your own f/j pool with the additional overhead, but at least you wouldn’t interfere with others using the common pool.
Your scenario doesn’t look bad. You’re not waiting for replies from the internet. Give it a try. If you run into difficulty with stalling threads, look into the ForkJoinPool.ManagedBlocker interface. Using that interface informs the f/j pool that you are doing blocking calls and the framework will create compensation threads.