According to the project loom documentation virtual threads behave like normal threads while having almost zero cost and the ability to turn blocking calls into non-blocking ones.
If this is true, then why are they separate things? Why not just make them the default? Is there any reason to not use them?
There are really two questions here: 1. Why are virtual threads not the default? and 2. Is there ever a reason not to use them.
Regarding the default, Java really has not concept of a "default" thread. Once virtual threads arrive, when you create a thread, you must specify whether you want a platform thread or a virtual thread. The question then becomes why have we decided not to automatically replace today's threads with virtual threads (i.e. make new Thread() create a virtual thread). The answer to that is quite simple: it would not be helpful at all and might well be quite harmful. It would not be helpful because the advantages of virtual threads come from the ability to create a great many of them. If your application creates N threads today, nothing would be gained by turning those N threads into virtual threads. The scaling advantage of virtual threads would only kick in when your application creates, say, 1000N threads, which means it would need to be changed, anyway (e.g. by replacing Executors.newFixedThreadPool with Executors.newVirtualThreadPerTaskExector). It might be harmful because while virtual threads' semantics are almost the same as platform threads, they are not perfectly backward compatible (see JEP 425 for details).
As to the question about when not to use virtual threads, there are some obvious cases. E.g. when your threads heavily interact with native code, which knows nothing about virtual threads, or when you depend on some detail that has changed for virtual threads, like the ability to subclass Thread. Other cases are not so clear. For example, CPU-bound operations do not benefit from having more threads than CPU cores, so they do not benefit from the multitude of virtual threads, but that doesn't mean that they would be harmed. We're still not ready to say that users should pick virtual threads by default, but we might well get there, as we learn more about how people use them.
Be aware that Project Loom is under active experimental development. Things may change.
No default
You asked:
Why not just make them the default?
In modern Java, we generally do not address threads directly. Instead, we use the Executors framework added years ago in Java 5.
In particular, in most cases a Java programmer uses the Executors utility class to produce an ExecutorService. That executor service is backed by various kinds of thread factories or thread pools.
For example, if you want to serialize one task after another, we would use an executor service backed by a single thread.
ExecutorService executorService = Executors.newSingleThreadExecutor() ;
If you browse through Executors class Javadoc, you will see a variety of options. 👉 None of them is "default". The programmer chooses one to suit the needs of her particular situation.
With Project Loom, we will have at least one more such option to choose from. In the preview build of Java, call the new Executors.newVirtualThreadPerTaskExecutor() to get an executor service backed by virtual threads. Go nuts, and throw a million tasks at it.
ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor() ;
You asked:
why are they separate things?
One of the highest priorities for the Java team is backward-compatibility: Existing apps should be able to run without surprise.
Virtual threads have a very different behavior and performance profile than platform threads. So I do not expect to see the Java team retrofitting virtual threads onto existing features of Java generally. They may choose to do so, but only if absolutely certain no detrimental effects will surface in the behavior of existing apps.
When to choose or avoid virtual threads
You asked:
Is there any reason to not use them?
Yes, certainly. Two reasons:
CPU-bound tasks
Tasks used to indirectly throttle other resources
CPU-bound tasks
The entire point of virtual threads is to keep the "real" thread, the platform host-OS thread, busy. When a virtual thread blocks, such as waiting for storage I/O or waiting network I/O, the virtual thread is "dismounted" from the host thread while another virtual thread is "mounted" on the host thread to get some execution done.
So, if your task’s code does not block, do not bother with virtual threads. But this kind of code is rare. Most tasks in most apps are often waiting for users, storage, networks, attached devices, etc. An example of a rare task that might not block is something that is CPU-bound like video-encoding/decoding, scientific data analysis, or some kind of intense number-crunching. Such tasks should be assigned to platform threads directly rather than virtual threads.
Throttling
Another reason to avoid virtual threads is with existing code that depends on the limits or bottlenecks of platform threads to throttle their app’s usage of other resources. For example, if a database server is limited to 10 simultaneous connections, then some app have been written to use an executor service backed by only 8 or 9 threads. Such existing code should not be blindly switched to virtual threads.
Of course such code is less than optimal. Such a code base would be better, clearer, more obvious to comprehend if explicit limiting/throttling mechanisms were utilized.
Using explicit throttling mechanisms will be needed if a programmer wants to benefit having thousands, even millions, of simultaneous virtual threads while avoiding exhausting/overloading other limited resources.
Java has long offered such throttling mechanisms. They just were not always used, given the simplicity/ease of relying on the limits/bottlenecks of a limited number of platform threads.
I am no expert on this. So rely on those who are experts. For details and insights, be sure to read the articles and watch the presentations and interviews by Ron Pressler, Alan Bateman, or other members of the Project Loom team.
Lets begin with
Why not just make them the default?
Virtual threads are wrapped upon platform threads, so you may consider them an illusion that JVM provides, the whole idea is to make lifecycle of threads to CPU bound operations.
Platform Threads versus Virtual threads. Platform threads take OS
threads hostage in IO based tasks and operations limited to number of
applicable threads with in thread pool and OS threads, by default
they are non Daemon threads
Virtual threads are implemented with JVM, in CPU bound operations the
associated to platform threads and retuning them to thread pool,
after IO bound operation finished a new thread will be called from
thread pool, so no hostage in this case.
Fourth level architecture to have better understanding.
CPU
Multicore CPU multicores with in cpu executing operations.
OS
OS threads the OS scheduler allocating cpu time to engaged OS
threads.
JVM
platform threads are wrapped totally upon OS threads with both task
operations
virtual threads are associated to platform threads in each CPU bound
operation, each virtual thread can be associated with multiple
platform threads as different times.
Virtual threads with Executer service
More effective to use executer service cause it associated to thread pool an limited to applicable threads with it, however in compare of virtual threads, with Executer service and virtual contained we do not ned to handle or manage the associated thread pool.
try(ExecutorService service = Executors.newVirtualThreadPerTaskExecutor()) {
service.submit(ExecutorServiceVirtualThread::taskOne);
service.submit(ExecutorServiceVirtualThread::taskTwo);
}
Executer service implements Auto Closable interface in JDK 19, thus when used with in 'try with resource', once it reach to end of 'try' block the 'close' api being called, alternatively main thread will wait till all submitted task with their dedicated virtual threads finish their lifecycle and associated thread pool being shutdown.
ThreadFactory factory = Thread.ofVirtual().name("user thread-", 0).factory();
try(ExecutorService service = Executors.newThreadPerTaskExecutor(factory)) {
service.submit(ExecutorServiceThreadFactory::taskOne);
service.submit(ExecutorServiceThreadFactory::taskTwo);
}
Executer service can be created with virtual thread factory as well, just putting thread factory with it constructor argument.
Can benefits features of Executer service like Future and Completable Future.
Virtual threads advantages
exhibits exact the same behavior as platform threads.
disposable and can be scaled to millions.
much more lightweight than platform threads.
fast creation time, as fast as creating string object.
the JVM does delimited continuation on IO operations, no IO for
virtual threads.
yet can have the sequential code as previous but way more effective.
the JVM gives an illusion of virtual threads, underneath whole story
goes on platform threads.
Just with usage of virtual thread CPU core become much more concurrent, the combination of virtual threads and multi core CPU with ComputableFutures to parallelized code is very powerful
Virtual threads usage cautions
Don not use monitor i.e the synchronized block, however this will fix in new release of JDK's, an alternative to do so is to use 'ReentrantLock' with try-final statement.
Blocking with native frames on stack, JNI's. its very rare
Control memory per stack (reduce thread locales and no deep recursion)
Monitoring tools not updated yet like debuggers, JConsole, VisualVM etc
Find more on JEP-425
If you make them default a good portion of the existing java code won't just be able to switch to java 19, because that code is optimized for OS threads.
Java has to be backward compatible.
There are cases where Virtual Threads don't make much sense for example
Applications that do heavy computations
If you make requests to a DB that has max connection pool, the bottleneck is not the threads
Using thread locals is not a good idea with Virtual Threads
Furthermore probably most of the existing code that deals with threads pools them which again goes against the main idea
I would like to write a verticle that renders graphs using GraphViz. I would like to do it by loading the native (shared) libs into my JVM and calling it via JNI. Now, GraphViz itself is not thread-safe. It is not enough to run each of the multi-instance verticles always in their own thread, I must additionally ensure that each verticle gets its own instance of the native code, or in other words, that every verticle runs in a separate process, each utilizing one of the cores.
Most descriptions of Vert.x say talk only about isolation between threads (not sharing data etc.) I have found nothing about process isolation.
Basically I'm looking for a framework to create a couple of instances of a REST server, all listening on the same socket or with a loadbalancer in front, and not having to write any of the code myself. Sort of what PM2 does for Node.js. Can I do that with Vert.x?
I realize this may be against the spirit of Vert.x, as the core documentation makes clear:
Instead of a single event loop, each Vertx instance maintains several event loops. By default we choose the number based on the number of available cores on the machine, but this can be overridden.
This means a single Vertx process can scale across your server, unlike Node.js.
But as I am using native libraries, which can only be loaded once per JVM, and in my case cannot execute concurrently, and so prevent scaling out to multiple cores, I guess I really do want the Node.js pattern, only in Java.
My requirement is also much simpler than what is described in the documentation of clustered event bus, e. g. the Zookeeper example, because I need no communication between the instances.
I know, using processbuilder we can create a process in java, but how do I create a thread inside a process? Also if i want to create multiple threads inside a process what is the best way of doing that?
Thanks in advance.
I'm looking for thread creation inside of new process.
Once launched, the launching application is not in control of the threads within the new process. Additional threads in the new process will be started as and when that processes code decides to. Only if you are the author of the code for the other process, would you be able to change how and when it spawns new threads.
The difference between processes and threads with respect to Java is that threads run within the same JVM instance, while processes run in different JVM instances.
For example, launching two instances of the same Java application, results in two processes, each running in their own JVM. Despite being the same application, they run independently of each other unless the application includes means of communicating between itself.
Thread creation in a different process would be the responsibility of that's process' Java code. If you are looking to create threads in one JVM under the direction of code in another JVM, you will have to implement an inter-process control mechanism (e.g. socket, control file, RMI, JMX, etc).
Without knowing your reason for spawning threads in a different process, I can only assume that you want some type of isolation. If it is data isolation you seek, consider revising your application's architecture to provide it intrinsically and follow one of the suggestions in Peter Lawrey's comment. A good starting point for ExecutorService is Java 8 Concurrency Tools: Threads and Executors.
So i have a existing Spring library that performs some blocking tasks(exposed as services) that i intend to wrap using Scala Futures to showcase multi processor capabilities. The intention is to get people interested in the Scala/Akka tech stack.
Here is my problem.
Lets say i get two services from the existing Spring library. These services perform different blocking tasks(IO,db operations).
How do i make sure that these tasks(service calls) are carried out across multiple cores ?
For example how do i make use of custom execution contexts?
Do i need one per service call?
How does the execution context(s) / thread pools relate to multi core operations ?
Appreciate any help in this understanding.
You cannot ensure that tasks will be executed on different cores. The workflow for the sample program would be as such.
Write a program that does two things on two different threads (Futures, Java threads, Actors, you name it).
JVM sees that you want two threads so it starts two JVM threads and submits them to the OS process dispatcher (or the other
way round, doesn't matter).
OS decides on which core to execute each thread. Usually, it will try to put threads on different cores to maximize the overall efficiency but it is not guaranteed; you might have a situation that your 10 JVM threads will be executed on one core, although this is extreme.
Rule of the thumb for writing concurrent and seemingly parallel applications is: "Here, take my e.g. 10 threads and TRY to split them among the cores."
There are some tricks, like tuning CPU affinity (low-level, very risky) or spawning a plethora of threads to make sure that they are parallelized (a lot of overhead and work for the GC). However, in general, OS is usually not that overloaded and if you create two e.g. actors, one for db one for network IO they should work well in parallel.
UPDATE:
The global ExecutionContext manages the thread pool. However, you can define your own and submit runnables to it myThreadPool.submit(runnable: Runnable). Have a look at the links provided in the comment.
does anyone know a parallel aplication/benchmark in java that simulates checkpointing? i mean, in my cluster there are running parallel processes in diferent nodes and i want to make them concurrently do some specific action (to have a checkpoint for example). this synchronization how is it achieved?
thanks
If the concurrent threads run in the same VM, just use a CyclicBarrier or a Latch. If they run in different VMs, you can use Terracotta to share a Latch or CyclicBarrier across JVMs, on which all your servers can then synchronize.
Works great, but it needs some work.
You can try hazelcast which offer this functionality but has a lighter touch on the rest of your system than terracotta (more than you need just for this)