I read somewhere sometime that we should have one JVM per core. I also read that Java is not good with multiple cores or CPUs hence scala is preferred.
Recently on a link I read that multiple threads do execute in different core if available.
Questions:
does JVM takes multiple cores/CPUs in consideration and execute each thread in separate core if available?
usage of scala for better usage of cores is somewhat different that just execution in separate core?
if JVM executes each thread in separate core (if available) what is the meaning of one JVM per core, with a single JVM I can utilize all the cores.
if answer to 1 is yes, what is the use of deploy a war/ear in multiple instances of server running in same machine.
Yes. Actually, all JVMs, nowadays, use native threads. So the thread scheduling and distribution across cores is done by the OS, not by the JVM.
Scala executes on the JVM, just as Java. So there is no difference in how threads use cores between Java and Scala
There is no reason to have one JVM per core. You could want that to avoid using two much memory per JVM, or to be able to go for version 1 to version 2 without having any service interruption, but that is a different matter.
No reason linked to multithreading problems. As I said, you might want to have two JVMs to be able to shutdown one without having any service interruption, or to isolate two versions of the same app for different customers for example. But that has nothing to do with multithreading.
Related
Suppose we have a very complex task. I know that if we use one thread then in practice we will use one core, but if I divide the task into threads equal to the number of processor cores does the program necessarily run on all the cores?
Or there is no correlation between the number of threads and cores used and the JVM 'decides'?
Actually, it is typically the Operating System that decides how many cores that a Java application gets to use. The scheduling of native threads to cores is handled by the operating system1. The JVM has little (if any) say thread scheduling.
But yes, an Java application won't necessarily get access to all of the cores. It will depend on what other applications, services, etc on system are doing. Indeed, the OS may provide ways for an administrator to externally limit the number of cores that may be used by a given (Java or not) application, or give one application priority over another.
... or there is no correlation between the number of thread and used core's
There is a correlation, but not one that is particularly useful. A JVM won't (cannot) use more cores than there are native threads in existence (including the JVM's internal and GC threads).
It is also worth noting that the OS (typically) doesn't assign a core to a native thread that is not currently runnable.
Basil Bourque notes in his answer that Project Loom will bring significant improvements to threading in Java, if and when it is incorporated into the standard releases. However, Loom won't alter the fact that the number of physical cores assigned an application JVM at any given time is controlled / limited by the OS.
1 - Or the operating system's hypervisor. Things can get a bit complicated in a cloud computing environment, for instance.
The Answer by Stephen C is true today, where Java threads are typically implemented as native threads provided by the host operating systems. But things change with Project Loom technology being developed for a future version of Java.
Project Loom brings virtual threads (fibers) to the concurrency toolbox of Java. Many virtual threads will be mapped to each of the few platform/kernel threads. A virtual thread when blocked (waiting on a call to slow resources such as file I/O, network I/O, database access, etc.) will be “parked” (set aside) allowing some other virtual thread to execute for a while on the “real” platform/kernel thread.
This parked/unparked switching will be very fast, and take little memory (using a flexible growing/shrinking stack). This makes threads “cheap”, so cheap that you might reasonably be able to run millions of threads at a time.
Returning to your question, the management of these virtual threads will be managed within the JVM rather than the host OS. But underneath our virtual threads, we rely on the same platform/kernel threads used today, and those are ultimately controlled by the host OS rather than the JVM.
By default, when setting up an executor service backed by virtual threads via Executors.newVirtualThreadExecutor we do not specify the number of platform/kernel threads to be used. That s handled by the implementation of your JVM.
Experimental builds of Project Loom are available now, based on early-access Java 17. The Loom team seeks feedback.
Concurrency in Java or some similar languages is achieved through threads or task level parallelism. But under the hood does the hardware or run time also use ILP to achieve best performance.
Little further elaboration: In a multi core processor (say 4 per system) with multiple threads (say 2 per core) ( i.e total 8 threads per system), a java thread is executed in one of the several (8 in this case) processor threads. But if the system determines that all or several other threads are doing nothing but staying ideal, can the hardware or runtime do any legal re-orderings and execute them in other threads on same or other cores and fetch the results back(or in to main memory)
I am bothered about does java implementation allow this or even otherwise it is up to hardware to handle this independently even with out the JVM even knowing anything.
It's a little unclear what you're asking, but I don't think it has much to do with Java.
I think you're talking about (at least) two different things:
"ILP" is generally used to refer to a set of techniques that occur within a single core (such as pipelining and branch prediction), and has little to do with threading or multi-core. These techniques are transparent implementation details of the CPU, and typically not exposed in a way that you (or the runtime) can interact with directly.
Threads are swapped on and off cores by the kernel scheduler if they become blocked (and even if they're not, to ensure fairness).
So i have a existing Spring library that performs some blocking tasks(exposed as services) that i intend to wrap using Scala Futures to showcase multi processor capabilities. The intention is to get people interested in the Scala/Akka tech stack.
Here is my problem.
Lets say i get two services from the existing Spring library. These services perform different blocking tasks(IO,db operations).
How do i make sure that these tasks(service calls) are carried out across multiple cores ?
For example how do i make use of custom execution contexts?
Do i need one per service call?
How does the execution context(s) / thread pools relate to multi core operations ?
Appreciate any help in this understanding.
You cannot ensure that tasks will be executed on different cores. The workflow for the sample program would be as such.
Write a program that does two things on two different threads (Futures, Java threads, Actors, you name it).
JVM sees that you want two threads so it starts two JVM threads and submits them to the OS process dispatcher (or the other
way round, doesn't matter).
OS decides on which core to execute each thread. Usually, it will try to put threads on different cores to maximize the overall efficiency but it is not guaranteed; you might have a situation that your 10 JVM threads will be executed on one core, although this is extreme.
Rule of the thumb for writing concurrent and seemingly parallel applications is: "Here, take my e.g. 10 threads and TRY to split them among the cores."
There are some tricks, like tuning CPU affinity (low-level, very risky) or spawning a plethora of threads to make sure that they are parallelized (a lot of overhead and work for the GC). However, in general, OS is usually not that overloaded and if you create two e.g. actors, one for db one for network IO they should work well in parallel.
UPDATE:
The global ExecutionContext manages the thread pool. However, you can define your own and submit runnables to it myThreadPool.submit(runnable: Runnable). Have a look at the links provided in the comment.
Could someone please provide explanation how Java multi-threaded program (e.g. Tomcat servlet container) is able to use all cores of CPU when JVM is only single process on linux? Is there any good in-depth article that describes the subject in details?
EDIT #1: I'm not looking for advice how to implement multi-threaded program in Java. I'm looking for explanation of how JVM internally manages to use multiple cores on linux/windows while still being single process on the OS.
EDIT #2: The best explanation I managed to find is that Hotspot (Sun/Oracle JVM) implements threads as native threads on Linux using NPTL. So more less each thread in Java is lightweight process (native thread) on Linux. It is clearly visible using ps -eLf command that print outs not only process id (PPID) but also native thread id (LWP).
More details can be also found here:
http://www.velocityreviews.com/forums/t499841-java-5-threads-in-linux.html
Distinguishing between Java threads and OS threads?
EDIT #3: Wikipedia has short but nice entry on NPTL with some further references http://en.wikipedia.org/wiki/Native_POSIX_Thread_Library
The Linux kernel supports threads as first-class citizens. In fact to the kernel a thread isn't much different to a process, except that it shares a address space with another thread/process.
Some old versions of ps even showed a separate process for each thread by default and newer versions can enable this behavior using the -m flag.
The JVM is a single process with many threads. Each thread can be scheduled on a different CPU core. A single process can have many threads.
When Java software running inside the JVM asks for another thread the JVM starts another thread.
That is how the JVM manages to use multiple cores.
If you use the concurrency library and split up your work as much as you can, the JVM should handle the rest.
Take a look at this http://embarcaderos.net/2011/01/23/parallel-processing-and-multi-core-utilization-with-java/
I would start by reading the Concurrency Tutorial.
In particular, it explains the differences (and relationship) between processes and threads.
On the architectures that I'm familiar with, the threads (including JVM-created threads) are managed by the OS. The JVM simply uses the threading facilities provided by the operating system.
A JVM runs in a single process and threads in a JVM share the heap belonging to that process. Then how does JVM make use of multiple cores which provide multiple OS threads for high concurrency?
You can make use of multiple cores using multiple threads. But using a higher number of threads than the number of cores present in a machine can simply be a waste of resources. You can use availableProcessors() to get the number of cores.
In Java 7 there is fork/join framework to make use of multiple cores.
Related Questions:
Is Multi-Threaded algorithm required to make use of Multi-core processors ?
Threads per Processor
Correctly multithreaded quicksort or mergesort algo in Java?
A JVM runs in a single process and threads in a JVM share the heap belonging to that process. Then how does JVM make use of multiple cores which provide multiple OS threads for high concurrency?
Java will utilize the underlying OS threads to do the actual job of executing the code on different CPUs, if running on a multi-CPU machine. When each Java thread is started, it creates an associated OS thread and the OS is responsible for scheduling, etc.. The JVM certain does some management and tracking of the thread and Java language constructs like volatile, synchronized, notify(), wait(), etc. all affect the run status of the OS thread.
A JVM runs in a single process and threads in a JVM share the heap belonging to that process.
JVM doesn't necessary "run in a single process" because even the garbage collector and other JVM code run in different threads and the OS often represents these different threads as different processes. In Linux, for example, the single process you see in the process list is often masquerading a bunch of different thread processes. This is even if you are on a single core machine.
However, you are correct that they all share the same heap space. They actually share the same entire memory space which means code, interned strings, stack space, etc..
Then how does JVM make use of multiple cores which provide multiple OS threads for high concurrency?
Threads get their performance improvements from a couple of reasons. Obviously straight concurrency often makes the program run faster. Being able to do multiple CPU tasks at the same time can (though not always) improve the throughput of the application. You are also able to isolate IO operations to a single thread meaning that other threads can be running while a thread is waiting on IO (read/write to disk/network, etc.).
But in terms of memory, threads get a lot of their performance improvements because of local per-CPU cached memory. When a thread runs on a CPU, the local high speed memory cache for the CPU helps the thread isolate storage requests locally without having to spend the time to read or write to central memory. This is why volatile and synchronized calls include memory synchronization constructs because the cache memory has to be flushed to main memory or invalidated when threads need to coordinate their work or communicate with each other.
Java will benefit from multiple cores, if the OS distribute threads over the available processors. JVM itself do not do anything special to get its threads scheduled evenly across multiple cores. A few things to keep in mind:
While implementing parallel algorithms, it might be better to spawn as many threads as there are cores. (Runtime.getRuntime().availableProcessors()). Not more, not less.
Make use of the facilities provided by the java.util.concurrent package.
Make sure that you have Java Concurrency in Practice in your personal library.
Green threads were replaced by native threads in Java 1.2.