I have a Java application where I am using PooledExecuter from oswego.
I strongly suspect that there is a thread contention in my application as irrespective of using PooledExecuter, requests are taking time = roughly no of requests * time for one request.
I want to gather evidence that there is definitely a thread contention.
Is there any way I can set some JVM parameters which show me what different threads are doing or any other way that can be useful to detect thread contention.
From your description it sounds like you have a resource which is single threaded and you code isn't able to use multiple threads efficiently. You should be able to see this by taking a thread dump while this is happening (a few times) You should see one thread doing "real" work, and all the other thread in the pool waiting for something or idle.
I don't know of jvm options that can tell you this, I would attach a profiler to it and see if the threads are contending (blocked / waiting) lots, and then see which locks they are contending for.
Related
I'm having a game server based on Java.
1 user need to use 2 threads to send and receive data. But whenever the thread come to 200-300 threads, the function where execute data doesnt work anymore. CPU, RAM of the server is not full, just around 15-20%.
I tried to use "garbage collector" when user disconnect, but this still happen.
Thanks for helping. Sorry with my bad English.
Your service, should ideally never be creating "too many threads".
Opt for a thread-pool using ExecutorService.
Number of threads you want to create a pool with, depends upon the kind of underlying task you have.
From a general practice:
1: For a CPU Intensive task your number of threads should be equal to
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
2: For an IO Intensive task you can create more number of threads than the number of available processors as most of your threads will be waiting if the IO task is taking long.
300 Java Threads does not sound like too much (compare with default settings for application servers like Wildfly). If your application is getting stuck but neither CPU nor RAM are the bottleneck, maybe try to figure out what is happening. You may be facing threads waiting for each other to finish as well.
Thus I recommend to look at the thread dump to see where the threads might be stuck. Check out Generate a Java thread dump without restarting.
Try checking out java ExecutorService to create a thread pool with a fixed number of threads.
There is a strange thing happening on our production box.
Code functionality:
A UI servlet takes a monitor lock on the document object which is being actioned upon by the user and performs some computation on it. The monitor lock is acquired to prevent the same document object from getting modified concurrently by multiple users simultaneously.
Issue Observed in Prod:
Few user actions are getting timed out.
Log Analysis:
The thread corresponding to the timed out user actions is printing all logs prior to acquiring the monitor lock on the document object. Then there is a gap of over 1 hour where the thread is not surfacing up in the logs and then it suddenly becomes alive and does the computation and attempts to send back a response which obviously errors out as the HTTP request has already timed out.
We have checked the logs and code and can confirm that there is no other thread which had acquired the monitor lock on that particular document object. So the lock was uncontested at the point in question.
What could be the possible issue? Is it just that the thread was put into a Runnable state on encountering a synchronized block and for the next 60-80 mins, the CPU never got a chance to run this particular runnable thread?
Ensure the application code is not messing around with thread priority via Thread.setPriority() method or the like. If you're using an IDE like IntelliJ and the Java sources are available, and assuming you can run the application and relevant flow locally in your development machine, you can put a breakpoint in Thread.setPriority() to see if anywhere it is getting invoked. This is an excerpt from Java Concurrency in Practice, Goetz 2006, regarding how unpredictable behavior can be if you try to set Thread priority manually:
10.3.1. Starvation
Starvation occurs when a thread is perpetually denied access to resources it needs in order to make progress; the most
commonly starved resource is CPU cycles. Starvation in Java applications can be caused by inappropriate use of thread
priorities. It can also be caused by executing nonterminating constructs (infinite loops or resource waits that do not
terminate) with a lock held, since other threads that need that lock will never be able to acquire it.
The thread priorities defined in the Thread API are merely scheduling hints. The Thread API defines ten priority levels
that the JVM can map to operating system scheduling priorities as it sees fit. This mapping is platform􀍲specific, so two
Java priorities can map to the same OS priority on one system and different OS priorities on another. Some operating
systems have fewer than ten priority levels, in which case multiple Java priorities map to the same OS priority.
Operating system schedulers go to great lengths to provide scheduling fairness and liveness beyond that required by the
Java Language Specification. In most Java applications, all application threads have the same priority, Thread.
NORM_PRIORITY. The thread priority mechanism is a blunt instrument, and it's not always obvious what effect changing
priorities will have; boosting a thread's priority might do nothing or might always cause one thread to be scheduled in
preference to the other, causing starvation.
It is generally wise to resist the temptation to tweak thread priorities. As soon as you start modifying priorities, the
behavior of your application becomes platform specific and you introduce the risk of starvation. You can often spot a
program that is trying to recover from priority tweaking or other responsiveness problems by the presence of
Thread.sleep or Thread.yield calls in odd places, in an attempt to give more time to lower priority threads.[5]
In my Java web app I have a method which ends out about 200 emails. Because of email server delay the whole process takes about 7 minutes. This bulk email sending has to take place as the result of user action. I of course don't want the user to have to wait that long before they are forwarded to the next, not mention that Apache times out anyway, so I am attempting to implement FutureTask to get the process to run in a separate thread while proceed with the rest of the code like this:
Some code;
Runnable r = (Runnable)new sendEmails(ids);
FutureTask task = new FutureTask(r, null);
Thread t = new Thread(task);
t.start();
Some more code;
The app, however, still waits for the FutureTask to finish before proceeding. I am open to the idea that this also not the best way to run some code on the side in another thread while continuing with the rest of the script. Are there better ways/How do I make this one work?
It looks like you are spinning up 200+ threads in a for loop. That will place a high burden on the machine, and due to the size of each stack that is allocated with each thread it will not take too many threads before the JVM runs out of memory, initially causing much GC and JVM locking up and then potentially under high enough load, a crash.
Sadly this may or may not explain why your code is waiting for the FutureTasks to complete. It may only appear to be waiting to due thrashing by creating/scheduling so many threads; but then again it may not. There could very well be something else synchronizing your code that has been cut out of the snippet above.
A way for you to find if there is a tricksy synchronisation hiding somewhere would be to hit ctrl-break while running the code (assuming that you are running from a command line, intellij/eclipse both have a stack dump icon that is handy). This will cause a stack dump for every thread in the system to appear. By doing this you will be able to find the user thread that is waiting for the future tasks to complete, and it will say which monitor it is waiting on. If it is not waiting, then you have a different problem. For example the system thrashes creating so many threads in short order that it appears to lock up or some such for a short period of time.
But first I would avoid the excessive Thread creation part, as that could be masking the issue. I suggest using code similar to the following:
ExecutorService scheduler = Executors.newCachedThreadPool()
scheduler.submit( task )
Can any one guide me with example about Thread and ThreadPool what is difference between them? which is best to use...? what are the drawback on its
Since a thread can only run once, you'd have to use a thread per task. However, creating and starting threads is somewhat expensive and can lead to a situation where too many threads are waiting for execution (don't remember the exact name for this right now) - which further reduces performance.
A thread pool is - as the name suggests - a pool of worker threads which are always running. Those threads then normally take tasks from a list, execute them, then try to take the next task. If there's no task, the thread will wait.
Using a thread pool has several advantages:
you don't have to create a thread per task
you normally have the optimal number of threads for your system (depending on the JVM too)
you can concentrate on writing tasks and use the thread pool to manage the infrastructure
Edit: Here are some quite good articles on concurrency in general: Sutter's Mill, look at the bottom for more links. Although they're primarily written for C/C++ the general concepts are the same, since it also describes the interdependence between concurrency solutions and hardware. A good article to understand concurrency performance issues is this article on drdobbs.com.
A thread pool is a collection of threads which are assigned to perform uniformed tasks.
The advantages of using thread pool pattern is that you can define how many threads is allowed to execute simultaneously. This is to avoid server crashing due to high CPU load or out of memory condition, e.g. the server's hardware capacity can support up to 100 requests per second only.
Database pooling has the similar concept with thread pool.
This pattern is widely used in most of the back-end servers' application process.
While a thread, is a unit which execute a task.
When writing a multithread internet server in java, the main-thread starts new
ones to serve incoming requests in parallel.
Is any problem if the main-thread does not wait ( with .join()) for them?
(It is obviously absurd create a new thread and then, wait for it).
I know that, in a practical situation, you should (or "you must"?) implement a pool
of threads to "re-use" them for new requests when they become idle.
But for small applications, should we use a pool of threads?
You don't need to wait for threads.
They can either complete running on their own (if they've been spawned to perform one particular task), or run indefinitely (e.g. in a server-type environment).
They should handle interrupts and respond to shutdown requests, however. See this article on how to do this correctly.
If you need a set of threads I would use a pool and executor methods since they'll look after thread resource management for you. If you're writing a multi-threaded network server then I would investigating using (say) a servlet container or a framework such as Mina.
The only problem in your approach is that it does not scale well beyond a certain request rate. If the requests are coming in faster than your server is able to handle them, the number of threads will rise continuously. As each thread adds some overhead and uses CPU time, the time for handling each request will get longer, so the problem will get worse (because the number of threads rises even faster). Eventually no request will be able to get handled anymore because all of the CPU time is wasted with overhead. Probably your application will crash.
The alternative is to use a ThreadPool with a fixed upper bound of threads (which depends on the power of the hardware). If there are more requests than the threads are able to handle, some requests will have to wait too long in the request queue, and will fail due to a timeout. But the application will still be able to handle the rest of the incoming requests.
Fortunately the Java API already provides a nice and flexible ThreadPool implementation, see ThreadPoolExecutor. Using this is probably even easier than implementing everything with your original approach, so no reason not to use it.
Thread.join() lets you wait for the Thread to end, which is mostly contrary to what you want when starting a new Thread. At all, you start the new thread to do stuff in parallel to the original Thread.
Only if you really need to wait for the spawned thread to finish, you should join() it.
You should wait for your threads if you need their results or need to do some cleanup which is only possible after all of them are dead, otherwise not.
For the Thread-Pool: I would use it whenever you have some non-fixed number of tasks to run, i.e. if the number depends on the input.
I would like to collect the main ideas of this interesting (for me) question.
I can't totally agree with "you
don't need to wait for threads".
Only in the sense that if you don't
join a thread (and don't have a
pointer to it) once the thread is
done, its resources are freed
(right? I'm not sure).
The use of a thread pool is only
necessary to avoid the overhead of
thread creation, because ...
You can limit the number of parallel
running threads by accounting, with shared variables (and without a thread pool), how many of then
were started but not yet finished.