Goal: Execute certain code every once in a while.
Question: In terms of performance, is there a significant difference between:
while(true) {
execute();
Thread.sleep(10 * 1000);
}
and
executor.scheduleWithFixedDelay(runnableWithoutSleep, 0, 10, TimeUnit.SECONDS);
?
Of course, the latter option is more kosher. Yet, I would like to know whether I should embark on an adventure called "Spend a few days refactoring legacy code to say goodbye to Thread.sleep()".
Update:
This code runs in super/mega/hyper high-load environment.
You're dealing with sleep times termed in tens of seconds. The possible savings by changing your sleep option here is likely nanoseconds or microseconds.
I'd prefer the latter style every time, but if you have the former and it's going to cost you a lot to change it, "improving performance" isn't a particularly good justification.
EDIT re: 8000 threads
8000 threads is an awful lot; I might move to the scheduled executor just so that you can control the amount of load put on your system. Your point about varying wakeup times is something to be aware of, although I would argue that the bigger risk is a stampede of threads all sleeping and then waking in close succession and competing for all the system resources.
I would spend the time to throw these all in a fixed thread pool scheduled executor. Only have as many running concurrently as you have available of the most limited resource (for example, # cores, or # IO paths) plus a few to pick up any slop. This will give you good throughput at the expense of latency.
With the Thread.sleep() method it will be very hard to control what is going on, and you will likely lose out on both throughput and latency.
If you need more detailed advice, you'll probably have to describe what you're trying to do in more detail.
Since you haven't mentioned the Java version, so, things might change.
As I recall from the source code of Java, the prime difference that comes is the way things are written internally.
For Sun Java 1.6 if you use the second approach the native code also brings in the wait and notify calls to the system. So, in a way more thread efficient and CPU friendly.
But then again you loose the control and it becomes more unpredictable for your code - consider you want to sleep for 10 seconds.
So, if you want more predictability - surely you can go with option 1.
Also, on a side note, in the legacy systems when you encounter things like this - 80% chances there are now better ways of doing it- but the magic numbers are there for a reason(the rest 20%) so, change it at own risk :)
There are different scenarios,
The Timer creates a queue of tasks that is continually updated. When the Timer is done, it may not be garbage collected immediately. So creating more Timers only adds more objects onto the heap. Thread.sleep() only pauses the thread, so memory overhead would be extremely low
Timer/TimerTask also takes into account the execution time of your task, so it will be a bit more accurate. And it deals better with multithreading issues (such as avoiding deadlocks etc.).
If you thread get exception and gets killed, that is a problem. But TimerTask will take care of it. It will run irrespective of failure in previous run
The advantage of TimerTask is that it expresses your intention much better (i.e. code readability), and it already has the cancel() feature implemented.
Reference is taken from here
You said you are running in a "mega... high-load environment" so if I understand you correctly you have many such threads simultaneously sleeping like your code example. It takes less CPU time to reuse a thread than to kill and create a new one, and the refactoring may allow you to reuse threads.
You can create a thread pool by using a ScheduledThreadPoolExecutor with a corePoolSize greater than 1. Then when you call scheduleWithFixedDelay on that thread pool, if a thread is available it will be reused.
This change may reduce CPU utilization as threads are being reused rather than destroyed and created, but the degree of reduction will depend on the tasks they're doing, the number of threads in the pool, etc. Memory usage will also go down if some of the tasks overlap since there will be less threads sitting idle at once.
Related
Well the title basically says it all, with the small addition that I would really like to know when to use them. And it might be simple enough - I've read the documentation for them both, still can't tell the difference much.
There are answers like this here that basically say:
Yielding also was useful for busy waiting...
I can't agree much with them for the simple reason that ForkJoinPool uses Thread::yield internally and that is a pretty recent addition in the jdk world.
The thing that really bothers me is usages like this in jdk too (StampledLock::tryDecReaderOverflow):
else if ((LockSupport.nextSecondarySeed() & OVERFLOW_YIELD_RATE) == 0)
Thread.yield();
else
Thread.onSpinWait();
return 0L;
So it seems there are cases when one would be preferred over the other. And no, I don't have an actual example where I might need this - the only one I actually used was Thread::onSpinWait because 1) I happened to busy wait 2) the name is pretty much self explanatory that I should have used it in the busy spin.
When blocking a thread, there are a few strategies to choose from: spin, wait() / notify(), or a combination of both. Pure spinning on a variable is a very low latency strategy but it can starve other threads that are contending for CPU time. On the other hand, wait() / notify() will free up the CPU for other threads but can cost thousands of CPU cycles in latency when descheduling/scheduling threads.
So how can we avoid pure spinning as well as the overhead associated with descheduling and scheduling the blocked thread?
Thread.yield() is a hint to the thread scheduler to give up its time slice if another thread with equal or higher priority is ready. This avoids pure spinning but doesn't avoid the overhead of rescheduling the thread.
The latest addition is Thread.onSpinWait() which inserts architecture-specific instructions to hint the processor that a thread is in a spin loop. On x86, this is probably the PAUSE instruction, on aarch64, this is the YIELD instruction.
What's the use of these instructions? In a pure spin loop, the processor will speculatively execute the loop over and over again, filling up the pipeline. When the variable the thread is spinning on finally changes, all that speculative work will be thrown out due to memory order violation. What a waste!
A hint to the processor could prevent the pipeline from speculatively executing the spin loop until prior memory instructions are committed. In the context of SMT (hyperthreading), this is useful as the pipeline will be freed up for other hardware threads.
Lets say I have written a infinite write loop but didn't have statement inside it? Will it create any issue like memory will be full etc or JVM will stop responding after sometime?
Why would you do something like that?
To answer, it wouldn't consume endless memory but Cpu usage could be a pain with really no instruction at all.
At minimum, you should help CPU preemption allowing the Thread to yield:
Thread.yield();
You can read this in Java Api Javadoc:
A hint to the scheduler that the current thread is willing to yield its current use of a processor. The scheduler is free to ignore this hint.
Yield is a heuristic attempt to improve relative progression between threads that would otherwise over-utilise a CPU. Its use should be combined with detailed profiling and benchmarking to ensure that it actually has the desired effect.
It is rarely appropriate to use this method. It may be useful for debugging or testing purposes, where it may help to reproduce bugs due to race conditions. It may also be useful when designing concurrency control constructs such as the ones in the java.util.concurrent.locks package.
An infinite loop might and probably will result in 100% CPU core utilization. Depending what you mean by "write loop" a similar technique is called Busy Waiting or Spinning.
spinning as a time delay technique often produces unpredictable or even inconsistent results unless code is implemented to determine how quickly the processor can execute a "do nothing" loop, or the looping code explicitly checks a real-time clock
You'll certainly keep one hardware thread busy. It wont create any objects, so memory isn't a direct issue as such.
However, the context is important.
If it is a high priority thread, the system may become unresponsive. This is implementation specific. Twenty years ago I wrote an infinite loop that made a Windows NT system unresponsive. (I think this was a TCP proxy and only happened when an IBM 3090 running CICS sent an empty keep alive frame to a 3270 terminal. Good times.)
If the thread is holding any locks, that wont be released.
If the thread does something useful, that useful thing wont happen. For instance if you were to write the loop in a finaliser (and the system only has one finaliser thread), no other object will get finalised and therefore not garbage collected either. The application may behave peculiarly. It'salways fun to run random code on the finaliser thread.
I have a web-app which has a background thread. This thread, which is a Executors.newSingleThreadExecutor is given tasks that are considered low priority: I don't care when they get done. Often the submitted tasks are a type of logging to a remote DB.
I have read many times that one should not change the actual priority of a thread using Thread.setPriority. That being the case, I don't want my background tasks to interfere with more important tasks of my server. What strategies can be used?
Here is what I am thinking about:
I am considering putting Thread.sleep(1) or yeild() at the beginning of the run() method of the background tasks. This would allow other threads to jump ahead if they have something to do at that moment. But, this seems hackish. Suggestions?
I am considering putting Thread.sleep(1) or yeild() at the beginning of the run() method of the background tasks. This would allow other threads to jump ahead if they have something to do at that moment. But, this seems hackish. Suggestions?
thread.yield() is often a no-op in many thread implementations and putting it or a sleep at the start of the run() method will do little or nothing since your background thread hasn't really started running yet.
I don't want my background tasks to interfere with more important tasks of my server. What strategies can be used?
You are correct that using thread priorities often have little to no effect on the number of cycles the thread will be given. This depends a lot of your architecture however so I'd certainly try them to see if it helps. If you are worried about some high performance operation taking cycles away from other more important threads then about your only recourse is to pepper the loops and other key places in your algorithm with Thread.sleep(...) calls. It may be hackish indeed but it can be effective.
The tricky parts is where to put the calls, what millis sleep should be used, etc.. That's going to take some testing and iteration to optimize the placement of the sleeps. Also, if you are calling 3rd party libraries or something you might not be able to put the sleep calls at the core places anyway.
If it doesn't work or you don't have access to the right pain points then you may have not choice except offloading your background processing to a remote system for processing.
Since the executor is single-threaded it can only occupy one core. As long as your server has N cores those background tasks will never take up more than 1/N of the server load, assuming they do not spawn new threads or dispatch to other thread pools on their own.
Alternatively you can schedule all tasks (low and high priority) through one thread pool configured with a priority queue and decorate each background task so that it has a low priority. I.e. you can move the scheduling from the kernel to userspace.
ExecutorService.newFixedThreadPool() Is there any real time scenarios where we prefer to have a fixed set of active threads even when there is nothing to process?
In practice, having a fixed number of threads is always better than spawning a new thread every time a task has to be processed.
Threads are expensive to create and maintain, and not being able to create the number of active threads in your application, can end up actually harming the performance. Fixed thread pools reuse already created threads and this removes the thread creation overhead.
When you keep a fixed number of threads, you can predict your memory and CPU usage better, at least IMHO.
Of course, there is no recipe that fits all use cases and, before choosing what paradigm is best for your particular situation, you should do rigorous testing and measurements. Experimenting with different configurations will give you a better understanding and point you to the best solution.
I have a multi-threaded application which creates hundreds of threads on the fly. When the JVM has less memory available than necessary to create the next Thread, it's unable to create more threads. Every thread lives for 1-3 minutes. Is there a way, if I create a thread and don't start it, the application can be made to automatically start it when it has resources, and otherwise wait until existing threads die?
You're responsible for checking your available memory before allocating more resources, if you're running close to your limit. One way to do this is to use the MemoryUsage class, or use one of:
Runtime.getRuntime().totalMemory()
Runtime.getRuntime().freeMemory()
...to see how much memory is available. To figure out how much is used, of course, you just subtract total from free. Then, in your app, simply set a MAX_MEMORY_USAGE value that, when your app has used that amount or more memory, it stops creating more threads until the amount of used memory has dropped back below this threshold. This way you're always running with the maximum number of threads, and not exceeding memory available.
Finally, instead of trying to create threads without starting them (because once you've created the Thread object, you're already taking up the memory), simply do one of the following:
Keep a queue of things that need to be done, and create a new thread for those things as memory becomes available
Use a "thread pool", let's say a max of 128 threads, as all your "workers". When a worker thread is done with a job, it simply checks the pending work queue to see if anything is waiting to be done, and if so, it removes that job from the queue and starts work.
I ran into a similar issue recently and I used the NotifyingBlockingThreadPoolExecutor solution described at this site:
http://today.java.net/pub/a/today/2008/10/23/creating-a-notifying-blocking-thread-pool-executor.html
The basic idea is that this NotifyingBlockingThreadPoolExecutor will execute tasks in parallel like the ThreadPoolExecutor, but if you try to add a task and there are no threads available, it will wait. It allowed me to keep the code with the simple "create all the tasks I need as soon as I need them" approach while avoiding huge overhead of waiting tasks instantiated all at once.
It's unclear from your question, but if you're using straight threads instead of Executors and Runnables, you should be learning about java.util.concurrent package and using that instead: http://docs.oracle.com/javase/tutorial/essential/concurrency/executors.html
Just write code to do exactly what you want. Your question describes a recipe for a solution, just implement that recipe. Also, you should give serious thought to re-architecting. You only need a thread for things you want to do concurrently and you can't usefully do hundreds of things concurrently.
This is an alternative, lower level solution Then the above mentioed NotifyingBlocking executor - it is probably not as ideal but will be simple to implement
If you want alot of threads on standby, then you ultimately need a mechanism for them to know when its okay to "come to life". This sounds like a case for semaphores.
Make sure that each thread allocates no unnecessary memory before it starts working. Then implement as follows :
1) create n threads on startup of the application, stored in a queue. You can Base this n on the result of Runtime.getMemory(...), rather than hard coding it.
2) also, creat a semaphore with n-k permits. Again, base this onthe amount of memory available.
3) now, have each of n-k threads periodically check if the semaphore has permits, calling Thread.sleep(...) in between checks, for example.
4) if a thread notices a permit, then update the semaphore, and acquire the permit.
If this satisfies your needs, you can go on to manage your threads using a more sophisticated polling or wait/lock mechanism later.