I was trying to read the implementation of Synchronous Queue
It is not so straightforward for me. It seems to be using a linked list where each node is associated with a thread.
And the core part uses a spin loop waiting for tasks to be placed in the queue.
I was wondering why is a spin loop being used instead of something like wait/notify?
Now this way one of the cores is gone due to this constant spin loop, right?
I am trying to understand this point and get a rough understanding of the design of the Synchronous Queue
UPDATE
What is also troubling me is how the waiter threads start/stop.
The point of the SynchronousQueue is to synchronize something which is usually quite asynchronous - one thread placing an item into the queue while another tries to take from it.
The SynchronousQueue is actually not a queue at all. It has no capacity, no internal storage. It only allows taking from the queue when another process is currently trying to put in the queue.
Example:
Process A tries to put in the queue. This blocks for now.
Process B tries to take from the queue. Since someone is trying to put, the item is transferred from A to B, and both are unblocked.
Process B tries to take from the queue, but no one tries to put. So B is now blocked.
Process A now wants to put an item. Now the item is transferred over to B, and A and B are no longer blocked.
About the blocking:
The Sun/Oracle JRE implementation does use polling instead of a wait/notify pattern if you do a timed operation (like "try to take for 1 second"). This makes sense: it periodically retries until the time is up. When you do a non-timed operation (like "take, no matter how long it takes" it does use park, which wakes again if the situation has changed. In neither situation would one of your cores be constantly busy spinning a loop. The for (;;) means "retry indefinately" in this case, it does not mean "constant spinning".
Related
lets say we have two threads with are connected by a ConcurrentLinkedQueue. What I want is something like a handler on the queue so that one thread knows when the other queue has added soemthing to the queue and to poll it. Is that possible?
Normally a ConcurrentLinkedQueue is used when there is at least one producer on a thread, and at least one consumer on a different thread.
The consumer will process the element as soon as they are available, to do so the read operation on the queue blocks, sometimes for a limited amount of time.
Depending on the application you can have a single producer and many consumer, or viceversa.
Blocking achieves exactly your requirement (the consumer thread knows when an element is inserted).
The fact that the consumer thread blocks is not a problem unless is your main process thread or unless you are planning to build several hundred concurrent consumers.
So, Queue#take() or Queue#poll(long timeout,TimeUnit unit) is your friend here, if you just run it on dedicated Thread.
In Java, is there a way for a thread to know it has been "interleaved"?
I would like to send a certain update to my clients (who are handled by individual threads) after their thread has been interleaved by another thread.
In case my use of the term "interleaved" is incorrect, I'm referring to the process where the processor stops running one thread and moves to another one.
So when the processor eventually returns to my thread, I would like a certain update to be sent to my client via the thread.
Apparently there is no simple way to detect that a thread has been interleaved.
Instead, I decided to use an atomic integer to track the amount of updates that were executed by all threads.
I then changed the code within my threads to monitor the amount of changes that had been done (since last notifying the client) and, once a certain threshold had been exceeded, I updated the client.
I have a multi-threaded application which creates hundreds of threads on the fly. When the JVM has less memory available than necessary to create the next Thread, it's unable to create more threads. Every thread lives for 1-3 minutes. Is there a way, if I create a thread and don't start it, the application can be made to automatically start it when it has resources, and otherwise wait until existing threads die?
You're responsible for checking your available memory before allocating more resources, if you're running close to your limit. One way to do this is to use the MemoryUsage class, or use one of:
Runtime.getRuntime().totalMemory()
Runtime.getRuntime().freeMemory()
...to see how much memory is available. To figure out how much is used, of course, you just subtract total from free. Then, in your app, simply set a MAX_MEMORY_USAGE value that, when your app has used that amount or more memory, it stops creating more threads until the amount of used memory has dropped back below this threshold. This way you're always running with the maximum number of threads, and not exceeding memory available.
Finally, instead of trying to create threads without starting them (because once you've created the Thread object, you're already taking up the memory), simply do one of the following:
Keep a queue of things that need to be done, and create a new thread for those things as memory becomes available
Use a "thread pool", let's say a max of 128 threads, as all your "workers". When a worker thread is done with a job, it simply checks the pending work queue to see if anything is waiting to be done, and if so, it removes that job from the queue and starts work.
I ran into a similar issue recently and I used the NotifyingBlockingThreadPoolExecutor solution described at this site:
http://today.java.net/pub/a/today/2008/10/23/creating-a-notifying-blocking-thread-pool-executor.html
The basic idea is that this NotifyingBlockingThreadPoolExecutor will execute tasks in parallel like the ThreadPoolExecutor, but if you try to add a task and there are no threads available, it will wait. It allowed me to keep the code with the simple "create all the tasks I need as soon as I need them" approach while avoiding huge overhead of waiting tasks instantiated all at once.
It's unclear from your question, but if you're using straight threads instead of Executors and Runnables, you should be learning about java.util.concurrent package and using that instead: http://docs.oracle.com/javase/tutorial/essential/concurrency/executors.html
Just write code to do exactly what you want. Your question describes a recipe for a solution, just implement that recipe. Also, you should give serious thought to re-architecting. You only need a thread for things you want to do concurrently and you can't usefully do hundreds of things concurrently.
This is an alternative, lower level solution Then the above mentioed NotifyingBlocking executor - it is probably not as ideal but will be simple to implement
If you want alot of threads on standby, then you ultimately need a mechanism for them to know when its okay to "come to life". This sounds like a case for semaphores.
Make sure that each thread allocates no unnecessary memory before it starts working. Then implement as follows :
1) create n threads on startup of the application, stored in a queue. You can Base this n on the result of Runtime.getMemory(...), rather than hard coding it.
2) also, creat a semaphore with n-k permits. Again, base this onthe amount of memory available.
3) now, have each of n-k threads periodically check if the semaphore has permits, calling Thread.sleep(...) in between checks, for example.
4) if a thread notices a permit, then update the semaphore, and acquire the permit.
If this satisfies your needs, you can go on to manage your threads using a more sophisticated polling or wait/lock mechanism later.
I have a java concurrency problem, it goes like this: There is a Servlet (serv1) that stores objects (representing messages) into a database. There is also a Thread (t1) that looks (in the database) for unsent messages and delivers them. t1 runs every 5 minutes so, for efficiency purposes, serv1 notifies t1 every time it stores a message.
The question is: How the notification process is going to behave on a highly concurred scenario where serv1 is receiving an extremely high amount of requests and thus t1 is being notified so much that it’d simulate a "while (true)"?.
Another question: How does the notification process will behave if serv1 wants to notify t1 but t1 is already awake/running?
Thanks in advance!!
I don't think this is an issue #Wilmer. I suspect that the notification itself is relatively cheap compared to the cost of consuming and processing your messages. If you are spinning consuming then messages then removing the notifications is not going to help the process and you will have to block your serv1 thread somehow or offload the jobs to run later.
In terms of notifications, if no one is waiting then the notify() is effectively a no-op. This is why it is important to check to see if there is anything to process before waiting on the signal -- all in a synchronized block. It is best practice to also loop around the wait and check again when we are notified. See this race condition.
In general, this is a very common practice that is used in virtually all producer/consumer thread models that I have seen. The alternative is not to handle the square wave traffic changes. Either your consumer (t1) is waiting too long and the buffers fill up or it is spinning too much and is consumer too much CPU checking the database.
Another thing to consider is to not use the database but to put the objects into a BlockingQueue for t1 to consume directly. If you need to store them in the database then put the IDs of the objects in the queue for t1 to consume. The thread will still need to poll the database on startup but you will save the polls later in the process.
Why are you notifying t1 at all? Why doesn't T1 on it's 5 minute sweep query the database and process all of the pending messages? Then you don't need a notification at all, you simply use the database.
In order to use Object o = new Object(); o.notify() correctly, it has to be done after obtaining that object's monitor (becoming its owner; AKA synchronize on it). Moreover, the awakened thread that waits upon that monitor will have to wait yet again for the notifying thread to release that monitor and try to obtain it. Only then it shall continue processing.
So, when t1 will be awakened, it will actually fight all other serv1 threads for becoming owner of the monitor. It might obtain the monitor, thus stalling all serv1 threads (not good). It might loose constantly to serv1's threads and not process the accumulating messages in the database (just as bad, I guess).
What you should do, is let the producers (serv1 threads) work asynchronously with the consumer (t1). t1 should continue to run every X minutes (or seconds) and process all the messages altogether.
Another option, if you want to keep the thread in low activity: configure several consumer threads (t1, t2, t3... etc.). You can use a Executors.newFixedThreadPool(int nThreads, ThreadFactory threadFactory) for this purpose.
I want to control the amount of time that each thread uses.
One thread does some processing and another processes data in the database, but the insertion is slower than processing because of the amount of generated data. I want to give more processor time to insert that data.
Is it possible do this with threads? At the moment, I'm putting a sleep in the thread doing the processing, but the time of insertion changes according to the machine. Is there another way I can do this? Is the way involving the use of thread synchronization inside my program?
You can increase the priority of a thread using Thread.setPriority(...) but this is not ideal.
Perhaps you can use some form of blocking queue from the java.util.concurrent package to make one Thread wait while another Thread is doing something. For example, a SynchronousQueue can be used to send a message from one Thread to another Thread that it can now do something.
Another approach is to use Runnables instead of Threads, and submit the Runnables to an Executor, such as ThreadPoolExecutor. This executor will have the role of making sure Runnables are using a fair amount of time.
The first thing to mention is that thread priority doesn't per se mean "share of the CPU". There seems to be a lot of confusion about what thread priority actually means, partly because it actually means different things under different OS's. If you're working in Linux, it actually does mean something close to relative share of CPU. But under Windows, it definitely doesn't. So in case it's of any help, you may firstly want to look at some information I compiled a little while ago about thread priorities in Java, which explains what Thread Priorities Actually Mean on different systems.
The general answer to your question is that if you want a thread to take a particular share of CPU, it's better to implicitly do that programmatically: periodically, for each "chunk" of processing, measure how much time elapsed (or how much CPU was used-- they're not strictly speaking the same thing), then sleep an appropriate amount of time so that the processing/sleep ratio comes to roughly the % of processing time you intended.
However, I'm not sure that will actually help your task here.
As I understand, basically you have an insertion task which is the rate determining step. Under average circumstances, it's unlikely that the system is "deliberately dedicating less CPU than it can or needs to" to the thread running that insertion.
So there's probably more mileage in looking at that insertion task and seeing if programmatically you can change how that insertion task functions. For example: can you insert in larger batches? if the insertion process really is CPU bound for some reason (which I am suspicious of), can you multi-thread it? why does your application actually care about waiting for the insertion to finish, and can you change that dependency?
If the insertion is to a standard DB system, I wonder if that insertion is terribly CPU bound anyway?
One way would be to set the priority of the processing thread to be lower than the other. But beware this is not recommended as it wont keep your code platform independent. (DIfferent thread priorities behave differently on different platforms).
Another way would be to use a service where database thread would keep sending messages about its current status (probably some flag "aboutToOver").
Or use synchronization say a binary semaphore. When the database thread is working, the other thread would be blocked and hence db thread would be using all the resources. But again processing thread would be blocked in the mean time. Actually this will be the best solution as the processign thread can perform say 3-4 tasks and then will get blocked by semaphore till later when it can again get up and do task