I'm building a GUI'd application with javaFX that supports a long-running CPU intensive operation, something like Prime95 or Orthos.
One of the problems I've run into is trying to get counters to increment nicely. If you think about an ElapsedTime field with an incrementing counter with millisecond resolution, what I need is a job on the UI thread to call elapsedTimeTextField.setText("00:00:00.001") to happen 1ms before a corresponding call elapsedTimeTextField.setText("00:00:00.002"). I also need to let the UI thread do more important jobs between those two calls.
Structuring code to do this has been tedious, and has resulted in a number of our controller classes creating threads that simply loop on code similar to:
Thread worker = new Thread(this::doUpdates);
worker.start();
//...
private void doUpdates(){
while(true){
String computedTime = computeTimeToDisplay();
runLaterOnUI(() -> textField.setText(computedTime));
sleep(DUTY_CYCLE_DOWNTIME);
}
}
While this does the job, its unfavorable because:
It's difficult to unit test: from a testing environment you either have to modify this code to give some kind of signal when it completes its first pass, (typically a count-down-latch) or you have to do silly non-deterministic & arbitrary sleep()s
It doesn't have any kind of backoff: if the UI thread is flooded with jobs this code is going to exacerbate the problem. Some kind of requeueing scheme, whereby the downtime takes into account the latency of the job and some kind of hard-coded sleep is preferable since it means that if the UI job is flooded we're not asking it to do work unduly.
It doesn't have centralized exception handling short of the threads default handler. This means that if an exception is raised in the computeTimeToDisplay() method (or for that fact, in the runLaterOnUI call or the sleep() call) the text field will no longer be updated.
I have addressed each of these concerns reasonably well individually, but I don't have any obvious and reusable idiom for tackling these three problems.
I suspect that the Future, Task, Executor, ServiceExecutor, etc classes (the classes in the java.util.concurrent package that aren't a lock or a collection) can help me to this goal, but I'm not sure how to use them.
Can somebody suggest some documentation to read and some idioms to follow that will help me in pursuit of these goals? Is there an agreed on idiom --that doesn't involve anonymous classes and contains minimal boiler-plate-- for this kind of concurrent-job?
I recommend using a ScheduledThreadPoolExecutor with a core pool size of 1 and optionally with a thread priority of Thread.NORM_PRIORITY + 1 (use a ThreadFactoryBuilder to create a ThreadFactory with higher than standard priority) for the UI thread - this will let you schedule tasks such as the counter increment using ScheduledThreadPoolExecutor#scheduleAtFixedRate. Don't execute anything other than UI tasks on this executor - execute your CPU tasks on a separate ThreadPoolExecutor with standard priority; if you have e.g. 16 logical cores then create a ThreadPoolExecutor with 16 core threads to make full use of your computer when the UI thread is idle, and let the virtual machine take care of ensuring that the UI thread executes its jobs when it's supposed to.
You question is multi-faceted and I am not going to pretend that I understand all of it. This answer will address only one part of the question.
It doesn't have any kind of backoff: if the UI thread is flooded with jobs this code is going to exacerbate the problem. Some kind of requeueing scheme, whereby the downtime takes into account the latency of the job and some kind of hard-coded sleep is preferable since it means that if the UI job is flooded we're not asking it to do work unduly.
The in-built java.util.concurrent classes such as Task, Service and ScheduledService include facilities to send message updates from a non-UI thread to a UI thread in way that does not flood the UI thread. You could use those classes directly (which would seem advisable, though perhaps that perception is naive of me as I don't fully understand your requirements). Or you can implement a similar custom facility in your code if you aren't using java.util.concurrent directly.
Here is the relevant code from the Task implementation:
/**
* Used to send message updates in a thread-safe manner from the subclass
* to the FX application thread. AtomicReference is used so as to coalesce
* updates such that we don't flood the event queue.
*/
private AtomicReference<String> messageUpdate = new AtomicReference<>();
private final StringProperty message = new SimpleStringProperty(this, "message", "");
/**
* Updates the <code>message</code> property. Calls to updateMessage
* are coalesced and run later on the FX application thread, so calls
* to updateMessage, even from the FX Application thread, may not
* necessarily result in immediate updates to this property, and
* intermediate message values may be coalesced to save on event
* notifications.
* <p>
* <em>This method is safe to be called from any thread.</em>
* </p>
*
* #param message the new message
*/
protected void updateMessage(String message) {
if (isFxApplicationThread()) {
this.message.set(message);
} else {
// As with the workDone, it might be that the background thread
// will update this message quite frequently, and we need
// to throttle the updates so as not to completely clobber
// the event dispatching system.
if (messageUpdate.getAndSet(message) == null) {
runLater(new Runnable() {
#Override public void run() {
final String message = messageUpdate.getAndSet(null);
Task.this.message.set(message);
}
});
}
}
}
The code works by ensuring that a runLater call is only made if the UI has processed (i.e. rendered) the last update.
Internally the JavaFX 8 system runs on a pulse system. Unless there is an unusually long time consuming operation on the UI thread or general system slowdown, each pulse will usually occur 60 times a second, or approximately every 16-17 milliseconds.
You mention the following:
what I need is a job on the UI thread to call elapsedTimeTextField.setText("00:00:00.001") to happen 1ms before a corresponding call elapsedTimeTextField.setText("00:00:00.002").
However, you can see from the JavaFX architecture description that updating the text more than 60 times a second is pointless as the additional updates will never be rendered. The sample code above from Task, takes care of this by ensuring that a UI update request is only ever issued at a time that the UI update thread can actually reflect the new value in the UI.
Some General Advice
This is just advice, it does not directly solve your problem, take it for what you will, some of it might not even be particularly relevant to your situation or problem.
Make clear the problem you are trying to solve in your questions. That is sometimes more important than a description of the symptoms you are experiencing and trying to resolve. It also helps prevent XY questions.
Be clear from the start on what you are actually doing to solve the problem. An mcve can sometimes help here.
For example, your initial problem statement does not state that you may have 10,000 controllers or provide code for what you term to be a controller. There is not much information on the expected length of time for tasks, what the UI display representing task progress and result is, why millisecond accuracy level might be important to display, if task results need to coalesced, if the tasks can be split and run concurrently, how many threads you are using, etc.
Don't try to develop your own higher level concurrency tools from primitives like ConcurrentLinkedQueue.
For your backend segmented work jobs, use high level concurrency utilities from Java SE, such as Executors, ForkJoin and BlockingQueue.
Orchestrate and synchronize the output of backend jobs with your UI using JavaFX concurrency utilities such as Task.
Know that the high level concurrency utilities and JavaFX concurrency tools can be used in unison, like in this example. I.e., the choice of concurrency tools doesn't need to be an either/or situation.
Extensive use of immutable objects can be a lifesaver in concurrent development.
If you will be doing a lot of concurrent development, take time for detailed study of high quality resources on concurrent programming such as Concurrency in Practice.
Concurrency in general is often simply hard to get right.
Related
I get that with threads being nonblocking, we don't need to have Thread sprawl depending on N concurrent requests, but rather we put our tasks in a single event loop in our reactive web programming pattern.
Yes, that can help, but since the event loop is a queue, what if the first task to be processed blocks forever? Then the event loop will never progress and thus end of responses and processing other than queueing more tasks. Yes, timeouts are probably possible, but I can't wrap my head around how the event loop can be a good solution.
Say you have 3 tasks that take 3 seconds to wait for IO and run each executions and they got submitted to the event queue. Then they will still take 9 seconds to be able to be processed and also to execute once IO resolved. In the case of making threads that block, this would have resolved in 3 seconds since they run concurrently.
Where I can see a benefit is if the event loop is not really a queue and upon signal that a task is ready to be processed, it dispatches that task to be processed. In that case though, this would mean that order of task execution is not maintained and also each task has to still be running a thread in order to be able to tell when IO is resolved.
Maybe I am not understanding the event loop and thread handling correctly. Can someone correct me please because it seems like this Reactor pattern seems to make things possibly worse.
Lastly, upon X requests in Spring Reactor, does only 1 thread get created to run handlers instead of the traditional X threads? In that case, if someone accidently wrote blocking code, doesnt that mean each subsequent requests get queued?
It is not a good idea to use the event loop for long running tasks. This is considered an anti-pattern. Usually it is merely used for quickly picking up imminent events, but not actually doing the work associated with these events if the work would block the event loop noticeably. You would want to use a separate thread pool for executing long running tasks. So the event loop would usually only initiate work using asynchronous and hence non-blocking structures (or actually doing the work only if it can be done very quickly) and pass the heavier and possibly blocking tasks to a separate thread pool (for CPU intensive computations) or to the operating system (such as data buffers to be sent over the network).
Also, don't be fooled by the fact that only one thread is dealing with the events, it is very fast and is usually enough for even demanding applications. Platforms like NodeJS or frameworks like Netty (used in Akka, Play framework, Apache Cassandra, etc.) are using an event loop at their heart with great success. One should just be aware of the fact, that performing blocking operations inside the event loop is generally a bad idea.
Please have a look at some of these posts for more information:
The reactor pattern and non blocking IO
Unix Network Programming
Kotlin Webflux
Slightly off topic but still a very prominent example: Don't Block the Event Loop (NodeJS)
I have a web-app which has a background thread. This thread, which is a Executors.newSingleThreadExecutor is given tasks that are considered low priority: I don't care when they get done. Often the submitted tasks are a type of logging to a remote DB.
I have read many times that one should not change the actual priority of a thread using Thread.setPriority. That being the case, I don't want my background tasks to interfere with more important tasks of my server. What strategies can be used?
Here is what I am thinking about:
I am considering putting Thread.sleep(1) or yeild() at the beginning of the run() method of the background tasks. This would allow other threads to jump ahead if they have something to do at that moment. But, this seems hackish. Suggestions?
I am considering putting Thread.sleep(1) or yeild() at the beginning of the run() method of the background tasks. This would allow other threads to jump ahead if they have something to do at that moment. But, this seems hackish. Suggestions?
thread.yield() is often a no-op in many thread implementations and putting it or a sleep at the start of the run() method will do little or nothing since your background thread hasn't really started running yet.
I don't want my background tasks to interfere with more important tasks of my server. What strategies can be used?
You are correct that using thread priorities often have little to no effect on the number of cycles the thread will be given. This depends a lot of your architecture however so I'd certainly try them to see if it helps. If you are worried about some high performance operation taking cycles away from other more important threads then about your only recourse is to pepper the loops and other key places in your algorithm with Thread.sleep(...) calls. It may be hackish indeed but it can be effective.
The tricky parts is where to put the calls, what millis sleep should be used, etc.. That's going to take some testing and iteration to optimize the placement of the sleeps. Also, if you are calling 3rd party libraries or something you might not be able to put the sleep calls at the core places anyway.
If it doesn't work or you don't have access to the right pain points then you may have not choice except offloading your background processing to a remote system for processing.
Since the executor is single-threaded it can only occupy one core. As long as your server has N cores those background tasks will never take up more than 1/N of the server load, assuming they do not spawn new threads or dispatch to other thread pools on their own.
Alternatively you can schedule all tasks (low and high priority) through one thread pool configured with a priority queue and decorate each background task so that it has a low priority. I.e. you can move the scheduling from the kernel to userspace.
Suppose I have a nametag, which is UI component in GUI program.
The nametag will constantly change its text based on the data.
If the user change his/her name data, then he/she will see the change in nametag.
For this task, my code looks like this:
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
String name = data.getName();
nametag.setText(name);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
});
Since the reaction time of 0.1s seems instant to people, I included Thread.sleep(100) for computer to take a break.
However, I am not sure if that helps the computer in terms of energy usage or something. Is sleep method in this case complete waste of time? No benefit at all?
Thread.Sleep has been used for many things it shouldn’t be used for.
Here’s a list of the common mistakes:
The thread needs to wait for another thread to complete
In this case no value, other than infinite, passed to Thread.Sleep will be correct. You simply don’t know when the other thread will complete using this method. If the thread completed after Sleep returned you’ll likely have synchronization problems. If the other thread completed before Sleep returned the thread was needlessly blocked for an amount of time rendering the benefits of multithreading limited or moot. In the control circumstances where you’ve tested this it may seem like it always works; it just takes a busy program to cause it to faile: a defrag program, a sudden influx of network traffic, a network hiccup, etc.
The thread needs perform logic every n milliseconds
As noted earlier, Sleep means relinquish control. When your thread gets control again isn’t up to the thread; so it can’t be used for periodic logic.
We don’t know why Thread.Sleep is required; but if we take it out the application stops working
This is flawed logic because the application still doesn’t work with Thread.Sleep. This is really just spackling over the problem on that particular computer. The original problem is likely a timing/synchronization issue, ignoring it by hiding it with Thread.Sleep is only going to delay the problem and make it occur in random, hard to reproduce ways.
Source: http://blogs.msmvps.com/peterritchie/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program/
This doesn't answer your direct question, but it does help address an XY Problem component of your question:
It looks like you're listening for object state changes by polling: by constantly testing an object to see what its state is and whether it's changed, and this is a bad idea, especially when coding for an event-driven GUI. Much better to use an observer pattern and be notified of state changes when or if they occur. That is how the Swing GUI library itself was written, and you should strongly consider emulating this.
Some ways to be notified of changes are to use component event listeners which can listen for changes to Swing components, such as ActionListeners, ChangeListeners, ItemListeners, and the like. Another way when listening to non Swing component items is to use SwingPropertyChangeSupport and PropertyChangeListeners and in this way to create "bound" properties of your class. This is often used for non-GUI model classes.
I am writing a simple top down space game, and am extending it to allow play over a network with multiple players. I've done a fair bit of reading, but this is the first time I've done this and I'd appreciate some advice on choosing a sensible design.
My GUI is written using Swing. 30 times a second, a timer fires, and repaints my GUI according to data in a gameWorld object in memory (essentially a list of ships & projectiles with positions, etc). Physics updates of the gameWorld are also carried out using this timer. Thus, for the single player implementation, everything happens on the EDT, and this works fine.
Now, I have separate thread dealing with incoming packets from other players. I would like to update the data in my gameWorld object based on what these packets contain. My question is, should I use invokeLater to make these changes, or should I use locks to avoid concurrency problems?
To illustrate what I mean:
runMethodOfInputThread() {
while(takingInput) {
data = receiveAndInterpretIncomingPacket(); // blocks
SwingUtilities.invokeLater(new Runnable() {
public void run() {
gameWorld.updateWithNewGameInfo(data);
}
});
}
}
vs
runMethodOfInputThread() {
while(takingInput) {
data = receiveAndInterpretIncomingPacket(); // blocks
synchronize (gameWorldLock) {
gameWorld.updateWithNewGameInfo(data);
}
}
}
The latter would also require using similar synchronize blocks wherever the EDT accesses the gameWorld, so it seems to me that using invokeLater would be simpler to implement. But am I right in thinking both approaches would work? Are there any other significant pros/cons to bear in mind?
Thanks,
Jeremy
Well, first of all you don not need to choose only one method. You can use locks to make you data structure thread-safe "just to be sure" (since your application is already multithreaded), and use invokeLater to actually apply changes only in EDT -- and in this case JIT likely to optimize you locks down, close to 0.
Next, from my point of view invokeLater is rather preferred way: if you can way around dealing with multi-threaded -- you should use the way, just because multithreading is hard and rich of possible errors.
But applying changes via invokeLater() will put additional pressure on EDT, so, if changes come with high rate you can observe GUI degradation. Also, if gameWorld.updateWithNewGameInfo(data) is havy method taking observable time to complete, it can makes you GUI even freeze. Also, invokeLater puts your task at the tail of event queue, so it'll be done after all events currently in queue. It may -- in some cases -- cause delays in applying changes, which can makes you game less user-friendly. It may, or may not be your case, but you'll should keep it in mind
As for general rule -- not use EDT for any time consuming task. As far, as I understand, network packet parsing is already in seperate thread in your application. Applying changes can (and should) be done in separate thread too, if it is time consuming.
Pros for approach 1:
Minimized complexity
Stability
By restricting access to the 'gameWorld' variable to the EDT thread, locking mechanisms are not required. Concurrent programming is complex and requires the programmer(s) to be vigilant throughout the source base when accessing objects shared amongst threads. It is possible for
a programmer to forget to synchronize in certain instances, leading to compromised game states or program failure.
Pros for approach 2:
Scalability
Performance
Minimizing the processing done on the EDT thread ensures that the games interface and display will remain responsive to the user. Approach 1 may work for now, but later revisions of your game will not be able to scale to a more advanced interface if the EDT thread is busy doing non-ui processing.
Not the second one. You want to have as little as possible running in the EDT. If you are waiting for a lock in the EDT, it's as bad as running all the other code (on the other side of the lock) directly in the EDT since the EDT has to wait for everything else to finish.
Also, it seems that your whole game is running on the EDT. That's bad practice. You should split your code using the model-view-controller pattern. I understand your game is small and can run in the EDT, but you should probably not get into the habit.
You should have your game logic running from a timer thread (java.util.concurrent.ScheduledThreadPoolExecutor) and at the end of every period you "send" your data to the EDT and repaint with invokeLater.
You should also have some separate thread that reads the socket and that thread should write to objects that share locks with the objects you are using in the timer game thread.
My suggestion is as follows
push all loaded data from different users (thread) to a queue
use another thread to read from that queue and update UI from EDT
It should avoid your concurrency issue. How it can be achived
runMethodOfInputThread() {
while(takingInput) {
data = receiveAndInterpretIncomingPacket(); // blocks
blockingQueue.add(data);
}
}
runMethodOfUPdateUIThread() {
while(updatingUI) {
data = blockingQueue.take();
SwingUtilities.invokeLater(new Runnable() {
public void run() {
gameWorld.updateWithNewGameInfo(data);
}
});
}
}
Assume that I have a set of objects that need to be analyzed in two different ways, both of which take relatively long time and involve IO-calls, I am trying to figure out how/if I could go about optimizing this part of my software, especially utilizing the multiple processors (the machine i am sitting on for ex is a 8-core i7 which almost never goes above 10% load during execution).
I am quite new to parallel-programming or multi-threading (not sure what the right term is), so I have read some of the prior questions, particularly paying attention to highly voted and informative answers. I am also in the process of going through the Oracle/Sun tutorial on concurrency.
Here's what I thought out so far;
A thread-safe collection holds the objects to be analyzed
As soon as there are objects in the collection (they come a couple at a time from a series of queries), a thread per object is started
Each specific thread takes care of the initial pre-analysis preparations; and then calls on the analyses.
The two analyses are implemented as Runnables/Callables, and thus called on by the thread when necessary.
And my questions are:
Is this a reasonable scheme, if not, how would you go about doing this?
In order to make sure things don't get out of hand, should I implement a ThreadManager or some thing of that sort, which starts and stops threads, and re-distributes them when they are complete? For example, if i have 256 objects to be analyzed, and 16 threads in total, the ThreadManager assigns the first finished thread to the 17th object to be analyzed etc.
Is there a dramatic difference between Runnable/Callable other than the fact that Callable can return a result? Otherwise should I try to implement my own interface, in that case why?
Thanks,
You could use a BlockingQueue implementation to hold your objects and spawn your threads from there. This interface is based on the producer-consumer principle. The put() method will block if your queue is full until there is some more space and the take() method will block if the queue is empty until there are some objects again in the queue.
An ExecutorService can help you manage your pool of threads.
If you are awaiting a result from your spawned threads then Callable interface is a good idea to use since you can start the computation earlier and work in your code assuming the results in Future-s. As far as the differencies with the Runnable interface, from the Callable javadoc:
The Callable interface is similar to Runnable, in that both are designed for classes whose instances are potentially executed by another thread. A Runnable, however, does not return a result and cannot throw a checked exception.
Some general things you need to consider in your quest for java concurrency:
Visibility is not coming by defacto. volatile, AtomicReference and other objects in the java.util.concurrent.atomic package are your friends.
You need to carefully ensure atomicity of compound actions using synchronization and locks.
Your idea is basically sound. However, rather than creating threads directly, or indirectly through some kind of ThreadManager of your own design, use an Executor from Java's concurrency package. It does everything you need, and other people have already taken the time to write and debug it. An executor manages a queue of tasks, so you don't need to worry about providing the threadsafe queue yourself either.
There's no difference between Callable and Runnable except that the former returns a value. Executors will handle both, and ready them the same.
It's not clear to me whether you're planning to make the preparation step a separate task to the analyses, or fold it into one of them, with that task spawning the other analysis task halfway through. I can't think of any reason to strongly prefer one to the other, but it's a choice you should think about.
The Executors provides factory methods for creating thread pools. Specifically Executors#newFixedThreadPool(int nThreads) creates a thread pool with a fixed size that utilizes an unbounded queue. Also if a thread terminates due to a failure then a new thread will be replaced in its place. So in your specific example of 256 tasks and 16 threads you would call
// create pool
ExecutorService threadPool = Executors.newFixedThreadPool(16);
// submit task.
Runnable task = new Runnable(){};;
threadPool.submit(task);
The important question is determining the proper number of threads for you thread pool. See if this helps Efficient Number of Threads
Sounds reasonable, but it's not as trivial to implement as it may seem.
Maybe you should check the jsr166y project.
That's probably the easiest solution to your problem.