We Keep Coding

How do I make a block aware execution context?

For some reason I can't wrap my head around implementing this. I've got an application running with Play that calls out to Elastic Search. As part of my design, my service uses the Java API wrapped with scala future's as shown in this blog post. I've updated the code from that post to hint to the ExecutionContext that it will be doing some blocking I/O like so:
import scala.concurent.{blocking, Future, Promise}
import org.elasticsearch.action.{ActionRequestBuilder, ActionListener, ActionResponse }
def execute[RB <: ActionRequestBuilder[_, T, _, _]](request: RB): Future[T] = {
blocking {
request.execute(this)
promise.future
}
}
My actual service that constructs the queries to send to ES takes an executionContext as a constructor parameter that it then uses for calls to elastic search. I did this so that the global execution context that play uses won't have it's threads tied down by the blocking calls to ES. This S.O. comment mentions that only the global context is blocking aware, so that leaves me to have to create my own. In that same post/answer there's a lot of information about using a ForkJoin pool, but I'm not sure how to take what's written in those docs and combine it with the hints in the blocking documentation to create an execution context that responds to blocking hints.
I think one of the issues I have is that I'm not sure exactly how to respond to the blocking context in the first place? I was reading the best practices and the example it uses is an unbounded cache of threads:
Note that here I prefer to use an unbounded "cached thread-pool", so it doesn't have a limit. When doing blocking I/O the idea is that you've got to have enough threads that you can block. But if unbounded is too much, depending on use-case, you can later fine-tune it, the idea with this sample being that you get the ball rolling.
So does this mean that with my ForkJoin backed thread pool, that I should try to use a cached thread when dealing with non-blocking I/O and create a new thread for blocking IO? Or something else? Pretty much every resource I find online about using seperate thread pools tends to do what the Neophytes guide does, which is to say:
How to tune your various thread pools is highly dependent on your individual application and beyond the scope of this article.
I know it depends on your application, but in this case if I just want to create some type of blocking aware ExecutionContext and understand a decent strategy for managing the threads. If the Context is specifically for a single part of the application, should I just make a fixed thread pool size and not use/ignore the blocking keyword in the first place?
I tend to ramble, so I'll try to break down what I'm looking for in an answer:
Code! Reading all these docs still leave me like I'm feeling just out of reach of being able to code a blocking-aware context, and I'd really appreciate an example.
Any links or tips on how to handle blocking threads, i.e. make a new thread for them endlessly, check the number of threads available and reject if too many, some other strategy
I'm not looking for performance tips here, I know I'll only get that with testing, but I can't test if I can't figure out how to code the context's in the first place! I did find an example of ForkJoins vs threadpools but I'm missing the crucial part about blocking there.
Sorry for the long question here, I'm just trying to give you a sense of what I'm looking at and that I have been trying to wrap my head around this for over a day and need some outside help.
Edit: Just to make this clear, the ElasticSearch Service's constructor signature is:
//Note that these are not implicit parameters!
class ElasticSearchService(otherParams ..., val executionContext: ExecutionContext)
And in my application start up code I have something like this:
object Global extends GlobalSettings {
val elasticSearchContext = //Custom Context goes here
...
val elasticSearchService = new ElasticSearchService(params, elasticSearchContext);
...
}
I am also reading through Play's recommendations for contexts, but have yet to see anything about blocking hints yet and I suspect I might have to go look into the source to see if they extend the BlockContext trait.

So I dug into the documentation and Play's best practices for the situation I'm dealing with is to
In certain circumstances, you may wish to dispatch work to other thread pools. This may include CPU heavy work, or IO work, such as database access. To do this, you should first create a thread pool, this can be done easily in Scala:
And provides some code:
object Contexts {
implicit val myExecutionContext: ExecutionContext = Akka.system.dispatchers.lookup("my-context")
}
The context is from Akka, so I ran down there searching for the defaults and types of Contexts they offer, which eventually led me to the documentation on dispatchers. The default is a ForkJoinPool whose default method for managing a block is to call the managedBlock(blocker). This led me to reading the documentation that stated:
Blocks in accord with the given blocker. If the current thread is a ForkJoinWorkerThread, this method possibly arranges for a spare thread to be activated if necessary to ensure sufficient parallelism while the current thread is blocked.
So it seems like if I have a ForkJoinWorkerThread then the behavior I think I want will take place. Looking at the source of ForkJoinPool some more I noted that the default thread factory is:
val defaultForkJoinWorkerThreadFactory: ForkJoinWorkerThreadFactory = juc.ForkJoinPool.defaultForkJoinWorkerThreadFactory
Which implies to me that if I use the defaults in Akka, that I'll get a context which handles blocking in the way I expect.
So reading the Akka documentation again it would seem that specifying my context something like this:
my-context {
type = Dispatcher
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 8
parallelism-factor = 3.0
parallelism-max = 64
task-peeking-mode = "FIFO"
}
throughput = 100
}
would be what I want.
While I was searching in the source code I did some looking for uses of blocking or of calling managedBlock and found an example of overriding the ForkJoin behavior in ThreadPoolBuilder
private[akka] class AkkaForkJoinWorkerThread(_pool: ForkJoinPool) extends ForkJoinWorkerThread(_pool) with BlockContext {
override def blockOn[T](thunk: ⇒ T)(implicit permission: CanAwait): T = {
val result = new AtomicReference[Option[T]](None)
ForkJoinPool.managedBlock(new ForkJoinPool.ManagedBlocker {
def block(): Boolean = {
result.set(Some(thunk))
true
}
def isReleasable = result.get.isDefined
})
result.get.get // Exception intended if None
}
}
Which seems like what I originally asked for as an example of how to make something that implements the BlockContext. That file also has code showing how to make an ExecutorServiceFactory, which is what I believe
is reference by the executor part of the configuration. So I think what I would do if I wanted to have
a totally custom context would be extend some type of WorkerThread and write my own ExecutorServiceFactory that uses the custom workerthread and then specify the fully qualified class name in the property like this post advises.
I'm probably going to go with using Akka's forkjoin :)

"Asynchronous while loop" in JavaFX thread

When I need to do an indeterminate number of pieces of work in the JavaFX thread without blocking the user interface, I use this class
public class AsyncWhile {
private final IntPredicate hook;
private int schedCount = 0;
private boolean terminated = false;
private int callCount = 0;
private static final int schedN = 1;
public AsyncWhile(IntPredicate hook) {
this.hook = hook;
schedule();
}
public void kill(){
terminated = true;
}
private void schedule(){
while(schedCount < schedN){
Platform.runLater(this::poll);
schedCount++;
}
}
private void poll(){
schedCount--;
if(!terminated){
terminated = !hook.test(callCount++);
if(!terminated){
schedule();
}
}
}
}
like this
asyncWhile = new AsyncWhile(i -> {
// return false when you're done
// or true if you want to be called again
});
// can asyncWhile.kill() should we need to
(
If you need a more concrete example, here I'm reading one line at a time from an InputStream and then parsing and displaying a plot parsed from that line:
asyncWhile = new AsyncWhile(i -> {
String line;
try {
if((line = reader.readLine()).startsWith(" Search complete.")){ // it so happens that this reader must be read in the JavaFX thread, because it automatically updates a console window
return false;
} else {
Task<MatchPlot> task = new ParsePlotTask(line);
task.setOnSucceeded(wse -> {
plotConsumer.accept(task.getValue());
// todo update progress bar
});
executorService.submit(task);
return true;
}
} catch (IOException ex) {
new ExceptionDialog(ex).showAndWait();
return false;
}
});
)
Chaining up runLaters like that feels like a hack. What is the proper way to solve this kind of problem? (By "this kind of problem" I mean the problem that would have been solved by a simple while loop, had it not been for the fact that its contents must run in the JavaFX thread without making the UI unresponsive.)

Recommended
In general, basing a solution off of the PartialResultsTask sample from the Task documentation (which relies on Platform.runLater invocations), is the standard way of solving this problem.
Alternate
Rather than scheduling runLater's you could use a BlockingDeque. In your processing task, you perform your time-consuming process just with a normal while loop, generate non-UI model objects which need to be represented in the JavaFX UI, stick those non-UI model objects into your queue. Then you setup a Timeline or AnimationTimer that polls the queue, draining it as necessary and to pick the items off the queue and represent them in the UI.
This approach is similar (but a bit different) to: Most efficient way to log messages to JavaFX TextArea via threads with simple custom logging frameworks.
Using your own queue in this case is not much different from using the implicit queue runLater invocations go on to, though, with your own queue, you might have a little more control over the process if you need that. It's a trade-off though, as it adds a bit more custom code and complexity, so probably just use the recommended PartialResults sample from Task and, if that doesn't fit your needs, then perhaps investigate the alternative custom queue based approach.
Aside
As a side note, you could use the custom logging framework linked earlier to log console messages from multiple threads to be displayed in your UI. That way you don't need to have your reader.readLine call execute I/O on the JavaFX UI, which is not recommended. Instead, have the I/O performed off the JavaFX UI thread and, as you process items, call into the logging framework to log messages that will eventually show up on the UI (the internal mechanisms within the logging framework take care of ensuring that JavaFX threading rules are respected).
Can you see any danger in using my approach?
Sorry for being non-specific here. I'm not going to directly answer this, but tangentially and not always applicably to your approach, using runLater can cause issues, mostly it is not a concern, but some things to consider:
If you send enough runLater calls faster than they can be processed, eventually you will either run out of memory or some runLater calls will start being ignored (depending on how the runLater system works).
Calls to runLater are sequential, not prioritized, so if there are internal events which are also being runLater, such as handling UI events, those might be delayed while your runLater calls are being processed.
runLater offers no guarantee of when later is. If your work is time sensitive, that might be an issue or at least something you need to account for in your implementation.
The runLater system is likely internally fairly complex and you won't know exactly how it is implemented unless you study the source code pretty closely.
Anything that you run on runLater is going to hold up the JavaFX application thread, probably until all of the outstanding runLater calls are complete
Once you have issued a bunch of runLater calls, you can't easily intersperse their processing over multiple pulses in the JavaFX animation system, they will likely all be executed on the next pulse. So you have to be careful not to send too many calls at once.
Those are just some things that come to mind.
In general though, runLater is a sound mechanism for many tasks and a core part of the JavaFX architecture. For most things the above considerations don't really have any consequence.
Writing quality multi-threaded code is pretty tricky. To the point where it often best avoided where possible, which is what the JavaFX system attempts to do for the most part by making scene graph access single-threaded. If you must do it, then stick to the patterns outlined in the Task documentation or utilizing some of the high level java.util.concurrent systems as much as possible rather than implementing your own systems. Also note that reading multi-threaded code is even trickier than writing it, so make sure what you do is clear to the next person.

Java: guide-line for when to use thread-pooling?

This is a high-volume production system, however, this particular code path is seldom used. Its an import feature that can potential result in a lot data coming in, but it's only occasionally used, a few times a month, perhaps.
Having a (polite) debate with a colleague. The issue is whether a simple thread created the old fashioned way:
Runnable thread = new Runnable() {
public void run() {
//... do the import work ...
};
}
new Thread(thread).start();
Is sufficient, or if this requires using a thread pool.
This is happening in a service-layer class that is called from a servlet (providing a RESTful interface). The purpose being to allow the response to return and free the UI while the import happens.
As a follow on - in this situation, is using a thread pool actually just going to add more unnecessary (coding and resource use) overhead?
After EJP's comment - is there a good guideline for when it becomes 'worth having a discussion' about using pooling instead of straight thread creation?

A threadpool would only be useful if you were planning on starting a lot of these threads, and then avoid thread creation overhead by re-using them instead of kill + re-creating them for subsequent work.
Since this code path is used so rarely, you will not need a threadpool.
However, it sounds like you are doing this heavy work in the same process that serves your REST API? You may want to consider passing this work to a worker that runs in a separate process.

JEE7: best way to create another thread that never exits

I'm writing a JEE7/Glassfish 4 application that reads data from an external queue (RabbitMQ) and processes it. It needs a method (I suppose an EJB method) that contains a loop that never exits that reads the queue. I suppose since this loop never exits, it needs to be on a separate thread. My question is, what is the correct way to do this in a JEE7 application?
This may be obvious, but the ReadQueue() method needs to start automatically when the app starts and must keep running permanently.
Is the ManagedExecutorService appropriate for this?

ManagedExecutorService is exactly what you want to use for this.
The availability of this service in JEE is a great benefit. In the past, we basically just ignored the guidelines and managed all of this stuff ourselves.
The MES allows you to capture the context information of the invoking component, and tie your task in to the life cycle of the container. These are both very important in the JEE environment.
As to where to start the task, you basically have two options.
One, you can use a ServletContextListener, and have that kick off the task during container startup.
Two, you can use an #Singleton EJB, and tie in to its lifecycle methods to start your task.
If you start the task up from the ServletContextListener, then the task will run as if it's in the WAR environment. If you start it up from the #Singleton, it will run within the Session Beans environment (this mostly relates to how the JNDI appears).
Either way, you only need to worry about starting the task via these mechanisms. You should rely on the ManagedTaskListener.taskAborted interface method to shut your task down.
In theory you can work with the Thread.interrupt that is sent to your task during shut down. I've never had good luck with that myself, I rely on an external mechanism to tell the long running tasks to shut off.
I wish I could give first hand experience with this new facility, but I haven't had an opportunity to try it out yet. But from the spec, this is what you want to do.

To start a thread with an infinite loop that polls the queue periodically is usually not a good idea. The nature of queues suggests an async, event-driven processing. For such problems in the JEE world you have MDBs. The only issue here is that MDB requires a JMS queue provider but RabbitMQ is using a different protocol (AMQP). You need a JMS-AMQP bridge to make this work. Could be Qpid JMS but no guarantee that it will work.

Here is one way to create a thread that never exits:
public class HelloRunnable implements Runnable {
public void run() {
while (true) {
// do ReadQueue() here
}
}
public static void main(String args[]) {
(new Thread(new HelloRunnable())).start();
}
}

In Swing can you post an event to the top of the EDT events queue?

I am looking for a way to do what the InvokeLater() function does only instead of putting the event on the bottom of the event queue it puts it on top. At least I think that will do what I want, maybe there's a better way. Below describes what I'm trying to replicate.
Years ago I use a c++ framework on the Mac that had feature that you could add a Chore object to a CriticalChore list while processing the current event. You would do this while executing code in what amounts to Swings EDT. After the current event was finished being processed and just before calling GetNextEvent() the Framework would check if the CriticalChore list was empty. If the list had items in it there Perform() (i.e. run()) function would be called. When finished with the list all the items were then deleted.
This feature came in really handy. Many times while handling an event early on you know you need to perform some additional code but only after a lot of other code is processed. But most importantly, is this code needs to be processed before any other events from the EDT queue are handled.

I don't see any method of doing that. I suppose that you could do some hacky stuff to make your own method of injecting higher priority actions.
I think the best answer, though, is to not do this at all. If you have a need to do so, the design probably needs to be reworked. The EventDispatchThread is supposed to be only for very short-running actions as it's never supposed to look to the end user as though the application has frozen. Because of this, the queue for the EDT should always be short enough that anything you put on it will happen "instantly" from the point of view of the user, so everything on it should have "instant" priority.
If anything needs to be done which is not a very short-lived action, there is a separate methodology for doing that. There is a Swing Worker class for that, and you are supposed to use this to set up tasks that run alongside the EDT and listen for its responses.
Here is a Swing Worker Tutorial. There are also some other good ones that Google pulls up with a "Java SwingWorker tutorial" query.

First, how's done
It's possible to install a global listener with its own queue and one each event polling the queue. Toolkit.addAWTEventListener(listener, eventMask)
There is a sun.awt.PeerEvent (for sun impl) that has an ultimate priority which offers the easiest impl since it's practically the same as EventQueue.invokeLater extending java.awt.event.InvocationEvent but again it's not standard.
Last:
Here how is done standard way, I have not tested the code, though (lazy & very late)
class EventQueueX extends EventQueue{
final ConcurrentLinkedQueue<AWTEvent> que=new ConcurrentLinkedQueue<AWTEvent>();
#Override
public AWTEvent getNextEvent() throws InterruptedException {
AWTEvent e = que.poll();
if (e!=null)
return e;
return super.getNextEvent();
}
#Override
public synchronized AWTEvent peekEvent() {
AWTEvent e = que.peek();
if (e!=null)
return e;
return super.peekEvent();
}
public void pushFirst(AWTEvent e){
que.offer(e);
synchronized (this) {
notify();
}
}
public void install(Toolkit toolkit){
EventQueue q = toolkit.getSystemEventQueue();
if (q!=this)
q.push(this);
}
};
Use the EventQueueX.install() and then pushFirst(e) when you want an event and you're set. Unfortunately the queue will get deinstalled on an exception and might be pushed away too.
Next, why is bad
To the question. putting an event in the front of the queue is a bad idea overall.
If you have to call any code later on just structure your own design and at the end of the function invoke the necessary code, use a Queue if you need be.
Adding an extra layer of super ultimate priority might look ok, but it's a hard to understand design concept for any regular AWT/Swing (UI mostly) developer. If you need to queue actions, use your own mini-framework w/o messing up with awt. While I am particularly good at hacking, even I, myself, would fine such an approach weird (to put it mildly).