We Keep Coding

Java Supporting Embedded Multi-Threading

Need help with Java multiple threading
I have a case as below:
There are many records. Each record has about 250 fields. Each field needs to be validated against on a predefined rule.
So I defined a class, FieldInfo, to represent each field:
public class FieldInfo {
private String name;
private String value;
private String error_code;
private String error_message;
// ignore getters and setters
}
a class Record to represent a record:
public class Record {
List<FieldInfo> fields;
// omit getter and setter here
}
and the rule interface and class:
public interface BusinessRule {
// validating one field needs some other fields' value in the same record. So the list of all fields for a certain record passed in as parameter
public FieldInfo validate(List<FieldInfo> fields);
}
public class FieldName_Rule implements BusinessRule {
public FieldInfo validate(List<FieldInfo> fields) {
// will do
// 1. pickup those fields required for validating this target field, including this target field
// 2. performs validation logics A, B, C...
// note: all rules only read data from a database, no update/insert operations.
}
}
User can submit 5000 records or more at a time for process. The performance requirement is high. I was thinking to have multiple threads for the submitted, for example 5000, records (means one thread run several records), and in each thread, fork another multiple threads on each record to run rules.
But unfortunately, such embedded multi-threading always died in my case.
Here are some key parts from the above solution:
public class BusinessRuleService {
#Autowired
private ValidationHandler handler;
public String process(String xmlRequest) {
List<Record> records = XmlConverter.unmarshall(xmlRequest).toList();
ExecutorService es = Executors.newFixedThreadPool(100);
List<CompletableFuture<Integer> futures =
records.stream().map(r->CompletableFuture.supplyAsync(()-> handler.invoke(r), es)).collect(Collectors.toList());
List<Integer> result = future.stream().map(CompletableFuture::join).collect(Collectors.toList());
System.out.println("total records %d processed.", result.size());
es.shutdown();
return XmlConverter.marshallObject(records);
}
}
#Component
public class ValidationHandlerImpl implements ValidationHandler {
#Autowired
private List<BusinessRule> rules;
#Override
public int invoke(Record record) {
ExecutorService es = Executors.newFixedThreadPool(250);
List<CompletableFuture<FieldInfo> futures =
rules.stream().map(r->CompletableFuture.supplyAsync(()-> r.validate(record.getFields()), es)).collect(Collectors.toList());
List<FieldInfo> result = future.stream().map(CompletableFuture::join).collect(Collectors.toList());
System.out.println("total records %d processed.", result.size());
es.shutdown();
return 0;
}
}
The workflow is:
User submits a list of records in an xml string format. One of the application endpoint launches the process method in a BusinessRuleService object. The process uses CompletableFuture to compose tasks and submit the tasks to a ExecutorService which has a thread pool of size 100. Each task in the CompletableFuture list then launches ValidationHandler object. The ValidationHandler object composes another CompletableFuture task and submit the task to another ExecutorService which has the pool size the same as the rule list size.
The above solution is proper?
Note: my current solution is: the submitted records are processed in sequence. And the 250 rules are processed in parallel for each record. With this solution, it takes more than 2 hours for 5000 records. Such poor performance is not acceptable by business.
I am very new to concurrent/multi-threading programming.
Much appreciate for all kind of helps!

This is a well known "single producer - multiple consumers" pattern. The classic solution is to create a BlockingQueue<Record> queue, and put records there at the pace of their reading. On the other end of the queue, a number of working threads read records from the queue and process them (in our case, validate the fields):
class ValidatingThread extends Tread {
BlockingQueue<Record> queue;
FieldName_Rule validator = new FieldName_Rule();
public Validator (BlockingQueue<Record> queue) {
this.queue = queue;
}
public void run() {
Record record = queue.take();
validator.validate(collectFields(record));
}
}
The optimal number of threads equals to the Runtime.getRuntime().availableProcessors().
Start them all at the beginning, and do not use "embedded multi-threading".
The task how to stop the threads after all the records are processed, is left as a learning assignment.

How to manage threads in Spring TaskExecutor framework

I have a BlockingQueue of Runnable - I can simply execute all tasks using one of TaskExecutor implementations, and all will be run in parallel.
However some Runnable depends on others, it means they need to wait when Runnable finish, then they can be executed.
Rule is quite simple: every Runnable has a code. Two Runnable with the same code cannot be run simultanously, but if the code differ they should be run in parallel.
In other words all running Runnable need to have different code, all "duplicates" should wait.
The problem is that there's no event/method/whatsoever when thread ends.
I can built such notification into every Runnable, but I don't like this approach, because it will be done just before thread ends, not after it's ended
java.util.concurrent.ThreadPoolExecutor has method afterExecute, but it needs to be implemented - Spring use only default implementation, and this method is ignored.
Even if I do that, it's getting complicated, because I need to track two additional collections: with Runnables already executing (no implementation gives access to this information) and with those postponed because they have duplicated code.
I like the BlockingQueue approach because there's no polling, thread simply activate when something new is in the queue. But maybe there's a better approach to manage such dependencies between Runnables, so I should give up with BlockingQueue and use different strategy?

If the number of different codes is not that large, the approach with a separate single thread executor for each possible code, offered by BarrySW19, is fine.
If the whole number of threads become unacceptable, then, instead of single thread executor, we can use an actor (from Akka or another similar library):
public class WorkerActor extends UntypedActor {
public void onReceive(Object message) {
if (message instanceof Runnable) {
Runnable work = (Runnable) message;
work.run();
} else {
// report an error
}
}
}
As in the original solution, ActorRefs for WorkerActors are collected in a HashMap. When an ActorRef workerActorRef corresponding to the given code is obtained (retrieved or created), the Runnable job is submitted to execution with workerActorRef.tell(job).
If you don't want to have a dependency to the actor library, you can program WorkerActor from scratch:
public class WorkerActor implements Runnable, Executor {
Executor executor=ForkJoinPool.commonPool(); // or can by assigned in constructor
LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueu<>();
boolean running = false;
public synchronized void execute(Runnable job) {
queue.put(job);
if (!running) {
executor.execute(this); // execute this worker, not job!
running=true;
}
public void run() {
for (;;) {
Runnable work=null;
synchronized (this) {
work = queue.poll();
if (work==null) {
running = false;
return;
}
}
work.run();
}
}
}
When a WorkerActor worker corresponding to the given code is obtained (retrieved or created), the Runnable job is submitted to execution with worker.execute(job).

One alternate strategy which springs to mind is to have a separate single thread executor for each possible code. Then, when you want to submit a new Runnable you simply lookup the correct executor to use for its code and submit the job.
This may, or may not be a good solution depending on how many different codes you have. The main thing to consider would be that the number of concurrent threads running could be as high as the number of different codes you have. If you have many different codes this could be a problem.
Of course, you could use a Semaphore to restrict the number of concurrently running jobs; you would still create one thread per code, but only a limited number could actually execute at the same time. For example, this would serialise jobs by code, allowing up to three different codes to run concurrently:
public class MultiPoolExecutor {
private final Semaphore semaphore = new Semaphore(3);
private final ConcurrentMap<String, ExecutorService> serviceMap
= new ConcurrentHashMap<>();
public void submit(String code, Runnable job) {
ExecutorService executorService = serviceMap.computeIfAbsent(
code, (k) -> Executors.newSingleThreadExecutor());
executorService.submit(() -> {
semaphore.acquireUninterruptibly();
try {
job.run();
} finally {
semaphore.release();
}
});
}
}
Another approach would be to modify the Runnable to release a lock and check for jobs which could be run upon completion (so avoiding polling) - something like this example, which keeps all the jobs in a list until they can be submitted. The boolean latch ensures only one job for each code has been submitted to the thread pool at any one time. Whenever a new job arrives or a running one completes the code checks again for new jobs which can be submitted (the CodedRunnable is simply an extension of Runnable which has a code property).
public class SubmissionService {
private final ExecutorService executorService = Executors.newFixedThreadPool(5);
private final ConcurrentMap<String, AtomicBoolean> locks = new ConcurrentHashMap<>();
private final List<CodedRunnable> jobs = new ArrayList<>();
public void submit(CodedRunnable codedRunnable) {
synchronized (jobs) {
jobs.add(codedRunnable);
}
submitWaitingJobs();
}
private void submitWaitingJobs() {
synchronized (jobs) {
for(Iterator<CodedRunnable> iter = jobs.iterator(); iter.hasNext(); ) {
CodedRunnable nextJob = iter.next();
AtomicBoolean latch = locks.computeIfAbsent(
nextJob.getCode(), (k) -> new AtomicBoolean(false));
if(latch.compareAndSet(false, true)) {
iter.remove();
executorService.submit(() -> {
try {
nextJob.run();
} finally {
latch.set(false);
submitWaitingJobs();
}
});
}
}
}
}
}
The downside of this approach is that the code needs to scan through the entire list of waiting jobs after each task completes. Of course, you could make this more efficient - a completing task would actually only need to check for other jobs with the same code, so the jobs could be stored in a Map<String, List<Runnable>> structure instead to allow for faster processing.

How to wait for completion of multiple tasks in Java?

What is the proper way to implement concurrency in Java applications? I know about Threads and stuff, of course, I have been programming for Java for 10 years now, but haven't had too much experience with concurrency.
For example, I have to asynchronously load a few resources, and only after all have been loaded, can I proceed and do more work. Needless to say, there is no order how they will finish. How do I do this?
In JavaScript, I like using the jQuery.deferred infrastructure, to say
$.when(deferred1,deferred2,deferred3...)
.done(
function(){//here everything is done
...
});
But what do I do in Java?

You can achieve it in multiple ways.
1.ExecutorService invokeAll() API
Executes the given tasks, returning a list of Futures holding their status and results when all complete.
2.CountDownLatch
A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes.
A CountDownLatch is initialized with a given count. The await methods block until the current count reaches zero due to invocations of the countDown() method, after which all waiting threads are released and any subsequent invocations of await return immediately. This is a one-shot phenomenon -- the count cannot be reset. If you need a version that resets the count, consider using a CyclicBarrier.
3.ForkJoinPool or newWorkStealingPool() in Executors is other way
Have a look at related SE questions:
How to wait for a thread that spawns it's own thread?
Executors: How to synchronously wait until all tasks have finished if tasks are created recursively?

I would use parallel stream.
Stream.of(runnable1, runnable2, runnable3).parallel().forEach(r -> r.run());
// do something after all these are done.
If you need this to be asynchronous, then you might use a pool or Thread.
I have to asynchronously load a few resources,
You could collect these resources like this.
List<String> urls = ....
Map<String, String> map = urls.parallelStream()
.collect(Collectors.toMap(u -> u, u -> download(u)));
This will give you a mapping of all the resources once they have been downloaded concurrently. The concurrency will be the number of CPUs you have by default.

If I'm not using parallel Streams or Spring MVC's TaskExecutor, I usually use CountDownLatch. Instantiate with # of tasks, reduce once for each thread that completes its task. CountDownLatch.await() waits until the latch is at 0. Really useful.
Read more here: JavaDocs

Personally, I would do something like this if I am using Java 8 or later.
// Retrieving instagram followers
CompletableFuture<Integer> instagramFollowers = CompletableFuture.supplyAsync(() -> {
// getInstaFollowers(userId);
return 0; // default value
});
// Retrieving twitter followers
CompletableFuture<Integer> twitterFollowers = CompletableFuture.supplyAsync(() -> {
// getTwFollowers(userId);
return 0; // default value
});
System.out.println("Calculating Total Followers...");
CompletableFuture<Integer> totalFollowers = instagramFollowers
.thenCombine(twitterFollowers, (instaFollowers, twFollowers) -> {
return instaFollowers + twFollowers; // can be replaced with method reference
});
System.out.println("Total followers: " + totalFollowers.get()); // blocks until both the above tasks are complete
I used supplyAsync() as I am returning some value (no. of followers in this case) from the tasks otherwise I could have used runAsync(). Both of these run the task in a separate thread.
Finally, I used thenCombine() to join both the CompletableFuture. You could also use thenCompose() to join two CompletableFuture if one depends on the other. But in this case, as both the tasks can be executed in parallel, I used thenCombine().
The methods getInstaFollowers(userId) and getTwFollowers(userId) are simple HTTP calls or something.

You can use a ThreadPool and Executors to do this.
https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html

This is an example I use Threads. Its a static executerService with a fixed size of 50 threads.
public class ThreadPoolExecutor {
private static final ExecutorService executorService = Executors.newFixedThreadPool(50,
new ThreadFactoryBuilder().setNameFormat("thread-%d").build());
private static ThreadPoolExecutor instance = new ThreadPoolExecutor();
public static ThreadPoolExecutor getInstance() {
return instance;
}
public <T> Future<? extends T> queueJob(Callable<? extends T> task) {
return executorService.submit(task);
}
public void shutdown() {
executorService.shutdown();
}
}
The business logic for the executer is used like this: (You can use Callable or Runnable. Callable can return something, Runnable not)
public class MultipleExecutor implements Callable<ReturnType> {//your code}
And the call of the executer:
ThreadPoolExecutor threadPoolExecutor = ThreadPoolExecutor.getInstance();
List<Future<? extends ReturnType>> results = new LinkedList<>();
for (Type Type : typeList) {
Future<? extends ReturnType> future = threadPoolExecutor.queueJob(
new MultipleExecutor(needed parameters));
results.add(future);
}
for (Future<? extends ReturnType> result : results) {
try {
if (result.get() != null) {
result.get(); // here you get the return of one thread
}
} catch (InterruptedException | ExecutionException e) {
logger.error(e, e);
}
}

The same behaviour as with $.Deferred in jQuery you can archive in Java 8 with a class called CompletableFuture. This class provides the API for working with Promises. In order to create async code you can use one of it's static creational methods like #runAsync, #supplyAsync. Then applying some computation of results with #thenApply.

I usually opt for an async notify-start, notify-progress, notify-end approach:
class Task extends Thread {
private ThreadLauncher parent;
public Task(ThreadLauncher parent) {
super();
this.parent = parent;
}
public void run() {
doStuff();
parent.notifyEnd(this);
}
public /*abstract*/ void doStuff() {
// ...
}
}
class ThreadLauncher {
public void stuff() {
for (int i=0; i<10; i++)
new Task(this).start();
}
public void notifyEnd(Task who) {
// ...
}
}

Singleton class to manage running tasks in multithreaded environment in Java

I have a similar situation to that described in this question:
Java email sending queue - fixed number of threads sending as many messages as are available
In that I have a blocking queue that gets fed commands(ICommandTask extends Callable{Object}) from which a thread pool takes off and runs. The blocking queue provides thread synchronization and isolation between calling thread and executing thread. Different objects throughout the program can submit ICommandTasks to the command queue which is why I've made AddTask() static.
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import com.mypackage.tasks.ICommandTask;
public enum CommandQueue
{
INSTANCE;
private final BlockingQueue<ICommandTask> commandQueue;
private final ExecutorService executor;
private CommandQueue()
{
commandQueue = new LinkedBlockingQueue<ICommandTask>();
executor = Executors.newCachedThreadPool();
}
public static void start()
{
new Thread(INSTANCE.new WaitForProducers()).start();
}
public static void addTask(ICommandTask command)
{
INSTANCE.commandQueue.add(command);
}
private class WaitForProducers implements Runnable
{
#Override
public void run()
{
ICommandTask command;
while(true)
{
try
{
command = INSTANCE.commandQueue.take();
executor.submit(task);
}
catch (InterruptedException e)
{
// logging etc.
}
}
}
}
}
In the main program during start up the Command Queue is started using the following which creates a New CommandQueue object and starts the WaitForProducers in a separate thread.
CommandQueue.Start();
I wanted to ask whether this method of setting up a multiple producers to single executor using the singleton enum (so that different parts of the program can access), and that uses a separate thread to take off tasks from the queue and submit to a ThreadPool is a recommended way of doing what I want to achieve. Particularly in a very multithreaded environment.
So far it seems to be working ok but I plan on creating similar objects to CommandQueue to handle different types of Tasks. They will be stored in their own queues. E.g. OrderQueue, EventQueue, NegotiationQueue etc. So it needs to be somewhat scaleable and threadsafe.
Thanks in advance.

Are there any executor in java concurrent package which guarantee that all tasks will be done in order they were submitted?

A code sample for demonstration of the idea from the title:
executor.submit(runnable1);
executor.submit(runnable2);
I need to be sure that runnable1 will finish before runnable2 start and I haven't found any proofs of such behavior in the executors documentation.
About the problem I'm solving:
I need write lots of logs to a file. Each log requires much precomputing (formatting and some other stuff). So, I want to put each logging task to a kind of queue and process these tasks in a separate thread. And, of course, it's important to keep logs ordering.

A single threaded executor will perform all tasks in the order submitted. You would only use a thread pool with multiple threads if you wanted the tasks to be perform concurrently.
Adding tasks to a queue can be expensive in itself. You can use an Exchanger like this
http://vanillajava.blogspot.com/2011/09/exchange-and-gc-less-java.html?z#!/2011/09/exchange-and-gc-less-java.html
This avoid using a queue or creating object.
An alternative which is faster is to use a memory mapped file which doesn't require a background thread (actually the OS is working in the background) This is much faster again. It supports sub-microsecond latencies and millions of messages per second.
https://github.com/peter-lawrey/Java-Chronicle

You could create a simple wrapper like the one below so that all your Runnables are executed in the same thread (i.e. sequentially), and submit that wrapper to the executor instead. That does not address the logging issue.
class MyRunnable implements Runnable {
private List<Runnable> runnables = new ArrayList<>();
public void add(Runnable r) {
runnables.add(r);
}
#Override
public void run() {
for (Runnable r : runnables) {
r.run();
}
}
}
//......
MyRunnable r = new MyRunnable();
r.add(runnable1);
r.add(runnable2);
executor.submit(r);

Presumably you are doing some post-analysis of the logfile? Have you considered not caring about the order they're written and re-ordering offline later. You could allocate a unique id at submit time using, a timestamp or AtomicLong?
a code sketch (untested) would look like this:
import java.util.concurrent.atomic.AtomicLong;
class MyProcessor {
public void work()
for (Object data: allData) {
executor.submit(new MySequencedRunnable(data);
}
}
}
class MySequencedRunnable implements Runnable {
private static final AtomicLong LOG_SEQUENCE_ID = new AtomicLong(0);
private final Object data;
MySequencedRunnable(Object data) {
this.data = data;
}
public void run() {
LOGGER.log(LOG_SEQUENCE_ID.incrementAndGet(), data);
}
}
Also consider, if you're using something like log4j, using NDC or MDC to assist with the re-ordering.