We Keep Coding

Execute each subtask in parallel in a multithreaded environment

I am working on a library which will take an object DataRequest as an input parameter and basis on that object, I will construct an URL and then make a call to our app servers using apache http client and then return the response back to the customer who is using our library. Some customer will call the executeSync method to get the same feature and some customer will call our executeAsync method to get the data.
executeSync() - waits until I have a result, returns the result.
executeAsync() - returns a Future immediately which can be processed after other things are done, if needed.
Below is my DataClient class which has above two methods:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16);
private CloseableHttpClient httpClientBuilder;
// initializing httpclient only once
public DataClient() {
try {
RequestConfig requestConfig =
RequestConfig.custom().setConnectionRequestTimeout(500).setConnectTimeout(500)
.setSocketTimeout(500).setStaleConnectionCheckEnabled(false).build();
SocketConfig socketConfig =
SocketConfig.custom().setSoKeepAlive(true).setTcpNoDelay(true).build();
PoolingHttpClientConnectionManager poolingHttpClientConnectionManager =
new PoolingHttpClientConnectionManager();
poolingHttpClientConnectionManager.setMaxTotal(300);
poolingHttpClientConnectionManager.setDefaultMaxPerRoute(200);
httpClientBuilder =
HttpClientBuilder.create().setConnectionManager(poolingHttpClientConnectionManager)
.setDefaultRequestConfig(requestConfig).setDefaultSocketConfig(socketConfig).build();
} catch (Exception ex) {
// log error
}
}
#Override
public List<DataResponse> executeSync(DataRequest key) {
List<DataResponse> responsList = null;
Future<List<DataResponse>> responseFuture = null;
try {
responseFuture = executeAsync(key);
responsList = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException | ExecutionException | InterruptedException ex) {
responsList =
Collections.singletonList(new DataResponse(DataErrorEnum.CLIENT_TIMEOUT,
DataStatusEnum.ERROR));
responseFuture.cancel(true);
// logging exception here
}
return responsList;
}
#Override
public Future<List<DataResponse>> executeAsync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.submit(task);
}
}
Below is my DataFetcherTask class which also has a static class DataRequestTask which calls our app servers by making URL:
public class DataFetcherTask extends RecursiveTask<List<DataResponse>> {
private final DataRequest key;
private final CloseableHttpClient httpClientBuilder;
public DataFetcherTask(DataRequest key, CloseableHttpClient httpClientBuilder) {
this.key = key;
this.httpClientBuilder = httpClientBuilder;
}
#Override
protected List<DataResponse> compute() {
// Create subtasks for the key and invoke them
List<DataRequestTask> requestTasks = requestTasks(generateKeys());
invokeAll(requestTasks);
// All tasks are finished if invokeAll() returns.
List<DataResponse> responseList = new ArrayList<>(requestTasks.size());
for (DataRequestTask task : requestTasks) {
try {
responseList.add(task.get());
} catch (InterruptedException | ExecutionException e) {
Thread.currentThread().interrupt();
return Collections.emptyList();
}
}
return responseList;
}
private List<DataRequestTask> requestTasks(List<DataRequest> keys) {
List<DataRequestTask> tasks = new ArrayList<>(keys.size());
for (DataRequest key : keys) {
tasks.add(new DataRequestTask(key));
}
return tasks;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
/** Inner class for the subtasks. */
private static class DataRequestTask extends RecursiveTask<DataResponse> {
private final DataRequest request;
public DataRequestTask(DataRequest request) {
this.request = request;
}
#Override
protected DataResponse compute() {
return performDataRequest(this.request);
}
private DataResponse performDataRequest(DataRequest key) {
MappingHolder mappings = DataMapping.getMappings(key.getType());
List<String> hostnames = mappings.getAllHostnames(key);
for (String hostname : hostnames) {
String url = generateUrl(hostname);
HttpGet httpGet = new HttpGet(url);
httpGet.setConfig(generateRequestConfig());
httpGet.addHeader(key.getHeader());
try (CloseableHttpResponse response = httpClientBuilder.execute(httpGet)) {
HttpEntity entity = response.getEntity();
String responseBody =
TestUtils.isEmpty(entity) ? null : IOUtils.toString(entity.getContent(),
StandardCharsets.UTF_8);
return new DataResponse(responseBody, DataErrorEnum.OK, DataStatusEnum.OK);
} catch (IOException ex) {
// log error
}
}
return new DataResponse(DataErrorEnum.SERVERS_DOWN, DataStatusEnum.ERROR);
}
}
}
For each DataRequest object there is a DataResponse object. Now once someone calls our library by passing DataRequest object, internally we make List<DataRequest> object and then we invoke each DataRequest object in parallel and return List<DataResponse> back where each DataResponse object in the list will have response for corresponding DataRequest object.
Below is the flow:
Customer will call DataClient class by passing DataRequest object. They can call executeSync() or executeAsync() method depending on their requirements.
Now in the DataFetcherTask class (which is a RecursiveTask one of ForkJoinTask's subtypes), given a key object which is a single DataRequest, I will generate List<DataRequest> and then invokes each subtask in parallel for each DataRequest object in the list. These subtasks are executed in the same ForkJoinPool as the parent task.
Now in the DataRequestTask class, I am executing each DataRequest object by making an URL and return its DataResponse object back.
Problem Statement:
Since this library is being called in a very high throughput environment so it has to be very fast. For synchronous call, executing in a separate thread is ok here? It will incur extra costs and resources for a Thread along with the cost of context switch of threads in this case so I am little bit confuse. Also I am using ForkJoinPool here which will save me in using extra thread pool but is it the right choice here?
Is there any better and efficient way to do the same thing which can be performance efficient as well? I am using Java 7 and have access to Guava library as well so if it can simplify anything then I am open for that as well.
It looks like we are seeing some contention when it runs under very heavy load. Is there any way this code can go into thread contention when runs under very heavy load?

I think in your situation it's better to use async http call, see link: HttpAsyncClient. And you don't need to use thread pool.
In executeAsync method create empty CompletableFuture<DataResponse>() and pass it to client call, there in callback call set the result of completableFuture by calling complete on it (or completeExceptionally if exceptions raise).
ExecuteSync method implementation looks good.
edit:
For java 7 it's only need to replace a completableFuture to promise implementation in guava, like ListenableFuture or anything similar

The choice to use the ForkJoinPool is correct, its designed for efficiency with many small tasks:
A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute tasks submitted to the pool and/or created by other active tasks (eventually blocking waiting for work if none exist). This enables efficient processing when most tasks spawn other subtasks (as do most ForkJoinTasks), as well as when many small tasks are submitted to the pool from external clients. Especially when setting asyncMode to true in constructors, ForkJoinPools may also be appropriate for use with event-style tasks that are never joined.
I suggest to try the asyncMode = true in the constructor since in your case the tasks are never joined:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16, ForkJoinPool.ForkJoinWorkerThreadFactory, null, true);
...
}
For the executeSync() you can use the forkJoinPool.invoke(task), this is the managed way to do a synchronous task execution in the pool for resources optimisation:
#Override
public List<DataResponse> executeSync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.invoke(task);
}
If you can use Java 8 then there is a common pool already optimised: ForkJoinPool.commonPool()

Returning a value from thread

First of all, yes I looked up this question on google and I did not find any answer to it. There are only answers, where the thread is FINISHED and than the value is returned. What I want, is to return an "infinite" amount of values.
Just to make it more clear for you: My thread is reading messages from a socket and never really finishes. So whenever a new message comes in, I want another class to get this message. How would I do that?
public void run(){
while(ircMessage != null){
ircMessage = in.readLine();
System.out.println(ircMessage);
if (ircMessage.contains("PRIVMSG")){
String[] ViewerNameRawRaw;
ViewerNameRawRaw = ircMessage.split("#");
String ViewerNameRaw = ViewerNameRawRaw[2];
String[] ViewerNameR = ViewerNameRaw.split(".tmi.twitch.tv");
viewerName = ViewerNameR[0];
String[] ViewerMessageRawRawRaw = ircMessage.split("PRIVMSG");
String ViewerMessageRawRaw = ViewerMessageRawRawRaw[1];
String ViewerMessageRaw[] = ViewerMessageRawRaw.split(":", 2);
viewerMessage = ViewerMessageRaw[1];
}
}
}

What you are describing is a typical scenario of asynchronous communication. Usually solution could be implemented with Queue. Your Thread is a producer. Each time your thread reads a message from socket it builds its result and sends it into a queue. Any Entity that is interested to receive the result should be listening to the Queue (i.e. be a consumer). Read more about queues as you can send your message so that only one consumer will get it or (publishing) means that all registered consumers may get it. Queue implementation could be a comercialy available products such as Rabbit MQ for example or as simple as Java provided classes that can work as in memory queues. (See Queue interface and its various implementations). Another way to go about it is communication over web (HTTP). Your thread reads a message from a socket, builds a result and sends it over http using let's say a REST protocol to a consumer that exposes a rest API that your thread can call to.

Why not have a status variable in your thread class? You can then update this during execution and before exiting. Once the thread has completed, you can still query the status.
public static void main(String[] args) throws InterruptedException {
threading th = new threading();
System.out.println("before run Status:" + th.getStatus());
th.start();
Thread.sleep(500);
System.out.println("running Status:" + th.getStatus());
while(th.isAlive()) {}
System.out.println("after run Status:" + th.getStatus());
}
Extend thread to be:
public class threading extends Thread {
private int status = -1; //not started
private void setStatus(int status){
this.status = status;
}
public void run(){
setStatus(1);//running
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
setStatus(0); //exit clean
}
public int getStatus(){
return this.status;
}
}
And get an output of:
before run Status:-1
running Status:1
after run Status:0

Rest method need long time

This is a design question and I am asking for some ideas.
I have a rest method and it will trigger long-time tasks (10~15 minutes)
As the function takes long time, I run it as a thread,
this can avoid method timeout, but how can I know if the thread went wrong?
Runnable loader = new Runnable() {
public void run() {
//tasks
}
};
(new Thread(loader)).start();
Update: the rest service like this
#path()
beginload(){
//let thread run and return info first
//how can i know if this thread went wrong?
(new Thread(loader)).start();
return "need 15 minutes";
}

Conceptually there has to be a way for the service to communicate a failure to the client. There are multiple ways you can do this. Here are three examples:
After the client calls the service, the service immediately returns a job ID. The client can use the job ID later to query the service for the status (including error). For example, when you launch instances at AWS EC2, it takes a while for EC2 to service the request, so the launch request returns a so-called "reservation ID" that you can use in subsequent operations (like querying for status, terminating the launch, etc.).
Pro: Usable in a wide variety of cases, and easy enough to implement.
Con: Requires polling. (I.e. more chatty.)
The client offers a callback URI that the service invokes upon job completion. The callback URI can either be configured into the service, or else passed along as a request parameter. (Don't hardcode the callback URI in the service since services shouldn't depend on their clients.)
Pro: Still pretty simple, and avoids polling.
Con: Client has to have URI for the service to call, which may not be convenient. (E.g. the client may be a desktop app rather than a service, firewall may prevent it, etc.)
The client pushes a notification into a message queue, and the client listens to that queue.
Pro: Avoids polling, and client doesn't need endpoints to call.
Con: More work to set up (requires messaging infrastructure).
There are other possibilities but those are typical approaches.

Do you need to differentiate between different requests? If several tasks to perform, you need an ID.
You can do something like the following:
private static final ExecutorService es = Executors.newFixedThreadPool(10);
private static final Map<Long, Future<Void>> map = new HashMap<>();
#GET
#Path("/submit")
public Response submitTask() {
long id = System.currentTimeMillis();
Future<Void> future = es.submit(new Callable<Void>() {
public Void call() throws Exception {
// long task
// you must throw exception for bad task
return null;
}
});
map.put(id, future);
return Response.ok(id, MediaType.TEXT_PLAIN).build();
}
#GET
#Path("/status/{id}")
public Response submitTask(#PathParam("id") long id) {
Future<Void> future = map.get(id);
if (future.isDone()) {
try {
future.get();
return Response.ok("Successful!", MediaType.TEXT_PLAIN).build();
} catch (InterruptedException | ExecutionException e) {
// log
return Response.ok("Bad task!", MediaType.TEXT_PLAIN).build();
}
}
return Response.ok("Wait a few seconds.", MediaType.TEXT_PLAIN).build();
}
This can give you an idea. Remember purge the map of old tasks.

If you want to get the return value of your thread and throw/catch possible exception, consider use Callable rather than Runnable, and it can be used along with ExecutorService which provide more functionality.
Callable : A task that returns a result and may throw an exception.
Implementors define a single method with no arguments called call.
public interface Callable<V> {
V call() throws Exception;
}

Netty - How to get server response in the client

I'm mostly there with Netty but one concept is still alluding me, and I can't find anything in the tutorials and so on. Firstly I do understand that Netty is asynchronous, but there must be a way for a client to call the server and be able to get a response beyond the handler. Let me explain more.
I have a client as illustrated below. And please note that I understand it's bootstrapped and a new connection is established on each call, that's just there to make the example smaller and more succinct. Please ignore that fact.
Client.java
// ServerResponse is a result from the server, in this case
// a list of users of the system (ignore that each time it's all bootstrapped).
public User[] callServerForInformationFromGUIWidget()
{
ClientBootstrap bootstrap = new ClientBootstrap(...);
bootstrap.setPipelineFactory(...);
ChannelFuture future = bootstrap.connect(new InetSocketAddress(host, port));
Channel channel = future.awaitUninterruptibly().getChannel();
// Where request is a POJO sent to the server,
// with a request such as get me a list of users
RequestPojo request = new RequestPojo(requestUserListCommand);
ChannelFuture lastWriteFuture = channel.write(request);
if(lastWriteFuture != null)
lastWriteFuture.awaitUninterruptibly();
}
Now I understand how to get the data on the server, and fire back the result. The only thing is how do I handle it on the client side? Yes the clientHandler class can do something like the following:
ClientHandler.java
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e)
{
User[] users = (User[])e.getMessage();
}
The problem is how does the client code actually get that result? All the examples are similar to a chat service, where the event fires off something else on the client that's not waiting on a response. Even the http client example I found lacking this. The documentation overall is really good, but it's lacking on how to do callbacks. Anyways, in this case I need the client to get the response from the server, and based on the results it will do what it needs.
In other words, how do I write the client to do something like this:
IdealClient.java
// ServerResponse is a result from the server, in this case
// a list of users of the system.
public User[] callServerForInformationFromGUIWidget()
{
...
RequestPojo request = new RequestPojo(requestUserListCommand);
ChannelFuture lastWriteFuture = channel.write(request);
if(lastWriteFuture != null)
lastWriteFuture.awaitUninterruptibly();
User[] users = resultFromCallToServer();
performSomeAction(users);
}
Because the handler doesn't know who is looking for the answer, or who asked the question. And if it's done in the handler, than how?
Back to my comments about the examples, the http client (and handler) examples just dump the result to System.out. If you had a GUI how would you pass the result from your request up to the GUI? I never saw any examples for this.

Jestan is correct. In my case I have a client that need to process price tick data. I use Antlr for the parsing. I fire my events in my parser, but in my case my protocol is String based. Below is an example without Antlr, I pass the String message in your case it could be the users.
//----------------- Event --------------
public class DataChangeEvent {
private String message;
public DataChangeEvent(String message) {
this.message = message;
}
public String getMessage() {
return message;
}
}
//----------------- Listener --------------
public interface DataChangeListenter {
public void dataChangeEvent(DataChangeEvent event);
}
//----------------- Event Handler that fires the dataChange events --------------
// This class needs to be static since you need to register all your classes that want to be notified of data change events
public class DataChangedHandler {
private static List<DataChangeListenter> listeners = new ArrayList<DataChangeListenter>();
public static void registerDataChangeListener(DataChangeListenter listener) {
listeners.add(listener);
}
public static void fireDataChange(DataChangeEvent dataChangeEvent) {
for(DataChangeListenter listenter : listeners) {
listenter.dataChangeEvent(dataChangeEvent);
}
}
}
//----------------- Example class that implements the listener and registers itself for events --------------
public class ProcessMessage implements DataChangeListenter {
public ProcessMessage() {
DataChangedHandler.registerDataChangeListener(this);
}
public void dataChangeEvent(DataChangeEvent event) {
//Depending on your protocal, I use Antlr to parse my message
System.out.println(event.getMessage());
}
}
//---------------- Netty Handler -----------
public class TelnetClientHandler extends SimpleChannelHandler {
private static final Logger logger = Logger.getLogger(TelnetClientHandler.class.getName());
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) {
String message = (String) e.getMessage();
DataChangedHandler.fireDataChange(message);
}
}

You have to handle it in the Handler with messageReceived(). I'm not sure what your issue is exactly. My guess is you have a response to a request that changes depending on what request was made? Maybe a concrete description of something you are doing of a response that has to know what request it came from. One thing you might be able to do is to pass a long living object the handler that knows the outstanding request, and it can match up the response when it receives it. The pipeline factory method can pass a reference to a manager type object to the Handler.
This was pretty much what I was trying to say. Your Handler is created in the PipelineFactory which is easy to pass parameters to the Handler from there:
bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
public ChannelPipeline getPipeline() throws Exception {
ChannelPipeline pipeline = Channels.pipeline();
pipeline.addLast("framer", new DelimiterBasedFrameDecoder(8192, Delimiters.nulDelimiter()));
pipeline.addLast("decoder", new XMLDecoder() );
pipeline.addLast("encoder", new XMLEncoder() );
// notice here I'm passing two objects to the Handler so it can
// call the UI.
pipeline.addLast("handler", new MyHandler(param1, param2));
return pipeline;
}
});
When you create your pipeline you'll add your Handler upon a new connection. Simply pass one or more objects that allows it to communicate back to the UI or a controller.

Design pattern to handle an asynchronous response in Java

I read answers from similar Q&A
How do you create an asynchronous HTTP request in JAVA? |
Asynchronous programming design pattern |
AsyncTask Android - Design Pattern and Return Values
I see a lot of solutions , but none really satifies me.
Listener way
Once the results are caught, the processing is implemented in onResult method.
public interface GeolocationListener {
public void onResult(Address[] addresses);
public void onError(Exception e);
}
This solution doesn't quite satify me , because I want to handle the results in the main method. I hate this interface because when the response is returned, it is processed in onResult resulting in chains of processing and no way to go back to the "main" method.
The servlet way
public class SignGuestbookServlet extends HttpServlet {
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
// ...
resp.sendRedirect("/guestbook.jsp");
}
}
There is no exposed Java code calling the servlet. All the configuration is done in the web.xml
The way I want
Wait for the response like this
Response a = getResponse();
// wait until the response is received, do not go further
// process
Response b = getResponse();
// wait until the response is received, do not go further
process(a,b);
Is there a design pattern to handle the async request and wait for the response like above ? Other way than the listener.
Please no library or framework.
EDIT
Thanks so far the responses. I didn't give you the full picture so I exposed the Geolocation class
I started the implementation . I don't know how to implement the method . Can someone shows "how to" ? He (or she) must also implement the listener to retrieve the results
private Address getFullAddress (String text, AddressListener listener, ... ){
// new Geolocation(text, listener, options).start()
// implements Geolocation.GeolocationListener
// how to return the Address from the onResult ?
}

First, you should not reject the first two methods you discuss. There are very good reasons people are using those techniques and you should try to learn them instead of creating new ones.
Otherwise, you should look at java.util.concurrent:
ExecutorService es = Executors.newFixedThreadPool(2);
...
Future<Response> responseA = es.submit(responseGetter);
Future<Response> responseB = es.submit(responseGetter);
process(responseA.get(), responseB.get());
where responseGetter is of type Callable<Response> (you must implement the method public Response call()).

Asynchronous code can always be made synchronous. The simplest/crudest way is to make the async call, then enter a while loop that just sleeps the current thread until the value comes back.
Edit: Code that turns an asynchronous callback into synchronous code--again, a crude implementation:
import java.util.concurrent.*;
public class MakeAsynchronousCodeSynchronous {
public static void main(String[] args) throws Exception {
final Listener listener = new Listener();
Runnable delayedTask = new Runnable() {
#Override
public void run() {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new IllegalStateException("Shouldn't be interrupted", e);
}
listener.onResult(123);
}
};
System.out.println(System.currentTimeMillis() + ": Starting task");
Executors.newSingleThreadExecutor().submit(delayedTask);
System.out.println(System.currentTimeMillis() + ": Waiting for task to finish");
while (!listener.isDone()) {
Thread.sleep(100);
}
System.out.println(System.currentTimeMillis() + ": Task finished; result=" + listener.getResult());
}
private static class Listener {
private Integer result;
private boolean done;
public void onResult(Integer result) {
this.result = result;
this.done = true;
}
public boolean isDone() {
return done;
}
public Integer getResult() {
return result;
}
}
}
You could also use a CountDownLatch as recommended by hakon's answer. It will do basically the same thing. I would also suggest you get familiar with the java.util.concurrent package for a better way to manage threads. Finally, just because you can do this doesn't make it a good idea. If you're working with a framework that's based on asynchronous callbacks, you're probably much better off learning how to use the framework effectively than trying to subvert it.

Could CountDownLatch help you? In the main method, you call getResponse and then countDownLatch.await(). Pass a count down latch to the getResponse method and then count down once getResponse the result from getResponse is finished:
CountDownLatch latch = new CountDownLatch(1);
Response a = getResponse(latch);
latch.await();
latch = new CountDownLatch(1);
Response b = getResponse(latch);
latch.await();
process(a, b);
Your getResponse needs to call latch.countDown() once it's asynch parts return a result.
e.g.:
public Response getResponse(CountDownLatch latch) {
someAsychBloc(final CountDownLatch latch) {
do work
latch.countDown();
}
}

Essentially you need a "listener" of sorts no matter what. This is because you do not know WHEN your return message will come back, if at all (that is one of the downsides of asynchronous processing...what to do if you do not get a return message).
So you either need to implement a listener that waits for events (ie, it is nudged by the returning message to be processed).
Or you could do a hybrid on that by having a separate thread that "polls" (or pulls) a response area on your service to see if the return message exists.
So it really comes down to whether you want more of a "pull" or "push" method of retrieving messages.
The SCA (Service Component Architecture) framework might be something to consider, but depending on what you are doing, it could be overkill too. But something to consider.
EDIT:
I just found this in the Java SE 6 Javadocs that may be helpful. The
interface CompletionService which abstracts the very thing you care
about --> asynchronous work. I suggest you take a look.

If you want a page flow in a web application, you have to handle in the web way : storing some data either in the session, or cookies or hidden fields, etc.
The problem you're trying to tackle, as far as I understand it, doesn't come from asynchronousness but from the stateless http protocole.
Regards,
Stéphane