We Keep Coding

ExecutorService does not work but individually creating threads work

I have a REST API where I am supposed to fetch large amount of data from an external API. I decided to try multi-threading to optimize the whole fetch-parse-persist cycle. But I am facing trouble with ExecutorService (I have not used it prior to this work). I am sharing the classes and relevant part of whole process
public class LogRetrievingService implements Runnable {
CustomHttpClient client;
public LogRetrievingService(CustomHttpClient client) {
this.client = client;
}
#Override
public void run() {
Response response = client.invokeExternalApi();
parse(response.readEntity(bytes[].class);
}
//skipping parse() for brevity, it basically selects some columns and sends them to DBwriter
My REST API resource is like this
public class LogRetrieverResource {
private CustomHttpClient client;
public LogRetrieverResource(CustomHttpClient client) {
this.client = client;
}
//this does not work
public void initLogRetrieval() {
ExecutorService service = Executors.newFixedThreadPool(4); //max_thread
for(int i = 0; i < 4; i++) {
service.submit(new LogRetrievingService (client));
}
}
//THIS WORKS
public void initLogRetrieval() {
for(int i = 0; i < 4; i++) {
Thread thread = new Thread(new LogRetrievingService(client));
thread.start();
}
}
}
Now when I hit my resource nothing happens, I can see that client's logs are hit but it does not go and fetch the data.
However if inside the loop of my LogRetrieverResource class I create a new Thread instance with same run method then multithreaded data fetching works as expected. Could someone point me in what am I doing wrong? I have no prior experience of using multithreading in java other than implementing the Runnable interface method.
edit: Adding client class details
import javax.ws.rs.client.Client;
public class CustomHttpClient {
public Response invokeExternalAPI() {
return client
.target("url") //removing url for confidentiality
.request()
.accept(MediaType.APPLICATION_JSON)
.cookie("SSO",<token>)
.get();
}
}

Just trying to notice the differences, and there shouldn't be much difference. First at the end of your executor submission loop add service.shutdown(). Then you will pretty much be doing exactly the same thing.
The next issue, exceptions are handled a bit differently. An executor service will capture all of the exceptions. For debugging purposes you can try.
service.submit(
() -> {
try{
new LogRetrievingService (client).run();
} catch(Exception e){
//log the exception so you can see if anything went wrong.
}
});
This is not the way to handle exceptions with an ExecutorService though, you should grab the future that you submitted and use that to handle any errors. Also, I suspect spring has some tools for doing this type of work.

Execute each subtask in parallel in a multithreaded environment

I am working on a library which will take an object DataRequest as an input parameter and basis on that object, I will construct an URL and then make a call to our app servers using apache http client and then return the response back to the customer who is using our library. Some customer will call the executeSync method to get the same feature and some customer will call our executeAsync method to get the data.
executeSync() - waits until I have a result, returns the result.
executeAsync() - returns a Future immediately which can be processed after other things are done, if needed.
Below is my DataClient class which has above two methods:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16);
private CloseableHttpClient httpClientBuilder;
// initializing httpclient only once
public DataClient() {
try {
RequestConfig requestConfig =
RequestConfig.custom().setConnectionRequestTimeout(500).setConnectTimeout(500)
.setSocketTimeout(500).setStaleConnectionCheckEnabled(false).build();
SocketConfig socketConfig =
SocketConfig.custom().setSoKeepAlive(true).setTcpNoDelay(true).build();
PoolingHttpClientConnectionManager poolingHttpClientConnectionManager =
new PoolingHttpClientConnectionManager();
poolingHttpClientConnectionManager.setMaxTotal(300);
poolingHttpClientConnectionManager.setDefaultMaxPerRoute(200);
httpClientBuilder =
HttpClientBuilder.create().setConnectionManager(poolingHttpClientConnectionManager)
.setDefaultRequestConfig(requestConfig).setDefaultSocketConfig(socketConfig).build();
} catch (Exception ex) {
// log error
}
}
#Override
public List<DataResponse> executeSync(DataRequest key) {
List<DataResponse> responsList = null;
Future<List<DataResponse>> responseFuture = null;
try {
responseFuture = executeAsync(key);
responsList = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException | ExecutionException | InterruptedException ex) {
responsList =
Collections.singletonList(new DataResponse(DataErrorEnum.CLIENT_TIMEOUT,
DataStatusEnum.ERROR));
responseFuture.cancel(true);
// logging exception here
}
return responsList;
}
#Override
public Future<List<DataResponse>> executeAsync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.submit(task);
}
}
Below is my DataFetcherTask class which also has a static class DataRequestTask which calls our app servers by making URL:
public class DataFetcherTask extends RecursiveTask<List<DataResponse>> {
private final DataRequest key;
private final CloseableHttpClient httpClientBuilder;
public DataFetcherTask(DataRequest key, CloseableHttpClient httpClientBuilder) {
this.key = key;
this.httpClientBuilder = httpClientBuilder;
}
#Override
protected List<DataResponse> compute() {
// Create subtasks for the key and invoke them
List<DataRequestTask> requestTasks = requestTasks(generateKeys());
invokeAll(requestTasks);
// All tasks are finished if invokeAll() returns.
List<DataResponse> responseList = new ArrayList<>(requestTasks.size());
for (DataRequestTask task : requestTasks) {
try {
responseList.add(task.get());
} catch (InterruptedException | ExecutionException e) {
Thread.currentThread().interrupt();
return Collections.emptyList();
}
}
return responseList;
}
private List<DataRequestTask> requestTasks(List<DataRequest> keys) {
List<DataRequestTask> tasks = new ArrayList<>(keys.size());
for (DataRequest key : keys) {
tasks.add(new DataRequestTask(key));
}
return tasks;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
/** Inner class for the subtasks. */
private static class DataRequestTask extends RecursiveTask<DataResponse> {
private final DataRequest request;
public DataRequestTask(DataRequest request) {
this.request = request;
}
#Override
protected DataResponse compute() {
return performDataRequest(this.request);
}
private DataResponse performDataRequest(DataRequest key) {
MappingHolder mappings = DataMapping.getMappings(key.getType());
List<String> hostnames = mappings.getAllHostnames(key);
for (String hostname : hostnames) {
String url = generateUrl(hostname);
HttpGet httpGet = new HttpGet(url);
httpGet.setConfig(generateRequestConfig());
httpGet.addHeader(key.getHeader());
try (CloseableHttpResponse response = httpClientBuilder.execute(httpGet)) {
HttpEntity entity = response.getEntity();
String responseBody =
TestUtils.isEmpty(entity) ? null : IOUtils.toString(entity.getContent(),
StandardCharsets.UTF_8);
return new DataResponse(responseBody, DataErrorEnum.OK, DataStatusEnum.OK);
} catch (IOException ex) {
// log error
}
}
return new DataResponse(DataErrorEnum.SERVERS_DOWN, DataStatusEnum.ERROR);
}
}
}
For each DataRequest object there is a DataResponse object. Now once someone calls our library by passing DataRequest object, internally we make List<DataRequest> object and then we invoke each DataRequest object in parallel and return List<DataResponse> back where each DataResponse object in the list will have response for corresponding DataRequest object.
Below is the flow:
Customer will call DataClient class by passing DataRequest object. They can call executeSync() or executeAsync() method depending on their requirements.
Now in the DataFetcherTask class (which is a RecursiveTask one of ForkJoinTask's subtypes), given a key object which is a single DataRequest, I will generate List<DataRequest> and then invokes each subtask in parallel for each DataRequest object in the list. These subtasks are executed in the same ForkJoinPool as the parent task.
Now in the DataRequestTask class, I am executing each DataRequest object by making an URL and return its DataResponse object back.
Problem Statement:
Since this library is being called in a very high throughput environment so it has to be very fast. For synchronous call, executing in a separate thread is ok here? It will incur extra costs and resources for a Thread along with the cost of context switch of threads in this case so I am little bit confuse. Also I am using ForkJoinPool here which will save me in using extra thread pool but is it the right choice here?
Is there any better and efficient way to do the same thing which can be performance efficient as well? I am using Java 7 and have access to Guava library as well so if it can simplify anything then I am open for that as well.
It looks like we are seeing some contention when it runs under very heavy load. Is there any way this code can go into thread contention when runs under very heavy load?

I think in your situation it's better to use async http call, see link: HttpAsyncClient. And you don't need to use thread pool.
In executeAsync method create empty CompletableFuture<DataResponse>() and pass it to client call, there in callback call set the result of completableFuture by calling complete on it (or completeExceptionally if exceptions raise).
ExecuteSync method implementation looks good.
edit:
For java 7 it's only need to replace a completableFuture to promise implementation in guava, like ListenableFuture or anything similar

The choice to use the ForkJoinPool is correct, its designed for efficiency with many small tasks:
A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute tasks submitted to the pool and/or created by other active tasks (eventually blocking waiting for work if none exist). This enables efficient processing when most tasks spawn other subtasks (as do most ForkJoinTasks), as well as when many small tasks are submitted to the pool from external clients. Especially when setting asyncMode to true in constructors, ForkJoinPools may also be appropriate for use with event-style tasks that are never joined.
I suggest to try the asyncMode = true in the constructor since in your case the tasks are never joined:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16, ForkJoinPool.ForkJoinWorkerThreadFactory, null, true);
...
}
For the executeSync() you can use the forkJoinPool.invoke(task), this is the managed way to do a synchronous task execution in the pool for resources optimisation:
#Override
public List<DataResponse> executeSync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.invoke(task);
}
If you can use Java 8 then there is a common pool already optimised: ForkJoinPool.commonPool()

How to read Message in netty in other class

I want to read a message at a specific position in an class other than InboundHandler. I can't find a way to read it expect in the channelRead0 method, which is called from the netty framework.
For example:
context.writeMessage("message");
String msg = context.readMessage;
If this is not possible, how can I map a result, which I get in the channelRead0 method to a specific call I made in another class?

The Netty framework is designed to be asynchronously driven. Using this analogy, it can handle large amount of connections with minimal threading usage. I you are creating an api that uses the netty framework to dispatch calls to a remote location, you should use the same analogy for your calls.
Instead of making your api return the value direct, make it return a Future<?> or a Promise<?>. There are different ways of implementing this system in your application, the simplest way is creating a custom handler that maps the incoming requests to the Promises in a FIFO queue.
An example of this could be the following:
This is heavily based on this answer that I submitted in the past.
We start with out handler that maps the requests to requests in our pipeline:
public class MyLastHandler extends SimpleInboundHandler<String> {
private final SynchronousQueue<Promise<String>> queue;
public MyLastHandler (SynchronousQueue<Promise<String>> queue) {
super();
this.queue = queue;
}
// The following is called messageReceived(ChannelHandlerContext, String) in 5.0.
#Override
public void channelRead0(ChannelHandlerContext ctx, String msg) {
this.queue.remove().setSuccss(msg);
// Or setFailure(Throwable)
}
}
We then need to have a method of sending the commands to a remote server:
Channel channel = ....;
SynchronousQueue<Promise<String>> queue = ....;
public Future<String> sendCommandAsync(String command) {
return sendCommandAsync(command, new DefaultPromise<>());
}
public Future<String> sendCommandAsync(String command, Promise<String> promise) {
synchronized(channel) {
queue.offer(promise);
channel.write(command);
}
channel.flush();
}
After we have done our methods, we need a way to call it:
sendCommandAsync("USER anonymous",
new DefaultPromise<>().addListener(
(Future<String> f) -> {
String response = f.get();
if (response.startWidth("331")) {
// do something
}
// etc
}
)
);
If the called would like to use our a api as a blocking call, he can also do that:
String response = sendCommandAsync("USER anonymous").get();
if (response.startWidth("331")) {
// do something
}
// etc
Notice that Future.get() can throw an InterruptedException if the Thread state is interrupted, unlike a socket read operation, who can only be cancelled by some interaction on the socket. This exception should not be a problem in the FutureListener.

Using java futures without requiring local mutable state

I've written some code that essentially is responsible for orchestrating a number of API's in sequence through a library method I provide to my clients called "orchestrate" (yes I know so original). What sits behind this orchestrate method is nothing more than a loop that executes API's in the order they are received, which in turn are delegated to a number of classes that contain some business logic for building a request, calling an API, performing some validation on the response and finally returning the api result. So, if a client sent in a list of apis:
{a1, a2, a3, a4, a5} it would execute each api in sequence in a completely blocking way.
I'm attempting to beef this up to where I'm able to call multiple API's in parallel depending on how I receive the instructions from a client. Think of it as the client sending me a list of lists like: { {a1, a2}, {a3}, {a4, a5} }
This means I'd like to execute a1 and a2 in parallel (which means build their request, call the apis, validate the response). Then wait until i'm sure both of them are done. Then execute a3, and wait until i'm sure it's done. Finally I want to execute a4 and a5 and follow the usual pattern.
Now, I'm tempted to use futures for the simple abstraction they provide to execute methods a wait for the response using the .get() method. But what I noticed that the executorService needs underneath is future's invocation of a call() method. This is fine, but makes me think that the call() method that is implemented per class may need access to "local" data in order to do its job (after all, I can't pass the call() method any particular parameters). I really want to avoid holding any local mutable state because that brings its own side-effects.
Is there a way for me to NOT hold local state and still use futures to handle my multithreading use-case? Or is my understanding of futures completely wrong and I'm missing something obvious? If not, are there any recommendations on a good path forward with some alternatives?

OK, so you have a class which gets you some data from a web page in a blocking way, and takes some arguments:
public class DataFetcher {
public Data fetchData(int param1, int param2) {
// ...
}
}
And now you want to execute this method twice, concurrently, and get back futures. So you just need to create a Callable:
final DataFetcher fetcher = new DataFetcher();
Callable<Data> task1 = new Callable<>() {
#Override
public Data call() {
return fetcher.fetchData(1, 2);
}
};
Callable<Data> task2 = new Callable<>() {
#Override
public Data call() {
return fetcher.fetchData(3, 4);
}
};
Future<Data> result1 = executorService.submit(task1);
Future<Data> result2 = executorService.submit(task2);
I don't see any mutable state here.
To avoid repeating the code and using anonymous classes, you can define a top-level class:
public DataFetcherTask implements Callable<Data> {
private final DataFetcher fetcher;
private final int param1;
private final int param2;
public DataFetcherTask(DataFetcher fetcher, int p1, int p1) {
this.fetcher = fetcher;
this.param1 = p1;
this.param2 = p2;
}
#Override
public Data call() {
return fetcher.fetchData(param1, param2);
}
};
and then use it like this:
Future<Data> result1 = executorService.submit(new DataFetcherTask(fetcher, 1, 2));
Future<Data> result2 = executorService.submit(new DataFetcherTask(fetcher, 3, 4));
Still no trace of mutable state here.

Design pattern to handle an asynchronous response in Java

I read answers from similar Q&A
How do you create an asynchronous HTTP request in JAVA? |
Asynchronous programming design pattern |
AsyncTask Android - Design Pattern and Return Values
I see a lot of solutions , but none really satifies me.
Listener way
Once the results are caught, the processing is implemented in onResult method.
public interface GeolocationListener {
public void onResult(Address[] addresses);
public void onError(Exception e);
}
This solution doesn't quite satify me , because I want to handle the results in the main method. I hate this interface because when the response is returned, it is processed in onResult resulting in chains of processing and no way to go back to the "main" method.
The servlet way
public class SignGuestbookServlet extends HttpServlet {
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
// ...
resp.sendRedirect("/guestbook.jsp");
}
}
There is no exposed Java code calling the servlet. All the configuration is done in the web.xml
The way I want
Wait for the response like this
Response a = getResponse();
// wait until the response is received, do not go further
// process
Response b = getResponse();
// wait until the response is received, do not go further
process(a,b);
Is there a design pattern to handle the async request and wait for the response like above ? Other way than the listener.
Please no library or framework.
EDIT
Thanks so far the responses. I didn't give you the full picture so I exposed the Geolocation class
I started the implementation . I don't know how to implement the method . Can someone shows "how to" ? He (or she) must also implement the listener to retrieve the results
private Address getFullAddress (String text, AddressListener listener, ... ){
// new Geolocation(text, listener, options).start()
// implements Geolocation.GeolocationListener
// how to return the Address from the onResult ?
}

First, you should not reject the first two methods you discuss. There are very good reasons people are using those techniques and you should try to learn them instead of creating new ones.
Otherwise, you should look at java.util.concurrent:
ExecutorService es = Executors.newFixedThreadPool(2);
...
Future<Response> responseA = es.submit(responseGetter);
Future<Response> responseB = es.submit(responseGetter);
process(responseA.get(), responseB.get());
where responseGetter is of type Callable<Response> (you must implement the method public Response call()).

Asynchronous code can always be made synchronous. The simplest/crudest way is to make the async call, then enter a while loop that just sleeps the current thread until the value comes back.
Edit: Code that turns an asynchronous callback into synchronous code--again, a crude implementation:
import java.util.concurrent.*;
public class MakeAsynchronousCodeSynchronous {
public static void main(String[] args) throws Exception {
final Listener listener = new Listener();
Runnable delayedTask = new Runnable() {
#Override
public void run() {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new IllegalStateException("Shouldn't be interrupted", e);
}
listener.onResult(123);
}
};
System.out.println(System.currentTimeMillis() + ": Starting task");
Executors.newSingleThreadExecutor().submit(delayedTask);
System.out.println(System.currentTimeMillis() + ": Waiting for task to finish");
while (!listener.isDone()) {
Thread.sleep(100);
}
System.out.println(System.currentTimeMillis() + ": Task finished; result=" + listener.getResult());
}
private static class Listener {
private Integer result;
private boolean done;
public void onResult(Integer result) {
this.result = result;
this.done = true;
}
public boolean isDone() {
return done;
}
public Integer getResult() {
return result;
}
}
}
You could also use a CountDownLatch as recommended by hakon's answer. It will do basically the same thing. I would also suggest you get familiar with the java.util.concurrent package for a better way to manage threads. Finally, just because you can do this doesn't make it a good idea. If you're working with a framework that's based on asynchronous callbacks, you're probably much better off learning how to use the framework effectively than trying to subvert it.

Could CountDownLatch help you? In the main method, you call getResponse and then countDownLatch.await(). Pass a count down latch to the getResponse method and then count down once getResponse the result from getResponse is finished:
CountDownLatch latch = new CountDownLatch(1);
Response a = getResponse(latch);
latch.await();
latch = new CountDownLatch(1);
Response b = getResponse(latch);
latch.await();
process(a, b);
Your getResponse needs to call latch.countDown() once it's asynch parts return a result.
e.g.:
public Response getResponse(CountDownLatch latch) {
someAsychBloc(final CountDownLatch latch) {
do work
latch.countDown();
}
}

Essentially you need a "listener" of sorts no matter what. This is because you do not know WHEN your return message will come back, if at all (that is one of the downsides of asynchronous processing...what to do if you do not get a return message).
So you either need to implement a listener that waits for events (ie, it is nudged by the returning message to be processed).
Or you could do a hybrid on that by having a separate thread that "polls" (or pulls) a response area on your service to see if the return message exists.
So it really comes down to whether you want more of a "pull" or "push" method of retrieving messages.
The SCA (Service Component Architecture) framework might be something to consider, but depending on what you are doing, it could be overkill too. But something to consider.
EDIT:
I just found this in the Java SE 6 Javadocs that may be helpful. The
interface CompletionService which abstracts the very thing you care
about --> asynchronous work. I suggest you take a look.

If you want a page flow in a web application, you have to handle in the web way : storing some data either in the session, or cookies or hidden fields, etc.
The problem you're trying to tackle, as far as I understand it, doesn't come from asynchronousness but from the stateless http protocole.
Regards,
Stéphane