We Keep Coding

multiple volley requests at same time

I need to send 4 http requests at same time and wait until all of them finished (i'm using volley)
I've tried to send them separately in 4 threads and use thread.join but it seems that onResponse and onError methods are running in main thread so the request threads finishes after call queue.add(jsonArrayRequest).
I can't use countdownlatch because as I know first it doesn't run threads at same time (it runs them in a queue) and second it blocks main thread.
what's your suggestion? let me know if there's better way to do this using Retrofit , OkHttp or other libraries.

To achieve it without using any patterns or other libraries, you can mark the request as finished if it responded, and call the method, in each of them, you want to execute if all the requests are finished. On that method, you just need to check if all the requests are done.
Example:
isRequest1Finished = false;
isRequest2Finished = false;
response1 = null;
response2 = null;
volleyRequest1(new Response.Listener<Something>() {
#Override
public void onResponse(Something response) {
isRequest1Finished = true;
response1 = response;
doSomething();
}
})
volleyRequest2(new Response.Listener<Something>() {
#Override
public void onResponse(Something response) {
isRequest2Finished = true;
response2 = response;
doSomething();
}
})
//....do on all of your requests
and in your doSomething() method:
void doSomething() {
if (isRequest1Finished && isRequest2Finished) {
// do something with the response1, response2, etc....
}
}
But my suggestion is using RxJava, where you can apply zip operator, in which it combines all of your asynchronous responses into one result:
Example:
Observable request1 = getRequest1();
Observable request2 = getRequest2();
Observable.zip(request1, request2,
new BiFunction<Something, Something, Pair<Something, Something>() {
#Override
public Pair<Something, Something> apply(Something response1, Something response2) {
return new Pair(response1, response2); // you can create a custom object to handle all of the responses
}
})
.map( // do something with your responses )

How to use RetroFit2 to return any data in OnResponse

I'm new with retrofit2 and I don't know how to return anything on the method OnResponse.
I have these code:
public boolean verificarUsuario (String login, String senha){
Retrofit retrofit = new Retrofit.Builder().baseUrl(BASE_URL).addConverterFactory(GsonConverterFactory.create(g)).build();
usuarioService service = retrofit.create(usuarioService.class);
retrofit2.Call<Boolean> user = service.verificarUsuario(login,senha);
user.enqueue(new Callback<Boolean>() {
#Override
public void onResponse(Call<Boolean> call, Response<Boolean> response) {
boolean result = response.body();
}
#Override
public void onFailure(Call<Boolean> call, Throwable t) {
}
});
}
And I need to return the variable result when my method atualizarUsuario is called.
Here I have the interface of my method verificarUsuario:
#GET("get/usuarios/login/{login,senha}")
Call<Boolean> verificarUsuario(#Path("login") String login, #Path("senha") String senha);
And here I have the Json of my method verificarUsuario:
#GET
#Produces(MediaType.APPLICATION_JSON)
#Path("/verificarUsuario/")
public String verificarUsuario(#QueryParam("login")String login, #QueryParam("senha") String senha) {
usuarioDAO dao = new usuarioDAO();
if(dao.verificarUsuario(login,senha)){
return "true";
}else{
return "false";
}
}
Does anybody know how to do it?

Retrofit offers two types of calls, asynchronous calls and synchronous calls. Asynchronous calls are triggered by enqueue, which is the method you are calling, and let you pass in a Callback object to tell it what should happen when the call is done. If you want the calling method to wait for the response to come back and then return something based on that, you'll want to use the synchronous method instead, which is execute. However, you can't call execute on the UI thread in android, so if this is happening on the main thread, you'll need to make verificarUsuario return void, and then trigger whatever result you want in the callback you pass into enqueue

Execute each subtask in parallel in a multithreaded environment

I am working on a library which will take an object DataRequest as an input parameter and basis on that object, I will construct an URL and then make a call to our app servers using apache http client and then return the response back to the customer who is using our library. Some customer will call the executeSync method to get the same feature and some customer will call our executeAsync method to get the data.
executeSync() - waits until I have a result, returns the result.
executeAsync() - returns a Future immediately which can be processed after other things are done, if needed.
Below is my DataClient class which has above two methods:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16);
private CloseableHttpClient httpClientBuilder;
// initializing httpclient only once
public DataClient() {
try {
RequestConfig requestConfig =
RequestConfig.custom().setConnectionRequestTimeout(500).setConnectTimeout(500)
.setSocketTimeout(500).setStaleConnectionCheckEnabled(false).build();
SocketConfig socketConfig =
SocketConfig.custom().setSoKeepAlive(true).setTcpNoDelay(true).build();
PoolingHttpClientConnectionManager poolingHttpClientConnectionManager =
new PoolingHttpClientConnectionManager();
poolingHttpClientConnectionManager.setMaxTotal(300);
poolingHttpClientConnectionManager.setDefaultMaxPerRoute(200);
httpClientBuilder =
HttpClientBuilder.create().setConnectionManager(poolingHttpClientConnectionManager)
.setDefaultRequestConfig(requestConfig).setDefaultSocketConfig(socketConfig).build();
} catch (Exception ex) {
// log error
}
}
#Override
public List<DataResponse> executeSync(DataRequest key) {
List<DataResponse> responsList = null;
Future<List<DataResponse>> responseFuture = null;
try {
responseFuture = executeAsync(key);
responsList = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException | ExecutionException | InterruptedException ex) {
responsList =
Collections.singletonList(new DataResponse(DataErrorEnum.CLIENT_TIMEOUT,
DataStatusEnum.ERROR));
responseFuture.cancel(true);
// logging exception here
}
return responsList;
}
#Override
public Future<List<DataResponse>> executeAsync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.submit(task);
}
}
Below is my DataFetcherTask class which also has a static class DataRequestTask which calls our app servers by making URL:
public class DataFetcherTask extends RecursiveTask<List<DataResponse>> {
private final DataRequest key;
private final CloseableHttpClient httpClientBuilder;
public DataFetcherTask(DataRequest key, CloseableHttpClient httpClientBuilder) {
this.key = key;
this.httpClientBuilder = httpClientBuilder;
}
#Override
protected List<DataResponse> compute() {
// Create subtasks for the key and invoke them
List<DataRequestTask> requestTasks = requestTasks(generateKeys());
invokeAll(requestTasks);
// All tasks are finished if invokeAll() returns.
List<DataResponse> responseList = new ArrayList<>(requestTasks.size());
for (DataRequestTask task : requestTasks) {
try {
responseList.add(task.get());
} catch (InterruptedException | ExecutionException e) {
Thread.currentThread().interrupt();
return Collections.emptyList();
}
}
return responseList;
}
private List<DataRequestTask> requestTasks(List<DataRequest> keys) {
List<DataRequestTask> tasks = new ArrayList<>(keys.size());
for (DataRequest key : keys) {
tasks.add(new DataRequestTask(key));
}
return tasks;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
/** Inner class for the subtasks. */
private static class DataRequestTask extends RecursiveTask<DataResponse> {
private final DataRequest request;
public DataRequestTask(DataRequest request) {
this.request = request;
}
#Override
protected DataResponse compute() {
return performDataRequest(this.request);
}
private DataResponse performDataRequest(DataRequest key) {
MappingHolder mappings = DataMapping.getMappings(key.getType());
List<String> hostnames = mappings.getAllHostnames(key);
for (String hostname : hostnames) {
String url = generateUrl(hostname);
HttpGet httpGet = new HttpGet(url);
httpGet.setConfig(generateRequestConfig());
httpGet.addHeader(key.getHeader());
try (CloseableHttpResponse response = httpClientBuilder.execute(httpGet)) {
HttpEntity entity = response.getEntity();
String responseBody =
TestUtils.isEmpty(entity) ? null : IOUtils.toString(entity.getContent(),
StandardCharsets.UTF_8);
return new DataResponse(responseBody, DataErrorEnum.OK, DataStatusEnum.OK);
} catch (IOException ex) {
// log error
}
}
return new DataResponse(DataErrorEnum.SERVERS_DOWN, DataStatusEnum.ERROR);
}
}
}
For each DataRequest object there is a DataResponse object. Now once someone calls our library by passing DataRequest object, internally we make List<DataRequest> object and then we invoke each DataRequest object in parallel and return List<DataResponse> back where each DataResponse object in the list will have response for corresponding DataRequest object.
Below is the flow:
Customer will call DataClient class by passing DataRequest object. They can call executeSync() or executeAsync() method depending on their requirements.
Now in the DataFetcherTask class (which is a RecursiveTask one of ForkJoinTask's subtypes), given a key object which is a single DataRequest, I will generate List<DataRequest> and then invokes each subtask in parallel for each DataRequest object in the list. These subtasks are executed in the same ForkJoinPool as the parent task.
Now in the DataRequestTask class, I am executing each DataRequest object by making an URL and return its DataResponse object back.
Problem Statement:
Since this library is being called in a very high throughput environment so it has to be very fast. For synchronous call, executing in a separate thread is ok here? It will incur extra costs and resources for a Thread along with the cost of context switch of threads in this case so I am little bit confuse. Also I am using ForkJoinPool here which will save me in using extra thread pool but is it the right choice here?
Is there any better and efficient way to do the same thing which can be performance efficient as well? I am using Java 7 and have access to Guava library as well so if it can simplify anything then I am open for that as well.
It looks like we are seeing some contention when it runs under very heavy load. Is there any way this code can go into thread contention when runs under very heavy load?

I think in your situation it's better to use async http call, see link: HttpAsyncClient. And you don't need to use thread pool.
In executeAsync method create empty CompletableFuture<DataResponse>() and pass it to client call, there in callback call set the result of completableFuture by calling complete on it (or completeExceptionally if exceptions raise).
ExecuteSync method implementation looks good.
edit:
For java 7 it's only need to replace a completableFuture to promise implementation in guava, like ListenableFuture or anything similar

The choice to use the ForkJoinPool is correct, its designed for efficiency with many small tasks:
A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute tasks submitted to the pool and/or created by other active tasks (eventually blocking waiting for work if none exist). This enables efficient processing when most tasks spawn other subtasks (as do most ForkJoinTasks), as well as when many small tasks are submitted to the pool from external clients. Especially when setting asyncMode to true in constructors, ForkJoinPools may also be appropriate for use with event-style tasks that are never joined.
I suggest to try the asyncMode = true in the constructor since in your case the tasks are never joined:
public class DataClient implements Client {
private final ForkJoinPool forkJoinPool = new ForkJoinPool(16, ForkJoinPool.ForkJoinWorkerThreadFactory, null, true);
...
}
For the executeSync() you can use the forkJoinPool.invoke(task), this is the managed way to do a synchronous task execution in the pool for resources optimisation:
#Override
public List<DataResponse> executeSync(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.httpClientBuilder);
return this.forkJoinPool.invoke(task);
}
If you can use Java 8 then there is a common pool already optimised: ForkJoinPool.commonPool()

Java - Converting wait/notify to java.util.concurrent

I am having hard time converting old fashioned wait notify with spurious waits to java.util.concurrent API
First Problem: What to use, Future or CountdownLatch or CyclicBarrier according to this question
Second Question: How to use it? Because in all the examples I have looked at are converting a single async method to sync which is not a problem
Thirdly: What is the best option in my case out of Future task's get method, CountDownLatch or CyclicBarrier, since I dont have multiple threads, but only 2.
My async code
Main class:
public static void main(String[] args) throws InterruptedException {
Request req = new Request(1);
Thread tReq = new Thread(req);
tReq.start();
synchronized(req){
req.wait();
}
LogProperties.log.info("Response is: " + req.responseString);
}
Request Class:
public class Request implements Runnable {
private int requestID;
public boolean isComplete;
public String responseString;
public Request(int id) {
this.requestID = id;
}
#Override
public void run() {
FutureTest.hmTest.put(requestID, this);
try {
//simulate a request
Thread.sleep(10000);
} catch (InterruptedException ex) {
}
Response response = new Response(requestID);
Thread tResponse = new Thread(response);
tResponse.start();
}
}
Response Class:
public class Response implements Runnable {
int id;
public Response(int responseId) {
this.id = responseId;
}
#Override
public void run() {
Request req = (Request) FutureTest.hmTest.get(id);
req.isComplete = true;
req.responseString = "Request for id [" + id + "] has been completed";
synchronized(req){
req.notify();
}
}
}
My Problem with using future callable and CyclicBarrier is that Im not returning a variable, I want to wait on a object, which is of type Request in this case, so what is the best solution

One of the most versatile means for threads to communicate is a BlockingQueue.
In your case, you have one thread that creates a "response" (i.e., a producer), and you have another thread that is waiting for the "response" (a consumer). One way to implement that is for the producer to put() the response into a BlockingQueue, and have the consumer take() the response out of the queue.
The take() operation will implicitly wait for the response to become available before it returns.

I think Pipes can be ideal for this, it can achieve synchronous communication easily.
Check out this link with pipes for producer consumer problem- http://www.informit.com/articles/article.aspx?p=26326&seqNum=10

Better way of implementing synchronous and asynchronous behavior in a client code?

I am working on a project in which I am supposed to make synchronous and asynchronous behavior of my client. In general, how our client will work as below -
Customer will call our client with a userId and we will construct a URL from that userId and make a HTTP call to that URL and we will get a JSON String back after hitting the URL. And after we get the response back as a JSON String, then we will send that JSON String back to our customer.
So in this case, as I mentioned above, I need to have synchronous and asynchronous methods, some customer will call synchronous method to get the same feature and some will call our asynchronous method to get the data back.
So now I am thinking, what is the best way to implement synchronous and asynchronous feature in a same client code. I know there might be different answers depending on how people implement but as a developer there might be better way of implementing it as the way currently I am doing.
I am still learning so wanted to know better way of doing this by which I can also tell other people that to solve these kind of problems, you need to do it like this.
For now, I have created an interface like this -
public interface Client {
// for synchronous
public String execute(final String userId);
// for asynchronous
public Future<String> executeAsync(final String userId);
}
And then I have our SmartClient which implements the above Client interface. I am not sure whether I should be doing like this or there is a better way of implementing synchronous and asynchronous feature.
Below is my SmartClient code and other high level code inside it which I think is sufficient to understand the whole flow.
public class SmartClient implements Client {
ExecutorService executor = Executors.newFixedThreadPool(5);
// This is for synchronous call
#Override
public String execute(String userId) {
String response = null;
Future<String> future = executor.submit(new Task(userId));
try {
response = future.get(3, TimeUnit.SECONDS);
} catch (TimeoutException e) {
System.out.println("Terminated!");
}
return response;
}
// This is for asynchronous call
#Override
public Future<String> executeAsync(String userId) {
// not sure what should I do here as well?
}
}
Below is my simple class which will perform the actual task -
class Task implements Callable<String> {
private final String userId;
public Task(String userId) {
this.userId = userId;
}
public String call() throws Exception {
String url = createURL(userId);
// make a HTTP call to the URL
RestTemplate restTemplate = new RestTemplate();
String jsonResponse = restTemplate.getForObject(url , String.class);
return jsonResponse;
}
// create a URL
private String createURL(String userId) {
String generateURL = somecode;
return generateURL;
}
}
But I was doing some research and I found out that the better way is -
Instead of having two methods one is synchronous and other is
asynchronous, have one interface method describing the synchronous
behavior. Write an implementation for it. And then provide a wrapper
class that wraps around the implementation of your interface that
basically calls a wrapped object's method while providing the
asynchronous behavior.
But somehow I am not able to understand how to accomplish this? Can anyone provide a simple example basis on my example which will help me to understand better how to do that?
I might be able to learn something new with this.

try this
#Override
public Future<String> executeAsync(String userId) {
return executor.submit(new Task(userId));
}