Application on which I am working is going to consume 2 REST web service in below sequence:
1) Count Records - To know the numbers of records within a particular time frame.
2) Fetch Records - Once we have number of records then we need to call this service. But this service has a threshold to fetch 10K records in a go. Lets say if first service tell me within particular time interval, it has 100K of records, then I need to call second web service 10 times in paginated way considering it's threshold is 10K in one go.
So if I will make 10 synchronous calls, my application would be too slow to respond back. So I need to a mechanism to make asynchronous calls.
I am using spring framework in the back end code and using rest template for web service call. I am looking to find the best way to make asynchronous call to the above mentioned POST web service
I have done some research and found Asynchronous method useful as below:
https://spring.io/guides/gs/async-method/
Can you please guide me if this is a right approach what I am looking at or is their a better way to make asynchronous call? Looking for your suggestions, Thanks!
What #Journycorner linked is a good start but it doesn't really show the whole picture as it only makes a single request. Working with Future is definitely on the right path. The fact that Spring 4 offers an AsyncRestTemplate that returns a Future is exactly what you want to use.
On my phone so can't write up the full code but this is roughly what you want to do.
#Component
public class SampleAsyncService {
private RestTemplate restTemplate;
private AsyncRestTemplate asyncRestTemplate;
#Value("${myapp.batchSize:1000}")
private int batchSize;
public SampleAsyncService(AsyncRestTemplate asyncRestTemplate, RestTemplate restTemplate) {
this.asyncRestTemplate = asyncRestTemplate;
this.restTemplate = restTemplate;
}
public List<Record> callForRecords() {
ResponseEntity<Integer> response = restTemplate.getForEntity("http://localhost:8081/countService",
Integer.class);
int totalRecords = response.getBody().intValue();
List<Future<ResponseEntity<List<Record>>>> futures = new ArrayList<Future<ResponseEntity<List<Record>>>>();
for (int offset = 0; offset < totalRecords;) {
ListenableFuture<ResponseEntity<List<Record>>> future = asyncRestTemplate.exchange(
"http://localhost:8081/records?startRow={}&endRow={}", HttpMethod.GET, null,
new ParameterizedTypeReference<List<Record>>() {
}, offset, batchSize);
futures.add(future);
offset = offset + batchSize;
}
int responses = 0;
List<Record> fullListOfRecords = new ArrayList<Record>();
while (responses < futures.size()) {
for (Future<ResponseEntity<List<Record>>> future : futures) {
if (future.isDone()) {
responses++;
try {
ResponseEntity<List<Record>> responseEntity = future.get();
fullListOfRecords.addAll(responseEntity.getBody());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
}
}
return fullListOfRecords;
}
public class Record {
}
}
* Update * Created complete code sample.
This is just an improvement over #shawn answer. With the implementation provided earlier, sometimes I was facing issue due to below block:
while (responses < futures.size()) {
for (Future<ResponseEntity<List<Record>>> future : futures) {
if (future.isDone()) {
responses++;
try {
ResponseEntity<List<Record>> responseEntity = future.get();
fullListOfRecords.addAll(responseEntity.getBody());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
}
Here if 6 threads are processed, sometimes same thread is allowed to enter multiple times in above block so it ends up having duplicate records in response. So to avoid such situation, I have added callback block to create final response which make sure no duplicate response, though we still need while(responses < futures.size()) loop block with future.get() to block method to return final combined response until all the asynchronous calls are processed.
#Component
public class SampleAsyncService {
private RestTemplate restTemplate;
private AsyncRestTemplate asyncRestTemplate;
#Value("${myapp.batchSize:1000}")
private int batchSize;
public SampleAsyncService(AsyncRestTemplate asyncRestTemplate, RestTemplate restTemplate) {
this.asyncRestTemplate = asyncRestTemplate;
this.restTemplate = restTemplate;
}
public List<Record> callForRecords() {
ResponseEntity<Integer> response = restTemplate.getForEntity("http://localhost:8081/countService",
Integer.class);
int totalRecords = response.getBody().intValue();
List<Future<ResponseEntity<List<Record>>>> futures = new ArrayList<Future<ResponseEntity<List<Record>>>>();
for (int offset = 0; offset < totalRecords;) {
ListenableFuture<ResponseEntity<List<Record>>> future = asyncRestTemplate.exchange(
"http://localhost:8081/records?startRow={}&endRow={}", HttpMethod.GET, null,
new ParameterizedTypeReference<List<Record>>() {
}, offset, batchSize);
future.addCallback(
new ListenableFutureCallback<ResponseEntity<ChatTranscript>>() {
#Override
public void onSuccess(ResponseEntity<ChatTranscript> response) {
fullListOfRecords.addAll(responseEntity.getBody());
log.debug("Success: " + Thread.currentThread());
}
#Override
public void onFailure(Throwable t) {
log.debug("Error: " + Thread.currentThread());
}
}
);
futures.add(future);
offset = offset + batchSize;
}
int responses = 0;
List<Record> fullListOfRecords = new ArrayList<Record>();
while (responses < futures.size()) {
for (Future<ResponseEntity<List<Record>>> future : futures) {
if (future.isDone()) {
responses++;
try {
future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
}
}
return fullListOfRecords;
}
public class Record {
}
}
If it really has to be asynchronous using the Spring Implementation seems like a smart idea.
I am assuming you need all the 100k records in a single execution so as to package all the data maybe into a file or perform some business logic in one go .
If thats not case it would be wise to reconsider the need to load all data into single execution straining the memory usage of jvm or running into Outofmemory errors.
Assuming the former, Async could be an option to execute parallel threads and capture and collate responses from each. However you need to keep an upper limit of number of threads to be executed in parallel using a "thread pool size" of a task executor.
One more reason to limit the thread size is avoid loading your partner rest webservice with too many parallel calls. Eventually the partner webservice is loading data from database, which will give optimum performance for certain limit of parallel query executions. Hope this helps!
Related
I have a REST API where I am supposed to fetch large amount of data from an external API. I decided to try multi-threading to optimize the whole fetch-parse-persist cycle. But I am facing trouble with ExecutorService (I have not used it prior to this work). I am sharing the classes and relevant part of whole process
public class LogRetrievingService implements Runnable {
CustomHttpClient client;
public LogRetrievingService(CustomHttpClient client) {
this.client = client;
}
#Override
public void run() {
Response response = client.invokeExternalApi();
parse(response.readEntity(bytes[].class);
}
//skipping parse() for brevity, it basically selects some columns and sends them to DBwriter
My REST API resource is like this
public class LogRetrieverResource {
private CustomHttpClient client;
public LogRetrieverResource(CustomHttpClient client) {
this.client = client;
}
//this does not work
public void initLogRetrieval() {
ExecutorService service = Executors.newFixedThreadPool(4); //max_thread
for(int i = 0; i < 4; i++) {
service.submit(new LogRetrievingService (client));
}
}
//THIS WORKS
public void initLogRetrieval() {
for(int i = 0; i < 4; i++) {
Thread thread = new Thread(new LogRetrievingService(client));
thread.start();
}
}
}
Now when I hit my resource nothing happens, I can see that client's logs are hit but it does not go and fetch the data.
However if inside the loop of my LogRetrieverResource class I create a new Thread instance with same run method then multithreaded data fetching works as expected. Could someone point me in what am I doing wrong? I have no prior experience of using multithreading in java other than implementing the Runnable interface method.
edit: Adding client class details
import javax.ws.rs.client.Client;
public class CustomHttpClient {
public Response invokeExternalAPI() {
return client
.target("url") //removing url for confidentiality
.request()
.accept(MediaType.APPLICATION_JSON)
.cookie("SSO",<token>)
.get();
}
}
Just trying to notice the differences, and there shouldn't be much difference. First at the end of your executor submission loop add service.shutdown(). Then you will pretty much be doing exactly the same thing.
The next issue, exceptions are handled a bit differently. An executor service will capture all of the exceptions. For debugging purposes you can try.
service.submit(
() -> {
try{
new LogRetrievingService (client).run();
} catch(Exception e){
//log the exception so you can see if anything went wrong.
}
});
This is not the way to handle exceptions with an ExecutorService though, you should grab the future that you submitted and use that to handle any errors. Also, I suspect spring has some tools for doing this type of work.
Say you have some long running task wrapped by an observable:
Observable.fromCallable(new Callable<String>() {
#Override
public String call() throws Exception {
return longRunningTask();
}
}
Is there any way to check whether the observable has been unsubscribed to determine if we should cancel the work and bail out?
More specifically, is it possible to check the status of a subscription (e.g. isUnsubscribed()) when using Observable.defer() or Observable.fromCallable()?
I'm aware that you can check subscriber.isUnsubscribed() when using Observable.create(), however, since it's ill-advised to use Observable.create(), how can this be done with other operators?
What about using Observable.doOnSubscribe(Action0) and Observable.doOnUnsubscribe(Action0). You can count the subscriptions and when there are none you can stop the job.
Greetings,
Martin
The fromCallable doesn't expose the consumer. For this, you need create with a body such as the following:
final SingleDelayedProducer<T> singleDelayedProducer =
new SingleDelayedProducer<T>(subscriber);
subscriber.setProducer(singleDelayedProducer);
try {
T result;
// computation
if (subscriber.isUnsubscribed()) {
return;
}
// more computation
result = ...
singleDelayedProducer.setValue(result);
} catch (Throwable t) {
Exceptions.throwOrReport(t, subscriber);
}
I am using following code
private void getOnlineConnections()
{
for (int i = 0; i < contacts.size(); i++)
{
final Persons person = contacts.get(i);
String queryString = null;
try {
queryString = String.format(Constants.GET_ONLINE_URL,
URLEncoder.encode(person.myId, "UTF-8"));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
ResponseCallback callback = new ResponseCallback()
{
#Override
public void onSuccess(String response)
{
response = response.trim();
}
#Override
public void onFailure(String exception)
{}
};
new MyAsyncTask(queryString, callback);
}
}
MyAsyncTask is an AsyncTask.
Now the number of contacts can be more than 128 or more than limit of TheadPoolExecutor.
How can Increase this limit so I won't get crash on RejectExecutionError.
If you are using API Level 11 or higher, create your own thread pool and use executeOnExecutor().
If you are supporting older devices, don't use AsyncTask. Create your own thread pool and use other means (e.g., runOnUiThread()) to arrange for work to be done on the main application thread.
This is something that sounds more like a thread than a task- I'd start off a thread and just have it chug through those contacts one at a time in the background. You can even partition the dataset into multiple chunks and create 1 thread per chunk if you want it to process in parallel (although unless the linux scheduler assigns those threads to different cores you won't actually achieve a speedup this way, so I'm not sure I'd bother unless its absolutely needed, and then I'd measure to make sure it worked).
I'm testing a Web application written in Java EE using servlets (version 7). I'm sending a lot of HTTP requests to my servlets and I want to know when all requests are completed.
To send requests I'm using an executor. Also I don't know if it is the most efficient way to do this.
for (int i=0; i < numRequests; i++) {
ExecutorService executor = Executors.newFixedThreadPool(1);
Future<util.Response> responseFromServlet = null;
responseFromServlet = executor.submit(new util.Request(new URL(url)));
if ( i != numRequests -1 ) {
executor.shutdown();
} else {
responseFromServlet.get().getBody(); // suspensive call for last invocation
executor.shutdown();
}
}
Actually the executor waits the end of the last invoked HTTP request but it usually is not the last one that completes.
I think that creating a new thread waiting for response of each HTTP servlet is crazy. I can't generate 100-200-300 threads, one for each request!
So is there a way to understand when all servlets end their execution? If needed, I can modify my servlets.
=== Edit ===
To be more precise, here is the Request class implementation:
public class Request implements Callable<Response> {
private URL url;
public Request(URL url) {
this.url = url;
}
#Override
public Response call() throws Exception {
return new Response(url.openStream());
}
}
And this it the Response class:
public class Response {
private InputStream body;
public Response(InputStream body) {
this.body = body;
}
public InputStream getBody() {
return body;
}
}
Using an executor is fine, you may want to increase the size of the ThreadPool though to have more concurrent threads performing your requests.
Use a CoutnDownLatch initialised with numRequests which sits waiting for all the threads to complete.
util.Request must call latch.countDown() in its runmethod
The code would look like this (handwritten - not tested)
ExecutorService executor = Executors.newFixedThreadPool(n);
final CountDownLatch latch = new CountDownLatch(numRequests);
for (int i=0; i < numRequests; i++) {
executor.submit(new util.Request(new URL(url), latch));
}
latch.await(someValue, TimeUnit.SECONDS)
`
Edit
Re-implement util.Request doing something like
public class Request implements Callable<Response> {
final private URL url;
final private CountDownLatch latch;
public Request(URL url, CountDownLatch latch) {
this.url = url;
this.latch = latch;
}
#Override
public Response call() throws Exception {
try {
return new Response(url.openStream());
}
catch (Exception e) {
//do something useful
}
finally {
latch.countDown();
}
}
}
You may want to consume the stream of you response before you countDown the latch to verify that you get what you expect as a response from your server.
If you are using this program to perform a load test, or even otherwise, I'd highly recommend that you use Jmeter instead. Jmeter already does what you are attempting to do and there are many plugins that will allow you to schedule the load / number of thread / time period etc. You can also monitor all HTTP requests through a variety of graphs.
Writing a test for your servlet should take you less than 5 minutes. The graphs are also easy to generate.
If you'd still like to use your custom program to contact the servlet, you can always limit the number of requests and back them up with a blocking queue through a threadpool executor.
Lastly, do not modify the servlet. You should be able to monitor it as a black box.
My current code uses series of asynchronous processes that culminate in results. I need to wrap each of these in such a way that each is accessed by a synchronous method with the result as a return value. I want to use executor services to do this, so as to allow many of these to happen at the same time. I have the feeling that Future might be pertinent to my implementation, but I can't figure out a good way to make this happen.
What I have now:
public class DoAJob {
ResultObject result;
public void stepOne() {
// Passes self in for a callback
otherComponent.doStepOne(this);
}
// Called back by otherComponent once it has completed doStepOne
public void stepTwo(IntermediateData d) {
otherComponent.doStepTwo(this, d);
}
// Called back by otherComponent once it has completed doStepTwo
public void stepThree(ResultObject resultFromOtherComponent) {
result = resultFromOtherComponent;
//Done with process
}
}
This has worked pretty well internally, but now I need to map my process into a synchronous method with a return value like:
public ResultObject getResult(){
// ??? What goes here ???
}
Does anyone have a good idea about how to implement this elegantly?
If you want to turn an asynchronous operation (which executes a callback when finished), into a synchronous/blocking one, you can use a blocking queue. You can wrap this up in a Future object if you wish.
Define a blocking queue which can hold just one element:
BlockingQueue<Result> blockingQueue = new ArrayBlockingQueue<Result>(1);
Start your asynchronous process (will run in the background), and write the callback such that when it's done, it adds its result to the blocking queue.
In your foreground/application thread, have it take() from the queue, which blocks until an element becomes available:
Result result = blockingQueue.take();
I wrote something similar before (foreground thread needs to block for an asynchronous response from a remote machine) using something like a Future, you can find example code here.
I've done something similar with the Guava library; these links might point you in the right direction:
Is it possible to chain async calls using Guava?
https://code.google.com/p/guava-libraries/wiki/ListenableFutureExplained
If you like to get your hands dirty, you can do this
ResultObject result;
public void stepOne()
otherComponent.doStepOne(this);
synchronized(this)
while(result==null) this.wait();
return result;
public void stepThree(ResultObject resultFromOtherComponent)
result = resultFromOtherComponent;
synchronized(this)
this.notify();
Or you can use higher level concurrency tools, like BlockingQueue, Semaphore, CountdownLatch, Phaser, etc etc.
Note that DoAJob is not thread safe - trouble ensured if two threads call stepOne at the same time.
I recommend using invokeAll(..). It will submit a set of tasks to the executor, and block until the last one completes (successfully/with exception). It then returns a list of completed Future objects, so you can loop on them and merge the results into a single ResultObject.
In you wish to run only a single task in a synchronous manner, you can use the following:
executor.invokeAll(Collections.singleton(task));
--edit--
Now I think I understand better your needs. I assume that you need a way to submit independent sequences of tasks. Please take a look at the code I posted in this answer.
Bumerang is my async only http request library which is constructed for Android http requests using Java -> https://github.com/hanilozmen/Bumerang . I needed to make synchronous calls without touching my library. Here is my complete code. npgall's answer inspired me, thanks! Similar approach would be applied to all kinds of async libraries.
public class TestActivity extends Activity {
MyAPI api = (MyAPI) Bumerang.get().initAPI(MyAPI.class);
BlockingQueue<Object> blockingQueue = new ArrayBlockingQueue<Object>(1);
static int indexForTesting;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_test);
Thread t = new Thread(new Runnable() {
#Override
public void run() {
for(int i = 0; i < 10; i++) {
getItems();
try {
Object response = blockingQueue.take(); // waits for the response
Log.i("TAG", "index " + indexForTesting + " finished. Response " + response.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}
});
t.start();
}
void getItems() {
Log.i("TAG", "index " + ++indexForTesting + " started");
api.getItems(new ResponseListener<Response<List<ResponseModel>>>() {
#Override
public void onSuccess(Response<List<ResponseModel>> response) {
List<ResponseModel> respModel = response.getResponse();
try {
blockingQueue.put(response);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
#Override
public void onError(Response<List<ResponseModel>> response) {
Log.i("onError", response.toString());
try {
blockingQueue.put(response);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
}
}