We've got a stipes (java) web-app that needs to make about 15 different webserivce calls all from one method. For example:
...
public Resolution userProfile()
{
serviceOneCall();
serviceTwoCall();
serviceThreeCall();
serviceFourCall();
....
serviceTenCall();
return new RedirectResolution("profiel.jsp");
}
All of these can be called in parallel and are not dependent on each other. The one thing that most all of these calls are doing is putting data in the session, and one or two may put data into the same object that is in the session, so thread safety is probably a concern there.
Can anyone suggest a good way of calling all of these concurrently?
All solutions to doing this work in parallel is going to involve spawning new threads or submitting jobs to a thread pool for the remote network calls to happen to.
A good way to avoid thread safety problems is to use an executorService and submit subclasses of Callable<T> (to either the submit(Callable) or invokeAll(Collection<Callable>) methods) and have the Callables return the response value. This way your initial method can simply handle the return values of each call and choose to set the responses in the session or update other objects, rather than this work occurring in another thread.
So basic algorithm:
Submit each of these calls to an executorService in Callable<T> subclasses
Collect the Future<T>s you get back from the executorService
Call Future.get() on each to block until you have a response, and then process the responses however you wish back on the main thread
Use an ExecutorService with a thread pool to submit Callables for each WS you need to call, and synchronize on the object which is updated when there is a chance of concurrent modification.
You may want to use Guava's concurrent extensions for an easier management of the Futures, using for example Futures.allAsList() which will convert a List<Future<T>> into a Future<List<T>>, so you only have one get() to do to wait for all the answers.
for (i = 0; i <= numOfServiceCalls; i++) {
new Thread(new Runnable() {
switch(i) {
case 1 : serviceOneCall();
break();
case 2 : serviceTwoCall();
break();
// Keep going with as many cases as you have.
}
});
}
Related
I'm working on a Java server application with the general following architecture:
Clients make RPC requests to the server
The RPC server (gRPC) I believe has its own thread pool for handling requests
Requests are immediately inserted into Thread Pool 1 for more processing
A specific request type, we'll call Request R, needs to run a few asynchronous tasks in parallel, judging the results to form a consensus that it will return to the client. These tasks are a bit more long running, so I use a separate Thread Pool 2 to handle these requests. Importantly, each Request R will need to run the same 2-3 asynchronous tasks. Thread Pool 2 therefore services ALL currently executing Request R's. However, a Request R should only be able to see and retrieve the asynchronus tasks that belong to it.
To achieve this, upon every incoming Request R, while its in Thread Pool 1, it will create a new CompletionService for the request, backed by Thread Pool 2. It will submit 2-3 async tasks, and retrieve the results. These should be strictly isolated from anything else that might be running in Thread Pool 2 belonging to other requests.
My questions:
Firstly, is Java's CompletionService isolated? I couldn't find good documentation on this after checking the JavaDocs. In other words, if two or more CompletionService's are backed by the same thread pool, are any of them at risk of pulling a future belonging to another CompletionService?
Secondly, is this bad practice to be creating this many CompletionService's for each request? Is there a better way to handle this? Of course it would be a bad idea to create a new thread pool for each request, so is there a more canonical/correct way to isolate futures within a CompletionService or is what I'm doing okay?
Thanks in advance for the help. Any pointers to helpful documentation or examles would be greatly appreciated.
Code, for reference, although trivial:
public static final ExecutorService THREAD_POOL_2 =
new ThreadPoolExecutor(16, 64, 60, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
// Gets created to handle a RequestR, RequestRHandler is run in Thread Pool 1
public class RequestRHandler {
CompletionService<String> cs;
RequestRHandler() {
cs = new ExecutorCompletionService<>(THREAD_POOL_2);
}
String execute() {
cs.submit(asyncTask1);
cs.submit(asyncTask2);
cs.submit(asyncTask3);
// Lets say asyncTask3 completes first
Future<String> asyncTask3Result = cs.take();
// asyncTask3 result indicates asyncTask1 & asyncTask2 results don't matter, cancel them
// without checking result
// Cancels all futures, I track all futures submitted within this request and cancel them,
// so it shouldn't affect any other requests in the TP 2 pool
cancelAllFutures(cs);
return asyncTask3Result.get();
}
}
Firstly, is Java's CompletionService isolated?
That's not garanteed as it's an interface, so the implementation decides that. But as the only implementation is ExecutorCompletionService I'd just say the answer is: yes. Every instance of ExecutorCompletionService has internally a BlockingQueue where the finished tasks are queued. Actually, when you call take on the service, it just passes the call to the queue by calling take on it. Every submitted task is wrapped by another object, which puts the task in the queue when it's finished. So each instance manages it's submitted tasks isolated from other instances.
Secondly, is this bad practice to be creating this many CompletionServices for each request?
I'd say it's okay. A CompletionService is nothing but a rather thin wrapper around an executor. You have to live with the "overhead" (internal BlockingQueue and wrapper instances for the tasks) but it's small and you are probably gaining way more from it than it costs. One could ask if you need one for just 2 to 3 tasks but it kinda depends on the tasks. At this point it's a question about if a CompletionService is worth it in general, so that's up to you to decide as it's out of scope of your question.
I'm working on a Java-based server in which I will have multiple threads (one thread for each connected user + some extra). There will be some database connection involved, so I was thinking that each time the server makes a SELECT query to the database it will start a new thread for this, to prevent blocking from the current thread. I'm planning on using a connection pool for this and I think I know how to do that. (I've looked into C3P0) However, there will be a lot of UPDATE statements involved also, but it's not important that these are ran directly, it's ok with a delay here. And since there might be a lot of UPDATE statements, I'm thinking of having a single worker thread for all UPDATE statements. As I see it, this will have the advantage of being able to re-use PreparedStatement-objects
The question:
How can I, from the other threads, tell the UPDATE-worker thread to run some statements? I know about multithreading and how to make threads "talk" to each other using synchronized blocks, but with the database involved it suddenly feels more complex. I have read that prepared statements and connections should not be shared between threads.
The idea I have right now on how to solve it: (doesn't feel like a good solution)
Use a LinkedBlockingQueue (or another kind of Queue) of a custom class with information about which kind of UPDATE statement to call and which parameters to send it. And then the worker thread will read from this queue when it's notified (which it will be when something is added to the queue) and there it will run the appropriate method which will use the appropriate prepared statement, set the params, and call it.
Edit: A bad think that I see myself with this approach is that the params might be ints, String, double, or whatever. How to store them in the custom class? Doesn't feel good to store them all as String.
Am I on the right track here or is there a better way to solve this?
No need for an explicit blocking queue. You can have a worker thread and a work queue encapsulated by an ExecutorService. As for the values, you can use generics:
class ThreadTask<T> implements Runnable {
private T value;
public ThreadTask(T value) {
this.value = value;
}
public void run() {
// update based on value
}
}
...
ExecutorService exec = Executors.newSingleThreadExecutor();
exec.submit(new ThreadTask<String>("asdf"));
exec.submit(new ThreadTask<Integer>(1));
The single thread executor is simply a worker thread waiting on a queue and executing the submitted tasks in order. No need for other explicit management.
The idea: I have a JAX-RS webservice servlet (Object called webServlet) which instantiates a data collecting Object dataCollector and passes this object on to multiple threads in their constructor. These threads query websites for results and then call the dataCollector.add(result) method to add the results to a Queue within the shared dataCollector.
I have two questions regarding this idea:
1) Can multiple threads call methods of a single shared object at the same time?
2) How does my webServlet object check when all threads are terminated to render a result page? Do I have to let my webServlet wait while all threads are running so I have a complete result list and how would I do that?
1) Yes, but perhaps not safely. In particular, if the queue in your dataCollector isn't a thread-safe queue like a ConcurrentLinkedQueue, you run the risk of a ConcurrentModificationException when a thread calls add() on it.
2) a) Use an ExecutorService (perhaps obtained from Executors) to submit Callables or Runnables. Keep the Futures that are returned and use get() to wait until he work is done.
b) You don't have to. The choice is up to you. If you send the response before the work is done, you obviously won't have a complete result yet.
c) See a).
If this is all new to you, you may want to check out Concurrency in the Java Tutorials.
I'm looking for a simple object that will hold my work threads and I need it to not limit the number of threads, and not keep them alive longer than needed.
But I do need it to have a method similar to an ExecutorService.shutdown();
(Waiting for all the active threads to finish but not accepting any new ones)
so maybe a threadpool isn't what I need, so I would love a push in the right direction.
(as they are meant to keep the threads alive)
Further clarification of intent:
each thread is an upload of a file, and I have another process that modifies files, but it waits for the file to not have any uploads. by joining each of the threads. So when they are kept alive it locks that process. (each thread adds himself to a list for a specific file on creation, so I only join() threads that upload a specific file)
One way to do what you awant is to use a Callable with a Future that returns the File object of a completed upload. Then pass the Future into another Callable that checks Future.isDone() and spins until it returns true and then do whatever you need to do to the file. Your use case is not unique and fits very neatly into the java.util.concurrent package capabilities.
One interesting class is ExecutorCompletionService class which does exactly what you want with waiting for results then proceeding with an additional calculation.
A CompletionService that uses a
supplied Executor to execute tasks.
This class arranges that submitted
tasks are, upon completion, placed on
a queue accessible using take. The
class is lightweight enough to be
suitable for transient use when
processing groups of tasks.
Usage Examples: Suppose you have a set of solvers for a certain problem,
each returning a value of some type
Result, and would like to run them
concurrently, processing the results
of each of them that return a non-null
value, in some method use(Result r).
You could write this as:
void solve(Executor e, Collection<Callable<Result>> solvers)
throws InterruptedException, ExecutionException
{
CompletionService<Result> ecs = new ExecutorCompletionService<Result>(e);
for (Callable<Result> s : solvers) { ecs.submit(s); }
int n = solvers.size();
for (int i = 0; i < n; ++i)
{
Result r = ecs.take().get();
if (r != null) { use(r); }
}
}
You don't want an unbounded ExecutorService
You almost never want to allow unbounded thread pools, as they actually can limit the performance of your application if the number of threads gets out of hand.
You domain is limited by disk or network I/O or both, so a small thread pool would be sufficient. You are not going to want to try and read from hundreds or thousands of incoming connections with a thread per connection.
Part of your solution, if you are receiving more than a handful of concurrent uploads is to investigate the java.nio package and read about non-blocking I/O as well.
Is there a reason that you don't want to reuse threads? Seems to me that the simplest thing would be to use ExecutorService anyway and let it reuse threads.
What are the correct ways/practice/implementation/strategies (or whatever we call it as) for not to wait for code block/method to finish execution in Java?
Assume the following method:
private void myMethod()
{
// Some lines of code here
.....
.....
.....
anotherMethod(); // or this could be a code block
// Again some lines of code here
.....
.....
.....
}
In this case, I expect myMethod() should not wait for code to finish execution of anotherMethod(). I can also assure here that subsequent lines of code do not depend on anything getting executed within anotherMethod().
You can start it in another Thread if there is no dependency .
new Thread(new Runnable(){
public void run(){
anotherMethod();
}
}).start();
Use
Executor executor = Executors.newSingleThreadExecutor();
executor.execute(new Runnable() {
#Override
public void run() {
anotherMethod();
}
});
// this is called automatically when the object is gc-ed,
// but you should not rely on this, hence the explicit call
executor.shutdown();
To quote Effective Java:
Not only should you refrain from writing your own work queues, but you
should generally refrain from working directly with threads. The key abstraction
is no longer Thread, which served as both the unit of work and the mechanism for
executing it. Now the unit of work and mechanism are separate. The key abstraction is the unit of work, which is called a task. There are two kinds of tasks: Runnable and its close cousin, Callable (which is like Runnable, except that it
returns a value). The general mechanism for executing tasks is the executor ser-
vice. If you think in terms of tasks and let an executor service execute them for
you, you gain great flexibility in terms of selecting appropriate execution policies.
In essence, the Executor Framework does for execution what the Collections
Framework did for aggregation.
Note that you'd better create your executor only once, store it in an instance field and reuse it (and shut it down afterwards)
If you are running in JavaEE 6 or spring, you can annotate your anotherMethod() with #Asynchronous and the container will start a new thread for you.
If you want to invoke the method in the same thread, then the method itself must provide an asynchronous (i.e. non-blocking) implementation. Usually this will involve returning some sort of callback such as a Future, which you can poll/query later to fetch the actual result. An example of this is the ExecutorService.submit() calls - the code you supply will be run, but in a background thread leaving you free to call other methods in the meantime.
I bolded the word invoke before, since fundamentally the only way in Java to have two things happening at once is to use multiple threads. So the method/code block/whatever will have to be executing in a background thread one way or another - usually this is handled for you in an asynchronous method by using some sort of thread pool or whatever is appropriate.
If the method doesn't provide an asynchronous interface, however, the only way to get its logic to run in another thread is to spawn that thread yourself, as org.life.java suggests.