We have 1000 threads that hit a web service and time how long the call takes. We wish for each thread to return their own timing result to the main application, so that various statistics can be recorded.
Please note that various tools were considered for this, but for various reasons we need to write our own.
What would be the best way for each thread to return the timing - we have considered two options so far :-
1. once a thread has its timing result it calls a singleton that provides a synchronised method to write to the file. This ensures that all each thread will write to the file in turn (although in an undetermined order - which is fine), and since the call is done after the timing results have been taken by the thread, then being blocked waiting to write is not really an issue. When all threads have completed, the main application can then read the file to generate the statistics.
2. Using the Executor, Callable and Future interfaces
Which would be the best way, or are there any other better ways ?
Thanks very much in advance
Paul
Use the latter method.
Your workers implement Callable. You then submit them to a threadpool, and get a Future instance for each.
Then just call get() on the Futures to get the results of the calculations.
import java.util.*;
import java.util.concurrent.*;
public class WebServiceTester {
public static class Tester
implements Callable {
public Integer call() {
Integer start = now();
//Do your test here
Integer end = now();
return end - start;
}
}
public static void main(String args[]) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(1000);
Set<Future<Integer>> set = new HashSet<Future<Integer>>();
for (int i =0 ; i < 1000 i++) {
set.add(pool.submit(new Tester()));
}
Set<Integer> results = new Set<Integer>();
for (Future<Integer> future : set) {
results.put(future.get());
}
//Manipulate results however you wish....
}
}
Another possible solution I can think of would be to use a CountDownLatch (from the java concurrency packages), each thread decrementing it (flagging they are finished), then once all complete (and the CountDownLatch reaches 0) your main thread can happily go through them all, asking them what their time was.
The executor framework can be implemented here. The time processing can be done by the Callable object. The Future can help you identify if the thread has completed processing.
You could pass an ArrayBlockingQueue to the threads to report their results to. You could then have a file writing thread that takes from the queue to write to the file.
Related
I have gotten my code into a state where I am creating a couple of threads and then inside those threads I use a library framework which spawns some additional threads over the life span of my application.
I have no control over how many threads are spawned inside the library framework, but I know they exist because I can see them in the eclipse debugger, I have kept the threads I use outside the library framework to a minimum, because I really don't want a multithreaded application, but sometimes you have too.
Now I am at the point where I need to do things with sockets and I/O, both of which are inherently hard to deal with in a multithreaded environment and while I am going to make my program thread safe i'd rather not get into the situation in the first place, or at least minimize the occurrences, the classes I am attempting to reduce multithreading in aren't time sensitive and i'd like them to complete "when they get the time". As it happens the lazy work is all in the same class definition but due to reasons, the class is instantiated a hell of a lot.
I was wondering if it was possible to make single type classes use only one thread when instantiated from multiple threads, and how?
I imagine the only way to achieve this would be to create a separate thread specifically for handling and processing of a instances of single class type.
Or do I just have to think of a new way to structure my code?
EDIT: included an example of my applications architecture;
public class Example {
public ArrayList<ThreadTypeA> threads = new ArrayList<ThreadTypeA>();
public static void main(String[] args) {
threads.add(new ThreadTypeA());
// left out how dataObj gets to ThreadTypeB for brevity
dataObj data = new dataObj(events);
}
}
public ThreadTypeA {
public ArrayList<ThreadTypeB> newThreads = new ArrayList<ThreadTypeB>();
public Thread thread = new Thread(this, "");
}
public ThreadTypeB {
// left out how dataObj gets to ThreadTypeB for brevity
public libObj libObj = new Library(dataObj);
}
public Library {
public Thread thread = new Thread(this, "");
#Override
public void editMe(dataObj) {
dataObj.callBack();
}
}
public dataObj(events) {
public void callMe() {
for (Event event: events) {
event.callMe();
}
}
}
there are a number of different events that can be called, ranging from writing to files making sql queries, sending emails and using proprietary ethernet-serial comms. I wish all events to run on the same thread, sequentially.
Rather than having Threads, consider having Callable or Runnables. These are objects which represent the work that is to be done. Your code can pass these to a thread pool for execution - you'll get a Future. If you care about the answer, you'll call get on the future and your code will wait for the execution to complete. If it's a fire-and-forget then you can be assured it's queued and will get done in good time.
Generally it makes more sense to divorce your execution code from the threads that run it to allow patterns like this.
To restrict thread resources use a limited thread pool:
ExecutorService executor = Executors.newFixedThreadPool(4);
for (int i = 0; i < 100; ++i) {
executor.execute(new Runnable() { ... });
}
executor.shutdown();
Also the reuse of threads of such a pool is said to be faster.
It might be a far hope that the library does a similar thing, and maybe even has the thread pool size configurable.
Our application requires all worker threads to synchronize at a defined point. For this we use a CyclicBarrier, but it does not seem to scale well. With more than eight threads, the synchronization overhead seems to outweigh the benefits of multithreading. (However, I cannot support this with measurement data.)
EDIT: Synchronization happens very frequently, in the order of 100k to 1M times.
If synchronization of many threads is "hard", would it help building a synchronization tree? Thread 1 waits for 2 and 3, which in turn wait for 4+5 and 6+7, respectively, etc.; after finishing, threads 2 and 3 wait for thread 1, thread 4 and 5 wait for thread 2, etc..
1
| \
2 3
|\ |\
4 5 6 7
Would such a setup reduce synchronization overhead? I'd appreciate any advice.
See also this featured question: What is the fastest cyclic synchronization in Java (ExecutorService vs. CyclicBarrier vs. X)?
With more than eight threads, the synchronization overhead seems to outweigh the benefits of multithreading. (However, I cannot support this with measurement data.)
Honestly, there's your problem right there. Figure out a performance benchmark and prove that this is the problem, or risk spending hours / days solving the entirely wrong problem.
You are thinking about the problem in a subtly wrong way that tends to lead to very bad coding. You don't want to wait for threads, you want to wait for work to be completed.
Probably the most efficient way is a shared, waitable counter. When you make new work, increment the counter and signal the counter. When you complete work, decrement the counter. If there is no work to do, wait on the counter. If you drop the counter to zero, check if you can make new work.
If I understand correctly, you're trying to break your solution up into parts and solve them separately, but concurrently, right? Then have your current thread wait for those tasks? You want to use something like a fork/join pattern.
List<CustomThread> threads = new ArrayList<CustomThread>();
for (Something something : somethings) {
threads.add(new CustomThread(something));
}
for (CustomThread thread : threads) {
thread.start();
}
for (CustomThread thread : threads) {
thread.join(); // Blocks until thread is complete
}
List<Result> results = new ArrayList<Result>();
for (CustomThread thread : threads) {
results.add(thread.getResult());
}
// do something with results.
In Java 7, there's even further support via a fork/join pool. See ForkJoinPool and its trail, and use Google to find one of many other tutorials.
You can recurse on this concept to get the tree you want, just have the threads you create generate more threads in the exact same way.
Edit: I was under the impression that you wouldn't be creating that many threads, so this is better for your scenario. The example won't be horribly short, but it goes along the same vein as the discussion you're having in the other answer, that you can wait on jobs, not threads.
First, you need a Callable for your sub-jobs that takes an Input and returns a Result:
public class SubJob implements Callable<Result> {
private final Input input;
public MyCallable(Input input) {
this.input = input;
}
public Result call() {
// Actually process input here and return a result
return JobWorker.processInput(input);
}
}
Then to use it, create an ExecutorService with a fix-sized thread pool. This will limit the number of jobs you're running concurrently so you don't accidentally thread-bomb your system. Here's your main job:
public class MainJob extends Thread {
// Adjust the pool to the appropriate number of concurrent
// threads you want running at the same time
private static final ExecutorService pool = Executors.newFixedThreadPool(30);
private final List<Input> inputs;
public MainJob(List<Input> inputs) {
super("MainJob")
this.inputs = new ArrayList<Input>(inputs);
}
public void run() {
CompletionService<Result> compService = new ExecutorCompletionService(pool);
List<Result> results = new ArrayList<Result>();
int submittedJobs = inputs.size();
for (Input input : inputs) {
// Starts the job when a thread is available
compService.submit(new SubJob(input));
}
for (int i = 0; i < submittedJobs; i++) {
// Blocks until a job is completed
results.add(compService.take())
}
// Do something with results
}
}
This will allow you to reuse threads instead of generating a bunch of new ones every time you want to run a job. The completion service will do the blocking while it waits for jobs to complete. Also note that the results list will be in order of completion.
You can also use Executors.newCachedThreadPool, which creates a pool with no upper limit (like using Integer.MAX_VALUE). It will reuse threads if one is available and create a new one if all the threads in the pool are running a job. This may be desirable later if you start encountering deadlocks (because there's so many jobs in the fixed thread pool waiting that sub jobs can't run and complete). This will at least limit the number of threads you're creating/destroying.
Lastly, you'll need to shutdown the ExecutorService manually, perhaps via a shutdown hook, or the threads that it contains will not allow the JVM to terminate.
Hope that helps/makes sense.
If you have a generation task (like the example of processing columns of a matrix) then you may be stuck with a CyclicBarrier. That is to say, if every single piece of work for generation 1 must be done in order to process any work for generation 2, then the best you can do is to wait for that condition to be met.
If there are thousands of tasks in each generation, then it may be better to submit all of those tasks to an ExecutorService (ExecutorService.invokeAll) and simply wait for the results to return before proceeding to the next step. The advantage of doing this is eliminating context switching and wasted time/memory from allocating hundreds of threads when the physical CPU is bounded.
If your tasks are not generational but instead more of a tree-like structure in which only a subset need to be complete before the next step can occur on that subset, then you might want to consider a ForkJoinPool and you don't need Java 7 to do that. You can get a reference implementation for Java 6. This would be found under whatever JSR introduced the ForkJoinPool library code.
I also have another answer which provides a rough implementation in Java 6:
public class Fib implements Callable<Integer> {
int n;
Executor exec;
Fib(final int n, final Executor exec) {
this.n = n;
this.exec = exec;
}
/**
* {#inheritDoc}
*/
#Override
public Integer call() throws Exception {
if (n == 0 || n == 1) {
return n;
}
//Divide the problem
final Fib n1 = new Fib(n - 1, exec);
final Fib n2 = new Fib(n - 2, exec);
//FutureTask only allows run to complete once
final FutureTask<Integer> n2Task = new FutureTask<Integer>(n2);
//Ask the Executor for help
exec.execute(n2Task);
//Do half the work ourselves
final int partialResult = n1.call();
//Do the other half of the work if the Executor hasn't
n2Task.run();
//Return the combined result
return partialResult + n2Task.get();
}
}
Keep in mind that if you have divided the tasks up too much and the unit of work being done by each thread is too small, there will negative performance impacts. For example, the above code is a terribly slow way to solve Fibonacci.
I frequently need to have a thread wait for the result of another thread. Seems like there should be some support for this in java.util.concurrent, but I can't find it.
Exchanger is very close to what I'm talking about, but it's bi-directional. I only want Thread A to wait on Thread B, not have both wait on each other.
Yes, I know I can use a CountDownLatch or a Semaphore or Thread.wait() and then manage the result of the computation myself, but it seems like I must be missing a convenience class somewhere.
What am I missing?
UPDATE
// An Example which works using Exchanger
// but you would think there would be uni-directional solution
protected Exchanger<Integer> exchanger = new Exchanger<Integer>();
public void threadA() {
// perform some computations
int result = ...;
exchanger.exchange(result);
}
public void threadB() {
// retrieve the result of threadA
int resultOfA = exchanger.exchange(null);
}
Are you looking for Future<T>? That's the normal representation of a task which has (usually) been submitted to a work queue, but may not have completed yet. You can find out its completion status, block until it's finished, etc.
Look at ExecutorService for the normal way of obtaining futures. Note that this is focused on getting the result of an individual task, not rather than waiting for a thread to finish. A single thread may complete many tasks in its life time, of course - that's the whole point of a thread pool.
So far, it seems like BlockingQueue may be the best solution I've found.
eg.
BlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(1);
The waiting thread will call queue.take() to wait for the result, and the producing queue will call queue.add() to submit the result.
The JDK doesn't provide a convenience class that provides the exact functionality you're looking for. However, it is actually fairly easy to write a small utility class to do just that.
You mentioned the CountDownLatch and your preference regarding it, but I would still suggest looking at it. You can build a small utility class (a "value synchronizer" if you will) pretty easily:
public class OneShotValueSynchronizer<T> {
private volatile T value;
private final CountDownLatch set = new CountDownLatch(1);
public T get() throws InterruptedException {
set.await();
return value;
}
public synchronized void set(T value) {
if (set.getCount() > 0) {
this.value = value;
set.countDown();
}
}
// more methods if needed
}
Since Java 8 you can use CompletableFuture<T>. Thread A can wait for a result using the blocking get() method, while Thread B can pass the result of computation using complete().
If Thread B encounters an exception while calculating the result, it can communicate this to Thread A by calling completeExceptionally().
What's inconvenient in using Thread.join()?
I recently had the same problem, tried using a Future then a CountdownLatch but settled on an Exchanger. They are supposed to allow two threads to swap data but there's no reason why one of those threads can't just pass a null.
In the end I think it was the cleanest solution, but it may depend on what exactly you are trying to achieve.
You might use java.util.concurrent.CountDownLatch for this.
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/CountDownLatch.html
Example:
CountDownLatch latch = new CountDownLatch(1);
// thread one
// do some work
latch.countDown();
// thread two
latch.await();
Please look at my following code....
private static final int NTHREDS = 10;
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
while(rs.next()){
webLink=rs.getString(1);
FirstName=rs.getString(2);
MiddleName=rs.getString(3);
Runnable worker = new MyRunnable(webLink,FirstName,MiddleName);// this interface has run method....
executor.execute(worker);
}
//added
public class MyRunnable implements Runnable {
MyRunnable(String webLink,String FirstName,String MiddleName){
** Assigning Values...***
}
#Override
public void run() {
long sum = 0;
**Calling method to crawl by passing those Values**
try {
Thread.sleep(200);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
In this part if the resultset(rs) having 100 records excutor creating 100 threads..... I need to run this process with in 10 threads. I need your help to know how to get control of threads.. If any thread has completed its task then it should process the immediate available task from the Result Set. Is it possible to achieve using executor framework.
Thanks...
vijay365
The code you've already posted does this. Your code will not immediately spawn 100 threads. It will spawn 10 threads that consume tasks from a queue containing your Runnables.
From the Executors.newFixedThreadPool Javadocs:
Creates a thread pool that reuses a
fixed set of threads operating off a
shared unbounded queue.
Instead of using a static number of threads (10 in this case) you should determine the number dynamically:
final int NTHREADS = Runtime.getRuntime().availableProcessors();
Also, I don't get why you are calling Thread.sleep?
ResultSet is probably a JDBC query result.
This design is almost certain to be doomed to failure.
The JDBC interface implementations are not thread-safe.
ResultSets are scare resources that should be closed in the same scope in which they were created. If you pass them around, you're asking for trouble.
Multi-threaded code is hard to write well and even harder to debug if incorrect.
You are almost certainly headed in the wrong direction with this design. I'd bet a large sum of money that you're guilty of premature optimization. You are hoping that multiple threads will make your code faster, but what will happen is ten threads time slicing on one CPU and taking the same time or longer. (Context switching takes time, too.)
A slightly better idea would be to load the ResultSet into an object or collection, close the ResultSet, and then do some multi-threaded processing on that returned object.
Try executor.submit(worker);
I'm making a Java application with an application-logic-thread and a database-access-thread.
Both of them persist for the entire lifetime of the application and both need to be running at the same time (one talks to the server, one talks to the user; when the app is fully started, I need both of them to work).
However, on startup, I need to make sure that initially the app thread waits until the db thread is ready (currently determined by polling a custom method dbthread.isReady()).
I wouldn't mind if app thread blocks until the db thread was ready.
Thread.join() doesn't look like a solution - the db thread only exits at app shutdown.
while (!dbthread.isReady()) {} kind of works, but the empty loop consumes a lot of processor cycles.
Any other ideas? Thanks.
Use a CountDownLatch with a counter of 1.
CountDownLatch latch = new CountDownLatch(1);
Now in the app thread do-
latch.await();
In the db thread, after you are done, do -
latch.countDown();
I would really recommend that you go through a tutorial like Sun's Java Concurrency before you commence in the magical world of multithreading.
There are also a number of good books out (google for "Concurrent Programming in Java", "Java Concurrency in Practice".
To get to your answer:
In your code that must wait for the dbThread, you must have something like this:
//do some work
synchronized(objectYouNeedToLockOn){
while (!dbThread.isReady()){
objectYouNeedToLockOn.wait();
}
}
//continue with work after dbThread is ready
In your dbThread's method, you would need to do something like this:
//do db work
synchronized(objectYouNeedToLockOn){
//set ready flag to true (so isReady returns true)
ready = true;
objectYouNeedToLockOn.notifyAll();
}
//end thread run method here
The objectYouNeedToLockOn I'm using in these examples is preferably the object that you need to manipulate concurrently from each thread, or you could create a separate Object for that purpose (I would not recommend making the methods themselves synchronized):
private final Object lock = new Object();
//now use lock in your synchronized blocks
To further your understanding:
There are other (sometimes better) ways to do the above, e.g. with CountdownLatches, etc. Since Java 5 there are a lot of nifty concurrency classes in the java.util.concurrent package and sub-packages. You really need to find material online to get to know concurrency, or get a good book.
Requirement ::
To wait execution of next thread until previous finished.
Next thread must not start until previous thread stops, irrespective of time consumption.
It must be simple and easy to use.
Answer ::
#See java.util.concurrent.Future.get() doc.
future.get() Waits if necessary for the computation to complete, and then retrieves its result.
Job Done!! See example below
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.junit.Test;
public class ThreadTest {
public void print(String m) {
System.out.println(m);
}
public class One implements Callable<Integer> {
public Integer call() throws Exception {
print("One...");
Thread.sleep(6000);
print("One!!");
return 100;
}
}
public class Two implements Callable<String> {
public String call() throws Exception {
print("Two...");
Thread.sleep(1000);
print("Two!!");
return "Done";
}
}
public class Three implements Callable<Boolean> {
public Boolean call() throws Exception {
print("Three...");
Thread.sleep(2000);
print("Three!!");
return true;
}
}
/**
* #See java.util.concurrent.Future.get() doc
* <p>
* Waits if necessary for the computation to complete, and then
* retrieves its result.
*/
#Test
public void poolRun() throws InterruptedException, ExecutionException {
int n = 3;
// Build a fixed number of thread pool
ExecutorService pool = Executors.newFixedThreadPool(n);
// Wait until One finishes it's task.
pool.submit(new One()).get();
// Wait until Two finishes it's task.
pool.submit(new Two()).get();
// Wait until Three finishes it's task.
pool.submit(new Three()).get();
pool.shutdown();
}
}
Output of this program ::
One...
One!!
Two...
Two!!
Three...
Three!!
You can see that takes 6sec before finishing its task which is greater than other thread. So Future.get() waits until the task is done.
If you don't use future.get() it doesn't wait to finish and executes based time consumption.
Good Luck with Java concurrency.
A lot of correct answers but without a simple example.. Here is an easy and simple way how to use CountDownLatch:
//inside your currentThread.. lets call it Thread_Main
//1
final CountDownLatch latch = new CountDownLatch(1);
//2
// launch thread#2
new Thread(new Runnable() {
#Override
public void run() {
//4
//do your logic here in thread#2
//then release the lock
//5
latch.countDown();
}
}).start();
try {
//3 this method will block the thread of latch untill its released later from thread#2
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
//6
// You reach here after latch.countDown() is called from thread#2
public class ThreadEvent {
private final Object lock = new Object();
public void signal() {
synchronized (lock) {
lock.notify();
}
}
public void await() throws InterruptedException {
synchronized (lock) {
lock.wait();
}
}
}
Use this class like this then:
Create a ThreadEvent:
ThreadEvent resultsReady = new ThreadEvent();
In the method this is waiting for results:
resultsReady.await();
And in the method that is creating the results after all the results have been created:
resultsReady.signal();
EDIT:
(Sorry for editing this post, but this code has a very bad race condition and I don't have enough reputation to comment)
You can only use this if you are 100% sure that signal() is called after await(). This is the one big reason why you cannot use Java object like e.g. Windows Events.
The if the code runs in this order:
Thread 1: resultsReady.signal();
Thread 2: resultsReady.await();
then thread 2 will wait forever. This is because Object.notify() only wakes up one of the currently running threads. A thread waiting later is not awoken. This is very different from how I expect events to work, where an event is signalled until a) waited for or b) explicitly reset.
Note: Most of the time, you should use notifyAll(), but this is not relevant to the "wait forever" problem above.
Try CountDownLatch class out of the java.util.concurrent package, which provides higher level synchronization mechanisms, that are far less error prone than any of the low level stuff.
You could do it using an Exchanger object shared between the two threads:
private Exchanger<String> myDataExchanger = new Exchanger<String>();
// Wait for thread's output
String data;
try {
data = myDataExchanger.exchange("");
} catch (InterruptedException e1) {
// Handle Exceptions
}
And in the second thread:
try {
myDataExchanger.exchange(data)
} catch (InterruptedException e) {
}
As others have said, do not take this light-hearted and just copy-paste code. Do some reading first.
The Future interface from the java.lang.concurrent package is designed to provide access to results calculated in another thread.
Take a look at FutureTask and ExecutorService for a ready-made way of doing this kind of thing.
I'd strongly recommend reading Java Concurrency In Practice to anyone interested in concurrency and multithreading. It obviously concentrates on Java, but there is plenty of meat for anybody working in other languages too.
If you want something quick and dirty, you can just add a Thread.sleep() call within your while loop. If the database library is something you can't change, then there is really no other easy solution. Polling the database until is ready with a wait period won't kill the performance.
while (!dbthread.isReady()) {
Thread.sleep(250);
}
Hardly something that you could call elegant code, but gets the work done.
In case you can modify the database code, then using a mutex as proposed in other answers is better.
This applies to all languages:
You want to have an event/listener model. You create a listener to wait for a particular event. The event would be created (or signaled) in your worker thread. This will block the thread until the signal is received instead of constantly polling to see if a condition is met, like the solution you currently have.
Your situation is one of the most common causes for deadlocks- make sure you signal the other thread regardless of errors that may have occurred. Example- if your application throws an exception- and never calls the method to signal the other that things have completed. This will make it so the other thread never 'wakes up'.
I suggest that you look into the concepts of using events and event handlers to better understand this paradigm before implementing your case.
Alternatively you can use a blocking function call using a mutex- which will cause the thread to wait for the resource to be free. To do this you need good thread synchronization- such as:
Thread-A Locks lock-a
Run thread-B
Thread-B waits for lock-a
Thread-A unlocks lock-a (causing Thread-B to continue)
Thread-A waits for lock-b
Thread-B completes and unlocks lock-b
You could read from a blocking queue in one thread and write to it in another thread.
Since
join() has been ruled out
you have already using CountDownLatch and
Future.get() is already proposed by other experts,
You can consider other alternatives:
invokeAll from ExecutorService
invokeAll(Collection<? extends Callable<T>> tasks)
Executes the given tasks, returning a list of Futures holding their status and results when all complete.
ForkJoinPool or newWorkStealingPool from Executors ( since Java 8 release)
Creates a work-stealing thread pool using all available processors as its target parallelism level.
This idea can apply?. If you use CountdownLatches or Semaphores works perfect but if u are looking for the easiest answer for an interview i think this can apply.