Syncing multiple asynchronous requests in Java - java

I am using the official Telegram Api (TDLib) in Java to request information about all members of a group. Using their IDs I am sending asynchronous requests to the server and I receive User objects for each request inside the ResultHandler like this:
private static ArrayList<TdApi.User> chatUsers= new ArrayList<>();
private static void addUsers(){
for (int i = 0; i < userIDs.length; i++){
client.send(new TdApi.GetUser(userIDs[i]), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
TdApi.User user = (TdApi.User)object;
chatUsers.add(user);
}
});
}
}
Since I am pretty new to anychronous requests in Java I am wondering the following:
What would be an appropriate approach to call this method and wait for all results received before moving on?
Generally, when calling several requests consecutively and waiting for each result before moving on with the next request, what is an usual approach instead of nesting the requests inside of each other to sync them in Java? I want to avoid something like this:
private static void getSupergroupId(int chatId){
//first step
client.send(new TdApi.GetChat(chatId), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
supergroupId = ((TdApi.ChatTypeSupergroup)((TdApi.Chat)object).type).supergroupId;
//second step when result received
client.send(new TdApi.GetSupergroupMembers(supergroupId, null, 0, 200), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
chatMembers = ((TdApi.ChatMembers)object).members;
//further steps which need to wait for the result of the step before
}
});
}
});
}
Thank you!

1 One of Java Synchronizers should work. I would start with CountDownLatch as it the simplest one.
private static final ArrayList<TdApi.User> chatUsers = Collections.synchronizedList(new ArrayList<>());
private static void addUsers() {
final CountDownLatch latch = new CountDownLatch(userIDs.length);
for (int i = 0; i < userIDs.length; i++) {
client.send(new TdApi.GetUser(userIDs[i]), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
TdApi.User user = (TdApi.User) object;
chatUsers.add(user);
latch.countDown();
}
});
}
// handle InterruptedException
latch.await(10, TimeUnit.SECONDS);
}
Notice that chatUsers is accessed from different threads so access to it should be guarded by a lock. I used Collections.synchronizedList in the example for simplicity. However you should use more fine-grained approach.
2 Take a look at Completablefuture, seems that is what you are looking for.
private static void getSupergroupId(int chatId) {
CompletableFuture.supplyAsync(() -> {
AtomicReference<TdApi.ChatTypeSupergroup> atomicReference = new AtomicReference<>();
CountDownLatch latch = new CountDownLatch(1);
client.send(new TdApi.GetChat(chatId), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
atomicReference.set(((TdApi.ChatTypeSupergroup) ((TdApi.Chat) object).type).supergroupId);
latch.countDown();
}
});
// handle InterruptedException
latch.await(10, TimeUnit.SECONDS);
return atomicReference.get();
}).thenApply(supergroupId -> {
AtomicReference<TdApi.ChatMembers> atomicReference = new AtomicReference<>();
CountDownLatch latch = new CountDownLatch(1);
client.send(new TdApi.GetSupergroupMembers(supergroupId, null, 0, 200), new Client.ResultHandler() {
#Override
public void onResult(TdApi.Object object) {
atomicReference.set((TdApi.ChatMembers) object).members;
latch.countDown();
}
});
// handle InterruptedException
latch.await(10, TimeUnit.SECONDS);
return atomicReference.get();
});
//further steps which need to wait for the result of the step before)
}
Notice that the same trick with CountDownLatch is used to wait for the result. Again you result from callback should be guarded by lock as it is accessed by different threads. To be 100% clear it's not required because of piggybacking on CountDownLatch however i would recommend to use explicit synchronization anyway, for example AtomicReference.

Related

How do javafx multithreading execute sequentially?

I don't know if there are any other good ways to achieve the results I want, thank you.
I have a requirement, according to the URL, create multiple webview threads, and execute them in order, such as thread execution, then trigger thread two execution, and so on, I use the synchronized (lobject) method, but in JAVAfx encountered a problem, the code is as follows:
public class LockObject {
public int orderNum = 1;
public final static int MaxValue=9;
public LockObject(int orderNum){
this.orderNum = orderNum;
}
}
public class DownloadThread extends Thread{
private LockObject lobject;
private int printNum =0;
private String url;
public DownloadThread(LockObject lobject,int printNum,String url){
this.lobject=lobject;
this.printNum = printNum;
this.url = url;
}
#Override
public void run() {
synchronized(lobject){
while(lobject.orderNum <= lobject.MaxValue){
if(lobject.orderNum == printNum){
System.out.print(printNum);
Platform.runLater(new Runnable() {
#Override
public void run() {
webView.getEngine().load(url);
webView.getEngine().getLoadWorker().stateProperty().addListener(new ChangeListener<Worker.State>() {
#Override
public void changed(ObservableValue<? extends Worker.State> observable, Worker.State oldValue, Worker.State newValue) {
if (newValue == Worker.State.SUCCEEDED) {
try {
//xxxxx
// java.lang.IllegalMonitorStateException
lobject.notifyAll();
} catch (Exception e) {
e.printStackTrace();
}
}
}
});
}
});
lobject.orderNum++;
if(lobject.orderNum==downloadThreads.length){
saveCsvFile(goodCSVS);
}
//lobject.notifyAll(); is ok
}else{
try {
lobject.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
}
Place of call
private DownloadThread[] downloadThreads;
LockObject lobject = new LockObject(1);
downloadThreads = new DownloadThread[tableView.getItems().size()];
for (int i = 0; i < tableView.getItems().size(); i++) {
UrlModel item = tableView.getItems().get(i);
downloadThreads[i] = new DownloadThread(lobject,tableView.getItems().size()-i,item.getLink());
downloadThreads[i].start();
}
Calling lobject.notifyAll() in the run method in Platform.runLater will report an IllegalMonitorStateException. After the address is processed, I want to wake up the next thread to execute.
If you need to execute multiple tasks in order, there's no need to create multiple threads. Just using a single thread will guarantee the next task only executes after the previous one has completed. You should also consider using a CountDownLatch instead of synchronizing on an object.
ExecutorService executor = Executors.newSingleThreadExecutor();
try {
for (UrlModel model : tableView.getItems()) {
executor.submit(() -> {
CountDownLatch latch = new CountDownLatch(1);
Platform.runLater(() -> {
engine.load(model.getLink())
engine.getLoadWorker().runningProperty().addListener((obs, ov, nv) -> {
if (!nv) {
latch.countDown();
}
});
});
latch.await();
// do whatever needs to happen after the WebEngine finishes loading
return null; // using #submit(Callable) and Callable needs to return something
});
}
} finally {
executor.shutdown();
}
Some notes:
You may want to avoid creating the ExecutorService if the table has no items to process. That is, assuming you don't reuse the same ExecutorService every time.
If you reuse the ExecutorService, don't call shutdown().
This ExecutorService uses non-daemon threads. You can customize this by supplying a ThreadFactory that creates daemon threads.
I added a listener to the Worker#running property instead of the status property to make it easier to ensure countDown() is invoked no matter the terminal status of the load (i.e. whether it's SUCCEEDED, CANCELLED or FAILED).
You may want to remove the the listener added to the Worker's property when it's finished. You can do this by using an anonymous class (rather than the lambda expression I used) and calling obs.removeListener(this) inside the changed method, where obs is the ObservableValue argument.

How I can return value from Thread [duplicate]

I have a method with a HandlerThread. A value gets changed inside the Thread and I'd like to return it to the test() method. Is there a way to do this?
public void test()
{
Thread uiThread = new HandlerThread("UIHandler"){
public synchronized void run(){
int value;
value = 2; //To be returned to test()
}
};
uiThread.start();
}
Usually you would do it something like this
public class Foo implements Runnable {
private volatile int value;
#Override
public void run() {
value = 2;
}
public int getValue() {
return value;
}
}
Then you can create the thread and retrieve the value (given that the value has been set)
Foo foo = new Foo();
Thread thread = new Thread(foo);
thread.start();
thread.join();
int value = foo.getValue();
tl;dr a thread cannot return a value (at least not without a callback mechanism). You should reference a thread like an ordinary class and ask for the value.
You can use a local final variable array. The variable needs to be of non-primitive type, so you can use an array. You also need to synchronize the two threads, for example using a CountDownLatch:
public void test()
{
final CountDownLatch latch = new CountDownLatch(1);
final int[] value = new int[1];
Thread uiThread = new HandlerThread("UIHandler"){
#Override
public void run(){
value[0] = 2;
latch.countDown(); // Release await() in the test thread.
}
};
uiThread.start();
latch.await(); // Wait for countDown() in the UI thread. Or could uiThread.join();
// value[0] holds 2 at this point.
}
You can also use an Executor and a Callable like this:
public void test() throws InterruptedException, ExecutionException
{
ExecutorService executor = Executors.newSingleThreadExecutor();
Callable<Integer> callable = new Callable<Integer>() {
#Override
public Integer call() {
return 2;
}
};
Future<Integer> future = executor.submit(callable);
// future.get() returns 2 or raises an exception if the thread dies, so safer
executor.shutdown();
}
What you are looking for is probably the Callable<V> interface in place of Runnable, and retrieving the value with a Future<V> object, which also lets you wait until the value has been computed. You can achieve this with an ExecutorService, which you can get from Executors.newSingleThreadExecutor() .
public void test() {
int x;
ExecutorService es = Executors.newSingleThreadExecutor();
Future<Integer> result = es.submit(new Callable<Integer>() {
public Integer call() throws Exception {
// the other thread
return 2;
}
});
try {
x = result.get();
} catch (Exception e) {
// failed
}
es.shutdown();
}
How about this solution?
It doesn't use the Thread class, but it IS concurrent, and in a way it does exactly what you request
ExecutorService pool = Executors.newFixedThreadPool(2); // creates a pool of threads for the Future to draw from
Future<Integer> value = pool.submit(new Callable<Integer>() {
#Override
public Integer call() {return 2;}
});
Now all you do is say value.get() whenever you need to grab your returned value, the thread is started the very second you give value a value so you don't ever have to say threadName.start() on it.
What a Future is, is a promise to the program, you promise the program that you'll get it the value it needs sometime in the near future
If you call .get() on it before it's done, the thread that's calling it will simply just wait until it's done
From Java 8 onwards we have CompletableFuture.
On your case, you may use the method supplyAsync to get the result after execution.
Please find some reference here.
CompletableFuture<Integer> completableFuture
  = CompletableFuture.supplyAsync(() -> yourMethod());
completableFuture.get() //gives you the value
If you want the value from the calling method, then it should wait for the thread to finish, which makes using threads a bit pointless.
To directly answer you question, the value can be stored in any mutable object both the calling method and the thread both have a reference to. You could use the outer this, but that isn't going to be particularly useful other than for trivial examples.
A little note on the code in the question: Extending Thread is usually poor style. Indeed extending classes unnecessarily is a bad idea. I notice you run method is synchronised for some reason. Now as the object in this case is the Thread you may interfere with whatever Thread uses its lock for (in the reference implementation, something to do with join, IIRC).
Using Future described in above answers does the job, but a bit less significantly as f.get(), blocks the thread until it gets the result, which violates concurrency.
Best solution is to use Guava's ListenableFuture. An example :
ListenableFuture<Void> future = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(1, new NamedThreadFactory).submit(new Callable<Void>()
{
#Override
public Void call() throws Exception
{
someBackgroundTask();
}
});
Futures.addCallback(future, new FutureCallback<Long>()
{
#Override
public void onSuccess(Long result)
{
doSomething();
}
#Override
public void onFailure(Throwable t)
{
}
};
With small modifications to your code, you can achieve it in a more generic way.
final Handler responseHandler = new Handler(Looper.getMainLooper()){
#Override
public void handleMessage(Message msg) {
//txtView.setText((String) msg.obj);
Toast.makeText(MainActivity.this,
"Result from UIHandlerThread:"+(int)msg.obj,
Toast.LENGTH_LONG)
.show();
}
};
HandlerThread handlerThread = new HandlerThread("UIHandlerThread"){
public void run(){
Integer a = 2;
Message msg = new Message();
msg.obj = a;
responseHandler.sendMessage(msg);
System.out.println(a);
}
};
handlerThread.start();
Solution :
Create a Handler in UI Thread,which is called as responseHandler
Initialize this Handler from Looper of UI Thread.
In HandlerThread, post message on this responseHandler
handleMessgae shows a Toast with value received from message. This Message object is generic and you can send different type of attributes.
With this approach, you can send multiple values to UI thread at different point of times. You can run (post) many Runnable objects on this HandlerThread and each Runnable can set value in Message object, which can be received by UI Thread.
Here is a cleaner approach, you just need a bit change to your existing code. The goal is to get the result from the the Thread. It doesn't really have to be return a result. Instead, using a callback style to take that result and do further processing.
public class Test {
public static void main(String[] args) {
String str = args[0];
int count = 0;
Thread t = new Thread(() ->
someFuncToRun(str, count, (value) -> {
System.out.println(value);
return value;
}));
t.start();
}
// Here I even run a recursive method because run things in the
// a thread sometime is to delegate those heavy lifting elsewhere
public static String someFuncToRun(String str, int ctn, Callback<String> p) {
++ctn;
if (ctn == 10) {
System.out.println("End here");
return p.cb(str);
}
System.out.println(ctn + " times");
return someFuncToRun(str + " +1", ctn, p);
}
}
// The key is here, this allow you to pass a lambda callback to your method
// update: use generic to allow passing different type of data
// you could event make it <T,S> so input one type return another type
interface Callback<T> {
public T cb(T a);
}

Fixing StackOverflowError recursivity using Runnable

I am getting StackOverflowError exception report while calling this recursive method :
private void downloadFiles(int index) {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
downloadFiles(index + 1);
}
});
}
}
I want to ask if I use a Runnable like this way:
int index = 0;
handler = new Handler();
Runnable runnable = new Runnable() {
#Override
public void run() {
downloadFiles();
}
};
handler.post(runnable);
private void downloadFiles() {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
index ++;
handler.post(runnable);
}
});
}
}
Will this be a recursivity as well and throw exception ?
Thanks
Your current use of recursion sort of defeats the purpose of using multiple threads. Currently, you only create a single thread which will enter downloadFiles(), and will then recursively try to download every file available. This is not really multithreading, it's single threading with recursion. There are several drawbacks to this approach. First, you are not taking advantage of the ability for multiple threads to do work in parallel. Second, since each subsequent recursive call is dependent on the previous one having succeeded, you are trying to download files in serial. If a given file download were to fail, it would break the rest of the recursive chain.
A better approach would be to spawn a new thread for each file download. This would allow you to use the power of multithreading to split the task in parallel, and it also allows progress to continue even if one thread were to encounter some problems.
Have a look at the following code snippet for an idea on how to approach your problem:
public class FileDownloader implements Runnable {
private index;
public FileDownloader(int index) {
this.index = index;
}
public void run() {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
// this may no longer be needed
}
});
}
}
// use a thread pool of size 5 to handle your file downloads
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int index=0; index < totalFiles; ++index) {
Runnable r = new FileDownloader(index);
executor.execute(r);
}
// shut down the thread pool executor and wait for it to terminate
executor.shutdown();
while (!executor.isTerminated()) {
}

Concurrency: how to implement an executor with both incoming and outgoing queues?

As we know, ThreadPoolExecutor uses some BlockingQueue as a queue of incoming tasks. What I want is to have ThreadPoolExecutor that has a second queue for the task results which are ready. I want to use this queue as a source for input/output services which send or store these results.
Why I want to create a separate queue? Because I want to decouple action of sending results from action of obtaining results. Also, I suppose any Exceptions and Delays that accompany input/output operations should not affect my ThreadPoolExecutor which is calculating the result.
I have created some naive implementation of this. I would like to get some criticism on this. May be, it can be implemented with out-of-the-box Java classes better? I use Java 7.
public class ThreadPoolWithResultQueue {
interface Callback<T> {
void complete(T t);
}
public abstract static class CallbackTask<T> implements Runnable {
private final Callback callback;
CallbackTask(Callback callback) {
this.callback = callback;
}
public abstract T execute();
final public void run() {
T t = execute();
callback.complete(t);
}
}
public static class CallBackTaskString extends CallbackTask<String> {
public CallBackTaskString(Callback callback) {
super(callback);
}
#Override
public String execute() {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
}
return hashCode() + "-" + System.currentTimeMillis();
}
}
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<Runnable>();
final BlockingQueue<String> resultQueue = new LinkedBlockingQueue<String>();
Callback<String> addToQueueCallback = new Callback<String>() {
#Override
public void complete(String s) {
System.out.println("Adding Result To Queue " + s);
resultQueue.add(s); //adding to outgoing queue. some other executor (or same one?) will process it
}
};
ThreadPoolExecutor executor = new ThreadPoolExecutor(3, 5, 1000l, TimeUnit.DAYS, workQueue);
for (int i = 0; i <= 5; i++) {
executor.submit(new CallBackTaskString(addToQueueCallback));
};
System.out.println("All submitted.");
executor.shutdown();
executor.awaitTermination(10l, TimeUnit.SECONDS);
System.out.println("Result queue size " + resultQueue.size());
}
}
For the sake of makinf a library component, you would have to wrap things up...
You could extend The thread pool executor which has a number of methods to intercept the submitted tasks, so you would queue thing out to a queue passed in the constructor.
That's basically ExecutorCompletionService, but you would allow the user to plug a queue instead of appearing as one.
Otherwise, this is typical proxying of the task. Fair job.

Returning value from Thread

I have a method with a HandlerThread. A value gets changed inside the Thread and I'd like to return it to the test() method. Is there a way to do this?
public void test()
{
Thread uiThread = new HandlerThread("UIHandler"){
public synchronized void run(){
int value;
value = 2; //To be returned to test()
}
};
uiThread.start();
}
Usually you would do it something like this
public class Foo implements Runnable {
private volatile int value;
#Override
public void run() {
value = 2;
}
public int getValue() {
return value;
}
}
Then you can create the thread and retrieve the value (given that the value has been set)
Foo foo = new Foo();
Thread thread = new Thread(foo);
thread.start();
thread.join();
int value = foo.getValue();
tl;dr a thread cannot return a value (at least not without a callback mechanism). You should reference a thread like an ordinary class and ask for the value.
You can use a local final variable array. The variable needs to be of non-primitive type, so you can use an array. You also need to synchronize the two threads, for example using a CountDownLatch:
public void test()
{
final CountDownLatch latch = new CountDownLatch(1);
final int[] value = new int[1];
Thread uiThread = new HandlerThread("UIHandler"){
#Override
public void run(){
value[0] = 2;
latch.countDown(); // Release await() in the test thread.
}
};
uiThread.start();
latch.await(); // Wait for countDown() in the UI thread. Or could uiThread.join();
// value[0] holds 2 at this point.
}
You can also use an Executor and a Callable like this:
public void test() throws InterruptedException, ExecutionException
{
ExecutorService executor = Executors.newSingleThreadExecutor();
Callable<Integer> callable = new Callable<Integer>() {
#Override
public Integer call() {
return 2;
}
};
Future<Integer> future = executor.submit(callable);
// future.get() returns 2 or raises an exception if the thread dies, so safer
executor.shutdown();
}
What you are looking for is probably the Callable<V> interface in place of Runnable, and retrieving the value with a Future<V> object, which also lets you wait until the value has been computed. You can achieve this with an ExecutorService, which you can get from Executors.newSingleThreadExecutor() .
public void test() {
int x;
ExecutorService es = Executors.newSingleThreadExecutor();
Future<Integer> result = es.submit(new Callable<Integer>() {
public Integer call() throws Exception {
// the other thread
return 2;
}
});
try {
x = result.get();
} catch (Exception e) {
// failed
}
es.shutdown();
}
How about this solution?
It doesn't use the Thread class, but it IS concurrent, and in a way it does exactly what you request
ExecutorService pool = Executors.newFixedThreadPool(2); // creates a pool of threads for the Future to draw from
Future<Integer> value = pool.submit(new Callable<Integer>() {
#Override
public Integer call() {return 2;}
});
Now all you do is say value.get() whenever you need to grab your returned value, the thread is started the very second you give value a value so you don't ever have to say threadName.start() on it.
What a Future is, is a promise to the program, you promise the program that you'll get it the value it needs sometime in the near future
If you call .get() on it before it's done, the thread that's calling it will simply just wait until it's done
From Java 8 onwards we have CompletableFuture.
On your case, you may use the method supplyAsync to get the result after execution.
Please find some reference here.
CompletableFuture<Integer> completableFuture
  = CompletableFuture.supplyAsync(() -> yourMethod());
completableFuture.get() //gives you the value
If you want the value from the calling method, then it should wait for the thread to finish, which makes using threads a bit pointless.
To directly answer you question, the value can be stored in any mutable object both the calling method and the thread both have a reference to. You could use the outer this, but that isn't going to be particularly useful other than for trivial examples.
A little note on the code in the question: Extending Thread is usually poor style. Indeed extending classes unnecessarily is a bad idea. I notice you run method is synchronised for some reason. Now as the object in this case is the Thread you may interfere with whatever Thread uses its lock for (in the reference implementation, something to do with join, IIRC).
Using Future described in above answers does the job, but a bit less significantly as f.get(), blocks the thread until it gets the result, which violates concurrency.
Best solution is to use Guava's ListenableFuture. An example :
ListenableFuture<Void> future = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(1, new NamedThreadFactory).submit(new Callable<Void>()
{
#Override
public Void call() throws Exception
{
someBackgroundTask();
}
});
Futures.addCallback(future, new FutureCallback<Long>()
{
#Override
public void onSuccess(Long result)
{
doSomething();
}
#Override
public void onFailure(Throwable t)
{
}
};
With small modifications to your code, you can achieve it in a more generic way.
final Handler responseHandler = new Handler(Looper.getMainLooper()){
#Override
public void handleMessage(Message msg) {
//txtView.setText((String) msg.obj);
Toast.makeText(MainActivity.this,
"Result from UIHandlerThread:"+(int)msg.obj,
Toast.LENGTH_LONG)
.show();
}
};
HandlerThread handlerThread = new HandlerThread("UIHandlerThread"){
public void run(){
Integer a = 2;
Message msg = new Message();
msg.obj = a;
responseHandler.sendMessage(msg);
System.out.println(a);
}
};
handlerThread.start();
Solution :
Create a Handler in UI Thread,which is called as responseHandler
Initialize this Handler from Looper of UI Thread.
In HandlerThread, post message on this responseHandler
handleMessgae shows a Toast with value received from message. This Message object is generic and you can send different type of attributes.
With this approach, you can send multiple values to UI thread at different point of times. You can run (post) many Runnable objects on this HandlerThread and each Runnable can set value in Message object, which can be received by UI Thread.
Here is a cleaner approach, you just need a bit change to your existing code. The goal is to get the result from the the Thread. It doesn't really have to be return a result. Instead, using a callback style to take that result and do further processing.
public class Test {
public static void main(String[] args) {
String str = args[0];
int count = 0;
Thread t = new Thread(() ->
someFuncToRun(str, count, (value) -> {
System.out.println(value);
return value;
}));
t.start();
}
// Here I even run a recursive method because run things in the
// a thread sometime is to delegate those heavy lifting elsewhere
public static String someFuncToRun(String str, int ctn, Callback<String> p) {
++ctn;
if (ctn == 10) {
System.out.println("End here");
return p.cb(str);
}
System.out.println(ctn + " times");
return someFuncToRun(str + " +1", ctn, p);
}
}
// The key is here, this allow you to pass a lambda callback to your method
// update: use generic to allow passing different type of data
// you could event make it <T,S> so input one type return another type
interface Callback<T> {
public T cb(T a);
}

Categories