We Keep Coding

Fixing StackOverflowError recursivity using Runnable

I am getting StackOverflowError exception report while calling this recursive method :
private void downloadFiles(int index) {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
downloadFiles(index + 1);
}
});
}
}
I want to ask if I use a Runnable like this way:
int index = 0;
handler = new Handler();
Runnable runnable = new Runnable() {
#Override
public void run() {
downloadFiles();
}
};
handler.post(runnable);
private void downloadFiles() {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
index ++;
handler.post(runnable);
}
});
}
}
Will this be a recursivity as well and throw exception ?
Thanks

Your current use of recursion sort of defeats the purpose of using multiple threads. Currently, you only create a single thread which will enter downloadFiles(), and will then recursively try to download every file available. This is not really multithreading, it's single threading with recursion. There are several drawbacks to this approach. First, you are not taking advantage of the ability for multiple threads to do work in parallel. Second, since each subsequent recursive call is dependent on the previous one having succeeded, you are trying to download files in serial. If a given file download were to fail, it would break the rest of the recursive chain.
A better approach would be to spawn a new thread for each file download. This would allow you to use the power of multithreading to split the task in parallel, and it also allows progress to continue even if one thread were to encounter some problems.
Have a look at the following code snippet for an idea on how to approach your problem:
public class FileDownloader implements Runnable {
private index;
public FileDownloader(int index) {
this.index = index;
}
public void run() {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
// this may no longer be needed
}
});
}
}
// use a thread pool of size 5 to handle your file downloads
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int index=0; index < totalFiles; ++index) {
Runnable r = new FileDownloader(index);
executor.execute(r);
}
// shut down the thread pool executor and wait for it to terminate
executor.shutdown();
while (!executor.isTerminated()) {
}

Using polling with isDone and Cancel on Java Future instead of blocking get

I am working on some legacy code which I can not refactor immediately.
The code uses blocking Java Future. It uses future.get(withTimeOut,...). This, means that we need to have a decent size thread pool to be responsive enough. As the calls will get blocked till they finish or times out.
Question:
I was thinking to grab the future and put it in a data structure which will be aware of start of execution of a task. Then have a dedicated thread or pool which will loop over the data structure and check if the future.isDone or has exceeded timeout limit. If yes, it can either get the result or cancel the execution. This way not many threads would be required. Will it be a correct implementation or it is not recommended at all?
Thanks in advance.
Edit:
Just to provide more context. These threads are used for logging to a downstream service. We really do not care about the response but we do not want the connection to hung. Thus, we need to grab the future and ensure that it is cancelled or timed out.
Here is a basic simulation which I wrote after asking the question.
#Component
public class PollingService {
private ExecutorService executorService = Executors.newFixedThreadPool(1);
PoorMultiplexer poorMultiplexer = new PoorMultiplexer();
private ConcurrentMap<Integer, Map<Future, Long>> futures = new ConcurrentHashMap<>();
public void startHandler(){
Thread handler = new Thread(new Runnable() {
#Override
public void run() {
while(true){
try {
//This should be handled better. If there is not anything stop and re-start it later.
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
for(Iterator<ConcurrentMap.Entry<Integer, Map<Future, Long>>> it = futures.entrySet().iterator(); it.hasNext();){
ConcurrentMap.Entry<Integer, Map<Future, Long>> entry = it.next();
Map<Future, Long> futureMap = entry.getValue();
boolean isProcessed = false;
if(futureMap.keySet().iterator().next().isDone()){
//mark completed
isProcessed = true;
}
if(futureMap.values().iterator().next() < (300 + System.currentTimeMillis()) && !isProcessed){
//cancel
futureMap.keySet().iterator().next().cancel(true);
isProcessed = true;
}
if(isProcessed){
futures.remove(entry.getKey());
System.out.println("Completed : " + entry.getKey());
}
}
System.out.println("Run completed");
}
}
});
handler.start();
}
public void run(int i) throws InterruptedException, ExecutionException{
System.out.println("Starting : " + i);
poorMultiplexer.send(new Runnable() {
#Override
public void run() {
long startTime = System.currentTimeMillis();
Future future = poorMultiplexer.send(execute());
Map<Future, Long> entry = new HashMap<>();
entry.put(future, startTime);
futures.put(i, entry);
System.out.println("Added : " + i);
}
});
}
public void stop(){
executorService.shutdown();
}
public Runnable execute(){
Worker worker = new Worker();
return worker;
}
}
//This is a placeholder for a framework
class PoorMultiplexer {
private ExecutorService executorService = Executors.newFixedThreadPool(20);
public Future send(Runnable task){
return executorService.submit(task);
}
}
class Worker implements Runnable{
#Override
public void run() {
//service call here
}
}

Asynchronously polling a set of Futures using a separate thread does sound like a reasonable implementation to me. That said, if you're able to add a library dependency, you might find it easier to switch to Guava's ListenableFuture, as Guava provides a wealth of utilities for doing asynchronous work.

Muti thread join & get return values

I'm trying to do multiple heavy calculations using threads.
Then I need to do something with the results after making sure all threads have finished its job.
Here's the basic code:
private class Runner implements Runnable {
private String _result = "fail";
public String get_results() {
return _result;
}
public void run() {
_result = "do complex calculation";
}
}
public void test() {
List<Thread> threads = new ArrayList<Thread>();
List<Runner> threadObjects = new ArrayList<Runner>();
for (int i = 0; i < 10; i++) {
Runner runner = new Runner();
Thread t = new Thread(runner);
t.start();
threads.add(t);
threadObjects.add(runner);
}
for (Thread thread : threads) {
try {
thread.join();
} catch (InterruptedException ex) {
}
}
for (Runner threadObject : threadObjects) {
System.out.println(threadObject.get_results());
}
}
My question is, is above snippet a common or a good approach to get calculation value?
If not please suggest a better ones.
Also sometimes I got runner.get_results() reponse = "fail", it seems calculation does not processed at all.
Thanks

You can use an ExecutorService such as the ScheduledThreadPoolExecutor;
ExecutorService executor = new ScheduledThreadPoolExecutor(numOfThreads);
With numOfThreads being the number of threads you want sitting in the thread pool.
You can then use the submit(Callable<T> task) method provided by the ScheduledThreadPoolExecutor class to execute the calculation.
You would then have a Callable implementation of your calculation and pass it to the submit() method in ExecutorService to execute the calculation;
class Calculation implements Callable {
#Override
public Object call() throws Exception { // The signature can be changed to return a different type (explained at the end)
return "do complex calculation";
}
}
As we can see from the method signature of the submit(Callable<T> task) method it returns a Future.
public <T> Future<T> submit(Callable<T> task)
The get() method of the Future class will return the result upon successful completion. This would ensure that your calculation completed before reading it.
A further note on the return type of the call() method;
Although this returns Object there is nothing stopping you changing the type of object it returns (this is known as co-variant returns)
For example the following is perfectly legal:
#Override
public String call() throws Exception {
return "do complex calculation";
}

How to manage return values of an unknown amount of Callables using ExecutorService?

I want to create a singleton-ExecutorService with a fixed threadpool size. Another thread will feed that ExecutorService with Callables and I want to parse the result of the Callables (optimally) immediately after the execution is done.
I am really uncertain how to implement this properly.
My initial thought was a method in the singleton-ES, which adds a Callable to the ExecutorService via "submit(callable)" and stores the resulting Future inside a HashMap or ArrayList inside the singleton. Another thread would check the Futures for results within a given interval.
But somehow this solution does not "feel right" and I didn't find a solution for this usecase elsewhere, so I am asking you guys before I code something I regret later.
How would you approach this problem?
I am looking forward to your responses!

import java.util.concurrent.*;
public class PostProcExecutor extends ThreadPoolExecutor {
// adjust the constructor to your desired threading policy
public PostProcExecutor(int corePoolSize, int maximumPoolSize,
long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
}
#Override
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
return new FutureTask<T>(callable) {
#Override
protected void done()
{
if(!isCancelled()) try {
processResult(get());
} catch(InterruptedException ex) {
throw new AssertionError("on complete task", ex);
} catch(ExecutionException ex) {
// no result available
}
}
};
}
protected void processResult(Object o)
{
System.out.println("Result "+o);// do your post-processing here
}
}

Use a ExecutorCompletionService. This way you can get the result of the Callable(s) as soon as they are ready. The take method of the completion service blocks waiting for each tasks to be done.
Here is an example from the java doc:
void solve(Executor e,
Collection<Callable<Result>> solvers)
throws InterruptedException, ExecutionException {
CompletionService<Result> ecs
= new ExecutorCompletionService<Result>(e);
for (Callable<Result> s : solvers)
ecs.submit(s);
int n = solvers.size();
for (int i = 0; i < n; ++i) {
Result r = ecs.take().get();
if (r != null)
use(r);
}
}

You can use MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(THREAD_NUMBER)); to create service
and use guava ListenableFuture for parsing result immidiatly also you can wtite your bike for Listen future result.
ListeningExecutorService service = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
ListenableFuture<Explosion> explosion = service.submit(new Callable<Explosion>() {
public Explosion call() {
return pushBigRedButton();
}
});
Futures.addCallback(explosion, new FutureCallback<Explosion>() {
// we want this handler to run immediately after we push the big red button!
public void onSuccess(Explosion explosion) {
walkAwayFrom(explosion);
}
public void onFailure(Throwable thrown) {
battleArchNemesis(); // escaped the explosion!
}
});

You can use ExecutorCompletionService to implement it.
The following steps can help you some.
Populate the number of available processors using Runtime.getRuntime().availableProcessors(). Let's keep the value in variable availableProcessors.
Initilize ExecutorService, like service = Executors.newFixedThreadPool(availableProcessors)
Initialize ExecutorCompletionService, assume the result from Callable is an Integer Array Integer[], ExecutorCompletionService completionService = new ExecutorCompletionService(service)
Use completionService.submit to submit the task.
Use completionService.take().get() to get each result of a task (callable).
Based on the above steps you can get the results of all callable, and do some business you would like to.

How to properly use threadpool and get a result from a thread?

i'm triyng to experiment the multithread programming (new for me) and i have some questions.
I'm using a ThreadPoolTaskExecutorwith a TestTask which implements Runnable and a run method wich sleeps for X seconds. Everyting went smoothly and all my TestTask were executed in a different thread. Ok.
Now the tricky part is that i want to know the result of an operation made in the thread. So i read some stuff on Google/stack/etc and i tried to use Future. And it's not working well anymore :/
I use the get method to get (oh really ?) the result of the call method and that part is working but the TestTask are executed one after another (and not at the same time like before). So i'm guessing i didn't understand properly something but i don't know what... and that's why i need your help !
The class wich launch test :
public void test(String test) {
int max = 5;
for (int i = 0; i < max; i++) {
TestThreadService.launch(i);
}
System.out.println("END");
}
The TestThreadService class :
public class TestThreadService {
private ThreadPoolTaskExecutor taskExecutor;
public void launch(int i) {
System.out.println("ThreadNumber : "+i);
taskExecutor.setWaitForTasksToCompleteOnShutdown(false);
TestTask testTask = new TestTask(i);
FutureTask<Integer> futureOne = new FutureTask<Integer>(testTask);
taskExecutor.submit(futureOne);
try {
Integer result = futureOne.get();
System.out.println("LAUNCH result : "+i+" - "+result);
} catch (Exception e) {
e.printStackTrace();
}
}
public void setTaskExecutor(ThreadPoolTaskExecutor taskExecutor) {
this.taskExecutor = taskExecutor;
}
}
And the TestTask Class :
public class TestTask implements Callable<Integer> {
public Integer threadNumber;
private Integer valeur;
public TestTask(int i) {
this.threadNumber = i;
}
public void setThreadNumber(Integer threadNumber) {
this.threadNumber = threadNumber;
}
#Override
public Integer call() throws Exception {
System.out.println("Thread start " + threadNumber);
// generate sleeping time
Random r = new Random();
valeur = 5000 + r.nextInt(15000 - 5000);
System.out.println("Thread pause " + threadNumber + " " + valeur);
try {
Thread.sleep(valeur);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Thread stop" + threadNumber);
return this.valeur;
}
}
I'm not bad in Java but this is the first time i'm trying to use different thread so i'ts kind a new for me.
What am i doing wrong ?
Thanks !

In your test method,
TestThreadService.launch(1);
should probably be
TestThreadService.launch(i);
Main thing though is the
Integer result = futureOne.get();
call in the launch method. Calling get() on a FutureTask is a blocking operation, meaning it will not return until the task is completed. That is why you are seeing a serial behavior. The use-case you are emulating (farming a bunch of activities and waiting for them to complete) is not one that the ThreadPoolTaskExecutor is ideally suited for. It does not have the "join" feature that raw threads have. That beeing said, what you want to do is something like
public Future<Integer> launch(int i) {
System.out.println("ThreadNumber : "+i);
taskExecutor.setWaitForTasksToCompleteOnShutdown(false);
TestTask testTask = new TestTask(i);
FutureTask<Integer> futureOne = new FutureTask<Integer>(testTask);
return taskExecutor.submit(futureOne);
}
And in your test method
public void test(String test) {
List<Future<Integer>> tasks = new ArrayList<Future<Integer>>();
int max = 5;
for (int i = 0; i < max; i++) {
tasks.add(TestThreadService.launch(i));
}
for (Future<Integer> task : tasks) {
System.out.println("LAUNCH result : " + task.get());
}
System.out.println("END");
}

also you can move setWaitForTasksToCompleteOnShutdown(false) into another method, for to dont be called each time you launch a thread, which is, as i see, (not very much threads), but in another scenario, with more tasks: an unnecessary and expensive job.
You can also create a public method on service, called: configure(); or, pre-launch(); before you start creating threads.
gluck!