I have a problem with ExecutorService, it does not execute all calls.
public class MainClass {
public static void main(String[] args) throws IOException, InvalidConnection, InterruptedException {
ExecutorService executor = Executors.newCachedThreadPool();
XMLUrlService xmlUrlService = new XMLUrlService();
LinkedBlockingQueue<String> linkedBlockingQueue = xmlUrlService.getAllXMLUrls();
System.out.println(linkedBlockingQueue.size());
for (int i = 0; i < linkedBlockingQueue.size(); i++) {
executor.execute(new XMLParser(linkedBlockingQueue));
}
executor.shutdown();
}
}
XMLUrlService class returns a list of URLs to which it then performs connections.
XMLPaser is a class that is responsible for making a connection to a given URL.
public class XMLParser implements Runnable {
private LinkedBlockingQueue queue;
public XMLParser(LinkedBlockingQueue queue) {
this.queue = queue;
}
public XMLParser(){}
public void getRates(String data) throws IOException, XMLStreamException, InvalidConnection {
URL url = new URL(data);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
InputStream inputStream = connection.getInputStream();
inputStream.close();
connection.disconnect();
}
public void run() {
try {
String data = (String) queue.take();
getRates(data);
} catch (InterruptedException | IOException ex) {
//Handle exception
} catch (InvalidConnection invalidConnection) {
invalidConnection.printStackTrace();
} catch (XMLStreamException e) {
e.printStackTrace();
}
}
}
My blocking queue contains 250 different url addresses, but the executorservice performs random number of calls (250 is only random number of elemnts to be tested). I think that when I take items from the queue, I can omit some, but I'm not sure.
There are three things you should change:
The way you execute the tasks
You are executing the tasks in a very odd way. Instead of passing linkedBlockingQueue, pass the String like so:
System.out.println(linkedBlockingQueue.size());
for (int i = 0; i < linkedBlockingQueue.size(); i++) {
String url = linkedBlockingQueue.remove()
executor.execute(new XMLParser(url));
}
The way you handle exceptions
Do not swallow exceptions - the easiest way is to use a Callable instead to get Futures so that you can later check what happened with your tasks:
ExecutorService executor = Executors.newCachedThreadPool();
LinkedBlockingQueue<String> linkedBlockingQueue = new LinkedBlockingQueue<String>();
System.out.println(linkedBlockingQueue.size());
List<Future<Void>> result = linkedBlockingQueue.stream().map(
url -> executor.submit(new XMLParser(url))
).collect(Collectors.toList());
for (Future<Void> future : result) {
try {
future.get();
} catch (Exception e) {
System.out.println("Something happened:" + e);
}
}
executor.shutdown();
Waiting for execution
After you shutdown the executor, wait until all your tasks are done.
executor.shutdown();
executor.awaitTermination(10, TimeUnit.HOURS);
Related
I am starting multiple threads and the idea is to wait for only one of them(does not matter which one) to continue the main thread and ignoring the other threads. In other words the main thread starts several threads and then waits until the fastest thread joins. Any ideas how to implement such a thing?
There's several possibilities.
1. Use a CountDownLatch
The idea is to have a CountDownLatch set to 1. All tasks running on the relevant threads will end by calling countDown() on it.
The main thread will simply call await() on it, and as soon as the first task is finished, the latch will hit 0, and release the waiting main thread.
This approach works with raw Threads, as well as with an ExecutorService.
public class UseCountDownLatch {
public static void main(String[] args) {
int numberOfThreads = 5;
CountDownLatch countDownLatch = new CountDownLatch(1);
ExecutorService executorService = Executors.newFixedThreadPool(numberOfThreads);
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
executorService.submit(() -> {
try {
TimeUnit.SECONDS.sleep(print * 3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
countDownLatch.countDown();
}
});
}
executorService.shutdown();
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
executorService.awaitTermination(1, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
2. Use a CompletionService
Wrap your ExecutionService in a CompletionService, then just wait for the first result to come in, and ignore further results.
public class UseCompletionService {
public static void main(String[] args) {
int numberOfThreads = 5;
ExecutorService executorService = Executors.newFixedThreadPool(numberOfThreads);
CompletionService<Void> completionService = new ExecutorCompletionService<Void>(executorService);
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
completionService.submit(() -> {
try {
TimeUnit.SECONDS.sleep(print * 3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
return null;
});
}
executorService.shutdown();
try {
completionService.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
executorService.awaitTermination(1, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
3. Use CompletableFutures
The CompletableFutures API is made to make tasks composable. The trick is simply to combine the CompletableFutures for each individual task in a CompletableFuture that completes as soon as any of its components completes. Then it's just a matter of getting the result from the composed CompletableFuture.
public class UseCompletableFuture {
public static void main(String[] args) {
int numberOfThreads = 5;
CompletableFuture<?>[] tasks = new CompletableFuture<?>[numberOfThreads];
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
tasks[i] = CompletableFuture.runAsync(() -> {
try {
TimeUnit.SECONDS.sleep(print*3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
}
try {
CompletableFuture.anyOf(tasks).get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
CompletableFuture.allOf(tasks).get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
I am using the below code in a java application.It spawns 3 worker threads. What I need is that if one thread fails (throws an unhandled exception from run)other threads should continue execution. If I call executor.shutdown() as below it is terminating the other threads(no new tasks are accepted).One of the options that I was thinking was to have a counter on the ExecutionException catch block and call shutdown only when counter==numConsumers. Is there a better way to do this?
public void execute() {
int numConsumers = Integer.parseInt(configs.getSINK_NUMBER_OF_CONSUMERs());
Future<?> future=null;
log.info("Creating {} consumers",numConsumers);
List<String> topics = Arrays.asList(configs.getTOPIC_NAME());
ExecutorService executor = Executors.newFixedThreadPool(numConsumers);
final List<SinkConsumer> consumers = new ArrayList<>();
for (int i = 0; i < numConsumers; i++) {
consumer = new SinkConsumer(UUID.randomUUID().toString(),topics,configs);
consumers.add(consumer);
future = executor.submit(consumer);
}
try {
future.get();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
executor.shutdown();
}
}
i am trying to poll a database in a server and check if any new records are added, and if any i'm going to send a http request to the java application with the new record.
This is the GET request:
public class PHPDataChecker implements Runnable {
public static String output;
public void run(){
try {
URL url = new URL("http://taxi.net/login.php");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I'm using a scheduler and here is that code:
public class Main {
private static boolean canStop=false;
public static void stopPHPDataChecker() {
canStop=true;
}
public static void runnner() {
// Setup a task for checking data and then schedule it
PHPDataChecker pdc = new PHPDataChecker();
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
final ScheduledFuture<?> pdcHandle = scheduler.scheduleAtFixedRate(pdc, 0L, 10L, TimeUnit.MILLISECONDS);// Start schedule
scheduler.schedule(new Runnable() {
public void run() {
System.out.println(">> TRY TO STOP!!!");
pdcHandle.cancel(true);
Main.stopPHPDataChecker();
System.out.println("DONE");
}
}, 10L, TimeUnit.MILLISECONDS);
do {
if (canStop) {
scheduler.shutdown();
}
} while (!canStop);
System.out.println("END");
}
another two programs to periodically poll it
RunMain.java:
public class RunMain implements Runnable {
public void run(){
Main m=new Main();
m.runnner();
}}
checkSchedule.java:
public class checkSchedule {
public static void main(String[] args) {
RunMain m = new RunMain();
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
final ScheduledFuture<?> pdcHandle = scheduler.scheduleAtFixedRate(m, 0L, 10L, TimeUnit.SECONDS);
}
}
This doensn't poll the database correctly is there anything wrong with the codings ?
this is the output i see in the nebeans IDE
Output from Server ....
{"return":"0"}
Output from Server ....
{"return":"0"}
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
TRY TO STOP!!!
DONE
END
I have the following code in a command line application. Once the loop completes, my app is still running. Why is it not shutting down. From the logs, I can see that endIndex has reached. But the app is still running?
Executor exec = Executors.newFixedThreadPool(4);
for (int i = startIndex; i <= endIndex; i++) {
final String spURL = urlPart + i;
Runnable requestHandler = new Runnable() {
#Override
public void run() {
try {
getImageForURL(spURL, 0);
} catch (IOException ex) {
} catch (Exception ex) {
}
}
};
exec.execute(requestHandler);
}
Try this
ExecutorService e = Executors.newFixedThreadPool(4);
...
e.shutdown();
You need to shutdown the executors after the loop (ideally in a finally block):
exec.shutdown() or exec.shutdownNow()
I have written a Bluetooth API for connecting with an external accessory.
The way that the API is designed is that there are a bunch of blocking calls such as getTime, setTime, getVolume, setVolume, etc.
The way these work is that they create a payload to send and call a method called sendAndReceive() which does some prep work and eventually does the following:
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
The problem is that sometimes this device becomes slow or non-responsive so I would like to put a timeout on this call.
I have tried several methods of putting this code in a thread\future task and running it with a timeout, for example:
FutureTask<byte[]> theTask = null;
// create new task
theTask = new FutureTask<byte[]>(
new Callable<byte[]>() {
#Override
public byte[] call() {
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
}
});
// start task in a new thread
new Thread(theTask).start();
// wait for the execution to finish, timeout after 6 secs
byte[] response;
try {
response = theTask.get(6L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
throw new CbtException(e);
} catch (ExecutionException e) {
throw new CbtException(e);
} catch (TimeoutException e) {
throw new CbtCallTimedOutException(e);
}
return response;
}
The problem with this approach is that I can't re-throw exceptions in the call method and since some of the methods throw exceptions I want to forward back to the API's client I can't use this methodology.
Can you recommend some other alternative?
Thanks!
You're saving you can't use the Future<> method because you want to re-throw the exception but in fact this is possible.
Most examples online do implement Callable with the prototype public ? call() but just change it to public ? call() throws Exception and all will be fine: you'll get the exception in the theTask.get() call and you can rethrow it to callers.
I have personally used Executors exactly for bluetooth socket timeout handling on android:
protected static String readAnswer(...)
throws Exception {
String timeoutMessage = "timeout";
ExecutorService executor = Executors.newCachedThreadPool();
Callable<String> task = new Callable<String>() {
public String call() throws Exception {
return readAnswerNoTimeout(...);
}
};
Future<String> future = executor.submit(task);
try {
return future.get(SOCKET_TIMEOUT_MS, TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
future.cancel(true);
throw new Exception(timeoutMessage);
}
}
Why not try something like
public class ReadTask extends Thread {
private byte[] mResultBuffer;
private Exception mCaught;
private Thread mWatcher;
public ReadTask(Thread watcher) {
mWatcher = watcher;
}
public void run() {
try {
mResultBuffer = sendAndReceive();
} catch (Exception e) {
mCaught = e;
}
mWatcher.interrupt();
}
public Exception getCaughtException() {
return mCaught;
}
public byte[] getResults() {
return mResultBuffer;
}
}
public byte[] wrappedSendAndReceive() {
byte[] data = new byte[1024];
ReadTask worker = new ReadTask(data, Thread.currentThread());
try {
worker.start();
Thread.sleep(6000);
} catch (InterruptedException e) {
// either the read completed, or we were interrupted for another reason
if (worker.getCaughtException() != null) {
throw worker.getCaughtException();
}
}
// try to interrupt the reader
worker.interrupt();
return worker.getResults;
}
There is an edge case here that the Thread calling wrappedSendAndReceive() may get interrupted for some reason other than the interrupt from the ReadTask. I suppose a done bit could be added to the ReadTask to allow the other thread to test if the read finished or the interrupt was caused by something else, but I'm not sure how necessary this is.
A further note is that this code does contain the possibility for data loss. If the 6 seconds expires and some amount of data has been read this will end up being discarded. If you wanted to work around this, you'd need to read one byte at a time in ReadTask.run() and then appropriately catch the InterruptedException. This obviously requires a little rework of the existing code to keep a counter and appropriately resize the read buffer when the interrupt is received.