After using this as a guide I'm trying to send a FileInputStream to two APIs concurrently, if either of them fails we error out.
I have two callables
private void submitCallablesWithExecutor(final FileInputStream content)
throws InterruptedException, ExecutionException, TimeoutException {
ExecutorService executorService = null;
try {
executorService = Executors.newFixedThreadPool(5);
//send to api1
final Future<String> futureOne = executorService.submit(new Callable<String>() {
#Override
public String call() throws Exception {
final String returnedValue = implOne.storeContent(content)
return returnedValue;
}
});
//send to api2
final Future<String> futureTwo = executorService.submit(new Callable<String>() {
#Override
public String call() throws Exception {
final String returnedValue = implTwo.storeContent(content);
return returnedValue;
}
});
final String valueOne = futureOne.get(4, TimeUnit.SECONDS);
final String valueTwo = futureTwo.get(4, TimeUnit.SECONDS);
} finally {
executorService.shutdown();
}
}
within api2s implementation it tries to calculate the filesize with the below method
protected Long calculateFilesize(InputStream data) throws StoreException {
try {
return (long) data.available();
} catch (IOException e) {
LOGGER.error("Error determining filesize of InputStream");
throw new StoreException( e);
}
}
I get the following error during data.available()
java.io.IOException: Stream Closed
I suspect at some point the stream is being closed and at this point it fails to read it.
Any advice would be highly appreciated.
It's worth noting that the method that calls submitCallablesWithExecutor(content) is surrounded with a try with resource. I wonder if it's auto closing the Stream?
try (final FileInputStream content = new FileInputStream(tmpFile)) {
submitCallablesWithExecutor(content)
}
It hard to tell without seeing more code of the tasks. This is one possible scenario for the closed stream. The javadoc for executorService.shutdown(); says:
This method does not wait for previously submitted tasks to complete execution.
If your first task take longer than 4 seconds to run you will get TimeoutException when calling futureOne.get(4, TimeUnit.SECONDS) which you don't catch so it leads to your own try with resource closing the FileInputStream. However the second callable might still be running in the executor service so could get this java.io.IOException: Stream Closed exception.
You could fix by using future.get() which blocks on first task completion, but it sounds like you don't want this, so it's better to cleanly shut down your service thread.
Try adding executorService.awaitTermination(long timeout, TimeUnit unit) after shutdown() and catch the TimeoutException and handle appropriately to ensure the second task exits cleanly in this timeout/error situation, such as using latch or flags to notify to end the task.
Related
I am trying to refactor code that sequentially waits on multiple futures to complete, to instead jointly wait for completion.
So I try to wait on multiple futures with a single timeout by using
// Example outcomes
final CompletableFuture<String> completedFuture
= CompletableFuture.completedFuture("hello");
final CompletableFuture<String> failedFuture
= new CompletableFuture<>();
failedFuture.completeExceptionally(new RuntimeException("Test Stub Exception"));
final CompletableFuture<String> incompleteFuture
= new CompletableFuture<>();
final AtomicBoolean timeoutHandled = new AtomicBoolean(false);
final CompletableFuture<String> checkedFuture
= incompleteFuture.whenComplete(
(x, e) -> timeoutHandled.set(e instanceof TimeoutException));
// this example timeouts after 1ms
try {
CompletableFuture
.allOf(completedFuture, checkedFuture, failedFuture)
.get(1, TimeUnit.MILLISECONDS);
} catch (final InterruptedException e) {
Thread.currentThread().interrupt();
} catch (final TimeoutException e) {
// probably do something here?
}
// but the incomplete future is still pending
assertTrue(checkedFuture.isCompletedExceptionally());
// this still fails even if checkedFuture.completeExceptionally(e) is called
assertTrue(timeoutHandled.get());
However the assert above fails because while the collective future timed out, the individual future did not time out yet. I would like to cancel such individual futures the same way as if they had run into timeouts individually, because they might have individual whenComplete() handlers handling TimeoutExceptions:
Expecting
<CompletableFuture[Incomplete]>
to be completed exceptionally.
Is there a useful/safe pattern by which I can loop over all exceptions and invoke completeExceptionally() to simulate a timeout in each of the futures, and make sure all "exception handlers" have been invoked before moving on?
You can create a varargs method with your try/catch that loops through each CompletableFuture and invokes completeExceptionally().
static void completeFutures(CompletableFuture<?>... completableFutures) throws ExecutionException {
try {
CompletableFuture.allOf(completableFutures).get(1, TimeUnit.MILLISECONDS);
} catch (final InterruptedException e) {
Thread.currentThread().interrupt();
} catch (final TimeoutException e) {
for (CompletableFuture<?> cf : completableFutures) {
cf.completeExceptionally(e);
}
}
}
I am trying to upload to S3 within my asynchronous Java code
private void submitCallablesWithExecutor()
throws InterruptedException, ExecutionException, TimeoutException {
ExecutorService executorService = null;
try {
executorService = Executors.newCachedThreadPool();
Future<String> task1Future = executorService.submit(new Callable<String>() {
public String call() {
try {
processExportRequest(xmlPutRequest_, customizedRequest_, response_);
return "Success";
} catch (Exception ex) {
return ex.getMessage();
}
}
});
} finally {
executorService.shutdown();
try {
if (!executorService.awaitTermination(800, TimeUnit.MILLISECONDS)) {
executorService.shutdownNow();
}
} catch (InterruptedException e) {
executorService.shutdownNow();
}
}
}
within processExportRequest I am calling upload to S3. I have tried both S3Client and S3AsyncClient. In both cases, I am getting following error:
Failed to upload to S3: java.lang.IllegalStateException: Interrupted waiting to refresh the value.
I don't see anywhere in my code that's calling Thread.interrupt(), and everything else seems to work fine, just not S3 upload. Maybe the multithreaded nature of Java Future is not compatible with AWS SDK? Thanks.
I changed Future to CompletableFuture, and combine two of them (in sequence):
private CompletableFuture<PutObjectResponse> processExportAndUploadAsync()
throws IOException {
CompletableFuture<PutObjectResponse> result = processExportAsync()
.thenCompose(fileName -> uploadS3Async(fileName));
return result;
}
It seems to work.
I have a java 8 based project which performs a certain function on a url. I need to modify the code snippet below so that it is capable of killing the thread/process running and run the next instance after a certain period of time irrespective of current process status.
I tried the following techniques to implement the thread kill procedure:
Executor service
Timer Task
Multithreaded thread kill
The code snippet for my most recent attempt is linked below.
#SuppressWarnings("static-access")
public static void main(String[] args) {
//fetch url from the txt file
List<String> careerUrls = getCareerUrls();
int a = 0;
DBConnection ds = null;
ds = DBConnection.getInstance();
try (java.sql.Connection con = ds.getConnection()) {
//read a single Url
for (String url : careerUrls) {
int c = a++;
ExecutorService executor = Executors.newFixedThreadPool(3);
Future<?> future = executor.submit(new Runnable() {
#Override
// <-- job processing
public void run() {
long end_time = System.currentTimeMillis() + 10000;
System.out.println("STARTED PROCESSING URL: " + url);
jobareaDeciderSample w = new jobareaDeciderSample();
w.mainSample(url, c, con);
}
});
// <-- reject all further submissions
executor.shutdown();
try {
future.get(120, TimeUnit.SECONDS); // <-- wait 2 Minutes to finish
} catch (InterruptedException e) { // <-- possible error cases
System.out.println("job was interrupted");
future.cancel(true);
Thread.currentThread().interrupt();
;
} catch (ExecutionException e) {
System.out.println("caught exception: " + e.getCause());
} catch (TimeoutException e) {
System.out.println("timeout");
future.cancel(true);
}
// wait all unfinished tasks for 2 sec
if (!executor.awaitTermination(0, TimeUnit.SECONDS)) {
// force them to quit by interrupting
executor.shutdownNow();
}
}
} catch (Exception e) {
LOGGER.error(e);
}
}
You are correct with your approach.
calling cancel(true); on future is the right way to stop this task.
You have another problem- you cannot just stop a thread. (well you can, using stop() in thread class, but you should never do this).
cancel(true); sends information to the thread, that it should be stopped. Some java classes are responding to this information and throw interrupted exception. But some dont. You have to modify your task code, to check if Thread.currentThread().isInterrupted(), and if so, stop execution.
This is something you have to do in your code, which you call by
jobareaDeciderSample w = new jobareaDeciderSample();
w.mainSample(url, c, con);
You should do this in some long time spinning code, if you said you do some stuff with url, you should do it in your while loop, where you download information for the web. In other words, do this check only when your code spends 99% of the time.
Also you are calling
Thread.currentThread().interrupt();
in your main thread, this does not do anything for you, as if you want to quit current thread, you can just call return
This question already has answers here:
I want my thread to handle interruption, but I can't catch InterruptedException because it is a checked exception
(4 answers)
Closed 8 years ago.
I have the following code:
public class Net {
public static void main(String[] args) {
Runnable task = new Runnable() {
#Override
public void run() {
String host = "http://example.example";
try {
URL url = new URL(host);
StringBuilder builder = new StringBuilder();
HttpURLConnection con = (HttpURLConnection) url.openConnection();
try(BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()))) {
String line;
while (null != (line = in.readLine())) builder.append(line);
}
out.println("data: " + builder.length());
con.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
};
Thread thread = new Thread(task);
thread.start();
thread.interrupt();
}
}
This "con.getInputStream()" blocks thread, when host is wrong. How to interrupt this code from another thread?
The general rule is to interrupt uninterruptible threads from 'outside', i.e.
Thread waiting for connection/stream - by closing the connection.
Thread waiting for hang up process to finish - by killing the process.
(not specifically this case) A running long loop - by introducing a boolean variable which is set from outside and checked inside the loop from time to time.
Unfortunately you cannot interrupt the thread which has blocked by some I/O operation(unless you utilize NIO).
you may need to close the stream(by another thread) which reading thread has blocked by.
something like this:
public class Foo implements Runnable{
private InputStream stream;
private int timeOut;
....
public void run(){
Thread.sleep(timeOut);
if(<<ensure the victim thread still is stuck>>){
stream.close();//this will throws an exception to the stuck thread.
}
}
....
}
Set a timeout value with setReadTimeout. Catch the SocketTimeoutException if timeout expires and recover or terminate the program the way you want.
This "con.getInputStream()" blocks thread, when host is wrong. How to interrupt this code from another thread?
This is a FAQ. Interrupting a thread will not cause the readLine(...) method to be interrupted. To quote from my answer here:
I want my thread to handle interruption, but I can't catch InterruptedException because it is a checked exception
It is important to realize that t.interrupt() only sets the interrupted bit in a thread -- it doesn't actually interrupt the thread's processing per se. A thread can be interrupted at any time safely.
So there is no way you can interrupt the thread if it is blocked in readLine(...). You could however change your loop to be something like:
while (!Thread.currentThread().isInterrupted()) {
String line = in.readLine();
if (line == null) {
break;
}
builder.append(line);
}
You can, as others have mentioned, closed the underlying InputStream which will cause the readLine() to throw an Exception.
I have written a Bluetooth API for connecting with an external accessory.
The way that the API is designed is that there are a bunch of blocking calls such as getTime, setTime, getVolume, setVolume, etc.
The way these work is that they create a payload to send and call a method called sendAndReceive() which does some prep work and eventually does the following:
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
The problem is that sometimes this device becomes slow or non-responsive so I would like to put a timeout on this call.
I have tried several methods of putting this code in a thread\future task and running it with a timeout, for example:
FutureTask<byte[]> theTask = null;
// create new task
theTask = new FutureTask<byte[]>(
new Callable<byte[]>() {
#Override
public byte[] call() {
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
}
});
// start task in a new thread
new Thread(theTask).start();
// wait for the execution to finish, timeout after 6 secs
byte[] response;
try {
response = theTask.get(6L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
throw new CbtException(e);
} catch (ExecutionException e) {
throw new CbtException(e);
} catch (TimeoutException e) {
throw new CbtCallTimedOutException(e);
}
return response;
}
The problem with this approach is that I can't re-throw exceptions in the call method and since some of the methods throw exceptions I want to forward back to the API's client I can't use this methodology.
Can you recommend some other alternative?
Thanks!
You're saving you can't use the Future<> method because you want to re-throw the exception but in fact this is possible.
Most examples online do implement Callable with the prototype public ? call() but just change it to public ? call() throws Exception and all will be fine: you'll get the exception in the theTask.get() call and you can rethrow it to callers.
I have personally used Executors exactly for bluetooth socket timeout handling on android:
protected static String readAnswer(...)
throws Exception {
String timeoutMessage = "timeout";
ExecutorService executor = Executors.newCachedThreadPool();
Callable<String> task = new Callable<String>() {
public String call() throws Exception {
return readAnswerNoTimeout(...);
}
};
Future<String> future = executor.submit(task);
try {
return future.get(SOCKET_TIMEOUT_MS, TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
future.cancel(true);
throw new Exception(timeoutMessage);
}
}
Why not try something like
public class ReadTask extends Thread {
private byte[] mResultBuffer;
private Exception mCaught;
private Thread mWatcher;
public ReadTask(Thread watcher) {
mWatcher = watcher;
}
public void run() {
try {
mResultBuffer = sendAndReceive();
} catch (Exception e) {
mCaught = e;
}
mWatcher.interrupt();
}
public Exception getCaughtException() {
return mCaught;
}
public byte[] getResults() {
return mResultBuffer;
}
}
public byte[] wrappedSendAndReceive() {
byte[] data = new byte[1024];
ReadTask worker = new ReadTask(data, Thread.currentThread());
try {
worker.start();
Thread.sleep(6000);
} catch (InterruptedException e) {
// either the read completed, or we were interrupted for another reason
if (worker.getCaughtException() != null) {
throw worker.getCaughtException();
}
}
// try to interrupt the reader
worker.interrupt();
return worker.getResults;
}
There is an edge case here that the Thread calling wrappedSendAndReceive() may get interrupted for some reason other than the interrupt from the ReadTask. I suppose a done bit could be added to the ReadTask to allow the other thread to test if the read finished or the interrupt was caused by something else, but I'm not sure how necessary this is.
A further note is that this code does contain the possibility for data loss. If the 6 seconds expires and some amount of data has been read this will end up being discarded. If you wanted to work around this, you'd need to read one byte at a time in ReadTask.run() and then appropriately catch the InterruptedException. This obviously requires a little rework of the existing code to keep a counter and appropriately resize the read buffer when the interrupt is received.