We Keep Coding

How to ask user for input to console (BufferedReader) and display countdown? Need to do it in two threads

Once user entered data timer stops and BuferredReader closed.
If 10 seconds passed and no input - BuferredReader closed and user unable to make input. Below code works, but not 100% correct.
Please suggest any solution.
public class Main {
public static void main(String[] args) throws IOException {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
NewThread nt = new NewThread(br);
Thread newThread = new Thread(nt);
newThread.start();
System.out.print("Please enter data: ");
System.out.println("");
String value = br.readLine();
System.out.println(value);
nt.shutdown();
}
}
class NewThread implements Runnable {
volatile BufferedReader br;
volatile boolean running ;
public NewThread(BufferedReader br) throws IOException {
this.br = br;
this.running = br.ready();
}
#Override
public void run() {
int count = 10;
try {
while (!running) {
System.out.print("("+count +")"+ '\r');
Thread.sleep(1000);
count--;
if (count <0){
shutdown();
}
}
} catch (InterruptedException | IOException e) {
e.printStackTrace();
}
}
public void shutdown () throws IOException {
running=true;
br.close();
}
}

So, firsty you calling method:
br.readLine()
BufferedReader implementation of this method uses synchornized block when waiting for user input. Below I put part of code this method:
String readLine(boolean ignoreLF) throws IOException {
StringBuffer s = null;
int startChar;
synchronized (lock) {
ensureOpen();
...}
Nextly, when you call method shutdown from NewThread(after time out) on your reader, which call close method on buffer - execution of this metod uses synchronized mechanism too:
public void close() throws IOException {
synchronized (lock) {
if (in == null)
return;
try {
in.close();
} finally {
in = null;
cb = null;
}
}
}
so it means that close method will be executed after finished readLine method (exactly after execution synchronized block in readLine method), which is finished when you pass parameter to console.
I suppose that is not possible to close this reader after calling readLine method by standard java mechanism when you use System.in.

Signal thread error to main thread

I have a thread in Java that is connecting to a socket and sending information to another thread, which is processing that information.
Now, if the "producer" thread fails for any reason, I want the whole program to stop, so some sort of notification must happen.
Here's my program (very simplified):
public class Main {
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
new Thread(new Producer(q)).start();
new Thread(new Consumer(q)).start();
// Catch any error in the producer side, then stop both consumer and producer, and do some extra work to notify me that there's an error...
}
}
Main code just creates a shared queue, and starts both producer and consumer. So far, I guess it's ok? Now the Producer code is like this:
public class Producer implements Runnable {
private Queue<String> q;
public Producer(Queue<String> q) {
this.q = q;
}
public void run() {
try {
connectToSocket();
while(true) {
String data = readFromSocket()
q.offer(data);
}
} catch (Exception e) {
// Something really bad happened, notify the parent thread so he stops the program...
}
}
}
Producer connects to socket, reads and sends to queue the string data... The consumer:
public class Consumer implements Runnable {
private Queue<String> q;
public Consumer(Queue<String> q) {
this.q = q;
}
public void run() {
while(true) {
String dataFromSocket = q.poll();
saveData(dataFromSocket);
}
}
}
My code does a lot more than that, but I think it's now self-explanatory what I'm trying to do. I've read about wait() and notify() but I think that wouldn't work, because I don't want to wait my thread for an exception, I want to deal with it in a better way. What are the alternatives?
In general, does my code look reasonable? Would using ExecutorService help here at all?
Thanks a lot!

you can use Thread's UncaughtExceptionHandler
Thread.setDefaultExceptionHandler(
new UncaughtExceptionHandler() {
public void unchaughtException(Thread th, Throwable exception) {
System.out.println("Exception from Thread" + th + ". Exception:" + exception);
}
});
Java docs
http://docs.oracle.com/javase/7/docs/api/java/lang/Thread.UncaughtExceptionHandler.html

The simplest solution given your current code would be to wait for the producer thread to finish and then interrupt the consumer:
Thread producerThread = new Thread(new Producer(q));
producerThread.start();
Thread consumerThread = new Thread(new Consumer(q));
consumerThread.start();
try {
producerThread.join();
} finally {
consumerThread.interrupt();
}
As you mention, an executor would give you a more general purpose way to shut down everything when you need to exit (for example, when a interrupted in the terminal with ctrl-c).
ScheduledExecutorService executor = Executors.newScheduledThreadPool(10);
Producer producer = new Producer(q);
Consumer consumer = new Consumer(q);
executor.submit(producer::run);
executor.submit(consumer::run);
Runtime.getRuntime().addShutdownHook(new Thread(executor::shutdownNow));
Note that your cleanup would have to be more comprehensive than just shutting down the executor. You would have to close the socket beforehand to allow the threads to be interrupted.
Here is a more complete example that handles shutdown from both sides. You can test it by starting a test server with nc -l 1234. Killing either process (nc or the java client) will result in a clean exit of the other.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Socket;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Main {
private ExecutorService executor;
private Socket socket;
private AtomicBoolean running = new AtomicBoolean(true);
public static void main(String[] args) throws IOException {
Main main = new Main();
main.run();
}
private Main() throws IOException {
executor = Executors.newCachedThreadPool();
socket = new Socket("localhost", 1234);
}
private void run() throws IOException {
BlockingQueue<String> q = new SynchronousQueue<>();
Producer producer = new Producer(socket, q);
Consumer consumer = new Consumer(q);
// Start the producer. When it ends call stop
CompletableFuture.runAsync(producer, executor).whenComplete((status, ex) -> stop());
// Start the consumer.
CompletableFuture.runAsync(consumer, executor);
// Add a shutdown hook to stop everything on break
Runtime.getRuntime().addShutdownHook(new Thread(this::stop));
}
private void stop() {
if (running.compareAndSet(true, false)) { // only once
// Close the socket to unblock the producer
try {
socket.close();
} catch (IOException e) {
// ignore
}
// Interrupt tasks
executor.shutdownNow();
try {
// Give tasks some time to clean up
executor.awaitTermination(1, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// ignore
}
}
}
static class Producer implements Runnable {
private BufferedReader in;
private BlockingQueue<String> q;
public Producer(Socket socket, BlockingQueue<String> q) throws IOException {
this.in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
this.q = q;
}
#Override
public void run() {
try {
while (true) {
String data = in.readLine();
if (data == null) {
break;
}
q.put(data);
}
} catch (InterruptedException | IOException e) {
// Fall through
}
System.err.println("Producer done");
}
}
static class Consumer implements Runnable {
private BlockingQueue<String> q;
public Consumer(BlockingQueue<String> q) {
this.q = q;
}
#Override
public void run() {
try {
while (true) {
System.out.println(q.take());
}
} catch (InterruptedException e) {
// done
}
System.err.println("Client done");
}
}
}

Start consumer thread as 'daemon' thread
Mark the consumer thread as 'daemon' and let the main thread end too:
From the Java API doc for Thread.setDaemon(boolean):
Marks this thread as either a daemon thread or a user thread. The Java Virtual Machine exits when the only threads running are all daemon threads.
public class Main {
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
Thread producerThread = new Thread(new Producer(q));
// producerThread.setDefaultUncaughtExceptionHandler(...);
producerThread.start();
Thread consumerThread = new Thread(new Consumer(q));
consumerThread.setDeamon(true);
consumerThread.start();
}
}
This way, your application automatically stops, when the main thread and the producer-thread have terminated (sucessfully or by exception).
You could combine this with the UncaughtExceptionHandler as #Manish suggested, if the main thread needs to know about the producerThread failing...

How about volatile?
public class Main {
volatile boolean isStopMain = false;
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
new Thread(new Producer(q)).start();
new Thread(new Consumer(q)).start();
// Catch any error in the producer side, then stop both consumer and producer, and do some extra work to notify me that there's an error...
while (true) {
if(isStopMain){
System.exit(0); //or other operation to stop the main thread.
}
}
}
}
And In Producer:
public class Producer implements Runnable {
private Queue<String> q;
public Producer(Queue<String> q) {
this.q = q;
}
public void run() {
try {
connectToSocket();
while(true) {
String data = readFromSocket()
q.offer(data);
}
} catch (Exception e) {
// Something really bad happened, notify the parent thread so he stops the program...
Main.isStopMain = true;
}
}
}
And I wonder if you are trying to kill the parent thread in child thread? If yes, here is something you may need to know:How do I terminate parent thread from child?

OutOfMemoryError - No trace in the console

I call the below testMethod, after putting it into a Callable(with other few Callable tasks), from an ExecutorService. I suspect that, the map.put() suffers OutOfMemoryError, as I'm trying to put some 20 million entries.
But, I'm not able to see the error trace in console. Just the thread stops still. I tried to catch the Error ( I know.. we shouldnt, but for debug I caught). But, the error is not caught. Directly enters finally and stops executing.. and the thread stands still.
private HashMap<String, Integer> testMethod(
String file ) {
try {
in = new FileInputStream(new File(file));
br = new BufferedReader(new InputStreamReader(in), 102400);
for (String line; (line= br.readLine()) != null;) {
map.put(line.substring(1,17),
Integer.parseInt(line.substring(18,20)));
}
System.out.println("Loop End"); // Not executed
} catch(Error e){
e.printStackTrace(); //Not executed
}finally {
System.out.println(map.size()); //Executed
br.close();
in.close();
}
return map;
}
Wt could be the mistake, I'm doing?
EDIT: This is how I execute the Thread.
Callable<Void> callable1 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod(inputFile);
return null;
}
};
Callable<Void> callable2 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod1();
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<Callable<Void>>();
taskList.add(callable1);
taskList.add(callable2);
// create a pool executor with 3 threads
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
//executor.invokeAll(taskList);
latch.await();
future.get(0);future.get(1); //Added this as per SubOptimal'sComment
But, this future.get() didn't show OOME in console.

You should not throw away the future after submitting the Callable.
Future future = pool.submit(callable);
future.get(); // this would show you the OOME
example based on the informations of the requestor to demonstrate
public static void main(String[] args) throws InterruptedException, ExecutionException {
Callable<Void> callableOOME = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableOOME");
HashMap<String, Integer> map = new HashMap<>();
// some code to force an OOME
try {
for (int i = 0; i < 10_000_000; i++) {
map.put(Integer.toString(i), i);
}
} catch (Error e) {
e.printStackTrace();
} finally {
System.out.println("callableOOME: map size " + map.size());
}
return null;
}
};
Callable<Void> callableNormal = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableNormal");
// some code to have a short "processing time"
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException ex) {
System.err.println(ex.getMessage());
}
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<>();
taskList.add(callableOOME);
taskList.add(callableNormal);
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
System.out.println("get future 0: ");
future.get(0).get();
System.out.println("get future 1: ");
future.get(1).get();
}

Try catching Throwable as it could be an Exception like IOException or NullPointerException, Throwable captures everything except System.exit();
Another possibility is that the thread doesn't die, instead it becomes increasingly slower and slower due to almost running out of memory but never giving up. You should be able to see this with a stack dump or using jvisualvm while it is running.
BTW Unless all you strings are exactly 16 characters long, you might like to call trim() on the to remove any padding in the String. This could make them shorter and use less memory.
I assume you are using a recent version of Java 7 or 8. If you are using Java 6 or older, it will use more memory as .substring() doesn't create a new underlying char[] to save CPU, but in this case wastes memory.

Text is not getting printed once the Threads are done [duplicate]

This question already has answers here:
How to wait for all threads to finish, using ExecutorService?
(27 answers)
Closed 8 years ago.
Please have a look at the following code.
public class BigFileWholeProcessor {
private static final int NUMBER_OF_THREADS = 2;
public void processFile(String fileName) {
BlockingQueue<String> fileContent = new LinkedBlockingQueue<String>();
BigFileReader bigFileReader = new BigFileReader(fileName, fileContent);
BigFileProcessor bigFileProcessor = new BigFileProcessor(fileContent);
ExecutorService es = Executors.newFixedThreadPool(NUMBER_OF_THREADS);
es.execute(bigFileReader);
es.execute(bigFileProcessor);
es.shutdown();
if(es.isTerminated())
{
System.out.println("Completed Work");
}
}
}
public class BigFileReader implements Runnable {
private final String fileName;
int a = 0;
public static final String SENTINEL = "SENTINEL";
private final BlockingQueue<String> linesRead;
public BigFileReader(String fileName, BlockingQueue<String> linesRead) {
this.fileName = fileName;
this.linesRead = linesRead;
}
#Override
public void run() {
try {
//since it is a sample, I avoid the manage of how many lines you have read
//and that stuff, but it should not be complicated to accomplish
BufferedReader br = new BufferedReader(new FileReader(new File("E:/Amazon HashFile/Hash.txt")));
String str = "";
while((str=br.readLine())!=null)
{
linesRead.put(str);
System.out.println(a);
a++;
}
linesRead.put(SENTINEL);
} catch (Exception ex) {
ex.printStackTrace();
}
System.out.println("Completed");
}
}
public class BigFileProcessor implements Runnable {
private final BlockingQueue<String> linesToProcess;
public BigFileProcessor (BlockingQueue<String> linesToProcess) {
this.linesToProcess = linesToProcess;
}
#Override
public void run() {
String line = "";
try {
while ( (line = linesToProcess.take()) != null) {
//do what you want/need to process this line...
if(line==BigFileReader.SENTINEL)
{
break;
}
String [] pieces = line.split("(...)/g");
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
I want to print the text "completed work" in BigFileWholeProcessor once all the thread work is done. But instead, it is not getting printed. Why is this? How to identify that all the threads are done and need printing?

shutdown() only signal ES to shutdown, you need
awaitTermination(long timeout, TimeUnit unit)
before print message

Use submit() method instead of execute(). The get() method can be used if you want to wait for the thread to finish at any point of time. Read documentation on use of Future object for further details.
ExecutorService es = Executors.newFixedThreadPool(2);
Future<?> f = es.submit(new Thread(new TestRun()));
f.get(); // Wait for result... (i.e similar to `join()` in this case)
es.shutdown(); // Shutdown ExecutorService
System.out.println("Done.");
I have defined a TestRun class implementing Runnable, not shown here. The Future object makes more sense in other scenarios.

Can't stop a task which is started using ExecutorService

Sorry I have to open a new thread to describe this problem.
This morning I asked this question, there're some replies but my problem is still not solved.
This time I will attach some runnable code(simplified but with the same problem) for you to reproduce the problem:
public class ThreadPoolTest {
public static void main(String[] args) throws Exception {
final ExecutorService taskExecutor = Executors.newFixedThreadPool(5);
Future<Void> futures[] = new Future[5];
for (int i = 0; i < futures.length; ++i)
futures[i] = startTask(taskExecutor);
for (int i = 0; i < futures.length; ++i)
System.out.println("futures[i].cancel(true): " + futures[i].cancel(true));
System.out.println("Cancel DONE.");
taskExecutor.shutdown();
}
private static Future<Void> startTask(final ExecutorService taskExecutor) {
Future<Void> f = taskExecutor.submit(new Callable<Void>() {
public Void call() throws Exception {
try {
downloadFile(new URI("http://stackoverflow.com"));
while(true) {
System.out.println(Thread.currentThread().getName() + ": " + Thread.currentThread().isInterrupted());
if(Thread.currentThread().isInterrupted())
break;
}
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}
});
return f;
}
private static void downloadFile (final URI uri) throws Exception {
// if(true) return;
Socket socket = new Socket (uri.getHost(), uri.getPort() == -1 ? 80 : uri.getPort());
return;
}
}
The code above will most likely be trapped in an infinite loop(you may want to run the code multiple times to witness what I saw), as you can see in the main method I have called futures[i].cancel(true) for all tasks, I don't know why this is happening, this has been torturing me for more than a day.
Your help will be greatly appreciated.

I've played with your code, and noticed that the thread's interrupt status is sometimes true before the socket creation, and false after.
I have tried interrupting a thread and calling the Socket constructor, and the thread always stays interrupted after. I also tried removing the shutdown of the threadpool, and the problem continued to happen.
Then I have tried using 5 different URIs, rather than always the same one. And the problem never happened.
So I wrote this simple program, showing that the thread pool is not the culprit, but the socket is:
public static void main(String[] args) throws Exception {
final URI uri = new URI("http://stackoverflow.com");
for (int i = 0; i < 5; i++) {
Runnable r = new Runnable() {
#Override
public void run() {
Thread.currentThread().interrupt();
System.out.println(Thread.currentThread().isInterrupted());
try {
Socket socket = new Socket (uri.getHost(), uri.getPort() == -1 ? 80 : uri.getPort());
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println(Thread.currentThread().isInterrupted());
}
};
new Thread(r).start();
}
}
And indeed, when 5 threads create a socket to the same host and port, 4 of them have their interrupt status cleared.
Then I tried to synchronize the socket creation (on a single lock, but I guess you might use one lock per host/port) :
synchronized(lock) {
try {
Socket socket = new Socket (uri.getHost(), uri.getPort() == -1 ? 80 : uri.getPort());
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
and TADA... the problem disappeared. I would open a bug at Oracle to signal the problem.

I ran your code, and it didn't stop, as you said.
Didn't have much time to investigate why it behaves so, but I found out that declaring the executor service's threads as daemons made the problem go away :
private static ExecutorService TaskExecutor = Executors.newFixedThreadPool(5, new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
});
I'll come back if I find a better explanation.

I think the problem that task are not started when you try to cancel them. I added Thread.sleep(100) like this:
for (int i = 0; i < futures.length; ++i)
futures[i] = startTask(taskExecutor);
Thread.sleep(100);
for (int i = 0; i < futures.length; ++i)
System.out.println("futures[i].cancel(true): " + futures[i].cancel(true));
and everything was cancelled ok.