We Keep Coding

How to triggerShutdown a Guava AbstractScheduledService?

I'm using a few services inheriting from the AbstractScheduledService, which get managed by a ServiceManager. Everything works fine, but now, there's a service whose runOneIteration takes a rather long time, and as the result, my process takes too long to terminate (more than five seconds).
There are other services inheriting from AbstractExecutionThreadService, which had a similar problem, which I could solve via
#Override
protected final void triggerShutdown() {
if (thread != null) thread.interrupt();
}
and storing private volatile thread in the run method. However, there's no triggerShutdown for AbstractScheduledService as stated in this issue.
I already considered alternatives like making runOneIteration do less work, but it's both ugly and inefficient.
I can't override stopAsync as it's final and I can't see anything else. Is there a hook for doing something like this?

Can you work with this? Was there any reason you couldn't add a triggerShutdown yourself?
class GuavaServer {
public static void main(String[] args) throws InterruptedException {
GuavaServer gs = new GuavaServer();
Set<ForceStoppableScheduledService> services = new HashSet<>();
ForceStoppableScheduledService ts = gs.new ForceStoppableScheduledService();
services.add(ts);
ServiceManager manager = new ServiceManager(services);
manager.addListener(new Listener() {
public void stopped() {
System.out.println("Stopped");
}
public void healthy() {
System.out.println("Health");
}
public void failure(Service service) {
System.out.println("Failure");
System.exit(1);
}
}, MoreExecutors.directExecutor());
manager.startAsync(); // start all the services asynchronously
Thread.sleep(3000);
manager.stopAsync();
//maybe make a manager.StopNOW()?
for (ForceStoppableScheduledService service : services) {
service.triggerShutdown();
}
}
public class ForceStoppableScheduledService extends AbstractScheduledService {
Thread thread;
#Override
protected void runOneIteration() throws Exception {
thread = Thread.currentThread();
try {
System.out.println("Working");
Thread.sleep(10000);
} catch (InterruptedException e) {// can your long process throw InterruptedException?
System.out.println("Thread was interrupted, Failed to complete operation");
} finally {
thread = null;
}
System.out.println("Done");
}
#Override
protected Scheduler scheduler() {
return Scheduler.newFixedRateSchedule(0, 1, TimeUnit.SECONDS);
}
protected void triggerShutdown() {
if (thread != null) thread.interrupt();
}
}
}

RejectedExecutionException coming from single thread executor

Below is my method in which I have single thread executor to execute some task in the run method.
private void trigger(final Packet packet) {
// this line is throwing exception
Executors.newSingleThreadExecutor().execute(new Runnable() {
#Override
public void run() {
// some code here
}
});
}
Below is the exception I am getting and I am not sure why? What is the best way to fix this?
error= java.util.concurrent.RejectedExecutionException: Task com.abc.stuffProc$2#e033da0 rejected from java.util.concurrent.ThreadPoolExecutor#76c2da8f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:628)
What happens if my trigger method is called many times and it is still working on the run method from my previous threads? Will it launch as many threads or it will wait for one thread to finish and then start another thread?

See here: What could be the cause of RejectedExecutionException
as you could see from the error log, your ThreadPoolExecutor is Terminated.
Maybe this is what you want:
private void trigger(final Packet packet) {
executor.execute(new Runnable() {
#Override
public void run() {
// some code here
}
});
}
private final ExecutorService executor = Executors.newFixedThreadPool(10);
EDIT Reproduce the Problem with:
public static void main(String[] args) {
final ExecutorTest et = new ExecutorTest();
for (int i = 0; i < 50000; i++) {
et.trigger(i);
}
System.out.println("Done");
}
private void trigger(int i) {
try {
Executors.newSingleThreadExecutor().execute(() -> {
try {
Thread.sleep(1000);
} catch (final InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
});
} catch (final Exception e) {
System.out.println("Test " + i + " with " + Thread.activeCount());
e.printStackTrace();
}
}

Create ThreadPoolExecutor outside of your trigger method. You should not create newSingleThreadExecutor for every call.
private ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
private void trigger(final Packet packet) {
executorService .execute(new Runnable() {
#Override
public void run() {
// some code here
}
});
}
Regarding your exception, please check execute method description.
public void execute(Runnable command)
Executes the given task sometime in the future. The task may execute in a new thread or in an existing pooled thread. If the task cannot be submitted for execution, either because this executor has been shutdown or because its capacity has been reached, the task is handled by the current RejectedExecutionHandler.
Since it's unbounded queue, most likely you have called shutdown some where else in your code.

Shutting down Kafka Consumer

I was reading up details of Kafka High level consumer in this link and saw the below statement -
In practice, a more common pattern is to use sleep indefinitely and
use a shutdown hook to trigger clean shutdown.
Are there any examples for doing this or pointers which could help?

This will be an example of an infinite loop
public void run() {
try {
consumer.subscribe(topics);
while (true) {
ConsumerRecords<String, String> records = consumer.poll(Long.MAX_VALUE);
//do something
}
} catch (WakeupException e) {
// do nothing we are shutting down
} finally {
consumer.close();
}
}
public void shutdown() {
consumer.wakeup();
}
}
And this will be your shutdown hook.
#PostConstruct
private void init(){
addShutdownHook();
}
private void addShutdownHook(){
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
shutdown();
}
}));
}

Java Thread with persistent resources

I have a pool of worker threads (an ExecutorService).
This pool is used to run shell commands.
I use a shell (/bin/sh) rather than creating a process for the executable directly, because I use shell redirects (>) to write the output directly to disk, without having to pass through the JVM, as well as some other niceties.
Spawning a shell process takes 2-3 milliseconds.
I want each thread to keep a shell process to avoid the overhead of starting it.
How do I allow each thread to own a process?
I am thinking of using a ThreadFactory with thread locals.
class ThreadFactory {
Thread newThread(Runnable r) {
return new Thread(new Runnable() {
Process process = Runtime.getRuntime().exec("/bin/sh")
try {
// store process as thread local here
r.run(); // then r can access thread local
} catch(Exception e) {
try {
process.close();
} catch(Exception e) {
}
throw e;
}
});
}
}
(Alternatively, I could subclass Thread and cast Thread.currentThread() to that class in my Runnable.)
Is this a good approach to solving this problem?

I would keep the Process reference in a ProcessRunnable that continuously executes commands. I think is more clear than using a ThreadLocal and a ThreadFactory. Something like this:
public class ShellCommandExecutor {
private int concurrency = 10;
private int capacity = 100;
private ExecutorService service = Executors.newFixedThreadPool(concurrency);
private BlockingQueue<String> commandsQueue = new LinkedBlockingQueue<>(capacity);
public void start() {
for (int i = 0; i < concurrency; i++)
service.submit(new Runnable() {
#Override
public void run() {
//todo deal with ioexception
Process process = Runtime.getRuntime().exec("/bin/sh");
while (!Thread.currentThread().isInterrupted()) {
try {
String command = commandsQueue.take();
//todo execute commands using the same process per thread
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
});
}
public void executeCommand(String command) throws InterruptedException {
commandsQueue.put(command);
}
public void shutdown() {
service.shutdownNow();
}
}
EDIT: a solution with thread local that should work easily with cached thread pools:
public class ShellCommandExecutor2 {
//todo limit queue
private ExecutorService service = Executors.newCachedThreadPool();
public void executeCommand(final String command) throws InterruptedException {
service.submit(new Runnable() {
#Override
public void run() {
Process process = ThreadLocalProcessFactory.get();
//todo execute command
}
});
}
public void shutdown() {
service.shutdownNow();
}
private static class ThreadLocalProcessFactory {
private static final ThreadLocal<Process> processThreadLocal =
new ThreadLocal<Process>() {
#Override protected Process initialValue() {
try {
return Runtime.getRuntime().exec("/bin/sh");
}
catch (IOException e) {
e.printStackTrace();
return null;
}
}
};
static Process get() {
return processThreadLocal.get();
}
}
}

Run a new thread and don't wait this thread finish

I am new to the Threading, so if please give me an advice for my case.
I would like create a new thread to do something and I don't care this thread can do complete or not.
I intend to use ExecutorCompletionService to do my job but this class is not suitable for me. It must call take or poll to drain a queue to avoid memory leak. So, this means I must wait until the thread complete. I read this from this question
This is the current code
ExecutorService executor = Executors.newCachedThreadPool();
CompletionService<Entity> completion = new ExecutorCompletionService<>(executor);
DoSomeThingClass doSomething = getInstance();
completion.submit(doSomething);
executor.shutdown();
// Continue to do other job and I don't care whenever doSomeThing is complete.
// However when doSomeThing finish, I don't need to do anything to avoid memory leak
For that reason, please give me an approach for my case and some skeleton code for example.
Thank you so much

You can mark this thread as "Daemon". And when your main thread completed, your app will exit.
public static void main(String[] args)
{
Thread t = new Thread(new Runnable() {
#Override
public void run() {
try {
TimeUnit.SECONDS.sleep(2);
} catch(InterruptedException e) {}
System.out.println("Thread 2 is finished");
}
});
t.setDaemon(true);
t.start();
System.out.println("Thread 1 is finished");
}

You can use Spring TaskExecutor, it is very useful to raise a thread to run a task.
import org.springframework.core.task.TaskExecutor;
public class TaskExecutorExample {
private class MessagePrinterTask implements Runnable {
private String message;
public MessagePrinterTask(String message) {
this.message = message;
}
public void run() {
System.out.println(message);
}
}
private TaskExecutor taskExecutor;
public TaskExecutorExample(TaskExecutor taskExecutor) {
this.taskExecutor = taskExecutor;
}
public void printMessages() {
for(int i = 0; i < 25; i++) {
taskExecutor.execute(new MessagePrinterTask("Message" + i));
}
}
}
You can check Spring Task Execution documentation here:
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/scheduling.html

Along with you code your Future concept
Future ft=completion.submit(doSomething);
ft.get(timeOut, TimeUnit.MILLISECONDS);
here you can specify Time to execute Thread if it fail to get execute thread get kill(not 100% sure)means it try to interrupt the thread and try to kill

I can resolve my problem as the following code
public static void main(
String[] args) {
ExecutorService executor = Executors.newCachedThreadPool();
executor.execute(new Runnable() {
#Override
public void run() {
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
} finally {
System.out.println("Thread 2 is finished");
}
}
});
executor.shutdown();
System.out.println("Thread 1 is finished");
}