I have a pool of worker threads (an ExecutorService).
This pool is used to run shell commands.
I use a shell (/bin/sh) rather than creating a process for the executable directly, because I use shell redirects (>) to write the output directly to disk, without having to pass through the JVM, as well as some other niceties.
Spawning a shell process takes 2-3 milliseconds.
I want each thread to keep a shell process to avoid the overhead of starting it.
How do I allow each thread to own a process?
I am thinking of using a ThreadFactory with thread locals.
class ThreadFactory {
Thread newThread(Runnable r) {
return new Thread(new Runnable() {
Process process = Runtime.getRuntime().exec("/bin/sh")
try {
// store process as thread local here
r.run(); // then r can access thread local
} catch(Exception e) {
try {
process.close();
} catch(Exception e) {
}
throw e;
}
});
}
}
(Alternatively, I could subclass Thread and cast Thread.currentThread() to that class in my Runnable.)
Is this a good approach to solving this problem?
I would keep the Process reference in a ProcessRunnable that continuously executes commands. I think is more clear than using a ThreadLocal and a ThreadFactory. Something like this:
public class ShellCommandExecutor {
private int concurrency = 10;
private int capacity = 100;
private ExecutorService service = Executors.newFixedThreadPool(concurrency);
private BlockingQueue<String> commandsQueue = new LinkedBlockingQueue<>(capacity);
public void start() {
for (int i = 0; i < concurrency; i++)
service.submit(new Runnable() {
#Override
public void run() {
//todo deal with ioexception
Process process = Runtime.getRuntime().exec("/bin/sh");
while (!Thread.currentThread().isInterrupted()) {
try {
String command = commandsQueue.take();
//todo execute commands using the same process per thread
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
});
}
public void executeCommand(String command) throws InterruptedException {
commandsQueue.put(command);
}
public void shutdown() {
service.shutdownNow();
}
}
EDIT: a solution with thread local that should work easily with cached thread pools:
public class ShellCommandExecutor2 {
//todo limit queue
private ExecutorService service = Executors.newCachedThreadPool();
public void executeCommand(final String command) throws InterruptedException {
service.submit(new Runnable() {
#Override
public void run() {
Process process = ThreadLocalProcessFactory.get();
//todo execute command
}
});
}
public void shutdown() {
service.shutdownNow();
}
private static class ThreadLocalProcessFactory {
private static final ThreadLocal<Process> processThreadLocal =
new ThreadLocal<Process>() {
#Override protected Process initialValue() {
try {
return Runtime.getRuntime().exec("/bin/sh");
}
catch (IOException e) {
e.printStackTrace();
return null;
}
}
};
static Process get() {
return processThreadLocal.get();
}
}
}
Related
Faced the fact that when the database is unavailable, the queue grows because tasks stop running. What is the best way to set some timeout for tasks executed in method run()? May be there is some good approach with using ExecutorService?
#Service
public class AsyncWriter implements Writer, Runnable {
private LinkedBlockingQueue<Entry> queue = new LinkedBlockingQueue<>();
private volatile boolean terminate = false;
private AtomicInteger completedCounter = new AtomicInteger();
#PostConstruct
private void runAsyncWriter() {
Thread async = new Thread(this);
async.setName("Writer Thread");
async.setPriority(2);
async.start();
}
#Override
public void run() {
while (!terminate) {
try {
Entry entry = queue.take();
dao.save(entry);
completedCounter.incrementAndGet();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
#Override
public void write(Entry entry) {
queue.add(entry);
}
}
Maybe you can try RxJava
https://www.baeldung.com/rx-java
And you can set your aync funtions
Timeout in RxJava
I'm using a few services inheriting from the AbstractScheduledService, which get managed by a ServiceManager. Everything works fine, but now, there's a service whose runOneIteration takes a rather long time, and as the result, my process takes too long to terminate (more than five seconds).
There are other services inheriting from AbstractExecutionThreadService, which had a similar problem, which I could solve via
#Override
protected final void triggerShutdown() {
if (thread != null) thread.interrupt();
}
and storing private volatile thread in the run method. However, there's no triggerShutdown for AbstractScheduledService as stated in this issue.
I already considered alternatives like making runOneIteration do less work, but it's both ugly and inefficient.
I can't override stopAsync as it's final and I can't see anything else. Is there a hook for doing something like this?
Can you work with this? Was there any reason you couldn't add a triggerShutdown yourself?
class GuavaServer {
public static void main(String[] args) throws InterruptedException {
GuavaServer gs = new GuavaServer();
Set<ForceStoppableScheduledService> services = new HashSet<>();
ForceStoppableScheduledService ts = gs.new ForceStoppableScheduledService();
services.add(ts);
ServiceManager manager = new ServiceManager(services);
manager.addListener(new Listener() {
public void stopped() {
System.out.println("Stopped");
}
public void healthy() {
System.out.println("Health");
}
public void failure(Service service) {
System.out.println("Failure");
System.exit(1);
}
}, MoreExecutors.directExecutor());
manager.startAsync(); // start all the services asynchronously
Thread.sleep(3000);
manager.stopAsync();
//maybe make a manager.StopNOW()?
for (ForceStoppableScheduledService service : services) {
service.triggerShutdown();
}
}
public class ForceStoppableScheduledService extends AbstractScheduledService {
Thread thread;
#Override
protected void runOneIteration() throws Exception {
thread = Thread.currentThread();
try {
System.out.println("Working");
Thread.sleep(10000);
} catch (InterruptedException e) {// can your long process throw InterruptedException?
System.out.println("Thread was interrupted, Failed to complete operation");
} finally {
thread = null;
}
System.out.println("Done");
}
#Override
protected Scheduler scheduler() {
return Scheduler.newFixedRateSchedule(0, 1, TimeUnit.SECONDS);
}
protected void triggerShutdown() {
if (thread != null) thread.interrupt();
}
}
}
I am creating my own thread pool and future object which can execute callable interface parallel. Executor provides shutdown method to stop all worker threads from running. If i am creating a thread pool like below how should I implement the shutdown method to stop after all threads have finished execution?
My custom thread pool looks like this
class MyThreadPool implements java.util.concurrent.Executor
{
private final java.util.concurrent.BlockingQueue<Callable> queue;
public MyThreadPool(int numThreads) {
queue = new java.util.concurrent.LinkedBlockingQueue<>();
for (int i=0 ; i<numThreads ; i++) {
new Thread(new Runnable(){
#Override
public void run() {
while(true) {
queue.take().call();
}
}
}).start();
}
}
#Override
public <T> Future<T> submit(Callable<T> callable) {
FutureTask<T> future = new FutureTask(callable);
queue.put(future);
return future;
}
public void shutdown(){ }
}
I couldnt think of a way to keep list of thread and then check if they are idle or not?
You definitely should hold references to the threads you're creating. For instance, set up a field threads of type List<Thread> and add the threads to this list from within the constructor.
Afterwards, you could implement shutdown() with the help of Thread#join():
public void shutdown() {
for (Thread t : threads) {
try {
t.join();
} catch (InterruptedException e) { /* NOP */ }
}
}
Don't forget to replace while (true) with an appropriate condition (which you toggle in shutdown()) and consider using BlockingQueue#poll(long, TimeUnit) rather than take().
EDIT: Something like:
public class MyThreadPool implements Executor {
private List<Thread> threads = new ArrayList<>();
private BlockingDeque<Callable> tasks = new LinkedBlockingDeque<>();
private volatile boolean running = true;
public MyThreadPool(int numberOfThreads) {
for (int i = 0; i < numberOfThreads; i++) {
Thread t = new Thread(() -> {
while (running) {
try {
Callable c = tasks.poll(5L, TimeUnit.SECONDS);
if (c != null) {
c.call();
}
} catch (Exception e) { /* NOP */ }
}
});
t.start();
threads.add(t);
}
}
public void shutdown() {
running = false;
for (Thread t : threads) {
try {
t.join();
} catch (InterruptedException e) { /* NOP */ }
}
}
// ...
}
I want to make a single thread which would contain 3 infinite tasks.
I want one task to run at a time and start/stop running task when required.
For example first I want task 1 to run, then I want task 2 to run but after stopping task 1 and again I want task 1 to run but after stopping of task 2 and so on.
Infinite task needs to check some condition and if that condition is satisfied perform some operations and if not satisfied sleep for few seconds and after wake up perform the above same operations again.
Infinite Runnable task looks some thing like this:
new Runnable(){
while(1){
if(TaskQueue.getInstance().size()<= 100){
TaskQueue.getInstance().push("add command to the end of queue");
}else{
try {
Thread.sleep(10000);
}catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
Any help would be appreciated?
Edit : I modified my question. I want a continuous single running thread(some thing like looper ) to monitor 3 infinite tasks and control this single continuous running thread tasks from outside.
Use this for start/stop thread in real-time:
class MyThread extends Thread {
private volatile boolean running = true; // Run unless told to pause
...
#Override
public void run() {
// Only keep painting while "running" is true
// This is a crude implementation of pausing the thread
while (true) {
if (Thread.currentThread().isInterrupted()) {
return;
}
if (running) {
//Your code
} else yield;
}
}
public void pauseThread() throws InterruptedException {
running = false;
}
public void resumeThread() {
running = true;
}
}
For pause thread use this:
myThread.pauseThread();
For resume thread use this:
myThread.resumeThread();
For stop thread use this (Not recommended):
myThread.stop();
For currently stop thread use this:
myThread.interrupt();
You must use a class like Thread that already implements Runnable.
new Thread(){....};
And the way it works it's:
Thread t = new Thread(){.....};
t.start();
t.stop();
You could also initialize a new thread, like:
Thread exampleThread = new thread();
After this you can start it at any point in your code by:
exampleThread.start();
you can use Semaphore,
to Manage the amount of signal.
private final static Semaphore semaphore = new Semaphore(0);
public static void main(String[] args) throws Exception {
//入口
threadTest();
}
public static void thread1() {
try{
//…… some code
}
finally{
semaphore.release();
}
}
public static void thread2() {
semaphore.acquire(1);
}
The question is my first answer,thanks.
I finally made my task scheduler. The API of which looks something like this:
TaskScheduler taskScheduler = TaskScheduler.getInstance();
taskScheduler.startTaskOne();
taskScheduler.stopTaskOne();
taskScheduler.startTaskTwo();
taskScheduler.stopTaskTwo();
Runs one task at a time (because I used Executors.newSingleThreadExecutor()).
We can control the execution of the task from outside:
public class TaskScheduler {
private static ExecutorService mTaskRunningService;
private static TaskScheduler mInstance;
private Future mFirstTaskFuture = null;
private Future mSecondTaskFuture = null;
static {
configure();
}
private static void configure() {
mTaskRunningService = Executors.newSingleThreadExecutor();
}
public static TaskScheduler getInstance() {
if (mInstance == null) {
mInstance = new TaskScheduler();
}
return mInstance;
}
private Runnable mTaskOneRunnable = new Runnable() {
#Override
public void run() {
try {
while (true) {
/** stop this single thread (i.e executing one task at time) service if this thread is interrupted
* from outside because documentation of {#link java.util.concurrent.ThreadPoolExecutor#shutdownNow()}
* says we need to do this*/
if (Thread.currentThread().isInterrupted()) {
return;
}
// task one work.......
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
private Runnable mTaskTwoRunnable = new Runnable() {
#Override
public void run() {
try {
while (true) {
/** stop this single thread (i.e executing one task at time) service if this thread is interrupted
* from outside because documentation of {#link java.util.concurrent.ThreadPoolExecutor#shutdownNow()}
* says we need to do this*/
if (Thread.currentThread().isInterrupted()) {
return;
}
// task two work......
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
public synchronized void startTaskOne() {
if (mFirstTaskFuture == null) {
// start executing runnable
mFirstTaskFuture = mTaskRunningService.submit(mTaskOneRunnable);
}
}
public synchronized boolean stopTaskOne() {
if (mFirstTaskFuture != null) {
// stop general reading thread
mFirstTaskFuture.cancel(true);
// cancel status
boolean status = mFirstTaskFuture.isDone();
// assign null because startTaskOne() again be called
mGeneralFuture = null;
return status;
}
return true;
}
public synchronized void startTaskTwo() {
if (mSecondTaskFuture == null) {
// start executing runnable
mSecondTaskFuture = mTaskRunningService.submit(mTaskTwoRunnable);
}
}
public synchronized boolean stopTaskTwo() {
if (mSecondTaskFuture != null) {
// clear task queue
mTaskQueue.clearTaskQueue();
// stop 22 probes reading thread
mSecondTaskFuture.cancel(true);
// cancel status
boolean status = mSecondTaskFuture.isDone();
// assign null because startTaskTwo() again be called
mSecondTaskFuture = null;
return status;
}
return true;
}
}
I have Java thread class which purpose is to process commands when they arrive.
My current implementation uses Thread.sleep(50) for checking for new commands, but I want to make it more elegant using wait/notify.
How to do that without introducing bugs ? Here is my code:
protected BlockingQueue<Command> currentCmds = new LinkedBlockingDeque<Command>();
#Override
public void run() {
while (!dead) {
Thread.sleep(50);
if (!currentCmds.isEmpty()) {
Command cmd = currentCmds.remove();
processCmd(cmd);
}
}
}
public void sendCommand(Command command) {
currentCmds.add(command);
}
I suspect this approach is slowing down my server when many clients are connected.
I have Java thread class which purpose is to process commands when they arrive.
Basically you want an ExecutorService which is a thread pool and a queue in one.
private final ExecutorService es = Executors.newSingleThreadExecutor();
public void sendRunnable(Runnable run) {
es.submit(run);
}
public void sendCommand(Command command) {
es.submit(new Runnable() {
public void run() {
try {
command.call();
} catch (Throwable t) {
t.printStackTrace();
}
}
});
}
An even simpler solution is to just submit Runnable or Callable and not use Command at all.