I had a look at JAVA and tried to play around with an ExecutorService. Unfortunately my executor doesn’t start my runnable objects.
I’m trying to get some information from different XML files, which are stored in the files list.
FileFinder fileFinder = new FileFinder(path);
List<File>files = fileFinder.getFiles();
ExecutorService threadPool = Executors.newFixedThreadPool(configReader.getThreadcount(), new ThreadFactory() {
#Override
public Thread newThread(Runnable r) {
return new EinbucherThread();
}
});
for(File file : files)
{
System.out.println("Started working");
USEinbucher einbucher = new USEinbucher(file, verbindung);
threadPool.execute(einbucher);
}
threadPool.shutdown();
try {
while(!threadPool.awaitTermination(1, TimeUnit.SECONDS)) {
i++;
System.out.println("waiting "+i );
;
}
} catch (InterruptedException e) {
e.printStackTrace();
}
I thought I could increase the performance by putting the unmarshaller in my threads. So I don’t need to create an unmarshaller for each file, but just once per thread (as far as I understood the API each thread can be used multiple times).
public class EinbucherThread extends Thread {
private Unmarshaller um;
public EinbucherThread() {
try {
JAXBContext jb = JAXBContext.newInstance("klassen");
um = jb.createUnmarshaller();
System.out.println("Thread was created");
} catch (JAXBException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public Unmarshaller getUm() {
return um;
}
Unfortunately it seems, as if the run method of my runnable class is never reached.
public class USEinbucher implements Runnable {
private File lieferung;
private Verbindung verbindung;
public USEinbucher(File lieferung, Verbindung verbindung) {
this.lieferung=lieferung;
this.verbindung=verbindung;
}
#Override
public void run()
{
System.out.println("Started to work");
einbuchen();
}
I inserted some println for debugging. With three files and a threadcount of two my output looks like:
Started working
Thread was created
Started working
Thread was created
Started working
Thread was created
waiting 1
waiting 2
waiting 3…
Any explanation is appreciated.
The ThreadFactory.newThread should return a Thread that is responsible for running the parameter Runnable object. Consider passing the Runnable parameter to your Thread object. For example:
#Override
public Thread newThread(Runnable r) {
return new EinbucherThread(r);
}
//in the constructor of EinbucherThread
public EinbucherThread (Runnable r){
super(r);
}
Related
Faced the fact that when the database is unavailable, the queue grows because tasks stop running. What is the best way to set some timeout for tasks executed in method run()? May be there is some good approach with using ExecutorService?
#Service
public class AsyncWriter implements Writer, Runnable {
private LinkedBlockingQueue<Entry> queue = new LinkedBlockingQueue<>();
private volatile boolean terminate = false;
private AtomicInteger completedCounter = new AtomicInteger();
#PostConstruct
private void runAsyncWriter() {
Thread async = new Thread(this);
async.setName("Writer Thread");
async.setPriority(2);
async.start();
}
#Override
public void run() {
while (!terminate) {
try {
Entry entry = queue.take();
dao.save(entry);
completedCounter.incrementAndGet();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
#Override
public void write(Entry entry) {
queue.add(entry);
}
}
Maybe you can try RxJava
https://www.baeldung.com/rx-java
And you can set your aync funtions
Timeout in RxJava
I am new to the multithreading programming. I am having a problem in understanding the behaviour of synchronized method access by multiple instance of the class.
In the below code am trying to implement wait and poll mechanism. Where am waiting for a response from one service for some time and if that service returned response within that time i will return back.
In this I have implemented two synchronized blocks. I can understand that two threads cant able to access the synchronized method at the same time. Here only I can get confused what if multiple instances of WaitAndPoll class created and invoked at the same time.
Will each instance execute one by one. If that so it will affect the performance badly in that case can anyone advise how to simplify this ?
WAIT and POLL:
public class WaitAndPoll {
Model model;
OSBService osbService;
WaitAndPoll(Model model, OSBService th1){
this.model = model;
this.osbService=th1;
}
// Prints a string and waits for consume()
public void waitingForOSBResponse()throws InterruptedException
{
synchronized(this)
{
System.out.println("waitingForOSBResponse thread running "+this.model.str);
this.osbService.start();
wait();
if(this.model.str==null) { // checking the response is still null if so calling java function
this.osbService.interrupt(); //This will interupt osb service thread
System.out.println(" Calling the java function");
}else{
System.out.println(" Response successfully returned from the OSBService :: "+this.model.str);
}
}
}
//Polling for every 1 second
public void pollingOSBResponse()throws InterruptedException
{
Thread.sleep(200);
synchronized(this)
{
int count=0;
while(this.model.str == null && count<3){
wait(1000);
System.out.println("wating for the modification");
++count;
}
System.out.println("Polling completed");
notify();
}
}
}
OSBService:
import java.util.Date;
public class OSBService extends Thread{
Model model;
OSBService(Model model){
this.model= model;
}
public void run(){
System.out.println("calling the osb webservice:: "+this.model.str);
try {
Thread.sleep(5000); //Simulating the wating period for the response
this.model.str="modified";
System.out.println("called the osb webservice:: "+this.model.str);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
System.err.println("OSB service interrupted because of longer time for response ::: "+this.model.str+" :: "+new Date().toString());
}catch (Exception e){
e.printStackTrace();
}
}
}
Main class:
public class Main
{
public static void main(String[] args) throws InterruptedException
{
Model model = new Model();
model.str=null;
OSBService osbService = new OSBService(model);
final WaitAndPoll waitAndPoll = new WaitAndPoll(model,osbService);
//Calling the OSB service and waiting for its response
Thread t1 = new Thread(new Runnable()
{
#Override
public void run()
{
try
{
waitAndPoll.waitingForOSBResponse();
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
});
//Polling whether the osb reponse received or not
Thread t2 = new Thread(new Runnable()
{
#Override
public void run()
{
try
{
waitAndPoll.pollingOSBResponse();
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
});
t1.start();
t2.start();
t1.join();
t2.join();
}
}
If multiple instance are created, then the methods will be try to acquire mutex lock in synchronized block on different instances of the WaitAndPoll class.
If you are passing different instances to the threads then there is no guarantee which thead will execute first the synchronized block.
I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?
There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)
I have a pool of worker threads (an ExecutorService).
This pool is used to run shell commands.
I use a shell (/bin/sh) rather than creating a process for the executable directly, because I use shell redirects (>) to write the output directly to disk, without having to pass through the JVM, as well as some other niceties.
Spawning a shell process takes 2-3 milliseconds.
I want each thread to keep a shell process to avoid the overhead of starting it.
How do I allow each thread to own a process?
I am thinking of using a ThreadFactory with thread locals.
class ThreadFactory {
Thread newThread(Runnable r) {
return new Thread(new Runnable() {
Process process = Runtime.getRuntime().exec("/bin/sh")
try {
// store process as thread local here
r.run(); // then r can access thread local
} catch(Exception e) {
try {
process.close();
} catch(Exception e) {
}
throw e;
}
});
}
}
(Alternatively, I could subclass Thread and cast Thread.currentThread() to that class in my Runnable.)
Is this a good approach to solving this problem?
I would keep the Process reference in a ProcessRunnable that continuously executes commands. I think is more clear than using a ThreadLocal and a ThreadFactory. Something like this:
public class ShellCommandExecutor {
private int concurrency = 10;
private int capacity = 100;
private ExecutorService service = Executors.newFixedThreadPool(concurrency);
private BlockingQueue<String> commandsQueue = new LinkedBlockingQueue<>(capacity);
public void start() {
for (int i = 0; i < concurrency; i++)
service.submit(new Runnable() {
#Override
public void run() {
//todo deal with ioexception
Process process = Runtime.getRuntime().exec("/bin/sh");
while (!Thread.currentThread().isInterrupted()) {
try {
String command = commandsQueue.take();
//todo execute commands using the same process per thread
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
});
}
public void executeCommand(String command) throws InterruptedException {
commandsQueue.put(command);
}
public void shutdown() {
service.shutdownNow();
}
}
EDIT: a solution with thread local that should work easily with cached thread pools:
public class ShellCommandExecutor2 {
//todo limit queue
private ExecutorService service = Executors.newCachedThreadPool();
public void executeCommand(final String command) throws InterruptedException {
service.submit(new Runnable() {
#Override
public void run() {
Process process = ThreadLocalProcessFactory.get();
//todo execute command
}
});
}
public void shutdown() {
service.shutdownNow();
}
private static class ThreadLocalProcessFactory {
private static final ThreadLocal<Process> processThreadLocal =
new ThreadLocal<Process>() {
#Override protected Process initialValue() {
try {
return Runtime.getRuntime().exec("/bin/sh");
}
catch (IOException e) {
e.printStackTrace();
return null;
}
}
};
static Process get() {
return processThreadLocal.get();
}
}
}
I am trying to develop active object pattern in concurrent Java using java.util.concurrent classes.
I describe it using a Client and a Server. A sample Server is as:
class Server implements Runnable {
public final LinkedBlockingQueue que = new LinkedBlockingQueue();
private final ExecutorService es = Executors.newCachedThreadPool();
private Message currentMessage;
private boolean state = false;
public init() {
es.submit(this);
}
public void requestForServer() {
if (state) {
this.currentMessage.await();
}
state = true;
}
public void run() {
for(;;) {
Message m = que.take();
this.currentMessage = m;
this.es.submit(m);
}
}
}
And a sample Client:
class Client {
private Server server;
public Client(Server s) {
this.server = s;
}
public void doSomething() {
Message m = new Message(new Callable() {
public Object call() {
server.requestForServer();
}
});
this.server.que.add(m);
}
}
And a sample Message encapsulation is:
class Message<V> extends FutureTask<V> {
private Lock lock = new ReentrantLock();
private Condition condition = new Condition();
public Message(Callable<V> callable) {
super(callable);
}
public void run() {
try {
lock.lock();
super.run();
lock.unlock();
} catch(Exception e) {}
}
public void await() {
try {
condition.await();
} catch(Exception e) {}
}
public void signal() {
try {
condition.signalAll();
} catch(Exception e) {}
}
}
And a sample running code:
Server s = new Server();
Client c = new Client (s);
s.init();
c.doSomething();
I dropped some implementation details to get my message across.
Now, the problem is when in Server the state is true so the incoming message should wait and the await is called on the current message. However, I get IllegalMonitorStateException which means that the current message does not own the current thread to await on it. But, I believe this is strange since the current message gets called in the Server and its thread pool so the current message has also an access to the current thread of execution.
I'd be most thankful for any ideas or suggestions, or with a known working implementation of this pattern using java.util.concurrent. Thanks in advance.
UPDATE:
I discussed the solution I could deploy in this blog post. I hope it could help.
You have to actually acquire the lock when you await on its corresponding condition. Without that lock you cannot associate yourself to the condition directly. To demonstrate this:
public void await() {
lock.lock();
try {
condition.await();
} catch(Exception e) {}
finally{
lock.unlock();
}
}
That should resolve your IllegalMonitorStateException
On a side note of correctness you should always release a lock in a try{ } finally{ } manner, you can observe what I wrote as an example. The reason for this is if an exception occurs between lock().lock(); and super.run(); lock.unlock() will never be called.