We Keep Coding

What's the best way to implement a list of elements that will have to have elements added/removed from different threads?

I'm currently trying to implement a system list that would run in a few different threads:
1) First thread is listening to incoming requests and adds them to the list.
2) A new thread is created for each request to perform certain operations.
3) Another thread iterates through the list, checks the status of each request, and removes them from the list when they're complete.
Now, the way I have it in a very simplified pseudocode can be viewed below:
private List<Job> runningJobs = new ArrayList<>(); // our list of requests
private Thread monitorThread;
private Runnable monitor = new Runnable() { // this runnable is later called in a new thread to monitor the list and remove completed requests
#Override
public void run() {
boolean monitorRun = true;
while(monitorRun) {
try {
Thread.sleep(1000);
if (runningJobs.size()>0){
Iterator<Job> i = runningJobs.iterator();
while (i.hasNext()) {
try {
Job job = i.next();
if (job.jobStatus() == 1) { // if job is complete
i.remove();
}
}
catch (java.util.ConcurrentModificationException e){
e.printStackTrace();
}
}
}
if (Thread.currentThread().isInterrupted()){
monitorRun = false;
}
} catch (InterruptedException e) {
monitorRun = false;
}
}
}
};
private void addRequest(Job job){
this.runningJobs.add(newJob);
// etc
}
In short, the Runnable monitor is what runs continuously in the third thread; the first thread is calling addRequest() occasionally.
While my current implementation somewhat works, I'm concerned about the order of operations here and possible java.util.ConcurrentModificationException (and the system is anything but robust). I'm certain there is a much better way to organize this mess.
What's the proper or a better way to do this?

Your requirements would be met nicely with an ExecutorService. For each request, create Job, and submit it to the service. Internally, the service uses a BlockingQueue, which would address your question directly, but you don't have to worry about it with an ExecutorService.
Specifically, something like this:
/* At startup... */
ExecutorService workers = Executors.newCachedThreadPool();
/* For each request... */
Job job = ... ;
workers.submit(job); /* Assuming Job implements Runnable */
// workers.submit(job::jobEntryPoint); /* If Job has some other API */
/* At shutdown... */
workers.shutdown();

There are a few different ways.
You can synchronize the list. This is possibly the most brute-force and still wouldn't help prevent an insert while you are iterating over it.
There are a few synchronized* collections. These tend to be better but have ramifications. For instance CopyOnWriteArrayList will work but it creates a new array list each time (that you would assign back to the variable). This is good for occasionally updated collections.
There is a ConcurrentLinkedQueue--Since it's "Linked" you can't reference an item in the middle.
Look through the implementations of the "List" interface and pick the one that best suits your problem.
If your problem is a queue instead of a list, there are a few implementations of that as well and they will tend to be better suited for that type of problem.
In general my answer is that you should probably scan through the Javadocs every time java does a major release and examine (at least) the new collections. You might be surprised at the stuff that's in there.

Why is 'create' asynchronous?

I was told, that creating a new instance is always an async message; but I don't understand why.
e.g.:
Foo myFoo = new Foo();
Here I will have to wait until the constructor finishes and returns my new object. But doesn't asynchronous mean, that I go on independently (no waiting) - like starting a thread?

I was told, that creating a new instance is always an async message;
Sorry, I have to say that either you heard it wrong or you were told something that is wrong. But first off, we should get some terminology straight. The term "async" or "asynchronous" means that the invocation returns immediately to the caller. We can easily demonstrate that this is not true with a constructor, with a simple experiment [1]. In other words, the constructor must return for the caller to make any progress.
Starting a thread is indeed asynchronous. The call to Thread.start() returns immediately and at some later point in time the thread actually starts running and executing the run() method.
1 The Experiment
Consider your class (for illustration only) is like below:
Foo.java
class Foo {
Foo() throws InterruptedException {
while (true) {
System.out.println("not returning yet ...");
Thread.sleep(2000);
}
}
public static void main(String[] args) throws InterruptedException {
Foo foo = new Foo();
}
}
If you compiled and run this class (I used Java 8 on my Mac, but that is not a requirement). As expected, this class runs forever producing the output every 2 seconds:
not returning yet ...
not returning yet ...
not returning yet ...
not returning yet ...
Note that the sleep call was added just to make it bearable. You could try this experiment without it, but then your program will overwhelm one of the CPU's by stressing it to 100%.
If, while it is running, you took a thread dump (for example, by using the command jstack), you see something like below (curtailed for brevity):
"main" #1 prio=5 os_prio=31 tid=0x00007f9522803000 nid=0xf07
waiting on condition [0x000000010408f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at Foo.<init>(Foo.java:5)
at Foo.main(Foo.java:9)
Regardless of the state of the thread (RUNNABLE, BLOCKED, WAITING, TIMED_WAITING), you will always see (take various thread dumps to see what this means) you will always see these two lines:
at Foo.<init>(Foo.java:5)
at Foo.main(Foo.java:9)
which means that the caller (in this case, the main thread) will never make any progress. And since this constructor never returns, no progress happens.

I was told, that creating a new instance is always an async message;
No, java constructors doesn't have implyed synchronization. Anyway, you can have concurrency issues within it. There is not guarantee that all the fields are initialized after the constructor call.
Here I will have to wait until the constructor finishes and returns my new object. But doesn't asynchronous mean, that I go on independently (no waiting) - like starting a thread?
Nope, you don't have to wait. You can access the object within another thread.
I suggest you to read this thread.

Example for concurrent access to an object while the constructor is still executing:
public class Demo {
private volatile int constructionCounter;
private volatile String string;
public Demo() throws InterruptedException {
super();
assert this.constructionCounter == 0;
this.constructionCounter++;
// From this point on, there's no way the constructionCounter can ever be != 1 again, because the constructor will never run more than once, concurrently or otherwise.
assert this.constructionCounter == 1;
final Demo newInstance = this;
Thread t = new Thread( new Runnable() {
public void run() {
// Access new instance from another thread.
// Race condition here; may print "Hello null" or "Hello World" depending on whether or not the constructor already finished.
System.out.println("Hello " + newInstance.getString());
}
});
t.start();
this.setString( "World!" );
}
public String setString( String str ) {
this.string = str;
}
public String getString() {
return this.string;
}
}
Note that this is only ever possible if and when the constructor itself somehow hands this out to another thread.

What happens when few threads trying to call the same synchronized method?

so I got this horses race and when a horse getting to the finishing line, I invoke an arrival method. Let's say I got 10 threads, one for each horse, and the first horse who arrives indeed invoking 'arrive':
public class FinishingLine {
List arrivals;
public FinishingLine() {
arrivals = new ArrayList<Horse>();
}
public synchronized void arrive(Horse hourse) {
arrivals.add(hourse);
}
}
Ofc I set the arrive method to synchronized but I dont completely understand what could happen if it wasnt synchronized, the professor just said it wouldn't be safe.
Another thing that I would like to understand better is how it is decided which thread will after the first one has been finished? After the first thread finished 'arrive' and the method get unlocked, which thread will run next?

1) It is undefined what the behaviour would be, but you should assume that it is not what you would want it to do in any way that you can rely upon.
If two threads try to add at the same time, you might get both elements added (in either order), only one element added, or maybe even neither.
The pertinent quote from the Javadoc is:
Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. (A structural modification is any operation that adds or deletes one or more elements, or explicitly resizes the backing array; merely setting the value of an element is not a structural modification.)
2) This is down to how the OS schedules the threads. There is no guarantee of "fairness" (execution in arrival order) for regular synchronized blocks, although there are certain classes (Semaphore is one) which give you the choice of a fair execution order.
e.g. you can implement a fair execution order by using a Semaphore:
public class FinishingLine {
List arrivals;
final Semaphore semaphore = new Semaphore(1, true);
public FinishingLine() {
arrivals = new ArrayList<Horse>();
}
public void arrive(Horse hourse) {
semaphore.acquire();
try {
arrivals.add(hourse);
} finally {
semaphore.release();
}
}
}
However, it would be easier to do this with a fair blocking queue, which handles the concurrent access for you:
public class FinishingLine {
final BlockingQueue queue = new ArrayBlockingQueue(NUM_HORSES, true);
public void arrive(Horse hourse) {
queue.add(hourse);
}
}

Non blocking function that preserves order

I have the following method:
void store(SomeObject o) {
}
The idea of this method is to store o to a permanent storage but the function should not block. I.e. I can not/must not do the actual storage in the same thread that called store.
I can not also start a thread and store the object from the other thread because store might be called a "huge" amount of times and I don't want to start spawning threads.
So I options which I don't see how they can work well:
1) Use a thread pool (Executor family)
2) In store store the object in an array list and return. When the array list reaches e.g. 1000 (random number) then start another thread to "flush" the array list to storage. But I would still possibly have the problem of too many threads (thread pool?)
So in both cases the only requirement I have is that I store persistantly the objects in exactly the same order that was passed to store. And using multiple threads mixes things up.
How can this be solved?
How can I ensure:
1) Non blocking store
2) Accurate insertion order
3) I don't care about any storage guarantees. If e.g. something crashes I don't care about losing data e.g. cached in the array list before storing them.

I would use a SingleThreadExecutor and a BlockingQueue.
SingleThreadExecutor as the name sais has one single Thread. Use it to poll from the Queue and persist objects, blocking if empty.
You can add not blocking to the queue in your store method.
EDIT
Actually, you do not even need that extra Queue - JavaDoc of newSingleThreadExecutor sais:
Creates an Executor that uses a single worker thread operating off an unbounded queue. (Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.) Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Unlike the otherwise equivalent newFixedThreadPool(1) the returned executor is guaranteed not to be reconfigurable to use additional threads.
So I think it's exactly what you need.
private final ExecutorService persistor = Executors.newSingleThreadExecutor();
public void store( final SomeObject o ){
persistor.submit( new Runnable(){
#Override public void run(){
// your persist-code here.
}
} );
}
The advantage of using a Runnable that has a quasi-endless-loop and using an extra queue would be the possibility to code some "Burst"-functionality. For example you could make it wait to persist only when 10 elements are in queue or the oldest element has been added at least 1 minute ago ...

I suggest using a Chronicle-Queue which is a library I designed.
It allows you to write in the current thread without blocking. It was originally designed for low latency trading systems. For small messages it takes around 300 ns to write a message.
You don't need to use a back ground thread, or a on heap queue and it doesn't wait for the data to be written to disk by default. It also ensures consistent order for all readers. If the program dies at any point after you call finish() the message is not lost. (Unless the OS crashes/loses power) It also supports replication to avoid data loss.

Have one separate thread that gets items from the end of a queue (blocking on an empty queue), and writes them to disk. Your main thread's store() function just adds items to the beginning of the queue.
Here's a rough idea (though I assume there will be cleaner or faster ways for doing this in production code, depending on how fast you need things to be):
import java.util.*;
import java.io.*;
import java.util.concurrent.*;
class ObjectWriter implements Runnable {
private final Object END = new Object();
BlockingQueue<Object> queue = new LinkedBlockingQueue();
public void store(Object o) throws InterruptedException {
queue.put(o);
}
public ObjectWriter() {
new Thread(this).start();
}
public void close() throws InterruptedException {
queue.put(END);
}
public void run() {
while (true) {
try {
Object o = queue.take();
if (o == END) {
// close output file.
return;
}
System.out.println(o.toString()); // serialize as appropriate
} catch (InterruptedException e) {
}
}
}
}
public class Test {
public static void main(String[] args) throws Exception {
ObjectWriter w = new ObjectWriter();
w.store("hello");
w.store("world");
w.close();
}
}

The comments in your question make it sound like you are unfamilier with multi-threading, but it's really not that difficult.
You simply need another thread responsible for writing to the storage which picks items off a queue. - your store function just adds the objects to the in-memory queue and continues on it's way.
Some psuedo-ish code:
final List<SomeObject> queue = new List<SomeObject>();
void store(SomeObject o) {
// add it to the queue - note that modifying o after this will also alter the
// instance in the queue
synchronized(queue) {
queue.add(queue);
queue.notify(); // tell the storage thread there's something in the queue
}
}
void storageThread() {
SomeObject item;
while (notfinished) {
synchronized(queue) {
if (queue.length > 0) {
item = queue.get(0); // get from start to ensure same order
queue.removeAt(0);
} else {
// wait for something
queue.wait();
continue;
}
}
writeToStorage(item);
}
}

Controlling thread using wait() and notify()

(Problem solved, solution below)
I have 2 classes: Equip and Command. The equip is an equipment that run commands, but I need it to be able to run only 1 command at the same time.
A command is a thread, that executes on the run() function, while Equip is a normal class that don't extend anything.
Currently I have the following setup to run the commands:
Command class:
#Override
public void run() {
boolean execute = equip.queueCommand(this);
if (!execute) {
// if this command is the only one on the queue, execute it, or wait.
esperar();
}
// executes the command.....
equip.executeNextCommand();
}
synchronized public void esperar() {
try {
this.wait();
} catch (Exception ex) {
Log.logErro(ex);
}
}
synchronized public void continue() {
this.notifyAll();
}
Equip class:
public boolean queueCommand(Command cmd) {
// commandQueue is a LinkedList
commandQueue.addLast(cmd);
return (commandQueue.size() == 1);
}
public void executeNextCommand() {
if (commandQueue.size() >= 1) {
Command cmd = commandQueue.pollFirst();
cmd.continue();
}
}
However, this is not working. Basically, the notify() isn't waking the command thread, so it'll never execute.
I searched about the wait and notify protocol, but I couldn't find anything wrong with the code. I also tried calling the wait() directly from the queueCommand() method, but then the execution of the queueCommand stopped, and it also didn't do what it was supposed to do.
Is this approach correct and I'm missing something or this is completely wrong and I should implement a Monitor class to manipulate the concurrent threads?
EDIT: I solved the problem using another completely different approach, using Executors, thanks to #Gray.
Here's the final code, it might help someone someday:
Equip class:
private ExecutorCompletionService commandQueue = new ExecutorCompletionService(Executors.newFixedThreadPool(1));
public void executeCommand(Command cmd, boolean waitCompletion) {
commandQueue.submit(cmd, null);
if (waitCompletion) {
try {
commandQueue.take();
} catch (Exception ex) {
}
}
}
In the Command class I just have a method to encapsulate the equip's execute method.
The boolean waitCompletion is used when I need the result of the command at the same time, and instead of calling a new thread to execute it, I just execute and wait, pretending that it's executing on the same thread. This question contains a good discussion on this matter: When would you call java's thread.run() instead of thread.start()?. And yes, this is a case where it's useful to call .run() instead of .start().

There are a large number of race conditions that exist in your code if Command.run() is called from multiple threads. Unless this is some sort of homework question where you have to implement the code yourself, I would highly recommend using one of the Java Executors which were added in 1.6. In this case the Executors.newSingleThreadExecutor() is what you need to limit the number of running background tasks to 1. This will allow an unlimited number of tasks to be submitted to the ExecutorService, but only one of those tasks will be executing at any one time.
If you need the thread that is submitting the tasks to block when another task is already running then you would use something like the following. This sets up a pool of a maximum of 1 thread and uses a SynchronousQueue which blocks until the worker thread consumes the job:
final ExecutorService executorServer =
new ThreadPoolExecutor(0, 1, 60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
But if that was the case then you would just call the task directly inside of a synchronized block and you wouldn't need the ExecutorService.
Lastly, for any new concurrency programmer (of any language) I would recommend that you take the time to read some documentation on the subject. Until you start recognizing the concurrent pitfalls inherent in threading even the simplest set of classes, it will be a frustrating process to get your code to work. Doug Lea's book is one of the bible's on the subject. My apologies if I have underestimated your experience in this area.

I think you should not have "synchronized" on the esperar method. That will block using the object instances as the locking object. Any other thread that attempts to wait will block AT ENTRY TO THE METHOD, not on the wait. So, the notifyAll will release the one thread that got into the method first. Of the remaining callers, only one will proceed with a call to esperar, which will then block on the wait(). Rinse and repeat.

ExectutorService is the way to go. But if you want to do-it-yourself, or need to do something fancier, I offer the following.
I gather than this whole thing is driven by Equip's queueCommand, which might be callled from any thread anywhere at any time. For starters, the two methods in Equip should by synchronized so commandQueue does not get trashed. (You might use ConcurrentLinkedQueue, but be careful with your counts.) Better still, put the code in each method in a block synchronized by queueCommand.
But further, I think your two classes work better combined. Switching Command to a simple Runnable, I'd try something like this:
class Equip {
private Object queueLock = new Object(); // Better than "this".
private LinkedList<Runnable> commandQueue = new LinkedList<Runnable>();
private void run() {
for (;;) {
Runnable cmd = equip.getNextCommand();
if (cmd == null) {
// Nothing to do.
synchronized (queueLock) { queueLock.wait(); }
}
else
cmd.run();
}
}
// Adds commands to run.
public boolean queueCommand( Runnable cmd ) {
synchronized (queueCommand) { commandQueue.addLast( cmd ); }
synchronized (queueLock) {
// Lets "run" know queue has something in it if it
// is in a wait state.
queueLock.notifyAll();
}
}
private Runnable getNextCommand() {
synchronized (queueCommand) { return commandQueue.pollFirst(); }
}
}
You'll need to catch some exceptions, and figure out how to start things up and shut them down, but this should give an idea of how the wait and notify work. (I'd look for some way to know when "run" was not waiting so I could skip synching on queueLock in queueCommand, but walk before you run.)