We Keep Coding

Why did we declare this method synchronized?

Consider this class:
// Synchronizing access to shared mutable data using Object
// methods wait and notifyAll.
public class SynchronizedBuffer implements Buffer
{
private int buffer = -1; // shared by producer and consumer threads
private boolean occupied = false;
// place value into buffer
public synchronized void blockingPut(int value) throws InterruptedException
{
// while there are no empty locations, place thread in waiting state
while (occupied)
{
// output thread information and buffer information, then wait
System.out.println("Producer tries to write."); // for demo only
displayState("Buffer full. Producer waits."); // for demo only
wait();
}
buffer = value; // set new buffer value
// indicate producer cannot store another value
// until consumer retrieves current buffer value
occupied = true;
displayState("Producer writes " + buffer); // for demo only
notifyAll(); // tell waiting thread(s) to enter runnable state
} // end method blockingPut; releases lock on SynchronizedBuffer
// return value from buffer
public synchronized int blockingGet() throws InterruptedException
{
// while no data to read, place thread in waiting state
while (!occupied)
{
// output thread information and buffer information, then wait
System.out.println("Consumer tries to read."); // for demo only
displayState("Buffer empty. Consumer waits."); // for demo only
wait();
}
// indicate that producer can store another value
// because consumer just retrieved buffer value
occupied = false;
displayState("Consumer reads " + buffer); // for demo only
notifyAll(); // tell waiting thread(s) to enter runnable state
return buffer;
} // end method blockingGet; releases lock on SynchronizedBuffer
// display current operation and buffer state; for demo only
private synchronized void displayState(String operation)
{
System.out.printf("%-40s%d\t\t%b%n%n", operation, buffer, occupied);
}
} // end class SynchronizedBuffer
And this paragraph from the book:
Notice that method displayState is a synchronized method. This is important because it, too, reads the SynchronizedBuffer’s shared mutable data. Though only one thread at a time may acquire a given object’s lock, one thread may acquire the same object’s lock multiple times—this is known as a reentrant lock and enables one synchronized method to invoke another on the same object.
Why did we declare method the displayState() as synchronized although it's called only from a synchronized method and thus when it's called the calling thread already has the monitor lock on the object?

You are right in questioning this source code. When a method owns the object’s monitor, entering a synchronized method or block acquiring the same monitor again, has no effect at all.
It’s also unlikely that the author considered the possibility that future changes to the code could invoke the method without owning the object’s monitor. First of all, the method’s entire purpose is to report the ongoing operation, second, enforcing correct usage could be achieve even simpler.
It’s also interesting that the method is used inconsistently. The method does always report the current value of buffer, but two of the four callers redundantly append the current value of buffer to the operation argument string before calling the method. Then, there are additional explicit print statements before the two of the callers. Since their message is “[Producer|Consumer] tries to [write|read]”, these statements are very likely supposed to report the attempt before the loop, instead of inside, at a place where it is already known that the attempt failed.
public class SynchronizedBuffer //implements Buffer
{
private int buffer = -1; // shared by producer and consumer threads
private boolean occupied = false;
// place value into buffer
public synchronized void blockingPut(int value)
throws InterruptedException
{
System.out.println("Producer tries to write."); // for demo only
// while there are no empty locations, place thread in waiting state
while(occupied)
{
// output thread information and buffer information, then wait
displayState("Buffer full. Producer waits.", buffer, occupied);// demo only
wait();
}
buffer = value; // set new buffer value
// indicate producer cannot store another value
// until consumer retrieves current buffer value
occupied = true;
displayState("Producer writes ", buffer, occupied); // for demo only
notifyAll(); // tell waiting thread(s) to enter runnable state
} // end method blockingPut; releases lock on SynchronizedBuffer
// return value from buffer
public synchronized int blockingGet() throws InterruptedException
{
System.out.println("Consumer tries to read."); // for demo only
// while no data to read, place thread in waiting state
while(!occupied)
{
// output thread information and buffer information, then wait
displayState("Buffer empty. Consumer waits.", buffer, occupied);// demo only
wait();
}
// indicate that producer can store another value
// because consumer just retrieved buffer value
occupied = false;
displayState("Consumer reads ", buffer, occupied); // for demo only
notifyAll(); // tell waiting thread(s) to enter runnable state
return buffer;
} // end method blockingGet; releases lock on SynchronizedBuffer
// display current operation and buffer state; for demo only
// not accessing the object, hence no synchronization needed
private static void displayState(String operation, int buffer, boolean occupied)
{
System.out.printf("%-40s%d\t\t%b%n%n", operation, buffer, occupied);
}
}
By turning the state to be reported into parameters, the responsibility to access them in a thread safe manner clearly lies at the caller and by making the method static, it’s impossible for the method to incorrectly access object state. Also, redundant reporting of a variable’s value would become apparent at the call site.

Java thread stuck after join

I have this Transmitter class, which contains one BufferedReader and one PrintWriter. The idea is, on the main class, to use Transmitter.receive() and Transmitter.transmit() to the main socket. The problem is:
public void receive() throws Exception {
// Reads from the socket
Thread listener = new Thread(new Runnable() {
public void run() {
String res;
try {
while((res = input.readLine()) != null) {
System.out.println("message received: " + res);
outputMessage = (res);
if (res.equals("\n")) {
break;
}
}
} catch (IOException e) {
e.printStackTrace();
}
};
});
listener.start();
listener.join();
}
The thread changes the 'outputMessage' value, which I can get using an auxiliary method. The problem is that, without join, my client gets the outputMessage but I want to use it several times on my main class, like this:
trans1.receive();
while(trans1.getOutput() == null);
System.out.println("message: " + trans1.getOutput());
But with join this system.out never executes because trans1.receive() is stuck... any thoughts?
Edit 1: here is the transmitter class https://titanpad.com/puYBvlVery

You might send \n; that doesn't mean that you will see it in your Java code.
As it says in the Javadoc for BufferedReader.readLine() (emphasis mine):
(Returns) A String containing the contents of the line, not including any line-termination characters
so "\n" will never be returned.

Doing this:
{
Thread listener = new Thread(new Runnable() {
public void run() {
doSomeWork();
};
});
listener.start();
listener.join();
}
will create a new thread and then wait for it to do its work and finish. Therefore it's more or less the same as just directly doing:
doSomeWork();
The new thread doesn't serve any real purpose here.
Also, the extra thread introduces synchronization problems because in your code you don't make sure your variables are synchronized.
Thirdly, your thread keeps reading lines from the input in a loop until there's nothing more to read and unless the other side closes the stream, it will block on the readLine() call. What you will see in with getOutput() will be a random line that just happens to be there at the moment you look, the next time you look it might be the same line, or some completely different line; some lines will be read and forgotten immediatelly without you ever noticing it from the main thread.
You can just call input.readLine() directly in your main thread when you actually need to get a new line message from the input, you don't need an extra reader thread. You could store the read messages into a Queue as yshavit suggests, if that's desirable, e.g. for performance reasons it might be better to read the messages as soon as they are available and have them ready in memory. But if you only need to read messages one by one then you can simply call input.readLine() only when you actually need it.

Non blocking function that preserves order

I have the following method:
void store(SomeObject o) {
}
The idea of this method is to store o to a permanent storage but the function should not block. I.e. I can not/must not do the actual storage in the same thread that called store.
I can not also start a thread and store the object from the other thread because store might be called a "huge" amount of times and I don't want to start spawning threads.
So I options which I don't see how they can work well:
1) Use a thread pool (Executor family)
2) In store store the object in an array list and return. When the array list reaches e.g. 1000 (random number) then start another thread to "flush" the array list to storage. But I would still possibly have the problem of too many threads (thread pool?)
So in both cases the only requirement I have is that I store persistantly the objects in exactly the same order that was passed to store. And using multiple threads mixes things up.
How can this be solved?
How can I ensure:
1) Non blocking store
2) Accurate insertion order
3) I don't care about any storage guarantees. If e.g. something crashes I don't care about losing data e.g. cached in the array list before storing them.

I would use a SingleThreadExecutor and a BlockingQueue.
SingleThreadExecutor as the name sais has one single Thread. Use it to poll from the Queue and persist objects, blocking if empty.
You can add not blocking to the queue in your store method.
EDIT
Actually, you do not even need that extra Queue - JavaDoc of newSingleThreadExecutor sais:
Creates an Executor that uses a single worker thread operating off an unbounded queue. (Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.) Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Unlike the otherwise equivalent newFixedThreadPool(1) the returned executor is guaranteed not to be reconfigurable to use additional threads.
So I think it's exactly what you need.
private final ExecutorService persistor = Executors.newSingleThreadExecutor();
public void store( final SomeObject o ){
persistor.submit( new Runnable(){
#Override public void run(){
// your persist-code here.
}
} );
}
The advantage of using a Runnable that has a quasi-endless-loop and using an extra queue would be the possibility to code some "Burst"-functionality. For example you could make it wait to persist only when 10 elements are in queue or the oldest element has been added at least 1 minute ago ...

I suggest using a Chronicle-Queue which is a library I designed.
It allows you to write in the current thread without blocking. It was originally designed for low latency trading systems. For small messages it takes around 300 ns to write a message.
You don't need to use a back ground thread, or a on heap queue and it doesn't wait for the data to be written to disk by default. It also ensures consistent order for all readers. If the program dies at any point after you call finish() the message is not lost. (Unless the OS crashes/loses power) It also supports replication to avoid data loss.

Have one separate thread that gets items from the end of a queue (blocking on an empty queue), and writes them to disk. Your main thread's store() function just adds items to the beginning of the queue.
Here's a rough idea (though I assume there will be cleaner or faster ways for doing this in production code, depending on how fast you need things to be):
import java.util.*;
import java.io.*;
import java.util.concurrent.*;
class ObjectWriter implements Runnable {
private final Object END = new Object();
BlockingQueue<Object> queue = new LinkedBlockingQueue();
public void store(Object o) throws InterruptedException {
queue.put(o);
}
public ObjectWriter() {
new Thread(this).start();
}
public void close() throws InterruptedException {
queue.put(END);
}
public void run() {
while (true) {
try {
Object o = queue.take();
if (o == END) {
// close output file.
return;
}
System.out.println(o.toString()); // serialize as appropriate
} catch (InterruptedException e) {
}
}
}
}
public class Test {
public static void main(String[] args) throws Exception {
ObjectWriter w = new ObjectWriter();
w.store("hello");
w.store("world");
w.close();
}
}

The comments in your question make it sound like you are unfamilier with multi-threading, but it's really not that difficult.
You simply need another thread responsible for writing to the storage which picks items off a queue. - your store function just adds the objects to the in-memory queue and continues on it's way.
Some psuedo-ish code:
final List<SomeObject> queue = new List<SomeObject>();
void store(SomeObject o) {
// add it to the queue - note that modifying o after this will also alter the
// instance in the queue
synchronized(queue) {
queue.add(queue);
queue.notify(); // tell the storage thread there's something in the queue
}
}
void storageThread() {
SomeObject item;
while (notfinished) {
synchronized(queue) {
if (queue.length > 0) {
item = queue.get(0); // get from start to ensure same order
queue.removeAt(0);
} else {
// wait for something
queue.wait();
continue;
}
}
writeToStorage(item);
}
}

Does this method in runnable object needs synchronization?

The following method belongs to an object A that implements Runnable. It's called asynchronously by other method from the object A and by code inside the run method (so, it's called from other thread, with a period of 5 seconds).
Could I end up with file creation exceptions?
If i make the method synchronized... the lock is always acquired over the object A ?
The fact that one of the callers is at the run() method confuses me :S
Thanks for your inputs.
private void saveMap(ConcurrentMap<String, String> map) {
ObjectOutputStream obj = null;
try {
obj = new ObjectOutputStream(new FileOutputStream("map.txt"));
obj.writeObject(map);
} catch (IOException ex) {
Logger.getLogger(MessagesFileManager.class.getName()).log(Level.SEVERE, null, ex);
} finally {
try {
obj.close();
} catch (IOException ex) {
Logger.getLogger(MessagesFileManager.class.getName()).log(Level.SEVERE, null, ex);
}
}
notifyActionListeners();
}

Synchronized instance methods use the this object as the lock and prevent simultaneous execution of all synchronized instance methods (even other ones) from different threads.
To answer your question regarding requirements for synchronization, the answer is basically yes because you have multiple threads accessing the same method, so output may collide.
As a design comment, I would make your saveMap method static, because it doesn't access any fields (it's stateless), and it more strongly indicates that output to the file is not dependent on the instance, so it's more obvious that file output may collide with other instances.
Edited:
Here's the code for what I'm suggesting:
private static synchronized void saveMap(Map<String, String> map) {
...
}
FYI, static synchronized methods use the class object (ie MyClass.class), which is a singleton, as the lock object.

It's called asynchronously by other method from the object A and by code inside the run method (so, it's called from other thread, with a period of 5 seconds).
Given that saveMap is called from multiple threads, without synchronization you cannot guarantee that two threads won't try to write to the same file concurrently. This will cause an incorrectly-formatted file when it happens.
The simplest solution is to make the method synchronized.
private synchronized void saveMap(ConcurrentMap<String, String> map) { ... }
If the map is large enough, this may cause unresponsiveness in your program. Another option is to write to a temporary file (a new file each time it's called) and then use synchronization while swapping the new file over map.txt by renaming and deleting.
private void saveMap(ConcurrentMap<String, String> map) {
File file = ... original code to write to a temporary file ...
if (file != null) {
synchronized(this) {
... move file over map.txt ...
}
notifyActionListeners();
}
}
Keep in mind that swapping two files won't be an atomic operation. Any external program or thread from the same program may catch the short time that map.txt doesn't exist. I was unable to find an atomic file-swap method in Java, but maybe with some searching you will.

Need advice on synchronization of Java Vector / ConcurrentModificationException

In a legacy application I have a Vector that keeps a chronological list of files to process and multiple threads ask it for the next file to process. (Note that I realize that there are likely better collections to use (feel free to suggest), but I don't have time for a change of that magnitude right now.)
At a scheduled interval, another thread checks the working directory to see if any files appear to have been orphaned because something went wrong. The method called by this thread occasionally throws a ConcurrentModificationException if the system is abnormally busy. So I know that at least two threads are trying to use the Vector at once.
Here is the code. I believe the issue is the use of the clone() on the returned Vector.
private synchronized boolean isFileInDataStore( File fileToCheck ){
boolean inFile = false;
for( File wf : (Vector<File>)m_dataStore.getFileList().clone() ){
File zipName = new File( Tools.replaceFileExtension(fileToCheck.getAbsolutePath(), ZIP_EXTENSION) );
if(wf.getAbsolutePath().equals(zipName.getAbsolutePath()) ){
inFile = true;
break;
}
}
return inFile;
}
The getFileList() method is as follows:
public synchronized Vector<File> getFileList() {
synchronized(fileList){
return fileList;
}
}
As a quick fix, would changing the getFileList method to return a copy of the vector as follows suffice?
public synchronized Vector<File> getFileListCopy() {
synchronized(fileList){
return (Vector<File>)fileList.clone();
}
}
I must admit that I am generally confused by the use of synchronized in Java as it pertains to collections, as simply declaring the method as such is not enough. As a bonus question, is declaring the method as synchronized and wrapping the return call with another synchronized block just crazy coding? Looks redundant.
EDIT: Here are the other methods which touch the list.
public synchronized boolean addFile(File aFile) {
boolean added = false;
synchronized(fileList){
if( !fileList.contains(aFile) ){
added = fileList.add(aFile);
}
}
notifyAll();
return added;
}
public synchronized void removeFile( File dirToImport, File aFile ) {
if(aFile!=null){
synchronized(fileList){
fileList.remove(aFile);
}
// Create a dummy list so I can synchronize it.
List<File> zipFiles = new ArrayList<File>();
synchronized(zipFiles){
// Populate with actual list
zipFiles = (List<File>)diodeTable.get(dirToImport);
if(zipFiles!=null){
zipFiles.remove(aFile);
// Repopulate list if the number falls below the number of importer threads.
if( zipFiles.size()<importerThreadCount ){
diodeTable.put(dirToImport, getFileList( dirToImport ));
}
}
}
notifyAll();
}
}

Basically, there are two separate issues here: sycnhronization and ConcurrentModificationException. Vector in contrast to e.g. ArrayList is synchronized internally so basic operation like add() or get() do not need synchronization. But you can get ConcurrentModificationException even from a single thread if you are iterating over a Vector and modify it in the meantime, e.g. by inserting an element. So, if you performed a modifying operation inside your for loop, you could break the Vector even with a single thread. Now, if you return your Vector outside of your class, you don't prevent anyone from modifyuing it without proper synchronization in their code. Synchronization on fileList in the original version of getFileList() is pointless. Returning a copy instead of original could help, as could using a collection which allows modification while iterating, like CopyOnWriteArrayList (but do note the additional cost of modifications, it may be a showstopper in some cases).

"I am generally confused by the use of synchronized in Java as it
pertains to collections, as simply declaring the method as such is not
enough"
Correct. synchronized on a method means that only one thread at a time may enter the method. But if the same collection is visible from multiple methods, then this doesn't help much.
To prevent two threads accessing the same collection at the same time, they need to synchronize on the same object - e.g. the collection itself. You have done this in some of your methods, but isFileInDataStore appears to access a collection returned by getFileList without synchronizing on it.
Note that obtaining the collection in a synchronized manner, as you have done in getFileList, isn't enough - it's the accessing that needs synchronizing. Cloning the collection would (probably) fix the issue if you only need read-access.
As well as looking at synchronizing, I suggest you track down which threads are involved - e.g. print out the call stack of the exception and/or use a debugger. It's better to really understand what's going on than to just synchronize and clone until the errors go away!

Where does the m_dataStore get updated? That's a likely culprit if it's not synchronized.

First, you should move your logic to whatever class is m_dataStore if you haven't.
Once you've done that, make your list final, and synchronize on it ONLY if you are modifying its elements. Threads that only need to read it, don't need synchronized access. They may end up polling an outdated list, but I suppose that is not a problem. This gets you increased performance.
As far as I can tell, you would only need to synchronize when adding and removing, and only need to lock your list.
e.g.
package answer;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Example {
public static void main(String[] args)
{
Example c = new Example();
c.runit();
}
public void runit()
{
Thread.currentThread().setName("Thread-1");
new Thread("Thread-2")
{
#Override
public void run() {
test1(true);
}
}.start();
// Force a scenario where Thread-1 allows Thread-2 to acquire the lock
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
}
// At this point, Thread-2 has acquired the lock, but it has entered its wait() method, releasing the lock
test1(false);
}
public synchronized void test1(boolean wait)
{
System.out.println( Thread.currentThread().getName() + " : Starting...");
try {
if (wait)
{
// Apparently the current thread is supposed to wait for some other thread to do something...
wait();
} else {
// The current thread is supposed to keep running with the lock
doSomeWorkThatRequiresALockLikeRemoveOrAdd();
System.out.println( Thread.currentThread().getName() + " : Our work is done. About to wake up the other thread(s) in 2s...");
Thread.sleep(2000);
// Tell Thread-2 that it we have done our work and that they don't have to spare the CPU anymore.
// This essentially tells it "hey don't wait anymore, start checking if you can get the lock"
// Try commenting this line and you will see that Thread-2 never wakes up...
notifyAll();
// This should show you that Thread-1 will still have the lock at this point (even after calling notifyAll).
//Thread-2 will not print "after wait/notify" for as long as Thread-1 is running this method. The lock is still owned by Thread-1.
Thread.sleep(1000);
}
System.out.println( Thread.currentThread().getName() + " : after wait/notify");
} catch (InterruptedException ex) {
Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
}
}
private void doSomeWorkThatRequiresALockLikeRemoveOrAdd()
{
// Do some work that requires a lock like remove or add
}
}