We Keep Coding

Java FileWriter thread safety [duplicate]

This question already has answers here:
Threads and file writing
(6 answers)
Closed 9 years ago.
i have 20 threads that write with the println() function on a file called results.txt. How can i synchronize them all?
I note every time my program run i have different number of lines of text in results.txt.
Thank you.

Access the file through a class that contains a synchronized method to write to the file. Only one thread at a time will be able to execute the method.
I think that Singleton pattern would fit for your problem:
package com.test.singleton;
public class Singleton {
private static final Singleton inst= new Singleton();
private Singleton() {
super();
}
public synchronized void writeToFile(String str) {
// Do whatever
}
public static Singleton getInstance() {
return inst;
}
}
Every time you need to write to your file, you only would have to call:
Singleton.getInstance().writeToFile("Hello!!");

Duplicate question ... duplicate answer. As I said here:
If you can hold your file as a FileOutputStream you can lock it like this:
FileOutputStream file = ...
....
// Thread safe version.
void write(byte[] bytes) {
try {
boolean written = false;
do {
try {
// Lock it!
FileLock lock = file.getChannel().lock();
try {
// Write the bytes.
file.write(bytes);
written = true;
} finally {
// Release the lock.
lock.release();
}
} catch ( OverlappingFileLockException ofle ) {
try {
// Wait a bit
Thread.sleep(0);
} catch (InterruptedException ex) {
throw new InterruptedIOException ("Interrupted waiting for a file lock.");
}
}
} while (!written);
} catch (IOException ex) {
log.warn("Failed to lock " + fileName, ex);
}
}

You are intend to write data into one file. So if you try to lock the whole file, it'd better to use a single thread to do this job. Although you spawn 20 threads, but there is only one of them is running every time you call the method, the others are just waiting for the lock.
I recommend you use RandomAccessFile to write data to your file. Then each thread can write some unique data into to the file without locking the whole file.
Some demo code as following
try {
final RandomAccessFile file = new RandomAccessFile("/path/to/your/result.txt", "rw");
final int numberOfThread = 20;
final int bufferSize = 512;
ExecutorService pool = Executors.newFixedThreadPool(numberOfThread);
final AtomicInteger byteCounter = new AtomicInteger(0);
final byte[] yourText = "Your data".getBytes();
for (int i = 0; i < yourText.length; i++) {
pool.submit(new Runnable() {
#Override
public void run() {
int start = byteCounter.getAndAdd(bufferSize);
int chunkSize = bufferSize;
if (start + bufferSize > yourText.length) {
chunkSize = yourText.length - start;
}
byte[] chunkData = new byte[chunkSize];
System.arraycopy(yourText, start, chunkData, 0, chunkSize);
try {
file.write(chunkData);
} catch (IOException e) {
//exception handle
}
}
});
}
file.close();
} catch (Exception e) {
//clean up
}

Why is my boolean not being changed?

So I'm trying to create a client/server program. I want to know when my client disconnects of his own accord, so I've setup a heartbeat system. Every 6 seconds my client sends a ping to my server, if the client doesn't send a ping for a total of 30 seconds the client is considered disconnected and removed from the current connections list (for which I plan to implement a GUI). Or at least, that's the plan.
ConnectionManager.java
public class ConnectionManager implements Runnable{
static Socket connection;
private ArrayList<Thread> allConnections;
private ArrayList<Connection> allConnectionList;
private ServerSocket server;
private int id = 0;
public ConnectionManager() {
allConnections = new ArrayList<Thread>();
allConnectionList = new ArrayList<Connection>();
}
#Override
public void run() {
try {
server = new ServerSocket(5555);
System.out.println("Server is running!");
while(true) {
connection = server.accept();
Connection a = new Connection(connection, id);
Runnable runnable = a;
allConnectionList.add(a);
allConnections.add(new Thread(runnable));
allConnections.get(allConnections.size() - 1).start();
id++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void removeConnection(int id) {
allConnections.remove(id);
allConnectionList.remove(id);
}
Connection.java
public class Connection implements Runnable {
private Socket a;
public boolean amIActive;
private int id;
public Connection(Socket a, int id) {
amIActive = true;
this.a = a;
this.id = id;
}
public void onConnect() {
try {
String TimeStamp = new java.util.Date().toString();
String formattedAddress = a.getInetAddress().toString().replace("/", "");
System.out.println("Received connection from: " + formattedAddress + " at " + TimeStamp);
Runnable runnable = new ConnectionListener(this);
Thread connectionThread = new Thread(runnable);
connectionThread.start();
String returnCode = "Server repsonded to " + a.getInetAddress().toString().replace("/", "") + " at "+ TimeStamp + (char) 13;
BufferedOutputStream os = new BufferedOutputStream(a.getOutputStream());
OutputStreamWriter osw = new OutputStreamWriter(os, "US-ASCII");
osw.write(returnCode);
osw.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public void run() {
onConnect();
System.out.println("We got this far!");
while(amIActive) {
whileTrue();
}
System.out.println("This code never gets run because we get stuck in the while loop above");
Main.b.removeConnection(id);
System.out.println("Connection was closed from " + a.getInetAddress());
}
public void setOffline(boolean state) {
this.amIActive = state;
}
public void whileTrue() {
}
public Socket getSocket() {
return a;
}
ConnectionListener.java
public class ConnectionListener implements Runnable{
public Connection myConnection;
public boolean receivedHeartbeat;
public int missedHeartbeats = 0;
public ConnectionListener(Connection a) {
this.myConnection = a;
}
#Override
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
while(myConnection.amIActive) {
try {
BufferedInputStream is;
is = new BufferedInputStream(myConnection.getSocket().getInputStream());
InputStreamReader isr = new InputStreamReader(is);
StringBuffer process = new StringBuffer();
int character;
while((character = isr.read()) != 13) { //GETTING STUCK HERE BECAUSE STUPID.
if(character == -1) {
myConnection.setOffline(true);
} else {
process.append((char)character);
}
}
handleInput(process);
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void handleInput(StringBuffer process) {
String messageSent = process.toString();
if(messageSent.equals("Ping!")) {
receivedHeartbeat = true;
}
}
Heartbeat.java
public class Heartbeat implements Runnable{
private ConnectionListener b;
public Heartbeat(ConnectionListener a) {
b = a;
}
#Override
public void run() {
while(true) {
try {
Thread.sleep(1000);
if(b.missedHeartbeats > 5) {
b.myConnection.amIActive = false;
System.out.println("Setting amIActiveToFalse!");
}
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
} else {
b.missedHeartbeats++;
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
My console is spammed with System.out.println("Setting amIActiveToFalse!"); from Heartbeat.java. But the while loop in Connection.java keeps running. I believe this might be something to do with my threading, but I can't figure it out.

When you have a non-volatile variable, there is no guarentee of visability of a change in one thread to another. In particular, if the JVM detects that a thread doesn't alter a boolean it can inline it, meaning you will never see the value change.
The simple solution is to make the boolean volatile and it will not be inlined and one thread will see when another changes it.
For more details http://vanillajava.blogspot.com/2012/01/demonstrating-when-volatile-is-required.html

The trivial answer to this is: make the variable volatile.
Without this, it is allowed for the thread changing the value to basically keep its updates in cache, committing them to main memory some time later.
This allows threaded code to run much faster, since it can keep its variables in cache rather than having to fetch from main memory. However, the consequence of this is that other threads don't see the update.
Making the variable volatile prevents this from happening: a thread always reads the value from main memory, and writes are immediately committed.
I say that this is the trivial answer because it doesn't necessarily fix all of your problems. There may also be an atomicity issue: in between one thread reading the variable and writing it again, another thread might sneak in and change its value, which may or may not put the first thread into an undefined state from the perspective of its invariants.
Specifically:
if(b.receivedHeartbeat) { b.receivedHeartbeat = false;
It is possible that some other thread can change b.receivedHeartbeat to false after this thread evaluates it to true, so this iteration is erroneously counted as a "non-missed" heartbeat.
This can be fixed by making the variable a (non-volatile) AtomicBoolean, on which there is an atomic compare-and-set method, which avoids such race conditions.
Java Concurrency In Practice is a great reference on these issues, I wholeheartedly recommend it. Look for the topics "visibility" and "atomicity".
Also read the advanced chapter on the Java Memory Model. That made me doubt myself at first, but made me a much stronger programmer after I digested it.

There are a couple issues I saw while debugging the code you posted, but I was able to successfully get the heartbeat functionality working.
In the Connection Listener class I don't think the if statement with .equals("Ping!") will match, because of the newline character at the end of each line.
In the Connection Listener class I would probably put the socket's Input Stream at the top of the loop not inside the loop. (I don't think this will break it but it's probably nicer this way)
ConnectionListener Updates:
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
BufferedReader br = null;
try {
//is = new BufferedInputStream(myConnection.getSocket().getInputStream());
br = new BufferedReader(new InputStreamReader(myConnection.getSocket().getInputStream()));
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
while(myConnection.amIActive) {
try {
String processLine = br.readLine();
System.out.println("handleInput:" + processLine);
handleInput(processLine);
} catch (Exception e) {
System.out.println("Exception!");
e.printStackTrace();
}
}
}
public void handleInput(String messageSent) {
if(messageSent.startsWith("Ping!")) { //Need to use startsWith, or add newline character
receivedHeartbeat = true;
System.out.println("receivedHeartbeat!");
}
}
Also, in your Heartbeat class make sure you reset the missedHeartbeats counter to 0 on true:
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
b.missedHeartbeats = 0;
} else {
b.missedHeartbeats++;
}

Does ReentrantReadWriteLock read while writeLock locked?

I research ReentrantReadWriteLock.
I write simple code for test(I know that use Thread.sleep() can not guarantee predictable result but I think that I am lucky:)):
public class RWLock {
private static String val = "old";
private static ReadWriteLock lock = new ReentrantReadWriteLock();
private static long time = System.currentTimeMillis();
public void read() {
try {
lock.readLock().lock();
System.out.println("read " + val +" - "+(System.currentTimeMillis()-time));
Thread.sleep(300);
} catch (InterruptedException e) {
} finally {
lock.readLock().unlock();
}
}
public void write() {
try {
lock.writeLock().lock();
val = "new";
System.out.println("write " + val+" - "+(System.currentTimeMillis()-time));
Thread.sleep(10000);
} catch (InterruptedException e) {
} finally {
lock.writeLock().unlock();
}
}
}
class Tester {
public static void main(String[] args) throws InterruptedException {
new MyThreadRead().start();
Thread.sleep(400);
new MyThreadWrite().start();
}
}
class MyThreadRead extends Thread {
#Override
public void run() {
for (int i = 0; i < 10; i++) {
new RWLock().read();
try {
Thread.sleep(200);
} catch (InterruptedException e) {
}
}
}
}
class MyThreadWrite extends Thread {
#Override
public void run() {
new RWLock().write();
}
}
output:
read old - 0
write new - 401
read new - 10401
read new - 10902
read new - 11402
read new - 11902
read new - 12402
read new - 12902
read new - 13402
read new - 13902
read new - 14402
10401 - 401 == 10000
10000 it is time of writing.
As I understood second read thread cannot finish before writing. Thus writing and second reading performs in parallel. It is not predictable behaviour for me.
What do you think about it?

Maybe you have more sleep() calls than you realize. The sleep calls in MyThreadRead#run() and in RWLock()#read() add up to 500 ms. So here's what's happening.
At T=0, the reader thread grabs the read lock and sleeps for 300ms.
At T=300, the reader releases the lock, and then sleeps for another 200ms.
At T=400, the writer grabs the write lock and sleeps for ten seconds,
At T=500, the reader tries to grab the read lock, but it is blocked by the writer.
At T=10400, the writer gives up the lock, and then the reader gets to go round
its loop nine more times.
P.S.: Move your lock() calls out of the try/finally statements. E.g.,
...lock()
try {
...
} finally {
...unlock();
}
That way, if the lock() call throws an exception, the unlock() will not be called.

Text is not getting printed once the Threads are done [duplicate]

This question already has answers here:
How to wait for all threads to finish, using ExecutorService?
(27 answers)
Closed 8 years ago.
Please have a look at the following code.
public class BigFileWholeProcessor {
private static final int NUMBER_OF_THREADS = 2;
public void processFile(String fileName) {
BlockingQueue<String> fileContent = new LinkedBlockingQueue<String>();
BigFileReader bigFileReader = new BigFileReader(fileName, fileContent);
BigFileProcessor bigFileProcessor = new BigFileProcessor(fileContent);
ExecutorService es = Executors.newFixedThreadPool(NUMBER_OF_THREADS);
es.execute(bigFileReader);
es.execute(bigFileProcessor);
es.shutdown();
if(es.isTerminated())
{
System.out.println("Completed Work");
}
}
}
public class BigFileReader implements Runnable {
private final String fileName;
int a = 0;
public static final String SENTINEL = "SENTINEL";
private final BlockingQueue<String> linesRead;
public BigFileReader(String fileName, BlockingQueue<String> linesRead) {
this.fileName = fileName;
this.linesRead = linesRead;
}
#Override
public void run() {
try {
//since it is a sample, I avoid the manage of how many lines you have read
//and that stuff, but it should not be complicated to accomplish
BufferedReader br = new BufferedReader(new FileReader(new File("E:/Amazon HashFile/Hash.txt")));
String str = "";
while((str=br.readLine())!=null)
{
linesRead.put(str);
System.out.println(a);
a++;
}
linesRead.put(SENTINEL);
} catch (Exception ex) {
ex.printStackTrace();
}
System.out.println("Completed");
}
}
public class BigFileProcessor implements Runnable {
private final BlockingQueue<String> linesToProcess;
public BigFileProcessor (BlockingQueue<String> linesToProcess) {
this.linesToProcess = linesToProcess;
}
#Override
public void run() {
String line = "";
try {
while ( (line = linesToProcess.take()) != null) {
//do what you want/need to process this line...
if(line==BigFileReader.SENTINEL)
{
break;
}
String [] pieces = line.split("(...)/g");
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
I want to print the text "completed work" in BigFileWholeProcessor once all the thread work is done. But instead, it is not getting printed. Why is this? How to identify that all the threads are done and need printing?

shutdown() only signal ES to shutdown, you need
awaitTermination(long timeout, TimeUnit unit)
before print message

Use submit() method instead of execute(). The get() method can be used if you want to wait for the thread to finish at any point of time. Read documentation on use of Future object for further details.
ExecutorService es = Executors.newFixedThreadPool(2);
Future<?> f = es.submit(new Thread(new TestRun()));
f.get(); // Wait for result... (i.e similar to `join()` in this case)
es.shutdown(); // Shutdown ExecutorService
System.out.println("Done.");
I have defined a TestRun class implementing Runnable, not shown here. The Future object makes more sense in other scenarios.

Cross process synchronization in Java

How can I synchornize two Java processes running on Windows ?
I am looking for something like the Win32 Named Mutex object which allows two processes to use the same locking object.
Thanks

Java cross process lock:
// Tester
try {
if (crossProcessLockAcquire(SomeClassInYourApp.class, 3000)) {
// Success - This process now has the lock. (Don't keep it too long.)
}
else {
// Fail (Timeout) - Another process still had the lock after 3 seconds.
}
} finally {
crossProcessLockRelease(); // try/finally is very important.
}
// Acquire - Returns success ( true/false )
private static boolean crossProcessLockAcquire(final Class<?> c, final long waitMS) {
if (fileLock == null && c != null && waitMS > 0) {
try {
long dropDeadTime = System.currentTimeMillis() + waitMS;
File file = new File(lockTempDir, c.getName() + ".lock");
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
FileChannel fileChannel = randomAccessFile.getChannel();
while (System.currentTimeMillis() < dropDeadTime) {
fileLock = fileChannel.tryLock();
if (fileLock != null) {
break;
}
Thread.sleep(250); // 4 attempts/sec
}
} catch (Exception e) {
e.printStackTrace();
}
}
return fileLock == null ? false : true;
}
// Release
private static void crossProcessLockRelease() {
if (fileLock != null) {
try {
fileLock.release();
fileLock = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Some class vars and a failsafe lock release.
private static File lockTempDir = new File(System.getProperty("java.io.tmpdir") + File.separator + "locks");
private static FileLock fileLock = null;
static {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run(){
crossProcessLockRelease();
}
});
}

I simplified Java42 answer
Usage
ProcessLock lock = new ProcessLock("lockKey");
lock.run(successLockRunnable, timeOutLockRunnable);
The code in successLockRunnable will lock any other process on the same machine using this implementation.
Source
/**
* Created by Ilya Gazman on 13/06/2016.
* Based on https://stackoverflow.com/a/9577667/1129332
*/
public class ProcessLock {
// Some class vars and a fail safe lock release.
private File lockTempDir = new File(System.getProperty("java.io.tmpdir") + File.separator + "locks");
private FileLock fileLock = null;
private String key;
public ProcessLock() {
this("lock");
}
public ProcessLock(String key) {
this.key = key;
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
crossProcessLockRelease();
}
});
}
public void run(Runnable successCallback) {
run(successCallback, null);
}
public void run(Runnable successCallback, Runnable timeOutCallback) {
try {
if (crossProcessLockAcquire(3000)) {
successCallback.run();
} else if (timeOutCallback != null) {
timeOutCallback.run();
}
} finally {
crossProcessLockRelease(); // try/finally is very important.
}
}
// Acquire - Returns success ( true/false )
private boolean crossProcessLockAcquire(final long waitMS) {
if (fileLock == null && waitMS > 0) {
try {
long dropDeadTime = System.currentTimeMillis() + waitMS;
File file = new File(lockTempDir, "_" + key + ".lock");
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
FileChannel fileChannel = randomAccessFile.getChannel();
while (System.currentTimeMillis() < dropDeadTime) {
fileLock = fileChannel.tryLock();
if (fileLock != null) {
break;
}
Thread.sleep(250); // 4 attempts/sec
}
} catch (Exception e) {
e.printStackTrace();
}
}
return fileLock != null;
}
// Release
private void crossProcessLockRelease() {
if (fileLock != null) {
try {
fileLock.release();
fileLock = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

It is not possible to do something like you want in Java. Different Java applications will use different JVM's fully separating themselves into different 'blackbox'es. However, you have 2 options:
Use sockets (or channels). Basically one application will open the listening socket and start waiting until it receives some signal. The other application will connect there, and send signals when it had completed something. I'd say this is a preferred way used in 99.9% of applications.
You can call winapi from Java. I do not remember the specifics, but you can find a plethora of example if you google "java winapi".

We use these kinds of statements to make sure only one process can do a block of code keyed by "myLockKey":
new LocalFileLock("myLockKey").doWithLock(() -> {
doSomething();
return null; // must return something
});
Here, we make use of this class:
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
import java.util.function.Supplier;
import com.headlandstech.utils.FileUtils;
import com.headlandstech.utils.Log;
public class LocalFileLock {
private final File lockFile;
public LocalFileLock(String name) {
this.lockFile = new File(FileUtils.TEMP_DIR, name + ".lock");
if (!lockFile.isFile()) {
FileUtils.writeStringToFile("", lockFile);
}
}
public <T> T doWithLock(Supplier<T> f) {
Log.log.info("Waiting on lock " + lockFile);
try (FileChannel channel = new RandomAccessFile(lockFile, "rw").getChannel()) {
final FileLock fileLock = channel.lock();
Log.log.info("Lock " + lockFile + " obtained");
T result = f.get();
fileLock.release();
Log.log.info("Lock " + lockFile + " released");
return result;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}

I don't think there are native methods in the java platform for this. However, there are several ways to go about obtaining the same type of effect depending on what synchronization you are trying to accomplish. In addition to having the processes communicate over network connections (direct sockets, multicast with an election, etc.) or dropping to platform specific calls, you can also explore obtaining a file lock to a shared file (see activemq passive stand-by with a shared file system for an example) or using a database either with something like a select for update or an optimistic update of a table row.

Not sure what you are trying to do, I'd possibly do this by exposing something via JMX and having the separate processes set a status flag which then programmatically revives your thread from a wait state. Instead of JMX you could of course use a socket/RMI.

using sockets for cross processes synchronizations is common practice . not only for java applications because in most *nix environments we have not system-wide mutexes as we have in Windows.