I use in project Striped locks from GUAVA for synchronizing input data for every unique identifier of process, it provides right order of data for next processing this data.
Striped<Lock> locks = Striped.lazyWeakLock(10);
public void put(String id, String data) {
Lock lock = locks.get(id);
lock.lock()
try {
// process data
} finally {
lock.unlock()
}
}
This code works good, provides right order of data.
But I want to get global lock (exclusive access) to current object for modification some data. So i want to stop getting locks for all unique id until I modify data.
Possible solutions:
I can use ReasWriteLock before getting striped lock. (read - for "locks.get(id)", write - for exclusive access) - Not good solution, extra locking in normal state.
I can use one lock for all, but it will slow down application
Global lock
Example:
if (locked) {
global.lock()
global.unlock()
Lock lock = locks.get(id);
...
} else {
Lock lock = locks.get(id);
if (locked){
...
}
lock.lock();
}
void extraLock(){
global.lock();
locked = true;
// modify data
locked = false;
global.unlock()
}
I think solution with global lock looks not good(flags, lock, double check and etc)
Question:
Can you advice any solution that can help me to solve this problem?
Or Can you tell some idea?
Thanks
Related
I have an application where I want to ensure that a method is called at most once concurrently, say when updating user balance in a database.
I am thinking of using the following locking mechanism: (showing Scala code below, but should be similar with Java Lambdas):
object Foo{
val dbLocked = new java.util.concurrent.atomic.AtomicBoolean(false)
def usingAtoimcDB[T](f: => T):T = {
if (dbLocked.get) throw new Exception("db is locked")
dbLocked.set(true)
try f
finally dbLocked.set(false)
}
}
Is this safe to use when usingAtoimcDB may be called concurrently?
EDIT: The corrected code below, as pointed in this answer:
def usingAtoimcDB[T](f: => T):T = {
if(dbLocked.compareAndSet(false, true)) {
//db is now locked
try f
finally dbLocked.set(false)
} else {
//db is already locked
throw new Exception("db is locked")
}
}
EDIT 2:
Using a spinloop. Is this also ok?
def usingAtoimcDB[T](f: => T):T = {
while (!dbLocked.compareAndSet(false, true)) {Thread.sleep(1)}
try f
finally dbLocked.set(false)
}
EDIT3: Based on the answers and comments below, I am also considering using queues.
Inadvisable. You are requesting that the same pieco of code running in the same application instance on tha same server is the single point to do that transaction. There also is no provision to let this code stand-out. When you are retired, someone may start a second application instance or whatever.
Whereas a database commit/rollback is a quite simple and sure mechanism.
When you cannot write an integration (unit) test to ensure this sole point, then do not do it.
If you do it:
Revoke rights to the table modifications for the normal database user
Add a new database use who has sufficient right granted
And still: do not do it.
The code you posted above is not thread-safe, because you are not using an atomic check-and-set operation. Two threads can both be executing the if (dbLocked.get) statement at the same time and both get false as the answer, and then both will do dbLocked.set(true) and call f.
If you really want to use AtomicBoolean, then you must use compareAndSet as #leshkin already showed - this is an atomic operation that does the check and set in one go without the possibility of another thread doing the same thing at the same time, so that it is thread-safe.
You are using an AtomicBoolean as a lock here. There are classes in the standard Java library which are better suited (and specifically made) for this purpose; have a look at the package java.util.concurrent.locks.
You could for example use class ReentrantReadWriteLock, which combines two locks for reading and writing. The write lock is exclusive (when it's locked, nobody else can read or write); the read lock is shared (when it's locked, nobody can write, but others can read at the same time). This allows for there to be multiple readers concurrently, but only one writer at a time, possibly improving efficiency (it's not necessary to make reading an exclusive operation).
Example:
import java.util.concurrent.locks._
object Foo {
private val lock: ReadWriteLock = new ReentrantReadWriteLock
def doWriteOperation[T](f: => T): T = {
// Locks the write lock
lock.writeLock.lock()
try {
f
} finally {
lock.writeLock.unlock()
}
}
def doReadOperation[T](f: => T): T = {
// Locks the read lock
lock.readLock.lock()
try {
f
} finally {
lock.readLock.unlock()
}
}
}
Yes, it should work as espected. I would slightly modify your function using compareAndSet call.
compareAndSet method has the advantage to be an atomic operation - there are no race conditions and the value will be changed atomically.
def usingAtoimcDB[T](f: => T):T = {
if(dbLocked.compareAndSet(false, true)) {
//db is now locked
try f
finally dbLocked.set(false)
} else {
//db is already locked
throw new Exception("db is locked")
}
}
I'm trying to write to a file, using a Java FileLock, to prohibit all other processes and threads from reading from it or writing to it until I'm finished with it. Given this question and the answers to it, it seems to me like this is the perfect tool for what I want--a mutex for file access.
However, I am very concerned about this text from the JavaDocs:
File locks are held on behalf of the entire Java virtual machine. They
are not suitable for controlling access to a file by multiple threads
within the same virtual machine.
Can someone either alleviate my fears or point me in the right direction? It sounds like FileLock won't work at all to keep a different thread out of the file, even if another thread has already obtained it. If this is the case, is there another canonical Java method to do this that will protect from other threads?
The FileLock is a process level lock and will thus not protect the file from concurrent access from multiple threads within the process that has the lock.
You need to use a combination of the FileLock to protect from concurrent access from other processes and some other synchronization mechanism (like a synchronized method for accessing the file) within your process to protect from concurrent access by your own threads.
I would implement this as follows:
interface FileOperator {
public void operate(File file);
}
class FileProxy {
private static final ConcurrentHashMap<URI, FileProxy> map =
new ConcurrentHashMap<>();
private final Semaphore mutex = new Semaphore(1, true);
private final File file;
private final URI key;
private FileProxy(File file) {
this.file = file;
this.key = file.toURI();
}
public static void operate(URI uri, FileOperator operator) {
FileProxy curProxy = map.get(uri);
if(curProxy == null) {
FileProxy newProxy = new FileProxy(new File(uri));
FileProxy curProxy = map.putIfAbsent(newProxy.key, newProxy);
if(curProxy == null) {
curProxy = newProxy; // FileProxy was not in the map
}
}
try {
curProxy.mutex.acquire();
operator.operate(curProxy.file);
} finally {
curProxy.mutex.release();
}
}
}
The threads that are using a file implement FileOperator or something similar. Files are hidden behind a FileProxy that maintains a static ConcurrentHashMap of key (URI, or absolute path, or some other file invariant) value (FileProxy) pairs. Each FileProxy maintains a Semaphore that acts as a mutex - this is initialized with one permit. When the static operate method is called, a new FileProxy is created from the URI if none exists; the FileOperator is then added to the FileProxy queue; acquire is called on the mutex to ensure that only one thread can operate on the file at a time; and finally the FileOperator does its thing.
In this implementation, FileProxy objects are never removed from the ConcurrentHashMap - if this is a problem then a solution is to wrap the FileProxy objects in a WeakReference or SoftReference so that they can be garbage collected, and then call map.replace if reference.get() == null to ensure that only one thread replaces the GC'd reference.
I have made a Java program that connects to a SQLite database using SQLite4Java.
I read from the serial port and write values to the database. This worked fine in the beginning, but now my program has grown and I have several threads. I have tried to handle that with a SQLiteQueue-variable that execute database operations with something like this:
public void insertTempValue(final SQLiteStatement stmt, final long logTime, final double tempValue)
{
if(checkQueue("insertTempValue(SQLiteStatement, long, double)", "Queue is not running!", false))
{
queue.execute(new SQLiteJob<Object>()
{
protected Object job(SQLiteConnection connection) throws SQLiteException
{
stmt.bind(1, logTime);
stmt.bind(2, tempValue);
stmt.step();
stmt.reset(true);
return null;
}
});
}
} // end insertTempValue(SQLiteStatement, long, double)
But now my SQLite-class can't execute the statements reporting :
DB[1][U]: disposing [INSERT INTO Temperatures VALUES (?,?)]DB[1][U] from alien thread
SQLiteDB$6#8afbefd: job exception com.almworks.sqlite4java.SQLiteException: [-92] statement is disposed
So the execution does not happen.
I have tried to figure out what's wrong and I think I need a Java wrapper that makes all the database operations calls from a single thread that the other threads go through.
Here is my problem I don't know how to implement this in a good way.
How can I make a method-call and ensure that it always runs from the same thread?
Put all your database access code into a package and make all the classes package private. Write one Runnable or Thread subclass with a run() method that runs a loop. The loop checks for queued information requests, and runs the appropriate database access code to find the information, putting the information into the request and marking the request complete before going back to the queue.
Client code queues data requests and waits for answers, perhaps by blocking until the request is marked complete.
Data requests would look something like this:
public class InsertTempValueRequest {
// This method is called from client threads before queueing
// Client thread queues this object after construction
public InsertTempValueRequest(
final long logTime,
final double tempValue
) {
this.logTime = logTime
this.tempValue = tempValue
}
// This method is called from client threads after queueing to check for completion
public isComplete() {
return isComplete;
}
// This method is called from the database thread after dequeuing this object
execute(
SQLiteConnection connection,
SQLiteStatement statement
) {
// execute the statement using logTime and tempValue member data, and commit
isComplete = true;
}
private volatile long logTime;
private volatile double tempValue;
private volatile boolean isComplete = false;
}
This will work, but I suspect there will be a lot of hassle in the implementation. I think you could also get by by using a lock that only permits one thread at a time to access the database, and also - this is the difference from your existing situation - beginning the access by creating the database resources - including statements - from scratch, and disposing of those resources before releasing the lock.
I found a solution to my problem. I have now implemented a wrapper-class that makes all operations with my older SQLite-class using an ExecutorService, inspired from Thread Executor Example and got the correct usage from Java Doc ExecutorService.
I have a write method that is supposed to safely write data to a file.
// The current file I am writing to.
FileOutputStream file = null;
...
// Synchronized version.
private void write(byte[] bytes) {
if (file != null && file.getChannel() != null) {
try {
boolean written = false;
do {
try {
// Lock it!
FileLock lock = file.getChannel().lock();
try {
// Write the bytes.
file.write(bytes);
written = true;
} finally {
// Release the lock.
lock.release();
}
} catch (OverlappingFileLockException ofle) {
try {
// Wait a bit
Thread.sleep(0);
} catch (InterruptedException ex) {
throw new InterruptedIOException("Interrupted waiting for a file lock.");
}
}
} while (!written);
} catch (IOException ex) {
log.warn("Failed to lock " + fileName, ex);
}
} else {
log.warn("Failing - " + (file == null ? "file" : "channel") + " is null!!");
}
}
It has worked fine for me for a while now, although I know there are some wrinkles in it.
I have recently changed a project that uses this code to build and run under Java 5 (from Java 6) and now it looks like it is deadlocked awaiting a lock on the file. It is a multithreaded app and it is quite possible for several threads to attempt to write to the same file.
The debugger tells me that the hung threads are waiting for the FileLock lock = file.getChannel().lock() call to return.
Some research brought up this interesting little nugget which mentions:
File locks are held on behalf of the entire Java virtual machine. They are not suitable for controlling access to a file by multiple threads within the same virtual machine.
So am I doing it wrong? If so what is the right way? If I am doing it right how come I hit a deadlock?
Added: Forgot to mention - each thread holds its own copy of this object so there should not be any synchronisation issues within the code. I felt safe to rely on the FileChannel.lock() method to ensure writes do not interleave.
Added too: I have indeed solved the issue using various synchronized mechanisms. I do, however, have outstanding questions:
Why is FileLock lock = file.getChannel().lock(); not suitable ...?
Why did my issues only appear when switching back to Java-5 when everything worked fine with Java-6?
FileLock is only for interprocess locking, javadoc reads:
"File locks are held on behalf of the entire Java virtual machine.
They are not suitable for controlling access to a file by multiple
threads within the same virtual machine."
To lock between java threads (same JVM) you need to use some shared lock. I would suggest within the file writing class to use a synchronized block (which according to these articles is likely to perform best):
final Object lock = new Object();
public void write(...){
synchronized(lock){
// do writing
}
}
Another approach is to use a ReentrantLock and then use the proven idiom of
final ReentrantLock lock = new ReentrantLock();
public void write(...){
try {
lock.lock()
// do the writing
} finally {
// forget this and you're screwed
lock.unlock();
}
}
you may need to implement critical section concept on the actual code using the file, rather than the hashmap. You can either create a synchronized block or separate the file access code into a separate procedure and make that method synchronized.
Essentially, only one thread executes a synchronized block at a time. It gives you the exclusive access you need.
Another way of doing it is, to use a serial thread Executor, depending on your functional requirements.
You may want to look at this thread:
Howto synchronize file access in a shared folder using Java (OR: ReadWriteLock on network level)
A multi-threaded piece of code accesses a resource (eg: a filesystem) asynchronously.
To achieve this, I'll use condition variables. Suppose the FileSystem is an interface like:
class FileSystem {
// sends a read request to the fileSystem
read(String fileName) {
// ...
// upon completion, execute a callback
callback(returnCode, buffer);
}
}
I have now an application accessing the FileSystem. Suppose I can issue multiple reads through a readFile() method.
The operation should write data to the byte buffer passed to it.
// constructor
public Test() {
FileSystem disk = ...
boolean readReady = ...
Lock lock = ...
Condition responseReady = lock.newCondition();
}
// the read file method in quesiton
public void readFile(String file) {
try {
lock.lock(); // lets imagine this operation needs a lock
// this operation may take a while to complete;
// but the method should return immediately
disk.read(file);
while (!readReady) { // <<< THIS
responseReady.awaitUninterruptibly();
}
}
finally {
lock.unlock();
}
}
public void callback(int returnCode, byte[] buffer) {
// other code snipped...
readReady = true; // <<< AND THIS
responseReady.signal();
}
Is this the correct way to use condition variables? Will readFile() return immediately?
(I know there is some sillyness in using locks for reads, but writing to a file is also an option.)
There's a lot missing from your question (i.e. no specific mention of Threads) but I will try to answer anyway.
Neither the lock nor the conditional variables give you background capabilities -- they just are used for a thread to wait for signals from other threads. Although you don't mention it, the disk.read(file) method could spawn a thread to do the IO and then return immediately but the caller is going to sit in the readReady loop anyway which seems pointless. If the caller has to wait then it could perform the IO itself.
A better pattern could be to use something like the Java 5 Executors service:
ExecutorService pool = Executors.newFixedThreadPool(int numThreads);
You can then call pool.submit(Callable) which will submit the job to be performed in the background in another thread (when the pool next has one available). Submit returns a Future which the caller can use to investigate if the background task has finished. It can return a result object as well. The concurrent classes take care of the locking and conditional signal/wait logic for you.
Hope this helps.
p.s. Also, you should make readReady be volatile since it is not synchronized.