I created a distributed lock class that I designed to be used like this:
myLock.lock();
doSomething();
myLock.unlock();
In my current implementation, lock() blocks until the lock is acquired. But I am running into some deadlock issues with this implementation. So I'd like to rewrite it to be asynchronous, but I have no idea how to do that in java.
Something like this would work I think:
myLock.lock(myCallbackFunction);
private void myCallbackFunction(boolean result){
if(result){
doSomething();
mylock.Unlock();
}
}
Is there a way to do this in java?
EDIT (More detail): The reasons why the synchronous implementation is deadlocking are complicated and not relevant. The distributed lock is acquiring mutual exclusion of a resource across a network with multiple systems. Really the only thing I'm looking for is how to write a method that accepts a callback function.
You can't do that in Java yet. What you can do is define a LockCallback interface:
interface LockCallback {
void run(boolean result, MyLock lock);
}
and have MyLock#lock take a LockCallback as a parameter. Then callers can call it as
myLock.lock(new LockCallback {
public void run(boolean result, MyLock lock) {
// ... do whatever needs to be done ...
lock.unlock();
});
Lambda syntax in Java 8 should make this a little less ugly looking.
Instead of writing your own and then abandoning the idea because you couldn't make it work, why not use a Semaphore, which is already there and is implemented correctly?
Related
I have been looking into scenarios where using Lock, ReentrantLock fits perfectly fine, but that scenario would be unachievable by using usual synchronisation. The official javadocs says:
There are occasions where you need to work with locks in a more
flexible way. For example, some algorithms for traversing concurrently
accessed data structures require the use of "hand-over-hand" or "chain
locking": you acquire the lock of node A, then node B, then release A
and acquire C, then release B and acquire D and so on. Implementations
of the Lock interface enable the use of such techniques by allowing a
lock to be acquired and released in different scopes, and allowing
multiple locks to be acquired and released in any order.
I failed to find any such example code snippet where only ReentrantLock solved the problem, but not the usual synchronization. I am aware of all the other advantages that Lock interfaces provide such as fairness, etc., but I am not looking for the theories.
Recently I had the case that a simple synchronize was no longer sufficient. It should be ensured that a long running job like a data import should not run in parallel. But if, for some reason, a second job would be started this second job should be blocked until the first job ends. Really simple. But because such a case would be an error situation in our system, there should be a log message in case of blocking. For this use case we need more than a simple synchronize.
We used the ReentrantLock's tryLock() method to implement this.
Here a simplified version of our implementation:
The job has to implement the interface ProtectedExecutable with the critical section in method execute() and blocking handler in onBlocking().
interface ProtectedExecutable {
/**
* The code to be executed as a critical section
* #return Result value
*/
int execute();
/**
* Is called only if the execution was blocked.
* #param reason Text with the blocking reason, which can be logged. Contains the description of the blocking ProtectedExecutable if
* possible
*/
void onBlocking(String reason);
}
In ProtectedExecutor tryLock() is first used to enter the critical section or to return false if it is blocked. So there is a chance to do some action before calling the blocking version lock() and wait.
class ProtectedExecutor {
private final Lock lock = new ReentrantLock();
public int execute(ProtectedExecutable executable) {
if(lock.tryLock()) {
try{
return executable.execute();
} finally {
lock.unlock();
}
} else {
executable.onBlocking("Execution blocked by another ProtectedExecutor!");
lock.lock();
try {
return executable.execute();
} finally {
lock.unlock();
}
}
}
}
Can ReentrantLock be naive/directly replaced by synchronized in all cases? -No. Just because the locking with ReentrantLock can cross method/block scope, and locking with synchronized cannot, it sets its own scope (at least you have to re-invent the lock with wait/notify(All) to have the same feature). What does it mean? For two locks a and b the ReentrantLock allows you to have any order of locking and any order of unlocking. For example, lock {a, b}, unlock {a, b} or unlock {b, a}. The synchronized block allows the order of unlocking to be only reverse order of locking (synchronized (a) { synchronized (b) {}} - lock {a, b}, unlock {b, a}, never unlock {a, b})
But, all practical problems can be resolved without ReentrantLock. And here I'd like to share some thoughts...
Most samples/comments will show how to use ReentrantLock to solve a problem, but in most cases there is another solution, and it may be even simpler, more clear, more natural. For example, "chain locking" tries to solve the problem of specific sequence of execution. This particular problem can be resolved with specific ExecutorService (for example with a single threaded executor for sequential execution) and a set of Tasks we put in specific order to be executed. More complex cases can be addressed with something like java.util.concurrent.Phaser etc. Or, there is another trick: to schedule next task(s) to be executed in the end of the current one (How to execute threads with ExecutorService in concrete order?).
Also, please note, that "chain locking" is deadlock-prone if different chains may have different order. Compare, for example "lock1.lock, do something, lock2.lock, lock1.unlock, do something, lock2.unlock" and "lock2.lock, do something, lock1.lock, lock2.unlock, do something, lock1.unlock"
Another example when a task "should not run in parallel" (ProtectedExecutable below) also perfectly can be resolved (by design) with a single threaded executor, and so on.
There is a known small difference in terms of performance between synchronized and ReentrantLock in cases of high contention, but not sure that's really important somewhere out of synthetic tests on specific JDK (Java 15 have disabled biased locking (JEP 374) in the Java virtual machine and this can affect test results).
I'd say that the only important thing, I've found in my personal practice, is design-related, since with a lock we can cross method scope. This is not about a problem which can be resolved with ReentrantLock only, but about a case when it may be convenient to use it. When you need a thread safe structure/class with a more or less complex behavior, there are 2 main patterns:
// each thread-safe business method is atomic
// and synchronized
class A {
// a lot of fields here
private void doX() {
// a portion of manipulations with the fields
}
private void doY() {
// another portion of manipulations with the fields
}
public synchronized void threadSafeOperation() {
doX();
doY();
}
}
A a = new A();
a.threadSafeOperation();
// each thread-safe business method is atomic
// and synchronized
class A {
private final ReenterantLock lock = new ReenterantLock();
// a lot of fields here
public void lock() {
lock.lock();
}
public void unlock() {
lock.unlock();
}
public void doX() {
// a portion of manipulations with the fields
}
public void doY() {
// another portion of manipulations with the fields
}
}
A a = new A();
a.lock();
try {
a.doX();
a.doY();
} finally {
a.unlock();
}
The first pattern requires each thread-safe operation/usage to be defined as one single synchronized method. The second pattern allows you to use different combinations in different use cases. Each one may be reasonable/preferable in specific case.
In the second case we also may say "use synchronized instead of lock", but specifically and explicitly defined lock/unlock operations say more about our intentions and rules of usage than:
A a = new A();
synchronized(a) {
a.doX();
a.doY();
}
Java object monitor's based synchronization (synchronized + wait + notify(All)) helped to resolve all practical problems for years, but one day Doug Lea and Co had decided to develop a rich, object-oriented library of concurrent primitives. So, now we have a number of more or less useful locks, things like java.util.concurrent.locks.Condition (instead of wait + notify(All)), effective concurrent collections and some other stuff. Now we can choose more expressive, effective way/tool to solve our day-to-day problems in concurrency.
How to skip method if some thread owns by this method in java, i know that in .net exists Monitor.TryEnter, and i can accomplish that by something like this:
if(Monitor.TryEnter()){
//do work
}
//else skip
But how can i accomplish that behavior in java, is any equivalent of Monitor.TryEnter in java.
Would ReentrantLock#tryLock() work for you?
It has the behavior that you want, but it requires an explicit lock object and as such it does not work with the embedded Java object monitor - I do not believe that there is an equivalent to tryLock() for Java object monitors.
I think you're probably best off using java.util.concurrent.Semaphore.
If you use a Semaphore that holds one permit :
Semaphore semaphore = new Semaphore(1);
the resulting code looks virtually the same :
if (semaphore.tryAcquire()) {
try {
// do work
} finally {
semaphore.release();
}
}
// else skip
I imagine in .Net you also have to make sure to release the lock again.
I have been making use of Java's synchronized blocks to make parts of my code thread safe. I am porting a data structure to java that can usually use synchronized blocks, but I don't always know how to use them in a typical Java way.
Here is an example of one scenario:
myMethod (Bool useLock)
{
if (useLock)
{
//locks the following section of code until unlocked.
lockObject.lock();
}
//do more stuff....
if (useLock)
{
//unlocks exclusive control of code.
lockObject.unlock();
}
}
How do I do an equivalent of this in Java? In this code sometimes I want to lock and sometimes I don't, but I want to be smart about it and not have to write two versions of the same code. Are there other ways of locking in Java other than using synchronized blocks?
You can use Lock objects, ReentrantLock in particular should do the work. http://docs.oracle.com/javase/tutorial/essential/concurrency/newlocks.html
Or you can still solve the issue with synchronized blocks. The code from your question will look like:
myMethod (Bool useLock) {
if (useLock) {
synchronized (this) {
criticalSection();
}
} else {
criticalSection();
}
}
criticalSection() {
//do more stuff....
}
Or if you want to guarantee the mutual exclusion among different instances of the class you should use other monitor object than this. For instance TheClassName.class or other explicitly defined static variable of that class.
Some good answers already posted, but I wanted to add another solution that worked for me:
I started with:
...
synchronized(<lockobject>) {
<huge section of code with lots of variables>
}
...
Changed it to:
...
synchronized(<condition> ? <lockobject> : new Object()) {
<huge section of code with lots of variables>
}
...
Basically if the condition is true it executes the code using synchronized on lockobject, but if the condition is false it uses "new Object()" which basically lets it run without any synchronization.
I Guess ReentrantLock can be used in your scenario
final Lock lock = new ReentrantLock();
Sample code snippet
class X {
private final ReentrantLock lock = new ReentrantLock();
// ...
public void m() {
lock.lock(); // block until condition holds
try {
// ... method body
} finally {
lock.unlock()
}
}
}
You can use Java lock objects and Condition objects. Condition class' java docs gives a very good isFull and isEmpty example too.
In your case, i guess, you can utilise the condition object to make the code more readable/understandable. something like this:
final Lock lock = new ReentrantLock();
final Condition useLock = lock.newCondition();
and use the condition as appropriate.
You can use Unsafe to monitorEnter and monitorExit but this is a bad idea IMHO.
If the locks are not needed the JVM will almost optimise them away in any case. synchronized is very efficient if only one thread ever acquires it. (Unlike Lock for example)
Are there other ways of locking in Java other than using synchronized blocks?
Yes - java.util.concurrent.locks, and the ReentrantLock class in particular, but as Peter says if you can get away with just using synchronized it'll probably be more efficient overall (and easier to maintain, particularly if you're working as part of a team).
in trying to improve my understanding on concurrency issues, I am looking at the following scenario (Edit: I've changed the example from List to Runtime, which is closer to what I am trying):
public class Example {
private final Object lock = new Object();
private final Runtime runtime = Runtime.getRuntime();
public void add(Object o) {
synchronized (lock) { runtime.exec(program + " -add "+o); }
}
public Object[] getAll() {
synchronized (lock) { return runtime.exec(program + " -list "); }
}
public void remove(Object o) {
synchronized (lock) { runtime.exec(program + " -remove "+o); }
}
}
As it stands, each method is by thread safe when used standalone. Now, what I'm trying to figure out is how to handle where the calling class wishes to call:
for (Object o : example.getAll()) {
// problems if multiple threads perform this operation concurrently
example.remove(b);
}
But as noted, there is no guarantee that the state will be consistent between the call to getAll() and the calls to remove(). If multiple threads call this, I'll be in trouble. So my question is - How should I enable the developer to perform the operation in a thread safe manner? Ideally I wish to enforce the thread safety in a way that makes it difficult for the developer to avoid/miss, but at the same time not complicated to achieve. I can think of three options so far:
A: Make the lock 'this', so the synchronization object is accessible to calling code, which can then wrap the code blocks. Drawback: Hard to enforce at compile time:
synchronized (example) {
for (Object o : example.getAll()) {
example.remove(b);
}
}
B: Place the combined code into the Example class - and benefit from being able to optimize the implementation, as in this case. Drawback: Pain to add extensions, and potential mixing unrelated logic:
public class Example {
...
public void removeAll() {
synchronized (lock) { Runtime.exec(program + " -clear"); }
}
}
C: Provide a Closure class. Drawback: Excess code, potentially too generous of a synchronization block, could in fact make deadlocks easier:
public interface ExampleClosure {
public void execute(Example example);
}
public Class Example {
...
public void execute(ExampleClosure closure) {
synchronized (this) { closure.execute(this); }
}
}
example.execute(new ExampleClosure() {
public void execute(Example example) {
for (Object o : example.getAll()) {
example.remove(b);
}
}
}
);
Is there something I'm missing? How should synchronization be scoped to ensure the code is thread safe?
Use a ReentrantReadWriteLock which is exposed via the API. That way, if someone needs to synchronize several API calls, they can acquire a lock outside of the method calls.
In general, this is a classic multithreaded design issue. By synchronizing the data structure rather than synchronizing concepts that use the data structure, it's hard to avoid the fact that you essentially have a reference to the data structure without a lock.
I would recommend that locks not be done so close to the data structure. But it's a popular option.
A potential technique to make this style work is to use an editing tree-walker. Essentially, you expose a function that does a callback on each element.
// pointer to function:
// - takes Object by reference and can be safely altered
// - if returns true, Object will be removed from list
typedef bool (*callback_function)(Object *o);
public void editAll(callback_function func) {
synchronized (lock) {
for each element o { if (callback_function(o)) {remove o} } }
}
So then your loop becomes:
bool my_function(Object *o) {
...
if (some condition) return true;
}
...
editAll(my_function);
...
The company I work for (corensic) has test cases extracted from real bugs to verify that Jinx is finding the concurrency errors properly. This type of low level data structure locking without higher level synchronization is pretty common pattern. The tree editing callback seems to be a popular fix for this race condition.
I think everyone is missing his real problem. When iterating over the new array of Object's and trying to remove one at a time the problem is still technically unsafe (though ArrayList implantation would not explode, it just wouldnt have expected results).
Even with CopyOnWriteArrayList there is the possibility that there is an out of date read on the current list to when you are trying to remove.
The two suggestions you offered are fine (A and B). My general suggestion is B. Making a collection thread-safe is very difficult. A good way to do it is to give the client as little functionality as possible (within reason). So offering the removeAll method and removing the getAll method would suffice.
Now you can at the same time say, 'well I want to keep the API the way it is and let the client worry about additional thread-safety'. If thats the case, document thread-safety. Document the fact that a 'lookup and modify' action is both non atomic and non thread-safe.
Today's concurrent list implementations are all thread safe for the single functions that are offered (get, remove add are all thread safe). Compound functions are not though and the best that could be done is documenting how to make them thread safe.
I think j.u.c.CopyOnWriteArrayList is a good example of similar problem you're trying to solve.
JDK had a similar problem with Lists - there were various ways to synchronize on arbitrary methods, but no synchronization on multiple invocations (and that's understandable).
So CopyOnWriteArrayList actually implements the same interface but has a very special contract, and whoever calls it, is aware of it.
Similar with your solution - you should probably implement List (or whatever interface this is) and at the same time define special contracts for existing/new methods. For example, getAll's consistency is not guaranteed, and calls to .remove do not fail if o is null, or isn't inside the list, etc. If users want both combined and safe/consistent options - this class of yours would provide a special method that does exactly that (e.g. safeDeleteAll), leaving other methods close to original contract as possible.
So to answer your question - I would pick option B, but would also implement interface your original object is implementing.
From the Javadoc for List.toArray():
The returned array will be "safe" in
that no references to it are
maintained by this list. (In other
words, this method must allocate a new
array even if this list is backed by
an array). The caller is thus free to
modify the returned array.
Maybe I don't understand what you're trying to accomplish. Do you want the Object[] array to always be in-sync with the current state of the List? In order to achieve that, I think you would have to synchronize on the Example instance itself and hold the lock until your thread is done with its method call AND any Object[] array it is currently using. Otherwise, how will you ever know if the original List has been modified by another thread?
You have to use the appropriate granularity when you choose what to lock. What you're complaining about in your example is too low a level of granularity, where the lock doesn't cover all the methods that have to happen together. You need to make methods that combine all the actions that need to happen together within the same lock.
Locks are reentrant so the high-level method can call low-level synchronized methods without a problem.
I have heard that there are different levels of Synchronization.are there?(If there are ,would you please explain them with a snippet code?)thanks.
Before Java 5 there was only one: the synchronized keyword. This waited for and obtained an exclusive lock on the reference object. When applied to a function:
public synchronized void doStuff() { ... }
the object being synchronized on is this.
Java 5 added a lot of concurrency utils, one of which was the Lock object. There are several versions of this including a ReadWriteLock. This is the only thing I can think of that you might be referring to.
The problem with synchronized is that its fairly crude. Done badly it can lead to deadlocks. The Java 5 utils allow non-blocking lock acquisition, timeout on lock acquisition and read/write lock support.
You'd really need to explain what you meant by a "level of synchronization". Are you talking about the difference between:
public synchronized void foo()
{
...
}
and
public void foo()
{
synchronized(lock)
{
...
}
}
? Or perhaps between the above and using the locks from java.util.concurrent.locks?
If you could give more context to what you've heard, we may be able to help you better. More importantly, what problem are you trying to solve that you think might need this information?
I am assuming OP is referring to synchronization of object used in multiple thread and not the keyword in Java.
This is my understanding, after reading Goetz, of different levels of synchronizations in Java.
Lets assume we have a object that we have to use across 132 threads, which use the object continuously, purely randomly, over say 100 years.
abstract Class ICount {
int i;
String iString;
public void add(int i);
public void sub(int i);
public void synchronized print(){
assert(i == Integer.valueOf(iString));
System.out.println("i"+String.valueof(i)+" == "+iString);
}
}
Now there could be different implementation which either by design or by carelessness, be thread safe at different levels.
Thread safe synchronization implementation : Guarantees that any java code, that doesn't use reflection to screw the object, using this implementation, will never hit assert.
Concurrent synchronization implementation : In addition to guaranteeing thread-safety, the author of this implementation tried to allow concurrent executions when possible, and also gave a shot at avoiding locks (contentions).
Thread unsafe synchronization implementation : Developer of this, at least knows his class his not thread safe, and requests you not to use this implementation unless you have exactly one thread.
If this were a real problem, the majority of the practical implementation falls in fourth category of 'synchronization', which is it works sometimes, and not documentation sufficiently to categorize in one of the above categories, and hence by default falls in fourth category.