What is the right way to solve following problem?
Writing a logic where at a same time 100 reader(Servlet requests) or one writer(Servlet requests) can be accessing critical section for one key in a map(Cache).
If writer comes into the picture in that case all reader should stop there progress and should restart once writer done with critical section processing(Re population cache element for same key).
I implemented one of the solution like in this question, where one instance of Resource class will be associated with single key.
class Resource {
private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private final Lock rlock = lock.readLock();
private final Lock wlock = lock.writeLock();
void read() { ... /* caller has to hold the read lock */ ... }
void write() { ... /* caller has to hold the write lock */ ... }
Lock readLock() { return rlock; }
Lock writeLock() { return wlock; }
}
Previously I implemented simple logic using Semaphore where I have associated one semaphore instance with single key and used 100 permits for the same and if writer thread is coming into the picture in that case I consumed all remaining permits(drainPermits) and letting all permit free by all readers and putted writer thread in waiting queue. But it leads to starvation to writer.
Other thing I was thinking that using ConcurrentHashMap could solve it? As ConcurrentHashMap have key based locking internally(Segments).
You don't have to expose the locking to the user of the resource, however if you start implementing that pattern you soon discover that you may as well use a ConcurrentHashMap which is actually optimized well for synchronized access
class Resource {
private Cache<Key, Value> yourcache;
private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private final Lock rlock = lock.readLock();
private final Lock wlock = lock.writeLock();
Value read(Key key) { try {
rlock.lock();
return yourcache.get(key)
} finally {
rlock.unlock();
}
}
void write(Key key) { ... /* similar pattern to above */ ... }
Lock readLock() { return rlock; } //don't expose these at all!
Lock writeLock() { return wlock; }//don't expose these at all!
}
Related
A server project, might run for very long time and create many threads.
In the following code I ask myself do i have to protect the lock somehow in addition to an overall try catch in method setData(MyData data):
note: assuming its thread-safe, i am not really familiar with other reasons or natural disasters that may cause thread termination, like windows 7 limitations or so. (i am not greedy, using +-5 threads, but still)
public class MyFactory {
private ReadWriteLock rwl = new ReentrantReadWriteLock();
private Lock readLock = rwl.readLock();
private Lock writeLock = rwl.writeLock();
private static MyFactory _instance = new MyFactory();
private static Map<Integer, MyData> mapDatas = new HashMap<>();
public static MyFactory getInstance() {
return _instance;
}
public void setData(MyData data) {
writeLock.lock();
try {
mapData.put(...);
} catch (Exception exc) {
...
} finally {
writeLock.unlock();
}
}
Assuming that you never expose the lock to other objects and you always use unlock the lock in a finally block, you are fine.
The reason is that the code in the finally block is always called if something happens in the try or catch blocks. Even if an Error is thrown this is true. These happen for example when you're out of memory or there is a linkage error with some DLL. If something worse happens than an Error, that will likely also end your process and make the problem moot.
I want to create a semaphore that prevents a certain method to be executed more than 1x at a time.
If any other thread requests access, it should wait until the semaphore is released:
private Map<String, Semaphore> map;
public void test() {
String hash; //prevent to run the long running method with the same hash concurrently
if (map.contains(hash)) {
map.get(hash).aquire(); //wait for release of the lock
callLongRunningMethod();
} else {
Semaphore s = new Semaphore(1);
map.put(hash, s);
callLongRunningMethod();
s.release(); //any number of registered threads should continue
map.remove(hash);
}
}
Question: how can I lock the semaphore with just one thread, but release it so that any number of threads can continue as soon as released?
Some clarifications:
Imagine the long running method is a transactional method. Looks into the database. If no entry is found, a heavy XML request is send and persisted to db. Also maybe further async processed might be triggered as this is supposed to be the "initial fetch" of the data. Then return the object from DB (within that method). If the DB entry had existed, it would directly return the entity.
Now if multiple threads access the long running method at the same time, all methods would fetch the heavy XML (traffic, performance), and all of them would try to persist the same object into the DB (because the long running method is transactional). Causing eg non-unique exceptions. Plus all of them triggering the optional async threads.
When all but one thread is locked, only the first is responsible for persisting the object. Then, when finished, all other threads will detect that the entry already exists in DB and just serve that object.
As far as I understand, you don't need to use Semaphore here. Instead, you should use ReentrantReadWriteLock. Additionally, the test method is not thread safe.
The sample below is the implementation of your logic using RWL
private ConcurrentMap<String, ReadWriteLock> map = null;
void test() {
String hash = null;
ReadWriteLock rwl = new ReentrantReadWriteLock(false);
ReadWriteLock lock = map.putIfAbsent(hash, rwl);
if (lock == null) {
lock = rwl;
}
if (lock.writeLock().tryLock()) {
try {
compute();
map.remove(hash);
} finally {
lock.writeLock().unlock();
}
} else {
lock.readLock().lock();
try {
compute();
} finally {
lock.readLock().unlock();
}
}
}
In this code, the first successful thread would acquire WriteLock while other Threads would wait for release of write lock. After release of a WriteLock all Threads waiting for release would proceed concurrently.
As far as I understand your need you want to be able to ensure that the task is executed by one single thread for the first time then you want to allow several threads to execute it if so you need to rely on a CountDownLatch as next:
Here is how it could be implemented with CountDownLatch:
private final ConcurrentMap<String, CountDownLatch> map = new ConcurrentHashMap<>();
public void test(String hash) {
final CountDownLatch latch = new CountDownLatch(1);
final CountDownLatch previous = map.putIfAbsent(hash, latch);
if (previous == null) {
try {
callLongRunningMethod();
} finally {
map.remove(hash, latch);
latch.countDown();
}
} else {
try {
previous.await();
callLongRunningMethod();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
I think you could do that by using a very high permit number (higher than the number of threads, e.g. 2000000).
Then in the function that should run exclusively you acquire the complete number of permits (acquire(2000000)) and in the other threads you acquire only a single permit.
I think that the easiest way to do this would be using an ExecutorService and Future:
class ContainingClass {
private final ConcurrentHashMap<String, Future<?>> pending =
new ConcurrentHashMap<>();
private final ExecutorService executor;
ContainingClass(ExecutorService executor) {
this.executor = executor;
}
void test(String hash) {
Future<?> future = pending.computeIfAbsent(
hash,
() -> executor.submit(() -> longRunningMethod()));
// Exception handling omitted for clarity.
try {
future.get(); // Block until LRM has finished.
} finally {
// Always remove: in case of exception, this allows
// the value to be computed again.
pending.values().remove(future);
}
}
}
Ideone Demo
Removing the future from the values is thread safe because computeIfAbsent and remove are atomic: either the computeIfAbsent is run before the remove, in which case the existing future is returned, and is immediately complete; or it is run after, and a new future is added, resulting in a new call to longRunningMethod.
Note that it removes the future from pending.values(), not from the pending directly: consider the following example:
Thread 1 and Thread 2 are run concurrently
Thread 1 completes, and removes the value.
Thread 3 is run, adding a new future to the map
Thread 2 completes, and tries to remove the value.
If the future were removed from the map by key, Thread 2 would remove Thread 3's future, which is a different instance from Thread 2's future.
This simplifies the longRunningMethod too, since it is no longer required to do the "check if I need to do anything" for the blocked threads: that the Future.get() has completed successfully in the blocking thread is sufficient to indicate that no additional work is needed.
I ended as follows using CountDownLatch:
private final ConcurrentMap<String, CountDownLatch> map = new ConcurrentHashMap<>();
public void run() {
boolean active = false;
CountDownLatch count = null;
try {
if (map.containsKey(hash)) {
count = map.get(hash);
count.await(60, TimeUnit.SECONDS); //wait for release or timeout
} else {
count = new CountDownLatch(1);
map.put(hash, count); //block any threads with same hash
active = true;
}
return runLongRunningTask();
} finally {
if (active) {
count.countDown(); //release
map.remove(hash, count);
}
}
}
I'm using something like
Cache<Integer, Item> cache;
where the Items are independent of each other and look like
private static class Item {
private final int id;
... some mutable data
synchronized doSomething() {...}
synchronized doSomethingElse() {...}
}
The idea is to obtain the item from the cache and call a synchronized method on it. In case of a miss, the item can be recreated, that's fine.
A problem occurs when an item gets evicted from the cache and recreated while a thread runs a synchronized method. A new thread obtains a new item and synchronizes on it... so for a single id, there are two threads inside the synchronized method. FAIL.
Is there an easy way around it? It's Guava Cache, if it helps.
I think the suggestion from Louis, using the the keys for locking is the most simple and practical one. Here is code some snippet, that, without the help of Guava libraries, illustrates the idea:
static locks[] = new Lock[ ... ];
static { /* initialize lock array */ }
int id;
void doSomething() {
final lock = locks[id % locks.length];
lock.lock();
try {
/* protected code */
} finally {
lock.unlock();
}
}
The size of the lock array limits the maximum amount of parallelism you get. If your code is only using CPU, you can initialize it by the number of available processors and this is the perfect solution. If your code waits for I/O you might need an arbitrary big array of locks or you limit the number of threads that can run the critical section. In this case another approach might be better.
Comments on a more conceptual level:
If you want to prevent the item from being evicted, you need a mechanism called pinning. Internally this is used by most cache implementations, e.g. for blocking during I/O operations. Some caches may expose a way to do it by the applications.
In a JCache compatible cache, there is the concept of an EntryProcessor. The EntryProcessor allows you to process a peace of code on an entry in an atomic way. This means the cache is doing all the locking for you. Depending of the scope of the problem, this may have an advantage, since this also works in clustered scenarios, which means the locking is cluster wide.
Another idea which comes to my mind is the vetoable eviction. This is a concept EHCache 3 is implementing. By specifying a vetoable eviction policy you can implement a pinning mechanism on your own.
I'm sure that there are multiple solutions for your issue.
I wrote down one of them with using a unique lock for each ietmId:
public class LockManager {
private Map<Integer, Lock> lockMap = new ConcurrentHashMap<>();
public synchronized Lock getOrCreateLockForId(Integer itemId) {
Lock lock;
if (lockMap.containsKey(itemId)) {
System.out.println("Get lock");
lock = lockMap.get(itemId);
} else {
System.out.println("Create lock");
lock = new ReentrantLock();
lockMap.put(itemId, lock);
}
return lock;
}
public synchronized Lock getLockForId(Integer itemId) {
Lock lock;
if (lockMap.containsKey(itemId)) {
System.out.println("get lock");
return lockMap.get(itemId);
} else {
throw new IllegalStateException("First lock, than unlock");
}
}
}
So, instead of using synchronised methods in class Item use LockManager to get Lock by itemId and call lock.lock() after it was retrieved.
Also note that LockManager should have singleton scope and the same instance should be shared across all usages.
Below you can see example of LockManager using:
try {
lockManager.getOrCreateLockForId(itemId).lock();
System.out.println("start doing something" + num);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("completed doing something" + num);
} finally {
lockManager.getLockForId(itemId).unlock();
}
I have a service bean which provides access to a Map. From time to time I need to rebuild the content of the Map wich takes several seconds and I want to block the access to the map while its rebuilding, because it can be accessed from different Threads.
#Service
public class MyService {
private Map<Key,Value> cache = null;
private ReentrantLock reentrantLock = new ReentrantLock();
public void rebuildCache(){
try {
reentrantLock.lock();
cache = new ConcurrentHashMap<>();
... //processing time consuming stuff and building up the cache
}finally {
reentrantLock.unlock();
}
}
public Value getValue(Key key){
while (lock.isLocked()){}
return cache.get(key);
}
...
}
As you can see I use
while (reentrantLock.isLocked()){}
to check if the lock is locked and wait until its unlocked. This solution seems to be quite dirty. Is there a better solution?
Use a ReentrantReadWriteLock instead.
In your write method:
theLock.writeLock().lock();
try {
// update the map
} finally {
theLock.writeLock().unlock();
}
In the read method, use the .readLock() instead.
This has the problem however that during the update of the map, all readers will be blocked; another solution would be to use a plain lock to replace the reference of the old map to a new, updated one, and use a plain old synchronized.
More importantly though, your use of locks is incorrect. You should do:
theLock.lock();
try {
// whatever
} finally {
theLock.unlock();
}
Imagine what happens if the locking fails with your current lock: you'll always try to unlock and you'll end up with an IllegalLockStateException.
I would propose a ReadWriteLock.
With it you can read as many times as you want, as long as read lock is not locked.
#Service
public class MyService {
private Map<Key,Value> cache = null;
private ReentrantLock reentrantLock = new ReentrantLock();
public void rebuildCache(){
try {
reentrantLock.writeLock().lock();
cache = new ConcurrentHashMap<>();
... //processing time consuming stuff and building up the cache
}finally {
reentrantLock.writeLock().unlock();
}
}
public Value getValue(Key key){
if(reentrantLock.getReadLock().lock()){
return cache.get(key);
}finally{
reentrantLock.getReadLock().unlock();
}
}
...
}
We need to lock a method responsible for loading database date into a HashMap based cache.
A possible situation is that a second thread tries to access the method while the first method is still loading cache.
We consider the second thread's effort in this case to be superfluous. We would therefore like to have that second thread wait until the first thread is finished, and then return (without loading the cache again).
What I have works, but it seems quite inelegant. Are there better solutions?
private static final ReentrantLock cacheLock = new ReentrantLock();
private void loadCachemap() {
if (cacheLock.tryLock()) {
try {
this.cachemap = retrieveParamCacheMap();
} finally {
cacheLock.unlock();
}
} else {
try {
cacheLock.lock(); // wait until thread doing the load is finished
} finally {
try {
cacheLock.unlock();
} catch (IllegalMonitorStateException e) {
logger.error("loadCachemap() finally {}",e);
}
}
}
}
I prefer a more resilient approach using read locks AND write locks. Something like:
private static final ReadWriteLock cacheLock = new ReentrantReadWriteLock();
private static final Lock cacheReadLock = cacheLock.readLock();
private static final Lock cacheWriteLock = cacheLock.writeLock();
private void loadCache() throws Exception {
// Expiry.
while (storeCache.expired(CachePill)) {
/**
* Allow only one in - all others will wait for 5 seconds before checking again.
*
* Eventually the one that got in will finish loading, refresh the Cache pill and let all the waiting ones out.
*
* Also waits until all read locks have been released - not sure if that might cause problems under busy conditions.
*/
if (cacheWriteLock.tryLock(5, TimeUnit.SECONDS)) {
try {
// Got a lock! Start the rebuild if still out of date.
if (storeCache.expired(CachePill)) {
rebuildCache();
}
} finally {
cacheWriteLock.unlock();
}
}
}
}
Note that the storeCache.expired(CachePill) detects a stale cache which may be more than you are wanting but the concept here is the same, establish a write lock before updating the cache which will deny all read attempts until the rebuild is done. Also, manage multiple attempts at write in a loop of some sort or just drop out and let the read lock wait for access.
A read from the cache now looks like this:
public Object load(String id) throws Exception {
Store store = null;
// Make sure cache is fresh.
loadCache();
try {
// Establish a read lock so we do not attempt a read while teh cache is being updated.
cacheReadLock.lock();
store = storeCache.get(storeId);
} finally {
// Make sure the lock is cleared.
cacheReadLock.unlock();
}
return store;
}
The primary benefit of this form is that read access does not block other read access but everything stops cleanly during a rebuild - even other rebuilds.
You didn't say how complicated your structure is and how much concurrency / congestion you need. There are many ways to address your need.
If your data is simple, use a ConcurrentHashMap or similar to hold your data. Then just read and write in threads regardlessly.
Another alternative is to use actor model and put read/write on the same queue.
If all you need is to fill a read-only map which is initialized from database once requested, you could use any form of double-check locking which may be implemented in a number of ways. The easiest variant would be the following:
private volatile Map<T, V> cacheMap;
public void loadCacheMap() {
if (cacheMap == null) {
synchronized (this) {
if (cacheMap == null) {
cacheMap = retrieveParamCacheMap();
}
}
}
}
But I would personally prefer to avoid any form of synchronization here and just make sure that the initialization is done before any other thread can access it (for example in a form of init method in a DI container). In this case you would even avoid overhead of volatile.
EDIT: The answer works only when initial load is expected. In case of multiple updates, you could try to replace the tryLock by some other form of test and test-and-set, for example using something like this:
private final AtomicReference<CountDownLatch> sync =
new AtomicReference<>(new CountDownLatch(0));
private void loadCacheMap() {
CountDownLatch oldSync = sync.get();
if (oldSync.getCount() == 0) { // if nobody updating now
CountDownLatch newSync = new CountDownLatch(1);
if (sync.compareAndSet(oldSync, newSync)) {
cacheMap = retrieveParamCacheMap();
newSync.countDown();
return;
}
}
sync.get().await();
}