I am aware that there are already a lot of similar questions on the internet, but my question is about my code, NOT ABOUT THREADS. I am making a small app that has a database of players. The code of the data storing class is as follows.
public class DataManager
{
static final int NO_OF_COLUMNS = 18;
static QDatabase pdb;
public DataManager()
{
pdb = new QDatabase(NO_OF_COLUMNS);
}
public void addPlayer(Object[] playerData)
{
pdb.add(playerData);
}
public void editPlayerInfo(int type, int playerRegNo, Object data)
{
pdb.set(type, playerRegNo, data);
}
public int getPlayerRegNo(String userID)
{
return (int) pdb.getData(USER_ID, userID, REG_NO);
}
public Boolean contains(int column, Object data)
{
return pdb.contains(column, data);
}
}
I have a server which keeps on recieving requests from multiple clients and creating a new thread for each of them. They all access this DataManager class which essentially acts as a database. Will it be possible for me to, in some way, make it possible for all the threads to be able to call the addPlayer() and editPlayerInfo() methods at the same time and yet not mess the whole thing up due to synchronization problems?
I also know that I can use databases. But here, I just thought that this would be easier. Assume that there will be about 200 threads running simultaneously. What is the best way for me to solve this?
Is there any way for me to hve all the threads access it at the same time as otherwise having 200 threads to wait on each other might become very slow?
EDIT 1:
The QDatabase class is as follows:
public class QDatabase implements Serializable
{
private ArrayList<ArrayList<Object>> database;
public final int NOT_EXISTS = 0, REGULAR = 0, TRANSPOSE = 1;
private int lastid = -1;
//Initializer taking the number of columns as an argument
public QDatabase(int noofcolumns)
{
database = new ArrayList<ArrayList<Object>>();
addColumns(noofcolumns);
}
//Method that adds an array of objects as a new row in the database.
public void add(Object[] object)
{
for(int index = 0; index < database.size(); index++)
{
if(object != null)
{
database.get(index).add(object[index]);
lastid = database.get(0).indexOf(object[0]);
}
}
}
//Method that finds the row in a column where an instance of a particular object is found and get the values at a
//cell with the same row and a given column.
public Object getData(int columntocheck, Object check, int columntoget)
{
Object ramobject = null;
int loc = database.get(columntocheck).indexOf(check);
ramobject = database.get(columntoget).get(loc);
return ramobject;
}
//Method to check if a column contains an instance of a given object.
public Boolean contains(int column, Object objecttocheck)
{
return database.get(column).contains(objecttocheck);
}
//Method to set a given cell to an object.
public void set(int column, int row, Object object)
{
database.get(column).set(row, object);
}
}
QDatabase is not thread-safe. You need either synchronize all its methods or use a thread-safe variant of ArrayList - CopyOnWriteArrayList from java.util.concurrent package. But be careful, using CopyOnWriteArrayList makes sense only if the number of reads from DB vastly outnumbers the number of writes. See API, it creates a fresh copy of unerlying array on all mutative operations.
UPDATE:
Actually, the most efficient solutiion in your situation seems to be ReadWriteLock. Use ReadLock for all reading operations and WriteLock for all mutative operations, like this
public class QDatabase implements Serializable {
private ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private Lock readLock = readWriteLock.readLock();
private Lock writeLock = readWriteLock.writeLock();
...
public void add(Object[] object) {
writeLock.lock();
try {
...
}
} finally {
writeLock.unlock();
}
}
public Object getData(int columntocheck, Object check, int columntoget) {
readLock.lock();
try {
...
} finally {
readLock.unlock();
}
}
...
just add synchronized block
public synchronized void addPlayer(Object[] playerData)
{
pdb.add(playerData);
}
public synchronized void editPlayerInfo(int type, int playerRegNo, Object data)
{
pdb.set(type, playerRegNo, data);
}
It will make sure that no two threads will access this method at the same time.
One of the ways to have simultaneous access by multiple threads but yet remain thread safe is to use local variables or use ThreadLocal.
None of them is feasible in your case, so you can't achieve simultaneous access of thread, it has to be sequential.
Have a look at the java.util.concurrency package. You could use classes there to manage your threading needs better.
In order for a class/ method to be "thread safe" it has to be designed so. Now, its not clear what the DATABASE object you have is doing internally but it does look like from the method names , that multiple threads ARE going to be an issue.
In order to INCREASE the number of threads , yet not keeping the ENTIRE method synchronized, look into the details of the add/edit methods implementations and , yes you would have to limit thread access to those lines of code that would cause issues.
You could use principles like multiple READ ,single WRITE locks etc.
Related
I have a Set with any type of values and an AtomicBoolean that indicates if the functionality provided by that class is running.
private Set<Object> set = new HashSet<>();
private AtomicBoolean running;
Now, i have two methods, one of them is adding objects to the set and the other serves as a setup method for my class.
public void start() {
// ...
set.foreEach(someApi::addObject);
// ...
running.set(true);
}
public void addObject(Object o) {
set.add(o);
if(running.get()) {
someApi.addObject(o);
}
}
However, there is a problem with that code. If the method is invoked from another thread while the start method is iterating through the set running is still false. Thus, the object will not be added to the api.
Question: How can i guarantee that all objects in the set and objects added with addObject will be added to the api exactly one time?
My ideas:
use a lock and block the addObject method if the setup is currently adding methods to the api (or make both methods synchronized, which will slightly decrease performence tough)
Question: How can i guarantee that all objects in the set and objects added with addObject will be added to the api exactly one time?
You have to be careful here because this gets close to the ole "double check locking bug".
If I understand you question you want to:
queue the objects passed into addObject(...) in the set before the call to start().
then when start() is called, call the API method on the objects in the set.
handle the overlap if additional objects are added during the call to start()
call the method once and only once on all objects passed to addObject(...).
What is confusing is that your API call is also named addObject(). I assume this is different from the addObject(...) method in your code sample. I'm going to rename it below to be someApiMethod(...) to show that it's not going recursive.
The easiest way is going to be, unfortunately, having a synchronized block in each of the methods:
private final Set<Object> set = new HashSet<>();
public void start() {
synchronized (set) {
set.forEach(someApi::someApiMethod);
}
}
public void addObject(Object obj) {
synchronized (set) {
if (set.add(obj)) {
someApi.addObject(obj);
}
}
}
}
To make it faster is going to take a lot more complicated code. One thing you could do is use a ConcurrentHashMap and a AtomicBoolean running. Something like:
private final ConcurrentMap<Object, Object> map = new ConcurrentHashMap<>();
private final Set<Object> beforeStart = new HashSet<>();
private final AtomicBoolean running = new AtomicBoolean();
public void start() {
synchronized (beforeStart) {
for (Object obj : beforeStart) {
doIfAbsent(obj);
}
running.set(true);
}
}
public void addObject(Object obj) {
if (running.get()) {
doIfAbsent(obj);
} else {
synchronized (beforeStart) {
// we have to test running again once we get the lock
if (running.get()) {
doIfAbsent(obj);
} else {
beforeStart.add(obj);
}
}
}
}
private void doIfAbsent(Object obj) {
if (map.putIfAbsent(obj, obj)) {
someApi.someApiMethod(obj);
}
}
This is pretty complicated and it may not be any faster depending on how large your hash map is and other factors.
We need to lock a method responsible for loading database date into a HashMap based cache.
A possible situation is that a second thread tries to access the method while the first method is still loading cache.
We consider the second thread's effort in this case to be superfluous. We would therefore like to have that second thread wait until the first thread is finished, and then return (without loading the cache again).
What I have works, but it seems quite inelegant. Are there better solutions?
private static final ReentrantLock cacheLock = new ReentrantLock();
private void loadCachemap() {
if (cacheLock.tryLock()) {
try {
this.cachemap = retrieveParamCacheMap();
} finally {
cacheLock.unlock();
}
} else {
try {
cacheLock.lock(); // wait until thread doing the load is finished
} finally {
try {
cacheLock.unlock();
} catch (IllegalMonitorStateException e) {
logger.error("loadCachemap() finally {}",e);
}
}
}
}
I prefer a more resilient approach using read locks AND write locks. Something like:
private static final ReadWriteLock cacheLock = new ReentrantReadWriteLock();
private static final Lock cacheReadLock = cacheLock.readLock();
private static final Lock cacheWriteLock = cacheLock.writeLock();
private void loadCache() throws Exception {
// Expiry.
while (storeCache.expired(CachePill)) {
/**
* Allow only one in - all others will wait for 5 seconds before checking again.
*
* Eventually the one that got in will finish loading, refresh the Cache pill and let all the waiting ones out.
*
* Also waits until all read locks have been released - not sure if that might cause problems under busy conditions.
*/
if (cacheWriteLock.tryLock(5, TimeUnit.SECONDS)) {
try {
// Got a lock! Start the rebuild if still out of date.
if (storeCache.expired(CachePill)) {
rebuildCache();
}
} finally {
cacheWriteLock.unlock();
}
}
}
}
Note that the storeCache.expired(CachePill) detects a stale cache which may be more than you are wanting but the concept here is the same, establish a write lock before updating the cache which will deny all read attempts until the rebuild is done. Also, manage multiple attempts at write in a loop of some sort or just drop out and let the read lock wait for access.
A read from the cache now looks like this:
public Object load(String id) throws Exception {
Store store = null;
// Make sure cache is fresh.
loadCache();
try {
// Establish a read lock so we do not attempt a read while teh cache is being updated.
cacheReadLock.lock();
store = storeCache.get(storeId);
} finally {
// Make sure the lock is cleared.
cacheReadLock.unlock();
}
return store;
}
The primary benefit of this form is that read access does not block other read access but everything stops cleanly during a rebuild - even other rebuilds.
You didn't say how complicated your structure is and how much concurrency / congestion you need. There are many ways to address your need.
If your data is simple, use a ConcurrentHashMap or similar to hold your data. Then just read and write in threads regardlessly.
Another alternative is to use actor model and put read/write on the same queue.
If all you need is to fill a read-only map which is initialized from database once requested, you could use any form of double-check locking which may be implemented in a number of ways. The easiest variant would be the following:
private volatile Map<T, V> cacheMap;
public void loadCacheMap() {
if (cacheMap == null) {
synchronized (this) {
if (cacheMap == null) {
cacheMap = retrieveParamCacheMap();
}
}
}
}
But I would personally prefer to avoid any form of synchronization here and just make sure that the initialization is done before any other thread can access it (for example in a form of init method in a DI container). In this case you would even avoid overhead of volatile.
EDIT: The answer works only when initial load is expected. In case of multiple updates, you could try to replace the tryLock by some other form of test and test-and-set, for example using something like this:
private final AtomicReference<CountDownLatch> sync =
new AtomicReference<>(new CountDownLatch(0));
private void loadCacheMap() {
CountDownLatch oldSync = sync.get();
if (oldSync.getCount() == 0) { // if nobody updating now
CountDownLatch newSync = new CountDownLatch(1);
if (sync.compareAndSet(oldSync, newSync)) {
cacheMap = retrieveParamCacheMap();
newSync.countDown();
return;
}
}
sync.get().await();
}
Suppose I have the following:
public class Foo {
private ReadingList mReadingList = new ReadingList();
public ReadingList getReadingList() {
synchronized (mReadingList) {
return mReadingList;
}
}
}
If I try modifying the ReadingList object in two threads, the synchronization above won't help me, right?:
// Thread 1
foo1.getReadingList().setName("aaa");
// Thread 2
foo2.getReadingList().setName("bbb");
do I have to wrap each method I want synchronized like so:
public class Foo {
private ReadingList mReadingList = new ReadingList();
public synchronized void setReadingListName(String name) {
mReadingList.setName(name);
}
public synchronized void setReadingListAuthor(String author) {
mReadingList.setAuthor(author);
}
...
and so on for each method of ReadingList I want exposed and synched? I'd end up just writing wrapper methods for each method of ReadingList.
Thanks
1. You have access to the ReadingList source
If you have access to the ReadingList object, add synchronized to all of the methods of ReadingList if you desire synchronized access to all of the fields or a certain group of setters if you only wish to interleave access to certain fields.
2. You do not have access to ReadingList
You would have to write something like:
public class Foo {
private ReadingList mReadingList = new ReadingList();
public void setReadingListName(String name) {
synchronized(mReadingList) {
mReadingList.setName(name);
}
}
public void setReadingListAuthor(String author) {
synchronized(mReadingList) {
mReadingList.setAuthor(author);
}
}
...
3. Use a general purpose lock object
Depending on the nature of Foo and how general-purpose this whole thing is, you may find that only a certain class or classes present the threading issue in ReadingList.
In such a class you could use a general purpose lock object:
public class Bar {
Object readingListLock = new Object();
public void someMethodThatModifiesReading() {
synchronized(readingListLock) {
foo.getReadingList().setName("1");
}
}
public void someOtherMethodThatModifiesReading() {
synchronized(readingListLock) {
foo.getReadingList().setName("2");
}
}
...
}
A quick (and not so efficient solution) is to synchronize all methods of ReadingList in ReadingList's implementation.
Look for Reader-Writer lock for a more efficient way to synchronize access: it allows multiple reads and single write at a time.
Your first solution only makes sure one thread gets the ReadingList at a time, and nothing else - many threads can read and modify the ReadingList concurrently.
The following is a simplified version of my current code. I am pretty sure I am not doing any thing wrong syntax-wise, and I can't locate my conceptual mistake.
This is sort of an observer pattern I tried to implement. I could not afford to inherit from Java.utils.observable as my class is already complicated and inherits from another class.
There are two parts here:
There's a Notifier class implementing Runnable :
public class Notifier implements Runnable{
public void run()
{
while(true)
{
MyDataType data = getData();
if(data.isChanged()==true)
{
refresh();
}
}
}
}
And then there is my main class which needs to respond to changes to MyDataType data.
public class abc {
private MyDataType data;
public void abc(){
Notifier notifier = new Notifier();
Thread thread = new Thread(notifier);
thread.start();
}
public MyDataType getData(){
return this.data;
}
public void refresh(){
MyDatatype data = getData();
//Do something with data
}
}
The problem : What's happening is that the notifier is calling refresh() when 'data' changes. However inside refresh(), when I do getData(), I am getting the old version of 'data'!
I should mention that there are other parts of the code which are calling the refresh() function too.
What am I overlooking?
Any other better solutions to this problem?
How should I approach designing Subject-Observer systems if I can't apply the default Java implementation out of the box?
when I do getData(), I am getting the old version of 'data'!
Your data field is shared among more than one thread so it must be marked with the volatile keyword.
private volatile MyDataType data;
This causes a "memory barrier" around the read and the the write that keeps the value visible to all threads. Even though the notifier thread is calling getData(), the value for data is being retrieved out if its memory cache. Without the memory barrier, the data value will be updated randomly or never.
As #JB mentioned in the comments, the volatile protects you against a re-assignment of the data field. If you update one of the fields within the current data value, the memory barrier will not be crossed that the notifier's memory will not be updated.
Looking back at your code, it looks like this is the case:
if(data.isChanged()==true)
{
refresh();
}
If data is not being assigned to a new object then making data to be volatile won't help you. You will have to:
Set some sort of volatile boolean dirty; field whenever data has been updated.
Update or read data within a synchronize block each and every time.
First, your data variable might be cached, so you will always need to get the latest value by making it volatile.
Second, what you are doing here is a producer / consumer pattern. This pattern is usually best implemented with messages. When you receive new data, you could create an immutable object and post it to the consumer thread (via a thread safe queue like a BlockingQueue) instead of having a shared variable.
Something along these lines:
public class Notifier extends Thread{
private BlockingQueue<E> consumerQueue = null;
public setConsumerQueue(BlockingQueue<E> val){
consumerQueue = val;
}
// main method where data is received from socket...
public void run(){
while(!interrupted()){
data = ... // got new data here
if(!data.isChanged()) continue;
// Post new data only when it has changed
if(consumerQueue!=null) consumerQueue.offer(data);
}
}
}
public class Consumer extends Thread{
private BlockingQueue<E> consumerQueue = new BlockingQueue<E>();
public Consumer (Producer val){
val.setConsumerQueue(consumerQueue);
}
public void run(){
while(!interrupted()){
data = consumerQueue.take();// block until there is data from producer
if(data !=null) processData(data);
}
}
}
I have a store of data objects and I wish to synchronize modifications that are related to one particular object at a time.
class DataStore {
Map<ID, DataObject> objects = // ...
// other indices and stuff...
public final void doSomethingToObject(ID id) { /* ... */ }
public final void doSomethingElseToObject(ID id) { /* ... */ }
}
That is to say, I do not wish my data store to have a single lock since modifications to different data objects are completely orthogonal. Instead, I want to be able to take a lock that pertains to a single data object only.
Each data object has a unique id. One way is to create a map of ID => Lock and synchronize upon the one lock object associated with the id. Another way is to do something like:
synchronize(dataObject.getId().toString().intern()) {
// ...
}
However, this seems like a memory leak -- the internalized strings may never be collected.
Yet another idea is to synchronize upon the data object itself; however, what if you have an operation where the data object doesn't exist yet? For example, what will a method like addDataObject(DataObject) synchronize upon?
In summary, how can I write a function f(s), where s is a String, such that f(s)==f(t) if s.equals(t) in a memory-safe manner?
Add the lock directly to this DataObject, you could define it like this:
public class DataObject {
private Lock lock = new ReentrantLock();
public void lock() { this.lock.lock(); }
public void unlock() { this.lock.unlock(); }
public void doWithAction( DataObjectAction action ) {
this.lock();
try {
action.doWithLock( this ) :
} finally {
this.unlock();
}
}
// other methods here
}
public interface DataObjectAction { void doWithLock( DataObject object ); }
And when using it, you could simply do it like this:
DataObject object = // something here
object.doWithAction( new DataObjectAction() {
public void doWithLock( DataObject object ) {
object.setProperty( "Setting the value inside a locked object" );
}
} );
And there you have a single object locked for changes.
You could even make this a read-write lock if you also have read operations happening while writting.
For such case, I normally have 2 level of lock:
First level as a reader-writer-lock, which make sure update to the map (add/delete) is properly synchronized by treating them as "write", and access to entries in map is considered as "read" on the map. Once accessed to the value, then synchronize on the value. Here is a little example:
class DataStore {
Map<ID, DataObject> objMap = // ...
ReadWritLock objMapLock = new ReentrantReadWriteLock();
// other indices and stuff...
public void addDataObject(DataObject obj) {
objMapLock.writeLock().lock();
try {
// do what u need, u may synchronize on obj too, depends on situation
objMap.put(obj.getId(), obj);
} finally {
objMapLock.writeLock().unlock();
}
}
public final void doSomethingToObject(ID id) {
objMapLock.readLock().lock();
try {
DataObject dataObj = this.objMap.get(id);
synchronized(dataObj) {
// do what u need
}
} finally {
objMapLock.readLock().unlock();
}
}
}
Everything should then be properly synchronized without sacrificing much concurrency
Yet another idea is to synchronize upon the data object itself; however, what if you have an operation where the data object doesn't exist yet? For example, what will a method like addDataObject(DataObject) synchronize upon?
Synchronizing on the object is probably viable.
If the object doesn't exist yet, then nothing else can see it. Provided that you can arrange that the object is fully initialized by its constructor, and that it is not published by the constructor before the constructor returns, then you don't need to synchronize it. Another approach is to partially initialize in the constructor, and then use synchronized methods to do the rest of the construction and the publication.