I'd like to be able to conditionally replace a value in a ConcurrentHashMap. That is, given:
public class PriceTick {
final String instrumentId;
...
final long timestamp;
...
And a class (let's call it TickHolder) which owns a ConcurrentHashMap (let's just call it map).
I wish to be able to implement a conditional put method, so that if there's no entry for the key, the new one is inserted, but if there is an existing entry, the new one is inserted only if the timestamp value in the new PriceTick is greater than the existing one.
For an old-school HashMap solution, TickHolder would have a put method:
public void add(PriceTick tick) {
synchronized(map) {
if ((map.get(tick.instrumentId) == null)
|| (tick.getTimestamp() > map.get(tick.instrumentId).getTimestamp()) )
map.put(tick.instrumentId, tick);
}
}
With a ConcurrentHashMap, one would want to drop the synchronization and use some atomic method like replace, but that's unconditional. So clearly the "conditional replace" method must be written.
However, since the test-and-replace operation is non-atomic, in order to be thread safe, it would have to be synchronized - but my initial reading of the ConcurrentHashMap source leads me to think that external synchronization and their internal locks will not work very well, so at a very minimum, every Map method which performs structural changes and the containing class performs would have to be synchronized by the containing class... and even then, I'm going to be fairly uneasy.
I thought about subclassing ConcurrentHashMap, but that seems to be impossible. It makes use of an inner final class HashEntry with default access, so although ConcurrentHashMap is not final, it's not extensible.
Which seems to mean that I have to fall back to implementing TickHolder as containing an old-school HashMap in order to write my conditional replace method.
So, the questions: am I right about the above? Have I (hopefully) missed something, whether obvious or subtle, which would lead to a different conclusion? I'd really like to be able to make use of that lovely striped locking mechanism here.
The non-deterministic solution is to loop replace():
do {
PriceTick oldTick = map.get(newTick.getInstrumentId());
} while ((oldTick == null || oldTick.before(newTick)) && !map.replace(newTick.getInstrumentId(), oldTick, newTick);
Odd though it may seem, that is a commonly suggested pattern for this kind of thing.
#cletus solution formed the base for my solution to an almost identical problem. I think a couple of changes are needed though as if oldTick is null then replace throws a NullPointerException as stated by #hotzen
PriceTick oldTick;
do {
oldTick = map.putIfAbsent(newTick.getInstrumentId());
} while (oldTick != null && oldTick.before(newTick) && !map.replace(newTick.getInstrumentId(), oldTick, newTick);
The correct answer should be
PriceTick oldTick;
do {
oldTick = map.putIfAbsent(newTick.getInstrumentId(), newTick);
if (oldTick == null) {
break;
}
} while (oldTick.before(newTick) && !map.replace(newTick.getInstrumentId(), oldTick, newTick);
As an alternative, could you create a TickHolder class, and use that as the value in your map? It makes the map slightly more cumbersome to use (getting a value is now map.getValue(key).getTick()), but it lets you keep the ConcurrentHashMap's behavior.
public class TickHolder {
public PriceTick getTick() { /* returns current value */
public synchronized PriceTick replaceIfNewer (PriceTick pCandidate) { /* does your check */ }
}
And your put method becomes something like:
public void updateTick (PriceTick pTick) {
TickHolder value = map.getValue(pTick.getInstrumentId());
if (value != null) {
TickHolder newTick = new TickHolder(pTick);
value = map.putIfAbsent(pTick.getInstrumentId(), newTick);
if (value == null) {
value = newTick;
}
}
value.replaceIfNewer(pTick);
}
Related
Do we get any benefit to change existing code from:
class MyClass {
volatile Object myVariable;
Object myMethod() {
if (myVariable == null) {
synchronized(this) {
if (myVariable == null) {
myVariable = this.getNewValue();
}
}
}
return myVariable;
}
}
to
class MyClass {
volatile Object myVariable;
Object myMethod() {
Object tmp = this.myVariable;
if (tmp == null) {
synchronized(this) {
tmp = this.myVariable;
if (tmp == null) {
this.myVariable = tmp = this.getNewValue();
}
}
}
return tmp;
}
}
I don't see the point of copying this.myVariable locally before using it, and I don't think it's a good practice to use "this." for every class variable.
Copying to a local variable is more efficient and more correct.
More Efficient: Assume that in the common case, myVariable is non-null. In the first version, you do two reads of myVariable, once to check for null and once to return the value. In the second version, you do one read of myVariable and two reads of tmp (local variable access, which is trivial). The whole point of using volatile is the strong memory guarantees, and those guarantees mean that two reads is a non-trivial performance hit over just one read.
More Correct: Assume that myVariable is some sort of "cache" which periodically needs to be refreshed. I.e. there is a background thread which periodically sets myVariable to null so that it will be reloaded on the next read. In the first version, you do two separate reads of myVariable. The first read could return non-null, then the "cache refresh" logic runs and sets myVariable to null. The second read (to return the value) then returns null! In the second version, tmp is always the value you tested for null (assuming that getNewValue() never returns null of course).
Note that using "this." is a stylistic choice in this code and has no bearing on the correctness or performance issues.
This is largely just expanding on what is stated in https://en.wikipedia.org/wiki/Double-checked_locking#Usage_in_Java .
So I have a method (of which I can't change the parameters, or else this could have been made easier through HashMaps...more on this later), that passes an item as a parameter. Now I have a list of instances from this other class, of which one of its attributes is of the same type of this item, and I want to find the instances in the list which correspond to this item (of which there should be only one). This is what I did to find this:
List<Instance> instances = ...
public static void checkItems(Item i) {
List<Instance> n = new ArrayList<>();
instances.forEach(p -> n.add(p.i == i ? p : null));
Instance currentInstance = n.get(0);
//Instance currentInstance = instances.stream().filter(p -> p.i == i).collect(Collectors.toList()).get(0);
}
You'll probably notice two things straight up:
I used a conditional operator which adds a null value to the list when the condition isn't passed
My commented code which was another attempt to solve this issue
So in the first case, I put null because it requires you to put something, and a null value is probably easier to work with, which is why the question arises: How do I access the first non-null value in a list (without resorting to iterating over the entire list to find it...)?
You may notice that I just assign the first value of the list with n.get(0) to currentInstance, because I know that only one value will have passed the test. However, due to some other code that I apply to currentInstance, this value cannot be null.
Just a note on the second point: the way I tried to solve it with streams actually works exactly as planned, except that for some reason the list of instances recovered is not a direct copy of the original instances. This resulted in the values of some of the attributed to have been reset to default values, therefore rendering this method useless.
EDIT: I just wanted to mention that the streams method wasn't working because of some silly mistake that I made in another class, there was nothing wrong with the code so I'll be using that bit to solve my problem :D
If you know that only one p passes the test, I don't know what the point of creating a list with a load of null values plus p is.
Your problem seems to stem from wanting to use forEach. In my opinion, you should almost always use a for loop in preference to forEach. With a simple for loop you can just use break when the item is found.
In detail:
Instance p = null;
for (Instance q : instances) {
if (q.i == i) {
p = q;
break;
}
}
if (p == null)
throw new IllegalStateException(); // It wasn't there.
// Do something with p.
You could do it this way:
Instance currentInstance = instances.stream()
.filter(p -> p.i == i)
.findFirst()
.get(); // you can use get if you are sure there's one instance
The predicate p -> p.i == i seems suspicious. Why not using equals() instead?
As described, this would normally be solved with streams in the following way:
Optional<Instance> first =
instances.stream().filter(p -> p.i == i).findFirst();
(of which there should be only one)
Of which there definitely is only one, or of which there might be more than one. (And if there's more than one, then what? Is that an error?) It sounds like it might be that you should have a Set<Instance>, not a List<Instance>. Just an observation.
You can do it like
instances.forEach(p -> {
if (p.i == i) n.add(p);
});
I would like to use ConcurrentHashMap to let one thread delete some items from the map periodically and other threads to put and get items from the map at the same time.
I'm using map.entrySet().removeIf(lambda) in the removing thread. I'm wondering what assumptions I can make about its behavior. I can see that removeIf method uses iterator to go through elements in the map, check the given condition and then remove them if needed using iterator.remove().
Documentation gives some info about ConcurrentHashMap iterators behavior:
Similarly, Iterators, Spliterators and Enumerations return elements
reflecting the state of the hash table at some point at or since the
creation of the iterator/enumeration. hey do not throw ConcurrentModificationException. However, iterators are designed to be used by only one thread at a time.
As the whole removeIf call happens in one thread I can be sure that the iterator is not used by more than one thread at the time. Still I'm wondering if the course of events described below is possible:
Map contains mapping: 'A'->0
Deleting Thread starts executing map.entrySet().removeIf(entry->entry.getValue()==0)
Deleting Thread calls .iteratator() inside removeIf call and gets the iterator reflecting the current state of the collection
Another thread executes map.put('A', 1)
Deleting thread still sees 'A'->0 mapping (iterator reflects the old state) and because 0==0 is true it decides to remove A key from the map.
The map now contains 'A'->1 but deleting thread saw the old value of 0 and the 'A' ->1 entry is removed even though it shouldn't be. The map is empty.
I can imagine that the behavior may be prevented by the implementation in many ways. For example: maybe iterators are not reflecting put/remove operations but are always reflecting value updates or maybe the remove method of the iterator checks if the whole mapping (both key and value) is still present in the map before calling remove on the key. I couldn't find info about any of those things happening and I'm wondering if there's something which makes that use case safe.
I also managed to reproduce such case on my machine.
I think, the problem is that EntrySetView (which is returned by ConcurrentHashMap.entrySet()) inherits its removeIf implementation from Collection, and it looks like:
default boolean removeIf(Predicate<? super E> filter) {
Objects.requireNonNull(filter);
boolean removed = false;
final Iterator<E> each = iterator();
while (each.hasNext()) {
// `test` returns `true` for some entry
if (filter.test(each.next())) {
// entry has been just changed, `test` would return `false` now
each.remove(); // ...but we still remove
removed = true;
}
}
return removed;
}
In my humble opinion, this cannot be considered as a correct implementation for ConcurrentHashMap.
After discussion with user Zielu in comments below Zielu's answer I have gone deeper into the ConcurrentHashMap code and found out that:
ConcurrentHashMap implementation provides remove(key, value) method which calls replaceNode(key, null, value)
replaceNode checks if both key and value are still present in the map before removing so using it should be fine. Documentation says that it
Replaces node value with v, conditional upon match of cv if
* non-null.
In the case mentioned in the question ConcurrentHashMap's .entrySet() is called which returns EntrySetView class. Then removeIf method calls .iterator() which returns EntryIterator.
EntryIterator extends BaseIterator and inherits remove implementation that calls map.replaceNode(p.key, null, null) which disables conditional removal and just always removes the key.
The negative course of events could be still prevented if iterators always iterated over 'current' values and never returned old ones if some value is modified. I still don't know if that happens or not, but the test case mentioned below seems to verify the whole thing.
I think that have created a test case which shows that the behavior described in my question can really happen. Please correct me if I there are any mistakes in the code.
The code starts two threads. One of them (DELETING_THREAD) removes all entries mapped to 'false' boolean value. Another one (ADDING_THREAD) randomly puts (1, true) or (1,false) values into the map. If it puts true in the value it expects that the entry will still be there when checked and throws an exception if it is not. It throws an exception quickly when I run it locally.
package test;
import java.util.Random;
import java.util.concurrent.ConcurrentHashMap;
public class MainClass {
private static final Random RANDOM = new Random();
private static final ConcurrentHashMap<Integer, Boolean> MAP = new ConcurrentHashMap<Integer, Boolean>();
private static final Integer KEY = 1;
private static final Thread DELETING_THREAD = new Thread() {
#Override
public void run() {
while (true) {
MAP.entrySet().removeIf(entry -> entry.getValue() == false);
}
}
};
private static final Thread ADDING_THREAD = new Thread() {
#Override
public void run() {
while (true) {
boolean val = RANDOM.nextBoolean();
MAP.put(KEY, val);
if (val == true && !MAP.containsKey(KEY)) {
throw new RuntimeException("TRUE value was removed");
}
}
}
};
public static void main(String[] args) throws InterruptedException {
DELETING_THREAD.setDaemon(true);
ADDING_THREAD.start();
DELETING_THREAD.start();
ADDING_THREAD.join();
}
}
I have created a method that uses an iterator that iterates through a map and for each pair it evaluates a statement with many OR conditions. If the condition is true, it adds the object of the pair (a Notification object) in a list (anomalies). However, at compilation time, the compiler gives a NullPointerException exception at this method. Based on my investigation, it seems that there is a problem in the if statement, but I can't see why. Can anyone give me an help in this? Thanks!
public List<Notification> getAnomalies(NotificationSearchCriteria notificationSearchCriteria) {
Map<String,Notification> messageList = new HashMap<String,Notification>();
List<Notification> anomalies = new ArrayList<Notification>();
Iterator iterator = messageList.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry pairs = (Map.Entry)iterator.next();
Notification message = (Notification) pairs.getValue();
if(message.getDescription().equals(notificationSearchCriteria.getDescription())||message.getSubjectName().equals(notificationSearchCriteria.getSubjectName())||message.getNotificationSubject().toString().equals(notificationSearchCriteria.getNotificationSubject().toString())||message.getNotificationType().toString().equals(notificationSearchCriteria.getNotificationType().toString())){
anomalies.add(message);
}
}
}
return anomalies;
}
This is most likely caused by one of the methods on message returning null. For example, if message.getDescription() returns null, then message.getDescription().equals(<something>) will throw a NullPointerException, since you can't call additional methods on a null object.
There are several ways to fix this. First off, I recommend inspecting your objects to see which can return a null value and add the appropriate handling code.
More generally, I always recommend calling equals on the variable you know not to be null to avoid these problems. For example
if ("accept".equals(command)) {
// do something
}
is generally better than
if (command.equals("accept")) {
// do something
}
because the second might through an NPE, while the first never will.
I would refactor the message-matching code into the NotificationSearchCriteria class. The if would end up being "if (notificationSearchCriteria.matches(message))". From the names, I am guessing that is NotificationSearchCriteria's only usage; in that sense, it would not increase coupling.
The check-for-null would be performed during NotificationSearchCriteria construction; which would ensure that all fields were non-null. In the matching code, within that class, things would look like:
boolean matches(Notification message) {
if (description.equals(message.getDescription()) || // LHS guaranteed non-null
foo.equals(message.getFoo()) ||
bar.equals(message.getBar()) || // ...
) { return true; }
}
The best way to code is to do null check.
Ideally I would have code like this :
while (iterator.hasNext()) {
Map.Entry pairs = (Map.Entry)iterator.next();
Notification message = (Notification) pairs.getValue();
if(null!=message && null!=message.getDescription() &&
null!=notificationSearchCriteria.getDescription() )
{
//Do your comparioson
}else{
//Handle the NullPointerException error the way you want
}
}
I was told it is not a good style to call potentially costly methods for boolean expressions (getSupercategories()).
private final SuperCategoriesResolver<ProductModel> catResolver = new SuperCategoriesResolver<ProductModel>() {
#Override
public Set<CategoryModel> getSuperCategories(final CategoryModel item) {
return item == null || item.getSupercategories() == null ? Collections.EMPTY_SET
: new LinkedHashSet<CategoryModel>(
item.getSupercategories());
}
};
As well that getSupercategories() is potentially dangerous since it's backed by a relation attribute which might not be coming from local data members (item is sent as a parameter to a public method in this class and after wards is sent to getSuperCategories() which is overriden in the same class when declaring catResolver).
Is this a better approach to tackle the argument above?
private final SuperCategoriesResolver<ProductModel> catResolver = new SuperCategoriesResolver<ProductModel>() {
#Override
public Set<CategoryModel> getSuperCategories(final ProductModel item) {
if (item != null) {
Set<CategoryModel> superCategories = (Set<CategoryModel>) item
.getSupercategories();
if (superCategories != null)
return superCategories;
}
return Collections.EMPTY_SET;
}
};
Where I first verify that item is not null. if it is, then a return empy_set if not then I called the costly method and get the collection and just if it is not null return the collection with elements.
Thank u very much for your advice.
It is likely to get more efficient to call getSupercategories() once instead of twice if it does any computation.
Do you need to return a copy of this set? You do in the first example but not the second.
Second approach is indeed faster because there is only one call to the getSupercategories method if item is not null. However, in your second approach, you no longer create a LinkedHashSet instance -- which means it will behave differently (though faster).
This sounds more like performance optimization as opposed to refactoring. Usually when you refactor something, there is a "factor" in there somwhere, that trims the code down by eliminating redundancies.
Nulls are your problem. Can you make a refactoring to push nulls away?
For example, you could refactor your code to make item.getSuperCategories never return null? Or do you need to distinguish between the empty set and null?
Similarly, why are you passing null into this method? If you can eliminate that scenario then the code just becomes a one liner.