I'm using Spring with Redis and I am working with a list inside a hash. Everything works great on a single thread, problems come when I have to update list value with more than one instances.
Here is my code to put and get value from Hash:
public void put(String hashName, int key , List<myObj> myList) {
redis.opsForHash().put(hashName, String.valueOf(key), myList);
}
public List<myObj> get(String hashName, string key) {
Object map = redis.opsForHash().get(hashName,key);
if (map==null) {
log.info("no keys found");
return new ArrayList<myObj>();
}
List<myObj> myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
return myList;
}
What I do to perform update is:
List<myObj> myList= hash.get(hashName,key);
myList.add(obj);
hash.put(hashName, key, myList);
When there is more than one instance I occur in race condition. Is there a way to update list values in an atomic way?
Your current implementation is not good, because in the put() you update the whole list. In case many threads want to add a single element to the list they first obtain the current list, then add an element, then put new list. Each thread will override the result of the previous one, the last one wins. Usage of synchronized doesn't matter here.
Solution
Don't replace the whole list. Instead, add a single element to the list. Remove the method put() and add a new one like following:
public synchronized void add(String hashName, int key, myObj element) {
List<myObj> myList;
Object map = redis.opsForHash().get(hashName,key);
if (map != null) {
myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
} else {
myList = new ArrayList<myObj>();
}
myList.add(element);
redis.opsForHash().put(hashName, String.valueOf(key), myList);
}
Besides make sure there are no attempts to modify the list directly and that the only way to add elements is to use your method add(). Use Collections.unmodifiableList():
public List<myObj> get(String hashName, string key) {
Object map = redis.opsForHash().get(hashName,key);
if (map==null) {
log.info("no keys found");
return new ArrayList<myObj>();
}
List<myObj> myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
return Collections.unmodifiableList(myList);
}
Related
Currently, I have a problem when I frequently iterating through a HashMap (1-time per second)
I add new element to the map on the main thread and iterate the map on the other thread
I return an ImmutableMap.copyOf() to iterate through it, and sometimes I add a new element to the map. But it throws me a ConcurrentModificationException
java.util.ConcurrentModificationException: null
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911) ~[?:1.8.0_261]
at java.util.ArrayList$Itr.next(ArrayList.java:861) ~[?:1.8.0_261]
My pseudo-code:
class Foo {
private final Map<String, List<User>> userMap = new HashMap<>();
public static void addUser(String houseId, User user) {
if(!userMap.containsKey(houseId)) {
userMap.put(houseId, new ArrayList<>());
}
List<User> list = userMap.get(houseId);
list.add(user);
}
public static Map<String, List<User>> getAll() {
return ImmutableMap.copyOf(userMap);
}
}
// iterating task
// this task run one per second
class Task extends TimerTask {
public void run() {
for (List<User> listUser : Foo.getAll().values()) {
for (User user : listUser) {
user.sayHello(); // print hello
}
}
}
}
I thought that use an ImmutableMap.copyOf() will prevent me from this problem. Because I read that you should use an immutable copy of your list/map when iterating through it from other thread.
I think I "solve" this by using CopyOnWriteArrayList. But I'm really curious why it throws me an error.
Thank you!!!
As you can see in the stack trace, the error occurs in the ArrayList, not the Map.
In method addUser, an element is added to the list via
list.add(user);
This operation might happen, while another thread loops over the elements with for (User user : listUser) causing the error.
When you exchange the ArrayList with a CopyOnWriteArrayList, the list.add(user) internally makes a copy and adds the new user into the copy. Therefore, a potentially running iteration is not affected since it loops over the old version of the array. So the running iteration will ignore the new value. Any new iterator will include the new value.
It's more of a Java question (although I'm using Firebase). I'm trying to understand what's best way to sort a list. I'm trying to create the following map:
Map<Character,List<QueryDocumentSnapshot>> docs;
In order to build this map, I iterate over the documents in my collection, get the first letter of each one and insert the document into the list where the key is the letter.
The code:
for (QueryDocumentSnapshot current_document : value) {
char letter = getFirstLetter(current_document.getString("type"));
List<QueryDocumentSnapshot> list = docs.get(letter);
if (list == null) {
list = new ArrayList<>();
list.add(current_document);
docs.put(letter, list);
} else {
list.add(current_document);
}
}
In order to make the docs map sorted, I use TreeMap:
docs = new TreeMap<>();
But how can I make the list of documents be sorted by the field type (String)?
EDIT Sorry for not mentioning it, value is of type QuerySnapshot (firebase). Also I'm using Android's Java API 16 (which means I don't have java 8).
If you can use TreeSet instead of ArrayList, it can look like this. More you can find here: https://stackoverflow.com/a/8725470/4228138
Map<Character,Set<QueryDocumentSnapshot>> docs;
...
Comparator<QueryDocumentSnapshot> comparator = new Comparator<QueryDocumentSnapshot>() {
#Override
public int compare(QueryDocumentSnapshot o1, QueryDocumentSnapshot o2) {
return o1.getString("type").compareTo(o2.getString("type"));
}
};
for (QueryDocumentSnapshot currentDocument : value) {
char letter = getFirstLetter(currentDocument.getString("type"));
Set<QueryDocumentSnapshot> documents = docs.get(letter);
if (documents == null) {
documents = new TreeSet<>(comparator);
documents.add(currentDocument);
docs.put(letter, documents);
} else {
documents.add(currentDocument);
}
}
Create a Comparator and apply it to each List in the map. Since you are not changing the actual List reference you can sort in place. Note that I presumed that "type" returned a String. This may need to be changed.
Also, TreeMap has nothing to do with sorting the values. That would only sort the keys to the map.
Comparator<QueryDocumentSnapshot> comp = new CompareByType();
for (List<QueryDocumentSnapshot> list : docs.values()) {
Collections.sort(list,comp);
}
Here is the Comparator class
class CompareByType implements Comparator<QueryDocumentSnapshot> {
public int compare(QueryDocumentSnapshot a, QueryDocumentSnapshot b) {
String a = QueryDocumentSnapshot.getString("type");
String b = QueryDocumentSnapshot.getString("type");
return a.compareTo(b);
}
}
And here is a slight mod to your map creation. You don't need the else clause because list will always have a valid list when it is time to add the document.
for (QueryDocumentSnapshot current_document : value) {
char letter = getFirstLetter(current_document.getString("type"));
List<QueryDocumentSnapshot> list = docs.get(letter);
if (list == null) {
list = new ArrayList<>();
docs.put(letter, list);
}
// At this point you will always have a list to access.
// you can add the current document here
list.add(current_document);
}
The simplest way is to sort all QueryDocumentSnapshots before they are classified by first key. If the collection is sorted before, then each particular list will be sorted as well.
Hence, add
Collections.sort(value, Comparator.comparing(q -> q.getString("type")));
before the code you posted (I assume value a List, for general collection change it to stream sorting).
You can
Let the QueryDocumentSnapshot implement Compareableable<QueryDocumentSnapshot>. So it will use your implementation to sort the entries.
Provide a Comparator to the TreeMap constructor: TreeMap(Comparator<? super K> comparator)
The comparator can look like this:
(o1, o2) -> o1.getString("type").compareTo(o2.getString("type"))
How can I put in a map of above type. I do not want to overwrite existing mapping.
So far my code is:
public class Store {
Map<String, List<String>> items;
public Store(){
items = new HashMap<String, List<String>>();
}
public boolean containsKey(String key) {
return items.containsKey(key);
}
public void put(String key, String item) {
List<String> myList = new ArrayList<>();
if (myList == null) {
myList = new ArrayList<String>();
items.put(key, item);
}
}
}
I stopped here because I received an error message stating "change the item type to list String from String". I can not figure out if I am doing something wrong.
Here, double wrong:
List<String> myList = new ArrayList<>();
if (myList == null) {
myList = new ArrayList<String>();
items.put(key, item);
}
myList will never be null, you just assigned a list to it! And item is just a single string, so you shouldn't use it as value for a map that expects lists of strings as value!
You go:
List<String> myList = items.get(key);
if (myList == null) {
myList = new ArrayList<String>();
}
myList.add(item);
items.put(key, myList);
instead.
Meaning: first you check if you already have a list for that key. If not, you create an empty one. Then you add your new item to the (potentially new) list. Before finally putting the list into the map (it could be already there, but then you just overwrite that information with "itself").
And if you want to know how the "pros" solve this problem, have a look at this questions and the answers I received upon asking it.
Or use Map.computeIfAbsent() that is designed for this requirement : add a new entry if not existing mapping for a specific key and getting the value for (the new one or the existing) :
items.computeIfAbsent(key, k -> new ArrayList<>())
.add(item);
I am using LinkedList data structure serverList to store the elements in it. As of now, it can also insert null in the LinkedList serverList which is not what I want. Is there any other data structure which I can use which will not add null element in the serverList list but maintain the insert ordering?
public List<String> getServerNames(ProcessData dataHolder) {
// some code
String localIP = getLocalIP(localPath, clientId);
String localAddress = getLocalAddress(localPath, clientId);
// some code
List<String> serverList = new LinkedList<String>();
serverList.add(localIP);
if (ppFlag) {
serverList.add(localAddress);
}
if (etrFlag) {
for (String remotePath : holderPath) {
String remoteIP = getRemoteIP(remotePath, clientId);
String remoteAddress = getRemoteAddress(remotePath, clientId);
serverList.add(remoteIP);
if (ppFlag) {
serverList.add(remoteAddress);
}
}
}
return serverList;
}
This method will return a List which I am iterating it in a for loop in normal way. I can have empty serverList if everything is null, instead of having four null values in my list. In my above code, getLocalIP, getLocalAddress, getRemoteIP and getRemoteAddress can return null and then it will add null element in the linked list. I know I can add a if check but then I need to add if check four time just before adding to Linked List. Is there any better data structure which I can use here?
One constraint I have is - This library is use under very heavy load so this code has to be fast since it will be called multiple times.
I am using LinkedList data structure serverList to store the elements in it.
That's most probably wrong, given that you're aiming at speed. An ArrayList is much faster unless you're using it as a Queue or alike.
I know I can add a if check but then I need to add if check four time just before adding to Linked List. Is there any better data structure which I can use here?
A collection silently ignoring nulls would be a bad idea. It may be useful sometimes and very surprising at other times. Moreover, it'd violate the List.add contract. So you won't find it in any serious library and you shouldn't implement it.
Just write a method
void <E> addIfNotNullTo(Collection<E> collection, E e) {
if (e != null) {
collection.add(e);
}
}
and use it. It won't make your code really shorter, but it'll make it clearer.
One constraint I have is - This library is use under very heavy load so this code has to be fast since it will be called multiple times.
Note that any IO is many orders of magnitude slower than simple list operations.
Use Apache Commons Collection:
ListUtils.predicatedList(new ArrayList(), PredicateUtils.notNullPredicate());
Adding null to this list throws IllegalArgumentException. Furthermore you can back it by any List implementation you like and if necessary you can add more Predicates to be checked.
Same exists for Collections in general.
There are data structures that do not allow null elements, such as ArrayDeque, but these will throw an exception rather than silently ignore a null element, so you'd have to check for null before insertion anyway.
If you're dead set against adding null checks before insertion, you could instead iterate over the list and remove null elements before you return it.
The simplest way would be to just override LinkedList#add() in your getServerNames() method.
List<String> serverList = new LinkedList<String>() {
public boolean add(String item) {
if (item != null) {
super.add(item);
return true;
} else
return false;
}
};
serverList.add(null);
serverList.add("NotNULL");
System.out.println(serverList.size()); // prints 1
If you then see yourself using this at several places, you can probably turn it into a class.
You can use a plain Java HashSet to store your paths. The null value may be added multiple times, but it will only ever appears once in the Set. You can remove null from the Set and then convert to an ArrayList before returning.
Set<String> serverSet = new HashSet<String>();
serverSet.add(localIP);
if (ppFlag) {
serverSet.add(localAddress);
}
if (etrFlag) {
for (String remotePath : holderPath) {
String remoteIP = getRemoteIP(remotePath, clientId);
String remoteAddress = getRemoteAddress(remotePath, clientId);
serverSet.add(remoteIP);
if (ppFlag) {
serverSet.add(remoteAddress);
}
}
}
serverSet.remove(null); // remove null from your set - no exception if null not present
List<String> serverList = new ArrayList<String>(serverSet);
return serverList;
Since you use Guava (it's tagged), I have this alternative if you have the luxury of being able to return a Collection instead of a List.
Why Collection ? Because List forces you to either return true or throw an exception. Collection allows you to return false if you didn't add anything to it.
class MyVeryOwnList<T> extends ForwardingCollection<T> { // Note: not ForwardingList
private final List<T> delegate = new LinkedList<>(); // Keep a linked list
#Override protected Collection<T> delegate() { return delegate; }
#Override public boolean add(T element) {
if (element == null) {
return false;
} else {
return delegate.add(element);
}
}
#Override public boolean addAll(Collection<? extends T> elements) {
return standardAddAll(elements);
}
}
I have an ArrayList:
ArrayList<Integer> example = new ArrayList<Integer>();
example.add(1);
example.add(1);
example.add(2);
example.add(3);
example.add(3);
So I want to make others three ArrayLists containing in which one the same values (where there is just one value the ArrayList would have just the one).
Is that possible?
Here's an approach to filtering out the elements, implemented: using a generic Map (in a generic class) to encapsulate the values.
The key is the object we want, and the value is determined as follows:
If the key never existed, we have a list with at most one element, which is the same as the key;
If the key has existed prior, we have a list with at least one element, which is the same as the key.
Here's how it's laid out. You instantiate it with the type of object you want to split.
public class UniqueSplitter<T> {
public Map<T, List<T>> filterOutElements(final List<?> theCandidateList) {
final Map<T, List<T>> candidateMap = new HashMap<>();
for(Object element : theCandidateList) {
if(candidateMap.containsKey(element)) {
candidateMap.get(element).add((T) element);
} else {
final List<T> elementList = new ArrayList<>();
elementList.add((T) element);
candidateMap.put((T)element, elementList);
}
}
return candidateMap;
}
}