I have two lists Map. I want to get different entries between them base on some fields of Object. I've created a method as below:
private List<Map<String,Object>> GetDifferentEntries(List<Map<String,Object>> setA, List<Map<String,Object>> setB) {
List<Map<String,Object>> tmp = new ArrayList<Map<String,Object>>(setA);
for(Map<String,Object> a : tmp){
for(Map<String,Object> b: setB){
if(a.get("ID").equals(b.get("ID")) && a.get("ID1").equals(b.get("ID1")) ){
setA.remove(a);
}
} }return setA;}
Can I write above method again by using Stream class in java 8?
I have an example:
List1={ [ID=1, actor="A", ID1 = 1, film="F"],
[ID=2, actor="B", ID1 = 1, film="F"],
[ID=3, actor="C", ID1 = 1, film="F"]}
List2={ [ID=1, director="A", ID1 = 1, film="F"],
[ID=5, director="E", ID1 = 1, film="F"]}
Result = { [ID=2, actor="B", ID1 = 1, film="F"],
[ID=3, actor="C", ID1 = 1, film="F"] }
I'm not that familiar with Java 8 streams yet but your method could be optimized (as already stated in my comment) and thus be easier to be refactored to streams.
First you should not use nested loops but rather 2 separate loops. Additionally you might want to use a special key object that provides for ID and ID1 as well as appropriate implementations of hashCode() and equals(). Example:
class Key {
String id;
String id1;
int hashCode() { ... }
boolean equals(Object o) { ... }
}
Then build a LinkedHashMap for the first list (you could use a HashMap but that way you retain the element order):
Map<Key, Map<String, Object>> mapA = new LinkedHashMap<>();
for( for(Map<String,Object> a : setA){
mapA.put( new Key( a.get("ID"), a.get("ID1") ), a );
}
Note that this assumes your lists don't contain duplicates, if they do, you need to do it slightly differently (but I'll leave that to you).
Now that you've got a map for setA you can eliminate the elements that are contained in setB:
for(Map<String,Object> b: setB){
//if the key is not present nothing will be removed
mapA.remove( new Key( b.get("ID"), b.get("ID1") ) );
}
Finally built a list out of mapA:
List<Map<String,Object>> prunedSetA = new ArrayList<>( mapA.values() );
As you can see, you now have 2 loops and at least the second loop might make use of streams. You could do that for the first loop as well but you might lose the ordering. That's not a problem if order isn't important or you re-sort the list at the end.
First of all you should be careful as the method has a side effect on the given parameters. You should NOT modify input parameters (here setA).
Your copy is not deep enough. You created a temporary list. That is ok. But you do not copy the entries as this will cause the side effect of modified input parameters as they come by reference and not by value. The caller cannot trust the input parameters to be the same after he passes them into your method. If you want to avoid the side effect you have also to copy the Maps. Beside that you should only remove elements from the copy.
Next your are only handling the case for the elements that are in SetA. Elements in SetB that are not in SetA ar not recognized. Is this correct?
private List<Map<String, Object>> getDifferentEntries(List<Map<String, Object>> setA, List<Map<String, Object>> setB) {
List<Map<String, Object>> result = makeDeepCopyOf(setA);
setA.stream().forEach((entryA) -> {
setB.stream().forEach((entryB) -> {
if (entryA.get("ID").equals(entryB.get("ID")) && entryA.get("ID1").equals(entryB.get("ID1"))) {
result.remove(entryA);
}
});
});
return result;
}
To go with parallelism you can use parallel streams. It will load your CPUs without caring about thread handling:
setA.stream().parallel().forEach(...
setB.stream().parallel().forEach(...
Then you have to extract the statement that removes an element to a synchronized method as this method will be called asynchronously.
private synchronized removeElementFrom(List<Map<String, Object>> list, Map<String, Object> entry) {
list.remove(entry);
}
OR you consider to wrap the copy of setA into a suitable synchronized datastructure:
List<Map<String, Object>> result = Collections.synchronizedList(makeDeepCopyOf(setA));
Related
I have an object Foo with the following elements:
class Foo {
int id;
int departmentId;
boolean condition1;
boolean condition2;
boolean condition3;
//...
}
and a list of Foo objects (~10k entries):
List<Foo> fooList = new ArrayList<>();
fooList.add(...);
//...
I need to iterate through each of the departmentIds of this list, and be able to stop any further iterations of a particular departmentId once its objects meet a particular combination of conditions.
For this purpose, I was thinking to simply create a new Map which holds my departmentId as a key and all related Foo objects as its value. So that I could iterate through my new objects based on the departmentId, and easily stop the iteration for other departments with same Id once the condition is met. Something like:
Map<Foo.departmentId, List<Foo>> departmentFoos = new HashMap<>();
Can this be achieved in a better way other than iterating through my fooList and putting/replacing the object of my HashMap one by one?
So in terms of number of iterations, it's unlikely that converting to a Map would give you any benefit, you're better off just going through the list and processing in place. This is required because there's no way to know if you've reached the last appearance of a specific departmentId until you've gone through the entire list of Foos.
So I would do something like:
for (Foo foo : fooList) {
if (hasBeenProcessed(foo.departmentId) {
continue;
}
process(foo);
}
Note that hasBeenProcessed could be as simple as processedDepartmentIds.contains(foo.departmentId) depending on your needs.
For just converting it to a map, there's nothing that can avoid going through the whole list. There are convenience methods for this in libraries like Guava: Maps.toMap or Guava: Multimaps.index.
Using Streams, It can be done this way:
Map<Integer, List<Foo>> output = fooList.stream()
.collect(Collectors.groupingBy(Foo::getDepartmentId, Collectors.toList()));
I'm currently trying to create a method that determine if an ArrayList(a2) contains an ArrayList(a1), given that both lists contain duplicate values (containsAll wouldn't work as if an ArrayList contains duplicate values, then it would return true regardless of the quantity of the values)
This is what I have: (I believe it would work however I cannot use .remove within the for loop)
public boolean isSubset(ArrayList<Integer> a1, ArrayList<Integer> a2) {
Integer a1Size= a1.size();
for (Integer integer2:a2){
for (Integer integer1: a1){
if (integer1==integer2){
a1.remove(integer1);
a2.remove(integer2);
if (a1Size==0){
return true;
}
}
}
}
return false;
}
Thanks for the help.
Updated
I think the clearest statement of your question is in one of your comments:
Yes, the example " Example: [dog,cat,cat,bird] is a match for
containing [cat,dog] is false but containing [cat,cat,dog] is true?"
is exactly what I am trying to achieve.
So really, you are not looking for a "subset", because these are not sets. They can contain duplicate elements. What you are really saying is you want to see whether a1 contains all the elements of a2, in the same amounts.
One way to get to that is to count all the elements in both lists. We can get such a count using this method:
private Map<Integer, Integer> getCounter (List<Integer> list) {
Map<Integer, Integer> counter = new HashMap<>();
for (Integer item : list) {
counter.put (item, counter.containsKey(item) ? counter.get(item) + 1 : 1);
}
return counter;
}
We'll rename your method to be called containsAllWithCounts(), and it will use getCounter() as a helper. Your method will also accept List objects as its parameters, rather than ArrayList objects: it's a good practice to specify parameters as interfaces rather than implementations, so you are not tied to using ArrayList types.
With that in mind, we simply scan the counts of the items in a2 and see that they are the same in a1:
public boolean containsAllWithCounts(List<Integer> a1, List<Integer> a2) {
Map<Integer,Integer> counterA1 = getCounter(a1);
Map<Integer,Integer> counterA2 = getCounter(a2);
boolean containsAll = true;
for (Map.Entry<Integer, Integer> entry : counterA2.entrySet ()) {
Integer key = entry.getKey();
Integer count = entry.getValue();
containsAll &= counterA1.containsKey(key) && counterA1.get(key).equals(count);
if (!containsAll) break;
}
return containsAll;
}
If you like, I can rewrite this code to handle arbitrary types, not just Integer objects, using Java generics. Also, all the code can be shortened using Java 8 streams (which I originally used - see comments below). Just let me know in comments.
if you want remove elements from list you have 2 choices:
iterate over copy
use concurrent list implementation
see also:
http://docs.oracle.com/javase/8/docs/api/java/util/Collections.html#synchronizedList-java.util.List-
btw why you don't override contains method ??
here you use simple Object like "Integer" what about when you will be using List< SomeComplexClass > ??
example remove with iterator over copy:
List<Integer> list1 = new ArrayList<Integer>();
List<Integer> list2 = new ArrayList<Integer>();
List<Integer> listCopy = new ArrayList<>(list1);
Iterator<Integer> iterator1 = listCopy.iterator();
while(iterator1.hasNext()) {
Integer next1 = iterator1.next();
Iterator<Integer> iterator2 = list2.iterator();
while (iterator2.hasNext()) {
Integer next2 = iterator2.next();
if(next1.equals(next2)) list1.remove(next1);
}
}
see also this answer about iterator:
Concurrent Modification exception
also don't use == operator to compare objects :) instead use equal method
about use of removeAll() and other similarly methods:
keep in mind that many classes that implements list interface don't override all methods from list interface - so you can end up with unsupported operation exception - thus I prefer "low level" binary/linear/mixed search in this case.
and for comparison of complex classes objects you will need override equal and hashCode methods
f you want to remove the duplicate values, simply put the arraylist(s) into a HashSet. It will remove the duplicates based on equals() of your object.
- Olga
In Java, HashMap works by using hashCode to locate a bucket. Each bucket is a list of items residing in that bucket. The items are scanned, using equals for comparison. When adding items, the HashMap is resized once a certain load percentage is reached.
So, sometimes it will have to compare against a few items, but generally it's much closer to O(1) than O(n).
in short - there is no need to use more resources (memory) and "harness" unnecessary classes - as hash map "get" method gets very expensive as count of item grows.
hashCode -> put to bucket [if many item in bucket] -> get = linear scan
so what counts in removing items ?
complexity of equals and hasCode and used of proper algorithm to iterate
I know this is maybe amature-ish, but...
There is no need to remove the items from both lists, so, just take it from the one list
public boolean isSubset(ArrayList<Integer> a1, ArrayList<Integer> a2) {
for(Integer a1Int : a1){
for (int i = 0; i<a2.size();i++) {
if (a2.get(i).equals(a1Int)) {
a2.remove(i);
break;
}
}
if (a2.size()== 0) {
return true;
}
}
return false;
}
If you want to remove the duplicate values, simply put the arraylist(s) into a HashSet. It will remove the duplicates based on equals() of your object.
In Java8 streams, am I allowed to modify/update objects within?
For eg. List<User> users:
users.stream().forEach(u -> u.setProperty("value"))
Yes, you can modify state of objects inside your stream, but most often you should avoid modifying state of source of stream. From non-interference section of stream package documentation we can read that:
For most data sources, preventing interference means ensuring that the data source is not modified at all during the execution of the stream pipeline. The notable exception to this are streams whose sources are concurrent collections, which are specifically designed to handle concurrent modification. Concurrent stream sources are those whose Spliterator reports the CONCURRENT characteristic.
So this is OK
List<User> users = getUsers();
users.stream().forEach(u -> u.setProperty(value));
// ^ ^^^^^^^^^^^^^
// \__/
but this in most cases is not
users.stream().forEach(u -> users.remove(u));
//^^^^^ ^^^^^^^^^^^^
// \_____________________/
and may throw ConcurrentModificationException or even other unexpected exceptions like NPE:
List<Integer> list = IntStream.range(0, 10).boxed().collect(Collectors.toList());
list.stream()
.filter(i -> i > 5)
.forEach(i -> list.remove(i)); //throws NullPointerException
The functional way would imho be:
import static java.util.stream.Collectors.toList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;
public class PredicateTestRun {
public static void main(String[] args) {
List<String> lines = Arrays.asList("a", "b", "c");
System.out.println(lines); // [a, b, c]
Predicate<? super String> predicate = value -> "b".equals(value);
lines = lines.stream().filter(predicate.negate()).collect(toList());
System.out.println(lines); // [a, c]
}
}
In this solution the original list is not modified, but should contain your expected result in a new list that is accessible under the same variable as the old one
To do structural modification on the source of the stream, as Pshemo mentioned in his answer, one solution is to create a new instance of a Collection like ArrayList with the items inside your primary list; iterate over the new list, and do the operations on the primary list.
new ArrayList<>(users).stream().forEach(u -> users.remove(u));
You can make use of the removeIf to remove data from a list conditionally.
Eg:- If you want to remove all even numbers from a list, you can do it as follows.
final List<Integer> list = IntStream.range(1,100).boxed().collect(Collectors.toList());
list.removeIf(number -> number % 2 == 0);
To get rid from ConcurrentModificationException Use CopyOnWriteArrayList
Instead of creating strange things, you can just filter() and then map() your result.
This is much more readable and sure. Streams will make it in only one loop.
As it was mentioned before - you can't modify original list, but you can stream, modify and collect items into new list. Here is simple example how to modify string element.
public class StreamTest {
#Test
public void replaceInsideStream() {
List<String> list = Arrays.asList("test1", "test2_attr", "test3");
List<String> output = list.stream().map(value -> value.replace("_attr", "")).collect(Collectors.toList());
System.out.println("Output: " + output); // Output: [test1, test2, test3]
}
}
.peek() is the answer.
users.stream().peek(u -> u.setProperty("value")).foreach(i->{
...
...
});
for new list
users.stream().peek(u -> u.setProperty("value")).collect(Collectors.toList());
This might be a little late. But here is one of the usage. This to get the count of the number of files.
Create a pointer to memory (a new obj in this case) and have the property of the object modified. Java 8 stream doesn't allow to modify the pointer itself and hence if you declare just count as a variable and try to increment within the stream it will never work and throw a compiler exception in the first place
Path path = Paths.get("/Users/XXXX/static/test.txt");
Count c = new Count();
c.setCount(0);
Files.lines(path).forEach(item -> {
c.setCount(c.getCount()+1);
System.out.println(item);});
System.out.println("line count,"+c);
public static class Count{
private int count;
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
#Override
public String toString() {
return "Count [count=" + count + "]";
}
}
Yes, you can modify or update the values of objects in the list in your case likewise:
users.stream().forEach(u -> u.setProperty("some_value"))
However, the above statement will make updates on the source objects. Which may not be acceptable in most cases.
Luckily, we do have another way like:
List<Users> updatedUsers = users.stream().map(u -> u.setProperty("some_value")).collect(Collectors.toList());
Which returns an updated list back, without hampering the old one.
I am building a couple of methods which are supposed to create a cache of input strings, load them in to a list, and then determine the number of occurrences of each string in that list, ranking them in order of the most common elements.
The string, or elements themselves are coming from a JUnit test. It's calling up a method called
lookupDistance(dest)
where "dest" is a String (destination airport code), and the lookupDistance returns the distance between two airport codes....
There's the background. The problem is that I want to load all of the "dest" strings in to a cache. What's the best way to do that?
I have skeleton code that has a method called:
public List<String> mostCommonDestinations()
How would I add "dest" strings to the List in a transparent way? The JUnit test case is only calling lookupDistance(dest), so how can I also redirect those "dest" strings to the List in this method?
How would I then quantify the number of occurrences of each element and say, rank the top three or four?
Have a Map<String, Integer> destinations = new HashMap<>();
In lookupDistance(dest), do something like this (untested pseudocode):
Integer count = destinations.get(dest);
if (count == null) {
destinations.put(dest, Integer.valueOf(1));
} else {
count = Integer.valueOf(count.intValue() + 1);
}
This way, you count the occurences of each dest.
Go through the Map and find the highest counts. That's a bit tricky. One approach might be:
List> list = new ArrayList<>();
list.addAll(destinations.entrySet());
// now you have a list of "entries", each of which maps from dest to its respective counter
// that list now has to be sorted
Collections.sort(list, comparator);
The comparator we used in this invocation has still to be written. It has to take two arguments, which are elements of the list, and compare them according to their counter value. the sort routine will do the rest.
Comparator<Map.Entry<String, Integer>> comparator = new Comparator<>() {
public #Override int compare(Map.Entry<String, Integer> a, Map.Entry<String, Integer> b) {
return a.getValue().intValue() - b.getValue().intValue();
}
}
Ok, so we have a sorted List of Entrys now from which you can pick the top 5 or so. Think that's pretty much it. All this looks more complicated than it should be, so I'm curios for other solutions.
You can add known destination at startup and keep adding new strings to cache as they arrive. That's one way. The other way is to cache strings as they are requested, keeping them for future request. In that case your lookupDistance should also cache string.
Start by making a small class that contains a Hashmap. The key would be your destination string, and the value can either be an object if you want to keep multiple information or just a number specifying how many times that string is used. I would recommend using a data object.
Please note that code below is just to you an idea, more like a pseudo-code.
class Cache {
private Hashmap<String, CacheObject>;
public void Add(string, CacheObject);
public CacheObject Lookup(string);
public CacheObject Remove(string);
public static Cache getInstance(); //single cache
}
class CacheObject {
public int lookupCount;
public int lastUsed;
}
In your lookupDistance you can simply do
if(Cache.getInstance().Lookup(string) == null) {
Cache.getInstance().Add(string, new CacheObject() { 1, Date.now});
}
I have a List of object and the list is very big. The object is
class Sample {
String value1;
String value2;
String value3;
String value4;
String value5;
}
Now I have to search for a specific value of an object in the list. Say if value3=='three' I have to return those objects (My search is not always based on value3)
The list is
List<Sample> list = new ArrayList<Sample>();
What is the efficient way of doing it?
Thanks.
You can give a try to Apache Commons Collections.
There is a class CollectionUtils that allows you to select or filter items by custom Predicate.
Your code would be like this:
Predicate condition = new Predicate() {
boolean evaluate(Object sample) {
return ((Sample)sample).value3.equals("three");
}
};
List result = CollectionUtils.select( list, condition );
Update:
In java8, using Lambdas and StreamAPI this should be:
List<Sample> result = list.stream()
.filter(item -> item.value3.equals("three"))
.collect(Collectors.toList());
much nicer!
Using Java 8
With Java 8 you can simply convert your list to a stream allowing you to write:
import java.util.List;
import java.util.stream.Collectors;
List<Sample> list = new ArrayList<Sample>();
List<Sample> result = list.stream()
.filter(a -> Objects.equals(a.value3, "three"))
.collect(Collectors.toList());
Note that
a -> Objects.equals(a.value3, "three") is a lambda expression
result is a List with a Sample type
It's very fast, no cast at every iteration
If your filter logic gets heavier, you can do list.parallelStream() instead of list.stream() (read this)
Apache Commons
If you can't use Java 8, you can use Apache Commons library and write:
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.collections.Predicate;
Collection result = CollectionUtils.select(list, new Predicate() {
public boolean evaluate(Object a) {
return Objects.equals(((Sample) a).value3, "three");
}
});
// If you need the results as a typed array:
Sample[] resultTyped = (Sample[]) result.toArray(new Sample[result.size()]);
Note that:
There is a cast from Object to Sample at each iteration
If you need your results to be typed as Sample[], you need extra code (as shown in my sample)
Bonus: A nice blog article talking about how to find element in list.
If you always search based on value3, you could store the objects in a Map:
Map<String, List<Sample>> map = new HashMap <>();
You can then populate the map with key = value3 and value = list of Sample objects with that same value3 property.
You can then query the map:
List<Sample> allSamplesWhereValue3IsDog = map.get("Dog");
Note: if no 2 Sample instances can have the same value3, you can simply use a Map<String, Sample>.
I modifie this list and add a List to the samples try this
Pseudocode
Sample {
List<String> values;
List<String> getList() {
return values}
}
for(Sample s : list) {
if(s.getString.getList.contains("three") {
return s;
}
}
As your list is an ArrayList, it can be assumed that it is unsorted. Therefore, there is no way to search for your element that is faster than O(n).
If you can, you should think about changing your list into a Set (with HashSet as implementation) with a specific Comparator for your sample class.
Another possibility would be to use a HashMap. You can add your data as Sample (please start class names with an uppercase letter) and use the string you want to search for as key. Then you could simply use
Sample samp = myMap.get(myKey);
If there can be multiple samples per key, use Map<String, List<Sample>>, otherwise use Map<String, Sample>. If you use multiple keys, you will have to create multiple maps that hold the same dataset. As they all point to the same objects, space shouldn't be that much of a problem.
You can filter the list:
list.stream().filter(
sample -> sample.getValue4().equals("4")
).forEach(System.out::println)
I propose for+if.
Object result;
for (Object o: objects){
if (o.value3.equals("three")){
result=o;
break;
}
}
no streams, no guavas, I think it's simple.