We Keep Coding

Best practice for iterating over Collection with impure functions in Java

Suppose this is the use case:
I want to update a Hashmap cache inside my class.
I have a set of keys and some conditions I wanna apply to the keys and the values which are retrieved with the key.
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
public class App {
Set<String> keysToUpdate;
HashMap<String, List<String>> cache;
void buildCache(){
keysToUpdate.stream()
.filter(k -> true) // some filter
.forEach(
key ->{
// now values list comes from outside the stream pipeline
List<String> values = cache.computeIfAbsent(key, k -> new ArrayList<>());
getValuesforKey(key)
.stream()
.filter(k -> true) //another filter
// side effects are introduced
.forEach(value -> {
//some other operation, for example logging the values added like
// log.info("{} value added", value);
values.add(value);
}
);
}
);
}
private List<String> getValuesforKey(String key) {
//some method to get the values for the key
return new ArrayList<>();
}
}
We are told that shared mutability like this is bad because the execution is not deterministic, but in this specific case I am adding values to a hashmap and I don't care about the order of execution if I know keysToUpdate doesn't contain repeated values.
Are there other aspects I haven't considered? Is this code safe if streams are parallelised?
If not would using the collection's iterator fix the problem? (code below).
Or would it be best to use imperative programming instead? In what cases is shared mutability OK within the stream
public class App {
Set<String> keysToUpdate;
HashMap<String, List<String>> cache;
void buildCache(){
keysToUpdate.stream()
.filter(k -> true)// some filter
.collect(Collectors.toList()) // Collect before iterating
.forEach(
key ->{
// now values list comes from outside the stream pipeline
List<String> values = cache.computeIfAbsent(key, k -> new ArrayList<>());
getValuesforKey(key)
.stream()
.filter(k -> true) 、、another filter
.collect(Collectors.toList()) // Collect before iterating
// side effects are introduced
.forEach(value -> {
//some other operation, for example logging the values added like
// log.info("{} value added", value);
values.add(value);
}
);
}
);
}
private List<String> getValuesforKey(String key) {
//some method to get the values for the key
return new ArrayList<>();
}
}

When dealing with multithreading, the question to be answered is whether race conditions may happen (that is, if two or more different threads may access the same resource at the same time with at least one of them trying to modify it).
In your example, the computeIfAbsent method will modify your map if the requested key is not already there. So, two threads can potentially modify the same resource (the cache object). To avoid this, you can obtain (at the beginning of the buildCache method) a thread-safe version of your map by using Collections.synchronizedMap() and then operate on the returned map.
For what regards the values list, safety depends on whether two threads may operate on the same key, and thus modify the same list. In your examples, keys are a unique as being obtained from a set, so the code is safe.
Side note: the expected increase of performance depends on the amount of processing that the getValuesForKey method has to perform. If negligible, most threads will just be waiting for a lock on the map, thus making the performance increase also minimal.

How to use Stream to get all Lists from a nested List that match a specific condition?

How can I achieve the same logic in my code using only Streams, without the for loop as shown in my code below?
I have tried using flatMap, but I get stuck on the condition part, since allMatch() only returns a boolean.
How can I retrieve all the rows from the nested ArrayList that passes the condition without using for loop?
ArrayList<ArrayList<Tile>> completeRows = new ArrayList<>();
for (ArrayList<Tile> rows: getGridTiles()) {
if (rows.stream().allMatch(p -> p.getOccupiedBlockType() != BlockType.EMPTY)) {
completeRows.add(rows);
}
}

You can apply filter() with a nested stream (exactly the same as you've used as a condition in your code) passed to it as a Predicate to verify that a list consists of only non-empty tiles.
And then collect all the lists (rows) that have passed the predicate into a List using collect().
public static List<List<Tile>> getNonEmptyRows(List<List<Tile>> rows) {
return rows.stream()
.filter(row -> row.stream().allMatch(tile -> tile.getOccupiedBlockType() != BlockType.EMPTY))
.collect(Collectors.toList()); // or .toList() with Java 16+
}
I have tried using flatMap
You need to use flatMap when your goal is to flatten the steam of collections (or objects holding a reference to a collection) to a stream of elements of these collections. In these case, turn a stream of lists of tiles Stream<List<Tile>> into a stream of tiles Stream<Tile>.
Judging by your code, it's not you what you want because you're accumulating the rows (lists of tiles) into another list and not "flattening" them.
But just in case, that's how it can be done:
public static List<Tile> getNonEmptyTiles(List<List<Tile>> rows) {
return rows.stream()
.filter(row -> row.stream().allMatch(tile -> tile.getOccupiedBlockType() != BlockType.EMPTY))
.flatMap(List::stream)
.collect(Collectors.toList()); // or .toList() with Java 16+
}
Sidenote: leverage abstract data types - write your code against interfaces. What does it mean to "program to an interface"?

How to propagate variables in a stream in java 8

I would be currious to know how to propagate variable into a stream in java 8.
An example is better than a long explaination, so how would you convert the following (abstract) code into streams:
Map<Integer,A> myMap = new HashMap();
for (Entry<Integer,A> entry : myMap)
{
int param1=entry.getValue().getParam1();
List param2=entry.getValue().getParam2();
for (B b : param2)
{
System.out.println(""+entry.getKey()+"-"+param1+"-"+b.toString());
}
}
Knowing that this example is a simplification of the problem (for example, i need "param1" more than once in the next for loop)
So far, the only idea i have is to store all the informations i need into a tuple to finally use the forEach stream method over this tuple.
(Not sure to be very clear....)
Edit:I simplified my example too much. My case is more something like that:
Map<Integer,A> myMap = new HashMap();
for (Entry<Integer,A> entry : myMap)
{
int param1=entry.getValue().getParam1();
CustomList param2=entry.getValue().getParam2();
for (int i = 0; i<param2.size(); i++)
{
System.out.println(""+entry.getKey()+"-"+param1+"-"+param2.get(i).toString());
}
}
I could write something like that with stream:
myMap.entrySet().stream()
.forEach(
e -> IntStream.range(0, e.getValue.getParam2().getSize())
.forEach(
i -> System.out.println(e.getKey()+"-"+e.getValue().getParam1()+"-"+e.getValue.getParam2.get(i))
)
);
However, what i have instead of "e.getValue.getParam2()" in my real case is much more complex (a sequence of 5-6 methods) and heavier than just retrieving a variable (it executes some logic), so i would like to avoid to repeat e.getValue.getParam2 (once in just before the forEach, and once in the forEach)
i know that it's maybe not the best use case for using stream, but I am learning about it and would like to know about the limits
Thanks!

Something like this:
myMap.forEach(
(key, value) -> value.getParam2().forEach(
b -> System.out.println(key+"-"+value.getParam1()+"-"+b)
)
);
That is, for each key/value pair, iterate through value.getParam2(). For each one of those, print out string formatted as you specified. I'm not sure what that gets you, other than being basically what you had before, but using streams.
Update
Responding to updates to your question, this:
myMap.forEach((key, value) -> {
final CustomList param2 = value.getParam2();
IntStream.range(0, param2.getSize()).forEach(
i -> System.out.println(key+"-"+value.getParam1()+"-"+param2.get(i))
)
});
Here we assign the result of getParam2() to a final variable, so it is only calculated once. Final (and effectively final) variables are visible inside lambda functions.
(Thank you to Holger for the suggestions.)

Note that there are more features in the Java 8 API than just streams. Especially, if you just want to process all elements of a collection, you don’t need streams.
You can simplify every form of coll.stream().forEach(consumer) to coll.forEach(consumer). This applies to map.entrySet() as well, however, if you want to process all mappings of a Map, you can use forEach on the Map directly, providing a BiConsumer<KeyType,ValueType> rather than a Consumer<Map.Entry<KeyType,ValueType>>, which can greatly improve the readability:
myMap.forEach((key, value) -> {
int param1 = value.getParam1();
CustomList param2 = value.getParam2();
IntStream.range(0, param2.size()).mapToObj(param2::get)
.forEach(obj -> System.out.println(key+"-"+param1+"-"+obj));
});
It’s worth thinking about adding a forEach(Consumer<ElementType>) method to your CustomList, even if the CustomList doesn’t support the other standard collection operations…

anyMatch inside allMatch

I have a couple of predicates that I all want to be satisfied.
The things that can satisfy those predicates are a handful of strings. An individual string doesn't have to satisfy all (or any) of those predicates, but after I've looked at the last string, all of the predicates have to be satisified.
My first take to represent this problem in Java was to use Stream's allMatch and anyMatch since I want all of the predicates to match any of the things to test:
Stream<String> thingsToTest = Stream.of("Hi", "predicates!", "oddball");
Predicate<String> startsWithH = string -> string.startsWith("H");
Predicate<String> endsWithBang = string -> string.endsWith("!");
Stream<Predicate<String>> predicates = Stream.of(startsWithH, endsWithBang);
// All of the strings have the chance to satisfy any predicate
boolean predicatesSatisfied = predicates.allMatch(pred -> thingsToTest.anyMatch(pred::test));
// I expect this to print "true"
System.out.println(predicatesSatisfied);
Sadly, this doesn't work but terminates with an IllegalStateException, telling me that the stream has already been operated upon or closed, which shouldn't come as a big surprise since for each predicate I give the strings a new chance to satisfy the predicate, using the string stream over and over.
And streams are not meant to be reused for good reasons.
So how do I avoid this exception? Is there a more elegant alternative to anyMatch or allMatch?

To get around the IllegalStateException I use a List of strings and call its stream() method:
// Use List instead of Stream
List<String> thingsToTest = Arrays.asList("Hi", "predicates!", "oddball");
// Same old
Predicate<String> startsWithH = string -> string.startsWith("H");
Predicate<String> endsWithBang = string -> string.endsWith("!");
Stream<Predicate<String>> predicates = Stream.of(startsWithH, endsWithBang);
// Call stream() on the List
boolean predicatesSatisfied = predicates.allMatch(pred -> thingsToTest.stream().
anyMatch(pred::test));
Although this works fine, I'm not sure if it is the most elegant way to do this, so if you have a better idea, please go ahead and post your code or suggestion.

When you need to use the Stream several times, the common solution is to create a Supplier<Stream> instead:
Supplier<Stream<String>> thingsToTest = () -> Stream.of("Hi", "predicates!", "oddball");
....
boolean predicatesSatisfied = predicates.allMatch(
pred -> thingsToTest.get().anyMatch(pred::test));
Unlike #MatthiasBraun suggestion, using the Supplier it's not always necessary to actually store all the stream elements in the verbatim collection. For example, such thing is possible:
Supplier<Stream<String>> thingsToTest =
() -> IntStream.range(0, 10000).mapToObj(String::valueOf);
You just have to care that supplier always returns the same stream elements.
If you already have a collection, then you can create a supplier as well:
List<String> list = Arrays.asList("Hi", "predicates!", "oddball");
Supplier<Stream<String>> thingsToTest = list::stream;

How to create a List<T> from Map<K,V> and List<K> of keys?

Using Java 8 lambdas, what's the "best" way to effectively create a new List<T> given a List<K> of possible keys and a Map<K,V>? This is the scenario where you are given a List of possible Map keys and are expected to generate a List<T> where T is some type that is constructed based on some aspect of V, the map value types.
I've explored a few and don't feel comfortable claiming one way is better than another (with maybe one exception -- see code). I'll clarify "best" as a combination of code clarity and runtime efficiency. These are what I came up with. I'm sure someone can do better, which is one aspect of this question. I don't like the filter aspect of most as it means needing to create intermediate structures and multiple passes over the names List. Right now, I'm opting for Example 6 -- a plain 'ol loop. (NOTE: Some cryptic thoughts are in the code comments, especially "need to reference externally..." This means external from the lambda.)
public class Java8Mapping {
private final Map<String,Wongo> nameToWongoMap = new HashMap<>();
public Java8Mapping(){
List<String> names = Arrays.asList("abbey","normal","hans","delbrook");
List<String> types = Arrays.asList("crazy","boring","shocking","dead");
for(int i=0; i<names.size(); i++){
nameToWongoMap.put(names.get(i),new Wongo(names.get(i),types.get(i)));
}
}
public static void main(String[] args) {
System.out.println("in main");
Java8Mapping j = new Java8Mapping();
List<String> testNames = Arrays.asList("abbey", "froderick","igor");
System.out.println(j.getBongosExample1(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
System.out.println(j.getBongosExample2(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
System.out.println(j.getBongosExample3(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
System.out.println(j.getBongosExample4(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
System.out.println(j.getBongosExample5(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
System.out.println(j.getBongosExample6(testNames).stream().map(Bongo::toString).collect(Collectors.joining(", ")));
}
private static class Wongo{
String name;
String type;
public Wongo(String s, String t){name=s;type=t;}
#Override public String toString(){return "Wongo{name="+name+", type="+type+"}";}
}
private static class Bongo{
Wongo wongo;
public Bongo(Wongo w){wongo = w;}
#Override public String toString(){ return "Bongo{wongo="+wongo+"}";}
}
// 1: Create a list externally and add items inside 'forEach'.
// Needs to externally reference Map and List
public List<Bongo> getBongosExample1(List<String> names){
final List<Bongo> listOne = new ArrayList<>();
names.forEach(s -> {
Wongo w = nameToWongoMap.get(s);
if(w != null) {
listOne.add(new Bongo(nameToWongoMap.get(s)));
}
});
return listOne;
}
// 2: Use stream().map().collect()
// Needs to externally reference Map
public List<Bongo> getBongosExample2(List<String> names){
return names.stream()
.filter(s -> nameToWongoMap.get(s) != null)
.map(s -> new Bongo(nameToWongoMap.get(s)))
.collect(Collectors.toList());
}
// 3: Create custom Collector
// Needs to externally reference Map
public List<Bongo> getBongosExample3(List<String> names){
Function<List<Wongo>,List<Bongo>> finisher = list -> list.stream().map(Bongo::new).collect(Collectors.toList());
Collector<String,List<Wongo>,List<Bongo>> bongoCollector =
Collector.of(ArrayList::new,getAccumulator(),getCombiner(),finisher, Characteristics.UNORDERED);
return names.stream().collect(bongoCollector);
}
// example 3 helper code
private BiConsumer<List<Wongo>,String> getAccumulator(){
return (list,string) -> {
Wongo w = nameToWongoMap.get(string);
if(w != null){
list.add(w);
}
};
}
// example 3 helper code
private BinaryOperator<List<Wongo>> getCombiner(){
return (l1,l2) -> {
l1.addAll(l2);
return l1;
};
}
// 4: Use internal Bongo creation facility
public List<Bongo> getBongosExample4(List<String> names){
return names.stream().filter(s->nameToWongoMap.get(s) != null).map(s-> new Bongo(nameToWongoMap.get(s))).collect(Collectors.toList());
}
// 5: Stream the Map EntrySet. This avoids referring to anything outside of the stream,
// but bypasses the lookup benefit from Map.
public List<Bongo> getBongosExample5(List<String> names){
return nameToWongoMap.entrySet().stream().filter(e->names.contains(e.getKey())).map(e -> new Bongo(e.getValue())).collect(Collectors.toList());
}
// 6: Plain-ol-java loop
public List<Bongo> getBongosExample6(List<String> names){
List<Bongo> bongos = new ArrayList<>();
for(String s : names){
Wongo w = nameToWongoMap.get(s);
if(w != null){
bongos.add(new Bongo(w));
}
}
return bongos;
}
}

If namesToWongoMap is an instance variable, you can't really avoid a capturing lambda.
You can clean up the stream by splitting up the operations a little more:
return names.stream()
.map(n -> namesToWongoMap.get(n))
.filter(w -> w != null)
.map(w -> new Bongo(w))
.collect(toList());
return names.stream()
.map(namesToWongoMap::get)
.filter(Objects::nonNull)
.map(Bongo::new)
.collect(toList());
That way you don't call get twice.
This is very much like the for loop, except, for example, it could theoretically be parallelized if namesToWongoMap can't be mutated concurrently.
I don't like the filter aspect of most as it means needing to create intermediate structures and multiple passes over the names List.
There are no intermediate structures and there is only one pass over the List. A stream pipeline says "for each element...do this sequence of operations". Each element is visited once and the pipeline is applied.
Here are some relevant quotes from the java.util.stream package description:
A stream is not a data structure that stores elements; instead, it conveys elements from a source such as a data structure, an array, a generator function, or an I/O channel, through a pipeline of computational operations.
Processing streams lazily allows for significant efficiencies; in a pipeline such as the filter-map-sum example above, filtering, mapping, and summing can be fused into a single pass on the data, with minimal intermediate state.

Radiodef's answer pretty much nailed it, I think. The solution given there:
return names.stream()
.map(namesToWongoMap::get)
.filter(Objects::nonNull)
.map(Bongo::new)
.collect(toList());
is probably about the best that can be done in Java 8.
I did want to mention a small wrinkle in this, though. The Map.get call returns null if the name isn't present in the map, and this is subsequently filtered out. There's nothing wrong with this per se, though it does bake null-means-not-present semantics into the pipeline structure.
In some sense we'd want a mapper pipeline operation that has a choice of returning zero or one elements. A way to do this with streams is with flatMap. The flatmapper function can return an arbitrary number of elements into the stream, but in this case we want just zero or one. Here's how to do that:
return names.stream()
.flatMap(name -> {
Wongo w = nameToWongoMap.get(name);
return w == null ? Stream.empty() : Stream.of(w);
})
.map(Bongo::new)
.collect(toList());
I admit this is pretty clunky and so I wouldn't recommend doing this. A slightly better but somewhat obscure approach is this:
return names.stream()
.flatMap(name -> Optional.ofNullable(nameToWongoMap.get(name))
.map(Stream::of).orElseGet(Stream::empty))
.map(Bongo::new)
.collect(toList());
but I'm still not sure I'd recommend this as it stands.
The use of flatMap does point to another approach, though. If you have a more complicated policy of how to deal with the not-present case, you could refactor this into a helper function that returns a Stream containing the result or an empty Stream if there's no result.
Finally, JDK 9 -- still under development as of this writing -- has added Stream.ofNullable which is useful in exactly these situations:
return names.stream()
.flatMap(name -> Stream.ofNullable(nameToWongoMap.get(name)))
.map(Bongo::new)
.collect(toList());
As an aside, JDK 9 has also added Optional.stream which creates a zero-or-one stream from an Optional. This is useful in cases where you want to call an Optional-returning function from within flatMap. See this answer and this answer for more discussion.

One approach I didn't see is retainAll:
public List<Bongo> getBongos(List<String> names) {
Map<String, Wongo> copy = new HashMap<>(nameToWongoMap);
copy.keySet().retainAll(names);
return copy.values().stream().map(Bongo::new).collect(
Collectors.toList());
}
The extra Map is a minimal performance hit, since it's just copying pointers to objects, not the objects themselves.