Related
I’m trying to obtain a only duplicated numbers list from a list of integers:
final Set<Integer> setOfNmums = new HashSet<>();
Arrays.asList(5,6,7,7,7,6,2,4,2,4).stream()
.peek(integer -> System.out.println("XX -> " + integer))
.filter(n -> !setOfNmums.add(n))
.peek(System.out::println)
.map(String::valueOf)
.sorted()
.collect(Collectors.toList());
The output is 2,4,6,7,7
Expected : 2,4,6,7
I don’t understand how that’s happening.. is this running in parallel? how am I getting two '7'?
The hashset should return false if it exists and that used by the filter?
Yes I can use distinct, but I’m curious to know why would the filter fail.. is it being done in parallel?
Your filter rejects the first occurrence of each element and accepts all subsequent occurrences. Therefore, when an element occurs n times, you’ll add it n-1 times.
Since you want to accept all elements which occur more than once, but only accept them a single time, you could use .filter(n -> !setOfNmums.add(n)) .distinct() or you enhance the set to a map, to be able to accept an element only on its second occurrence.
Map<Integer, Integer> occurrences = new HashMap<>();
List<String> result = Stream.of(5,6,7,7,7,6,2,4,2,4)
.filter(n -> occurrences.merge(n, 1, Integer::sum) == 2)
.map(String::valueOf)
.sorted()
.collect(Collectors.toList());
But generally, using stateful filters with streams is discouraged.
A cleaner solution would be
List<String> result = Stream.of(5,6,7,7,7,6,2,4,2,4)
.collect(Collectors.collectingAndThen(
Collectors.toMap(String::valueOf, x -> true, (a,b) -> false, TreeMap::new),
map -> { map.values().removeIf(b -> b); return new ArrayList<>(map.keySet()); }));
Note that this approach doesn’t count the occurrences but only remembers whether an element is unique or has seen at least a second time. This works by mapping each element to true with the second argument to the toMap collector, x -> true, and resolving multiple occurrences with a merge function of (a,b) -> false. The subsequent map.values().removeIf(b -> b) will remove all unique elements, i.e. those mapped to true.
You can use .distinct() function in your stream check this out.
Since Holger already explained why your solution didn't work, I'll just provide an alternative.
Why not use Collections.frequency(collection, element) together with distinct()?
The solution would be quite simple(i apologize for the formatting, i just copied it from my ide and there doesn't seem to be an autoformat feature in SOF):
List<Integer> numbers = List.of(5, 6, 7, 7, 7, 6, 2, 4, 2, 4);
List<String> onlyDuplicates = numbers.stream()
.filter(n -> Collections.frequency(numbers, n) > 1)
.distinct()
.sorted()
.map(String::valueOf)
.toList();
This simply keeps all elements that occur more than once and then filters out the duplicates before sorting, converting each element to a string and collecting to a list since that seems to be what you want.
if you need a mutable list you can use collect(toCollection(ArrayList::new)) instead of toList()
In previous question I asked previously Which FunctionalInterface should I use?
Now I was trying to add to List<Integer> and not just two Integers a and b, such that each index adds to the same index of another list.
I had previously
BinaryOperator<Integer> binaryOperator = Integer::sum;
for adding two integers using binaryOperator.apply(int a,int b). Is there a similar way like
BinaryOperator<List<Integer>> binaryOperator = List<Integer>::sum;
and then get the result in List<Integer> cList?
if you want to perform some computation on the elements at corresponding indices (in this specific case the summation) there's no need to use a BinaryOperator, instead use IntStream.range to generate the indices:
// generates numbers from 0 until list.size exclusive
IntStream.range(0, list.size())....
// generates numbers from 0 until the minimum of the two lists exclusive if needed
IntStream.range(0, Math.min(list.size(), list2.size()))....
The common name for this type of logic is "zip"; i.e. when given two input sequences it produces an output sequence in which every two elements from the input sequences at the same position are combined using some function.
There's no built-in method in the standard library for this, but you can find some generic implementations here.
for example, using the zip method in the accepted answer of the linked post, you could simply do:
List<Integer> result = zip(f.stream(), s.stream(), (l, r) -> l + r).collect(toList());
or using method reference:
List<Integer> result = zip(f.stream(), s.stream(), Math::addExact).collect(toList());
where f and s are your list of integers.
You can use IntStream.range() to iterate over elements, then mapToObj() to map them to their sum and the collect() them in the third list.
Given that your lists are of same size
List<Integer> first = List.of(); // initialised
List<Integer> second = List.of(); // initialised
you can get the third list as :
List<Integer> third = IntStream.range(0, first.size())
.mapToObj(i -> first.get(i) + second.get(i))
.collect(Collectors.toList());
In terms of BinaryOperator, you can represent it as :
BinaryOperator<List<Integer>> listBinaryOperator = (a, b) -> IntStream.range(0, first.size())
.mapToObj(i -> first.get(i) + second.get(i))
// OR from your existing code
// .mapToObj(i -> binaryOperator.apply(first.get(i), second.get(i)))
.collect(Collectors.toList());
Or you can make it more readable by abstracting out the logic into a method and using it as :
BinaryOperator<List<Integer>> listBinaryOperator = YourClass::sumOfList;
where sumOfList is defined as :
private List<Integer> sumOfList(List<Integer> first, List<Integer> second) {
return IntStream.range(0, first.size())
.mapToObj(i -> first.get(i) + second.get(i))
.collect(Collectors.toList());
}
What you can do is define your own utility method whose single task is zipping the two input lists :
<T> List<T> zip(final List<? extends T> first, final List<? extends T> second, final BinaryOperator<T> operation)
{
return IntStream.range(0, Math.min(first.size(), second.size()))
.mapToObj(index -> operation.apply(first.get(index), second.get(index)))
.collect(Collectors.toList());
}
This way, you can sum your two input lists as :
zip(first, second, Integer::sum)
PROBLEM
I have a list of arrays and I want to count the occurrences of duplicates.
For example, if I have this :
{{1,2,3},
{1,0,3},
{1,2,3},
{5,2,6},
{5,2,6},
{5,2,6}}
I want a map (or any relevant collection) like this :
{ {1,2,3} -> 2,
{1,0,3} -> 1,
{5,2,6} -> 3 }
I can even lose the arrays values, I'm only interested in cardinals (e.g. 2, 1 and 3 here).
MY SOLUTION
I use the following algorithm :
First hash the arrays, and check if each hash is in an HashMap<Integer, ArrayList<int[]>>, let's name it distinctHash, where the key is the hash and the value is an ArrayList, let's name it rowList, containing the different arrays for this hash (to avoid collisions).
If the hash is not in distinctHash, put it with the value 1 in another HashMap<int[], Long> that counts each occurrence, let's call it distinctElements.
Then if the hash is in distinctHash, check if the corresponding array is contained in rowList. If it is, increment the value in distinctElements associated to the identical array found in rowList. (If you use the new array as a key you will create another key since their reference are different).
Here is the code, the boolean returned tells if a new distinct array was found, I apply this function sequentially on all of my arrays :
HashMap<int[], Long> distinctElements;
HashMap<Integer, ArrayList<int[]>> distinctHash;
private boolean addRow(int[] row) {
if (distinctHash.containsKey(hash)) {
int[] indexRow = distinctHash.get(hash).get(0);
for (int[] previousRow: distinctHash.get(hash)) {
if (Arrays.equals(previousRow, row)) {
distinctElements.put(
indexRow,
distinctElements.get(indexRow) + 1
);
return false;
}
}
distinctElements.put(row, 1L);
ArrayList<int[]> rowList = distinctHash.get(hash);
rowList.add(row);
distinctHash.put(hash, rowList);
return true;
} else {
distinctElements.put(row, 1L);
ArrayList<int[]> newValue = new ArrayList<>();
newValue.add(row);
distinctHash.put(hash, newValue);
return true;
}
}
QUESTION
The problem is that my algorithm is too slow for my needs (40s for 5,000,000 arrays, and 2h-3h for 20,000,000 arrays). Profiling with NetBeans told me that the hashing takes 70% of runtime (using Google Guava murmur3_128 hash function).
Is there another algorithm that could be faster? As I said I'm not interested in arrays values, only in the number of their occurrences. I am ready to sacrifice precision for speed so a probabilistic algorithm is fine.
Wrap the int[] in a class that implements equals and hashCode, then build Map of the wrapper class to instance count.
class IntArray {
private int[] array;
public IntArray(int[] array) {
this.array = array;
}
#Override
public int hashCode() {
return Arrays.hashCode(this.array);
}
#Override
public boolean equals(Object obj) {
return (obj instanceof IntArray && Arrays.equals(this.array, ((IntArray) obj).array));
}
#Override
public String toString() {
return Arrays.toString(this.array);
}
}
Test
int[][] input = {{1,2,3},
{1,0,3},
{1,2,3},
{5,2,6},
{5,2,6},
{5,2,6}};
Map<IntArray, Long> map = Arrays.stream(input).map(IntArray::new)
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
map.entrySet().forEach(System.out::println);
Output
[1, 2, 3]=2
[1, 0, 3]=1
[5, 2, 6]=3
Note: The above solution is faster and uses less memory than solution by Ravindra Ranwala, but it does require the creation of an extra class, so it is debatable which is better.
For smaller arrays, use the simpler solution below by Ravindra Ranwala.
For larger arrays, the above solution is likely better.
Map<List<Integer>, Long> map = Stream.of(input)
.map(a -> Arrays.stream(a).boxed().collect(Collectors.toList()))
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
You may do it like so,
Map<List<Integer>, Long> result = Stream.of(source)
.map(a -> Arrays.stream(a).boxed().collect(Collectors.toList()))
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
And here's the output,
{[1, 2, 3]=2, [1, 0, 3]=1, [5, 2, 6]=3}
If the sequence of elements for all duplication of that array is like each other and the length of each array is not much, you can map each array to an int number and using from last part of your method. Although this method decrease the time of hashing, there are some assumptions here which might not be true for your case.
Let's say I have a list
ArrayList<String> arr = new ArrayList(Arrays.asList("N1", "N2", "N3", "N5"));
How do I find "N4", I mean, how I find that the missing integer is 4?
What I've tried so far
Integer missingID = arr.stream().map(p -> Integer.parseInt(p.substring(1))).sorted()
.reduce((p1, p2) -> (p2 - p1) > 1 ? p1 + 1 : 0).get();
This doesn't work because reduce is not intended to work in the way I need in this situation, actually, I have no idea how do that.
If there's no missing number, than the next must be "N6" - or just 6 - (in this example)
It must be done with java standard stream's library, no use of third parties.
The algorithm to implement here is based from this one: to find the missing number in a sequence of integers, the trick is to:
calculate the sum of the elements in the sequence.
calculate the sum of the elements the sequence would have with the missing number: this is easy to do since we can determine the minimum, the maximum and we know that the sum from a sequence of integer going from min to max is max*(max+1)/2 - (min-1)*min/2.
find the difference between those two sums: that's our missing number
In this case, we can collect statistics on our Stream by first mapping to an IntStream formed by only the numbers themselves and then calling summaryStatistics(). This returns a IntSummaryStatistics that has all the values we want: min, max and sum:
public static void main(String[] args) {
List<String> arr = Arrays.asList("N3", "N7", "N4", "N5", "N2");
IntSummaryStatistics statistics =
arr.stream()
.mapToInt(s -> Integer.parseInt(s.substring(1)))
.summaryStatistics();
long max = statistics.getMax();
long min = statistics.getMin();
long missing = max*(max+1)/2 - (min-1)*min/2 - statistics.getSum();
System.out.println(missing); // prints "6" here
}
If there is no missing number, this will print 0.
Here's the solution involving the pairMap operation from my free StreamEx library. It prints all the missing elements of the sorted input:
ArrayList<String> arr = new ArrayList(Arrays.asList("N1", "N2", "N3", "N5"));
StreamEx.of(arr).map(n -> Integer.parseInt(n.substring(1)))
.pairMap((a, b) -> IntStream.range(a+1, b))
.flatMapToInt(Function.identity())
.forEach(System.out::println);
The pairMap operation allows you to map every adjacent pair of the stream to something else. Here we map them to the streams of the skipped numbers, then flatten these streams.
The same solution is possible without third-party library, but looks more verbose:
ArrayList<String> arr = new ArrayList(Arrays.asList("N1", "N2", "N3", "N5"));
IntStream.range(0, arr.size()-1)
.flatMap(idx -> IntStream.range(
Integer.parseInt(arr.get(idx).substring(1))+1,
Integer.parseInt(arr.get(idx+1).substring(1))))
.forEach(System.out::println);
If there's only ONE missing number in the array, and if all numbers are positive, you could use the XOR algorithm, as explained in this question and its answers:
List<String> list = Arrays.asList("N5", "N2", "N3", "N6");
int xorArray = list.stream()
.mapToInt(p -> Integer.parseInt(p.substring(1)))
.reduce(0, (p1, p2) -> p1 ^ p2);
int xorAll = IntStream.rangeClosed(2, 6)
.reduce(0, (p1, p2) -> p1 ^ p2);
System.out.println(xorArray ^ xorAll); // 4
The advantage of this approach is that you don't need to use extra data structures, all you need is a couple of ints.
EDIT as per #Holger's comments below:
This solution requires you to know the range of the numbers in advance. Although on the other hand, it doesn't require the list and stream to be sorted.
Even if the list wasn't sorted, you could still get min and max (hence, the range) with IntSummaryStatistics, but this would require an extra iteration.
You could create a state object which is used to transform a single input stream into multiple streams of missing entries. These missing entry streams can then be flat mapped to produce a single output:
public class GapCheck {
private String last;
public GapCheck(String first) {
last = first;
}
public Stream<String> streamMissing(String next) {
final int n = Integer.parseInt(next.replaceAll("N", ""));
final int l = Integer.parseInt(last.replaceAll("N", ""));
last = next;
return IntStream.range(l + 1, n).mapToObj(Integer::toString);
}
}
Usage:
final List<String> arr = new ArrayList(Arrays.asList("N1", "N3", "N5"));
arr.stream()
.flatMap(new GapCheck(arr.get(0))::streamMissing)
.forEach(System.out::println);
output:
2
4
This is more work than you might expect, but it can be done with a collect call.
public class Main {
public static void main(String[] args) {
ArrayList<String> arr = new ArrayList<String>(Arrays.asList("N1", "N2", "N3", "N5", "N7", "N14"));
Stream<Integer> st = arr.stream().map(p -> Integer.parseInt(p.substring(1))).sorted();
Holder<Integer> holder = st.collect(() -> new Holder<Integer>(),
(h, i) -> {
Integer last = h.getProcessed().isEmpty() ? null : h.getProcessed().get(h.getProcessed().size() - 1);
if (last != null) {
while (i - last > 1) {
h.getMissing().add(++last);
}
}
h.getProcessed().add(i);
},
(h, h2) -> {});
holder.getMissing().forEach(System.out::println);
}
private static class Holder<T> {
private ArrayList<T> processed;
private ArrayList<T> missing;
public Holder() {
this.processed = new ArrayList<>();
this.missing = new ArrayList<>();
}
public ArrayList<T> getProcessed() {
return this.processed;
}
public ArrayList<T> getMissing() {
return this.missing;
}
}
}
This prints
4
6
8
9
10
11
12
13
Note that this sort of thing isn't really a particularly strong fit for Streams. All of the stream processing methods will tend to pass you each item exactly one time, so you need to handle all runs of missing numbers at once, and in the end, you're writing kind of a lot of code to avoid just writing a loop.
Here is one solution using pure streams, albeit not very efficient.
public void test() {
List<String> arr = new ArrayList(
Arrays.asList("N1", "N2", "N3", "N5", "N7"));
List<Integer> list = IntStream
.range(1, arr.size())
.mapToObj(t -> new AbstractMap.SimpleEntry<Integer, Integer>(
extract(arr, t), extract(arr, t) - extract(arr, t - 1)))
.filter(t -> t.getValue() > 1)
.map(t -> t.getKey() - 1)
.collect(Collectors.toList());
System.out.println(list);
}
private int extract(List<String> arr, int t) {
return Integer.parseInt(arr.get(t).substring(1));
}
Major performance block will be because of repeated parsing of list elements. However, this solution will be able to provide all missing numbers.
I have a 2D array that I want to print using IntStream.
this is the array,
int[][] twoD = { { 1, 2 }, { 3, 4 }, { 5, 6 } };
Now, using nested loop this can be done like,
for (int i = 0; i < twoD.length; i++) {
for (int j = 0; j < twoD[i].length; j++) {
System.out.println(twoD[i][j]);
}
}
but i want to use IntStream. and I recently learned about its flatMap method which I can use to achieve that, so I tried this,
IntStream.range(0, twoD.length)
.flatMap(j -> IntStream.range(0, twoD[j].length))
.forEach(System.out::print);
and it outputs 010101.
One reason for the output being 010101 is that 010101 are index values not the values in the array and I have to map these values to the array values using something like, i -> twoD[i]
so I tried this,
IntStream.range(0, twoD.length)
.map(i -> twoD[i])
.flatMap(j -> IntStream.range(0, twoD[j].length))
.forEach(System.out::print);
but it gives error on map(i -> twoD[i]),
Type mismatch: cannot convert from int[] to int
but if it was 1D array then it would have worked, for example,
int[] oneD = { 1, 2, 3, 4, 5, 6 };
IntStream.range(0, oneD.length)
.map(i -> oneD[i])
.forEach(System.out::print);
How can I print the 2D array using the above approach ?
I think you overcomplicate things. You could just do it like this:
Stream.of(twoD).flatMapToInt(IntStream::of).forEach(System.out::println);
What it does is:
get a Stream<int[]> from the int[][] array
flatMap each int[] to an IntStream so that you get back an IntStream with all the elements of the 2D array
for each element, print it
What you want to do is achievable but not very readable. A formal translation of your nested loop would be:
IntStream.range(0, twoD.length)
.forEach(i -> IntStream.range(0, twoD[i].length)
.forEach(j -> System.out.println(twoD[i][j])));
which produces the same output, but as you can see it's not very readable. Here you don't need to stream the indices so the first approach with flatMapToInt is the best.
Now why your solution doesn't compile?
It's because map on an IntStream expect a mapping function that gives you back an int but you give an int[]. You need to use mapToObj and then again flatMapToInt to get an IntStream and finally print the contents (this isn't the only solution though).
IntStream.range(0, twoD.length)
.mapToObj(i -> twoD[i])
.flatMapToInt(IntStream::of)
.forEach(System.out::print);
Do you gain in readability? Not really, so I suggest to use the first approach which is clear and concise.
Note that the last solution could be also written as:
IntStream.range(0, twoD.length)
.flatMap(i -> IntStream.of(twoD[i]))
.forEach(System.out::print);
... but I still prefer the first approach ! :)
Why not stream the array itself:
Arrays.stream(twoD)
.flatMapToInt(Arrays::stream)
.forEach(System.out::println);