I have a collection constisting of Map<Pair<DateTime, String>, List<Entity>> which was previously grouped using streams. Entity is a simple class with int property and getValue() method.
Now, I want to aggregate values of Entity with usage of my simple EntityAccumulator modyfing the type of the previous map to Map<Pair<DateTime, String>, EntityAccumulator>. The only way to achieve this as far as I understand is to create my own custom collector, howevere I've stucked at finisher() method which should return Pair.
Or, maybe is there simpler way to achieve the result I want ?
StreamProcessing
Map<Pair<DateTime, String>, EntityAccumulator> collect = entities.stream()
.collect(Collectors.groupingBy(entity-> Pair.of(entity.getTimestamp(), entity.getName())))
.entrySet().stream()
.collect(new EntityCollector()));
EntityAccumulator
private static class EntityAccumulator {
private int result = 0.0;
public EntityAccumulator() { }
public EntityAccumulator(int result) {
this.result = result;
}
public void calculate(Entity entity) {
result += entity.getValue();
}
public EntityAccumulatoradd(EntityAccumulator other) {
return new EntityAccumulator(this.result + other.result);
}
}
Collector
public class EntityCollector implements Collector<Map.Entry<Pair<DateTime, String>, List<Entity>>, EntityAccumulator, Map.Entry<Pair<DateTime, String>, EntityAccumulator>> {
#Override
public Supplier<EntityAccumulator> supplier() {
return EntityAccumulator::new;
}
#Override
public BiConsumer<EntityAccumulator, Map.Entry<Pair<DateTime, String>, List<Entity>>> accumulator() {
return (result, pairListEntry) -> pairListEntry.getValue().forEach(result::calculate);
}
#Override
public BinaryOperator<EntityAccumulator> combiner() {
return EntityAccumulator::add;
}
#Override
public Function<EntityAccumulator, Map.Entry<Pair<DateTime, String>, EntityAccumulator>> finisher() {
return (k) -> {
return null; // ??? HELP HERE
}
}
#Override
public Set<Characteristics> characteristics() {
return EnumSet.of(Characteristics.UNORDERED);
}
}
Apparently, you actually want to do
Map<Pair<DateTime, String>, Double> collect = entities.stream()
.collect(Collectors.groupingBy(
entity -> Pair.of(entity.getTimestamp(), entity.getName()),
Collectors.summingDouble(Entity::getValue)));
or
Map<Pair<DateTime, String>, Integer> collect = entities.stream()
.collect(Collectors.groupingBy(
entity -> Pair.of(entity.getTimestamp(), entity.getName()),
Collectors.summingInt(Entity::getValue)));
depending on the actual value type. Your declaration int result = 0.0 isn’t quite clear.
First, if you want to perform reduction on the groups, you should provide the Collector for the values as a second argument to the groupingBy collector. Then, it doesn’t have to deal with neither, Map nor Map.Entry.
Since it’s basically folding the entities to a single number (for each group), you can use an existing collector, i.e. summingInt or summingDouble.
When you create your own collector, you can’t reconstitute information in the finisher function that you have dropped in the accumulator function. If your container type EntityAccumulator contains a single number only, there is no way to produce a Map.Entry<Pair<DateTime, String>, EntityAccumulator> from it.
By the way, you rarely need to implemented the Collector interface with a class, even when creating a custom collector. You can simply use Collector.of, specifying the functions and characteristics, to create a Collector.
So using your original EntityAccumulator class (assuming, result should be int and 0.0 is a typo), you could use
Map<Pair<DateTime, String>, Integer> collect = entities.stream()
.collect(Collectors.groupingBy(
entity -> Pair.of(entity.getTimestamp(), entity.getName()),
Collector.of(EntityAccumulator::new,
EntityAccumulator::calculate,
EntityAccumulator::add,
ea -> ea.result,
Collector.Characteristics.UNORDERED)));
to achieve the same as above. It would also be possible to perform the operation in two steps, like in your attempt, using
Map<Pair<DateTime, String>, Integer> collect = entities.stream()
.collect(Collectors.groupingBy(e -> Pair.of(e.getTimestamp(), e.getName())))
.entrySet().stream()
.collect(Collectors.toMap(Map.Entry::getKey, e -> e.getValue().stream().collect(
Collector.of(EntityAccumulator::new,
EntityAccumulator::calculate,
EntityAccumulator::add,
ea -> ea.result,
Collector.Characteristics.UNORDERED))));
but, of course, this is only for completeness. The solution shown at the beginning of this answer is simpler and more efficient.
Related
I have the following method:
public void caller(){
List<Class1> data1 = Arrays.asList(new Class1(), new Class1() ...);
List<Class2> data2 = Arrays.asList(new Class2(), new Class2() ...);
// The following is what I'm trying to implement:
List<BiConsumer<Class1, Double>> peeks1 = Arrays.asList(Class1::setOneNum, Class1::setAnotherNum, Class1:: setMoreNum);
List<BiConsumer<Class2, Double>> peeks2 = Arrays.asList(Class2::setSomeNum1, Class2::setSomeNum2);
helper(data1, peeks1);
helper(data2, peeks2);
...
}
private <T> List<T> helper(List<T> data, List<BiConsumer<T, Double>> peeks) {
for(BiConsumer<T, Double> singlePeek: peeks){
data = data.stream()
.peek(a -> singlePeek.accept(a, math.random()))
.collect(Collectors.toList());
}
return data;
}
There are other implementation in common for Class1 and Class2, the only difference are the methods called after the .stream() which is why I'm trying to "merge" the functions into one helper.
Where BiConsumer is a setter. I want to call a list of setters after stream(). But I cannot input a list of functional interface into helper() (what I tried was Arrays.asList(Class1::setNum, Class1::setAnotherNum, Class1::setMoreNum) won't work as an input since Array.asList() only accepts Object). So is there any work-around? Thanks!
#user7 Thanks for pointing it out. I was careless but I've fixed the "typo". And added the caller function.
You have to specify the target type, when you call the .asList method:
Arrays.<BiConsumer<Object, Double>>asList(Class1::setOneNum, ...)
Update:
According to the updated code of the question the result of Arrays.asList is not directly handed over to the helper method, so no explicit typing is required.
The only possible reasons left why the code is not working are:
At least one of the methods (setOneNum, setSomeNum1, ...) has wrong parameters types
At least one of the methods is not static
Could I advise you in trying to make it a little bit more functional?
For your code consider the following helper, this one will make use of function as a first class citizen concept and make some High Order Functions:
private <T, V> Function<Supplier<T>, Supplier<T>> helper(Supplier<V> v,
BiConsumer<T, V> bc) {
return (Supplier<T> r) -> {
bc.accept(r.get(), v.get());
return r;
};
}
This helper function expects a Supplier of some value kind of value and a BiConsumer that will be your setter function. The returns is a function of Suppliers of the same class you are working with.
With that we can make something like a pipe operator of functional languages. Their premises is that the data should processed in a pipeline operation.
List<Class1> data1 = Arrays.asList(new Class1(), new Class1());
List<Class2> data2 = Arrays.asList(new Class2(), new Class2());
Supplier<Double> random = () -> Math.random();
This will be our data, you have the same array and now a Supplier with the random value you want.
Now lets compose our pipeline with andThem:
data1.stream()//
.forEach(data -> {
helper(random, Class1::setOneNum)//
.andThen(helper(random, Class1::setAnotherNum))//
.andThen(helper(random, Class1::setMoreNum))//
.apply(() -> data);
System.out.println(data.toString());
});
data2.stream()//
.forEach(data -> {
helper(random, Class2::setSomeNum1)//
.andThen(helper(random, Class2::setSomeNum2))//
.apply(() -> data);
System.out.println(data.toString());
});
As you can see the helper function can be chained together with "andThem" method of Function interface. This will make Java execute the helper function and use it's return as the parameter of the next Function.
The data parameter will hole the values of classes and will be changed each chain. As we iterated all objects will
And the result:
Class1 [oneNum=0,047, anotherNum=0,482, moreNum=0,339]
Class1 [oneNum=0,131, anotherNum=0,889, moreNum=0,411]
Class2 [someNum1=0,18, someNum2=0,004]
Class2 [someNum1=0,497, someNum2=0,702]
I think it is the same result you want. And as you can see you don't need to pass any generics as the Java will understand it well.
The classes that I made for reference:
class Class1 {
double oneNum;
double anotherNum;
double moreNum;
public double getOneNum() {
return oneNum;
}
public void setOneNum(double oneNum) {
this.oneNum = oneNum;
}
public double getAnotherNum() {
return anotherNum;
}
public void setAnotherNum(double anotherNum) {
this.anotherNum = anotherNum;
}
public double getMoreNum() {
return moreNum;
}
public void setMoreNum(double moreNum) {
this.moreNum = moreNum;
}
#Override
public String toString() {
return MessageFormat.format("Class1 [oneNum={0}, anotherNum={1}, moreNum={2}]", oneNum, anotherNum, moreNum);
}
}
class Class2 {
double someNum1;
double someNum2;
public double getSomeNum1() {
return someNum1;
}
public void setSomeNum1(double someNum1) {
this.someNum1 = someNum1;
}
public double getSomeNum2() {
return someNum2;
}
public void setSomeNum2(double someNum2) {
this.someNum2 = someNum2;
}
#Override
public String toString() {
return MessageFormat.format("Class2 [someNum1={0}, someNum2={1}]", someNum1, someNum2);
}
}
I have a custom Collector of the sort
public class ClusteringCollector extends java.util.stream.Collector<MyModel, Map<String, ClusterModel>, SortedSet<Map.Entry<Integer, ClusterModel>>> {
#Override
public Supplier<Map<String, MyOtherModel>> supplier() {
return HashMap::new;
}
#Override
public BiConsumer<Map<String, ClusterModel>, MyModel> accumulator() {
return (l, r) -> {
String mapKey = r.getURI();
if(l.containsKey(mapKey)) {
l.get(mapKey).addCluster(r.getCluster());
} else {
l.put(mapKey, r.getCluster());
}
}
}
#Override
public BinaryOperator<Map<String, ClusterModel>> combiner() {
return (left, right) -> {
for(Map.Entry<String, ClusterModel> e : right.entrySet()) {
e.getValue().setClusterCount(1);
if(left.containsKey(e.getKey())) {
left.get(e.getKey()).merge(e.getValue());
} else {
left.put(e.getKey(), e.getValue());
}
}
return left;
};
}
#Override
public Function<Map<String, ClusterModel>, SortedSet<Map.Entry<Integer, ClusterModel>>> finisher() {
return (accumulated) -> {
SortedSet<Map.Entry<Integer, ClusterModel>> finished = new TreeSet<>((mine, theirs) -> {
Double t1 = mine.getValue().getClusterCount() * mine.getValue().getClusterWeight();
Double t2 = theirs.getValue().getClusterCount() * theirs.getValue().getClusterWeight();
return t2.compareTo(t1);
});
Map<Integer, ClusterModel> tempMap = new LinkedHashMap<>();
for(Map.Entry<String, ClusterModel> e : accumulated.entrySet()) {
if(tempMap.containsKey(e.getValue().hashCode())) {
tempMap.get(e.getValue().hashCode()).merge(e.getValue());
} else {
tempMap.put(e.getValue().hashCode(), e.getValue());
}
}
finished.addAll(tempMap.entrySet());
return finished;
};
}
#Override
public Set<Characteristics> characteristics() {
return EnumSet.of(Characteristics.UNORDERED, Characteristics.IDENTITY_FINISH);
}
}
I use the collector in the following way
try (Stream<MyModel> resultStream = generateDataStream()) {
SortedSet<Map.Entry<Integer, ClusterModel>> clusters = resultStream.collect(new ClusteringCollector()); // This line throws a ClassCastException
}
The problem though is that I keep getting a ClassCastException when I try to run the collect method above. Here is the stacktrace
java.lang.ClassCastException: java.util.HashMap cannot be cast to java.util.SortedSet
com.mycomp.abc.core.services.DefaultClusteringServiceImpl.findClusters(DefaultClusteringServiceImpl.java:78)
com.mycomp.abc.core.webservices.ClusteringWebService.getClustersFromQuery(ClusteringWebService.java:67)
com.mycomp.abc.core.webservices.ClusteringWebService$Proxy$_$$_WeldClientProxy.getClustersFromQuery(Unknown Source)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:237)
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
org.jboss.resteasy.core.SynchronousDispatcher.invokePropagateNotFound(SynchronousDispatcher.java:217)
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:224)
org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.doFilter(FilterDispatcher.java:62)
Can someone tell me why this is happening? I am not getting any compilation errors though and the finisher transforms the Map into a SortedSet correctly.
There are several errors in the posted code which suggest that this is not the actual code, however, the main problem is recognizable. You specified the IDENTITY_FINISH characteristic, despite you have a complex conversion in the finisher.
The IDENTITY_FINISH characteristic implies that the finisher was just like Function.identity(), but at runtime, the Stream implementation can’t check whether the generic signatures are compatible with that declaration. When it uses this characteristic to decide to skip the finisher, it will just return the container object, which is a HashMap in your case, which, of course, is not assignable to SortedSet.
In the end, that’s the better outcome of this mistake. The worse would be if the container and result type are compatible and the skipping of a nontrivial finisher stays unnoticed at first. So be careful about specifying the IDENTITY_FINISH characteristic.
Note that when you don’t implement Collector, but rather construct one by passing the functions to Collector.of(…), you never need to specify that characteristic, as it will be injected based on whether you specified a finisher function or not. For the overloaded method without a finisher, the generic signature will even ensure that the container type matches the result type.
I got a function which returns a map.
i want to konw if it's possible to apply a function over the key of the map in order to store function(Id) instead of the key Id.
it would be perfect to do this in one shot.
here is the code of my function :
public Map<String, List<Object>> getMap(final List<Object> listOfObjects) {
Map<String, List<Object>> map = (listOfObjects.stream().collect(Collectors.groupingBy(Object::getId)));
return map;
}
public class Object {
int id,
String msg
int getId() {
return id;
}
}
int function( int id) {
// some code
}
what i want to store in the map is function(id) and not the id
You can have a generic utility method that receives the list and a function that transforms each item of the list into whatever you want, and use the result of this function as the key of the map:
public static <K, V> Map<K, List<V>> getMap(
List<V> list,
Function<? super V, ? extends K> keyMapper) {
return list.stream()
.collect(Collectors.groupingBy(keyMapper));
}
Usage:
Map<Integer, List<Item>> map = getMap(listOfItems, Item::getId);
This works exactly as your code, except that I've renamed your Object class to Item, because Object was a misleading name that clashes with java.lang.Object.
Now, if you want to transform the id of each object before putting each entry in the map:
Function<Item, Integer> function = Item::getId;
Map<Integer, List<Item>> mapWithIncrementedKeys = getMap(
listOfItems,
function.andThen(i -> i + 1));
This would transform the id of each item by incrementing its value before using it as the keys of the returned map.
I currently have something like below
List<String> myNewList = myList
.stream()
.map(item->{
return mappedItem
})
.collect(Collectors.toList());
repository.save(myNewList);
In Optional, I can perform operations on the mapped item by using ifPresent method like below
myOptional
.map(item -> {
return mappedItem
})
.ifPresent(newItem -> {
repository.save(newItem);
});
I was wondering if I can do something like the above on stream. Rather than declaring myNewList, is there a way I can collect the new List and apply my function on the new list?
Update: Based on the answer from #tagir-valeev, I modified my code as below
myList
.stream()
.map(item->{
return mappedItem
})
.collect(Collectors.collectingAndThen(Collectors.toList(),
list -> {
repository.save(list);
return list;
}
));
You can create your custom collector like this:
myList.stream().map(..)
.collect(Collectors.collectingAndThen(Collectors.toList(), repository::save));
If save return type is void, it would be more ugly as you need to return something from collect:
myList.stream().map(..)
.collect(Collectors.collectingAndThen(Collectors.toList(),
list -> {repository.save(list);return list;}));
You may declare special method in your Repository class:
class Repository {
Collector<MyItemType, ?, List<MyItemType>> saving() {
return Collectors.collectingAndThen(Collectors.toList(),
list -> {this.save(list);return list;});
}
void save(List<MyItemType> list) { ... }
}
And use it:
myList.stream().map(..).collect(repository.saving());
I would like to apply a function to a Java collection, in this particular case a map. Is there a nice way to do this? I have a map and would like to just run trim() on all the values in the map and have the map reflect the updates.
With Java 8's lambdas, this is a one liner:
map.replaceAll((k, v) -> v.trim());
For the sake of history, here's a version without lambdas:
public void trimValues(Map<?, String> map) {
for (Map.Entry<?, String> e : map.entrySet()) {
String val = e.getValue();
if (val != null)
e.setValue(val.trim());
}
}
Or, more generally:
interface Function<T> {
T operate(T val);
}
public static <T> void replaceValues(Map<?, T> map, Function<T> f)
{
for (Map.Entry<?, T> e : map.entrySet())
e.setValue(f.operate(e.getValue()));
}
Util.replaceValues(myMap, new Function<String>() {
public String operate(String val)
{
return (val == null) ? null : val.trim();
}
});
I don't know a way to do that with the JDK libraries other than your accepted response, however Google Collections lets you do the following thing, with the classes com.google.collect.Maps and com.google.common.base.Function:
Map<?,String> trimmedMap = Maps.transformValues(untrimmedMap, new Function<String, String>() {
public String apply(String from) {
if (from != null)
return from.trim();
return null;
}
}
The biggest difference of that method with the proposed one is that it provides a view to your original map, which means that, while it is always in sync with your original map, the apply method could be invoked many times if you are manipulating said map heavily.
A similar Collections2.transform(Collection<F>,Function<F,T>) method exists for collections.
Whether you can modify your collection in-place or not depends on the class of the objects in the collection.
If those objects are immutable (which Strings are) then you can't just take the items from the collection and modify them - instead you'll need to iterate over the collection, call the relevant function, and then put the resulting value back.
Might be overkill for something like this, but there are a number of really good utilities for these types of problems in the Apache Commons Collections library.
Map<String, String> map = new HashMap<String, String>();
map.put("key1", "a ");
map.put("key2", " b ");
map.put("key3", " c");
TransformedMap.decorateTransform(map,
TransformerUtils.nopTransformer(),
TransformerUtils.invokerTransformer("trim"));
I highly recommend the Jakarta Commons Cookbook from O'Reilly.
I ended up using a mutation of #erickson's answer, mutated to:
return a new Collection, not modify in place
return Collections with elements of type equal to the return type of the Function
support mapping over either the values of a map or the elements of a list
Code:
public static interface Function<L, R> {
L operate(R val);
}
public static <K, L, R> Map<K, L> map(Map<K, R> map, Function<L, R> f) {
Map<K, L> retMap = new HashMap<K, L>();
for (Map.Entry<K, R> e : map.entrySet()) retMap.put(e.getKey(), f.operate(e.getValue()));
return retMap;
}
public static <L, R> List<L> map(List<R> list, Function<L, R> f) {
List<L> retList = new ArrayList<L>();
for (R e : list) retList.add(f.operate(e));
return retList;
}
You'll have to iterate over all the entries and trim each String value. Since String is immutable you'll have to re-put it in the map. A better approach might be to trim the values as they're placed in the map.
I have come up with a "Mapper" class
public static abstract class Mapper<FromClass, ToClass> {
private Collection<FromClass> source;
// Mapping methods
public abstract ToClass map(FromClass source);
// Constructors
public Mapper(Collection<FromClass> source) {
this.source = source;
}
public Mapper(FromClass ... source) {
this.source = Arrays.asList(source);
}
// Apply map on every item
public Collection<ToClass> apply() {
ArrayList<ToClass> result = new ArrayList<ToClass>();
for (FromClass item : this.source) {
result.add(this.map(item));
}
return result;
}
}
That I use like that :
Collection<Loader> loaders = new Mapper<File, Loader>(files) {
#Override public Loader map(File source) {
return new Loader(source);
}
}.apply();
You could also take a look at Google Collections