We Keep Coding

Can not modify value in JavaRDD

I have a question about how to update JavaRDD values.
I have a JavaRDD<CostedEventMessage> with message objects containing information about to which partition of kafka topic it should be written to.
I'm trying to change the partitionId field of such objects using the following code:
rddToKafka = rddToKafka.map(event -> repartitionEvent(event, numPartitions));
where the repartitionEvent logic is:
costedEventMessage.setPartitionId(1);
return costedEventMessage;
But the modification does not happen.
Could you please advice why and how to correctly modify values in a JavaRDD?

Spark is lazy, so from the code you pasted above it's not clear if you actually performed any action on the JavaRDD (like collect or forEach) and how you came to the conclusion that data was not changed.
For example, if you assumed that by running the following code:
List<CostedEventMessage> messagesLst = ...;
JavaRDD<CostedEventMessage> rddToKafka = javaSparkContext.parallelize(messagesLst);
rddToKafka = rddToKafka.map(event -> repartitionEvent(event, numPartitions));
Each element in messagesLst would have partition set to 1, you are wrong.
That would hold true if you added for example:
messagesLst = rddToKafka.collect();
For more details refer to documentation

Obtain java primitive from mongo aggregation without a new output class

I have an aggregation:
AggregationResults<Integer> result = mongoTemplate.aggregate(
Aggregation.newAggregation(
Aggregation.group().count().as("value"),
Aggregation.project("value").andExclude("_id"),
MyData.class, Integer.class);
In the mongo shell, when I don't have to map an object, I get: { "value" : 2 }
However, I get the following error when trying to map this lone value: org.springframework.data.mapping.model.MappingException: No mapping metadata found for java.lang.Integer
Can I get around having to create a new output type class, when I only want to get a single java primitive?
Note: I'm going this approach instead of db.collection.count() for the sharding inaccuracies stated here - https://docs.mongodb.com/manual/reference/method/db.collection.count/#sharded-clusters

AggregationResults<DBObject> result = mongoTemplate.aggregate(
Aggregation.newAggregation(
Aggregation.group().count().as("value"),
Aggregation.project("value").andExclude("_id"),
MyData.class, DBObject.class);
int count = (Integer) result.getUniqueMappedResult().get("value");
So, not exactly what I wanted, because I still have to traverse over an object, but it's not any more code than I had before and I didn't need to make another class as the outputType.

Java 8 Stream API Filtering

I have a collection of objects, with the following type:
{
String action_name; //add or delete
long action_time;
String action_target;
}
Need to get the latest merged operation on each action_target
Sample input data:
[add|1001|item1, add|1002|item2, delete|1003|item1, add|1004|item1]
Expected result:
[add|1002|item2, add|1004|item1]
Sample input data:
[add|1001|item1, add|1002|item2, delete|1003|item1]
Expected result:
[add|1002|item2]
Sample input data:
[delete|1001|item1, add|1002|item2, add|1003|item1]
Expected result:
[add|1002|item2, add|1003|item1]
Is this approachable using Java8 stream APIs? Thanks.

You want to group by one criteria (the action_target) combined with reducing the groups to the maximum of their action_time values:
Map<String,Item> map=items.stream().collect(
Collectors.groupingBy(item->item.action_target,
Collectors.collectingAndThen(
Collectors.maxBy(Comparator.comparing(item->item.action_time)),
Optional::get)));
This returns a Map<String,Item> but, of course, you may call values() on it to get a collection of items.
Beautified with static imports, the code looks like:
Map<String,Item> map=items.stream().collect(groupingBy(item->item.action_target,
collectingAndThen(maxBy(comparing(item->item.action_time)), Optional::get)));
Your additional request of taking care of idempotent "add" and follow-up "delete" actions can be simplified to “remove items whose last action is "delete"” which can be implemented just by doing that after collecting using a mutable map:
HashMap<String,Item> map=items.stream().collect(groupingBy(
item->item.action_target, HashMap::new,
collectingAndThen(maxBy(comparing(item->item.action_time)), Optional::get)));
map.values().removeIf(item->item.action_name.equals("delete"));

Freemarker: iterating nested list in hash

I want to iterate a List nested in a Map, the data structure is like:
Map<Integer, List<Integer>> groups = new TreeMap<>()
// Some code else to put values into groups ...
Freemarker template:
<#list groups?keys as groupKey>
${groupKey} // It's OK here.
<#list groups[groupKey] as item> // Exception threw here, detail message is pasted below
${item}
</#list>
</#list>
Detail exception message:
FreeMarker template error:
For "...[...]" left-hand operand: Expected a sequence or string or something automatically convertible to string (number, date or boolean), but this evaluated to an extended_hash (wrapper: f.t.SimpleHash):
==> groups
So, what is the problem?
P.S.
I have tried groups.get(groupKey) instead of groups[groupKey], it throws a new Exception stack:
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
java.lang.String.compareTo(String.java:108)
java.util.TreeMap.getEntry(TreeMap.java:346)
java.util.TreeMap.get(TreeMap.java:273)
freemarker.template.SimpleHash.get(SimpleHash.java:160)
freemarker.core.Dot._eval(Dot.java:40)
freemarker.core.Expression.eval(Expression.java:76)

The problem in the original question is that FTL's hash type is not like Map. It's a collection of "variables", that is, the keys must be String-s. (Even that ?keys works is a glitch in BeansWrapper... though now it comes handy.) Since the key is a number, FTL assumes that you want to get an item from a sequence (a List or array), or that you want to get a character from a string, hence the original error message.
The solution is using the Java API-s, like get in Dev-an's answer. (On the long term FTL meant to introduce the map type, so all this problems with non-string keys will end, but who knows when that will be...)
Update: Since 2.3.22 there's ?api to access the Java API of objects, like myMap?api.get(nonStringKey). However, it's by default not allowed (see the api_builtin_enabled configuration setting and more in the Manual: http://freemarker.org/docs/ref_builtins_expert.html#ref_buitin_api_and_has_api). Also note that as Java maps are particular about the numerical type, if the key is not an Integer coming from a Java, you have to use myMap?api.get(myNumericalKey?int).

Try the following:
<#list groups?keys as groupKey>
${groupKey}
<#list groups.get(groupKey) as item>
${item}
</#list>
</#list>

ADOBE CQ5 JCR - How to orderby/sorting query builder result using node's property

basically below is the java coding part, which the result will then be populated to a .csv file. However, I dont seem get the ordering part right (last line in below snippet).
Map<String, String> map = new HashMap<String, String>();
map.put("path", "/etc/crx-db/form-data/career");
map.put("type", "nt:unstructured");
map.put("p.limit", "-1");
map.put("daterange.property", "created");
map.put("daterange.lowerBound", from);
map.put("daterange.lowerOperation", ">=");
map.put("daterange.upperOperation", "<=");
map.put("daterange.upperBound", to);
map.put("orderby", "created"); //<--here
Providing that in crx repositry (/etc/crx-db/form-data/career), I have nodes: data1, data2, data3...
Then for each node, there is one property - Name: created | Type: Date | Value: 2014-01-28T23:21:15.029+08:00 (eg)
However my result in .csv is incorrect like (row 1 to 5):
2014-01-28T23:21:15.029+08:00
2014-01-28T23:48:12.219+08:00
2014-02-10T18:44:38.914+08:00 <-- unsorted
2014-02-10T18:43:32.426+08:00 <-- unsorted
2014-02-10T18:46:53.319+08:00
Pretty sure my code wasn't running. Any idea on how can I tweak my java code to make the sorting happen? As in returning sorted data1, data2, data3... based on the property created. Thanks.

You were almost there. It can be done as follows.
map.put("orderby", "#created");
map.put("orderby.sort", "desc"); // in case you want it descending
In case you need to check property within a child node, you can provide the relative path to that for the orderby value. For eg., if you are searching for dam:Asset and want to order them based on the jcr:lastModified property of its metadata, then your query would be something similar to this.
map.put("path", "/content/dam/geometrixx");
map.put("type", "dam:Asset");
map.put("orderby","#jcr:content/metadata/jcr:lastModified");
For further learning refer this