Jolt nested array transformation - java

I'm a beginner in jolt transformations and I'm having below input payload that needs to be transformed before sending out
i want to create separate node based on nested array values and insert it back into array and remove the nested array completely from the output
{
"Def": {
"capacity": {
"accounttype": {
"value": "customer"
},
"customer": {
"name": "abc"
},
"config": [
{
"restriction": {
"value": "inbound"
},
"serviceConfig": [
{
"serviceType": {
"value": "standard"
}
},
{
"serviceType": {
"value": "special"
}
}
]
}
]
}
}
}
and expected output is below
{
"Def": {
"capacity": {
"accounttype": {
"value": "customer"
},
"customer": {
"name": "abc"
},
"config": [
{
"restriction": {
"value": "inbound"
},
"serviceType": {
"value": "standard"
}
},
{
"restriction": {
"value": "inbound"
},
"serviceType": {
"value": "special"
}
}
]
}
}
}
I did try multiple things but none of them seems to work,

You can use a shift transformation spec in which walk through the indexes of serviceConfig array and then tile the other elements by using "*": "&2.&1.&" (as else case) such as
[
{
"operation": "shift",
"spec": {
"Def": {
"capacity": {
"*": "&2.&1.&", // &2 stands for going the tree 2 levels up and grabbing the value "Def", &1 for "capacity", & as leaf node which copies the current node's value
"config": {
"*": {
"serviceConfig": {
"*": {
"#(2,restriction)": "&6.&5.&4[&1].restriction",
"*": "&6.&5.&4[&1].&" // &6 -> "Def", &5 -> "capacity", &4 -> "config", [&1] -> indexes of serviceConfig, & -> leaf node which copies the current node's value
}
}
}
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is

Related

JOLT shift transformation: collect the items from all the levels and change the property name

I am trying to transform a JSON using Jolt transformation looking for some input here.
I am trying to get a part of the items into an one array.
My goal is to get an array that contains a part of the items.
Here is my input and expected output:
Input:
{
"item": [
{
"ac": "i",
"id": "c5b9e16076fe4faaaace5c7d0cbe0d9b",
"foo": {
"id": "c5b9e16076fe4faaaace5c7d0cbe0d9b",
"nn": "de38c8a2e6a844d1a9dd7a573eedee15"
},
"item": [
{
"ac": "y",
"id": "c5b9e16076fe4faaaace5c7d0cbe0d9b",
"foo": {
"id": "c5b9e16076fe4faaaace5c7d0cbe0d9b",
"nn": "b1754500dde646f1af495814fcd2d65e"
}
},
{
"id": "r",
"ac": "ac",
"foo": {
"id": "c5b9e16076fe4faaaace5c7d0cbe0d9b",
"nn": "bfb5550a6d754892b313ab9dd8604725"
}
}
]
}
]
}
Expected output:
[
{
"ac": "y",
"foo": {
"nn": [
"b1754500dde646f1af495814fcd2d65e"
]
}
},
{
"ac": "i",
"foo": {
"nn": "de38c8a2e6a844d1a9dd7a573eedee15"
}
},
{
"ac": "ac",
"foo": {
"nn": "bfb5550a6d754892b313ab9dd8604725"
}
}
]
My spec:
[
{
"operation": "shift",
"spec": {
"item": {
"*": {
"item": {
"*": {
"item": {
"*": {
"ac": "[&1].ac",
"foo": {
"nn": "[&2].foo.&"
}
}
},
"ac": "[&1].ac",
"foo": {
"nn": "[&2].foo.&"
}
}
},
"ac": "[&1].ac",
"foo": {
"nn": "[&2].foo.&"
}
}
}
}
}
]
The result isn't what I expected, I got:
My output:
[ {
"ac" : [ "y", "i" ],
"foo" : {
"nn" : [ "b1754500dde646f1af495814fcd2d65e", "de38c8a2e6a844d1a9dd7a573eedee15" ]
}
}, {
"ac" : "ac",
"foo" : {
"nn" : "bfb5550a6d754892b313ab9dd8604725"
}
} ]
a list of the ac and list of the KK not each properties of item together in a one list of items
Can you please told me how to do it right?
It's important to determine the valid separation patterns to construct the independent objects; this case I've used "ac": "#(1,id).&1.&" and "nn": "#(3,id).&3.foo.KK" (as staying two levels inner) patterns for each construction model such as
[
{
"operation": "shift",
"spec": {
"item": {
"*": {
"item": {
"*": {
"item": {
"*": {
"ac": "#(1,id).&1.&",
"foo": {
"*": {
"nn": "#(3,id).&3.foo.KK"
}
}
}
},
"ac": "#(1,id).&1.&",
"foo": {
"*": {
"nn": "#(3,id).&3.foo.KK"
}
}
}
},
"ac": "#(1,id).&1.&",
"foo": {
"*": {
"nn": "#(3,id).&3.foo.KK"
}
}
}
}
}
},
{
// get rid of separator labels of objects
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
Edit (based on the edited question) : You can use the following spec for the new case
[
{
"operation": "shift",
"spec": {
"item": {
"*": {
"item": {
"*": {
"item": {
"*": {
"ac": "#(1,ac).&",
"foo": {
"nn": "#(2,ac).&1[#1].&"
}
}
},
"ac": "#(1,ac).&",
"foo": {
"nn": "#(2,ac).&1[#1].&"
}
}
},
"ac": "#(1,ac).&",
"foo": {
"nn": "#(2,ac).&1[#1].&"
}
}
}
}
},
{
// get rid of separator labels of objects
"operation": "shift",
"spec": {
"*": ""
}
}
]

Jolt how to include object outside list and add it to each object of list

I have an array which I have to iterate and append object that is outside the list to each of the object inside list. It is not happening properly. Since offerings is an array if it has multiple objects to each object I need to have typeId of that object.
Request:
[
{
"offers": [
{
"type": {
"id": "5"
},
"offerings": [
{
"id": "10"
},
{
"id": "9"
}
]
},
{
"type": {
"id": "3"
},
"offerings": [
{
"id": "11"
}
]
},
{
"type": {
"id": "2"
},
"offerings": [
{
"id": "14"
},
{
"id": "16"
}
]
}
]
}
]
My spec:
[
{
"operation": "shift",
"spec": {
"*": {
"offers": {
"*": {
"type": {
"id": "[&4].[&2].typeId"
},
"offerings": {
"*": {
"id": "[&3].[&1].offerId"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Response now:
[
{
"typeId": "5",
"offerId": "10"
},
{
"offerId": "9",
"typeId": "3"
},
{
"typeId": "2"
},
{
"offerId": "11"
},
{
"offerId": "14"
},
{
"offerId": "16"
}
]
Expected:
[
{
"typeId": "5",
"offerId": "10"
},
{
"typeId": "5",
"offerId": "9"
},
{
"typeId": "3",
"offerId": "11"
},
{
"typeId": "2",
"offerId": "14"
},
{
"typeId": "2",
"offerId": "16"
}
]
Please help to achieve as expected.
You can transfer "id": "[&4].[&2].typeId" key-value pair into the object tagged "offerings" key name as "#(2,type.id)": "[&3].[&1].typeid" in order to combine them as desired such like the below one
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"offerings": {
"*": {
"#(2,type.id)": "[&3].[&1].typeid",
"*": "[&3].[&1].offer&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]

Java Stream representation of extracting data from jsonobj

I have a service that returns me a response that has the below object
{
"features": [
{
"name": "climateControl"
},
{
"name": "drivingControl"
},
{
"name": "breakeControl",
"type": [
{
"name": "safety"
}
]
},
{
"name": "seatAdjustments",
"type": [
{
"name": "motion"
}
]
},
{
"name": "enginePower",
"type": [
{
"name": "motion"
}
]
}
]
}
Now I need to extract the feature names that are either null or doesn't have any type with name "safety"
I can write something like below
List<Features> features = service.getFeatures();
List<String> refinedList = new ArrayList<>();
for(Feature feature:features){
if(feature.getType==null || feature.getType().getName().equalsIgnoreCase("safety") )
{
refinedList.add(feature.getName());
}
}
Looking for a way to do the same using java streams.
You can use filter for the if conditions and map to getName, then collect as:
List<String> refinedList = features.stream()
.filter(feature -> feature.getType == null || feature.getType().getName().equalsIgnoreCase("safety"))
.map(feature -> feature.getName())
.collect(Collectors.toList());

How to count number of elements from JSON using java

I am pretty new to java and rest. I was trying to count elements from the item object using rest. If you check below Json then you will find 6 elements associated with the item object.
{
"response": {
"status": "active",
"timestamp": "2019-01-02 20:07:42"
},
"group": [
{
"best_option": "Offer",
"item": {
"F": [
{
"code": "228"
}
],
"E": [
{
"code": "228"
}
]
},
"review": {
"F": [
{
"code": "110"
},
{
"code": "212"
}
],
"E": [
{
"code": "110"
},
{
"code": "212"
}
]
}
},
{
"best_option": "Offer",
"item": {
"D": [
{
"code": "228"
}
],
"C": [
{
"code": "228"
}
]
},
"review": {
"D": [
{
"code": "110"
},
{
"code": "212"
}
],
"C": [
{
"code": "110"
},
{
"code": "212"
},
{
"code": ""
}
]
}
},
{
"best_option": "Offer",
"item": {
"A": [
{
"code": "228"
}
]
},
"review": {
"A": [
{
"code": "110"
},
{
"code": "212"
}
]
}
},
{
"best_option": "Offer",
"item": {
"B": [
{
"code": "228"
},
{
"code": "662"
}
]
},
"review": {
"B": [
{
"code": "110"
},
{
"code": "662"
}
]
}
}
]
}
I have tried jsonResponse.getBody().jsonPath().get() to count element from item object. I want to count number of elements present in the item object for e.g: A, B, C,D,E F . Can someone please help me to sort out my issue?
Iterate over "group" object, get "item" as JsonObject and get the size.
JsonObject jsonObject = (JsonObject)new JsonParser().parse(input);
JsonArray groupObject = jsonObject.getAsJsonArray("group");
int countItem=0;
for(int i=0;i<groupObject.size();i++) {
JsonObject items = ((JsonObject)groupObject.get(i)).getAsJsonObject("item");
countItem+=items.size();
}

Aggregation operation to modify the key of a property from a subdocument of documents

I have in mongo db a collection with documents like this:
{
"prop1": "val1",
"prop2": {
"prop21": {
"toBeChanged1_xyz": "val2",
"toBeChanged2_zyx": "val3",
},
"prop22": {
"whatever": "something"
}
}
}
Trying to create aggregation operation in Java Spring to remove from the keys of prop21's properties the substring after _. This if what I need to have:
{
"prop1": "val1",
"prop2": {
"prop21": {
"toBeChanged1": "val2",
"toBeChanged2": "val3",
},
"prop22": {
"whatever": "something"
}
}
}
Tried something like this:
List<AggregationOperation> operations = new ArrayList<AggregationOperation>();
operations.add( Aggregation.project( "code" ).and( "code" ).minus( "_"+"$entityType" ).as( "newId" ));
It didn't worked, of course. I have no idea how can it be done or can it be done easier in other ways.
You should update your structure to below.
{
"prop1": "val1",
"prop2": [
{
"prop": "prop21",
"value": [
{
"prop": "toBeChanged1_xyz",
"value": "val2"
},
{
"prop": "toBeChanged2_zyx",
"value": "val3"
}
]
},
{
"prop": "prop22",
"value": [
{
"prop": "whatever",
"value": "something"
}
]
}
]
}
You can try below query in 3.4. The query will iterate over property elements and locate prop2 and $split to modify the property name.
[
{
"$addFields": {
"prop2": {
"$map": {
"input": "$prop2",
"as": "prop_2",
"in": {
"prop": "$$prop_2.prop",
"value": {
"$cond": [
{
"$eq": [
"$$prop_2.prop",
"prop21"
]
},
{
"$map": {
"input": "$$prop_2.value",
"as": "prop_21",
"in": {
"prop": {
"$arrayElemAt": [
{
"$split": [
"$$prop_21.prop",
"_"
]
},
0
]
},
"value": "$$prop_21.value"
}
}
},
"$$prop_2.value"
]
}
}
}
}
}
}
]
You can use the below query if you cant modify the structure from 3.4.4 version.
The query uses $arrayToObject and $objectToArray operator to modify the documents to above structure with same logic to modify the property name.
[
{
"$addFields": {
"prop2": {
"$arrayToObject": {
"$map": {
"input": {
"$objectToArray": "$prop2"
},
"as": "prop_2",
"in": {
"k": "$$prop_2.k",
"v": {
"$cond": [
{
"$eq": [
"$$prop_2.k",
"prop21"
]
},
{
"$arrayToObject": {
"$map": {
"input": {
"$objectToArray": "$$prop_2.v"
},
"as": "prop_21",
"in": {
"k": {
"$arrayElemAt": [
{
"$split": [
"$$prop_21.k",
"_"
]
},
0
]
},
"v": "$$prop_21.v"
}
}
}
},
"$$prop_2.v"
]
}
}
}
}
}
}
}
]

Categories