I have some issue finding the right way to filter in my input json file using jolt as described in the following:
Input :
{
"test1": 2242,
"test2": 23456,
"description": "desc, desc, desc",
"name1": "foo",
"active": true,
"date1": "xx-xx-xx",
"date2": "xx-xx-xx",
"grade": "A",
"quantity": 10,
"information": "info, info, info",
"favorite": "Y",
"maxValue": 100,
"approachTypeMin[zone=MAR, method=a]": "12",
"approachTypeMax[zone=MAR, method=a]": "50",
"approachTypeMin[zone=NOR, method=b]": "3",
"approachTypeMax[zone=NOR, method=b]": "10"
}
This is my Spec
[
{
"operation": "shift",
"spec": {
"test1": "value1",
"test2": "value2",
"description": "desc1",
"name1": "nameValue",
"active": {
"true": {
"#Active": "status"
},
"false": {
"#Inactive": "status"
}
},
"date1": "beginDate",
"date2": "endDate",
"grade": {
"*": {
"$": "testGrade.value",
"#(2,type)": "testGrade.type",
"#2": "testGrade.defaultValue"
}
},
"quantity": "qtn",
"additionalInfo": "information",
"favorite": "fvt",
"maxValue": "max",
"approachType*, method=*]": "test.&(0,2)"
}
}
]
This is my actual Output
{
"value1" : 2242,
"value2" : 23456,
"desc1" : "desc, desc, desc",
"nameValue" : "foo",
"status" : "Active",
"beginDate" : "xx-xx-xx",
"endDate" : "xx-xx-xx",
"testGrade" : {
"value" : "A",
"defaultValue" : "2"
},
"qtn" : 10,
"fvt" : "Y",
"max" : 100,
"test" : {
"A" : [ "12", "50" ],
"B" : [ "3", "10" ]
}
}
My issue is that i can't find a way to get something like the result above depending on the grade(e.g grade A==Approach method):
{
"value1": 2242,
"value2": 23456,
"desc1": "desc, desc, desc",
"nameValue": "foo",
"status": "Active",
"beginDate": "xx-xx-xx",
"endDate": "xx-xx-xx",
"value": "X",
"testGrade": {
"type": "A",
"defaultValue": "2"
},
"qtn": "10",
"fvt": "Y",
"max": "100",
"convertedApproachMin": 12,
"convertedApproachMax": 50
}
Thanks for any help!
Need to divide key names starting with approachType one more time in order to seperate them by means of Min and Max, to derive the lower case grade to match the extracted methods (a and b).
So, use the following specs :
[
{
"operation": "shift",
"spec": {
"test1": "value1",
"test2": "value2",
"description": "desc1",
"name1": "nameValue",
"active": {
"true": {
"#Active": "status"
},
"false": {
"#Inactive": "status"
}
},
"date1": "beginDate",
"date2": "endDate",
"grade": "&",
"quantity": "qtn",
"additionalInfo": "information",
"favorite": "fvt",
"maxValue": "max",
"approachType*\\[zone*, method=*]": "cA&(0,1).&(0,3)"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"grade_l": "=toLower(#(1,grade))"
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"grade_l": {
"*": {
"#(2,grade)": "testGrade.value",
"#2": "testGrade.defaultValue",
"#(2,cAMin.&)": "convertedApproachMin",
"#(2,cAMax.&)": "convertedApproachMax"
}
}
}
},
{
"operation": "remove",
"spec": {
"grade": "",
"cA*": ""
}
}
]
the demo on the site http://jolt-demo.appspot.com/ :
Related
I am trying to transform the below JSON:
Input:
{
"steps": [
{
"end": "2023-01-27T09:19:29.849298Z",
"id": "1",
"start": "2023-01-27T09:18:24.59325Z",
"name": "foo"
},
{
"end": "2023-01-28T09:19:29.849298Z",
"id": "2",
"start": "2023-01-28T09:18:24.59325Z",
"name": "bar"
}
]
}
Output:
{
"steps": [
{
"end": "2023-01-27T09:19:29.849298Z",
"id": "1",
"name": "foo",
"start": "2023-01-27T09:18:24.59325Z"
},
{
"end": "2023-01-28T09:19:29.849298Z",
"id": "2",
"name": "bar",
"start": "2023-01-28T09:18:24.59325Z"
}
],
"date": [
{
"name": "startDate",
"value": "2023-01-27T09:18:24.59325Z" //steps[0].start
},
{
"name": "endDate",
"value": "2023-01-27T09:19:29.849298Z" //steps[0].end
}
]
}
I tried using the below spec:
[
{
"operation": "shift",
"spec": {
"steps": {
"*": "steps[]",
"0": {
"#startDate": "date[0].name",
"start": "date[0].value",
"end": "date[1].value",
"#endDate": "date[1].name"
}
}
}
}
]
But "*": "steps[]" only transforms the last element of the array steps. Please guide me as to what is wrong with the above spec, as I am new to jolt. Also, any pointers to the correct operations needed to achieve the above output will be greatly appreciated.
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"#": "&1",
"0": {
"#startDate": "date[0].name",
"start": "date[0].value",
"#endDate": "date[1].name",
"end": "date[1].value"
}
}
}
}
]
You can use # to get the steps array and put it into the steps key with &1.
The issue with the given Jolt spec is that the transformation of the steps array elements is only happening for the last element of the array. This is because the "*": "steps[]" directive only applies to the last element of the array.
To transform all elements of the steps array, you can use the following Jolt spec:
[
{
"operation": "shift",
"spec": {
"steps": {
"0": {
"start": "date[0].value",
"end": "date[1].value"
},
"*": {
"*": "steps[&1].&",
"#startDate": "date[0].name",
"#endDate": "date[1].name"
}
}
}
}
]
One option is to conditionally pick by the existing(steps) array's indexes while walking through it such as
[
{
"operation": "shift",
"spec": {
"steps": {
"#1": "",// derive the whole value from the upper level, eg. replicate it
"0": {// the first index
"#startDate": "date[&1].name",
"start": "date[&1].value"
},
"*": {// the other index(this case there's only one)
"#endDate": "date[&1].name",// bring the value of the wrapper object'S index by using &1(going up one level the tree)
"end": "date[&1].value"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Working with Nifi JoltTransformJSON processor, fairly new to using JOLT Specs and can't figure out how to move a set of {} from each item of an array to a single set of {} around all items of the array form the output JSON. Please see the OUTPUT and the NEEDED OUTPUT sections below for comparisons. I tried creating the spec from reading the JOLT definition and some examples which got me pretty close, but this last part has been kicking my rear for a while, any hints would be awesome!
INPUT JSON:
{
"AssetID": "1",
"AssetNumber": "2",
"AssetMaterial": "Cisco MDS 9706",
"RackUnits": "9.0",
"MaterialType": "Chassis",
"AssetName": "Cisco-MDS-9706_1",
"CustRID": "A001",
"SerialNumber": "OU812",
"Room": "ROOM5",
"Datacenter": "DC69",
"UMountingID": "86",
"CabinetAssetID": "181",
"CabinetName": "CAB666"
}
CURRENT SPEC:
[
{
"operation": "shift",
"spec": {
"AssetID": "data[].6.value",
"AssetNumber": "data[].7.value",
"AssetMaterial": "data[].8.value",
"AssetName": "data[].9.value",
"CustRID": "data[].10.value",
"SerialNumber": "data[].11.value",
"Room": "data[].12.value",
"Datacenter": "data[].13.value",
"UMountingID": "data[].14.value",
"CabinetAssetID": "data[].15.value",
"CabinetName": "data[].16.value"
}
},
{
"operation": "default",
"spec": {
"to": "table1"
}
},
{
"operation": "default",
"spec": {
"fieldsToReturn": [6, 7, 8, 9, 10, 11, 12]
}
}
]
OUTPUT JSON:
{
"data": [
{
"6": {
"value": "1"
}
},
{
"7": {
"value": "2"
}
},
{
"8": {
"value": "Cisco MDS 9706"
}
},
{
"9": {
"value": "Cisco-MDS-9706_1"
}
},
{
"10": {
"value": "A001"
}
},
{
"11": {
"value": "OU812"
}
},
{
"12": {
"value": "ROOM5"
}
},
{
"13": {
"value": "DC69"
}
},
{
"14": {
"value": "86"
}
},
{
"15": {
"value": "181"
}
},
{
"16": {
"value": "CAB666"
}
}
],
"to": "table1",
"fieldsToReturn": [ 6, 7, 8, 9, 10, 11, 12 ]
}
REQUIRED/EXPECTED OUTPUT:
{
"data" : [
{
"6" : {
"value" : "1"
},
"7" : {
"value" : "2"
},
"8" : {
"value" : "Cisco MDS 9706"
},
"9" : {
"value" : "Cisco-MDS-9706_1"
},
"10" : {
"value" : "A001"
},
"11" : {
"value" : "OU812"
},
"12" : {
"value" : "ROOM5"
},
"13" : {
"value" : "DC69"
},
"14" : {
"value" : "86"
},
"15" : {
"value" : "181"
},
"16" : {
"value" : "CAB666"
}
}
],
"to" : "table1",
"fieldsToReturn" : [ 6, 7, 8, 9, 10, 11, 12 ]
}
What you need seems that the top level object should reside at the common index of the array data, so use index 0 such as data[0]. to combine those object as single object such as
[
{
"operation": "shift",
"spec": {
"AssetID": "data[0].6.value",
"AssetNumber": "data[0].7.value",
"AssetMaterial": "data[0].8.value",
...
...
"#table1": "to"
}
},
{
"operation": "default",
"spec": {
"fieldsToReturn": [6, 7, 8, 9, 10, 11, 12]
}
},
{
"operation": "sort"
}
]
btw, the first default transformation spec is superfluous, rather use
"#table1": "to"
within the shift transformation spec
For input
{
"field1": {
"array1": [
"a",
"b"
],
"array2": [
"c",
"d"
]
}
}
I want the output:
{
"objectArray": [
{
"role": "to",
"id": "a"
},
{
"role": "to",
"id": "b"
},
{
"role": "to",
"id": "c"
},
{
"role": "to",
"id": "d"
}
]
}
I currently am trying:
[
{
"operation": "shift",
"spec": {
"field1": {
"array1": {
"#to": "objectArray[].role",
"#": "objectArray[].id"
},
"array2": {
"#to": "objectArray[].role",
"#": "objectArray[].id"
}
}
}
}
]
However this seems to separate out the roles and ids into their own separate objects, after the first one.
You can use this two step of shift transformation spec.
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "o[].id"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "objectArray[&1].&",
"#to": "objectArray[&1].role"
}
}
}
}
]
where you got and array for all ids in the first step, and then adding an arbitrarily chose key-value pairs within the second step while renaming the target array
I am trying to do Transform the jolt but I face problems in the nested array. I am not getting the description value.
I put some code
JSON Input is
{
"id": 3,
"name": "Sample Product",
"attribute_set_id": 4,
"price" : 10,
"custom_attributes": [
{
"attribute_code": "image",
"value": "/1/_/1.jpg"
},
{
"attribute_code": "description",
"value": "<p>This is Sample Product for test</p>"
}
]
}
And my Jolt Spec is
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"description": "custom_attributes[0].attribute_code.value"
}
]
My expected output is
{
"id" : 3,
"name" : "Sample Product",
"description" : "<p>This is Sample Product for test</p>",
"image" : "/1/_/1.jpg",
"start_price" : 10,
"current_price" : 10
}
If "attribute_code": "image" and "attribute_code": "description" always appears in the 1st and 2nd element of the array, you can simply transform the given JSON string with following Jolt Spec:
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"custom_attributes": {
"1": {
"value": "description"
},
"0": {
"value": "image"
}
},
"price": ["start_price", "current_price"]
}
}
]
I have the following JSON:
{
"items": [
{
"id": "1",
"name": "John",
"location": {
"town": {
"id": "10"
},
"address": "600 Fake Street",
},
"creation_date": "2010-01-19",
"last_modified_date": "2017-05-18"
},
{
"id": "2",
"name": "Sarah",
"location": {
"town": {
"id": "10"
},
"address": "76 Evergreen Street",
},
"creation_date": "2010-01-19",
"last_modified_date": "2017-05-18"
},
{
"id": "3",
"name": "Hamed",
"location": {
"town": {
"id": "20"
},
"address": "50 East A Street",
},
"creation_date": "2010-01-19",
"last_modified_date": "2017-05-18"
}
]
}
And I need to get something like this, count how many times each townId appears:
[ { "10": 2 }, {"20": 1 }]
I'm trying to find the most eficient way to do this. Any idea?
Most efficient way is to load the String in a StringBuilder and remove all line breaks and white spaces. Then search for index of "town":{"id":" string (town start index) and then search for the end index (String `"}'). Using the 2 indexes you can extract town ids and count them.
No need to deserialize the JSON into POJO objects:) and extract values by xpath from the POJOs.