I am trying to transform the below JSON:
Input:
{
"steps": [
{
"end": "2023-01-27T09:19:29.849298Z",
"id": "1",
"start": "2023-01-27T09:18:24.59325Z",
"name": "foo"
},
{
"end": "2023-01-28T09:19:29.849298Z",
"id": "2",
"start": "2023-01-28T09:18:24.59325Z",
"name": "bar"
}
]
}
Output:
{
"steps": [
{
"end": "2023-01-27T09:19:29.849298Z",
"id": "1",
"name": "foo",
"start": "2023-01-27T09:18:24.59325Z"
},
{
"end": "2023-01-28T09:19:29.849298Z",
"id": "2",
"name": "bar",
"start": "2023-01-28T09:18:24.59325Z"
}
],
"date": [
{
"name": "startDate",
"value": "2023-01-27T09:18:24.59325Z" //steps[0].start
},
{
"name": "endDate",
"value": "2023-01-27T09:19:29.849298Z" //steps[0].end
}
]
}
I tried using the below spec:
[
{
"operation": "shift",
"spec": {
"steps": {
"*": "steps[]",
"0": {
"#startDate": "date[0].name",
"start": "date[0].value",
"end": "date[1].value",
"#endDate": "date[1].name"
}
}
}
}
]
But "*": "steps[]" only transforms the last element of the array steps. Please guide me as to what is wrong with the above spec, as I am new to jolt. Also, any pointers to the correct operations needed to achieve the above output will be greatly appreciated.
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"#": "&1",
"0": {
"#startDate": "date[0].name",
"start": "date[0].value",
"#endDate": "date[1].name",
"end": "date[1].value"
}
}
}
}
]
You can use # to get the steps array and put it into the steps key with &1.
The issue with the given Jolt spec is that the transformation of the steps array elements is only happening for the last element of the array. This is because the "*": "steps[]" directive only applies to the last element of the array.
To transform all elements of the steps array, you can use the following Jolt spec:
[
{
"operation": "shift",
"spec": {
"steps": {
"0": {
"start": "date[0].value",
"end": "date[1].value"
},
"*": {
"*": "steps[&1].&",
"#startDate": "date[0].name",
"#endDate": "date[1].name"
}
}
}
}
]
One option is to conditionally pick by the existing(steps) array's indexes while walking through it such as
[
{
"operation": "shift",
"spec": {
"steps": {
"#1": "",// derive the whole value from the upper level, eg. replicate it
"0": {// the first index
"#startDate": "date[&1].name",
"start": "date[&1].value"
},
"*": {// the other index(this case there's only one)
"#endDate": "date[&1].name",// bring the value of the wrapper object'S index by using &1(going up one level the tree)
"end": "date[&1].value"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Related
I have a following JSON :
{
"timestamp": 1665923527,
"place": "us",
"event": "custom",
"Users": [
{
"email": "test1#test.com",
"key1": "Admin"
},
{
"email": "test2#test.com",
"key1": "Admin"
}
]
}
which I want to modify it to multiple array based on the users key dynamically. In the above example I have 2 arrays inside user. So I want the output to be:
[
{
"timestamp": 1665923527,
"place": "us",
"event": "custom",
"email": "test1#test.com",
"key1": "Admin"
},
{
"timestamp": 1665923527,
"place": "us",
"event": "custom",
"email": "test2#test.com",
"key1": "Admin"
}
]
Is it possible with Jolt?
I did search in all the documents i found, but no luck
You can walk through the attributes of the innermost array(Users) by using a shift transformation spec while copying the values of the others grabbed from after going two levels up the tree such as
[
{
"operation": "shift",
"spec": {
"Users": {
"*": {
"#(2,timestamp)": "[&1].timestamp", // go 2 levels up the tree to grab the value of timestamp by using "#(2,timestamp)", [&1] stands for going 1 level up to reach the level of "Users" array to combine all values at that level
"#(2,place)": "[&1].place",
"#(2,event)": "[&1].event",
"*": "[&1].&"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Edit : alternatively you can prefer the below one to make it dynamic :
[
{
// separate the attributes into two groups "x" and "y"
"operation": "shift",
"spec": {
"Users": {
"*": {
"*": {
"#": "x.&2.&"
}
}
},
"*": "y.&"
}
},
{
// combine each attributes under common index values
"operation": "shift",
"spec": {
"x": {
"*": {
"#(2,y)": "&2.&"
},
"#": "&"
}
}
},
{
// dive deep into indexes
"operation": "shift",
"spec": {
"x": {
"*": {
"*": {
"*": "[&2].&"
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I wanted to know if there is way using jolt to select only fields that contain the value of another ?
Here is my input
{
"type": "A",
"field1": "value1",
"field2": "value2",
"example1-A1[zone=fr,method=A]": "80",
"example2-A1[zone=fr,method=A]": "90",
"example1-B1[zone=fr,method=B]": "50",
"example2-B1[zone=fr,method=B]": "10"
}
My output will depend on the value in the "type" field and check the method if it is equal to that type(eg: A) in order to select the values and will be like
{
"type": "A",
"field1": "value1",
"field2": "value2",
"new-filed1": "80",
"new-field2": "90"
}
Any help would be appreciate
Answer for the updated question:
[
{
"operation": "shift",
"spec": {
"*-*,method=*]": "fieldsByMethod.&(0,3).&(0,1)",
"type": [
"&",
"otherFields.&"
],
"*": "otherFields.&"
}
},
{
"operation": "shift",
"spec": {
"type": {
"*": {
"#(2,fieldsByMethod.&)": "&1"
}
},
"otherFields": "otherFields"
}
},
{
"operation": "shift",
"spec": {
"*": {
"example*": "new-field&(0,1)",
"*": "&"
}
}
}
]
Comparing the old and the updated input we have three additional requirements:
Fields without the ,method= substring should also be in the output.
For that we can temporarily place them in otherFields subdocument.
The type field should also be in the output:
That's why
"type": [
"&",
"otherFields.&"
]
comes in handy. & rewrites the type field to it's original place, so we can use it as in the old answer. otherFields.& puts it also to the otherFields subdocument.
Rename the example* fields to new-field*. That happens in the last operation
Old answer for the spec:
{
"type": "A",
"example1-A1[zone=fr,method=A]": "80",
"example2-A1[zone=fr,method=A]": "90",
"example1-B1[zone=fr,method=B]": "50",
"example2-B1[zone=fr,method=B]": "10"
}
and the expected output:
{
"example1": "80",
"example2": "90"
}
This spec produces the expected output for the example:
[
{
"operation": "shift",
"spec": {
"type": "type",
"*-*,method=*]": "fieldsByMethod.&(0,3).&(0,1)"
}
},
{
"operation": "shift",
"spec": {
"type": {
"*": {
"#(2,fieldsByMethod.&)": "&1"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "&"
}
}
}
]
To understand it, try to apply it step by step.
For the second (I think the trickiest one) operation I found the issue https://github.com/bazaarvoice/jolt/issues/480 that helped me to create it.
To understand the first operation, try to analyze the https://jolt-demo.appspot.com/#prefixSoupToBuckets example.
I have JSON Input as below
{
"data": [
{
"items": [
{
"item": "R11",
"code": "8611A"
},
{
"item": "R2",
"code": "8611B"
}
]
}
]
}
I need to extract items array as below
[
{
"item": "R11",
"code": "8611A"
},
{
"item": "R2",
"code": "8611B"
}
]
Please copy and paste the above INPUT and OUTPUT https://jolt-demo.appspot.com/#inception
I have tested [{"operation":"shift","spec":{"data":{"*":""}}}]
But it returns {"items" : [ {...}, {...} ] }
You can use a shift transformation as below
[
{
"operation": "shift",
"spec": {
"*": { //might be replaced with "data"
"*": {
"*": //might be replaced with "items"
{
"#": ""
}
}
}
}
}
]
where the asterisks wildcards represent the array data, its indexes, and the array items respectively. The "#":"" represents the items as array of objects with no key.
Alternative Spec:
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"items": ""
}
}
}
}
]
Here is the answer
[{
"operation": "shift",
"spec": {
"data": {
"0": {
"*": ""
}
}
}
}]
For input
{
"field1": {
"array1": [
"a",
"b"
],
"array2": [
"c",
"d"
]
}
}
I want the output:
{
"objectArray": [
{
"role": "to",
"id": "a"
},
{
"role": "to",
"id": "b"
},
{
"role": "to",
"id": "c"
},
{
"role": "to",
"id": "d"
}
]
}
I currently am trying:
[
{
"operation": "shift",
"spec": {
"field1": {
"array1": {
"#to": "objectArray[].role",
"#": "objectArray[].id"
},
"array2": {
"#to": "objectArray[].role",
"#": "objectArray[].id"
}
}
}
}
]
However this seems to separate out the roles and ids into their own separate objects, after the first one.
You can use this two step of shift transformation spec.
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "o[].id"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "objectArray[&1].&",
"#to": "objectArray[&1].role"
}
}
}
}
]
where you got and array for all ids in the first step, and then adding an arbitrarily chose key-value pairs within the second step while renaming the target array
For a student project I have to improve data quality. The first step is to request an API. Secondly, we have to edit the json structure.
This is the response from the API :
{
"lists": [
[
0,
451,
"test",
"953"
],
[
2,
1010,
"hello",
"610"
]
]
}
Now using jolt I want to have a result like that :
{
"lists": [
{
"id": 0,
"clientId": 451,
"name": "test",
"custom_value": "953"
},
{
"id": 2,
"clientId": 1010,
"name": "hello",
"custom_value": "610"
}
]
}
Currently, I can access to data values but I don't know how to separate it into array with objects.
My 'code' :
[
{
"operation": "shift",
"spec": {
"lists": {
"*": {
"*": {
"*": {
"$0": "lists"
}
}
}
}
}
}
]
Where I'm wrong and how can I edit the structure of the original array properly?
Spec
[
{
"operation": "shift",
"spec": {
"lists": {
"*": { // lists array
"0": "lists[&1].id",
"1": "lists[&1].clientId",
"2": "lists[&1].name",
"3": "lists[&1].custom_value"
}
}
}
}
]