We Keep Coding

Need JOLT spec for array input JSON

I am working on transforming a complex json using JOLT.
Input JSON:
{ "data":
[
{
"fieldname": "Name",
"fieldvalue": [ "John Doe" ]
},
{ "fieldname": "Title",
"fieldvalue": [ "Manager" ]
},
{ "fieldname": "Company",
"fieldvalue": [ "Walmart" ]
}
] }
Expected Output:
{
"finalPayload":{
"PI":{
"EmpName":"John Doe",
"EmpRole":"Manager"
},
"Company":"Walmart"
}
}
I am unable to understand how to access and assign "fieldvalue" in output based on "fieldname". Please help me with the JOLT spec.
Note: The order of name, title and company in input JSON will be jumbled and random meaning its not mandatory that under "data" array first object will be related to "Name" only.

Hi hope this helps you in resolving your issue.
You can have condition in Jolt too, by going inside the variable and checking the fieldname.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"fieldname": {
"Name": {
"#(2,fieldvalue)": "finalPayload.PI.EmpName"
},
"Title": {
"#(2,fieldvalue)": "finalPayload.PI.EmpRole"
},
"Company": {
"#(2,fieldvalue)": "finalPayload.Company"
}
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"finalPayload": {
"PI": {
"EmpName": "ONE",
"EmpRole": "ONE"
},
"Company": "ONE"
}
}
}
]

May I introduce an alternative library to solve the issue.
https://github.com/octomix/josson
implementation 'com.octomix.josson:josson:1.3.21'
-------------------------------------------------
Josson josson = Josson.fromJsonString(
"{\"data\":[{\"fieldname\":\"Name\",\"fieldvalue\":[\"JohnDoe\"]},{\"fieldname\":\"Title\",\"fieldvalue\":[\"Manager\"]},{\"fieldname\":\"Company\",\"fieldvalue\":[\"Walmart\"]}]}");
JsonNode node = josson.getNode(
"map(" +
" finalPayload: map(" +
" PI: map(" +
" EmpName: data[fieldname='Name'].fieldvalue[0]," +
" EmpRole: data[fieldname='Title'].fieldvalue[0]" +
" )," +
" Company: data[fieldname='Company'].fieldvalue[0]" +
" )" +
")");
System.out.println(node.toPrettyString());
Output
{
"finalPayload" : {
"PI" : {
"EmpName" : "JohnDoe",
"EmpRole" : "Manager"
},
"Company" : "Walmart"
}
}

How to merge two JSON string

I have two JSON strings, I want to merge these two response string into a single one. Is there any way to merge these two JSON string using java ?
String str1 = "{
"data" : {
"values" : {
"name" : "kiran",
"age" : "24"
}
}
}"
String str2 = "{
"data" : {
"values" : {
"name" : "Mamu",
"age" : "26"
}
}
}"
I wnat to merge these two JSON string as follows
String mergeResult = "{
"data" : {
"values" : [
{
"name" : "kiran",
"age" : "24"
},
{
"name" : "Manu",
"age" : "26"
}
]
}
}"

From your example JSON it looks like there can be many more input objects than two, so I'd use a JSON to JSON transform via JOLT library (https://github.com/bazaarvoice/jolt) as follows:
Form a JSON array of all the input {"data" : ...} objects (either by collecting the original objects and putting them in a List before serialization or just manually by concatenating their JSONs with square brackets):
[
{
"data": {
"values": {
"name": "kiran",
"age": "24"
}
}
},
{
"data": {
"values": {
"name": "Mamu",
"age": "26"
}
}
}
]
Use the JOLT spec:
[
{
"operation": "shift",
"spec": {
"*": {
"data": {
"values": {
"#": "data.values[]"
}
}
}
}
}
]
The resulting JSON:
{
"data" : {
"values" : [ {
"name" : "kiran",
"age" : "24"
}, {
"name" : "Mamu",
"age" : "26"
} ]
}
}
You can test or modify the spec yourself at http://jolt-demo.appspot.com before using it in your application.
Here's what Java side might look like:
Resource transformationSpec = ...
String inputJson = ...
List<Object> specs = JsonUtils.jsonToList(transformationSpec.getInputStream());
Chainr chainr = Chainr.fromSpec(specs);
Object inputObject = JsonUtils.jsonToObject(inputJson);
Object transformedObject = chainr.transform(inputObject);
String transformedJson = JsonUtils.toJsonString(transformedObject);

Library Josson & Jossons has concatenate operation to combine array from datasets.
https://github.com/octomix/josson
Jossons jossons = new Jossons();
jossons.putDataset("resp1", Josson.fromJsonString(
"{" +
" \"data\": {" +
" \"values\": {" +
" \"name\": \"kiran\"," +
" \"age\": \"24\"" +
" }" +
" }" +
"}"));
jossons.putDataset("resp2", Josson.fromJsonString(
"{" +
" \"data\": {" +
" \"values\": {" +
" \"name\": \"Mamu\"," +
" \"age\": \"26\"" +
" }" +
" }" +
"}"));
JsonNode mergeValues = jossons.evaluateQuery(
"resp1->data.toArray() <+< resp2->data.toArray()");
System.out.println(mergeValues.toPrettyString());
JsonNode mergeResult = Josson.create(mergeValues).getNode(
"toObject('values').toObject('data')");
System.out.println(mergeResult.toPrettyString());
mergeValues
[ {
"name" : "kiran",
"age" : "24"
}, {
"name" : "Mamu",
"age" : "26"
} ]
mergeResult
{
"data" : {
"values" : [ {
"name" : "kiran",
"age" : "24"
}, {
"name" : "Mamu",
"age" : "26"
} ]
}
}

May be not the most efficient solution but very simple one would be to parse each String into a Map (or specific POJO if you have one) and merge them as maps, and than serialize back into Json String. To parse a json string to map or a specific POJO you can use Jackson library - method readValue() of ObjectMapper Or Gson library. Also, I wrote my own simple wrapper over Jackson library that simplifies the use. Class JsonUtils is available as part of MgntUtils library (written and maintained by me). This class just has the methods that allow you to parse and de-serialize from/to Json String from/to a class instance. So your code could be as follows:
public String mergeJsonStrings(String str1, Str2) throws IOException {
Map<String, Object> map1 = convertJsonStringToMap(str1);
Map<String, Object> map2 = convertJsonStringToMap(str2);
Map<String, Object> dataMap1 = (Map<String, Object>)map1.get("data");
Map<String, Object> valuesMap1 = (Map<String, Object>)dataMap1.get("values");
Map<String, Object> dataMap2 = (Map<String, Object>)map2.get("data");
Map<String, Object> valuesMap2 = (Map<String, Object>)dataMap2.get("values");
valuesMap1.putAll(valuesMap2);
return convertMapToJsonString(map1);
}
Map<String, Object> convertJsonStringToMap(String str) throws IOException {
return JsonUtils.readObjectFromJsonString(str, Map.class);
}
Strong convertMapToJsonString(Map<String,Object> map) throws JsonProcessingException{
return JsonUtils.writeObjectToJsonString(map);
}
This example uses MgntUtils library to parse and serialize JSON but of course you can use Jackson, Gson or any other library you want. MgntUtils library available as Maven artifact and on Github (including source code and Javadoc)

Jolt don't transform json to another json

I am writing a project using Java 11, spring boot 2.1.9 and jolt (core and utils) 0.1.1 on Ubuntu
I have next JSON source:
{
"restaurant": {
"rating": {
"value": 3
},
"address": {
"value": "India"
}
}
}
And I have the following spec file:
[
{
"operation": "shift",
"spec": {
"restaurant": {
"rating": {
"value": "Restaurant Rating"
}
}
}
}
]
On the output I expect next json:
{
"Restaurant Rating" : 3
}
I wrote a simple test to check this:
List<Object> specs = JsonUtils.jsonToList(Files.readString(Paths.get("path/to/spec.json")));
Chainr converter = Chainr.fromSpec(specs);
Object inputJSON = JsonUtils.toJsonString(Files.readString(Paths.get("path/to/example.json")));
Object transformedOutput = converter.transform(inputJSON);
System.out.println(JsonUtils.toPrettyJsonString(transformedOutput));
But variable transformedOutput always is null
Please tell me what I'm doing wrong.

jsonToObject will get input json as object and map with spec
List specs = JsonUtils.jsonToList(Files.readString(Paths.get("path/to/spec.json")));
Chainr converter = Chainr.fromSpec(specs);
Object inputJSON = JsonUtils.jsonToObject(Files.readString(Paths.get("path/to/example.json")));
Object transformedOutput = converter.transform(inputJSON);
System.out.println(JsonUtils.toPrettyJsonString(transformedOutput));
----output-----
{
"Restaurant Rating" : 3
}

Read out elements of a string in specific order

For some reasons I have to use a specific string in my project. This is the text file (it's a JSON File):
{"algorithm":
[
{ "key": "onGapLeft", "value" : "moveLeft" },
{ "key": "onGapFront", "value" : "moveForward" },
{ "key": "onGapRight", "value" : "moveRight" },
{ "key": "default", "value" : "moveBackward" }
]
}
I've defined it in JAVA like this:
static String input = "{\"algorithm\": \n"+
"[ \n" +
"{ \"key\": \"onGapLeft\", \"value\" : \"moveLeft\" }, \n" +
"{ \"key\": \"onGapFront\", \"value\" : \"moveForward\" }, \n" +
"{ \"key\": \"onGapRight\", \"value\" : \"moveRight\" }, \n" +
"{ \"key\": \"default\", \"value\" : \"moveBackward\" } \n" +
"] \n" +
"}";
Now I have to isolate the keys and values in an array:
key[0] = onGapLeft; value[0] = moveLeft;
key[1] = onGapFront; value[1] = moveForward;
key[2] = onGapRight; value[2] = moveRight;
key[3] = default; value[3] = moveBackward;
I'm new to JAVA and don't understand the string class very well. Is there an easy way to get to that result? You would help me really!
Thanks!
UPDATE:
I didn't explained it well enough, sorry. This program will run on a LEGO NXT Robot. JSON won't work there as I want it to so I have to interpret this JSON File as a normal STRING! Hope that explains what I want :)

I propose a solution in several step.
1) Let's get the different parts of your ~JSON String. We will use a pattern to get the different {.*} parts :
public static void main(String[] args) throws Exception{
List<String> lines = new ArrayList<String>();
Pattern p = Pattern.compile("\\{.*\\}");
Matcher matcher = p.matcher(input);
while (matcher.find()) {
lines.add(matcher.group());
}
}
(you should take a look at Pattern and Matcher)
Now, lines contains 4 String :
{ "key": "onGapLeft", "value" : "moveLeft" }
{ "key": "onGapFront", "value" : "moveForward" }
{ "key": "onGapRight", "value" : "moveRight" }
{ "key": "default", "value" : "moveBackward" }
Given a String like one of those, you can remove curly brackets with a call to String#replaceAll();
List<String> cleanLines = new ArrayList<String>();
for(String line : lines) {
//replace curly brackets with... nothing.
//added a call to trim() in order to remove whitespace characters.
cleanLines.add(line.replaceAll("[{}]","").trim());
}
(You should take a look at String String#replaceAll(String regex))
Now, cleanLines contains :
"key": "onGapLeft", "value" : "moveLeft"
"key": "onGapFront", "value" : "moveForward"
"key": "onGapRight", "value" : "moveRight"
"key": "default", "value" : "moveBackward"
2) Let's parse one of those lines :
Given a line like :
"key": "onGapLeft", "value" : "moveLeft"
You can split it on , character using String#split(). It will give you a String[] containing 2 elements :
//parts[0] = "key": "onGapLeft"
//parts[1] = "value" : "moveLeft"
String[] parts = line.split(",");
(You should take a look at String[] String#split(String regex))
Let's clean those parts (remove "") and assign them to some variables:
String keyStr = parts[0].replaceAll("\"","").trim(); //Now, key = key: onGapLeft
String valueStr = parts[1].replaceAll("\"","").trim();//Now, value = value : moveLeft
//Then, you split `key: onGapLeft` with character `:`
String key = keyStr.split(":")[1].trim();
//And the same for `value : moveLeft` :
String value = valueStr.split(":")[1].trim();
That's it !
You should also take a look at Oracle's tutorial on regular expressions (This one is really important and you should invest time on it).

You need to use a JSON parser library here. For example, with org.json you could parse it as
String input = "{\"algorithm\": \n"+
"[ \n" +
"{ \"key\": \"onGapLeft\", \"value\" : \"moveLeft\" }, \n" +
"{ \"key\": \"onGapFront\", \"value\" : \"moveForward\" }, \n" +
"{ \"key\": \"onGapRight\", \"value\" : \"moveRight\" }, \n" +
"{ \"key\": \"default\", \"value\" : \"moveBackward\" } \n" +
"] \n" +
"}";
JSONObject root = new JSONObject(input);
JSONArray map = root.getJSONArray("algorithm");
for (int i = 0; i < map.length(); i++) {
JSONObject entry = map.getJSONObject(i);
System.out.println(entry.getString("key") + ": "
+ entry.getString("value"));
}
Output :
onGapLeft: moveLeft
onGapFront: moveForward
onGapRight: moveRight
default: moveBackward

Parsing nested JSON

I have the following JSON:
{
"registration": {
"name": "Vik Kumar",
"first_name": "Vik",
"last_name": "Kumar",
"bloodGroup": "B-",
"gender": "male",
"birthday": "10\/31\/1983",
"email": "vik.ceo\u0040gmail.com",
"cellPhone": "1234123456",
"homePhone": "1234123457",
"officePhone": "1234123458",
"primaryAddress": "jdfjfgj",
"area": "jfdjdfj",
"location": {
"name": "Redwood Shores, California",
"id": 103107903062719
},
"subscribe": true,
"eyePledge": false,
"reference": "fgfgfgfg"
}
}
I am using the following code to parse it:
JsonNode json = new ObjectMapper().readTree(jsonString);
JsonNode registration_fields = json.get("registration");
Iterator<String> fieldNames = registration_fields.getFieldNames();
while(fieldNames.hasNext()){
String fieldName = fieldNames.next();
String fieldValue = registration_fields.get(fieldName).asText();
System.out.println(fieldName+" : "+fieldValue);
}
This works fine and it print all the values except for location which is kind of another level of nesting. I tried the same trick as above code to pass json.get("location") but that does not work. Please suggest how to make it work for location.

You need to detect when you are dealing with a (nested) Object using JsonNode#isObject:
public static void printAll(JsonNode node) {
Iterator<String> fieldNames = node.getFieldNames();
while(fieldNames.hasNext()){
String fieldName = fieldNames.next();
JsonNode fieldValue = node.get(fieldName);
if (fieldValue.isObject()) {
System.out.println(fieldName + " :");
printAll(fieldValue);
} else {
String value = fieldValue.asText();
System.out.println(fieldName + " : " + value);
}
}
}
Thus, when you reach an object, such as location, you'll call the printAll recursively to print all its inner values.
org.codehaus.jackson.JsonNode json = new ObjectMapper().readTree(jsonString);
org.codehaus.jackson.JsonNode registration_fields = json.get("registration");
printAll(registration_fields);

Since location is nested within registration, you need to use:
registration_fields.get("location");
to get it. But isn't it already processed by the while-loop, why do you need to get it separately?