We Keep Coding

Transform JSON and create new one

I'm working with JSON in Java, and I want to trasnform a JSON I have as entry :
{
"index":{
"mutlipleIndices":[
{
"index":"languages",
"values":[
"English",
"German"
]
},
{
"index":"editors",
"values":[
"AC. Renus",
"Lodiga"
]
}
],
"simpleIndices":[
{
"index":"bookName",
"values":"A song Of Ice and fire"
},
{
"index":"nbPages",
"valeurs":1600
}
]
}
}
int this :
{
"data": { ... Put here the entry JSON ... },
"flatData": [
{
"key": "languages",
"type": "string",
"key_type": "languages.string",
"value_string": ["English", "German"]
},
{
"key": "editors",
"type": "string",
"key_type": "editors.string",
"value_string": ["AC. Renus", "Lodiga"]
},
{
"key": "bookName",
"type": "string",
"key_type": "bookName.string",
"value_string": "A song Of Ice and fire"
},
{
"key": "nbPages",
"type": "float",
"key_type": "nbPages.float",
"value_float": 1600
}
]
}
My entry JSON is a JsonNode (Jackson library), how can I iterate on that object and create a new one ?
Basically what I thought I'd do is iternate on the entry json, find an entry and transform it
Thanks in advance

Create 2 pojos
Mapping the input json (let's call it MyJsonInput.java)
Mapping the output you want (let's call it MyOutputObject.java).
These 2 classes should have fields which matches the json structure.
Create a mapper:
ObjectMapper objectMapper = new ObjectMapper();
Read the json and translate it into your object:
MyJsonInput myInput = objectMapper.readValue(json, MyJsonInput.class); //where json is a string with the content of your json file.
Now you have a MyJsonInput instance (myInput) containing the json data and you can do your logic to flatten it and fill your MyOutputObject.
To get the json string of MyOutputObject instance use:
String outputAsString = objectMapper.writeValueAsString(myOutputObject);
You can also threat your both input and output as a generic Map<String,Object> but in my opinion not the best way because it can get very complicated reading and parsing maps containing other maps, and not to mention that you need to cast your data into the appropriate type (string, number etc..)

https://github.com/octomix/josson
https://mvnrepository.com/artifact/com.octomix.josson/josson
implementation 'com.octomix.josson:josson:1.3.21'
-------------------------------------------------
Jossons jossons = new Jossons();
jossons.putDataset("input", Josson.fromJsonString(
"{\n" +
" \"index\":{\n" +
" \"multipleIndices\":[\n" +
" {\n" +
" \"index\":\"languages\",\n" +
" \"values\":[\n" +
" \"English\",\n" +
" \"German\"\n" +
" ]\n" +
" },\n" +
" {\n" +
" \"index\":\"editors\",\n" +
" \"values\":[\n" +
" \"AC. Renus\",\n" +
" \"Lodiga\"\n" +
" ]\n" +
" }\n" +
" ],\n" +
" \"simpleIndices\":[\n" +
" {\n" +
" \"index\":\"bookName\",\n" +
" \"values\":\"A song Of Ice and fire\"\n" +
" },\n" +
" {\n" +
" \"index\":\"nbPages\",\n" +
" \"values\":1600\n" +
" }\n" +
" ]\n" +
" }\n" +
"}"));
Map<String, String> dictionaryFinder = new HashMap<>();
// Define a dictionary function, $0 is the function parameter.
dictionaryFinder.put("flatIndices()",
"$0->map(" +
"key:index," +
"type:if(values.isArray(),values[0],values).if(isText(),'string','float')," +
"key_type:concat(index,'.',if(values.isArray(),values[0],values).if(isText(),'string','float'))," +
"value_string:values)");
// Define a left concatenate operation to combine two arrays.
dictionaryFinder.put("allIndices",
"flatIndices(input->index.multipleIndices) <+< flatIndices(input->index.simpleIndices)");
ResolverProgress progress = new ResolverProgress("\nProgress:");
// The query is a left concatenate operation to combine two objects.
JsonNode output = jossons.evaluateQueryWithResolver(
"input->toObject('data') <+< allIndices->toObject('flatData')",
dictionaryFinder::get, null, progress);
// Print result.
System.out.println(output == null ? null : output.toPrettyString());
// Print resolution progress.
System.out.println(String.join("\n", progress.getSteps()));
Output
{
"data" : {
"index" : {
"multipleIndices" : [ {
"index" : "languages",
"values" : [ "English", "German" ]
}, {
"index" : "editors",
"values" : [ "AC. Renus", "Lodiga" ]
} ],
"simpleIndices" : [ {
"index" : "bookName",
"values" : "A song Of Ice and fire"
}, {
"index" : "nbPages",
"values" : 1600
} ]
}
},
"flatData" : [ {
"key" : "languages",
"type" : "string",
"key_type" : "languages.string",
"value_string" : [ "English", "German" ]
}, {
"key" : "editors",
"type" : "string",
"key_type" : "editors.string",
"value_string" : [ "AC. Renus", "Lodiga" ]
}, {
"key" : "bookName",
"type" : "string",
"key_type" : "bookName.string",
"value_string" : "A song Of Ice and fire"
}, {
"key" : "nbPages",
"type" : "float",
"key_type" : "nbPages.float",
"value_string" : 1600
} ]
}
Progress:
Round 1 : Resolving allIndices from flatIndices(input->index.multipleIndices) <+< flatIndices(input->index.simpleIndices)
Round 1 : Resolving flatIndices(input->index.multipleIndices) from $0->map(key:index,type:if(values.isArray(),values[0],values).if(isText(),'string','float'),key_type:concat(index,'.',if(values.isArray(),values[0],values).if(isText(),'string','float')),value_string:values)
Round 1 : Resolved flatIndices(input->index.multipleIndices) = Array with 2 elements
Round 2 : Resolving flatIndices(input->index.simpleIndices) from $0->map(key:index,type:if(values.isArray(),values[0],values).if(isText(),'string','float'),key_type:concat(index,'.',if(values.isArray(),values[0],values).if(isText(),'string','float')),value_string:values)
Round 2 : Resolved flatIndices(input->index.simpleIndices) = Array with 2 elements
Round 3 : Resolved allIndices = Array with 4 elements
Round 4 : Resolved query result = Object with 2 elements

Need JOLT spec for array input JSON

I am working on transforming a complex json using JOLT.
Input JSON:
{ "data":
[
{
"fieldname": "Name",
"fieldvalue": [ "John Doe" ]
},
{ "fieldname": "Title",
"fieldvalue": [ "Manager" ]
},
{ "fieldname": "Company",
"fieldvalue": [ "Walmart" ]
}
] }
Expected Output:
{
"finalPayload":{
"PI":{
"EmpName":"John Doe",
"EmpRole":"Manager"
},
"Company":"Walmart"
}
}
I am unable to understand how to access and assign "fieldvalue" in output based on "fieldname". Please help me with the JOLT spec.
Note: The order of name, title and company in input JSON will be jumbled and random meaning its not mandatory that under "data" array first object will be related to "Name" only.

Hi hope this helps you in resolving your issue.
You can have condition in Jolt too, by going inside the variable and checking the fieldname.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"fieldname": {
"Name": {
"#(2,fieldvalue)": "finalPayload.PI.EmpName"
},
"Title": {
"#(2,fieldvalue)": "finalPayload.PI.EmpRole"
},
"Company": {
"#(2,fieldvalue)": "finalPayload.Company"
}
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"finalPayload": {
"PI": {
"EmpName": "ONE",
"EmpRole": "ONE"
},
"Company": "ONE"
}
}
}
]

May I introduce an alternative library to solve the issue.
https://github.com/octomix/josson
implementation 'com.octomix.josson:josson:1.3.21'
-------------------------------------------------
Josson josson = Josson.fromJsonString(
"{\"data\":[{\"fieldname\":\"Name\",\"fieldvalue\":[\"JohnDoe\"]},{\"fieldname\":\"Title\",\"fieldvalue\":[\"Manager\"]},{\"fieldname\":\"Company\",\"fieldvalue\":[\"Walmart\"]}]}");
JsonNode node = josson.getNode(
"map(" +
" finalPayload: map(" +
" PI: map(" +
" EmpName: data[fieldname='Name'].fieldvalue[0]," +
" EmpRole: data[fieldname='Title'].fieldvalue[0]" +
" )," +
" Company: data[fieldname='Company'].fieldvalue[0]" +
" )" +
")");
System.out.println(node.toPrettyString());
Output
{
"finalPayload" : {
"PI" : {
"EmpName" : "JohnDoe",
"EmpRole" : "Manager"
},
"Company" : "Walmart"
}
}

How to filter list of data in list in java with sql or nested dynamic condition

How to filter data dynamically using java, Assume we have data (list of map / json array without pojo mapping).
[
{
"id": "1001",
"type": "Internal",
"status": "Closed"
},
{
"id": "1002",
"type": "External",
"status": "Closed"
},
{
"id": "1003",
"type": "Internal",
"status": "Open"
},
{
"id": "1004",
"type": "Internal",
"status": "Open"
}
]
Now we need out put filtered data as id > 1001 and ( type: 'External' or status: 'Open" )
[
{
"id": "1002",
"type": "External",
"status": "Closed"
},
{
"id": "1003",
"type": "Internal",
"status": "Open"
},
{
"id": "1004",
"type": "Internal",
"status": "Open"
}
]
Any Suggestions how to achieve this ?

Use JSON path , below are few JSONPath queries I have posted as an sample example
P.S :- Ur "Id" should be of type Integer for operation > or <
Code:-
public static void main(String[] args) {
String jsonData = "[\r\n" +
" {\r\n" +
" \"id\": 1001,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Closed\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1002,\r\n" +
" \"type\": \"External\",\r\n" +
" \"status\": \"Closed\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1003,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Open\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1004,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Open\"\r\n" +
" }\r\n" +
"]";
String filterId = "$.[?(#.id > 1001)]"; //For Id > 1001
String filterType = "$.[?(#.type in ['External'])]"; //for External type
String filterTypeAndId = "$.[?((#.id > 1001) && (#.type in ['Internal']))]"; //for External type with Id > 1001
String filterTypeAndId2 = "$.[?((#.id > 1001) && (#.type in ['Internal', 'External']))]"; //for External type with Id > 1001
DocumentContext documentContext = JsonPath.parse(jsonData);
System.out.println(documentContext.read(filterId).toString());
System.out.println(documentContext.read(filterType).toString());
System.out.println(documentContext.read(filterTypeAndId).toString());
System.out.println(documentContext.read(filterTypeAndId2).toString());
}
results are :-
[{"id":1002,"type":"External","status":"Closed"},{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]
[{"id":1002,"type":"External","status":"Closed"}]
[{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]
[{"id":1002,"type":"External","status":"Closed"},{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]

We can use javascript ability to resolve condition. We can use Java ScriptEngineManager to run javascript expression :
Expression can be type = 'Internal' AND status ='Open'
private String getResult(String expression) {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
try {
return engine.eval(expression).toString();
} catch (Exception e) {
}
return null;
}

You can achieve this with the following approach :-
Convert json to string
Construct JSONArray >> JSONArray jsonArray = new JSONArray(converted_json_String)
pull out jsonArrayObject from JsonArray and construct a list
Use Collections.sort method and deploy your comparision logic in it on the JSON keys
and construct your sorted json.

try this code, I am assuming you have a list of objects.
List<YourCLass> filteredList = list
.stream()
.filter(obj -> obj.getId() > 1001 && (obj.getStatus().equals("Open") || obj.getType().equals("External")))
.collect(Collectors.toList());
filteredList have the list of objects you are expecting, using Object mapper you can get it as json format.
String filteredObjJson = new ObjectMapper().writeValuesAsString(filteredList);
add this dependency in pom.xml or download the lib and add it to your libraries
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>

Map a String into JSON using JsonGenerator

Given is a JSON array
[
{
"ref": "idValue",
"data": {
"filter": [
{
"property": "number"
}
],
"stateId": "contacts"
}
}
]
I store this array as String in a database.
Now I want to embed the given String into a new JSON object, generated dynamically by javax.json.stream.JsonGenerator.
A possible result should look like this:
{
"newValue": "someValue",
"storedFilters": [
{
"ref": "idValue",
"data": {
"filter": [
{
"property": "number"
}
],
"stateId": "contacts"
}
}
]
}
Is there a possibility to achieve this?
My approach:
String jsonStringValue = ... // Contains JSON content of a previous request
HttpServletResponse resp = ...
List<Event> events = ...
OutputStream os = resp.getOutputStream();
Map<String, Boolean> config = new HashMap<String, Boolean>();
if (prettyPrint) {
config.put(JsonGenerator.PRETTY_PRINTING, Boolean.TRUE);
}
JsonGeneratorFactory jgf = Json.createGeneratorFactory(config);
JsonGenerator jg = jgf.createGenerator(os);
if (events.size() > 1) {
jg.writeStartArray();
}
// Some generic stuff happens here which is based on the 'events' list
jg.writeStartObject();
jg.write(JsonConstants.FILTERS, jsonStringValue);
jg.writeEnd();
if (events.size() > 1) {
jg.writeEnd();
}
But this ends up into a field which simply contains the String value in quotes.

You can create a JSON Object as below statically
{
"newValue": "someValue",
"storedFilters": []
}
Then you can add the javax.json.stream.JsonGenerator generated value to the JSON Array storedFilters dynamically.

You could try below implementation, if it works for you.
String str = "[\n" + " {\n" + " \"ref\": \"idValue\",\n" + " \"data\": {\n"
+ " \"filter\": [\n" + " {\n"
+ " \"property\": \"number\"\n" + " }\n" + " ],\n"
+ " \"stateId\": \"contacts\"\n" + " }\n" + " }\n" + "]";
JsonFactory factory = new JsonFactory();
StringWriter jsonObjectWriter = new StringWriter();
JsonGenerator generator = factory.createJsonGenerator(jsonObjectWriter);
generator.useDefaultPrettyPrinter(); // pretty print JSON
generator.writeStartObject();
generator.writeFieldName("newValue");
generator.writeString("someValue");
generator.writeFieldName("storedFilters");
generator.writeRawValue(str);
generator.writeEndObject();
generator.close(); // to close the generator
System.out.println(jsonObjectWriter.toString());
Output:
{
"newValue": "someValue",
"storedFilters": [
{
"ref": "idValue",
"data": {
"filter": [
{
"property": "number"
}
],
"stateId": "contacts"
}
}
]
}

JSON Object creation is failing

I have json file which i down from internet and saved to my app. Then i read this file and create json object . But i am not able to create json object .
This is the exception I am getting
org.json.JSONException: Expected literal value at character 3 of { \"resources\": { ..........
Below is my code to read input stream and create json object
private JSONObject readFileFromInpputStream(InputStream inst) throws JSONException
{
// TODO Auto-generated method stub
StringBuilder responseStrBuilder=null;
try {
BufferedReader streamReader = new BufferedReader(new InputStreamReader(inst, "UTF-8"));
responseStrBuilder = new StringBuilder();
String inputStr;
while ((inputStr = streamReader.readLine()) != null){
responseStrBuilder.append(inputStr);
}
}
catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String str = responseStrBuilder.toString();
String k=str.replace("\"", "\\\"");
// String m =k.replaceAll("\t", "");
// String s= m.replaceAll("\n", "");
//String p = s.replaceAll("\\s", "");
Log.i(loggerName, loggerName + " str " +str);
//Log.i(loggerName, loggerName + " k " +p);
JSONObject obj = new JSONObject(k);
return obj;
}
Below is output string
{
"resources": {
"-xmlns:xliff": "urn:oasis:names:tc:xliff:document:1.2",
"string": [
{
"name": "sample1",
"value": "To <xliff:g id=\"receiver_name\"> Europe </xliff:g>"
},
{
"name": "cdata",
"value": "<![CDATA[<p>Text<p>]]>"
},
{
"name": "content_description_sample",
"value": " <b>Something</b>"
},
{
"name": "countdown",
"value": " <xliff:g id="time" example="5days">%1$s</xliff:g> until holiday"
},
{
"name": "picker_combined_view_fmt",
"value": " Combined view (<xliff:g id="count">%s</xliff:g>)"
},
{
"name": "configure_email_text",
"value": "No corporate email accounts have been configured on this device. To configure them, click <b>Here</b> "
},
{
"name": "invalid_credentials",
"value": "Authentication failed. Enter valid credentials."
},
{
"name": "link",
"value": "<b>Hello World</b> This is a test of the URL Example"
},
{
"name": "bold",
"value": "<b>This text is bold</b>"
},
{
"name": "emphasis",
"value": "<em>This text is emphasized</em>"
},
{
"name": "check_availability_button",
"value": "Check availability How are you"
}
],
"string-array": [
{
"name": "Array1",
"item": [
"en_US",
"en_GB"
]
},
{
"name": "Array2",
"item": [
"en_US",
"en_GB"
]
}
]
}
}{
\"resources\": {
\"-xmlns: xliff\": \"urn: oasis: names: tc: xliff: document: 1.2\",
\"string\": [
{
\"name\": \"sample1\",
\"value\": \"To<xliff: gid=\\"receiver_name\\">Europe</xliff: g>\"
},
{
\"name\": \"cdata\",
\"value\": \"<![
CDATA[
<p>Text<p>
]
]>\"
},
{
\"name\": \"content_description_sample\",
\"value\": \"<b>Something</b>\"
},
{
\"name\": \"countdown\",
\"value\": \"<xliff: gid=\"time\"example=\"5days\">%1$s</xliff: g>untilholiday\"
},
{
\"name\": \"picker_combined_view_fmt\",
\"value\": \"Combinedview(<xliff: gid=\"count\">%s</xliff: g>)\"
},
{
\"name\": \"configure_email_text\",
\"value\": \"Nocorporateemailaccountshavebeenconfiguredonthisdevice.Toconfigurethem,
click<b>Here</b>\"
},
{
\"name\": \"invalid_credentials\",
\"value\": \"Authenticationfailed.Entervalidcredentials.\"
},
{
\"name\": \"link\",
\"value\": \"<b>HelloWorld</b>ThisisatestoftheURLExample\"
},
{
\"name\": \"bold\",
\"value\": \"<b>Thistextisbold</b>\"
},
{
\"name\": \"emphasis\",
\"value\": \"<em>Thistextisemphasized</em>\"
},
{
\"name\": \"check_availability_button\",
\"value\": \"CheckavailabilityHowareyou\"
}
],
\"string-array\": [
{
\"name\": \"Array1\",
\"item\": [
\"en_US\",
\"en_GB\"
]
},
{
\"name\": \"Array2\",
\"item\": [
\"en_US\",
\"en_GB\"
]
}
]
}
}
and below is the exception i am getting
org.json.JSONException: Expected literal value at character 3 of { \"resources\": { ..........
What am I doing wrong?

There is no reason to escape quotation marks("). They are part of how the json object constructor identifies strings.
Just using
JSONObject obj = new JSONObject(str);
should be fine.
In addition,
in " Combined view (%s)" the two quotation marks are treated as string delimiters and SHOULD be escaped, but it indicates a problem with the server you got this message from. Escaping these yourself can be impossible because there is no sure way to know which quotation marks are real and which are part of the text.

On validating your JSON output through Jlint it gives validation error, but on removing space on line 2 of your output(is the space intentional ? or you added it on posting question by mistake?)
"cdata", "value": "Text
]]>" }, { "nam
it validates successfully.
In any case whether the space is initial source of error or not ,like CurlyCorvus said, simply pass the String to new JSONObject(str); without escaping the ".

Thans Mustafa and Curly.
The issue was due to quotes mark in tag
for ex in original one value is
"value": " %1$s until holiday"
It worked fine when I replaced it to
"value": " %1$s until holiday"
So I guess when quotes is present inside it consider it as new object.