We Keep Coding

Jolt transform: extract nested array from an object

I have JSON Input as below
{
"data": [
{
"items": [
{
"item": "R11",
"code": "8611A"
},
{
"item": "R2",
"code": "8611B"
}
]
}
]
}
I need to extract items array as below
[
{
"item": "R11",
"code": "8611A"
},
{
"item": "R2",
"code": "8611B"
}
]
Please copy and paste the above INPUT and OUTPUT https://jolt-demo.appspot.com/#inception
I have tested [{"operation":"shift","spec":{"data":{"*":""}}}]
But it returns {"items" : [ {...}, {...} ] }

You can use a shift transformation as below
[
{
"operation": "shift",
"spec": {
"*": { //might be replaced with "data"
"*": {
"*": //might be replaced with "items"
{
"#": ""
}
}
}
}
}
]
where the asterisks wildcards represent the array data, its indexes, and the array items respectively. The "#":"" represents the items as array of objects with no key.

Alternative Spec:
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"items": ""
}
}
}
}
]

Here is the answer
[{
"operation": "shift",
"spec": {
"data": {
"0": {
"*": ""
}
}
}
}]

Null Check in Jolt while converting string to integer For an Object

I am doing conversion of string to integer/double in my jolt spec. If elements does not come in request, then empty object is coming in response. Instead I do not want to pass that object itself as its empty.
Input Request Working:
{
"data": {
"first": "1",
"second": "2"
}
}
Jolt Spec:
[
{
"operation": "shift",
"spec": {
"data": {
"first": "datas.firstTag",
"second": "datas.second.secondTag"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"datas": {
"firstTag": "=toInteger",
"second": {
"secondTag": "=toInteger"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
}
]
Output if tag is there in request:
{
"datas" : {
"firstTag" : 1,
"second" : {
"secondTag" : 2
}
}
}
But when input request is like below where I do not get second tag:
{
"data": {
"first": "1"
}
}
Current Output:
{
"datas" : {
"firstTag" : 1,
"second" : { }
}
}
Getting second object as empty
"second" : { }
Expected Output:
{
"datas" : {
"firstTag" : 1
}
}
Please help to fix this issue.

Rethink in a dynamical manner by using ampersand placeholders such as
[
{
"operation": "shift",
"spec": {
"*": {
"first": "&1s.&Tag",
"*": { "#(1,&)": "&2s.&.&Tag" }
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"firstTag": "=toInteger",
"*": {
"*": "=toInteger"
}
}
}
}
]
the suffixed integers for those ampersands represents the level to go up to grab the desired value such as &1 and &2 stand for data, & without integer suffixes represent the value from the current level.
"*" wildcard represents the rest of the elements, other than first in this case(you might have third, fourth ... as other elements to be formatted like the second)
first case :
second case :
Edit : For the current case which's mentioned within the edit in order to get
{
"datas" : {
"firstTag" : "1",
"second" : {
"secondTag" : "2",
"secondNew" : "3"
}
}
}
from the input
{
"data": {
"first": "1",
"second": "2",
"secondNew": "3"
}
}
which had a new entry "secondNew": "3", you'll need to write each key explicitly within the spec such as
[
{
"operation": "shift",
"spec": {
"*": {
"first": "&1s.&Tag",
"second": "&1s.&.&Tag",
"secondNew": "&1s.second.&"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"firstTag": "=toInteger",
"*": {
"*": "=toInteger"
}
}
}
}
]
in order to combine the rest of the elements under common object.

JSON to JSON Transform of input sample using any existing java library/tools

Input:
{
"Student": {
"name" :"abc",
"id" : 588,
"class : "12"
}
}
Reqired Output:
{
"Student": {
"key" :"name",
"value":"abc",
"key" :"id",
"value":"588",
"key" :"class",
"value":"12"
}
}

Your output json invalid. Json object can not duplicate key .
You can use the library org.json and do something like this:
JSONObject jsonObject = new JSONObject(inputJson);
JSONObject outputJson = new JSONObject();
JSONArray array = new JSONArray();
for (Object key : jsonObject.keySet()) {
JSONObject item = new JSONObject();
String keyStr = (String)key;
Object keyvalue = jsonObj.get(keyStr);
item.put(keyStr, keyvalue);
array.put(item);
}
outputJson.put("Student", array);
System.out.println(json.toString());
Output :
{
"Student": [
{
"key": "name",
"value": "abc"
},
{
"key": "id",
"value": "588"
},
{
"key": "class",
"value": "12"
}
]
}

Similar to the other answer, the desired output JSON format is not valid.
The closest valid output would be
{
"Student" : [ {
"key" : "name",
"value" : "abc"
}, {
"key" : "id",
"value" : 588
}, {
"key" : "class",
"value" : "12"
} ]
}
This can be generated via Jolt with the following spec
[
{
"operation": "shift",
"spec": {
"Student": {
"name": {
"$": "Student[0].key",
"#": "Student[0].value"
},
"id": {
"$": "Student[1].key",
"#": "Student[1].value"
},
"class": {
"$": "Student[2].key",
"#": "Student[2].value"
}
}
}
}
]

This is easy to solve with JSLT if we assume the output is made valid JSON by making an array of key/value objects like the other respondents do.
The array function converts an object into an array of key/value objects exactly like you ask for, so the transform becomes:
{"Student" : array(.Student)}

Neo4j: REST API slower when using parameters

I use REST API to perform a query and I have performance issue when I use parameters.
{
"statements" : [ {
"statement" : "MATCH (n:NetworkElement)-[:Attribute]-(r:Realm) WHERE n.tag IN {tags} RETURN r.name, collect(n.tag)",
"parameters" : {
"tags" : [
"tag1",
"tag2",
"tag3", ...]
}
} ]
}
This query take 5s. This is very long.
I try now to include the tags arry inside the statement :
{
"statements" : [ {
"statement" : "MATCH (n:NetworkElement)-[:Attribute]-(r:Realm) WHERE n.tag IN [\"tag1\",\"tag2\",\"tag3\", ...] RETURN r.name, collect(n.tag)",
"parameters" : {}
} ]
}
And now the query take only 40ms ?
Someone can explain me ? Give me some optimization way ?
Thanks in advance
Edit 2015-05-27
After test with 2.2.2 the problem occur but in different condition :
I run this query without parameters :
{
"statements": [
{
"statement" : "PROFILE MATCH (ne:NetworkElement {_type:'interface'})-[:Connect*1..]->(s:NetworkElement) WHERE s.tag IN [\"mytag\"] RETURN s.tag, collect(ne.tag)",
"parameters" : {}
}
]
}
The query execute in ~100ms without cache, the plan is :
{
"root":{
"operatorType":"EagerAggregation",
"DbHits":1468,
"Rows":1,
"version":"CYPHER 2.2",
"KeyNames":"s.tag",
"EstimatedRows":0,
"planner":"COST",
"identifiers":[
"collect(ne.tag)",
"s.tag"
],
"children":[
{
"operatorType":"Projection",
"LegacyExpression":"ne",
"Rows":734,
"DbHits":1468,
"EstimatedRows":0,
"identifiers":[
" UNNAMED46",
"ne",
"s",
"s.tag"
],
"children":[
{
"operatorType":"Filter",
"LegacyExpression":"(ne:NetworkElement AND ne._type == { AUTOSTRING0})",
"Rows":734,
"DbHits":3402,
"EstimatedRows":0,
"identifiers":[
" UNNAMED46",
"ne",
"s"
],
"children":[
{
"operatorType":"VarLengthExpand(All)",
"ExpandExpression":"(s)-[ UNNAMED46:Connect*]->(ne)",
"Rows":1134,
"DbHits":2269,
"EstimatedRows":0,
"identifiers":[
" UNNAMED46",
"ne",
"s"
],
"children":[
{
"operatorType":"NodeUniqueIndexSeek",
"Index":":NetworkElement(tag)",
"Rows":1,
"DbHits":1,
"EstimatedRows":0.9999999999971109,
"identifiers":[
"s"
],
"children":[
]
}
]
}
]
}
]
}
]
}
}
Now I run the same query with parameters :
{
"statements": [
{
"statement" : "PROFILE MATCH (ne:NetworkElement)-[:Connect*1..]->(s:NetworkElement) WHERE ne._type = {endNeType} AND s.tag IN {startTags} RETURN s.tag, collect(ne.tag)",
"parameters" : {
"endNeType" : "interface",
"startTags" : ["mytag"]
}
}
]
}
The query take 980ms to execute and the plan is not the same :
{
"root":{
"operatorType":"EagerAggregation",
"DbHits":1468,
"Rows":1,
"version":"CYPHER 2.2",
"KeyNames":"s.tag",
"EstimatedRows":0,
"planner":"COST",
"identifiers":[
"collect(ne.tag)",
"s.tag"
],
"children":[
{
"operatorType":"Projection",
"LegacyExpression":"ne",
"Rows":734,
"DbHits":1468,
"EstimatedRows":0,
"identifiers":[
" UNNAMED26",
"ne",
"s",
"s.tag"
],
"children":[
{
"operatorType":"Filter",
"LegacyExpression":"(any(-_-INNER-_- in {startTags} where s.tag == -_-INNER-_-) AND s:NetworkElement)",
"Rows":734,
"DbHits":104427,
"EstimatedRows":0,
"identifiers":[
" UNNAMED26",
"ne",
"s"
],
"children":[
{
"operatorType":"VarLengthExpand(All)",
"ExpandExpression":"(ne)-[ UNNAMED26:Connect*]->(s)",
"Rows":34809,
"DbHits":105113,
"EstimatedRows":0,
"identifiers":[
" UNNAMED26",
"ne",
"s"
],
"children":[
{
"operatorType":"NodeIndexSeek",
"Index":":NetworkElement(_type)",
"Rows":35495,
"DbHits":35496,
"EstimatedRows":0.9999999999971109,
"identifiers":[
"ne"
],
"children":[
]
}
]
}
]
}
]
}
]
}
}
I have an unique constraint on NetworkElement.tag and index on NetworkElement._type.

If you have more than 1 potential index usage, the plan might look different, you can force the other (or both) index lookups with USING INDEX
MATCH (ne:NetworkElement)-[:Connect*1..]->(s:NetworkElement)
USING INDEX ne:NetworkElement(_type)
USING INDEX s:NetworkElement(tag)
WHERE ne._type = {endNeType} AND s.tag IN {startTags}
RETURN s.tag, collect(ne.tag)

JSON Object creation is failing

I have json file which i down from internet and saved to my app. Then i read this file and create json object . But i am not able to create json object .
This is the exception I am getting
org.json.JSONException: Expected literal value at character 3 of { \"resources\": { ..........
Below is my code to read input stream and create json object
private JSONObject readFileFromInpputStream(InputStream inst) throws JSONException
{
// TODO Auto-generated method stub
StringBuilder responseStrBuilder=null;
try {
BufferedReader streamReader = new BufferedReader(new InputStreamReader(inst, "UTF-8"));
responseStrBuilder = new StringBuilder();
String inputStr;
while ((inputStr = streamReader.readLine()) != null){
responseStrBuilder.append(inputStr);
}
}
catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String str = responseStrBuilder.toString();
String k=str.replace("\"", "\\\"");
// String m =k.replaceAll("\t", "");
// String s= m.replaceAll("\n", "");
//String p = s.replaceAll("\\s", "");
Log.i(loggerName, loggerName + " str " +str);
//Log.i(loggerName, loggerName + " k " +p);
JSONObject obj = new JSONObject(k);
return obj;
}
Below is output string
{
"resources": {
"-xmlns:xliff": "urn:oasis:names:tc:xliff:document:1.2",
"string": [
{
"name": "sample1",
"value": "To <xliff:g id=\"receiver_name\"> Europe </xliff:g>"
},
{
"name": "cdata",
"value": "<![CDATA[<p>Text<p>]]>"
},
{
"name": "content_description_sample",
"value": " <b>Something</b>"
},
{
"name": "countdown",
"value": " <xliff:g id="time" example="5days">%1$s</xliff:g> until holiday"
},
{
"name": "picker_combined_view_fmt",
"value": " Combined view (<xliff:g id="count">%s</xliff:g>)"
},
{
"name": "configure_email_text",
"value": "No corporate email accounts have been configured on this device. To configure them, click <b>Here</b> "
},
{
"name": "invalid_credentials",
"value": "Authentication failed. Enter valid credentials."
},
{
"name": "link",
"value": "<b>Hello World</b> This is a test of the URL Example"
},
{
"name": "bold",
"value": "<b>This text is bold</b>"
},
{
"name": "emphasis",
"value": "<em>This text is emphasized</em>"
},
{
"name": "check_availability_button",
"value": "Check availability How are you"
}
],
"string-array": [
{
"name": "Array1",
"item": [
"en_US",
"en_GB"
]
},
{
"name": "Array2",
"item": [
"en_US",
"en_GB"
]
}
]
}
}{
\"resources\": {
\"-xmlns: xliff\": \"urn: oasis: names: tc: xliff: document: 1.2\",
\"string\": [
{
\"name\": \"sample1\",
\"value\": \"To<xliff: gid=\\"receiver_name\\">Europe</xliff: g>\"
},
{
\"name\": \"cdata\",
\"value\": \"<![
CDATA[
<p>Text<p>
]
]>\"
},
{
\"name\": \"content_description_sample\",
\"value\": \"<b>Something</b>\"
},
{
\"name\": \"countdown\",
\"value\": \"<xliff: gid=\"time\"example=\"5days\">%1$s</xliff: g>untilholiday\"
},
{
\"name\": \"picker_combined_view_fmt\",
\"value\": \"Combinedview(<xliff: gid=\"count\">%s</xliff: g>)\"
},
{
\"name\": \"configure_email_text\",
\"value\": \"Nocorporateemailaccountshavebeenconfiguredonthisdevice.Toconfigurethem,
click<b>Here</b>\"
},
{
\"name\": \"invalid_credentials\",
\"value\": \"Authenticationfailed.Entervalidcredentials.\"
},
{
\"name\": \"link\",
\"value\": \"<b>HelloWorld</b>ThisisatestoftheURLExample\"
},
{
\"name\": \"bold\",
\"value\": \"<b>Thistextisbold</b>\"
},
{
\"name\": \"emphasis\",
\"value\": \"<em>Thistextisemphasized</em>\"
},
{
\"name\": \"check_availability_button\",
\"value\": \"CheckavailabilityHowareyou\"
}
],
\"string-array\": [
{
\"name\": \"Array1\",
\"item\": [
\"en_US\",
\"en_GB\"
]
},
{
\"name\": \"Array2\",
\"item\": [
\"en_US\",
\"en_GB\"
]
}
]
}
}
and below is the exception i am getting
org.json.JSONException: Expected literal value at character 3 of { \"resources\": { ..........
What am I doing wrong?

There is no reason to escape quotation marks("). They are part of how the json object constructor identifies strings.
Just using
JSONObject obj = new JSONObject(str);
should be fine.
In addition,
in " Combined view (%s)" the two quotation marks are treated as string delimiters and SHOULD be escaped, but it indicates a problem with the server you got this message from. Escaping these yourself can be impossible because there is no sure way to know which quotation marks are real and which are part of the text.

On validating your JSON output through Jlint it gives validation error, but on removing space on line 2 of your output(is the space intentional ? or you added it on posting question by mistake?)
"cdata", "value": "Text
]]>" }, { "nam
it validates successfully.
In any case whether the space is initial source of error or not ,like CurlyCorvus said, simply pass the String to new JSONObject(str); without escaping the ".

Thans Mustafa and Curly.
The issue was due to quotes mark in tag
for ex in original one value is
"value": " %1$s until holiday"
It worked fine when I replaced it to
"value": " %1$s until holiday"
So I guess when quotes is present inside it consider it as new object.