I am new to JOLT. I am creating a JOLT specification to parse a JSON from one format to another. However when i use the RHS as "&1.name" it doesn't work as expected and creates a different format. But when i use "[&1].name", it works.
I cannot share the data due to policy constraints. Giving an example
eg.
Input JSON:
[
{
"name": "my name1"
},
{
"name": "my name2"
}
]
JOLT spec (Not working) :
[
{
"operation": "shift",
"spec": {
"*": {
"name": "&1.name"
}
}
}
]
JOLT spec (working) :
[
{
"operation": "shift",
"spec": {
"*": {
"name": "[&1].name"
}
}
}
]
Please help me understand the purpose of [] on RHS.
Both two spec is valid.
&1: get 1 level up key.
[&1]: get 1 level up index.
For example in your spec, &1 means 0 and 1 as key and your output can be like this:
{
"0": {
"name": "my name1"
},
"1": {
"name": "my name2"
}
}
And when you using the [&1] means 0 and 1 as index of array and your output can be like this:
[
{
"name": "my name1"
},
{
"name": "my name2"
}
]
You can even rephrase by replacing the name on the right hand side by an ampersand & as
[
{
"operation": "shift",
"spec": {
"*": {
"name": "[&1].&"
}
}
}
]
in order to replicate it
Prepending both [&1] and &1 targets the same level which's going up the tree 1 level in order to reach the level of the indexes of the outermost array by traversing opening curly brace(s) ({) once. But the first one yields arraywise result instead of index numbers 0,1,2 ... those are generated by the second one.
Related
I want to transform a flat key-value map into a complex json object.
Structure is as follows:
the map-keys of course represent the json keys.
nested elements are separated by a dot
list elements are signaled by a digit at the end of the key. there may be multiple digits at the end. The keys unfortunately stand in reversed order. Mean that the first "key-fragment" maps to the last digit, and the innermost key-fragment maps to the first digit.
Following example:
service.fee.1.1=a
service.fee.2.1=b
service.fee.3.1=c
Here the "service" key maps always to index=1. This means "service" is an array, but with only one element in this case.
The one element has the key "fee", with 3 values inside.
The resulting json should thus be:
{
"service": [
{
"fee": ["a", "b", "c"]
}
]
}
Another example:
service.fee.name.1.1=a
service.fee.age.2.1=b
service.fee.test.2.1=c
{
"service": [
{
"fee": [
{
"name": "a"
},
{
"age": "b",
"test": "c"
}
]
}
]
}
That's what I started with, but I cannot get the point where I probably have to use recursion to handle nested objects and lists:
JSONObject json = new JSONObject();
for (Map.Entry<String, String> entry : map.entrySet()) {
String key = entry.getKey();
if (endswithdigit(key)) {
} else {
if (key.contains("-")) {
//complex object
JSONObject subjson = new JSONObject();
json.put(key, subjson);
//TODO probably have to apply some kind of recursion here with subjson??
} else {
//plain value
json.put(key, entry.getValue());
}
}
}
Maybe you could give advise how to properly build a nested JSONObject with nested lists and recursion?
If you need to tackle this problem yourself (IE: a library cannot handle it), then I would break it down so that it can be coherently tackled with Composite Pattern.
I'll address this answer in two parts: first, a proposed solution to create the heirarchy; and second, how to utilize the Composite pattern to turn your internal heirarchy into the JSON you want.
Part 1: Creating the Heirarhcy
One approach for this would be to iteratively create objects by dividing elements in to bins (starting with a common composite root object that contains every element). This will form the composite structure of your data. The flow will be:
For each element in the bin of the object composite:
Strip off the top-level identifier from the left of the element
Create an identifier to be associated with it.
If it is keyed:
Strip off the key from the right
Create a composite array for the identifier (if it does not exist).
If there is further data left of the = of the element:
Create a composite object for the element bin associated with that array index (if it does not exist).
Place the element in a bin for that object.
Otherwise create a leaf node for the value of the index.
Otherwise, if there is further data left of the = of the element:
Create a composite object for the element bin associated with that array index (if it does not exist).
Place the element in a bin for that object.
Otherwise, create a leaf node for the value of the identifier.
Repeat for all new bins.
For example's sake, lets assume we are working with the given dataset:
x.y.z.1.1=A
x.y.z.3.1=B
x.y.w.1.1=C
x.u.1=D
a.b.1=E
a.c.1=F
e.1=G
e.2=H
i=I
m.j=J
m.k=K
The process would then follow as:
ITERATION 0 (initialize):
root // x.y.z.1.1=A, x.y.z.3.1=B, x.y.w.1.1=C, x.u.1=D, a.b.1=E, a.c.1=F, e.1=G, e.2=H, i=I, m.j=J, m.k=K
ITERATION 1:
root :
x[1] // y.z.1=A, y.z.3=B, y.w.1=C, u=D
a[1] // b=E, c=F
e[1] : "G"
e[2] : "H"
i : "I"
m : // j=J, k=K
ITERATION 2:
root :
x[1] :
y[1] // z=A, w=C
y[3] // z=B
u : "D"
a[1] :
b : "E"
c : "F"
e[1] : "G"
e[2] : "H"
i : "I"
m :
j : "J"
k : "K"
ITERATION 3:
root :
x[1] :
y[1] :
z : "A"
w : "C"
y[3] :
z : "B"
u: "D"
a[1] :
b : "E"
c : "F"
e[1] : "G"
e[2] : "H"
i : "I"
m :
j : "J"
k : "K"
Part 2: Composite Pattern
At this point, we've iteratively divided our data into a heirarchical composite structure; now we just need to get our internalized data structure into JSON. This is where the Composite pattern will come in handy; each of your objects will implement the following interface:
// All objects in the composite tree must implement this.
public interface Jsonable {
// The non-leaf objects will need to have their implementation of this
// call it for each child object (and handle gaps).
JsonObject toJsonObject();
}
If following the above, we would likely have three implementations of this interface: ArrayComposite, ObjectComposite, and ValueLeaf.
Calling toJsonObject() on your root element will give you your complete JsonObject. A textural representation of that for the above example is below (notice the added gap in the y array; this needs to be handled in the toJsonObject() call of your array composites):
{
"x" : [
{
"y" : [
{
"z" : "A",
"w" : "C"
},
"",
{
"z" : "B"
}
]
}
],
"a" : [
{
"b" : "D",
"c" : "E"
}
],
"e" : [
"F",
"G"
]
"i" : "I"
"m" : {
"j" : "J",
"k" : "K"
}
}
Which, neglecting white spacing, seems to be what you're looking for.
Note that this assumes a data set does not contain elements that would result in invalid JSON. IE: the dataset could not contain the following:
i=I
i.1=I
As it would be saying that i is both an array and a value.
Please try the Gson library and use new Gson().toJson(yourmap); this will convert your map to JSON format.
Perhaps you resolve it by splitting the key where the numbers start, and using a LIFO for the subkeys and a FIFO for the value and indexes. Instead for splitting it can be done by parsing the key and detecting where the numbers start:
For example:
x.y.z.2.1 = val
This is split to show how it would work, but it can be done just parsing the string (: is to delimit the separation).
x.y.z : 2.1 : val
Then put the subkeys in a LIFO (x goes in first, z last):
LIFO
head: z
y
x
and a FIFO for the value and indexes (2 goes in first, val goes last)
Fifo
top:2
1
val
Then you can pop out from the LIFO and match it to the pop of the FIFO. The first assignment will be for the value of the map, then, the assignments will be done to the object or array of the last step.
z = val
y[1] = z
x[2] = y
I am reading some data from the internet into my array like this:
final String[] relations = entity.getAsJsonPrimitive("rel").getAsString().substring(1).split("\\/");
final String relation = entity.getAsJsonPrimitive("rel").getAsString();
System.out.println(relations[1]);
break;
Anyway, the output is really weird (without the break statement the output is hundred of lines) since after I added the break statement I am getting two lines of output like this:
proerty
is_a
How can I split this output to print only the first or the second line?
I checked how the function spilt works but it did not work for me it says "ArrayIndexOutOfBound"
Also, would any one tell me why the first index contains hundred of lines but the second index throws an error when I print it and says out of bound.
This is the input, which is only one element. The real input has hundreds elements like this one:
Note: I want to print the relation part only.
{
"context": "/ctx/all",
"dataset": "/d/verbosity",
"end": "/c/en/sleep",
"features": [
"/c/en/bed /r/RelatedTo -",
"/c/en/bed - /c/en/sleep",
"- /r/RelatedTo /c/en/sleep"
],
"id": "/e/bffe7778135422c9fc697e5c3ada0be01ef026f0",
"license": "/l/CC/By-SA",
"rel": "/r/RelatedTo",
"source_uri": "/s/site/verbosity",
"sources": [
"/s/site/verbosity"
],
"start": "/c/en/bed",
"surfaceText": "[[bed]] is related to [[sleep]]",
"uri": "/a/[/r/RelatedTo/,/c/en/bed/,/c/en/sleep/]",
"weight": 6.954197039975413
},
Thanks in advance!
In Mule/Dataweave how do I convert/transform a HashMap into a Array. I have got a HashMap whose key & value are dynamic numbers.
Example:
{"3.2" : 1, "22" : 8, "2.0" : 1}
I want to transform it to this structure:
[
{
"name": "app-a",
"value1": 3.2,
"value2": 1
},
{
"name": "app-a",
"value1": 22,
"value2": 8
},
{
"name": "app-a",
"value1": 2,
"value2": 1
}
]
Solution (Thanks to #Sulthony H)
%dw 1.0
%output application/json
---
payload pluck $$ map {
value1: ($ as :string) as :number,
value2: payload[$]
}
In order to transform a HashMap into an Array, I will do the following steps:
Iterate the HashMap by its key using pluck operator in DataWeave: payload pluck $$ map {}
Transform the key as number: value1: ($ as :string) as :number
Get the value based on that key: value2: payload[$]
Another different solution:
%dw 1.0
%output application/json
---
payload map {
($ mapObject {
name: "app-a",
value1: $$ as :string,
value2: $
})
}
1 - Use the map operator to iterate on the list of elements. payload map
2 - Use mapObject on each of the element of your array $ mapObject. With mapObject, $$ refers to the key name, $ refers to the value
3 - print out the values with value1: $$ as :string, value2: $
Even more simple...
payload pluck (
{
value1:$$,
value2:$
}
)
I have a string that looks like this
"{"resturant_name": "Chipotle", "street": "431 Liberty St"},
{"resturant_name": "MCDoNalds", "street": "1 Main St"},
{"resturant_name": "Wednys", "street": "5 Main St"}"
And I want to turn into a JSONArray so I can loop though and get the name?
But when I do
JSONArray jsonArray = new JSONArray(string);
I get an error
type of org.json.JSONObject cannot be converted to JSONArray,
How can I make this a JSONArray so I can loop though it?
Thanks
That's because your string is not JSON. It's close, but not quite. JSON standards dictate that the structure should either be an Object or an Array. To create an Array, it must begin with "[" and end with "]". So, your string should look like:
[{"resturant_name": "Chipotle", "street": "431 Liberty St"},
{"resturant_name": "MCDoNalds", "street": "1 Main St"},
{"resturant_name": "Wednys", "street": "5 Main St"}]
I'm new to JSON and I'm really struggling to parse this layout with GSON in Java
{"time_entries":
[
{"hours":1.0,
"id":311,
"created_on":"2012-11-02T14:53:38Z",
"user":{"id":7,"name":"blah"},
"issue":{"id":266},
"activity":{"id":10,"name":"blah"},
"updated_on":"2012-11-02T14:53:38Z",
"comments":"blah",
"spent_on":"2012-11-02",
"project":{"id":10,"name":"blah"}},
{"hours":6.0,
"id":310,
"created_on":"2012-11-02T13:49:24Z",
"user":{"id":4,"name":"blah"},
"issue":{"id":258},
"activity":{"id":9,"name":"blah"},
"updated_on":"2012-11-02T13:49:24Z",
"comments":"blah",
"spent_on":"2012-11-02",
"project":{"id":11,"name":"blah"
}}
],
"offset":0,
"limit":2,
"total_count":306
}
If it helps it's the output the Redmine API gives you for time entries.
I'm struggling to understand some of the basic JSON concepts like objects and arrays and I haven't been able to find an example with a layout similar to this.
My main concern in using the tutorials I have read is that the multiple ID fields will get confused.
What's the best way to parse this without tying myself in knots?
I'm not set on using Gson and would be happy for a solution using Jackson or the built in library. The end goal is for Android implementation so I would prefer to use use serialization.
Thanks
EDIT:
My attempt at an "object model"
public class TimeResponse {
public List<Time_Entry> time_entries;
#SerializedName("hours")
public String hours;
#SerializedName("id")
public int id;
#SerializedName("created_on")
public String created_on;
#SerializedName("name")
public String name;
#SerializedName("updated_on")
public int updated_on;
public int page;
#SerializedName("comments")
public double comments;
#SerializedName("spent_on")
public String spent_on;
#SerializedName("offset")
public String offset;
#SerializedName("limit")
public String limit;
#SerializedName("total_count")
public String total_count;
}
I'm am unsure as to what I should write for my results list (if I need one) and I've have only declared an id and name string once despite it being used multiple times?
I am aware I shouldn't be using strings for my hours I'm in the process of looking into what the hours field actually contains. I believe the tutorial is slightly out of date in that the last three fields are not represented in the same way now in the Twitter API.
I am not sure what you mean by 'multiple ID fields'. There is no such thing as an ID in JSON.
Regarding the basic JSON concepts, see http://json.org/:
Object:
An object is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace). Each name is followed by : (colon) and the name/value pairs are separated by , (comma).
Array:
An array is an ordered collection of values. An array begins with [ (left bracket) and ends with ] (right bracket). Values are separated by , (comma).
Value:
A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested.
String:
A string is a sequence of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. A character is represented as a single character string. A string is very much like a C or Java string.
Number:
A number is very much like a C or Java number, except that the octal and hexadecimal formats are not used.
Edit:
There is not much you can do to simlify the JSON from your question except pretty-print it:
{
"time_entries": [
{
"hours": 1,
"id": 311,
"created_on": "2012-11-02T14:53:38Z",
"user": {
"id": 7,
"name": "blah"
},
"issue": {
"id": 266
},
"activity": {
"id": 10,
"name": "blah"
},
"updated_on": "2012-11-02T14:53:38Z",
"comments": "blah",
"spent_on": "2012-11-02",
"project": {
"id": 10,
"name": "blah"
}
},
{
"hours": 6,
"id": 310,
"created_on": "2012-11-02T13:49:24Z",
"user": {
"id": 4,
"name": "blah"
},
"issue": {
"id": 258
},
"activity": {
"id": 9,
"name": "blah"
},
"updated_on": "2012-11-02T13:49:24Z",
"comments": "blah",
"spent_on": "2012-11-02",
"project": {
"id": 11,
"name": "blah"
}
}
],
"offset": 0,
"limit": 2,
"total_count": 306
}
Perhaps you can see that you have one JSON Object with four name/value pairs:
time_entries
offset
limit
total_count
The last three of these have simple numeric values while the first (time_entries) is an Array of two more Objects. Each one of these two Objects conssits of various name/value pairs. The name/value pair id is just one of these.
Data is in name/value pairs
Data is separated by commas
Curly braces hold objects
Square brackets hold arrays
I ve used javascript here.. it may useful for you.. if you 've any other help let me knw
var jsonText = xmlhttp.responseText;
var obj = eval ("(" + jsonText + ")");
row_num=Object.keys(obj.time_entries).length;
this line give the length of time_entries array length
keydata[c]=Object.keys(obj.time_entries[0]);
columndata=keydata[0].toString();
my = columndata.split(",");
columndata contain the key values of time entries as a string of zero th index in that array
columnndata={hours,id,create_on..... and so on}
my={"hours","id".... etc}