This question has been asked many times but I couldn't find the answer that fixes my issue.
I'm trying to convert nested JSON format to CSV format like this :
The JSON structure is arbitrary and could be anything, nested or not.
I'm not suppose to know it, it's a database answer and I need to export this JSON answer into CSV file.
Here is an example
Input :
{
"_id": 1,
"name": "Aurelia Menendez",
"scores": [
{
"type": "exam",
"score": 60.06045071030959
},
{
"type": "quiz",
"score": 52.79790691903873
},
{
"type": "homework",
"score": 71.76133439165544
}
]
}
The output I'm looking for :
_id,name,scores.type,scores.score,scores.type,scores.score,scores.type,scores.score
1,Aurelia Menendez,exam,60.06...,quiz,52.79...,homework,71.76..
This is an example, it could be any other JSON document.
The idea here is to use dot notation in the CSV column name.
I've already used CDL but the output is not what I want :
_id scores name
1 "[{score:60.06045071030959,type:exam},{score:52.79790691903873,type:quiz},{score:71.76133439165544,type:homework}]" Aurelia Menendez
So how can I convert nested JSON to CSV with dot notation and in a generic way ?
Edits
Deserialisation of the JSON with Jackson :
ObjectMapper mapper=new ObjectMapper();
JsonNode jsonNode=mapper.readValue(new File("C:\\...\\...\...\\test.json"), JsonNode.class);
Ismail
Like you said :
The JSON structure is arbitrary and could be anything, nested or not.
The JSON to CSV conversion can't be generalized as it varies from user to user and also depends specific requirements.
But still there's a library json2flat which tries to achieve it. But it may differ from user's requirement. Still it's worth a try.
For example for the JSON given above:
{
"_id": 1,
"name": "Aurelia Menendez",
"scores": [
{
"type": "exam",
"score": 60.06045071030959
},
{
"type": "quiz",
"score": 52.79790691903873
},
{
"type": "homework",
"score": 71.76133439165544
}
]
}
can be interpreted as follows :
/_id,/name,/scores/type,/scores/score
1,"Aurelia Menendez","exam",60.06045071030959
1,"Aurelia Menendez","quiz",52.79790691903873
1,"Aurelia Menendez","homework",71.76133439165544
Converting JSON to XLS/CSV in Java has what you are looking for.
Basically, you need to use org.json.CDL to convert from JSON to CSV format
Comments are not convenient place to post longs answers, so I post my answer here.
Analyze your JSON and all possible JSON structures you can get from your database. It should be a limited number of JSON forms.
As you have analyzed your JSON structure build a class/class hierarchy, that fully reflects this structure.
Use JSON serializer/deserializer library at your choice, to deserialize JSON to a java object.
Employ StringBuffer/StringBuilder classes, and iterate over your object information, and build comma delimited (or tab-delimited) strings.
Write strings you have built on the previous stage to the file.
That's it.
Related
My mobile app receives a JSON document from my server, but then my app sends it back to the server with additional fields populated, some fields modified.
Upon receipt at the server I can convert this JSON string to a generic Mongo Document and dump it into the database, I can then read it back as a POJO. This is wasteful, all I want to do is convert the JSON to a POJO using the mongo API, manipulate the data a little, then store it.
Likewise I sometimes don't want to query as the generic Document but a specific typed POJO then create the Document / JSON from it after manipulating it in a type-safe way.
Ideally I'd like to support parsing a list of documents too. Using the Java Mongo API is it possible to
Convert a POJO to a JSON String
Convert a JSON string (or Document) to POJO
Convert a JSON array to a List
Convert a List to a JSON List of documents
Please note that I'm trying to avoid using JAX-RS / JAXB or Spring, or other third party libraries such as GSON as so far they don't keep the JSON in the same format as if I queried the mongo database using the generic Document type.
As a really simple example, my JSON has the _id and date properties like this, some entities will have nested child documents/types ...
{
"questions": [
{
"_id": {
"$oid": "605b3d62958763a2e18be480"
},
"companyId": {
"$oid": "554a0123c2e625bd02259318"
},
"groupId": null,
"title": "Test Title 1",
"desc": "Description 1",
"style": "range",
"minRange": 1,
"maxRange": 10,
"comments": false,
"startDate": {
"$date": "2021-03-01T00:00:00Z"
},
"endDate": {
"$date": "2021-03-31T23:59:00Z"
},
"deleted": true,
"frequencyMins": 1
},
..
],
}
}
Suppose that I serialise two different objects and save them to a directory.
Problem: Upon application start up, parsing the JSON files are not a problem - since GSON is employed, I can write my own serialisers and deserialisers for both of the JSON files for their respective objects to be constructed.
But the problem is, how can I differentiate between the numerous JSON files in terms of what they store within them, so I can apply the correct deserialiser to it.
Thank you, best.
Consider standardizing your JSON structure to include document type. You can even store the target object type in that field. Good practice is to include document version number as well. Example below shows two different versions of the 'account' document and a transaction document. All three can be stored in, say, the same Couchbase bucket. The way to differentiate between different documents would be to look at the "doc_type" field and the document version (if required). From the GSON serializer selection standpoint, you can look at at the "doc_type" in a switch/if-else statement or store the target object type in place of "account" or "transaction" and then, at the expense of performance, dynamically parse JSON to POJO.
{
"doc_type": "account",
"doc_ver": 1,
"content": {
"accnt_no": "12321645645484",
"name": "Name or alias",
"email": "Email address",
"password": "Password in raw format",
"exp_date": "06/10/2017"
}
}
{
"doc_type": "account",
"doc_ver": 2,
"content": {
"accnt_no": "12321645645484",
"name": "customer name",
"email": "customer email",
"password": "pass",
"timezone": "customer timezone",
"ip": "IP address",
"spoken_languages": [ "EN", "RU" ],
"exp_date": "06/10/2017"
}
}
{
"doc_type": "transaction",
"doc_ver": 1,
"content": {
"accnt_no": "12321645645484",
"tran_date": "06/04/2017",
"tran_time": "09:15:84.953"
}
}
Hope this helps.
I think that the best way is parse JSON to a HashMap<String, Object> with multiple level. GSON will parse your JSON to HashMap with key is object name and value is an object (This object will belong to 3 type: HashMap for a object in JSON, List for an array in JSON and String for a string in JSON). To using this HashMap you need to iterate through the HashMap using a recursive method.
How would i map this JSON to an object?
{"d":"[{\"Key\":\"3\",\"ExternalKey\":\"143234A\",\"Name\":\"cup of juice\",\"Litres\":\"2 litres\",\"Date\":\"2016-10-06T08:32:27\",\"Capacity\":5.4900,\"CapacityType\":\"%\"}, {\"Key\":\"3\",\"ExternalKey\":\"143234A\",\"Name\":\"cup of milk\",\"Litres\":\"2.4 litres\",\"Date\":\"2016-10-06T08:32:27\",\"Capacity\":1667.6100,\"CapacityType\":\"M\"}]"}
I've tried using a HashMap but it just puts "d" as the string and the rest as a String object with one element
This is an extremely common problem known as data marshaling. In java, Jackson is the general best solution. Read this tutorial: http://wiki.fasterxml.com/JacksonInFiveMinutes
Your Json string has some extra " characters.
This is the final json:
{
"d": [
{
"Key": "3",
"ExternalKey": "143234A",
"Name": "cup of juice",
"Litres": "2 litres",
"Date": "2016-10-06T08:32:27",
"Capacity": 5.49,
"CapacityType": "%"
},
{
"Key": "3",
"ExternalKey": "143234A",
"Name": "cup of milk",
"Litres": "2.4 litres",
"Date": "2016-10-06T08:32:27",
"Capacity": 1667.61,
"CapacityType": "M"
}
]
}
Now you can copy the json and paste it here to get the pojo.
There are n numbers of libraries available for parsing the JSON and convert into in java classes.
Some of the examples are,
GSON
Jackson
Logan Square
Moshi, etc
You need to create a java class which map with your JSON Response and need to change your pojo according to parsing library.
For Example,
Using Logan Square you must annotate your POJO class to #JsonObject and all properties must be annotated with #JsonField.
Now, you can parse your json using method
...
LoganSquare.parse(jsonString, pojoClassName);
...
Hopefully this will help you.
Cheers
if i substring out the "d" key and map it as a JSON Array it works, that'll have to be the solution for now...
I am trying to parse the below JSON using retrofit. Here JSON object is having same name as JSON Array. Is there a way to correctly create pojo's
Here button is an Object in first record whereas in the second record it is an Array.
Is this valid as per JSON specification.
{
"Single": [{
"button": {
"target": "https://m.jjj.com",
"title": "Shop Now"
},
"description": "W1",
"image": "http://dojo.scene11.com/is/image/anf/anf-US-20150629",
"title": "Sports at $25"
}, {
"button": [{
"target": "https://m.abz.com",
"title": "Shop Now"
}],
"description": "Good",
"image": "http://axys.com.m/is/image/brands",
"title": "gebra "
}]
}
This is a valid JSON as per JSON specifications, but as we know an Array must have same type of values, you are violating that logic. So when you would be trying to directly map this JSON to the Java Object, it might give an exception, but in case you have written a custom parser, you might be able to use it properly.
Yes, this is valid if you are using proper function to parse your json. There can be json object and json array of same 'name'.
But there should not be two objects of same name, or two arrays of same name. Even this will be valid json having two object(or array) of same name, but you will be able to get only one json object(or array) while parsing your json.
I have a result set coming from SQL and the result set is such that it might contain results of same kind as children (kind of a tree). I want to covert this tree kind of structure to a JSON format. I am wondering how to achieve this basically. I want to convert my ResultSet into a tree and then into JSON. My JSON should look something like this:
{
"name": "value",
"children":
[
{
"name": "value",
"children":
[
{
"name": "value"
}
]
}
]
}
I want to use Jackson library.
Check out ObjectMapper. Should do the trick for you I think.