We Keep Coding

Document.parse() constructor not working for nested json array

I have one extended json string.
{"_id": {"oid": "59a47286cfa9a3a73e51e72c"}, "theaterId": {"numberInt": "101100"}, "location": {"address": {"street1": "340 XDW Market", "city": "Bloomington", "state": "MN", "zipcode": "12427"}, "geo": {"type": "Point", "coordinates": [{"$numberDouble": "-193.24565"}, {"$numberDouble": "144.85466"}]}}}
Trying to convert above json string to document in order to insert it into MongoDB. For this I am using org.bson.Document.Document.parse(json_string) constructor.
But the document I am getting after parsing, doesn't preserve the datatype inside geo.coordinate arraylist (Check below Document). While it preserve datatype of theaterId.
{
"_id": {
"oid": "59a47286cfa9a3a73e51e72c"
},
"theaterId": {
"numberInt": "101100"
},
"location": {
"address": {
"street1": "340 XDW Market",
"city": "Bloomington",
"state": "MN",
"zipcode": "12427"
},
"geo": {
"type": "Point",
"coordinates": [-193.24565, 144.85466]
}
}
}
Is this a potential issue in Document.parse() API ?

Your fields in geo.coordinate are starting with dollar sign $. In theaterId you have numberInt, while in coordinate - $numberDouble.
Check the docs and this question for how to handle it depending on what you need. Considering, that it looks like numberInt satisfies your needs, you might just need to remove the dollars from field names.
Edit: After digging somewhat deeper into those docs, the one you provided as well, {"numberInt": "101100"} is not extended json with datatype, it's just a normal json object with property and value for that property. It would need to be {"$numberInt": "101100"} to be extended json. On the other hand {"$numberDouble": "-193.24565"} is extended. The datatype is not lost, it's parsed into List<Double>, since we know each element is of type Double the datatype can be reconstructed back.
If you take at Document.toJson(), under the hood it's working with RELAXED output mode, which will output coordinates as you are seeing them - [-193.24565, 144.85466]. If you provide EXTENDED output mode, for example like this:
JsonWriterSettings settings = JsonWriterSettings.builder().outputMode(JsonMode.EXTENDED).build();
System.out.println(document.toJson(settings));
then the datatype will be reconstructed back from the java type, and coordinates will look like so:
[{"$numberDouble": "-193.24565"}, {"$numberDouble": "144.85466"}]
In conclusion, there is no problem with Document.parse("json"), but there might be a problem with the json you are supplying to it.
Edit2:
As in showed in example, the datatypes can be reconstructed back from java types. I am not familiar with the way collection.insertOne(Document.parse(json_string)) works under the hood, but if you don't explicitly specify the mode, it might be using RELAXED by default, instead of EXTENDED. The docs here state - This format prioritizes type preservation at the loss of human-readability and interoperability with older formats., so it would make sense. But this is just a wild guess on my part though, you would need to dig into docs to make sure.

JSON parsing vs Regex based String parsing

I need to process a big JSON payload(~1MB) coming from an API, a portion of the JSON is something like this:
{
"id": "013dd2a7-fec4-4cc5-b819-f3cf16a1f820",
//more attributes
"entry_mode": "LDE",
"periods": [
{
"type": "quarter",
"id": "fe96dc03-660c-423c-84cc-e6ae535edd2d",
"number": 1,
"sequence": 1,
"scoring": {
//more attribtues
},
"events": [
{
"id": "e4426708-fadc-4cae-9adc-b7f170f5d607",
"clock": "12:00",
"updated": "2013-12-22T03:41:40+00:00",
"description": "J.J. Hickson vs. DeAndre Jordan (Blake Griffin gains possession)",
"event_type": "opentip",
"attribution": {
"name": "Clippers",
"market": "Los Angeles",
"id": "583ecdfb-fb46-11e1-82cb-f4ce4684ea4c",
"team_basket": "left"
},
"location": {
"coord_x": 572,
"coord_y": 296
},
"possession": {
"name": "Clippers",
"market": "Los Angeles",
"id": "583ecdfb-fb46-11e1-82cb-f4ce4684ea4c"
}
},
//more events
]
}
]
}
This is a nearly-realtime API that I need to process only the events, identify a set of event UUIDs, look for duplicates in the database and save new events.
I could use a JSONObject/JSONArray or use regex with string parsing to and fetch the events portion. Processing time is critical since this should be nearly-realtime and memory efficiency is important since there can be multiple payloads coming in at once.
Which one is more efficient for this use case?

Use a proper streaming JSON parser. You know what you want to pull out of the stream, you know when you can quit parsing it, so read the stream in small, manageable chunks, and quit as soon as you know you are done.
Circa 2017, I'm not aware of any browser/native JSON streaming APIs, so you'll need to find a Javascript-based streaming library. Fortunately, streaming is not a new concept, so there are a number of options already in existence:
http://oboejs.com/
https://github.com/dominictarr/JSONStream
https://github.com/creationix/jsonparse
https://github.com/dscape/clarinet
http://danieltao.com/lazy.js/demos/json/

Way to implement real time searching in local data in android

I have just started android development and here is my situation elaborately.
Sample data in JSON format:
{
"ht_id": 1086,
"name": "Mobile Phones",
"rate": 12
},
{
"ht_id": 1111,
"name": "Primary cells and primary batteries",
"rate": 28
},
{
"ht_id": 1112,
"name": "Electric accumulators, including separators therefor, whether or not rectangular (including square)",
"rate": 28
}
I have some 250 static records total and planning to ship the same with the app. I want to search if input string is contained in the data and return corresponding "rate" in real time.
As I want to do this in real time, search speed is the utmost criteria.
So, I am wondering how to store data with the app (DB/CSV/text/JSON) and later how to read in memory for super fast retrieval in android app(eg: hashSet)

How to/best practice on how to differentiate between serialised objects stored within JSON format?

Suppose that I serialise two different objects and save them to a directory.
Problem: Upon application start up, parsing the JSON files are not a problem - since GSON is employed, I can write my own serialisers and deserialisers for both of the JSON files for their respective objects to be constructed.
But the problem is, how can I differentiate between the numerous JSON files in terms of what they store within them, so I can apply the correct deserialiser to it.
Thank you, best.

Consider standardizing your JSON structure to include document type. You can even store the target object type in that field. Good practice is to include document version number as well. Example below shows two different versions of the 'account' document and a transaction document. All three can be stored in, say, the same Couchbase bucket. The way to differentiate between different documents would be to look at the "doc_type" field and the document version (if required). From the GSON serializer selection standpoint, you can look at at the "doc_type" in a switch/if-else statement or store the target object type in place of "account" or "transaction" and then, at the expense of performance, dynamically parse JSON to POJO.
{
"doc_type": "account",
"doc_ver": 1,
"content": {
"accnt_no": "12321645645484",
"name": "Name or alias",
"email": "Email address",
"password": "Password in raw format",
"exp_date": "06/10/2017"
}
}
{
"doc_type": "account",
"doc_ver": 2,
"content": {
"accnt_no": "12321645645484",
"name": "customer name",
"email": "customer email",
"password": "pass",
"timezone": "customer timezone",
"ip": "IP address",
"spoken_languages": [ "EN", "RU" ],
"exp_date": "06/10/2017"
}
}
{
"doc_type": "transaction",
"doc_ver": 1,
"content": {
"accnt_no": "12321645645484",
"tran_date": "06/04/2017",
"tran_time": "09:15:84.953"
}
}
Hope this helps.

I think that the best way is parse JSON to a HashMap<String, Object> with multiple level. GSON will parse your JSON to HashMap with key is object name and value is an object (This object will belong to 3 type: HashMap for a object in JSON, List for an array in JSON and String for a string in JSON). To using this HashMap you need to iterate through the HashMap using a recursive method.

JSON Conversion Object having same name as Array in separate record

I am trying to parse the below JSON using retrofit. Here JSON object is having same name as JSON Array. Is there a way to correctly create pojo's
Here button is an Object in first record whereas in the second record it is an Array.
Is this valid as per JSON specification.
{
"Single": [{
"button": {
"target": "https://m.jjj.com",
"title": "Shop Now"
},
"description": "W1",
"image": "http://dojo.scene11.com/is/image/anf/anf-US-20150629",
"title": "Sports at $25"
}, {
"button": [{
"target": "https://m.abz.com",
"title": "Shop Now"
}],
"description": "Good",
"image": "http://axys.com.m/is/image/brands",
"title": "gebra "
}]
}

This is a valid JSON as per JSON specifications, but as we know an Array must have same type of values, you are violating that logic. So when you would be trying to directly map this JSON to the Java Object, it might give an exception, but in case you have written a custom parser, you might be able to use it properly.

Yes, this is valid if you are using proper function to parse your json. There can be json object and json array of same 'name'.
But there should not be two objects of same name, or two arrays of same name. Even this will be valid json having two object(or array) of same name, but you will be able to get only one json object(or array) while parsing your json.