I'm trying to dynamically parse a lot of data in Java. In other words I have tons of different types of data so I am putting them in a String or a JSON String (via GSON) instead of doing:
switch(data){
case instanceof Class1:
Class1 data = gson.fromJson(jsonString);
//Do Stuff for class 1 layout
case instanceof Class2:
Class2 data = gson.fromJson(jsonString);
etc.
So instead of doing this for 80 or so classes (which more may be added/removed/changed at any given time) I am planning to put the data in either a String or Json String and parse out the values (depth first).
I am on a low end PC (single core Atom processor) and am trying to reduce the amount of taxing I put on the CPU, so determining which would be faster... regex string manipulation with splits or using a recursive JSON parser.
Let us discuss both the cases you have mentioned here:
CASE 1:
Creating instances for each and every json input using gson (mapped to a Class)
and
CASE 2: Create a Document (or similar type object) itself and try accessing data from there.
For Case 2, you don't need to write a parser yourself: there are parsers already available.
I'll just write down a jackson example here (There must be similar stuff available with gson as well):
String myJson = "{ \"type\" : \"foo\", \"class\" : 3 }";
ObjectMapper objectMapper = new ObjectMapper();
JsonNode node = objectMapper.readValue(myJson, JsonNode.class);
JsonNode type = node.get("type");
System.out.println(type.asText());
As per my experience, the performance difference hasn't been much in both these cases as the libraries handle the parsing quite efficiently but if you have a lot of different types of JSON, then it doesn't make any sense to create POJOs out of each and every JSON (So much mapping !!).
I have personally not worked with gson because of performance reasons like this but you can create what's called an ObjectMapper using jackson and use it quite efficiently. I assume there should be similar thing in gson as well.
The only downside would be that every field will now be accessible through a string input which might make your code a bit unreadable.
EDIT:
If you really wish to iterate through all the fields of the json, you'll have to do DFS ultimately, but parsing can still be avoided by using the fields method of the JsonNode.
root.fields() will return with an iterator with the entries in it which can be then gracefully used as described in answer here.
Remember, similar stuff with different names will be available in gson too. :)
Related
I'm trying to parse the Wikidata JSON dump using the Gson streaming API, since the file is around 70GB of json. The overall structure of the file is as follows:
[
{"type":"item",... other fields ...},
{"type":"property",... other fields ...},
.....
]
It is an array of objects in which each object can be of type item or property and I would like to instantiate a different class (namely I have a corresponding Item and Property class in my Java code) according to the object that I encounter.
Basically, I'd like to look at the type field and then parse the following JSON accordingly. Since the JsonReader doesn't seem to provide a getNextJsonObject() or similar function, is there a way to do this besides preprocessing the whole file and splitting the entries into two separate ones? The file is so big that I'd like to avoid the extra preprocessing step when I could do everything on the fly.
I actually found a very easy solution after a bit of thinking. The Gson API provides the method:
Gson.fromJson(JsonReader reader, Class class)
This will read the next object from the reader and deserialize to the class you pass as parameter. Since in my case I don't know which class to serialize to I can do the following:
JsonObject asd = gson.fromJson(reader, JsonObject.class);
if (asd.get("type").getAsString().equals("item")) {
// Instantiate item
} else {
// Instantiate property
}
I have a JSON file that is marshalled into custom object using GSON.
All works fine.
An example of the structure I have:
public class Domain {
private String id;
private String name;
//other fields
private ArrayList<Structures> structs;
private ArrayList<Buildings> buildings;
private ArrayList<CustomObject> objects;
// some more lists and fields
}
So basically I create a builder and parse the json
Gson gson = new GsonBuilder().create();
gson.fromJson(jsonString, Domain.class);
This works absolutely fine. Parsing is done and I get my data.
Problem:
I don't really need to have various fields of the Domain class populated from the start because e.g. I may have the Domain class with a lot of elements in the e.g. list for structs but I might not really need to access them.
What I need to do is some kind of pattern for lazy loading.
I.e. I would like to not load some parts of class during the json parsing and only load them when needed.
Question:
If I understand correctly the way to skip fields from being parsed is by making them transient.
But then if at some later time I need to access e.g. the structs how would I actually load them at that point? I think that reloading/reparsing all the json again is suboptimal.
What is a standard/good approach for this?
This is a really lengthy topic. There are many approaches to this and all of them are usually a lot more complicated. The easiest one, if you really value something very simple for me was so far not using gson, but for example something like JSONObject and then populate the object myself. using this you could easily split this up into multiple steps. The problem that now arises is, that you never know, what exactly is already loaded - or more - what is maybe loaded, but just not filled as a field.
Lazy loading using automatic conversions like gson is unfortunately always gonna involve unnecessary object creation too, so question then is if its not less pain just to do it yourself from the beginning.
if it has to be gson, you could have different objects for different stages. read them in through json and then apply to your main object.
a favourable version of that is probably to split up the object into different components (or aspects or whatever you want to call it) that match the different loading stages. Different possibilities but lets just pick one of them:
class Domain {
private String id;
private DomainStructs domainStructs;
}
class DomainStructs {
private ArrayList<Structures> structs;
}
Now you need a new Object in this version of doing this. This means the overall size of the model is slightly (but not much really) bigger and you should probably match together things that are necessary together anyway - so not load every field separate, but this way you can leave out parts and easily add them later by populating them from Gson like 2 steps here:
Gson gson = new GsonBuilder().create();
Domain domain = gson.fromJson(jsonString, Domain.class); // first time
domain.structs = gson.fromJson(jsonString, DomainStructs.class); // now adding
I am not saying this is the best idea ever, but it fulfills your idea while still using gson.
I would though consider splitting up the Data already - so not storing the strings, but holding the data in different components in this case if it is possible. Basically you want a domainJsonString and a domainStructsJsonString if you get what i mean. stored in a way so you can easily retrieve them separately.
I hope this helps you to move a bit forward
So basically, I'm using GSON to parse some JSON on Android, and when grabbing a List of objects using JavaBeans I came across an issue - one of the object types has 2 variants
These 2 JSON objects are exactly identical except there can either be a String field (field 'a'), or an object field (field 'b')
Is it possible to get GSON to accept a JSON object with either one of these 2 options, and just leave the field that's not included in the JSON as null in the Java object?
Thanks
I recommend using either Jackson or Moshi for working with JSONs and avoid using Gson in your next project, as it is much slower for deserialization and while marginally faster than Moshi at serializing objects, the resulting JSON might be larger due to the HTML-compatible escape characters that Gson inserts into it.
I want a method that can convert an object (with a few parameters String or int) in a String, and more importantly, convert that String back in an object. Do you know any way to develop a converter for this purpose?
The problem is that the object is a DTO used by other developpers so I can't modify the toString() method. Thus I created my own method toString, but I don't know how can I reliably make a converter working both ways. I thought about several solutions (storing the objects in a map, etc), but they all seem unclear to read/understand. Is there any usual way to do it that every experienced programmers would understand in a blink of an eye what it does?
EDIT: Well, I know serialization, but my question wasn't clear, I'm sorry. The fact is that I want to display the objects in a ListView (JavaFX), to make them readable, like String "N°:xxxx; Version : 1", serialized objects aren't really esthetic nor readable.
So my final solution is something more like:
listView.setCellFactory(new Callback<ListView<ProductDto>, ListCell<ProductDto>>() {
#Override
public ListCell<ProductDto> call(final ListView<ProductDto> productDtoListView) {
return new ListCell<ProductDto>(){
#Override
protected void updateItem(final ProductDto productDto, final boolean b) {
if(productDto!=null){
setText(fromProductDtoToString(productDto));
}
}
};
}
});
This way, I store the objects themselves in the ListView, and can still display a fairly esthetic String representing the object.
Thanks for your time.
Fabien
You can for example use Jackson to convert object to JSON and vice versa. Check this article How To Convert Java Object To / From JSON
If you want, you can use XML binding too.
In general, there are number of serialization libraries/approaches:
XStream (for XML but not for JSON)
Jackson (for JSON)
Jackson xml module (faster than XStream)
Kryo (a fast, compact binary serialization format)
Smile (a binary format that comes with Jackson 1.6 and later).
Java Object Serialization.
SimpleXML seems solid, runs at 2x the speed of XStream, but requires more configuration effort
YamlBeans
SnakeYAML
Jackson JSON, Kryo, and Jackson Smile are all significantly faster than good old Java Object Serialization, by about 3x to 4.5x. XStream is on the slow side.
I have the following String code and I want to convert to json and iterate through items and put them into an arraylist in Java.
I also have a class called Users whith its attributes and getters/setters(id, nick, age, online, avatar)how should i do:
code:
//this is the real code
String a = "{"id":"1","nick":"jhon","age":20,"online":1,"avatar":"http:\/\/www.example.com\/image.jpeg"},{"id":"2","nick":"mike","age":45,"online":0,"avatar":"http:\/\/www.example.com\/image.jpeg"},{"id":"3","nick":"carl","age":12,"online":1,"avatar":"http:\/\/www.example.com/image.jpeg"},{"id":"4","nick":"ana","age":22,"online":0,"avatar":"http:\/\/www.example.com\/image.jpeg"}";
//this is what i want to do
String a = real code sample;
Json b = a.toJson; // something like this
Arraylist<User> list = new Arraylist<User>();
for each b{
list.add( new user(b.getId(),b.getNick()....));
}
i want to do something like that and of course the code is an example and here it's not well written.
You may prefer to use Gson library which has good documentation for how to use. You can easly parse objects to JSON and JSON to object. So it may be a better alternative for your own parser.
It also supports Arrays and Collections.
I think you need to validate you array first on JSONLINT
Jackson is quite nice for this sort of thing - I've been using it for some time with no problems. I've heard nice things about Gson as well, but had no reason to switch from Jackson. Xstream also can work with Json.
Instead of working with a special "JSON" object, Jackson (and I believe Gson as well) stuff the JSON values into a POJO. You use the annotations to customize how the values are read and written.
FWIW, JSON processing is easy with a library like this, but also something you don't want to do much - it's relatively expensive in terms of CPU. It's not that the libs are slow - that kind of text processing is a lot of work.