I have to turn a List<Map> into a JSON string. The Maps are flat and containing primitive and String data. Right now I'm using GSON. Basically like this:
List<Map> list = new ArrayList();
Map map = new HashMap();
map.put("id",100);
map.put("name","Joe");
map.put("country","US");
// ...
list.add(map);
Gson gson = new GsonBuilder().disableHtmlEscaping().setPrettyPrinting().create();
Stopwatch sw = Stopwatch.createStarted(); // guava's stopwatch
String s = gson.toJson(list);
System.err.println("toJson laps " + sw);
return s;
The list may have 100 entries and each map aprox. 20 fields. GSON really takes long time to create the JSON string. The JSON string will be returned by a HTTP response and right now it took too much time (8000ms). So I try other ones: json-smart, jackson, etc. But no one gives a significant speed boost. I trace the JSON string creation as the hot spot in execution time.
For 100 x 20 fields I really won't expect more than a second but it takes significant much more time. Is there any heal for this ?
Update
I have overseen some BLOB data that is returned. Thank you all.
You better use Jackson2
https://github.com/FasterXML/jackson
Java EE 7 has also a new JSON processing API, brand new!
http://docs.oracle.com/javaee/7/api/javax/json/package-summary.html
Profiling the different libs will provide answers. For enterprise applications, I never use GSON.
If you want to write benchmarks, use JMH. Writing high quality benchmarks isn't easy and JMH does a lot of the heavy lifting for you (although there are still a few gotchas).
If you wonder which Java Json library serializes/deserializes faster for various payload sizes, you can look at this extensive benchmark which compares a dozen of them with JMH. Fasted is dsljson, second is jackson. Gson is actually pretty slow for a 'modern' lib.
Related
I have a large JSON file roughly around 300 MB, I am parsing it using Jackson object mapper:-
private void parseJson(Object obj) {
ObjectMapper map = new ObjectMapper();
Map<String,List<POJOClass>> result = new HashMap<>();
String str = map.writeValueAsString(obj);
map.registerModule(new JSR10module());
result = map.readValue(new ByteInputStream(str.getBytes(StandardCharSets.UTF_8)),
new TypeReference<Map<String,List<POJOClass>>>
() {});
}
Parameter to parseJson is an Object which contains JSON String.
This works fine for JSON files of around 150 MB, however, it starts to fail with Heap Space error when JSON files are around 250-300 MB. I am using jackson 2.4.0
You don't have enough memory in your java process to handle such huge file.
When launching your app use the -xmx option to increase the maximum memory used by your java process:
java -Xmx512m ...
or
java -Xmx1G ...
Have you tried streaming over the JSON using, for instance: https://www.baeldung.com/jackson-streaming-api so that you do not have to put everything into memory at once.
For huge big json file, you should use jackson stream api.
You can find the document here. It cost little memory than normal way. In general, your POJO must satisfy certain shapes, it should be able to be separated into many independent parts. Such as a list of object. Then with jackson stream api, you can achieve the object in the list one by one.
My spring application does a rest request to a server and the response from the server is a JSONObject string. The JSON string is very huge(200MB). I want to convert the json string to a JSONObject. Below are my code for conversion:
exchange = restTemplate.exchange(Url, HttpMethod.POST, postEntity, String.class);
jsonObject = objectMapper.readValue(exchange.getBody(), JSONObject.class);
For a single request, it is taking 3-5 seconds for conversion. But, if there are multiple requests the conversion is taking so much time (60 seconds for 8-10 requests in parallel).
Is there any better way to do this?
I'd say that transforming a 200MB chunk of JSON to an object using jackson-databind's ObjectMapper will almost always consume a lot of computing time and furthermore huge amounts of memory.
If you don't need the whole object represented by the JSON in memory at a single time, i. e. chunks of it will suffice, I would advise to switch to an approach that utilizes jackson's streaming API. You can combine it with databind on smaller subsets of the JSON, passing the resulting DTOs to some consumer (kind of visitor pattern).
I hope this is of any help for your special use-case.
I am currently writing an application in Java, and am struggling to extract the values from a String which is in a JSON format.
Could someone help me with the easiest, most simplest way to extract data from this string? I'd prefer not to use external library if at all possible.
{"exchange":{"status":"Enabled","message":"Broadband on Fibre Technology","exchange_code":"NIWBY","exchange_name":"WHITEABBEY"},"products":[{"name":"20CN ADSL Max","likely_down_speed":1.5,"likely_up_speed":0.15,"availability":true....
Could someone explain how I could return the "likely down speed" of "20CN ADSL Max for example?
Thanks
Currently , there is no way in Java to parse json without an external lib (or your own implementation).
The org.json library is a standard when working with JSON.
You can use this snippet along with the library to achieve what you asked:
JSONObject obj = new JSONObject(" .... ");
JSONArray arr = obj.getJSONArray("products");
for (int i = 0; i < arr.length(); i++) {
String name = arr.getJSONObject(i).getString("name");
if ( name.equals("20CN ADSL Max") ) {
String s = arr.getJSONObject(i).getString("likely down speed");
}
}
Hope this helps.
For sure it's possible to do the parsing yourself, but it'll be much faster if you rely upon an existing library such as org.json.
With that, you can easily convert the string into a JSON object and extract all the fields you need.
If an existing library is not an option, you'll need to build yourself the tree describing the object in order to extract the pair key-values
While this may seem like a very simple, straightforward task, it gets rather complicated rather quickly.
Check out the SO thread How to parse JSON in Java. There is unfortunately not a single, clear solution to that question as shown in that thread. But I guess the org.json library seems to be the most popular solution.
If your application needs to handle arbitrary JSON, I would advise against trying to build your own parser.
Whatever your objections are to using an external library, get over them.
I used to use new JSONObject(string) to convert string to JSONObject. however, it is too slow performance-wise. Anybody have the faster solution?
Take a look at Jackson. They claim to be faster than any other Java JSON parser. It also parses the data in a stream, lowering memory consumption.
I've used Gson with some good success: http://code.google.com/p/google-gson/
I want to create a string which will look like the following:
"[{"product_id":"123","name":"stack"},{"product_id":"456","name":"overflow"}]"
I have the product_id and name in two arrays. How can I create the above type of string most efficiently?
This looks like JSON. You should use a JSON library.
There are plenty out there (see this page, scroll almost to the end), but
Gson (Sample Usage) or
Jackson (Sample Usage)
are the ones I'd use.
If it turns out you already have a List<ProductInformation>, then the default serialization with flexjson with an exclude will give you the string:
String jsonResult = new JSONSerializer().exclude("*.class").serialize(listOfProductInformation);
Regarding performance, your best bet is to use a good profiler to look at your overall application. This will identify your true hotspots/bottlenecks and if it turns out this serialization process is a hotspot, you can spend time tuning as #Ali mentions and running it back through the profiler or performance test harness to measure the impact.
You may want to have a look on the Java String Formatter:
http://download.oracle.com/javase/6/docs/api/java/util/Formatter.html
I'm assuming you have 2 arrays one containing and product_id and the other containing the values and both of these are in correct order.
I would create my own java object (POJO) that had two fields product_id and name then take your two arrays and create one array (or list) of POJOs each POJO containing the product_id and name. Then I would just use Jackson or Gson to create my JSON.
These libraries give you a JSON representation of a Java Objects, so in this case you will have to make sure that you have a list or even an array of objects which contain your product_id and name.
If you insist on doing it the hard-way (or no external library way), then I would create a template String and repeatedly call replace on it and add it to a StringBuffer. The StringBuffer is important if the string can be very large. Something like:
String template = "{\"product_id\":\"productId\",\"name\":\"productName\"}";
StringBuffer result = new StringBuffer("[");
for(int i=0; i<myProductArray.length; i++){
String temp = template.replace("productId",myProductArray[i]);
temp = temp.replace("productName",myNameArray[i]);
if(result.length() > 1)
result.append(",");
result.append(temp);
}
result.append("]");
return result.toString();
If you can replace the template String with a StringBuffer and manipulate that directly, it'll make a ton of difference performance wise for large Strings.
Regards,