We Keep Coding

Query a JSON file with Java-Large file

I am trying to parse below JSON file using java.
I need to be able to
search the file by id or name or any of the fields in the object.
search for empty values in the field as well.
The search should return entire object.
The File will be huge and the search should still be time efficient.
[
{
"id": 1,
"name": "Mark Robb",
"last_login": "2013-01-21T05:13:41 -11:30",
"email": "markrobb#gmail.com",
"phone": "12345",
"locations": [
"Germany",
"Austria"
]
},
{
"id": 2,
"name": "Matt Nish",
"last_login": "2014-02-21T07:10:41 -11:30",
"email": "mattnish#gmail.com",
"phone": "456123",
"locations": [
"France",
"Italy"
]
}
]
This is what I have tried so far using Jackson library.
public void findById(int id) {
List<Customer> customers = objectMapper.readValue(new File("src/main/resources/customers.json"), new TypeReference<List<Customer>>(){});
for(Customer customer: customers) {
if(customer.getId() == id) {
System.out.println(customer.getName());
}
}
}
I just don't think this is an efficient method for a huge JSON file(About 20000 customers in a file). And there could be multiple files. Search time should not increase linearly.
How can I make this time efficient? Should I use any other library?

The most efficient (both CPU and memory) way to parse is to use stream oriented parsing instead of object mapping. Usually, it takes a bit more code to be written, but also usually it is a good deal :) Both Gson and Jackson support such lightweight technique. Also, you should avoid memory allocation in the main/hot path to prevent GC pauses. To illustrate the idea I use a small GC-free library https://github.com/anatolygudkov/green-jelly:
import org.green.jelly.*;
import java.io.CharArrayReader;
import java.io.Reader;
import java.util.ArrayList;
import java.util.List;
public class SelectById {
public static class Customer {
private long id;
private String name;
private String email;
public void clear() {
id = 0;
name = null;
email = null;
}
public Customer makeCopy() {
Customer result = new Customer();
result.id = id;
result.name = name;
result.email = email;
return result;
}
#Override
public String toString() {
return "Customer{" +
"id=" + id +
", name='" + name + '\'' +
", email='" + email + '\'' +
'}';
}
}
public static void main(String[] args) throws Exception {
final String file = "\n" +
"[\n" +
" {\n" +
" \"id\": 1,\n" +
" \"name\": \"Mark Robb\",\n" +
" \"last_login\": \"2013-01-21T05:13:41 -11:30\",\n" +
" \"email\": \"markrobb#gmail.com\",\n" +
" \"phone\": \"12345\",\n" +
" \"locations\": [\n" +
" \"Germany\",\n" +
" \"Austria\"\n" +
" ]\n" +
"},\n" +
" {\n" +
" \"id\": 2,\n" +
" \"name\": \"Matt Nish\",\n" +
" \"last_login\": \"2014-02-21T07:10:41 -11:30\",\n" +
" \"email\": \"mattnish#gmail.com\",\n" +
" \"phone\": \"456123\",\n" +
" \"locations\": [\n" +
" \"France\",\n" +
" \"Italy\"\n" +
" ]\n" +
" }\n" +
"]\n";
final List<Customer> selection = new ArrayList<>();
final long selectionId = 2;
final JsonParser parser = new JsonParser().setListener(
new JsonParserListenerAdaptor() {
private final Customer customer = new Customer();
private String currentField;
#Override
public boolean onObjectStarted() {
customer.clear();
return true;
}
#Override
public boolean onObjectMember(final CharSequence name) {
currentField = name.toString();
return true;
}
#Override
public boolean onStringValue(final CharSequence data) {
switch (currentField) {
case "name":
customer.name = data.toString();
break;
case "email":
customer.email = data.toString();
break;
}
return true;
}
#Override
public boolean onNumberValue(final JsonNumber number) {
if ("id".equals(currentField)) {
customer.id = number.mantissa();
}
return true;
}
#Override
public boolean onObjectEnded() {
if (customer.id == selectionId) {
selection.add(customer.makeCopy());
return false; // we don't need to continue
}
return true;
}
}
);
// now let's read and parse the data with a buffer
final CharArrayCharSequence buffer = new CharArrayCharSequence(1024);
try (final Reader reader = new CharArrayReader(file.toCharArray())) { // replace by FileReader, for example
int len;
while((len = reader.read(buffer.getChars())) != -1) {
buffer.setLength(len);
parser.parse(buffer);
}
}
parser.eoj();
System.out.println(selection);
}
}
It should work almost as fast as possible in Java (in case we cannot use SIMD instructions directly). To get rid of memory allocation at all (and GC pauses) in the main path, you have to replace ".toString()" (it creates new instance of String) by something reusable like StringBuilder.
The last thing which may affects overall performance is method of the file reading. And RandomAccessFile is one of the best options we have in Java. Since your encoding seems to be ASCII, just cast byte to char to pass to the JsonParser.

It should be possible to do this with Jackson. The trick is to use JsonParser to stream/parse the top-level array and then parse each record using ObjectMapper.readValue().
ObjectMapper objectMapper = new ObjectMapper();
File file = new File("customers.json");
try (JsonParser parser = objectMapper.getFactory().createParser(file))
{
//Assuming top-level array
if (parser.nextToken() != JsonToken.START_ARRAY)
throw new RuntimeException("Expected top-level array in JSON.");
//Now inside the array, parse each record
while (parser.nextToken() != JsonToken.END_ARRAY)
{
Customer customer = objectMapper.readValue(parser, Customer.class);
//Do something with each customer as it is parsed
System.out.println(customer.id + ": " + customer.name);
}
}
#JsonIgnoreProperties(ignoreUnknown = true)
public static class Customer
{
public String id;
public String name;
public String email;
}
In terms of time efficiency it will need to still scan the entire file - not much you can do about that without an index or something fancier like parallel parsing. But it will be more memory efficient than reading the entire JSON into memory - this code only loads one Customer object at a time.
Also:
if(customer.getId() == id) {
Use .equals() for comparing strings, not ==:
if (customer.getId().equals(id)) {

You can try the Gson library. This library implements a TypeAdapter class that converts Java objects to and from JSON by streaming serialization and deserialization.
The API is efficient and flexible especially for huge files. Here is an example:
public class GsonStream {
public static void main(String[] args) {
Gson gson = new Gson();
try (Reader reader = new FileReader("src/main/resources/customers.json")) {
Type listType = new TypeToken<List<Customer>>(){}.getType();
// Convert JSON File to Java Object
List<Customer> customers = gson.fromJson(reader, listType);
List<Customer> names = customers
.stream()
.filter(c -> c.getId() == id)
.map(Customer::getName)
.collect(Collectors.toList());
} catch (IOException e) {
e.printStackTrace();
}
}
}
If you want to understand how to Override the TypeAdapter abstract class here you have and example:
public class GsonTypeAdapter {
public static void main(String args[]) {
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(Customer.class, new customerAdapter());
builder.setPrettyPrinting();
Gson gson = builder.create();
try {
reader = new JsonReader(new FileReader("src/main/resources/customers.json"));
Customer customer = gson.fromJson(jsonString, Customer.class);
System.out.println(customer);
jsonString = gson.toJson(customer);
System.out.println(jsonString);
} catch (IOException e) {
e.printStackTrace();
}
}
}
class customerAdapter extends TypeAdapter<Customer> {
#Override
public customer read(JsonReader reader) throws IOException {
Customer customer = new customer();
reader.beginObject();
String fieldName = null;
while (reader.hasNext()) {
JsonToken token = reader.peek();
if (token.equals(JsonToken.NAME)) {
//get the current token
fieldName = reader.nextName();
}
if ("name".equals(fieldName)) {
//move to next token
token = reader.peek();
customer.setName(reader.nextString());
}
if("id".equals(fieldName)) {
//move to next token
token = reader.peek();
customer.setRollNo(reader.nextInt());
}
}
reader.endObject();
return customer;
}
#Override
public void write(JsonWriter writer, Customer customer) throws IOException {
writer.beginObject();
writer.name("name");
writer.value(customer.getName());
writer.name("id");
writer.value(customer.getId());
writer.endObject();
}
}
class Customer {
private int id;
private String name;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String toString() {
return "Customer[ name = " + name + ", id: " + id + "]";
}
}

Read JSON Nested element using reflection without using JsonFactory or ObjectMapper

[Unable to access property of another object stored in Arraylist]
I am creating an function to get JSON input in object from RESTful Web service input and format it again in JSON format to call other web service.
I have limitation that I can not use any JSON API for object mapping hence using Java reflection core API.
I am able to create JSON format from Input for simple elements but unable to access nested elements (another user defined POJO class ). I am using arraylist.
Input
{
"GenesisIncidents": {
"service": "Transmission",
"affectedCI": "22BT_ORNC03",
"opt_additionalAffectedItems": [
{
"itemType": "NODE-ID",
"ItemName": "22BT_ORNC03"
},
{
"ItemType": "CCT",
"ItemName": "A_circuit_id"
}]
}
}
GenesisIncidents.class
import java.util.ArrayList;
import java.util.Date;
public class GenesisIncidents {
private String service;
private String affectedCI;
private ArrayList<AdditionalAffectedItems> opt_additionalAffectedItems;
public GenesisIncidents(){}
public String getService() {
return service;
}
public void setService(String service) {
this.service = service;
}
public String getAffectedCI() {
return affectedCI;
}
public void setAffectedCI(String affectedCI) {
this.affectedCI = affectedCI;
}
public ArrayList<AdditionalAffectedItems> getOpt_additionalAffectedItems() {
return opt_additionalAffectedItems;
}
public void setOpt_additionalAffectedItems(ArrayList<AdditionalAffectedItems> opt_additionalAffectedItems) {
this.opt_additionalAffectedItems = opt_additionalAffectedItems;
}
}
AdditionalAffectedItems.class
public class AdditionalAffectedItems {
private String itemType;
private String itemName;
public AdditionalAffectedItems(){
super();
}
public String getItemType() {
return itemType;
}
public void setItemType(String itemType) {
this.itemType = itemType;
}
public String getItemName() {
return itemName;
}
public void setItemName(String itemName) {
this.itemName = itemName;
}
}
Implemetation
public void updateTicketExt(GenesisIncidents genesisIncidents) {
try{
Field allFields[]=genesisIncidents.getClass().getDeclaredFields();
Method allMethods[] = genesisIncidents.getClass().getDeclaredMethods();
String jsonString ="{\r\n \""+genesisIncidents.getClass().getName().toString().substring(48)+"\": {";
final String preStr="\r\n \""; //To create a JSON object format.
final String postStr="\": "; //To create a JSON object format.
int totalNoOfFields=allFields.length;
for (Field field : allFields) {
System.out.println(field.getType());
String getter="get"+StringUtils.capitalize(field.getName());
Method method= genesisIncidents.getClass().getMethod(getter, null);
try{
if(field.getType().toString().contains("Integer"))
jsonString=jsonString + preStr + field.getName() + postStr +method.invoke(genesisIncidents).toString()+",";
else
jsonString=jsonString + preStr + field.getName() + postStr +"\""+method.invoke(genesisIncidents).toString()+"\",";
if(field.getType().toString().contains("ArrayList")){
System.out.println("ArrayListElement found");
genesisIncidents.getOpt_additionalAffectedItems().forEach(obj->{System.out.println(obj.getItemName());});
//convertArrayToJSON(field, genesisIncidents);
}
}catch(NullPointerException npe)
{
System.out.println("Null value in field.");
continue;
}
}
jsonString=jsonString.substring(0,jsonString.length()-1);
jsonString=jsonString+"\r\n }\r\n }";
System.out.println("\n"+jsonString);
}catch(Exception jex){
jex.printStackTrace();
}
}
My below code line is unable to access object stored under array list.
genesisIncidents.getOpt_additionalAffectedItems().forEach(obj->{System.out.println(obj.getItemName());});
OUTPUT
karaf#root>class java.lang.String
class java.lang.String
class java.lang.String
class java.util.ArrayList
ArrayListElement found
null
null
{
"GenesisIncidents": {
"service": "Transmission",
"affectedCI": "22BT_ORNC03",
"opt_additionalAffectedItems": " [org.apache.servicemix.examples.camel.rest.model.AdditionalAffectedItems#5881a 895, org.apache.servicemix.examples.camel.rest.model.AdditionalAffectedItems#399b4e eb]"
}
}

I have fiddled around with your example I have managed to get it working. This will produce the correct JSON string by passing in an instance of a GenesisIncident object. I guess that there is much room for improvement here but this can serve as an example.
public static String genesisToJson(GenesisIncidents incidents) {
try{
StringBuilder jsonBuilder = new StringBuilder();
jsonBuilder.append("{\r\n \"")
.append(incidents.getClass().getSimpleName())
.append("\": {");
Field allFields[] = incidents.getClass().getDeclaredFields();
for (Field field : allFields) {
String getter = getGetterMethod(field);
Method method = incidents.getClass().getMethod(getter, null);
try{
if(field.getType().isAssignableFrom(Integer.class)) {
jsonBuilder.append(preStr).append(field.getName()).append(postStr)
.append(method.invoke(incidents).toString()).append(",");
} else if (field.getType().isAssignableFrom(String.class)) {
jsonBuilder.append(preStr).append(field.getName()).append(postStr).append("\"")
.append(method.invoke(incidents).toString()).append("\",");
} else if (field.getType().isAssignableFrom(List.class)) {
System.out.println("ArrayListElement found");
getInnerObjectToJson(field, incidents.getOptItems(), jsonBuilder);
}
} catch(NullPointerException npe) {
System.out.println("Null value in field.");
continue;
}
}
jsonBuilder.append("\r\n } \r\n }");
return jsonBuilder.toString();
}catch(Exception jex){
jex.printStackTrace();
}
return null;
}
private static void getInnerObjectToJson(Field field, List<AdditionalAffectedItems> items, StringBuilder builder)
throws NoSuchMethodException, InvocationTargetException, IllegalAccessException {
builder.append(preStr).append(field.getName()).append(postStr).append("[");
for (var item : items) {
var fields = List.of(item.getClass().getDeclaredFields());
builder.append("{");
for (var f : fields) {
String getter = getGetterMethod(f);
Method method = item.getClass().getMethod(getter, null);
builder.append(preStr).append(f.getName()).append(postStr).append("\"")
.append(method.invoke(item).toString()).append("\"");
if (!(fields.indexOf(f) == (fields.size() - 1))) {
builder.append(",");
}
}
if (items.indexOf(item) == (items.size() - 1)) {
builder.append("}\r\n");
} else {
builder.append("},\r\n");
}
}
builder.append("]");
}
private static String getGetterMethod(Field field) {
return "get" + StringUtils.capitalize(field.getName());
}

Extracting the value of an element from a JSON

I'm trying to make a test where I get some documents based on the id of the batch they belong to. More specifically, I want to check that a specific batchPublicId is in the response body. I am using okhttp for the test.
This a shorter version of the json:
{
"_embedded": {
"invoices": [
{
"type": "INVOICE",
"publicId": "27bc8426-17cf-4fe5-9278-64108ae05e4b",
"deliveryStatus": null,
"processingStatus": "INITIATED",
"batchPublicId": "0000000000000000000000001"
}
]
}
}
I'm new to json and this is how far I got with the problem:
String invoicesJsonData = response.body().string();
JSONObject invoicesJsonObject = new JSONObject(invoicesJsonData);
Assert.assertTrue(invoicesJsonObject.getJSONObject("_embedded") !=null && invoicesJsonObject.getJSONObject("_embedded").has("invoices"));
I would like to verify that batchPublicId has the value mentioned in the json. Is there a way to do this? Thank you.

String invoicesJsonData = response.body().string();
JSONObject invoicesJsonObject = new JSONObject(invoicesJsonData);
JSONObject invoicesJsonObject1 = invoicesJsonObject.getJSONObject("_embedded");
JSONArray f2=invoicesJsonObject1.getJSONArray("invoices");
for(int i=0;i<f2.length();i++){
JSONObject obj=f2.getJSONObject(i);
if(obj.get("batchPublicId")!=null){
System.out.println(obj.get("batchPublicId"));
}

You can do something like this,Which worked out for me sometimes back.
String invoicesJsonData = response.body().string();
JSONObject invoicesJsonObject = new JSONObject(invoicesJsonData);
JSONObject invoicesJsonObject = json.getJSONObject("invoicesJsonObject");
String batchPublicId = invoicesJsonObject.getString("batchPublicId");
System.out.println( "batchPublicId: " + batchPublicId );
if(batchPublicId !=null){
// do something
}
Not sure about the syntax.Giving you a hint.

you can check any keys is there in json object or not like below :
if(jsonObject1.has("batchPublicId")){
String batchPublicId = jsonObject1.optString("batchPublicId");
Log.i(getClass().getSimpleName(), "batchPublicId=" + batchPublicId);}
has method is used to find any key is there in jsonobject or not.

In my opinion, a better approach for this would be to create a POJO from this JSON string, and extract the information you need using simply the getters
For example:
Wrapper class:
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonRootName(value = "_embedded")
public class Embeded {
#JsonProperty("invoices")
private List<Invoice> invoices;
public Embeded() {}
public List<Invoice> getInvoices() {
return invoices;
}
public void setInvoices(List<Invoice> invoices) {
this.invoices = invoices;
}
}
Invoice class:
#JsonIgnoreProperties(ignoreUnknown = true)
public class Invoice {
#JsonProperty("type")
private String type;
#JsonProperty("publicId")
private String publicId;
#JsonProperty("deliveryStatus")
private String deliveryStatus;
#JsonProperty("processingStatus")
private String processingStatus;
#JsonProperty("batchPublicId")
private String batchPublicId;
public Invoice() {}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getPublicId() {
return publicId;
}
public void setPublicId(String publicId) {
this.publicId = publicId;
}
public String getDeliveryStatus() {
return deliveryStatus;
}
public void setDeliveryStatus(String deliveryStatus) {
this.deliveryStatus = deliveryStatus;
}
public String getProcessingStatus() {
return processingStatus;
}
public void setProcessingStatus(String processingStatus) {
this.processingStatus = processingStatus;
}
public String getBatchPublicId() {
return batchPublicId;
}
public void setBatchPublicId(String batchPublicId) {
this.batchPublicId = batchPublicId;
}
}
Test:
public void json_test() throws JsonParseException, JsonMappingException, IOException {
String json = "{"
+ "\"_embedded\": {"
+ "\"invoices\": ["
+ "{"
+ "\"type\": \"INVOICE\","
+ "\"publicId\": \"27bc8426-17cf-4fe5-9278-64108ae05e4b\","
+ "\"deliveryStatus\": null,"
+ "\"processingStatus\": \"INITIATED\","
+ "\"batchPublicId\": \"0000000000000000000000001\""
+ "}"
+ "]"
+ "}"
+ "}";
ObjectMapper mapper = new ObjectMapper();
mapper.configure(Feature.UNWRAP_ROOT_VALUE, true);
List<Invoice> invoices = mapper.readValue(json, Embeded.class).getInvoices();
Assert.assertTrue(StringUtils.equals(invoices.get(0).getBatchPublicId(), "0000000000000000000000001"));
}

If I understand your right, you just need to call:
Assert.assertTrue(invoicesJsonObject.getString("batchPublicId").equals("0000000000000000000000001"));"

If you want to create a test for JSON Validation, you can use the JSONAssert.
JSONAsset give the method assertEquals, that compare two json structures, strict identic or not.
final String expected_result = YOUR_EXPECTED_RESULT;
JSONAssert.assertEquals(YOUR_EXPECTED_JSON_RESULT, RESULT_FROM_RESPONSE_BODY, false);
The last boolean parameter defines if you want an strict comparation or just compare if your expected result is in result from response.

Generic tuple de-serialization in Jackson

It so happens that I need to support in Java JSON data coming from external data sources. There is one common pattern. It's an array containing fixed number of elements of certain different types. We call it tuple. Here is my example of de-serialization for 3-element tuple with particular expected types of elements using FasterXML Jackson:
public class TupleTest {
public static void main(String[] args) throws Exception {
String person = "{\"name\":\"qqq\",\"age\":35,\"address\":\"nowhere\",\"phone\":\"(555)555-5555\",\"email\":\"super#server.com\"}";
String jsonText = "[[" + person + ",[" + person + "," + person + "],{\"index1\":" + person + ",\"index2\":" + person + "}]]";
ObjectMapper om = new ObjectMapper().registerModule(new TupleModule());
List<FixedTuple3> data = om.readValue(jsonText, new TypeReference<List<FixedTuple3>>() {});
System.out.println("Deserialization result: " + data);
System.out.println("Serialization result: " + om.writeValueAsString(data));
}
}
class Person {
public String name;
public Integer age;
public String address;
public String phone;
public String email;
#Override
public String toString() {
return "Person{name=" + name + ", age=" + age + ", address=" + address
+ ", phone=" + phone + ", email=" + email + "}";
}
}
class FixedTuple3 {
public Person e1;
public List<Person> e2;
public Map<String, Person> e3;
#Override
public String toString() {
return "Tuple[" + e1 + ", " + e2 + ", " + e3 + "]";
}
}
class TupleModule extends SimpleModule {
public TupleModule() {
super(TupleModule.class.getSimpleName(), new Version(1, 0, 0, null, null, null));
setSerializers(new SimpleSerializers() {
#Override
public JsonSerializer<?> findSerializer(SerializationConfig config,
JavaType type, BeanDescription beanDesc) {
if (isTuple(type.getRawClass()))
return new TupleSerializer();
return super.findSerializer(config, type, beanDesc);
}
});
setDeserializers(new SimpleDeserializers() {
#Override
public JsonDeserializer<?> findBeanDeserializer(JavaType type,
DeserializationConfig config, BeanDescription beanDesc) throws JsonMappingException {
Class<?> rawClass = type.getRawClass();
if (isTuple(rawClass))
return new TupleDeserializer(rawClass);
return super.findBeanDeserializer(type, config, beanDesc);
}
});
}
private boolean isTuple(Class<?> rawClass) {
return rawClass.equals(FixedTuple3.class);
}
public static class TupleSerializer extends JsonSerializer<Object> {
public void serialize(Object value, JsonGenerator jgen, SerializerProvider provider) throws IOException, JsonProcessingException {
try {
jgen.writeStartArray();
for (int i = 0; i < 3; i++) {
Field f = value.getClass().getField("e" + (i + 1));
Object res = f.get(value);
jgen.getCodec().writeValue(jgen, res);
}
jgen.writeEndArray();
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
}
public static class TupleDeserializer extends JsonDeserializer<Object> {
private Class<?> retClass;
public TupleDeserializer(Class<?> retClass) {
this.retClass = retClass;
}
public Object deserialize(JsonParser p, DeserializationContext ctx) throws IOException, JsonProcessingException {
try {
Object res = retClass.newInstance();
if (!p.isExpectedStartArrayToken()) {
throw new JsonMappingException("Tuple array is expected but found " + p.getCurrentToken());
}
JsonToken t = p.nextToken();
for (int i = 0; i < 3; i++) {
final Field f = res.getClass().getField("e" + (i + 1));
TypeReference<?> tr = new TypeReference<Object>() {
#Override
public Type getType() {
return f.getGenericType();
}
};
Object val = p.getCodec().readValue(p, tr);
f.set(res, val);
}
t = p.nextToken();
if (t != JsonToken.END_ARRAY)
throw new IOException("Unexpected ending token in tuple deserializer: " + t.name());
return res;
} catch (IOException ex) {
throw ex;
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
}
}
But this approach means I have to make new class every time I face new type configuration in tuple of certain size. So I wonder if there is any way to define deserializer dealing with generic typing. So that it will be enough to have one tuple class per tuple size. For instance my generic tuple of size 3 could be defined like:
class Tuple3 <T1, T2, T3> {
public T1 e1;
public T2 e2;
public T3 e3;
#Override
public String toString() {
return "Tuple[" + e1 + ", " + e2 + ", " + e3 + "]";
}
}
And usage of it would look like:
List<Tuple3<Person, List<Person>, Map<String, Person>>> data =
om.readValue(jsonText,
new TypeReference<List<Tuple3<Person, List<Person>, Map<String, Person>>>>() {});
Is it something doable or not?

Ok. So... there may be a simpler way to do "tuple"-style. You can actually annotate POJOs to be serialized as arrays:
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
#JsonPropertyOrder({ "name", "age" }) // or use "alphabetic"
public class POJO {
public String name;
public int age;
}
and if so, they'll get written as arrays, read from arrays.
But if you do what to handle custom generic types, you probably need to get type parameters resolved. This can be done using TypeFactory, method findTypeParameters(...). While this may seem superfluous, it is needed for general case if you sub-type (if not, JavaType actually has accessors for direct type parameters).

Yes, you must use Reflection to get ALL FIELDS, not to get the known fields by name.

How i can recover a Map of objects from json?

I have this class
public static class SomeClass {
public SomeClass(String field) {
this.field = field;
}
private final String field;
public String getField() {
return field;
}
}
I have also this test (edited)
#Test
public void testStringifyMapOfObjects() {
Map<String, SomeClass> original = Maps.newTreeMap();
original.put("first", new SomeClass("a"));
original.put("second", new SomeClass("b"));
String encoded = JsonUtil.toJson(original);
Map<String, SomeClass> actual = JsonUtil.fromJson(encoded, Map.class);
Assert.assertEquals("{'first':{'field':'a'},'second':{'field':'b'}}", encoded.replaceAll("\\s", "").replaceAll("\"", "'"));
Assert.assertEquals(original.get("first"), actual.get("first"));
}
The test fails with
junit.framework.AssertionFailedError: expected:<eu.ec.dgempl.eessi.facade.transport.test.TestToolTest$SomeClass#6e3ed98c> but was:<{field=a}>
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:277)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:71)
at eu.ec.dgempl.eessi.facade.transport.test.TestToolTest.testStringifyMapOfObjects(TestToolTest.java:90)
Can I make json to properly serialize objects as the values of the map or should I use something else?
edited
public class JsonUtil {
private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(JsonUtil.class);
public static <T> String toJson(T data) {
ObjectMapper mapper = new ObjectMapper();
mapper.configure(Feature.INDENT_OUTPUT, true);
try {
return mapper.writeValueAsString(data);
} catch (IOException e) {
LOG.warn("can't format a json object from [" + data + "]", e);
return null;
}
//
// return Json.stringify(Json.toJson(data));
}
public static <T> T fromJson(String description, Class<T> theClass) {
try {
JsonNode parse = new ObjectMapper().readValue(description, JsonNode.class);
T fromJson = new ObjectMapper().treeToValue(parse, theClass);
return fromJson;
} catch (JsonParseException e) {
// throw new RuntimeException("can't parse a json object of type " + theClass.getName() + " from [" + description + "]", e);
LOG.warn("can't parse a json object from [" + description + "]", e);
return null;
} catch (JsonMappingException e) {
// throw new RuntimeException("can't parse a json object of type " + theClass.getName() + " from [" + description + "]", e);
LOG.warn("can't parse a json object from [" + description + "]", e);
return null;
} catch (IOException e) {
// throw new RuntimeException("can't parse a json object of type " + theClass.getName() + " from [" + description + "]", e);
LOG.warn("can't parse a json object from [" + description + "]", e);
return null;
}
}
}

You are running into a problem related to Java generics. To summarize, when deserializing data into a non-reifiable type (aka a type for which actual type information is not available at runtime) you need to use a supertype token. You can get more detail about what a supertype token is (and why you need to use one) by reading these SO posts:
Pass parameterized type to method as argument
Error using Jackson and JSON
Deserialize JSON to ArrayList using Jackson
And also from the Jackson documentation:
Data Binding With Generics
TypeReference Javadoc
The basic problem is that when you use a typical generic object, the actual type parameters for the object aren't available at runtime. Therefore Jackson doesn't know which actual class to instantiate and deserialize your data into.
The easiest way to get around the problem would be adding an overload to your JSON utility class, that accepts a type reference (as opposed to a Class<T>). For example:
public static <T> T fromJson(String json, TypeReference<T> typeRef) {
if(json == null || typeRef == null) return null;
return new ObjectMapper().readValue(json, typeRef);
}
To be used as such:
Map<String, SomeClass> actual = JsonUtil.fromJson(
encoded,
new TypeReference<Map<String, SomeClass>>(){});

I discovered that the simplest solution is to create a "container" class that will contain the map. This is working probably because the container has enough type details for the map, as opposed to the case when a map is used directly.
public static class SomeClass {
private final String field;
private SomeClass() {
this("wrong");
}
public SomeClass(String field) {
this.field = field;
}
public String getField() {
return field;
}
#Override
public String toString() {
return "SomeClass[" + field + "]";
}
}
public static class SomeClassContainer {
private final Map<String, SomeClass> all = Maps.newTreeMap();
public Map<String, SomeClass> getAll() {
return all;
}
}
After this ... the updated test is
#Test
public void testStringifyMapOfObjects() {
SomeClassContainer original = new SomeClassContainer();
original.getAll().put("first", new SomeClass("a"));
original.getAll().put("second", new SomeClass("b"));
String encoded = JsonUtil.toJson(original);
System.out.println(encoded);
SomeClassContainer actual = JsonUtil.fromJson(encoded, SomeClassContainer.class);
System.out.println(ObjectUtils.toString(actual));
Assert.assertEquals("{'all':{'first':{'field':'a'},'second':{'field':'b'}}}", encoded.replaceAll("\\s", "").replaceAll("[\"]", "'"));
Assert.assertEquals("class eu.ec.dgempl.eessi.facade.transport.test.TestToolTest$SomeClass", actual.getAll().get("first").getClass().toString());
Assert.assertEquals(original.getAll().get("first").toString(), actual.getAll().get("first").toString());
Assert.assertEquals(original.getAll().get("second").toString(), actual.getAll().get("second").toString());
}