How can I convert a json string like
{
"a.b": 1,
"a.c.d": 2
}
into json string
{
"a": {
"b": 1,
"c": {
"d": 2
}
}
}
by using ObjectMapper?
The easiest option is to create custom deserializer or custom been with #JsonAnySetter and #JsonAnyGetter Here is example:
public static final String json = "{\"a.b\": 1,\"a.c.d\": 2}";
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
HelperBean bean = mapper.readValue(json, HelperBean.class);
System.out.println(mapper.writeValueAsString(bean));
// result: {"a":{"b":1,"c":{"d":2}}}
}
public static class HelperBean{
#JsonIgnore
private Map<String, Object> data = new TreeMap<>();
#JsonAnySetter
public void setDays(String key, Object value){
String[] parts = key.split("\\.");
Map<String, Object> currMap = data;
for (int i = 0; i< parts.length; i++){
String part = parts[i];
Object subMap = currMap.get(part);
if (i == parts.length - 1) // last node
currMap.put(part, value);
else if(subMap == null){ // new node
subMap = new TreeMap<>();
currMap.put(part, subMap);
currMap = (Map<String, Object>) subMap;
}else if (subMap instanceof Map){ // existing node
currMap.put(part, subMap);
currMap = (Map<String, Object>) subMap;
} else { // node conflict
// handle exception when a.b = 1 and a.b.c = 1
}
}
}
#JsonAnyGetter
public Map<String, Object> getData() {
return data;
}
}
Here's the problem:
public static <T> T execute(String query, Function<List<JsonNode>, T> function) {
String jsonString = HttpRequest.get("http://overpass-api.de/api/interpreter", true, "data", query).body();
List<JsonNode> list = toJsonNodeList(jsonString);
T res = function.apply(list);
return res;
}
This method:
perform a query that returns a json string
transform that string into a JsonNode list
finally, transform each JsonNode into a particular object
This is an example of function that convert each JsonNode into a geojson geometry and returns a geojson result set:
public class GeojsonMapper implements Function<List<JsonNode>, GeojsonSingleListResultSet> {
#Override
public GeojsonSingleListResultSet apply(List<JsonNode> list) {
List<Element> elementList = list.parallelStream()
.map(jsonNode -> {
String id = jsonNode.get("id").asText();
JsonNode tags = jsonNode.get("tags");
switch (jsonNode.get("type").asText()) {
case "node":
return new Point(jsonNode.get("lat").asDouble(), jsonNode.get("lon").asDouble(), id, tags);
case "way":
ArrayList<Point> points = new ArrayList<>();
JsonNode nodeList = jsonNode.get("geometry");
for (int j = 0; j < nodeList.size(); j++) {
JsonNode wayNode = nodeList.get(j);
points.add(j, new Point(wayNode.get("lat").asDouble(), wayNode.get("lon").asDouble()));
}
if (Polygon.isPolygon(points, tags)) {
return new Polygon(points, id, tags);
} else {
return new LineString(points, id, tags);
}
default:
Iterator<JsonNode> iterator = jsonNode.get("members").getElements();
List<List<Point>> rings = new ArrayList<>();
List<Point> ring = null;
while (iterator.hasNext()) {
JsonNode member = iterator.next();
JsonNode geometry = member.get("geometry");
ring = new ArrayList<>();
for (int ringIndex = 0; ringIndex < geometry.size(); ringIndex++) {
JsonNode coordinates = geometry.get(ringIndex);
ring.add(new Point(coordinates.get("lat").asDouble(), coordinates.get("lon").asDouble()));
}
rings.add(ring);
}
return new Multipolygon(Polygon.buildPolygons(rings), id, tags);
}
})
.collect(toList());
return new GeojsonSingleListResultSet(elementList);
}
}
Everything works well, but the toJsonNodeList() method is very slow in comparison to the method function.apply() that use a Stream. This is the toJsonNodeList()'s code:
private static List<JsonNode> toJsonNodeList(String s){
ObjectMapper mapper = new ObjectMapper();
List<JsonNode> list = new ArrayList<>();
try{
JsonNode resultSet = mapper.readTree(s).get("elements");
Iterator<JsonNode> iterator = resultSet.getElements();
while (iterator.hasNext()) {
list.add(iterator.next());
}
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
Is there a way to parse a json string using a parallelStream so as to extract each element and convert it to a JsonNode?
I have the following JSON string
[
{
"channel": "/bvmt/initValues",
"data": {
"value": {
"instrumentIds": "['TN0007250012','TN0007500010']",
"instruments": "[{'mnemonic':'ADWYA','marche':'ADWYA','phaut':5.82,'open':5.82,'nbrTrans':7,'veille':5.82,'time':'11:14:28','recapChange':0.00,'state':'','variation':'.','ref':5.82,'stateGrp':'S','percentChange':-0.34,'last':5.80,'bestTransaction':[{'value':5.82,'qte':'3','time':'10:00:00'},{'value':5.82,'qte':'5','time':'10:02:26'},{'value':5.82,'qte':'145','time':'10:23:27'},{'value':5.81,'qte':'100','time':'10:23:42'},{'value':5.80,'qte':'1000','time':'10:23:42'},{'value':5.73,'qte':'1','time':'10:31:21'},{'value':5.80,'qte':'100','time':'11:14:28'}],'volume':7857.19,'id':'TN0007250012','bestLimits':[{'quantiteAchat':2600,'timeVente':'11:44:10','ordreAchat':1,'prixAchat':5.76,'quantiteVente':100,'timeAchat':'11:44:10','ordreVente':1,'prixVente':5.90},{'quantiteAchat':50,'timeVente':'11:44:10','ordreAchat':1,'prixAchat':5.74,'quantiteVente':210,'timeAchat':'11:44:10','ordreVente':1,'prixVente':5.95},{'quantiteAchat':250,'timeVente':'11:44:10','ordreAchat':2,'prixAchat':5.75,'quantiteVente':187,'timeAchat':'11:44:10','ordreVente':1,'prixVente':5.94},{'quantiteAchat':189,'timeVente':'11:44:10','ordreAchat':3,'prixAchat':5.73,'quantiteVente':1112,'timeAchat':'11:44:10','ordreVente':1,'prixVente':5.97},{'quantiteAchat':44,'timeVente':'11:44:10','ordreAchat':1,'prixAchat':5.72,'quantiteVente':400,'timeAchat':'11:44:10','ordreVente':1,'prixVente':5.98}],'openStatus':'','cto':0,'valuer':'ADWYA','pbas':5.73,'grp':'S','abrv':'ADWYA','houv':'','qto':0,'seuilBas':5.65,'vto':0,'quantite':1354,'seuilHaut':5.99},{'mnemonic':'WIFAK','marche':'WIFAK','phaut':7.11,'open':7.00,'nbrTrans':2,'veille':7.13,'time':'10:24:15','recapChange':0.00,'state':'','variation':'.','ref':7.13,'stateGrp':'S','percentChange':-0.28,'last':7.11,'bestTransaction':[{'value':7.00,'qte':'99','time':'10:17:00'},{'value':7.11,'qte':'100','time':'10:24:15'}],'volume':1404.00,'id':'TN0007200017','bestLimits':[{'quantiteAchat':100,'timeVente':'11:00:19','ordreAchat':1,'prixAchat':6.80,'quantiteVente':100,'timeAchat':'11:00:19','ordreVente':1,'prixVente':7.09},{'quantiteAchat':0,'timeVente':'11:00:19','ordreAchat':0,'prixAchat':0.00,'quantiteVente':82,'timeAchat':'11:00:19','ordreVente':1,'prixVente':7.11},{'quantiteAchat':0,'timeVente':'11:00:19','ordreAchat':0,'prixAchat':0.00,'quantiteVente':284,'timeAchat':'11:00:19','ordreVente':2,'prixVente':7.10},{'quantiteAchat':0,'timeVente':'11:00:19','ordreAchat':0,'prixAchat':0.00,'quantiteVente':222,'timeAchat':'11:00:19','ordreVente':1,'prixVente':7.12},{'quantiteAchat':0,'timeVente':'11:00:19','ordreAchat':0,'prixAchat':0.00,'quantiteVente':110,'timeAchat':'11:00:19','ordreVente':2,'prixVente':7.13}],'openStatus':'','cto':0,'valuer':'WIFACK INT BANK','pbas':7.00,'grp':'S','abrv':'WIFAK','houv':'','qto':0,'seuilBas':6.92,'vto':0,'quantite':199,'seuilHaut':7.34}]"
},
"action": "initValues",
"infoSession": {
"lastInstrumentOrder": 11672,
"state": 1,
"lastInstrumentTime": "12:03:00",
"tradingTime": "08:04:02",
"tradingDate": "2017-04-24"
}
},
"id": "5"
},
{
"channel": "/bvmt/process",
"successful": true,
"id": "5"
}
]
I'm interested only in the content of the "instruments" field , I want to get only the "mnemonic" and "percentChange" fields and deserialize them into an array of Objects like this
public class Data
{
public List<MyObject> objects;
}
public class MyObject
{
String mnemonic;
Float percentChange;
}
How can I do this using Gson ?
Actually you have dozen ways of doing it. It only depends on how you manage your JSON documents. Let's declare a couple of DTOs first:
final class Data {
final List<MyObject> objects;
Data(final List<MyObject> objects) {
this.objects = objects;
}
}
final class MyObject {
final String mnemonic;
final Float percentChange;
MyObject(final String mnemonic, final Float percentChange) {
this.mnemonic = mnemonic;
this.percentChange = percentChange;
}
}
Here are some ways:
Pure JsonElement trees
The following example uses Java 8 Stream API and Gson JSON trees facilities, and it appears to be the simplest way to me:
private static final Gson gson = new Gson();
static Data testUsingJsonTreesOnly(final Reader reader) {
final List<MyObject> objects = StreamSupport.stream(gson.fromJson(reader, JsonElement.class).getAsJsonArray().spliterator(), false)
.map(JsonElement::getAsJsonObject)
.map(jsonObject -> jsonObject.get("data"))
.filter(Objects::nonNull)
.map(JsonElement::getAsJsonObject)
.map(jsonObject -> jsonObject.get("value"))
.filter(Objects::nonNull)
.map(JsonElement::getAsJsonObject)
.map(jsonObject -> jsonObject.get("instruments"))
.filter(Objects::nonNull)
.map(JsonElement::getAsJsonPrimitive)
.map(JsonPrimitive::getAsString)
.map(json -> gson.fromJson(json, JsonElement.class))
.map(JsonElement::getAsJsonArray)
.flatMap(jsonArray -> StreamSupport.stream(jsonArray.spliterator(), false))
.map(jsonElement -> gson.fromJson(jsonElement, MyObject.class))
.collect(toList());
return new Data(objects);
}
Two-pass mappings
This approach way attempts to extract the values in two passes:
deserialize the "outer" object;
get the inner object string and deserialize it in the second pass.
private static final Gson gson = new Gson();
private static final Type channelViewListType = new TypeToken<List<ChannelView>>() {
}.getType();
private static final Type myObjectListType = new TypeToken<List<MyObject>>() {
}.getType();
static Data testUsingDeserializationWithStrings(final Reader reader) {
final List<MyObject> objects = gson.<List<ChannelView>>fromJson(reader, channelViewListType)
.stream()
.filter(Objects::nonNull)
.map(view -> view.data)
.filter(Objects::nonNull)
.map(view -> view.value)
.filter(Objects::nonNull)
.map(view -> view.instruments)
.map((Function<String, List<MyObject>>) instruments -> gson.fromJson(instruments, myObjectListType))
.flatMap(Collection::stream)
.collect(toList());
return new Data(objects);
}
private static final class ChannelView {
final DataView data = null;
}
private static final class DataView {
final ValueView value = null;
}
private static final class ValueView {
final String instruments = null;
}
One-pass mappings using type adapters
This is, I would say, level #3: you can implement a specific type adapter to "unwrap" the encoded JSON document. #JsonAdapter can be used to specified the field that contains the specific "inner" JSON document:
private static final Gson gson = new Gson();
private static final Type channelViewListType = new TypeToken<List<ChannelView>>() {
}.getType();
static Data testUsingDeserializationWithJsonAdapter(final Reader reader) {
final List<MyObject> objects = gson.<List<ChannelView>>fromJson(reader, channelViewListType)
.stream()
.filter(Objects::nonNull)
.map(view -> view.data)
.filter(Objects::nonNull)
.map(view -> view.value)
.filter(Objects::nonNull)
.map(view -> view.instruments)
.flatMap(Collection::stream)
.collect(toList());
return new Data(objects);
}
private static final class ChannelView {
final DataView data = null;
}
private static final class DataView {
final ValueView value = null;
}
private static final class ValueView {
#JsonAdapter(UnpackedJsonTypeAdapterFactory.class)
final List<MyObject> instruments = null;
}
private static final class UnpackedJsonTypeAdapterFactory
implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
return new UnpackedJsonTypeAdapter<>(gson.getAdapter(typeToken));
}
private static final class UnpackedJsonTypeAdapter<T>
extends TypeAdapter<T> {
private final TypeAdapter<T> typeAdapter;
private UnpackedJsonTypeAdapter(final TypeAdapter<T> typeAdapter) {
this.typeAdapter = typeAdapter;
}
#Override
#SuppressWarnings("resource")
public void write(final JsonWriter out, final T value)
throws IOException {
out.value(typeAdapter.toJson(value));
}
#Override
public T read(final JsonReader in)
throws IOException {
final String json = in.nextString();
final JsonReader lenientIn = new JsonReader(new StringReader(json));
lenientIn.setLenient(true);
return typeAdapter.read(lenientIn);
}
}
}
Pure streaming
Probably the easiest by concept way, but not that easy to implement because of creating a high-level parser that deals with JSON token stream directly from the beginning to the end. Note that no even Gson instances are introduced.
static Data testUsingStreaming(final Reader reader)
throws IOException {
final List<MyObject> myObjects = new ArrayList<>();
final JsonReader jsonReader = new JsonReader(reader);
jsonReader.beginArray();
while ( jsonReader.hasNext() ) {
jsonReader.beginObject();
while ( jsonReader.hasNext() ) {
switch ( jsonReader.nextName() ) {
case "data":
jsonReader.beginObject();
while ( jsonReader.hasNext() ) {
switch ( jsonReader.nextName() ) {
case "value":
jsonReader.beginObject();
while ( jsonReader.hasNext() ) {
switch ( jsonReader.nextName() ) {
case "instruments":
final String instrumentsJson = jsonReader.nextString();
parseInstruments(instrumentsJson, myObjects);
break;
default:
jsonReader.skipValue();
break;
}
}
jsonReader.endObject();
break;
default:
jsonReader.skipValue();
break;
}
}
jsonReader.endObject();
break;
default:
jsonReader.skipValue();
break;
}
}
jsonReader.endObject();
}
jsonReader.endArray();
return new Data(myObjects);
}
private static void parseInstruments(final String instrumentsJson, final Collection<MyObject> myObjects)
throws IOException {
final JsonReader jsonReader = new JsonReader(new StringReader(instrumentsJson));
jsonReader.setLenient(true);
jsonReader.beginArray();
while ( jsonReader.hasNext() ) {
String mnemonic = null;
Float percentChange = null;
jsonReader.beginObject();
while ( jsonReader.hasNext() ) {
final String name = jsonReader.nextName();
switch ( name ) {
case "mnemonic":
mnemonic = jsonReader.nextString();
break;
case "percentChange":
percentChange = (float) jsonReader.nextDouble();
break;
default:
jsonReader.skipValue();
break;
}
}
if ( mnemonic != null && percentChange != null ) {
myObjects.add(new MyObject(mnemonic, percentChange));
}
jsonReader.endObject();
}
jsonReader.endArray();
}
All of the approaches above produce the same output:
ADWYA: -0.34
WIFAK: -0.28
I have a custom map implementation called ObjectMap as follows:
public class ObjectMap extends LinkedHashMap<String, Object> implements Serializable {
...
}
I can convert any POJO to this ObjectMap using jackson ObjectMapper as follows:
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setVisibility(
objectMapper.getSerializationConfig().
getDefaultVisibilityChecker().
withFieldVisibility(JsonAutoDetect.Visibility.ANY).
withGetterVisibility(JsonAutoDetect.Visibility.NONE).
withIsGetterVisibility(JsonAutoDetect.Visibility.NONE)
);
ObjectMap objectMap = objectMapper.convertValue(object, new TypeReference<ObjectMap>() {});
But the problem is if my POJO's complex fields are being mapped to a LinkedHashMap not ObjectMap. So how do I enforce ObjectMapper to map internal fields also into a ObjectMap instead of LinkedHashMap?
I have come up with below solution at last:
public class Document extends LinkedHashMap<String, Object> implements Serializable {
}
public class JacksonMapper {
private ObjectMapper objectMapper;
public <T> Document asDocument(T object) {
ObjectMapper objectMapper = getObjectMapper();
JsonNode node = objectMapper.convertValue(object, JsonNode.class);
return loadDocument(node);
}
protected ObjectMapper getObjectMapper() {
if (objectMapper == null) {
objectMapper = new ObjectMapper();
objectMapper.setVisibility(
objectMapper.getSerializationConfig().
getDefaultVisibilityChecker().
withFieldVisibility(JsonAutoDetect.Visibility.ANY).
withGetterVisibility(JsonAutoDetect.Visibility.NONE).
withIsGetterVisibility(JsonAutoDetect.Visibility.NONE)
);
objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
objectMapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
}
return objectMapper;
}
private Document loadDocument(JsonNode node) {
Document document = new Document();
Iterator<Map.Entry<String, JsonNode>> fields = node.fields();
while (fields.hasNext()) {
Map.Entry<String, JsonNode> entry = fields.next();
String name = entry.getKey();
JsonNode value = entry.getValue();
document.put(name, loadObject(value));
}
return document;
}
private Object loadObject(JsonNode node) {
if (node == null) return null;
try {
switch (node.getNodeType()) {
case ARRAY:
return loadArray(node);
case BINARY:
return node.binaryValue();
case BOOLEAN:
return node.booleanValue();
case MISSING:
case NULL:
return null;
case NUMBER:
return node.numberValue();
case OBJECT:
return loadDocument(node);
case POJO:
return loadDocument(node);
case STRING:
return node.textValue();
}
} catch (IOException e) {
return null;
}
return null;
}
private List loadArray(JsonNode array) {
if (array.isArray()) {
List list = new ArrayList();
Iterator iterator = array.elements();
while (iterator.hasNext()) {
Object element = iterator.next();
if (element instanceof JsonNode) {
list.add(loadObject((JsonNode) element));
} else {
list.add(element);
}
}
return list;
}
return null;
}
}
Hope this will help someone someday.
Say I'm building my GSON object like this
new GsonBuilder().setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE).create();
and now I want to deserialize the following JSON
{
"MyMap" : {
"Key1" : "Foo",
"Key2" : "Bar"
}
}
into the following class (which is working just fine)
public class MapClass {
Map<String,String> myMap;
}
but I also would like the keys to be named "key1" and "key2". How would I go about this?
You can try in this way:
try {
JSONObject jObj = new JSONObject("{"
+ " \"MyMap\" : {"
+ " \"Key1\" : \"Foo\","
+ " \"Key2\" : \"Bar\""
+ " }"
+ "}"); // this parses the json
JSONObject jObjt = jObj.getJSONObject("MyMap");
//old version with out GSON
Map<String, String> map = new HashMap();
Iterator itr = jObjt.keys();
while (itr.hasNext()) {
String key = (String) itr.next();
String value = jObjt.getString(key);
map.put(key, value);
}
//desalinized one with GSON
Map<String, String> map1 = new Gson().fromJson(jObjt.toString(), HashMap.class);
for (String str : map1.keySet()) {
System.out.println("k:" + str + " v:" + map1.get(str));
}
} catch (JSONException ex) {
//log the error
}
FieldNamingPolicy is applied to fields of json value. It is not possible to apply this to maps' keys (a map which has key, value pairs) at json.
Easy solution:
After deserialization, iterate over your map and rename key names. i.e. Key1 to key1 and Key2 to key2.
Other solution:
Write a custom TypeAdapter which handles deserialization process and renames keys.
public class MapClassTypeAdapter extends TypeAdapter<MapClass> {
#Override
public MapClass read(JsonReader in) throws IOException {
final MapClass mapClassInstance = new MapClass();
mapClassInstance.myMap = new HashMap<String, String>();
in.beginObject();
if("myMap".equalsIgnoreCase(in.nextName())) {
in.beginObject();
while (in.hasNext()) {
String key = in.nextName();
// You want keys as camel case
String newKey = key.substring(0,1).toLowerCase() + key.substring(1);
String value = in.nextString();
mapClassInstance.myMap.put(newKey, value);
}
in.endObject();
}
in.endObject();
return mapClassInstance;
}
#Override
public void write(JsonWriter out, MapClass mapClass) throws IOException {
throw new RuntimeException("MapClassTypeAdapter.write method not implemented yet!");
}
}
Test Other solution:
String json = "{\"myMap\":{\"Key1\":\"Foo\",\"Key2\":\"Bar\"}}";
final GsonBuilder gsonBuilder = new GsonBuilder();
gsonBuilder.registerTypeAdapter(MapClass.class, new MapClassTypeAdapter());
final Gson gson = gsonBuilder.create();
MapClass mapClass = gson.fromJson(json, MapClass.class);