We Keep Coding

Remove some keys when converting from custom type into Map using Jackson's convertValue

I have a rather simple Java object that I would like to convert it into a Map<String, Object> using Jackson's convertValue(fromValue, toValueTypeRef) — this is rather trivial.
At the same time, there are some fields/members (e.g.: createdOn and modifiedOn) I would like to remove from the resultant Map, but I can't remove them in all (de)serialization routines. I was wondering if there is a way to tell Jackson to do that, rather than removing them "by hand". I've usually seen #JsonView used for such purposes, but I think it doesn't work with convertValue — or I'm still unable to configure it correctly.
This is how I'm doing the conversion:
#NoArgsConstructor
public final class CustomTypeMapper {
private static final TypeReference<Map<String, Object>> MAP_TYPE_REFERENCE = new TypeReference<>() { };
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); // ...with some customizations
public Map<String, Object> convertFrom(final CustomType input) {
// OBJECT_MAPPER.writerWithView(Views.Public.class);
return OBJECT_MAPPER.convertValue(input, MAP_TYPE_REFERENCE);
}
}
CustomType looks roughly like this:
#Getter
#Builder
#Jacksonized
public final class CustomType {
#JsonProperty("id")
private String id;
#JsonProperty("user")
private User user;
...
#Builder.Default
#JsonProperty("created_on") // TODO: Exclude this field/member some times
#JsonView(Views.Internal.class)
private final Instant createdOn = Instant.now();
#Builder.Default
#JsonView(Views.Internal.class)
#JsonProperty("modified_on") // TODO: Exclude this field/member some times
private Instant modifiedOn = Instant.now();
}

Converting the unstructured object in java

I'm using MongoDb for unstructured documents. When I do the aggregations, I'm getting final output as unstructured objects. I post some sample data for the easiness. Actual objects have many fields.
Eg :
[
{ _id : "1", type: "VIDEO", videoUrl : "youtube.com/java"},
{ _id : "2", type: "DOCUMENT", documentUrl : "someurl.com/spring-boot-pdf"},
{ _id : "3", type: "ASSESSMENT", marks : 78}
]
The respective class for the types of above objects are
#Data
public class Video{
private String _id;
private String type;
private String videoUrl;
}
#Data
public class Document{
private String _id;
private String type;
private String documentUrl;
}
#Data
public class Assessment{
private String _id;
private String type;
private Integer marks;
}
Since I can't specify the converter class, I get all objects as list of Object.class which is a general type for all.
List<Object> list = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
It's working, but this is not readable and not maintainable for backend and front-end developers (eg : swagger ui). So I came up with a solution, that put all fields as a class.
#Data
#JsonInclude(JsonInclude.Include.NON_NULL)
class MyConvetor{
private String _id;
private String type;
private String videoUrl;
private String documentUrl;
private Integer marks;
}
Here Jackson helps to ignore all null fields
Now I can use MyConverter as Type
List<MyConverter> list = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), MyConverter.class).getMappedResults();
But I feel this is not a good practice when we implementing a standard application. I'd like to know, is there any way to avoid the general type class (e.g. extending any abstract class)? Or is this the only way I can do?

I don't think so (or I don't know) if MongoDB in Java provides this kind of dynamic conversion by some field (it would require specify what field and what classes). But you can do it by hand.
First, you need to define your types (enum values or some map) for matching string to class. You can create abstract parent class (eg. TypedObject) for easier usage and binding all target classes (Video, Document, Assessment) .
Next you have to read and map values from Mongo to anything because you want to read all data in code. Object is good but I recommend Map<String, Object> (your Object actually is that Map - you can check it by invoking list.get(0).toString()). You can also map to String or DBObject or some JSON object - you have to read "type" field by hand and get all data from object.
At the end you can convert "bag of data" (Map<String, Object> in my example) to target class.
Now you can use converted objects by target classes. For proving these are actually target classes I print objects with toString all fields.
Example implementation
Classes:
#Data
public abstract class TypedObject {
private String _id;
private String type;
}
#Data
#ToString(callSuper = true)
public class Video extends TypedObject {
private String videoUrl;
}
#Data
#ToString(callSuper = true)
public class Document extends TypedObject {
private String documentUrl;
}
#Data
#ToString(callSuper = true)
public class Assessment extends TypedObject {
private Integer marks;
}
Enum for mapping string types to classes:
#RequiredArgsConstructor
public enum Type {
VIDEO("VIDEO", Video.class),
DOCUMENT("DOCUMENT", Document.class),
ASSESSMENT("ASSESSMENT", Assessment.class);
private final String typeName;
private final Class<? extends TypedObject> clazz;
public static Class<? extends TypedObject> getClazz(String typeName) {
return Arrays.stream(values())
.filter(type -> type.typeName.equals(typeName))
.findFirst()
.map(type -> type.clazz)
.orElseThrow(IllegalArgumentException::new);
}
}
Method for converting "bag of data" from JSON to your target class:
private static TypedObject toClazz(Map<String, Object> objectMap, ObjectMapper objectMapper) {
Class<? extends TypedObject> type = Type.getClazz(objectMap.get("type").toString());
return objectMapper.convertValue(objectMap, type);
}
Read JSON to "bags of data" and use of the above:
String json = "[\n" +
" { _id : \"1\", type: \"VIDEO\", videoUrl : \"youtube.com/java\"},\n" +
" { _id : \"2\", type: \"DOCUMENT\", documentUrl : \"someurl.com/spring-boot-pdf\"},\n" +
" { _id : \"3\", type: \"ASSESSMENT\", marks : 78}\n" +
"]";
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
List<Map<String, Object>> readObjects = objectMapper.readValue(json, new TypeReference<>() {});
for (Map<String, Object> readObject : readObjects) {
TypedObject convertedObject = toClazz(readObject, objectMapper);
System.out.println(convertedObject);
}
Remarks:
In example I use Jackson ObjectMapper for reading JSON. This makes the example and testing simpler. I think you can replace it with mongoTemplate.aggregate(). But anyway I need ObjectMapper in toClazz method for converting "bags of data".
I use Map<String, Object> instead of just Object. It is more complicated: List<Map<String, Object>> readObjects = objectMapper.readValue(json, new TypeReference<>() {});. If you want, you can do something like this: List<Object> readObjects2 = (List<Object>) objectMapper.readValue(json, new TypeReference<List<Object>>() {});
Result:
Video(super=TypedObject(_id=1, type=VIDEO), videoUrl=youtube.com/java)
Document(super=TypedObject(_id=2, type=DOCUMENT), documentUrl=someurl.com/spring-boot-pdf)
Assessment(super=TypedObject(_id=3, type=ASSESSMENT), marks=78)
Of course you can cast TypedObject to target class you need (I recommend checking instance of before casting) and use:
Video video = (Video) toClazz(readObjects.get(0), objectMapper);
System.out.println(video.getVideoUrl());
I assumed you read whole collection once and you get all types mixed up in one list (as in example in your question). But you can try find documents in MongoDB by field "type" and get data separately for each of type. With this you can easily convert to each type separately.

Jackson Collection<Object> attributes to top level JSON output string

I'm using Jackson in order to have log outputs as JSON.
The things is that we allow logging with the following syntax:
log.info("yourMessage {}", innerMessageObject, Meta.of("key", ObjectValue))
OUTPUT I HAVE
{
"level": INFO,
... classic logging attributes
"metadata": {
"object1": "value 1",
"object2": { ... }
...
}
}
OUTPUT I WANT
{
"level": INFO,
... classic logging attributes
"object1": "value 1",
"object2": { ... }
...
}
My log POJO
#Data
class JsonLog {
#JsonIgnore
private static final ObjectMapper mapper = JsonLog.initMapper();
private final String message;
private final String class_name;
private final Collection<Object> metadata;
private final Marker marker;
private final String level;
private final Long timestamp;
private final String thread;
private final String logger;
private final LoggerContextVO logger_context;
private final Map<String, String> environment_vars;
}
I don't succeed to only have the metadata attribute to be serialized as top level attributes.
It seems I cannot use #JsonUnwrapped because of this issue, I also tried this solution but cannot see how to implement it.
Have you got any ideas ?
Thank you :)

Would it be OK to convert metadata to be Map<String, Object>? It seems so by you example JSON and that's actually the natural generic representation of JSON in Java. In that case:
private final Map<String, Object> metadata;
#JsonAnyGetter
Map<String, Object> getMetadata() {
return metadata;
}
According to the docs, this annotation marks the getter method:
to be serialized as part of containing POJO (similar to unwrapping)
and can only be used with methods returning a Map. I am not aware of a solution for Collection

RestClientException Could not extract response: no suitable HttpMessageConverter found for response type

I have following PoJo:
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
public class KostenpflichtigeBuchung {
private String buchungZahlungsId;
private String warenkorbId;
private String pseudocardpan;
private Zahlungsmittel zahlungsmittel;
private String landKreditkartenInhaber;
private String nameKreditkartenInhaber;
private String touchpointId;
private String vertriebspartnerId;
private Kundendaten kundendaten;
private Fulfillmentart fulfillmentart;
private final List<Reisender> reisenderList = new ArrayList<>();
#JsonIgnore
private Map<String, Object> payload;
#JsonAnyGetter
public Map<String, Object> getPayload() {
return payload;
}
#JsonAnySetter
public void setPayload(String name, Object value) {
if (payload == null) {
payload = new HashMap<>();
}
payload.put(name, value);
}
}
When I execute a cucumber Test on it, I get the following Exception:
org.springframework.web.client.RestClientException: Could not extract response: no suitable HttpMessageConverter found for response type [class de.db.vendo.bue.buchung.model.KostenpflichtigeBuchung] and content type [application/json;charset=utf-8]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:109)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:917)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:901)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:655)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
I have some other test and other POJO with the same annotation which are working without any problems. I have really not a single idea what is going wrong.
Thank for any suggestion!

Most probably issue between lombok constructor annotations and related generated constructors, often when I face this error, I just revert response class to #NoArgConstructor only and try again.
If you really need all these constructors and builders, try to code them manually and go with #JsonCreator like describe in this thread.

Parsing JSON with Retrofit and GSON, error when trying to parse and obtain callback.

I am using to Retrofit to handle Calls to my API for an Android Application. I am trying to get Retrofit to handle the parsing of the JSON, and creating a list of Objects in accordance with the POJO i have created.
The error i receive is "com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 176".
I used JsonSchema2Pojo to generate my java classes. The classes and associated JSON are as follows.
{"status":"success","data":[{"sort_key":1,"event_id":1947357,"title":"2014 US Open Tennis Session 15 (Mens\/Womens Round of 16)","datetime_utc":"2014-09-01T15:00:00","venue":{"city":"Flushing","name":"Louis Armstrong Stadium","extended_address":"Flushing, NY 11368","url":"https:\/\/seatgeek.com\/venues\/louis-armstrong-stadium\/tickets\/?aid=10918","country":"US","display_location":"Flushing, NY","links":[],"slug":"louis-armstrong-stadium","state":"NY","score":0.73523,"postal_code":"11368","location":{"lat":40.7636,"lon":-73.83},"address":"1 Flushing Meadows Corona Park Road","timezone":"America\/New_York","id":2979},"images":["https:\/\/chairnerd.global.ssl.fastly.net\/images\/performers-landscape\/us-open-tennis-45e2d9\/5702\/huge.jpg","https:\/\/chairnerd.global.ssl.fastly.net\/images\/performers\/5702\/us-open-tennis-c1ccf7\/medium.jpg","https:\/\/chairnerd.global.ssl.fastly.net\/images\/performers\/5702\/us-open-tennis-01f513\/large.jpg","https:\/\/chairnerd.global.ssl.fastly.net\/images\/performers\/5702\/us-open-tennis-4e07f2\/small.jpg"]}
From this i believe i need to generate 3 POJO's, my higher level "EventObject" Class, A Location Class, and a Venue Class. These classes and their variables follow:
EventObject Class:
public class EventObject {
private Integer sortKey;
private Integer eventId;
private String title;
private String datetimeUtc;
private Venue venue;
private List<String> images = new ArrayList<String>();
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
Location Class:
public class Location {
private Float lat;
private Float lon;
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
Venue Class:
public class Venue {
private String city;
private String name;
private String extendedAddress;
private String url;
private String country;
private String displayLocation;
private List<Object> links = new ArrayList<Object>();
private String slug;
private String state;
private Float score;
private String postalCode;
private Location location;
private String address;
private String timezone;
private Integer id;
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
My interface for the Api Call is as follows:
public interface UserEvents {
#GET("/user/get_events")
void getEvents(#Header("Authorization")String token_id,
#Query("event_type")String event_type,
#Query("postal_code")int postalCode,
#Query("per_page") int perPage ,
#Query("lat") int lat,
#Query("lon") int lon,
#Query("month")int month,
#Query("page")int page,
Callback<List<EventObject>>callback) ;
}
Here is its implementation in my code :
UserEvents mUserEvents = mRestAdapter.create(UserEvents.class);
mUserEvents.getEvents(token_Id, "sports",11209,25,0, 0, 9, 2, new Callback <List<EventObject>>() {
#Override
public void success(List<EventObject> eventObjects, retrofit.client.Response response) {
Log.d(TAG,"Success");
}
There is alot going on here, but i believe that i am probably going wrong with how i am handling the JSON. When i copied and pasted in my JSON to the Pojo generator, i did not include "status:success, " data:{
I essentially just used the entire entry of an element in the Array ( everything from {sort_key onward until the next sort key ) and pushed that through the converter.
This is my first try at Retrofit and API work, and parsing anything this complicated.
I am hoping its something that someone else will be able to point out. I have googled as well i could to sort this out with no luck.
Thanks for looking.

The main problem is that you are not getting the root element of the response. You need to create an entity "response" that gets the items status and data. It would look something like this:
public class RootObject {
#Expose
private String status;
#Expose
private EventObject data;
//getters and setters here
}
Then when you make the callback you should point to your RootObject, mUserEvents.getEvents(token_Id, "sports",11209,25,0, 0, 9, 2, new Callback <RootObject>()
One more thing, Retrofit uses GSON to parse your json reponse. It means that when you create the entities, the variables need to match the name of the objects coming in the response. If it doesn't you need to tell GSON how to map the variables, like this:
#SerializedName("extended_address")
#Expose
private String extendedAddress;
In that case the value coming in the json is "extended_address" and will be mapped to the String extendedAddress. If you don't put that #SerializedName line the parsing will fail. If you want to skip that line then you can call your variable "extended_address" so it matches the response.
The #Expose is needed by GSON to parse the variable below it. If a variable doesn't have it then GSON will ignore that parsing. So you need to fix both the #Expose and #SerializedName on your entities so GSON works correctly.
Hope it helps.