I'm using Jackson in Spring MVC application. I want to use a String value as key name for Java POJO --> JSON
"record": {
"<Dynamic record name String>": {
"value": {
....
}
}
}
So the dynamic record name String could be "abcd","xyz" or any other string value. How can I define my "record" POJO to have a key like that ?
Unfortunately, you cannot have dynamic fields in Java classes (unlike some other languages), so you have two choices:
Using Maps
Using JSON objects (i.e. JsonNode in case of Jackson)
Suppose, you have a data like this:
{
"record": {
"jon-skeet": {
"name": "Jon Skeet",
"rep": 982706
},
"darin-dimitrov": {
"name": "Darin Dimitrov",
"rep": 762173
},
"novice-user": {
"name": "Novice User",
"rep": 766
}
}
}
Create two classes to capture it, one for user and another for the object itself:
User.java:
public class User {
private String name;
private Long rep;
public String getName() { return name; }
public void setName(String name) { this.name = name; }
public Long getRep() { return rep; }
public void setRep(Long rep) { this.rep = rep; }
#Override
public String toString() {
return "User{" +
"name='" + name + '\'' +
", rep=" + rep +
'}';
}
}
Data.java:
public class Data {
private Map<String, User> record;
public Map<String, User> getRecord() { return record; }
public void setRecord(Map<String, User> record) { this.record = record; }
#Override
public String toString() {
return "Data{" +
"record=" + record +
'}';
}
}
Now, parse the JSON (I assume there is a data.json file in the root of your classpath):
public class App {
public static void main(String[] args) throws Exception {
final ObjectMapper objectMapper = new ObjectMapper();
System.out.println(objectMapper.readValue(App.class.getResourceAsStream("/data.json"), Data.class));
System.out.println(objectMapper.readTree(App.class.getResourceAsStream("/data.json")));
}
}
This will output:
Data{record={jon-skeet=User{name='Jon Skeet', rep=982706}, darin-dimitrov=User{name='Darin Dimitrov', rep=762173}, novice-user=User{name='Novice User', rep=766}}}
{"record":{"jon-skeet":{"name":"Jon Skeet","rep":982706},"darin-dimitrov":{"name":"Darin Dimitrov","rep":762173},"novice-user":{"name":"Novice User","rep":766}}}
In case of a Map you can use some static classes, like User in this case, or go completely dynamic by using Maps of Maps (Map<String, Map<String, ...>>. However, if you find yourself using too much maps, consider switching to JsonNodes. Basically, they are the same as Map and "invented" specifically for highly dynamic data. Though, you'll have some hard time working with them later...
Take a look at a complete example, I've prepared for you here.
This is in Kotlin but I have found a solution to the same problem using Jackson.
You don't need the root node "record", so you will need to get rid of it or start one node deeper(you're on your own there) but to turn the list of records that are children of their id into a list of records with id in the object follows:
val node = ObjectMapper().reader().readTree(json)
val recordList = mutableListOf<Record>()
node.fields().iterator().forEach {
val record = record(
it.key,
it.value.get("name").asText(),
it.value.get("rep").asText()
)
recordList.add(event)
}
node.fields() returns a map of children(also maps)
iterating through the parent map you will get the id from the key and then the nested data is in the value (which is another map)
each child of fields is key : value where
key = record id
value = nested data (map)
This solution, you don't need multiple classes to deserialize a list of classes.
I have my data in this format:
{
"0" : {"a": {}}, {"b": {}}, ...
"1" : {"c": {}}, {"d": {}}, ...
.
.
.
}
I am able to capture it into a map using the dynamic capture feature of jackson by using #JsonAnySetter annotation.
public class Destination{
Map<String, Object> destination = new LinkedHashMap<>();
#JsonAnySetter
void setDestination(String key, Object value) {
destination.put(key, value);
}
}
Related
I have the following JSON
{
"ads": [
{
"228029_228029": {
"ad_id": "228029",
"duration": 10,
"m3u8_text": {
"_1280p": "#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-ALLOW-CACHE:YES\n#EXT-X-TARGETDURATION:7\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:7.120000,\n_1280p_0000.ts\n#EXTINF:2.880000,\n_1280p_0001.ts\n#EXT-X-ENDLIST\n",
"_320p": "#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-ALLOW-CACHE:YES\n#EXT-X-TARGETDURATION:7\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:7.120000,\n_320p_0000.ts\n#EXTINF:2.880000,\n_320p_0001.ts\n#EXT-X-ENDLIST\n"
}
}
},
{
"228845_228845": {
"ad_id": "228845",
"duration": 24,
"m3u8_text": {
"_1280p": "#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-ALLOW-CACHE:YES\n#EXT-X-TARGETDURATION:8\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:7.840000,\n_1280p_0000.ts\n#EXTINF:6.880000,\n_1280p_0001.ts\n#EXTINF:6.680000,\n_1280p_0002.ts\n#EXTINF:2.600000,\n_1280p_0003.ts\n#EXT-X-ENDLIST\n",
"_320p": "#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-ALLOW-CACHE:YES\n#EXT-X-TARGETDURATION:8\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:7.840000,\n_320p_0000.ts\n#EXTINF:6.880000,\n_320p_0001.ts\n#EXTINF:6.680000,\n_320p_0002.ts\n#EXTINF:2.600000,\n_320p_0003.ts\n#EXT-X-ENDLIST\n"
}
}
}
],
"total_duration": 80
}
I have created the respective model class as
This is the root model
#JsonIgnoreProperties(ignoreUnknown = true)
public class AdsResponse {
#JsonProperty("ads")
List<Ad> ads;
#JsonProperty("total_duration")
long totalDuration;
}
Then the ads model
public class Ad {
Map<String,AdInfo> ad;}
Then the AdInfo model
public class AdInfo {
#JsonProperty("m3u8_text")
AdManifest adManifest;
int duration;
#JsonProperty("ad_id")
String adId;}
Then the manifest model
public class AdManifest {
#JsonProperty("_1280p")
String _1280p;
#JsonProperty("_320p")
String _320p;}
When I try to parse this using below code
AdsResponse response = new ObjectMapper().readValue(
res,
AdsResponse.class);
I get the empy ad object
AdsResponse{ads=[Ad{ad=null}, Ad{ad=null}, totalDuration=80}
What is wrong here?
You don't actually need the Ad class, it's just a Map<>. AdResponse can look like this:
public class AdsResponse {
#JsonProperty("ads")
List<Map<String, AdInfo>> ads;
#JsonProperty("total_duration")
long totalDuration;
If the keys in the map were predictable, you could make them properties on the Ad class and then Jackson would map them properly. But since they're not (they look like some kind of ID), mapping them to a Map<> is probably the best option.
As an alternative, if you want or need to have the Ad objects, you can map them like this:
public class Ad {
Map<String, AdInfo> adInfo = new HashMap<>();
#JsonAnySetter
public void setAds(String key, AdInfo value) {
adInfo.put(key, value);
}
}
With that, and AdResponse defined the way you have it in the question, you'll get populated Ad instances, each of which has a Map<> with only 1 key/value pair. For an even simpler (and probably more sensible model, you can eliminate the Map if there is only ever 1 key in an Ad, like this:
public class Ad {
private AdInfo adInfo;
#JsonAnySetter
public void setAdInfo(String ignored, AdInfo value) {
this.adInfo = value;
}
}
So far, I've been able to create a KStream with the help of a topic.
KStream<String, Object> testqa2 = builder.stream("testqa2", Consumed.with(Serdes.String(), Serdes.String()))
.mapValues(value -> {
System.out.println(value);
return value;
});
It doesn't print anything, so on debbuging - I realized I am just creating my KStream. There is no data in it.
I am having a litte trouble creating serializer/deserializer for worker class.
package com.copart.mwa.Avro;
public class Worker {
private static String WorkerActivityName;
private static String WorkerSid;
private static String WorkerPreviousActivityName;
private static String WorkerPreviousActivitySid;
public String getWorkerActivityName() {
return WorkerActivityName;
}
public void setWorkerActivityName(String workerActivityName) {
WorkerActivityName = workerActivityName;
}
public static String getWorkerSid() {
return WorkerSid;
}
public void setWorkerSid(String workerSid) {
WorkerSid = workerSid;
}
public String getWorkerPreviousActivityName() {
return WorkerPreviousActivityName;
}
public void setWorkerPreviousActivityName(String workerPreviousActivityName) {
WorkerPreviousActivityName = workerPreviousActivityName;
}
public String getWorkerPreviousActivitySid() {
return WorkerPreviousActivitySid;
}
public void setWorkerPreviousActivitySid(String workerPreviousActivitySid) {
WorkerPreviousActivitySid = workerPreviousActivitySid;
}
#Override
public String toString() {
return "Worker(" + WorkerSid + ", " + WorkerActivityName + ")";
} }
And the message from the producer to the consumer is a JSON
{
"WorkerActivityName": "Available",
"EventType": "worker.activity.update",
"ResourceType": "worker",
"WorkerTimeInPreviousActivityMs": "237",
"Timestamp": "1626114642",
"WorkerActivitySid": "WAc9030ef021bc1786d3ae11544f4d9883",
"WorkerPreviousActivitySid": "WAf4feb231e97c1878fecc58b26fdb95f3",
"WorkerTimeInPreviousActivity": "0",
"AccountSid": "AC8c5cd8c9ba538090da104b26d68a12ec",
"WorkerName": "Dorothy.Finegan#Copart.Com",
"Sid": "EV284c8a8bc27480e40865263f0b42e5cf",
"TimestampMs": "1626114642204",
"P": "WKe638256376188fab2a98cccb3c803d67",
"WorkspaceSid": "WS38b10d521442ecb74fcc263d5a4d726e",
"WorkspaceName": "Copart-MiPhone",
"WorkerPreviousActivityName": "Unavailable(RNA)",
"EventDescription": "Worker Dorothy.Finegan#Copart.Com updated to Available Activity",
"ResourceSid": "WKe638256376188fab2a98cccb3c803d67",
"WorkerAttributes": "{\"miphone_dept\":[\"USA_YRD_OPS\"],\"languages\":[\"en\"],\"home_region\":\"GL\",\"roles\":[\"supervisor\"],\"miphone_yards\":[\"81\"],\"miphone_enabled\":true,\"miphone_states\":[\"IL\"],\"home_state\":\"IL\",\"skills\":[\"YD_SELLER\",\"YD_TITLE\"],\"home_division\":\"Northern\",\"miphone_divisions\":[\"Northern\"],\"miphone_functions\":[\"outbound_only\"],\"full_name\":\"Dorothy Finegan\",\"miphone_regions\":[\"GL\"],\"home_country\":\"USA\",\"copart_user_id\":\"USA3204\",\"home_yard\":\"81\",\"home_dept\":\"USA_YRD_OPS\",\"email\":\"dorothy.finegan#copart.com\",\"home_dept_category\":\"OPS\",\"contact_uri\":\"client:Dorothy_2EFinegan_40Copart_2ECom\",\"queue_activity\":\"Available\",\"teams\":[],\"remote_employee\":false,\"miphone_call_center_units\":[\"USA_YRD_OPS|81\"],\"miphone_call_center_teams\":[]}"
}
I want to implemenet a customer deserializer where
"WorkspaceSid": "WS38b10d521442ecb74fcc263d5a4d726e", is the key and the remaining values of the other attributes act as the value for the key-value pair.
Thanks,
Anmol
It doesn't print anything
If there is data in testqa2 topic, and you have auto.offset.reset=earliest, then it should.
having a litte trouble creating serializer/deserializer for worker class
Kafka has bulit-in JSON serializers that you can build a Serde for. You don't need to make your own.
"WorkspaceSid", is the key
use selectKey, or map if you want to modify the key, not mapValues
Serializer<JsonNode> jsonNodeSerializer = new JsonSerializer();
Deserializer<JsonNode> jsonNodeDeserializer = new JsonDeserializer();
final Serde<JsonNode> jsonNodeSerde = Serdes.serdeFrom(jsonNodeSerializer,jsonNodeDeserializer);
KStream<String, JsonNode> testqa2 = builder.stream("testqa2", Consumed.with(Serdes.String(), jsonSerde))
.selectKey((k, json) -> json.get("WorkspaceSid"))
.print(Printed.toSysOut());
Alternatively, fix your producer code to get the Sid from the value, and set the key there...
If you want to use Avro, you wouldn't write a Worker class - you would generate it from an Avro schema.
I am trying to parse a json string to java object but i am not sure on the object hierarchy.
below is the json string
{
"TABLE_Detail":{
"1":{
"TABLE":"table1",
"RUN_DATE":"20170313",
"SEQ_NUM":"123",
"START_TIME":"20170313133144",
"END_TIME":"20170313133655"
},
"2":{
"TABLE":"table2",
"RUN_DATE":"20170313",
"SEQ_NUM":"123",
"START_TIME":"20170313133142",
"END_TIME":"20170313133723"
}
}
}
Here the number 1, 2 are dynamic and can go up to any number, I tried to create a outer object and have a Map of type key String and value as object TableData. The map having variable name TABLE_Detail. but the TableData object is always null. TableData object has all the variables.
Please help me on how to convert this json string to object.
Change 1 to table1 and 2 to table2:
public class TableDetails {
private TableData table1;
private TableData table2;
public TableDetails(){
}
// getter and setter
}
And if modify json format to "Koen Van Looveren" mentioned:
public class TableDetails {
List<TableData> tables;
public TableDetails() {
}
// getter and setter
}
The table class:
Table.java:
public class TableData {
private String table;
private String run_date;
private String seq_num;
private String start_time;
private String end_time;
public TableData() {
}
// getter and setter
}
you have two choice for such painfully json structure when using Gson.
using Gson parsing json as Map and write some class access returned Map.this mode works fine for access data only!
//usage
TableDetails details=new TableDetails(gson.fromJson(json, Map.class));
//classes
class TableDetails {
private Map<String, Map> context;
public TableDetails(Map root) {
this.context = (Map<String, Map>) root.get("TABLE_Detail");
}
public int size() {
return context.size();
}
public Table get(String key) {
return new Table(context.get(key));
}
}
class Table {
private Map context;
public Table(Map context) {
this.context = context;
}
public String getName() {
return get("TABLE");
}
private <T> T get(String name) {
return (T) context.get(name);
}
...
}
write your own Gson TypeAdapter,but this way may be more complex.if you interesting on write custom TypeAdapter there is a demo that I written for extract json root.gson-enclosing-plugin
You can try deserialize it into a Map<String, Map<String, TableData>>. The reason why Map<String, TableData> doesn't work it that the pesudo-array is wrapped in another object.
The following example converts a response into a List<TableData>:
public List<TableData> deserialize(String json) {
return Gson().<Map<String, Map<String, TableData>>>fromJson(json, new TypeToken<Map<String, Map<String, TableData>>>(){}.getType())
.values().iterator().next()
.entrySet().stream()
.sorted(Comparator.comparingInt(e -> Integer.parseInt(e.getKey())))
.map(Map.Entry::getValue)
.collect(Collectors.toList());
}
I was in a search for the solution, and i came across one of the site where the solution worked. i wanted to credit the below site. Thanks for all the support.
I am able to map the dynamic value 1, 2 as map keys and values are mapped correspondingly to the TableData object properties using #SerializedName gson annootation.
http://findnerd.com/list/view/Parse-Json-Object-with-dynamic-keys-using-Gson-/24094/
When using an array in json you need to use [ for opening and ] for closing
{
"TABLE_Detail": [
{
"TABLE": "table1",
"RUN_DATE": "20170313",
"SEQ_NUM": "123",
"START_TIME": "20170313133144",
"END_TIME": "20170313133655"
},
{
"TABLE": "table2",
"RUN_DATE": "20170313",
"SEQ_NUM": "123",
"START_TIME": "20170313133142",
"END_TIME": "20170313133723"
}
]
}
I have a JSON file (that I have no control over) that looks like this:
{
"some-identifier": {
"#class": "some-prefix.ClassA",
"<classA-property1>": "value1",
"<classA-property2>": "value2",
},
"some-other-identifier": {
"#class": "some-other-prefix.ClassB",
"<classB-property1>": <... possibly nested objects ...>
},
<...>
}
(The classA-properties and classB-properties are the actual names of the members of ClassA and ClassB respectively.)
I would like to deserialize this into a HashMap (mapping each identifier to the actual object) and I want to use a custom TypeIdResolver to determine the actual class to be instantiated (which I can determine from the prefix and class name). The objects themselves should then be deserialized using the default deserializer.
After a lot of searching I couldn't make this work. I need some way to annotate the HashMap in order to set JsonTypeInfo and JsonTypeIdResolver for its content. All examples I've seen so far have those annotations on a base type that all subclasses extend from. However, in my case, there is no common parent class for the classes contained in the JSON (except Object of course). I thought about annotating Object itself with a mixin, but even then this would break default deserialization for the contained objects since it would then expect an #class property on all child objects.
Is there a solution for this scenario?
I think you can accomplish this by enabling the default type information for the object mapper like this:
new ObjectMapper().enableDefaultTyping(
ObjectMapper.DefaultTyping.JAVA_LANG_OBJECT,
JsonTypeInfo.As.PROPERTY);
Here is a complete example:
public class JacksonDefaultTypeInfo {
static class Bean1 {
public String value;
Bean1() {}
public Bean1(final String value) {
this.value = value;
}
#Override
public String toString() {
return "Bean1{" +
"value='" + value + '\'' +
'}';
}
}
static class Bean2 {
public int number;
Bean2() {}
Bean2(final int number) {
this.number = number;
}
}
public static void main(String[] args) throws IOException {
final Map<String, Object> map = new HashMap<>();
map.put("bean1", new Bean1("string"));
map.put("bean2", new Bean2(123));
final ObjectMapper objectMapper = new ObjectMapper()
.enableDefaultTyping(
ObjectMapper.DefaultTyping.JAVA_LANG_OBJECT,
JsonTypeInfo.As.PROPERTY);
final String json = objectMapper
.writerWithDefaultPrettyPrinter()
.writeValueAsString(map);
System.out.println(json);
final Map<String, Object> result = objectMapper.readValue(
json,
new TypeReference<Map<String, Object>>() {});
System.out.println(result);
}
}
Output:
{
"bean1" : {
"#class" : "stackoverflow.JacksonDefaultTypeInfo$Bean1",
"value" : "string"
},
"bean2" : {
"#class" : "stackoverflow.JacksonDefaultTypeInfo$Bean2",
"number" : 123
}
}
{bean1=Bean1{value='string'}, bean2=Bean2{number=123}}
I have the following problem: I export my user object as usual with the jackson.databind.ObjectMapper and that works fine: ({"address":{"village":"NY"},"prename":"Joe"}).
Now I have to get the key (for address and prename) with Java reflection.
If the field has the annotation #JsonProperty, there is no problem to get this key. But this annotation isn't pressent on all fields (for example the m_address field).
At How does the Jackson mapper know what field in each Json object to assign to a class object? I read that the ObjectMapper tries to call the getter or so.
But I have no clue how I can find the right getter to my field.
I know that this isn't probably the most beautiful way to solve my problem, but I haven't found any method on the ObjectMapper like: mapper.getJSONKeyByName(field).
If something like that exist even better. :)
Is there a way to find the right getter to a field and does something like mapper.getJSONKeyByName(field) exist on the ObjectMapper?
Main.java
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
// Object to JSON as usual
mapper.writeValue(System.out, new User("Joe", new Address("NY")));
// {"address":{"village":"NY"},"prename":"Joe"}
// Lookup with reflection
for (Field field : User.class.getDeclaredFields()) {
field.setAccessible(true);
try {
if (field.isAnnotationPresent(JsonProperty.class)) {
System.out.println("JSON-Key with annotation: " +
field.getAnnotation(JsonProperty.class).value());
// JSON-Key with annotation: prename
} else {
//TODO do something to get "JSON-Key without annotation: address
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
User.java
public class User implements Serializable {
#JsonProperty(value="prename")
#Validationinfo(name="prename", required=true, maxlenght=20)
private String m_name;
private Address m_address;
public User(String name, Address a) {
m_name = name;
m_address = a;
}
#JsonIgnore
public String getName() {
return m_name;
}
public void setName(String name) {
m_name = name;
}
public Address getAddress() {
return m_address;
}
public void setAddress(Address address) {
m_address = address;
}
}
Address.java
public class Address implements Serializable {
#JsonProperty(value="village")
#Validationinfo(name="village", required=false, maxlenght=10)
private String m_village;
public Address(String village) {
m_village = village;
}
public String getVillage() {
return m_village;
}
public void setVillage(String village) {
m_village = village;
}
}
EDIT:
The code is simplified. I have a REST service which does the writeValue part. The reflection part is done in a static recursive method on the User.
The thing is I have a custom Validationinfo annotation (with things like required, maxlength and so on) on my fields and also a name parameter. This name is the same as it is on the #JsonProperty annotation.
On an JavaScript application I want merge the value of the user fields with the ValidationInfos. For that I have to ensure that every validationInfos->name is unique. So I have to prefix the annotated Vaditioninfo->name whith the JSON serialized name/key of its parent (see in the REST respons "address.village").
The rest response I am locking for:
{
"user": {
"prename": "Joe",
"address" : {
"village": "NY"
}
}, "validationInfos": [{
"name": "prename",
"required": true,
"maxlenght": 10
}, {
"name": "address.village",
"required": false,
"maxlenght": 20
}]
}
In JavaScript I planning to do something like:
for (var i = 0; i < data.validationInfos.length;; i++) {
var element = data.validationInfos;
element.value = eval ("data.user." + element.name);
}
You should use jackson introspection instead of pure java reflection. It will allow you to discover json properties mapped to java fields/methods according to your serialization config.
JavaType userType = mapper.getTypeFactory().constructType(User.class);
BeanDescription introspection =
mapper.getSerializationConfig().introspect(userType);
List<BeanPropertyDefinition> properties = introspection.findProperties();
// do some processing over properties...