We Keep Coding

Spring Data Mongodb Aggregation - Group by nested objects and build DTO

I have the following Employee data in MongoDB
{
"_id": {
"$oid": "625f09bb1a96bf42ff4c4006"
},
"employeeId": 1234,
"email": "jason#acme.com",
"firstName": "Jason",
"lastName": "Stuart",
"currentCTC": 1201117.61,
"department": {
"$ref": "department",
"$id": {
"$oid": "625f09bb1a96bf42ff4c4005"
}
}
}
{
"_id": {
"$oid": "625f09bb1a96bf42ff4c4006"
},
"employeeId": 1235,
"email": "jasons#acme.com",
"firstName": "Jasons",
"lastName": "Stuarts",
"currentCTC": 1201117.61,
"department": {
"$ref": "department",
"$id": {
"$oid": "625f09bb1a96bf42ff4c4005"
}
}
}
My Spring #Document looks like this:
// Employee.java
#Data
#Document
public class Employee {
#Id
private String id;
private Long employeeId;
private String email;
private String firstName;
private String middleName;
private String lastName;
private Gender gender;
private double currentCTC;
#DBRef
private Department department;
}
// Department.java
#Document
#Data
public class Department {
#Id
private String id;
private String name;
}
Now, my requirement is to find the sum of salaries Department-wise.. I need the data to be in the following way:
[
{
"department": {
"id": "625f09bb1a96bf42ff4c4006",
"name": "Engineering"
},
"cost": 31894773.01
},
{
"department": {
"id": "625f09bb1a96bf42ff4c4006",
"name": "Marketing"
},
"cost": 4552325.25
}
]
I created an aggregate function like this in Spring Data:
public List<DepartmentCost> getDepartmentCosting() {
GroupOperation groupByDepartment = group("department").sum("currentCTC").as("cost").first("$$ROOT").as("department");
Aggregation aggregation = Aggregation.newAggregation(groupByDepartment);
AggregationResults<DepartmentCost> results = mongoTemplate.aggregate(aggregation, "employee", DepartmentCost.class);
return results.getMappedResults();
}
And my expected DepartmentCost.java
#Data
#Document
public class DepartmentCost {
#DBRef
private Department department;
private double cost;
}
Now when I try this API out, I get the data correctly, but I do not get department name. It comes as null. I get a response like
[
{
"department": {
"id": "625f09bb1a96bf42ff4c4006",
"name": null,
},
"cost": 2241117.6100000003
},
{
"department": {
"id": "625f09bb1a96bf42ff4c400a",
"name": null,
},
"cost": 14774021.43
},
{
"department": {
"id": "625f09bc1a96bf42ff4c4013",
"name": null,
},
"cost": 14879633.97
}
]
How can I get the department details expanded in my model? Please help..

After a couple of attempts, I figured it out. All I had to do was this:
GroupOperation groupByDepartment = group("department").sum("currentCTC").as("cost").first("$department").as("department");
as opposed to:
GroupOperation groupByDepartment = group("department").sum("currentCTC").as("cost").first("$$ROOT").as("department");

how to mapped java embbeded object with JSON

I build an google Calendar API, and i miss understand a point with my json files.
I succeed to create my java object with my json files but here the issue:
i have two classes :
public class User {
private String email;
private String firstname;
private String lastname;
Entity entity;
``
and my Entity
`` public class Entity {
private String name;
private String entityType;
private Entity rootEntity;``
here my json file :
for user
``[
{
"firstname": "Jean-Marc",
"lastname": "Chevereau",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"BFA",
"entityType":"secteur"
}
},
{
"firstname": "Florent",
"lastname": "Hamlin",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"IT",
"entityType":"secteur"
}
},
{
"firstname": "Benoit",
"lastname": "Micaud",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"EX",
"entityType":"offre",
"rootEntity":{
"name":"BFA"
}
}
}
]``
And a Entity json file
```[
{
"name": "BFA",
"entityType": "secteur",
"rootEntity": "",
},
{
"name": "EX",
"entityType": "Offre",
"rootEntity": "BFA",
}
}
]
But here the trouble. if in my User.json i write theEntity Name, i dont want to write entitytype and rootEntity, because if i write Entity Name is BFA, it will always be the same entitType and the rootEntity.
In others words, my json Entity will be always the same,and if i just put the name we know that refers to an entity object.
For instance, in this user.json file, I will just need to put
[
{
"firstname": "Jean-Marc",
"lastname": "Chevereau",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"BFA",
}
},
{
"firstname": "Florent",
"lastname": "Hamlin",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"IT",
}
},
{
"firstname": "Benoit",
"lastname": "Micaud",
"email": "xxxxxxx#xxxxx.com",
"entity": {
"name":"EX",
}
}
]

In Json-lib you have a JsonConfig to specify the allowed fields:
JsonConfig jsonConfig=new JsonConfig();
jsonConfig.registerPropertyExclusion(Entity.class,"rootEntity");
jsonConfig.registerPropertyExclusion(Entity.class,"entityType");
JSON json = JSONSerializer.toJSON(objectToWrite,jsonConfig);

I suppose com.fasterxml.jackson's #JsonIgnore annotation should help.
public class Entity {
private String name;
#JsonIgnore
private String entityType;
#JsonIgnore
private Entity rootEntity;
}

Custom Gson serializer with fields that are unknown at runtime

I'm attempting to do custom Gson serialization to create a Json object to send to a service except their are some fields that are not known at runtime.
The json I wish to create should look something like this:
{
"type": "configuration/entityTypes/HCP",
"attributes": {
"FirstName": [
{
"type": "configuration/entityTypes/HCP/attributes/FirstName",
"value": "Michael"
}
]
},
"crosswalks": [
{
"type": "configuration/sources/AMA",
"value": "10000012"
}
]
}
I am able to successfully create this json using Gson, but the issue is that I have thousands of fields that could be under the attributes object, in this example there is only the FirstName but if I was doing a create there would be as many attributes as that person, place or thing had associated with them.
Because currently I am able to create this using Gson by having 4 different classes:
Type
Attributes
FirstName
Crosswalks
But I want to be able to have FirstName, LastName, MiddleName, etc. all underneath the attributes object without creating an individual java class for all of them. The json would look like this in that case:
{
"type": "configuration/entityTypes/HCP",
"attributes": {
"FirstName": [
{
"type": "configuration/entityTypes/HCP/attributes/FirstName",
"value": "Doe"
}
],
"LastName": [
{
"type": "configuration/entityTypes/HCP/attributes/LastName",
"value": "John"
}
],
"MiddleName": [
{
"type": "configuration/entityTypes/HCP/attributes/MiddleName",
"value": "Michael"
}
]
},
"crosswalks": [
{
"type": "configuration/sources/AMA",
"value": "10000012"
}
]
}
Is there a way to use Gson to create the attributes object without creating java objects for all of the different attributes I have?

You can use Map<String, Object> where Object will be an one-element-array. See, for example, below model:
class Attributes {
private Map<String, Object> attributes;
// getters, setters
}
class Type {
private final String type;
private final String value;
public Type(String type, String value) {
this.type = type;
this.value = value;
}
// getters
}
Now, let's build attributes manually:
import com.google.gson.Gson;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
public class GsonApp {
public static void main(String[] args) {
Map<String, Object> map = new HashMap<>();
map.put("FirstName", Collections.singletonList(new Type("url/FirstName", "Rick")));
map.put("LastName", Collections.singletonList(new Type("url/LastName", "Pickle")));
Attributes attributes = new Attributes();
attributes.setAttributes(map);
String json = new Gson().newBuilder().setPrettyPrinting().create().toJson(attributes);
System.out.println(json);
}
}
Above code prints:
{
"attributes": {
"FirstName": [
{
"type": "url/FirstName",
"value": "Rick"
}
],
"LastName": [
{
"type": "url/LastName",
"value": "Pickle"
}
]
}
}

Deserialize complex JSON to Java, classes nested multiple levels deep

I am trying to make the Json output from Cucumber into a single Java object. This contains objects nested four levels deep, and I am having trouble deserializing it. I am presently using Jackson, but open to suggestions.
Here is my Json code:
{
"line": 1,
"elements": [
{
"line": 3,
"name": "Converteren centimeters naar voeten/inches",
"description": "",
"id": "applicatie-neemt-maten-in-cm-en-converteert-ze-naar-voet/inch,-en-vice-versa;converteren-centimeters-naar-voeten/inches",
"type": "scenario",
"keyword": "Scenario",
"steps": [
{
"result": {
"duration": 476796588,
"status": "passed"
},
"line": 4,
"name": "maak Maten-object aan met invoer in \"centimeters\"",
"match": {
"arguments": [
{
"val": "centimeters",
"offset": 37
}
],
"location": "StepDefinition.maakMatenObjectAanMetInvoerIn(String)"
},
"keyword": "Given "
},
{
"result": {
"duration": 36319,
"status": "passed"
},
"line": 5,
"name": "ik converteer",
"match": {
"location": "StepDefinition.converteerMaten()"
},
"keyword": "When "
},
{
"result": {
"duration": 49138,
"status": "passed"
},
"line": 6,
"name": "uitvoer bevat maat in \"voeten/inches\"",
"match": {
"arguments": [
{
"val": "voeten/inches",
"offset": 23
}
],
"location": "StepDefinition.uitvoerBevatMaatIn(String)"
},
"keyword": "Then "
}
]
},
{
"line": 8,
"name": "Converteren voeten/inches naar centimeters",
"description": "",
"id": "applicatie-neemt-maten-in-cm-en-converteert-ze-naar-voet/inch,-en-vice-versa;converteren-voeten/inches-naar-centimeters",
"type": "scenario",
"keyword": "Scenario",
"steps": [
{
"result": {
"duration": 84175,
"status": "passed"
},
"line": 9,
"name": "maak Maten-object aan met invoer in \"voeten/inches\"",
"match": {
"arguments": [
{
"val": "voeten/inches",
"offset": 37
}
],
"location": "StepDefinition.maakMatenObjectAanMetInvoerIn(String)"
},
"keyword": "Given "
},
{
"result": {
"duration": 23928,
"status": "passed"
},
"line": 10,
"name": "ik converteer",
"match": {
"location": "StepDefinition.converteerMaten()"
},
"keyword": "When "
},
{
"result": {
"duration": 55547,
"status": "passed"
},
"line": 11,
"name": "uitvoer bevat maat in \"centimeters\"",
"match": {
"arguments": [
{
"val": "centimeters",
"offset": 23
}
],
"location": "StepDefinition.uitvoerBevatMaatIn(String)"
},
"keyword": "Then "
}
]
}
],
"name": "Applicatie neemt maten in cm en converteert ze naar voet/inch, en vice versa",
"description": "",
"id": "applicatie-neemt-maten-in-cm-en-converteert-ze-naar-voet/inch,-en-vice-versa",
"keyword": "Feature",
"uri": "sample.feature"
}
I have tried a number of different approaches. First I used nested inner classes, but it appeared you had to make them static, which I feared would not work since I have multiple instances of the same object within one (multiple "element"-objects in the root, for example). Then I tried putting them in separate classes, with Json annotations. Here's where that got me (omitting setters):
public class CucumberUitvoer {
private String name;
private String description;
private String id;
private String keyword;
private String uri;
private int line;
#JsonProperty("elements")
private List<FeatureObject> elements;
public CucumberUitvoer(){}
}
public class FeatureObject {
private String name;
private String description;
private String id;
private String type;
private String keyword;
private int line;
#JsonProperty("steps")
private List<StepObject> steps;
public FeatureObject() {
}
}
public class StepObject {
#JsonProperty("result")
private ResultObject result;
private String name;
private String given;
private String location;
private String keyword;
private int line;
#JsonProperty("match")
private MatchObject match;
public StepObject(){}
}
public class ResultObject {
private int duration;
private String status;
public ResultObject(){}
}
public class MatchObject {
#JsonProperty("arguments")
private List<ArgumentObject> arguments;
private String location;
public MatchObject(){}
}
public class ArgumentObject {
private String val;
private String offset;
public ArgumentObject(){}
}
For clarification, here's a class diagram of how the nesting works.
This solution gives me the following error:
com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of nl.icaprojecten.TestIntegratieQuintor.JSONInterpreter.CucumberUitvoer out of START_ARRAY token
Here is the code doing the actual mapping:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
CucumberUitvoer obj1 = null;
try {
obj1 = mapper.readValue(json, CucumberUitvoer.class);
} catch (IOException e) {
e.printStackTrace();
}
Is there a quick fix to this approach to make it work, or should I try something entirely different?

Ok I spent some time debugging and trying to figure out what was the problem, and finally was something pretty obvious.
implements Serializable
Thats the line I added to MatchObject and worked.
When we try to deserialize some object first we have to make those classes implements the interface Serializable

I just tried your sample code and oddly, it works.
Can you please double check your imports, if the JSON is coming in as provided and the getters, setters, constructors are actually there?

You can get the idea from this code to deserialize,
public class testCustomDeSerializer extends JsonDeserializer<test> {
public testCustomDeSerializer() {
this(null);
}
public TestCustomDeSerializer(Class t) {
// super(t);
}
#Override
public Test deserialize(JsonParser p, DeserializationContext ctx) throws IOException, JsonProcessingException {
ObjectCodec objectCodec = p.getCodec();
JsonNode node = objectCodec.readTree(p);
ObjectMapper objectMapper = new ObjectMapper();
Test test= new Test();
test.setId(node.get("line").asText());
List<elements> elementList = new ArrayList<>();
JsonNode elementsNode = node.get("elements");
Iterator<JsonNode> slaidsIterator = elementsNode.elements();
while (slaidsIterator.hasNext()) {
Steps steps= new Steps();
JsonNode slaidNode = slaidsIterator.next();
JsonNode stepNode= (JsonNode) slaidNode.get("Steps");
BoundingPoly in = objectMapper.readValue(stepNode.toString(), Steps.class);
elementsNode.setSteps(in);
/// continue
return
}
Hope it helps

Mapping JSONArray in RestTemplate Spring

I am trying to map this JSONArray using Spring RestTemplate:
[{
"Command": "/usr/sbin/sshd -D",
"Created": 1454501297,
"Id": "e00ca61f134090da461a3f39d47fc0cbeda77fbbc0610439d3c16a932686b612",
"Image": "ubuntu:latest",
"Labels": {
},
"Names": [
"/nova-c1896fbd-1309-4da2-8d77-b4fe4c02fa8e"
],
"Ports": [
],
"Status": "Up 2 hours"
}, {
"Command": "/usr/sbin/sshd -D",
"Created": 1450106126,
"Id": "7ffc9dbdd200e2c23adec442abd656ed57306955332697cb7da979f36ebf3b22",
"Image": "ubuntu:latest",
"Labels": {
},
"Names": [
"/nova-93b9ae40-8135-48b7-ac17-12094603b28c"
],
"Ports": [
],
"Status": "Up 2 hours"
}]
Here is ContainersInfo class:
#JsonIgnoreProperties(ignoreUnknown = true)
public class ContainersInfo {
private String Id;
private List<String> Names;
public String getId() {
return Id;
}
public void setId(String id) {
Id = id;
}
public List<String> getNames() {
return Names;
}
public void setNames(List<String> names) {
Names = names;
}
}
However I get null when I want to get the data:
ContainersInfo[] containers = syncRestTemplate.getForObject("http://192.168.1.2:4243/containers/json?all=1", ContainersInfo[].class);
for (int i = 0; i < containers.length; i++)
System.out.println("id:" + containers[i].getId());
The resulting output is as follows:
id:null
id:null
Any idea, what I should do?

Your JSON field names are in pascal case as opposed to camel case (which is usually the case). Set Jackson naming strategy to PascalCaseStrategy, i.e by adding #JsonNaming(PascalCaseStrategy.class) annotation into ContainersInfo class.