We Keep Coding

Null values being returned in json while using gson in java

I have the following Json file which I am trying to read:
{
"billingInformation": {
"taxes": {
"gst": 2.5,
"hst": 7.8
},
"billTo": {
"name" : "Mike",
"address" : "123, Lake Shore Drive, California",
"phoneNumber" : "601 855 1249"
},
"salesAgent": {
"name" : "Charlotte Thompson",
"agentCode" : 44551
},
"items": {
"item": [
{
"hsnCode": "5112",
"description": "TV Set",
"originCountry": "US",
"quantity": 1,
"unitPrice": 150.00
}
],
"currency": "USD"
}
}
}
I used direct Object Mapping provided by Gson:
result = new String(Files.readAllBytes(Paths.get(path)));
BillingInformation billingInformation = gson.fromJson(result, BillingInformation.class);
But it always resulted in:
class BillingInformation {
taxes: null
billTo: null
salesAgent: null
items: null
}
How can I get the data inside the other objects?
Edit:
Here is the BillingInformation Class:
public class BillingInformation {
#SerializedName("taxes")
private Taxes taxes;
#SerializedName("billTo")
private BillTo billTo;
#SerializedName("salesAgent")
private SalesAgent salesAgent;
#SerializedName("items")
private Items items;
}
and I have the usual getters and setters for the above fields.

You should create a class that looks like this:
class Data {
private BillingInformation billingInformation;
}
And then do the deserialization:
Data fromFile = gson.fromJson(result, Data.class);

Mongo aggregation for a DbRef field with mongoTemplate

Assume I have two mongo collections as follows.
Collection A
{
"_id" : ObjectId("582abcd85d2dfa67f44127e0"),
"level" : "super"
"dataReference" : Object
B : DbRef(B, 5b618a570550de0021aaa2ef, undefined)
}
Collection B
{
"_id" : ObjectId("5b618a570550de0021aaa2ef"),
"role" : "admin"
}
What I need is retrieve the records from Collection A, which records have "level" field's value as "super" and its related Collection B record's "role" value as "admin".
For this, I am trying to use aggregation and java mongoTemplate.
Following is the code that I tried but it returns 0 records.
final TypedAggregation<A> typedAggregation = Aggregation.newAggregation(A.class,
Aggregation.match(Criteria.where("level").equals(level)),
Aggregation.lookup("B", "_id", "dataReference.B.$id", "Basb"),
Aggregation.match(new Criteria().andOperator(
Criteria.where("B.role").regex("admin")
)));
final AggregationResults<Map> A = mongoTemplate.aggregate(typedAggregation, "A", Map.class);
Please note that I am new to Mongo aggregation.

It's quiet ugly solution:
MongoTemplate
You cannot use TypedAggregation because we need to transform A collection to be able join with B collection
Aggregation typedAggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("level").is("super")),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return Document.parse("{\"$addFields\":{\"dataReference\":{\"$reduce\":{\"input\":{\"$objectToArray\":\"$dataReference\"},\"initialValue\":null,\"in\":{\"$cond\":[{\"$eq\":[{\"$type\":\"$$this.v\"},\"objectId\"]},\"$$this.v\",\"$$value\"]}}}}}");
}
},
Aggregation.lookup("B", "dataReference", "_id", "B"),
Aggregation.match(new Criteria().andOperator(
Criteria.where("B.role").regex("admin")
)
)
);
final AggregationResults<Document> A = mongoTemplate.aggregate(typedAggregation, "A", Document.class);
MongoDB Aggregation
db.A.aggregate([
{
"$match": {
"level": "super"
}
},
{
"$addFields": {
"B": {
"$reduce": {
"input": {
"$objectToArray": "$dataReference"
},
"initialValue": null,
"in": {
"$cond": [
{
"$eq": [
{
"$type": "$$this.v"
},
"objectId"
]
},
"$$this.v",
"$$value"
]
}
}
}
}
},
{
"$lookup": {
"from": "B",
"localField": "B",
"foreignField": "_id",
"as": "B"
}
},
{
"$match": {
"$and": [
{
"B.role": {
"$regex": "admin",
"$options": ""
}
}
]
}
}
])
MongoPlayground

Delete duplicates items based on condition

I have the below list that has duplicate elements
I want to remove all duplicates from the list based on the version and date property
This means, if there's a diplicate element, i get the one that has state actif, if no one has state actif then i get the one with the recent date
[
{
"_id" : ObjectId("5e1832df02f04352705457dd"),
"product":"1",
"version":{
"state":"Actif",
"name":"1.0.0"
},
"createdDate":"01/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705457ff"),
"product":"1",
"version":{
"state":"A faire",
"name":"3.0.0"
},
"createdDate":"01/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705457ee"),
"product":"1",
"version":{
"state":"Archiver",
"name":"2.0.0"
},
"createdDate":"02/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705457gg"),
"product":"2",
"version":null,
"createdDate":"01/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705457yy"),
"product":"2",
"version":{
"state":"Archiver",
"name":"2.0.0"
},
"createdDate":"02/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705455ss"),
"product":"3",
"version":{
"state":"Archiver",
"name":"2.0.0"
},
"createdDate":"01/01/2020"
}
]
The output should look like:
[
{
"_id" : ObjectId("5e1832df02f04352705457dd"),
"product":"1",
"version":{
"state":"Actif",
"name":"1.0.0"
},
"createdDate":"01/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705457yy"),
"product":"2",
"version":{
"state":"Archiver",
"name":"2.0.0"
},
"createdDate":"02/01/2020"
},
{
"_id" : ObjectId("5e1832df02f04352705455ss"),
"product":"3",
"version":{
"state":"Archiver",
"name":"2.0.0"
},
"createdDate":"01/01/2020"
}
]
public List<Product> search() {
final Query query = new Query().with(new Sort(new Order(Direction.DESC, "createdDate")));
return mongoOperations.find(query, Product.class);
}
How can we do this ?

You need to use aggregate method.
Java code
Not tested!
If AggregationOperation expects to implement toDocument method, change all BasicDBObject to Document.
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation aggregation = newAggregation(
sort(Sort.Direction.ASC, "product", "version", "createdDate"),
group("product").last("$$ROOT").as("root"),
sort(Sort.Direction.ASC, "_id"),
//$replaceRoot
);
return mongoTemplate.aggregate(aggregation, Product.class, Product.class).getMappedResults();
MongoDB shell
db.Product.aggregate([
{
$sort: {
product: 1,
version: -1,
createdDate: -1
}
},
{
$group: {
_id: "$product",
root: {
$push: "$$ROOT"
}
}
},
{
$sort: {
_id: 1
}
},
{
$replaceRoot: {
newRoot: {
$arrayElemAt: [
{
$concatArrays: [
{
$filter: {
input: "$root",
cond: {
$eq: [
"$$this.version.state",
"Actif"
]
}
}
},
[
{
$arrayElemAt: [
"$root",
0
]
}
]
]
},
0
]
}
}
}
])
MongoPlayground

The aggregation using Spring Data 2.2 and MongoDB 3.4 compatible aggregation operators.
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "spr_test");
Aggregation agg = newAggregation(
project("product", "version")
.and(Concat.valueOf(SubstrCP.valueOf("createdDate").substringCP(6, 4))
.concat("-")
.concatValueOf(SubstrCP.valueOf("createdDate").substringCP(3, 2))
.concat("-")
.concatValueOf(SubstrCP.valueOf("createdDate").substringCP(0, 2)) )
.as("createdDate"),
group("product")
.push("$$ROOT").as("details")
.push("createdDate").as("createdDates"),
facet(
project("details")
.and(Size.lengthOfArray("details")).as("arrSize"),
match(where("arrSize").is(new Integer(1))))
.as("c1")
.and(
match(where("details.version.state").is("Actif" )),
project()
.and(filter("details")
.as("d")
.by(Eq.valueOf("d.version.state").equalToValue("Actif" )))
.as("details"))
.as("c2")
.and(
project("details", "createdDates")
.and(Size.lengthOfArray("details")).as("arrSize"),
match(where("arrSize").gt(new Integer(1))
.and("details.version.state").ne("Actif")),
project()
.and(filter("details")
.as("d")
.by(Eq.valueOf("d.createdDate").equalTo(Max.maxOf("createdDates"))))
.as("details"))
.as("c3"),
project()
.and(arrayOf("c1").concat("c2").concat("c3"))
.as("result"),
unwind("result"),
project()
.and(arrayOf("result.details").elementAt(0)).as("result")
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "test", Document.class);
results.forEach(System.out::println);

How to implement aggregation query in Spring Data MongoDB?

I am new with Spring Data MongoDB and I am trying to implement an aggregation query in Java with Spring Data MongoDB. I have tried searching from this problem and approached it using MongoTemplate, but still to no result.
The format of my data:
[{
"_id" : ObjectId("5e1aea6c275360baf96bac29"),
"title" : "postim",
"upvotesBy" : [
"5e18b4c12753608718dfa007",
"5e19ac0f5161a4994ded1f35"
],
"file" : "test",
"description" : "description",
"postedBy" : "5e18b4c12753608718dfa007",
"createdAt" : ISODate("2020-01-12T09:44:12.119+0000"),
"_class" : "com.socialnetwork.post.Post"
},
{
"_id" : ObjectId("5e1aeaf8275360bb4bb47325"),
"title" : "postim2",
"upvotesBy" : [
"5e18b4c12753608718dfa007",
"5e19ac0f5161a4994ded1f35"
],
"file" : "test2",
"description" : "description2",
"postedBy" : "5e18b4c12753608718dfa007",
"createdAt" : ISODate("2020-01-12T09:46:32.909+0000"),
"_class" : "com.socialnetwork.post.Post"
}]
My query:
db.post.aggregate([
{
$match: {}
},
{
$lookup: {
from: "users",
localField: "postedBy",
foreignField: "_id",
as: "user"
}
},
{
$group: {
_id: {
username: "$user.name",
title: "$title",
description: "$description",
upvotes: { $size: "$upvotesBy" },
upvotesBy: "$upvotesBy",
isUpvoted: { $in: [req.query.userId, "$upvotesBy"] },
isPinned: {
$cond: {
if: { $gte: [{ $size: "$upvotesBy" }, 3] },
then: true,
else: false
}
},
file: "$file",
createdAt: {
$dateToString: {
format: "%H:%M %d-%m-%Y",
timezone: "+01",
date: "$createdAt"
}
},
id: "$_id"
}
}
},
{ $sort: { "_id.isPinned": -1, "_id.createdAt": -1 } }
])
This is the query I use in my Javascript backend and I can do this fairly easy with Mongoose. However I am having some difficulty with the Java implementation of it.
private LookupOperation getLookupOperation() {
return LookupOperation.newLookup().from("user")
.localField("postedBy")
.foreignField("_id")
.as("user");
}
#Override
public List<PostSummary> aggregate() {
LookupOperation lookupOperation = getLookupOperation();
return mongoTemplate.aggregate(Aggregation.newAggregation(lookupOperation, Aggregation.group("id")
.addToSet("user.name").as("username")
.addToSet("title").as("title")
.addToSet("description").as("description")
.addToSet("id").as("id")
.push("upvotesBy").as("upvotesBy")
.addToSet("file").as("file")
.addToSet("createdAt").as("createdAt")
), Post.class, PostSummary.class).getMappedResults();
}
When I try to run this I get the following error:
"Cannot convert [] of type class java.util.ArrayList into an instance of class java.lang.Object! Implement a custom Converter<class java.util.ArrayList, class java.lang.Object> and register it with the CustomConversions. Parent object was: com.socialnetwork.post.PostSummary#7159d908"
When I delete the .addToSet("user.name").as("username") from the group aggregation I also get an error from .push("upvotesBy").as("upvotesBy") as it can not convert [] of type class java.util.ArrayList into an instance of class java.lang.String
Also the implementation of the Post Class and the PostSummary Class is simple:
Post.java:
#Document
public class Post {
#Id
private String id;
private String title;
private List<String> upvotesBy;
private String file;
private String description;
private String postedBy;
private Date createdAt = new Date();
// ... Getters and Setters for each field
}
PostSummary.java:
public class PostSummary {
private String username;
private String title;
private String description;
private List<String> upvotesBy;
private String file;
private String createdAt;
private String id;
//... Getters and Setters for the class
}
I also need to implement the isUpvoted and isPinned part of the query, but getting the idea on how to approach the first problem would be a great start.
EDIT: My desired output:
[
{
"username" : "user1",
"title" : "postim2",
"upvotesBy" : [
"5e18b4c12753608718dfa007",
"5e19ac0f5161a4994ded1f35"
],
"file": "file1",
id: "5e18b4c12753608718dber01"
... Other fields of the original post
},
{
"username" : "user2",
"title" : "postim2",
"upvotesBy" : [
"5e18b4c12753608718dfa007",
"5e19ac0f5161a4994ded1f35"
],
id: "5e18b4c12753608718dber02",
"file": "file2",
... Other fields of the original post
}
]
So from the lookup operation I need only to get the name of the user.

Let's do it
We need to update your aggregation to make it work.
Errors:
users's _id is ObjectId type, but in your post you have stored as String, so $lookup should be changed to Uncorrelated sub-queries
We replace $group by '$addFields' which fits better
We add as last stage $project operator to exclude all unsed fields.
db.post.aggregate([
{
$match: {}
},
{
$lookup: {
from: "users",
let: {
postedBy: "$postedBy"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
{
"$toString": "$_id"
},
"$$postedBy"
]
}
}
}
],
as: "user"
}
},
{
$unwind: "$user"
},
{
$addFields: {
id: {
$toString: "$_id"
},
username: "$user.name",
upvotes: {
$size: "$upvotesBy"
},
isUpvoted: {
$in: [
"5e18b4c12753608718dfa007",
"$upvotesBy"
]
},
isPinned: {
$cond: [
{
$gte: [
{
$size: "$upvotesBy"
},
3
]
},
true,
false
]
},
createdAt: {
$dateToString: {
format: "%H:%M %d-%m-%Y",
timezone: "+01",
date: "$createdAt"
}
}
}
},
{
$sort: {
"isPinned": -1,
"createdAt": -1
}
},
{
$project: {
_id: 0,
user: 0,
upvotesBy: 0,
_class: 0
}
}
])
Now, we transform this query to Spring-Data syntax.
Java Implementation
package postman;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.match;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.project;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.sort;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.unwind;
import java.util.Arrays;
import java.util.List;
import org.bson.Document;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Sort.Direction;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.aggregation.Aggregation;
import org.springframework.data.mongodb.core.aggregation.AggregationOperation;
import org.springframework.data.mongodb.core.aggregation.AggregationOperationContext;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.stereotype.Service;
#Service
public class PostmanService {
#Autowired
private MongoTemplate mongoTemplate;
public List<PostSummary> find(String userId){
Aggregation aggregation = Aggregation.newAggregation(
match(new Criteria()),
//lookup("users", "postedBy", "_id", "user")
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document("$lookup",
new Document("from", "users")
.append("let", new Document("postedBy", "$postedBy"))
.append("pipeline", Arrays.asList(
new Document("$match",
new Document("$expr",
new Document("$eq", Arrays.asList(
new Document("$toString", "$_id"),
"$$postedBy"
))))))
.append("as", "user"));
}
},
unwind("$user"),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document("$addFields",
new Document("id", new Document("$toString", "$_id"))
.append("username", "$user.name")
.append("upvotes", new Document("$size", "$upvotesBy"))
.append("isUpvoted", new Document("$in", Arrays.asList(userId, "$upvotesBy")))
.append("isPinned", new Document("$cond",
Arrays.asList(new Document("$gte",
Arrays.asList(new Document("$size", "$upvotesBy"), 3)), Boolean.TRUE, Boolean.FALSE)))
.append("createdAt", new Document("$dateToString",
new Document("format", "%H:%M %d-%m-%Y")
.append("timezone", "+01")
.append("date", "$createdAt")
)));
}
},
sort(Direction.DESC, "isPinned", "createdAt"),
project().andExclude("user", "_class")
);
System.out.println("Aggregation: " + aggregation.toString());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Post.class), PostSummary.class).getMappedResults();
}
}
Now, we call aggregation pipeline:
List<PostSummary> l = postmanService.find("5e18b4c12753608718dfa007");
for(PostSummary post: l) {
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
System.out.println(ow.writeValueAsString(post));
}
2020-01-12 16:15:22.043 INFO 11148 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2020-01-12 16:15:22.047 INFO 11148 --- [ main] Postman.PostmanApplication : Started PostmanApplication in 4.602 seconds (JVM running for 5.301)
Aggregation: { "aggregate" : "__collection__", "pipeline" : [{ "$match" : {}}, { "$lookup" : { "from" : "users", "let" : { "postedBy" : "$postedBy"}, "pipeline" : [{ "$match" : { "$expr" : { "$eq" : [{ "$toString" : "$_id"}, "$$postedBy"]}}}], "as" : "user"}}, { "$unwind" : "$user"}, { "$addFields" : { "id" : { "$toString" : "$_id"}, "username" : "$user.name", "upvotes" : { "$size" : "$upvotesBy"}, "isUpvoted" : { "$in" : ["5e18b4c12753608718dfa007", "$upvotesBy"]}, "isPinned" : { "$cond" : [{ "$gte" : [{ "$size" : "$upvotesBy"}, 3]}, true, false]}, "createdAt" : { "$dateToString" : { "format" : "%H:%M %d-%m-%Y", "timezone" : "+01", "date" : "$createdAt"}}}}, { "$sort" : { "isPinned" : -1, "createdAt" : -1}}, { "$project" : { "user" : 0, "_class" : 0}}]}
2020-01-12 16:15:22.161 INFO 11148 --- [ main] org.mongodb.driver.connection : Opened connection [connectionId{localValue:2, serverValue:277}] to localhost:27017
{
"username" : "user1",
"title" : "postim2",
"description" : "description2",
"upvotesBy" : [ "5e18b4c12753608718dfa007", "5e19ac0f5161a4994ded1f35" ],
"file" : "test2",
"createdAt" : "10:46 12-01-2020",
"id" : "5e1aeaf8275360bb4bb47325"
}
{
"username" : "user1",
"title" : "postim",
"description" : "description",
"upvotesBy" : [ "5e18b4c12753608718dfa007", "5e19ac0f5161a4994ded1f35" ],
"file" : "test",
"createdAt" : "10:44 12-01-2020",
"id" : "5e1aea6c275360baf96bac29"
}

MongoDb Lookup gives empty aggregation result when using Spring Data Mongodb

The following is a app_relation Collection :
{
"_id" : ObjectId("5bf518bb1e9f9d2f34a8299b"),
"app_id" : "123456789",
"dev_id" : "1",
"user_id" : "1",
"status" : "active",
"created" : NumberLong(1542789294)
}
The other collection is app :
{
"_id" : ObjectId("5bd02abb1e9f9d2adc211138"),
"app_id" : "123456789",
"custom_app_name" : "Demo",
"price" : 10,
"created" : NumberLong(1540369083)
}
Using Lookup in mongodb I want to embed App collection in AppRelation
For the same my mongodb query is:
db.app_relation.aggregate([
{
$lookup: {
"from": "app",
"localField": "app_id",
"foreignField": "app_id",
"as": "data"
}
},
{
$match: {
"data": {
"$size": 1
}
}
}
])
The equivalent code in Spring Java is :
LookupOperation lookupOperation = LookupOperation.newLookup().from("app").localField("app_id")
.foreignField("app_id").as("data");
AggregationOperation match = Aggregation.match(Criteria.where("data").size(1));
Aggregation aggregation = Aggregation.newAggregation(lookupOperation, match)
.withOptions(Aggregation.newAggregationOptions().cursor(new BasicDBObject()).build());
List<AppRelation> results = mongoTemplate.aggregate(aggregation, AppRelation.class, AppRelation.class)
.getMappedResults();
When executing the above code it provides the empty collection, whereas executing mongo db query it provides proper result.
The query generated in Debug logs is:
{
"aggregate": "app_relation",
"pipeline": [
{
"$lookup": {
"from": "app",
"localField": "app_id",
"foreignField": "app_id",
"as": "data"
}
},
{
"$match": {
"data": {
"$size": 1
}
}
}
],
"cursor": {}
}