I'm looking for a way to implement graphlookup using Java in a Resful Web API. I'm trying to implement the hierarchy like on the MongoDB (https://docs.mongodb.com/v3.4/reference/operator/aggregation/graphLookup/)
{ "_id" : 1, "name" : "Dev" }
{ "_id" : 2, "name" : "Eliot", "reportsTo" : "Dev" }
{ "_id" : 3, "name" : "Ron", "reportsTo" : "Eliot" }
{ "_id" : 4, "name" : "Andrew", "reportsTo" : "Eliot" }
{ "_id" : 5, "name" : "Asya", "reportsTo" : "Ron" }
{ "_id" : 6, "name" : "Dan", "reportsTo" : "Andrew" }
This is the employee collection, what I want is to be able to create this structure stored in MongoDB
{
"_id" : 1,
"name" : "Dev",
"reportingHierarchy" : [ ]
}
{
"_id" : 2,
"name" : "Eliot",
"reportsTo" : "Dev",
"reportingHierarchy" : [
{ "_id" : 1, "name" : "Dev" }
]
}
{
"_id" : 3,
"name" : "Ron",
"reportsTo" : "Eliot",
"reportingHierarchy" : [
{ "_id" : 1, "name" : "Dev" },
{ "_id" : 2, "name" : "Eliot", "reportsTo" : "Dev" }
]
}
I've seen examples like this for aggregation, but nothing on graphlookup
Aggregation agg = newAggregation(
match(Criteria.where("pageId").is("2210")),
unwind("postIds"),
group("_id").sum("1").as("sum")
//project("$sum").and("pageId").previousOperation()
);
Is there a way to get graphlookup into a format like this? Where instead of using match, unwind, group, I can use GraphLookupOperation and then something like get map result.
AggregationOperation aggregationOperation = new AggregationOperation() {
#Override public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
DBObject graphLookup = new BasicDBObject(
"from", "individual").append(
"startWith", "$reportsTo").append(
"connectFromField", "reportsTo").append(
"connectToField", "firstName").append(
"maxDepth", 2).append(
"as", "reportingHierarchy");
return new BasicDBObject("$graphLookup", graphLookup);
This code allows for a workaround since i'm using an older version, I think 1.10.0 spring mongo. Now I have the issues of the "reportingHierarchy" not looking like I want it to. It's not giving me a name. Just reportsTo and reportingHierarchy that is also including _class which I don't want.
Related
I use the Java Spring MongoDB repository in my project.
I have this collection in MongoDB called Info:
{ "_id" : 1, "hosting" : "hostgator.com", count:7 }
{ "_id" : 2, "hosting" : "aws.amazon.com", count:7}
{ "_id" : 3, "hosting" : "aws.amazon.com", count:3}
{ "_id" : 4, "hosting" : "hostgator.com", count:5 }
{ "_id" : 5, "hosting" : "aws.amazon.com", count:1 }
{ "_id" : 6, "hosting" : "cloud.google.com", count:1 }
{ "_id" : 7, "hosting" : "aws.amazon.com", count:5 }
{ "_id" : 8, "hosting" : "hostgator.com", count:2 }
{ "_id" : 9, "hosting" : "cloud.google.com", count:3 }
{ "_id" : 10,"hosting" : "godaddy.com", count:7 }
...
{ "_id" : 100, "hosting" : "godaddy.com", count:5 }
Here is DTO definition:
public class Info{
public int _id;
public String hosting;
public int count;
}
I need to write a query and to get from the database all values of count property and remove the duplications. For example, the result that I expect according to the collection above is:
List<int> counts = [1,2,3,5,7];
For this purpose I use the aggregation group method and MongoTemplate:
GroupOperation groupOperation = Aggregation.group("count");
Aggregation aggregation = Aggregation.newAggregation(groupOperation);
var result = template.aggregate(aggregation, Info.class, Info[].class);
System.out.println(result.getMappedResults());
But the result that I get is an empty array - [].
Why I don't get the expected result?
I have an index where each of my objects has status field which can have some predefined values. I want to fetch all of them which has statusINITIATED, UPDATED, DELETED, any match with these and hence created this query by java which I got printing on console, using Querybuilder and nativeSearchQuery, executing by ElasticsearchOperations:
{
"bool" : {
"must" : [
{
"terms" : {
"status" : [
"INITIATED",
"UPDATED",
"DELETED"
],
"boost" : 1.0
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
I have data in my index with 'INITIATED' status but not getting anyone with status mentioned in the query. How to fix this query, please?
If you need anything, please let me know.
Update: code added
NativeSearchQueryBuilder nativeSearchQueryBuilder=new NativeSearchQueryBuilder();
QueryBuildersingleQb=QueryBuilders.boolQuery().must(QueryBuilders.termsQuery("status",statusList));
Pageable pageable = PageRequest.of(0, 1, Sort.by(Defs.START_TIME).ascending());
FieldSortBuilder sort = SortBuilders.fieldSort(Defs.START_TIME).order(SortOrder.ASC);
nativeSearchQueryBuilder.withQuery(singleQb);
nativeSearchQueryBuilder.withSort(sort);
nativeSearchQueryBuilder.withPageable(pageable);
nativeSearchQueryBuilder.withIndices(Defs.SCHEDULED_MEETING_INDEX);
nativeSearchQueryBuilder.withTypes(Defs.SCHEDULED_MEETING_INDEX);
NativeSearchQuery searchQuery = nativeSearchQueryBuilder.build();
List<ScheduledMeetingEntity> scheduledList=elasticsearchTemplate.queryForList(searchQuery, ScheduledMeetingEntity.class);
Update 2: sample data:
I got this from kibana query on this index:
"hits" : [
{
"_index" : "index_name",
"_type" : "type_name",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-03-03T13:09:59.198",
"createTimeInMs" : 1614755399198,
"createdBy" : "user1#domain.com",
"editTime" : "2021-03-03T13:09:59.198",
"editTimeInMs" : 1614755399198,
"editedBy" : "user1#domain.com",
"versionId" : 1,
"id" : "1",
"meetingId" : "47",
"userId" : "129",
"username" : "user1#domain.com",
"recipient" : [
"user1#domain.com"
],
"subject" : "subject",
"body" : "hi there",
"startTime" : "2021-03-04T07:26:00.000",
"endTime" : "2021-03-04T07:30:00.000",
"meetingName" : "name123",
"meetingPlace" : "placeName",
"description" : "sfsafsdafsdf",
"projectName" : "",
"status" : "INITIATED",
"failTry" : 0
}
}
]
Confirm your mapping:
GET /yourIndexName/_mapping
And see if it is valid
Your mapping needs to have keyword for TermsQuery to work.
{
"status": {
"type" "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
ES can automatically do the mapping for you (without you having to do it yourself) when you first push a document. However you probably have finer control if you do the mapping yourself.
Either way, you need to have keyword defined for your status field.
=====================
Alternative Solution: (Case Insensitive)
If you have a Field named (status), and the values you want to search for are (INITIATED or UPDATED, or DELETED).
Then you can do it like this:
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
.must(createStringSearchQuery());
public QueryBuilder createStringSearchQuery(){
QueryStringQueryBuilder queryBuilder = QueryBuilders.queryStringQuery(" INITIATED OR UPDATED OR DELETED ");
queryBuilder.defaultField("status");
return queryBuilder;
}
Printing the QueryBuilder:
{
"query_string" : {
"query" : "INITIATED OR UPDATED OR DELETED",
"default_field" : "status",
"fields" : [ ],
"type" : "best_fields",
"default_operator" : "or",
"max_determinized_states" : 10000,
"enable_position_increments" : true,
"fuzziness" : "AUTO",
"fuzzy_prefix_length" : 0,
"fuzzy_max_expansions" : 50,
"phrase_slop" : 0,
"escape" : false,
"auto_generate_synonyms_phrase_query" : true,
"fuzzy_transpositions" : true,
"boost" : 1.0
}
}
I have the following documents in one collection named as mail_test. Some of them have a tags field which is an array:
/* 1 */
{
"_id" : ObjectId("601a7c3a57c6eb4c1efb84ff"),
"email" : "aaaa#bbb.com",
"content" : "11111"
}
/* 2 */
{
"_id" : ObjectId("601a7c5057c6eb4c1efb8590"),
"email" : "aaaa#bbb.com",
"content" : "22222"
}
/* 3 */
{
"_id" : ObjectId("601a7c6d57c6eb4c1efb8675"),
"email" : "aaaa#bbb.com",
"content" : "33333",
"tags" : [
"x"
]
}
/* 4 */
{
"_id" : ObjectId("601a7c8157c6eb4c1efb86f4"),
"email" : "aaaa#bbb.com",
"content" : "4444",
"tags" : [
"yyy",
"zzz"
]
}
There are two documents with non-empty-tags, so I want the result to be 2.
I use the the following statement to aggregate and get the correct tag_count:
db.getCollection('mail_test').aggregate([{$group:{
"_id":null,
"all_count":{$sum:1},
"tag_count":{"$sum":{$cond: [ { $ne: ["$tags", undefined] }, 1, 0]}}
//if replace `undefined` with `null`, I got the tag_count as 4, that is not what I want
//I also have tried `$exists`, but it cannot be used here.
}}])
and the result is:
{
"_id" : null,
"all_count" : 4.0,
"tag_count" : 2.0
}
and I use spring data mongo in java to do this:
private void test(){
Aggregation agg = Aggregation.newAggregation(
Aggregation.match(new Criteria()),//some condition here
Aggregation.group(Fields.fields()).sum(ConditionalOperators.when(Criteria.where("tags").ne(null)).then(1).otherwise(0)).as("tag_count")
//I need an `undefined` instead of `null`,or is there are any other solution?
);
AggregationResults<MailTestGroupResult> results = mongoTemplate.aggregate(agg, MailTest.class, MailTestGroupResult.class);
List<MailTestGroupResult> mappedResults = results.getMappedResults();
int tag_count = mappedResults.get(0).getTag_count();
System.out.println(tag_count);//get 4,wrong
}
I need an undefined instead of null but I don't know how to do this,or is there are any other solution?
You can use Aggregation operators to check if the field tags exists or not with one of the following constructs in the $group stage of your query (to calculate the tag_count value):
"tag_count":{ "$sum": { $cond: [ { $gt: [ { $size: { $ifNull: ["$tags", [] ] }}, 0 ] }, 1, 0] }}
// - OR -
"tag_count":{ "$sum": { $cond: [ $eq: [ { $type: "$tags" }, "array" ] }, 1, 0] }
Both, return the same result (as you had posted).
I have a question about how to retrieve data from a mongodb and how to store it in my Java code. I select all the documents (there are only 5) from my mongodb and then store its content in my Java application.
So my code is as follows:
public class MongoReader {
private Invoice invoice;
public void mongoReader () {
MongoClientURI uri = new MongoClientURI("my-url");
try (MongoClient mongoClient = new MongoClient(uri)) {
MongoDatabase database = mongoClient.getDatabase("BiFiBEP02");
MongoCollection<Document> mongoCollection = database.getCollection("bifi");
FindIterable<Document> documents = mongoCollection.find();
for (Document document : documents){
invoice.setCustomerId(document.getInteger("customerId"));
invoice.setDate(document.getDate("date"));
invoice.setInvoiceId(document.getInteger("invoiceId"));
invoice.setInvoiceLines(document.getList("invoiceLines", ArrayList<InvoiceLine>));
invoice.setNote(document.getString("note"));
invoice.setPersonId(document.getInteger("personId"));
}
}
catch (MongoException mongoException) {
mongoException.printStackTrace();
}
}
}
The document.getList() doesn't seem to be satisfied with my input.
An example of the the mongo document:
"customerId" : 2,
"date" : ISODate("2018-05-16T10:23:40.049Z"),
"invoiceId" : 1,
"invoiceLines" : [
{
"btwCode" : "hoog",
"productId" : 1,
"productName" : "BiFi worstjes voordeelstrip",
"quantity" : 20,
"totalPrice" : 30,
"unit" : "kg"
},
{
"btwCode" : "hoog",
"productId" : 2,
"productName" : "BiFi worstjes kip",
"quantity" : 20,
"totalPrice" : 30,
"unit" : "kg"
},
{
"btwCode" : "laag",
"productId" : 3,
"productName" : "BiFi worstjes extra scherp",
"quantity" : 30,
"totalPrice" : 100.22,
"unit" : "kg"
},
{
"btwCode" : "geen",
"productId" : 1,
"productName" : "BiFi worstjes promotiestand",
"quantity" : -1,
"totalPrice" : 30.32,
"unit" : "kg"
}
],
"note" : "This invoice is very important!",
"personId" : 2
So my question here is: I have to put the invoiceLines in arraylist in my Java object, but I cannot get the array items out of the mongoDb. How do I do this?
I am using mongodb as a document oriented database, and spring data as the ODM with it. I am facing hard time, performing a max aggregation on complex bson structure.
I have to find the max date, from all documents but if the document has an embedded document, it has to consider that embedded document for the max date.
Here is an example, lets suppose i have a collection name person and person collection contains following documents.
{
"_id" : ObjectId("55def1ceb5b5ed74ddf2b5ce"),
"name" : "abc",
"birth_date_time" : '15 June 1988'
"children" : {
"_id" : ObjectId("55def1ceb2223ed74ddf2b5ce"),
"name" : "def",
"birth_date_time" : '10 April 2010'
}
},
{
"_id" : ObjectId("55def1ceb5b5ed74dd232323"),
"name" : "xyz",
"birth_date_time" : '15 June 1986'
},
{
"_id" : ObjectId("55def1ceb5b5ed74ddf2b5ce"),
"name" : "mno",
"birth_date_time" : '18 March 1982'
"children" : {
"_id" : ObjectId("534ef1ceb2223ed74ddf2b5ce"),
"name" : "pqr",
"birth_date_time" : '10 April 2009'
}
}
It should return 10 April 2010 as this the max birth date for a person in the collection person. I want to know who to achieve it using spring data repository.
Here are the MongoDB aggregations. They should be easily implemented in Spring Data.
db.person.aggregate([
{$group: {
_id: null,
maxDate: {$max : {
$cond: [
{$gt : ["$birth_date_time","$children.birth_date_time"]},
"$birth_date_time",
"$children.birth_date_time"
]}}
}}
])
or using a $project:
db.person.aggregate([{
$project: {
mDate: {
$cond: [
{$gt : ["$birth_date_time","$children.birth_date_time"]},
"$birth_date_time",
"$children.birth_date_time"
]
}
}},
{$group: {
_id: null,
maxDate: {$max : "$mDate"}
}},
])