For this example from the MongoDB documentation, how do I write the query using MongoTemplate?
db.sales.aggregate(
[
{
$group : {
_id : { month: { $month: "$date" }, day: { $dayOfMonth: "$date" }, year: { $year: "$date" } },
totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } },
averageQuantity: { $avg: "$quantity" },
count: { $sum: 1 }
}
}
]
)
Or in general, how do I group by a calculated field?
You can actually do something like this with "project" first, but to me it's a little counter-intuitive to require a $project stage before hand:
Aggregation agg = newAggregation(
project("quantity")
.andExpression("dayOfMonth(date)").as("day")
.andExpression("month(date)").as("month")
.andExpression("year(date)").as("year")
.andExpression("price * quantity").as("totalAmount"),
group(fields().and("day").and("month").and("year"))
.avg("quantity").as("averavgeQuantity")
.sum("totalAmount").as("totalAmount")
.count().as("count")
);
Like I said, counter-intuitive as you should just be able to declare all of this under $group stage, but the helpers don't seem to work this way. The serialization comes out a bit funny ( wraps the date operator arguments with arrays ) but it does seem to work. But still, this is two pipeline stages rather than one.
What is the problem with this? Well by separating the stages the stages the "project" portion forces the processing of all of the documents in the pipeline in order to get the calculated fields, that means it passes through everything before moving on to the group stage.
The difference in processing time can be clearly seen by running the queries in both forms. With a separate project stage, on my hardware takes three times longer to execute than the query where all fields are calculated during the "group" operation.
So it seems the only present way to construct this properly is by building the pipeline object yourself:
ApplicationContext ctx =
new AnnotationConfigApplicationContext(SpringMongoConfig.class);
MongoOperations mongoOperation = (MongoOperations) ctx.getBean("mongoTemplate");
BasicDBList pipeline = new BasicDBList();
String[] multiplier = { "$price", "$quantity" };
pipeline.add(
new BasicDBObject("$group",
new BasicDBObject("_id",
new BasicDBObject("month", new BasicDBObject("$month", "$date"))
.append("day", new BasicDBObject("$dayOfMonth", "$date"))
.append("year", new BasicDBObject("$year", "$date"))
)
.append("totalPrice", new BasicDBObject(
"$sum", new BasicDBObject(
"$multiply", multiplier
)
))
.append("averageQuantity", new BasicDBObject("$avg", "$quantity"))
.append("count",new BasicDBObject("$sum",1))
)
);
BasicDBObject aggregation = new BasicDBObject("aggregate","collection")
.append("pipeline",pipeline);
System.out.println(aggregation);
CommandResult commandResult = mongoOperation.executeCommand(aggregation);
Or if all of that seems to terse to you, then you can always work with the JSON source and parse that. But of course, it has to be valid JSON:
String json = "[" +
"{ \"$group\": { "+
"\"_id\": { " +
"\"month\": { \"$month\": \"$date\" }, " +
"\"day\": { \"$dayOfMonth\":\"$date\" }, " +
"\"year\": { \"$year\": \"$date\" } " +
"}, " +
"\"totalPrice\": { \"$sum\": { \"$multiply\": [ \"$price\", \"$quantity\" ] } }, " +
"\"averageQuantity\": { \"$avg\": \"$quantity\" }, " +
"\"count\": { \"$sum\": 1 } " +
"}}" +
"]";
BasicDBList pipeline = (BasicDBList)com.mongodb.util.JSON.parse(json);
Another alternative is to use a custom aggregation operation class defined like so:
public class CustomAggregationOperation implements AggregationOperation {
private DBObject operation;
public CustomAggregationOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
Then implement it in the pipeline like so:
Aggregation aggregation = newAggregation(
new CustomAggregationOperation(
new BasicDBObject(
"$group",
new BasicDBObject("_id",
new BasicDBObject("day", new BasicDBObject("$dayOfMonth", "$date" ))
.append("month", new BasicDBObject("$month", "$date"))
.append("year", new BasicDBObject("$year", "$date"))
)
.append("totalPrice", new BasicDBObject(
"$sum", new BasicDBObject(
"$multiply", Arrays.asList("$price","$quantity")
)
))
.append("averageQuantity", new BasicDBObject("$avg", "$quantity"))
.append("count",new BasicDBObject("$sum",1))
)
)
)
So it's basically just an interface that is consistent with that used by the existing pipeline helpers, but instead of using other helper methods it directly takes a DBObject definition to return in pipeline construction.
Related
I trying the following query in spring onto the template, that query works in Mongoserver, but when I using the query in the SpringMongo template, the filter does not work, it gets all date, instead of filters date. If anyone knows this issue. pls, help me out this.
The Native Mongo Query:-
db.Task_Status.aggregate([{
{"$addFields":{"timeDiff": { $floor: { $divide: [ { $subtract:[ new Date() , "$currenttime" ] } , 60000 ] } } }},
{$match:{$and:[{"timeDiff":{"$lt": 12}}]}} }]);
The Spring MongoTempalte java Code:
String nativeFilterQuery = "{\"$addFields\":{\"timeDiff\": { $floor: { $divide: [ { $subtract:[ new Date() , \"$currenttime\" ] } , 60000 ] } } }},{$match:{$and:[{\"timeDiff\":{\"$lt\": 12}}]}}";
List<TaskStatus> amTaskLists = Collections.EMPTY_LIST;
List<AggregationOperation> aggregationOperations = new ArrayList<>();
if (StringUtility.isNotNullOrEmpty(nativeFilterQuery)) {
aggregationOperations.add(new CustomProjectAggregationOperation(nativeFilterQuery));
}
TypedAggregation<TaskStatus> aggregation = Aggregation.newAggregation(TaskStatus.class, aggregationOperations);
AggregationResults result = execute(aggregation, TaskStatus.class);
amTaskLists = result.getMappedResults();
When I split the query and Run like following its works
String queryOne = "{\"$addFields\":{\"timeDiff\": { $floor: { $divide: [ { $subtract:[ new Date() , \"$currenttime\" ] } , 60000 ] } } }}";
String queryTwo = "{$match:{$and:[{\"timeDiff\":{\"$lt\": 12}}]}}";
List<AggregationOperation> aggregationOperations = new ArrayList<>();
if (StringUtility.isNotNullOrEmpty(queryOne)) {
aggregationOperations.add(new CustomProjectAggregationOperation(queryOne));
}
if (StringUtility.isNotNullOrEmpty(queryTwo)) {
aggregationOperations.add(new CustomProjectAggregationOperation(queryTwo));
}
You can use AggregationOperation for performing different aggregation in springmongo,but $addFields not support till 3.2 version for that use DBObject for creating query.
For example i used addFields to add score in textcriria my AggregationOperation is:
AggregationOperation score = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
return new BasicDBObject(
"$addFields",
new BasicDBObject("score", new BasicDBObject("$meta","textScore"))
);
}
};
So you can use in your way to find timeDiff and also create match function and you can use like:
List<AggregationOperation> dataListopeartion = new ArrayList<>();
MatchOperation match = new MatchOperation(new Criteria().andOperator(Criteria.where("timeDiff").lte(12));
dataListopeartion.add(match);
dataListopeartion.add(score); //your timeDiff query with AggregationOperation
Aggregation agg = Aggregation.newAggregation(dataListopeartion);
AggregationResults<TaskStatus> groupResults = mongoTemplate.aggregate(agg, Task_Status,TaskStatus.class);
ListMongoWrapper<TaskStatus> mongoWrapper = groupResults.getMappedResults();
You can also refer following answer of SO for better understanding:
link1, link2
I wrote this command in mongodb:
db.getCollection('bnaTask').update(
{"_id":ObjectId("5a2d21823be8c34e903245b7") },
{ $pull: { "status.events": { "event": { $ne: "mybnaCsvUploaderExecuted" } , "time": { $ne: "2017-12-10T11:58:53.543Z" } } } },
{ multi: true }
)
and I want to know how to write it in java, thanks.
Everything is Object in java. you are just constructing BSON objects to match their respective JSON counterparts as used in the segment.
BasicDBObject query = new BasicDBObject("_id", 73);
BasicDBObject event = new BasicDBObject("event", new BasicDBObject( "$ne",
"mybnaCsvUploaderExecuted");
BasicDBObject time = new BasicDBObject("time",
new BasicDBObject( "$ne","2017-12-10T11:58:53.543Z"));
Map<String,Object> map=new HashMap<String,Object>();
map.put("event",event);
map.put("time",time);
BasicDBObject fields = new BasicDBObject("status.events", map);
BasicDBObject update = new BasicDBObject("$pull",fields);
BasicDBObject multiple = new BasicDBObject("multi",true);
getCollection('bnaTask').update( query, update ,multiple);
I have a collection containing contacts and each contact document has a firstName and lastName attribute.
Now I want to query the database by using Java and the MongoDb Java driver in version 3.2.
I try to find a contact with a concatenated firstName + lastName. My query looks like the following for the MongoDb shell:
db.getCollection('contacts').aggregate(
{
$project:{
fullName:{
$concat: [ "$firstName", " ", "$lastName" ]
}
}
},
{
$match:{
fullName:"John Doe"
}
}
);
Now I tried to get my head around the MongoDb Java driver to get the same accomplished in Java:
AggregateIterable<Document> documents = contactUserCollection.aggregate(Arrays.asList(project(computed("fullName", "$firstName $lastName")), match(eq("fullName", firstLastName))));
But this isn't working.
Does someone have an idea how I could accomplish this?
Thank you
You could try the following:
/*
MONGO SHELL :
var pipeline = [
{
"$project": {
"otherfieldsA": 1,
"otherfieldsB": 1,
"otherfieldsC": 1,
"fullName": {
"$concat": [ "$fistName", " ", "$lastName" ]
}
}
}
{
"$match": { "fullName": "John Doe" }
}
];
db.contacts.aggregate(pipeline);
*/
public class JavaAggregation {
public static void main(String args[]) throws UnknownHostException {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("test");
DBCollection coll = db.getCollection("contacts");
// create the pipeline operations, build the $project operations
BasicDBList concatenate = new BasicDBList();
concatenate.add("$firstName");
concatenate.add(" ");
concatenate.add("$lastName");
DBObject fullName = new BasicDBObject("$concat", concatenate);
DBObject fields = new BasicDBObject("otherfieldsA", 1);
fields.put("otherfieldsB", 1);
fields.put("otherfieldsC", 1);
fields.put("fullName", fullName);
DBObject project = new BasicDBObject("$project", fields);
// create the $match operator
DBObject match = new BasicDBObject("$match",
new BasicDBObject("fullName", "John Doe")
);
AggregationOutput documents = coll.aggregate(match, project, group, sort);
for (DBObject result : documents.results()) {
System.out.println(result);
}
}
}
I have values that coming from MongoDB stored in a DBObject. and I needed to store that value in a set one by one. As being new to MongoDB I am not actually getting idea coherently how to proceed that.
String date = sdf.format(cal2.getTime());
List<String> dateList = new ArrayList<String>();
for (int i = 0; i < 6; i++) {
Date dateParsed = sdf.parse(date);
dateParsed.setDate(dateParsed.getDate() - i);
dateList.add(sdf.format(dateParsed));
}
Set<String> values2= new HashSet<String>();
for (String str : dateList) {
BasicDBObject find1 = new BasicDBObject("_ky", str);
DBObject values1= someDB.findOne(find1);
Iterator iter = values1.iterator(); /*giving error the method not found (becasue values1 is a dbObject)*/
while (iter.hasNext()) {
values2.add(//???//);
}
}
Any help on how can I iterate the DBObject- values1 and add those values in a set- values2 would be great.
You can call values1.keySet() and iterate over that and get() any values or use values1.toMap() and iterate that Map like you would any other.
The primary abstraction in the Mongo Java Driver is the DBObject which acts like a wrapper around Java's Map<String,Object>.
import java.util.Arrays;
import com.mongodb.BasicDBList;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
public class DBObjectKeySetDemo {
public static void main( String[] args ) {
DBObject dbo = new BasicDBObject( "firstName", "Robert" ).append( "lastName", "Kuhar" );
dbo.put( "age", 49 );
dbo.put( "hobbies", Arrays.asList( "Fly Fishing", "Board Games", "Roller Derby" ) );
DBObject browser = new BasicDBObject( "implementation", "Chrome" );
browser.put( "vendor", "Google" );
BasicDBList bookmarks = new BasicDBList();
bookmarks.add( new BasicDBObject( "name", "StackOverflow" ).append( "URL", "http://stackoverflow.com" ) );
bookmarks.add( new BasicDBObject( "name", "MMS" ).append( "URL", "https://mms.mongodb.com" ) );
browser.put( "bookmarks", bookmarks );
dbo.put( "browser", browser );
for ( String key : dbo.keySet() ) {
System.out.println( "key: " + key + " value: " + dbo.get( key ) );
}
System.out.println( "dbo: " + dbo );
}
}
The "gotcha" is that you can only work directly with "top level" element. For example, in the above example, through the Java API, you have no way to directly reference "browser.vendor". Through the Java API you would have to first get the "browser" sub-document, and then get the "vendor" field.
Clear? As mud? It helped me to just think of the abstraction as a Map<String,Object> where Object, in the case of a sub-document, may itself be a Map<String,Object>.
I need to implement "Popular Keyword" in my web app so that most searched keywords will show there.. For that i have created a db in mongo. I need to retrieve data from mongo so that mostly occuring keywords should sort and should return to my page. In sql it is just like as,
select names,count(names) from keyword_db group by names order by names;
need both java code and mongo shell query..
You might want to try for the mongo shell query (I'm sorting in descending order:
db.keyword_db.aggregate ( [
{ $group: { _id: "$names", count: { $sum: 1 } } },
{ $sort: { count: -1 } } ] )
For the Java version:
DBCollection myColl = db.getCollection("keyword_db");
// for the $group operator
DBObject groupFields = new BasicDBObject( "_id", "$names");
groupFields.put("count", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields );
// for the $sort operator
DBObject sortFields = new BasicDBObject("count", -1);
DBObject sort = new BasicDBObject("$sort", sortFields );
// run the aggregation
AggregationOutput output = myColl.aggregate(group, sort);
System.out.println( output.getCommandResult() );
The following page may help you also:
http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/
For sorting you can use following..1 for ascending and -1 for desc.
db.keyword_db.find( { names: "A" } ).sort( { user_id: 1 } )
and for groupBy You can use Aggregation framework..
http://docs.mongodb.org/manual/reference/method/db.collection.group/