{ "_id" : { "$oid" : "52568424036439f2c5107571"} , "start" : { "x:" : 71 , "y:" : 9} , "end" : { "x:" : 30 , "y:" : 84}}
{ "_id" : { "$oid" : "52568424036439f2c5107572"} , "start" : { "x:" : 28 , "y:" : 59} , "end" : { "x:" : 72 , "y:" : 64}}
{ "_id" : { "$oid" : "52568424036439f2c5107573"} , "start" : { "x:" : 16 , "y:" : 71} , "end" : { "x:" : 18 , "y:" : 79}}
Need all docs say where start.x > 40
Query tried mongoDb console :
db.lines.find({"start.x": {$gt:40} })
Java Driver:
DBObject query = QueryBuilder.start("start.x").greaterThan(50).get();
collection.find(query)
Both cases no error, no documents retrieved.
You named your field x: not x so this should do the work:
db.lines.find({"start.x:": {$gt:40} })
Haven't used Mongo with Java, but... have you tried using QueryBuilder.get instead of QueryBuild.start? That's the method used for find operations.
You can also use a more specific one for this case: QueryBuild.greaterThan
Related
I have a collection with a dataset that looks like:
{
"resource" : "10.10.10.10",
"statistics" : {
"connections" : 17
}
}
{
"resource" : "10.10.10.10",
"statistics" : {
"connections" : 24
}
}
I want to use Mongo's $group/$push mechanism to return a dataset that looks like:
{ "_id" : "10.10.10.10", "statTotals" : [ 17, 24 ] }
In Mongo shell, I can that by doing:
db.testcol.aggregate([ { "$project" : { "resource" : 1 , "total" : "$statistics.connections"}} , { "$group" : { "_id" : "$resource" , "statTotals" : { "$push" : "$total"}}} ])
Now I want to do this using Spring's Mongo data solution in Java. The operations I'm currently trying to use are:
ProjectionOperation projOper = Aggregation.project("resource").and("statistics.connections").as("total");
GroupOperation groupOper = Aggregation.group("resource").push("total").as("statTotals");
Unfortunately, this is generating a pipeline which looks like:
{ "aggregate" : "event" , "pipeline" : [ { "$project" : { "resource" : 1 , "total" : "$statistics.connections"}} , { "$group" : { "_id" : "$resource" , "statTotals" : { "$push" : "$statistics.connections"}}}]}
In the $group, the $push is being done against $statistics.connections instead of $total so the results come back blank.
Any help would be greatly appreciated on this. At first I thought this might be https://jira.spring.io/browse/DATAMONGO-1254 but I've tried using both 1.7.2 as well as 1.8.1 and get the same results.
I am trying to sort cursor by two fields "start" and "end". Both of them have indexes.
This is the code attempting to sort.
DBCursor cursor = store.colConcepts.find(q);
cursor.addOption(Bytes.QUERYOPTION_NOTIMEOUT);
BasicDBObject sortObj = new BasicDBObject( "start", filter.isEventTimeSortDirAscending() ? 1 : -1 ).append( "end", filter.isEventTimeSortDirAscending() ? 1 : -1 );
cursor = cursor.sort( sortObj );
In above code query q is { "tags" : { "$all" : [ "Person"]}}
And following are the indices on collection store.colConcepts.
colConcepts.ensureIndex(new BasicDBObject("tags", 1));
colConcepts.ensureIndex(new BasicDBObject("roles.concept",1));
colConcepts.ensureIndex(new BasicDBObject("keys",1));
colConcepts.ensureIndex(new BasicDBObject("start", 1));
colConcepts.ensureIndex(new BasicDBObject("end", 1));
Following is the result of cursor.explain().
{ "cursor" : "BtreeCursor tags_1" , "isMultiKey" : true , "n" : 237267 , "nscannedObjects" : 237267 , "nscanned" : 237267 , "nscannedObjectsAllPlans" : 237267 , "nscannedAllPlans" :
237267 , "scanAndOrder" : false , "indexOnly" : false , "nYields" : 1853 , "nChunkSkips" : 0 , "millis" : 274 , "indexBounds" : { "tags" : [ [ "Person" , "Person"]]} , "allPlans" : [
{ "cursor" : "BtreeCursor tags_1" , "isMultiKey" : true , "n" : 237267 , "nscannedObjects" : 237267 , "nscanned" : 237267 , "scanAndOrder" : false , "indexOnly" : false ,
"nChunkSkips" : 0 , "indexBounds" : { "tags" : [ [ "Person" , "Person"]]}}] , "server" : "xxx:27017" , "filterSet" : false , "stats" : { "type" : "FETCH" , "works" : 237269 ,
"yields" : 1853 , "unyields" : 1853 , "invalidates" : 0 , "advanced" : 237267 , "needTime" : 1 , "needFetch" : 0 , "isEOF" : 1 , "alreadyHasObj" : 0 , "forcedFetches" : 0 ,
"matchTested" : 0 , "children" : [ { "type" : "IXSCAN" , "works" : 237268 , "yields" : 1853 , "unyields" : 1853 , "invalidates" : 0 , "advanced" : 237267 , "needTime" : 1 ,
"needFetch" : 0 , "isEOF" : 1 , "keyPattern" : "{ tags: 1 }" , "isMultiKey" : 1 , "boundsVerbose" : "field #0['tags']: [\"Person\", \"Person\"]" , "yieldMovedCursor" : 0 ,
"dupsTested" : 237267 , "dupsDropped" : 0 , "seenInvalidated" : 0 , "matchTested" : 0 , "keysExamined" : 237267 , "children" : [ ]}]}}
As you can see tags,start, end all of them have indices.
Upon execution it is producing the exception :
com.mongodb.MongoException: Runner error: Overflow sort stage buffered data usage of 33554442 bytes exceeds internal limit of 33554432 bytes
I did some research on the issue and found that this problem can come up if you have no index on the field. or if the fields are indexed as sparse which is not the case in situation I have.
I am using mongodb 2.6.1. I did run the code with 2.6.4 but that didn't stop mongo from throwing exception.
Any idea how this can be solved?
You don't have the right index for the query. The query planner selected the index on tags to fulfill the query, but that index doesn't help with the sort. Since you want to select based on tags and then sort on (start, end), try putting an index on { "tags" : 1, "start" : 1, "end" : 1 }. Having indexes on each separately isn't helpful here.
I'm testing the MongoDB Java API and I wanted to do a mapReduce.
I implemented it as follow :
String map = "function() { " +
"emit(this.ts, this.1_bid);}";
String reduce = "function(key, values) {" +
"return Array.sum(values);}";
MapReduceCommand cmd = new MapReduceCommand(collection, map, reduce, null, MapReduceCommand.OutputType.INLINE, null);
MapReduceOutput out = collection.mapReduce(cmd);
for (DBObject o : out.results()) {
System.out.println(o.toString());
}
But when I execute it I have the following exception stack :
[tick_engine] 16:51:53.600 ERROR [MongoTickDataReader] Failed to read data from mongoDB
com.mongodb.CommandFailureException: { "serverUsed" : "/127.0.0.1:27017" , "errmsg" : "exception: SyntaxError: Unexpected token ILLEGAL" , "code" : 16722 , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:71) ~[mongo-2.11.1.jar:na]
at com.mongodb.CommandResult.throwOnError(CommandResult.java:110) ~[mongo-2.11.1.jar:na]
at com.mongodb.DBCollection.mapReduce(DBCollection.java:1265) ~[mongo-2.11.1.jar:na]
at com.smarttrade.tickEngine.in.MongoTickDataReader.mapReduce(MongoTickDataReader.java:321) ~[classes/:na]
at com.smarttrade.tickEngine.in.MongoTickDataReader.readData(MongoTickDataReader.java:157) ~[classes/:na]
at com.smarttrade.tick.engine.TickEngine.onMarketDataRequest(TickEngine.java:203) [classes/:na]
at com.smarttrade.tick.sttp.TickMarketDataRequestCommand.execute(TickMarketDataRequestCommand.java:62) [classes/:na]
at com.smarttrade.st.commands.Command.process(Command.java:140) [src/:na]
at com.smarttrade.st.server.STTPInvoker$1.process(STTPInvoker.java:385) [src/:na]
at com.smarttrade.st.server.STTPInvoker$1.process(STTPInvoker.java:1) [src/:na]
at com.smarttrade.util.concurrent.queue.MultiSessionsBlockingQueue$SimpleSession.run(MultiSessionsBlockingQueue.java:122) [src/:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
The problem seems to be with the attribute name that you have defined - 1_bid
I created sample documents to test your map-reduce -
{ "_id" : ObjectId("533ef7d0e1687dd644410d88"), "ts" : "TSKEY", "1_bid" : 200 }
{ "_id" : ObjectId("533ef7d3e1687dd644410d89"), "ts" : "TSKEY", "1_bid" : 300 }
{ "_id" : ObjectId("533ef7d5e1687dd644410d8a"), "ts" : "TSKEY", "1_bid" : 400 }
{ "_id" : ObjectId("533ef7dce1687dd644410d8b"), "ts" : "TSKEY2", "1_bid" : 800 }
{ "_id" : ObjectId("533ef7dfe1687dd644410d8c"), "ts" : "TSKEY2", "1_bid" : 300 }
I ran following map-reduce command -
db.sample4.mapReduce(function() { emit(this.ts, this.1_bid);},function(key, values) {return Array.sum(values);})
The error that I got is SyntaxError: missing ) after argument list (shell):1
I realized, that the function, that mapper is executing, is a JavaScript function and in Javascript, you cannot have a variable that starts with a number. Hence you get a syntax error. I then created new set of documents -
{ "_id" : ObjectId("533eff29e1687dd644410d8d"), "ts" : "TSKEY", "bid_1" : 200 }
{ "_id" : ObjectId("533eff2de1687dd644410d8e"), "ts" : "TSKEY", "bid_1" : 300 }
{ "_id" : ObjectId("533eff34e1687dd644410d8f"), "ts" : "TSKEY", "bid_1" : 400 }
{ "_id" : ObjectId("533eff7fe1687dd644410d92"), "ts" : "TSKEY2", "bid_1" : 800 }
{ "_id" : ObjectId("533eff85e1687dd644410d93"), "ts" : "TSKEY2", "bid_1" : 300 }
and then modified the mapper to use "bid_1" and ran the following command -
db.sample4.mapReduce(function() { emit(this.ts, this.bid_1);},function(key, values) {return Array.sum(values);},"pivot")
The output was -
{
"result" : "pivot",
"timeMillis" : 61,
"counts" : {
"input" : 12,
"emit" : 12,
"reduce" : 2,
"output" : 2
},
"ok" : 1,
}
db.pivot.find()
{ "_id" : "TSKEY", "value" : 900 }
{ "_id" : "TSKEY2", "value" : 1100 }
I tested this in Java using the same program that you have pasted and just changed the attribute name to "bid_1" and it worked
To prevent syntax errors on field names, you can also write the map function this way :
function() {
emit(this["ts"], this["1_bid"]);
}
I am using java and mongoDB. I have the following json,
{ "_id" : { "$oid" : "524a27a318c533dc95edafe1"} , "RoomNumber" : 516 , "RoomType" : "presidential" , "Reserved" : true , "RegularRate" : 400.0 , "Discount" : [ 0.85 , 0.75 , 1.0 , 1.0] , "DiscountedRate" : 0}
{ "_id" : { "$oid" : "524a27a318c533dc95edafe2"} , "RoomNumber" : 602 , "RoomType" : "presidential" , "Reserved" : false , "RegularRate" : 500.0 , "Discount" : [ 1 , 0.75 , 1.0 , 1.0] , "DiscountedRate" : 0}
{ "_id" : { "$oid" : "524a27a318c533dc95edafe3"} , "RoomNumber" : 1315 , "RoomType" : "Single" , "Reserved" : true , "RegularRate" : 100.0 , "Discount" : [ 1 , 1 , 1.0 , 1.0] , "DiscountedRate" : 0}
In the collection the documents have different room numbers. If I know a room number how would i get a document with that room number and then get all ther other values in that document.
for exmaple if I have 602, i want to be able to get roomtype: presidential, reserved: false, Regular rate: 500
Thanks
The shell query for that would be:
db.rooms.findOne({RoomNumber: 602}, {RoomType: 1, Reserved: 1, RegularRate:1, _id: 0});
And using the native mongo driver:
BasicDBObject query = new BasicDBObject("RoomNumber", 602);
BasicDBObject projection = new BasicDBObject("RoomType", 1);
projection
.append("Reserved", 1)
.append("RegularRate", 1)
.append("_id", 0);
DBObject result = collection.findOne(query, projection);
I am new to java. How can I build this mongo query in java. Any help/hint will be appreciated.
db.places.find({loc : { $near :[ -122.934326171875,37.795268017578] , $maxDistance : 50 } ,$or:[{"uid":"at"},{"myList.$id" :ObjectId("4fdeaeeede2d298262bb80") } ] ,"searchTag" : { $regex : "Union", $options: "i"} } );
By using the QueryBuilder you can create the query you wanted. I have created it as follows.
QueryBuilder query = new QueryBuilder();
query.put("loc").near(-122.934326171875, 37.795268017578, 50);
query.or(
QueryBuilder.start("uid").is("at").get(),
QueryBuilder.start("myList.$id").is(new ObjectId("5024f2f577a59dd9736ddc38")).get()
);
query.put("searchTag").regex(Pattern.compile("Union",Pattern.CASE_INSENSITIVE));
When I print the query into the console it looks like :
{ "loc" : { "$near" : [ -122.934326171875 , 37.795268017578 , 50.0]} ,
"$or" : [ { "uid" : "at"} , { "myList.$id" : { "$oid" : "5024f2f577a59dd9736ddc38"}}] ,
"searchTag" : { "$regex" : "Union" , "$options" : "i"}
}
I didn't try it but hope it will work.