I'm testing out spring-data and it's mongodb support.
I have a question about the query creation when using or-queries. Consider the following:
Query query = new Query().or(new Query(where("receiverId").is(userId)), new Query(where("requesterId").is(userId)));
query.and(where("status").is(status));
This will result in the following mongodb query:
"$or" : [ { "receiverId" : { "$oid" : "4d78696025d0d46b42d9c579"}} , { "requesterId" : { "$oid" : "4d78696025d0d46b42d9c579"}}] , "status" : "REQUESTED"}
This returns zero results while one is expected. Running this query in mongodb command results in following error:
error: { "$err" : "invalid operator: $oid", "code" : 10068 }
Modifying the query and running it in mongodb command works fine:
{ "$or" : [ { "receiverId" : ObjectId("4d78696025d0d46b42d9c579")} , { "requesterId" : ObjectId("4d78696025d0d46b42d9c579")}] , "status" : "REQUESTED"}
Notice the use of ObjectId("...") instead of $oid.
Am I going about something the wrong way? Maybe setting up the query wrong?
Are you inspecting that query variable at runtime or is that what you are seeing in MongoDB's logs?
Int he C# driver, if you inspect the query variable, you see $oid as well, but that is not the actual query that is sent to the server. At some point, it changes that to a valid MongoDB query.
If you are running on linux, you may want to start up mongosniff which will show you realtime queries, updates and inserts as they happen. If you are on Windows, you should start up mongod.exe with -vvvv flag which will enable it to log every query, update, insert, or command to the log file.
Then you can actually see the exact query that is being submitted.
Related
I've setup full text search and MongoDB and it's working quite well (Mongo 2.6.5).
However it does an OR instead of and AND.
1) Is it possible to make the query an AND query, while still getting all the benefits of full text search (stemming etc.)
2) And if so, is it possible to add this option via the Morphia wrapper library
EDIT
I see that the full text search includes a 'score' for each document returned. Is it possible to only return docs with a certain score or above. Is there some score that would represent a 'fuzzy' and query. That is usually all tokens are in the document but not absolutely always. If so this would solve the problem as well.
Naturally if possible to do this via Morphia that would be super helpful. But I can use the native java driver as well.
Any pointers in the correct direction, much appreciated.
EDIT
Code looks like this, I'm using Morphia 1.0.1:
Datastore ds = Dao.instance().getDatabase();
Query<Product> q = ds.createQuery(Product.class).search("grey vests");
List<Product> prods = q.asList();
Printing the query gives:
{ "$text" : { "$search" : "grey vests"}}
Note: I am able to do take an intersection of multiple result sets to create an AND query. However this is very slow since something like "grey" will return a massive result set and be slow at feeding the results back.
EDIT
I've tried to chain the search() calls and add a single 'token' to each call. But I am getting a run time error. Code becomes:
q.search("grey").search("vests");
The query I get is (which seems like it's doing the right thing) ...
{ "$and" : [ { "$text" : { "$search" : "grey"}} , { "$text" : { "$search" : "vests"}}]}
The error is:
com.mongodb.MongoQueryException: Query failed with error code 17287 and error message 'Can't canonicalize query: BadValue Too many text expressions' on server ...
at com.mongodb.connection.ProtocolHelper.getQueryFailureException(ProtocolHelper.java:93)
I'm using spring data mongodb in my application. It uses mongodb 2.6. I want to query documents of a collection by the id which mongo assigns during insertion. I'm doing something like this:
Query query = new Query();
String id = "542385a91f00bf7dbeae1fc7";
query.addCriteria(Criteria.where("_id").new Object(id));
template.find(query, entity.class);
This query translates to:
{ "_id" : { "$oid" : "542385a91f00bf7dbeae1fc7"}}
When I execute the same on mongo shell, it gives an error:
error: {
"$err" : "Can't canonicalize query: BadValue unknown operator: $oid",
"code" : 17287
}
How do I query by id using spring data mongodb?
It should read
new ObjectId(id)
instead of new Object(id). Please see the API docs for details.
I've ran this query and it worked well.
curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/test",
"user" : "myaccount",
"password" : "myaccount",
"sql" : "select * from orders"
}
}'
Everything seems to be indexed. However, when I changed a data from the Orders Table, the changes did not reflect the document in ElasticSearch. Is it possible to automatically sync updated/changed data?
You need to add another parameter for schedule to tell jdbc-river to pull data periodically.
Here is a reference to this.
I'm using elastic search on Windows 7. I have a trouble with load river-jdbc to sync elasticsearch with mysql. I try loading all river-jdbc river but the result is :
Execution Error
[java.lang.NoClassDefFoundError:org/elasticsearch/rest/XcontentThrowableRestResponse], NoClassDefFoundError[org/elasticsearch/rest/XcontentThrowableRestResponse], ClassNotFoundExeption[org/elasticsearch/rest/XcontentThrowableRestResponse]
P/s: OS: Window 7, elasticsearch 1.2.1, mysql-connector-java-5.1.25-bin.jar
So I am trying to do a mongodb query using Spring data with limit attached. I have noticed that Spring data doesn't add limit to the query itself, but rather gets all data, performs limit on it on Java side then returns limited result. Is this true or am I doing something wrong in my code here.
Criteria criteria = queryBuilder.getQuery(searchCriteria);
Query query = new Query(criteria);
query.limit(500);
logger.Debug("Query: " + query);
if (query.getQueryObject() != null){
resultSet = (List<T>) _mongoDb.find(query, model.getClass(), _collectionName);
}
The query I see in the log is this:
Query: Query: { "$or" : [ { "PARTY" : { "$elemMatch" : { "PARTY_ID" : "32135"}}} , { "ABBR_NUM" : "6873"} , { "ANN_ABBR_NUM" : "6873"}]}, Fields: null, Sort: null
I don't see a limit of 500 attached to this query. Is there something I have missed? By the way, the fields in the query are part of "searchCriteria" passed in.
Your log statement is just logging the query. Behind the scenes the limit actually gets applied to the cursor, so you're not going to see it using that log statement.
Try using the MongoDB profiler (set profile level 2) then query the system.profile collection and you will see that your limit is actually applied to the query, indicated by the ntoreturn field in the profile record.
We have a user_audit entity(table) in mongodb which stores all the login/logout information.
For example,
"_id" : ObjectId("5228cf0156961de6693b74c0"),
"active" : true,
"fname" : "Tom",
"lastlogin" : ISODate("2013-09-05T18:35:45.608Z"),
"lastloginip" : "0:0:0:0:0:0:0:1",
"lname" : "Bailey",
"lastlogout" : ISODate("2013-09-05T18:36:45.568Z"),
There are thousands of records in this table in production.
Now, the admin wants to look for all the logins on a particular date. i am not able to look for exact match of date because of the "time" information attached to ISODate in the "lastlogin" field. In Java, new Date() had been used to insert this field value.
The issue is the time information keeps changing for logins on a particular day.
Tried,
query.put("lastlogin", new BasicDBObject("$lte", givenDate).append("$gte", givenDate));
Obviously it does not return results because the time part of the date does not match.
The query passed from Java to mongo is:
query={ "lastlogin" : { "$lte" : { "$date" : "2013-09-05T04:00:00.000Z"} , "$gte" : {
"$date" : "2013-09-05T04:00:00.000Z"}}}
[Note: It defaults to 04:00:00.000Z always if we format with MM_dd_yyyy in java, not sure why..]
The issue is we have a lot of records in production database. For fresh records, i can blank out the time portion before inserting from Java. But for existing records, not sure how to handle it. How can i get the records on a given date?
According to mongodb cookbook, you are in the right direction. You may just query for something like
query.put("lastlogin", new BasicDBObject("$lt", givenDatePlusOneDay).append("$gte", givenDateAt00h00min));
which is an interval and not the same date.