In mySQL the describe statement can be used to retrieve the schema of a given table, unfortunately I could not locate a similar functionality for the MongoDB java driver :(
Let's say I have I have the flowing BSON documents:
{
_id: {$oid:49},
values: { a:10, b:20}
}
,
{
_id: {$oid:50},
values: { b:21, c:31}
}
Now let's suppose I do:
DBObject obj = cursor.next();
DBObject values_1 = (DBObject) obj.get("values");
and the schema should be something like this:
>a : int
>b : int
and for the next document:
DBObject obj = cursor.next();
DBObject values_2 = (DBObject) obj.get("values");
the schema should be:
>b : int
>c : int
Now that I explained what the schema retrial is, can some1 be nice and tell me how to do it?
If it helps, In ma case I only need to know the field names (because the datatype is always the same, but it would be nice also know how to retrieve the datatypes).
A work arround, is convert the DBObject to a Map, then the Map to a Set, the Set to an Iterator and extract the attribute names/values... Still have now idea how to extract data types.
this is:
DBObject values_1 = (DBObject) obj.get("values");
Map _map = values_1.toMap();
// Set set = _map.entrySet(); // if you want the <key, value> pairs
Set _set_keys = _map.keySet();
Iterator _iterator = _set_keys.iterator();
while (_iterator.hasNext())
System.out.println("-> " + _iterator.next());
A DBObject has a method called keySet (documentation). You shouldn't need to convert to a Map first.
There's no exposed method to determine the underlying BSON data type at this point, so you'd need to investigate using instanceof or getClass to determine the underlying data type of the Object that is returned from get.
If you look at the source code for BasicBSONObject for example, you'll see how the helper functions that cast do some basic checks and then force the cast.
Related
I have just started using Mongo Db . Below is my data structure .
It has an array of skillID's , each of which have an array of activeCampaigns and each activeCampaign has an array of callsByTimeZone.
What I am looking for in SQL terms is :
Select activeCampaigns.callsByTimeZone.label,
activeCampaigns.callsByTimeZone.loaded
from X
where skillID=50296 and activeCampaigns.campaign_id= 11371940
and activeCampaigns.callsByTimeZone='PT'
The output what I am expecting is to get
{"label":"PT", "loaded":1 }
The Command I used is
db.cd.find({ "skillID" : 50296 , "activeCampaigns.campaignId" : 11371940,
"activeCampaigns.callsByTimeZone.label" :"PT" },
{ "activeCampaigns.callsByTimeZone.label" : 1 ,
"activeCampaigns.callsByTimeZone.loaded" : 1 ,"_id" : 0})
The output what I am getting is everything under activeCampaigns.callsByTimeZone while I am expecting just for PT
DataStructure :
{
"skillID":50296,
"clientID":7419,
"voiceID":1,
"otherResults":7,
"activeCampaigns":
[{
"campaignId":11371940,
"campaignFileName":"Aaron.name.121.csv",
"loaded":259,
"callsByTimeZone":
[{
"label":"CT",
"loaded":6
},
{
"label":"ET",
"loaded":241
},
{
"label":"PT",
"loaded":1
}]
}]
}
I tried the same in Java.
QueryBuilder query = QueryBuilder.start().and("skillID").is(50296)
.and("activeCampaigns.campaignId").is(11371940)
.and("activeCampaigns.callsByTimeZone.label").is("PT");
BasicDBObject fields = new BasicDBObject("activeCampaigns.callsByTimeZone.label",1)
.append("activeCampaigns.callsByTimeZone.loaded",1).append("_id", 0);
DBCursor cursor = coll.find(query.get(), fields);
String campaignJson = null;
while(cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
campaignJson = campaignDBO.toString();
System.out.println(campaignJson);
}
the value obtained is everything under callsByTimeZone array. I am currently parsing the JSON obtained and getting only PT values . Is there a way to just query the PT fields inside activeCampaigns.callsByTimeZone .
Thanks in advance .Sorry if this question has already been raised in the forum, I have searched a lot and failed to find a proper solution.
Thanks in advance.
There are several ways of doing it, but you should not be using String manipulation (i.e. indexOf), the performance could be horrible.
The results in the cursor are nested Maps, representing the document in the database - a Map is a good Java-representation of key-value pairs. So you can navigate to the place you need in the document, instead of having to parse it as a String. I've tested the following and it works on your test data, but you might need to tweak it if your data is not all exactly like the example:
while (cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
List callsByTimezone = (List) ((DBObject) ((List) campaignDBO.get("activeCampaigns")).get(0)).get("callsByTimeZone");
DBObject valuesThatIWant;
for (Object o : callsByTimezone) {
DBObject call = (DBObject) o;
if (call.get("label").equals("PT")) {
valuesThatIWant = call;
}
}
}
Depending upon your data, you might want to add protection against null values as well.
The thing you were looking for ({"label":"PT", "loaded":1 }) is in the variable valueThatIWant. Note that this, too, is a DBObject, i.e. a Map, so if you want to see what's inside it you need to use get:
valuesThatIWant.get("label"); // will return "PT"
valuesThatIWant.get("loaded"); // will return 1
Because DBObject is effectively a Map of String to Object (i.e. Map<String, Object>) you need to cast the values that come out of it (hence the ugliness in the first bit of code in my answer) - with numbers, it will depend on how the data was loaded into the database, it might come out as an int or as a double:
String theValueOfLabel = (String) valuesThatIWant.get("label"); // will return "PT"
double theValueOfLoaded = (Double) valuesThatIWant.get("loaded"); // will return 1.0
I'd also like to point out the following from my answer:
((List) campaignDBO.get("activeCampaigns")).get(0)
This assumes that "activeCampaigns" is a) a list and in this case b) only has one entry (I'm doing get(0)).
You will also have noticed that the fields values you've set are almost entirely being ignored, and the result is most of the document, not just the fields you asked for. I'm pretty sure you can only define the top-level fields you want the query to return, so your code:
BasicDBObject fields = new BasicDBObject("activeCampaigns.callsByTimeZone.label",1)
.append("activeCampaigns.callsByTimeZone.loaded",1)
.append("_id", 0);
is actually exactly the same as:
BasicDBObject fields = new BasicDBObject("activeCampaigns", 1).append("_id", 0);
I think some of the points that will help you to work with Java & MongoDB are:
When you query the database, it will return you the whole document of
the thing that matches your query, i.e. everything from "skillID"
downwards. If you want to select the fields to return, I think those will only be top-level fields. See the documentation for more detail.
To navigate the results, you need to know that a DBObjects are returned, and that these are effectively a Map<String,
Object> in Java - you can use get to navigate to the correct node,
but you will need to cast the values into the correct shape.
Replacing while loop from your Java code with below seems to give "PT" as output.
`while(cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
campaignJson = campaignDBO.get("activeCampaigns").toString();
int labelInt = campaignJson.indexOf("PT", -1);
String label = campaignJson.substring(labelInt, labelInt+2);
System.out.println(label);
}`
Consider a collection of objects having fields like:
{
id: // String
type: //Integer
score: //Double value
}
I would like to query on collection using type and for returned documents divide their scores by their maximum. Consider following query oject:
DBObject searchQuery = new BasicDBObject("type", 2);
collection.find(searchQuery);
With above query it'll return some documents. I want to get maximum of scores among all those documents and then divide all those documents' score by obtained maximum.
How can I do this??
I could find maximum using aggregation as follows:
String propertyToOperateOn = "score";
DBObject match = new BasicDBObject("$match", searchQuery);
DBObject groups = new BasicDBObject("_id", null);
DBObject operation = new BasicDBObject("$max", "$" + propertyToOperateOn);
groups.put("maximum", operation);
DBObject apply = new BasicDBObject("$group", groups);
AggregationOutput output = mongoConstants.IAScores.aggregate(match, apply);
Here output will contain the maximum value. But then how can I update (divide) all documents' scores by this maximum??
I hope there could be better way to do this task, but I'm unable to get it as I'm very much new to mongodb (or any database as such).
This is technically the same issue as "mongodb: java: How to update a field in MongoDB using expression with existing value", but I'll repeat the answer:
At the moment, MongoDB doesn't allow you to update the value of a field according to an existing value of a field. Which means, you can't do the following SQL:
UPDATE foo SET field1 = field1 / 2;
In MongoDB, you will need to do this in your application, but be aware that this is no longer an atomic operation as you need to read and then write.
I'm trying to get a list of mongo "_ids" from a database using Java. I don't need any other part of the objects in the database, just the "_id".
This is what I'm doing right now:
// Another method queries for all objects of a certain type within the database.
Collection<MyObject> thingies = this.getMyObjects();
Collection<String> ids = new LinkedList<String>();
for (MyObject thingy : thingies) {
ids.add(thingy.getGuid());
}
This seems horribly inefficient though... is there a way just to query mongo for objects of a certain type and return only their "_ids" without having to reassemble the entire object and extract it?
Thanks!
The find() method has an overload where you can pass the keys that you want to retrieve back from the query or those that you don't want.
So you could try this:
BasicDBObject qyery = new BasicDBObject("someKey","someValue");
BasicDBObject keys = new BasicDBObject("_id", 1);
DBCursor cursor = collection.find(query, keys);
I'm very new to MongoDB, and I'm using it along with the Java driver. I have this document structure:
{ "_id" : ObjectId("4f7d2ba6fd5a306d82687d48"), "room" : "Den" }
{ "_id" : ObjectId("4f7d2baafd5a306d82687d49"), "room" : "Foyer" }
{ "_id" : ObjectId("4f7d2fdcfd5a306d82687d4a"), "room" : "Master Bedroom" }
{ "_id" : ObjectId("4f7d301afd5a306d82687d4b"), "room" : "Guest Bedroom" }
{ "_id" : ObjectId("4f7d2b98fd5a306d82687d47"), "code" : "A", "lights" : [ { "name" : "Overhead", "code" : "1" } ], "room" : "Kitchen" }
Where the last line is of particular interest in illustrating what I want to do. Each document is a room and may have a "lights" key corresponding to a value that is an array of sub-documents. From a modeling perspective, I have a house, which has 0-n rooms, each of which has 0-n lights in it. What I want to do in Java is take the name of the room as a parameter, and return a collection of DBObject corresponding to the sub-documents in the lights array -- "get me all lights for room 'kitchen'", for example.
So far, proceeding incrementally in TDD style, I've constructed this query:
public static final String ROOM_KEY = "room";
public static final String EQUALS_KEY = "$eq";
private BasicDBObject buildRoomNameQuery(String roomName) {
BasicDBObject myQuery = new BasicDBObject();
myQuery.put(ROOM_KEY, new BasicDBObject(EQUALS_KEY, roomName));
return myQuery;
}
I realize that this is going to get me the entire room document for the room name I pass in. I'm a bit stuck on what the best way to proceed from here is to get what I want. Is what I'm doing even possible with a simple query, or will I have to retrieve the array and iterate through it in code, casting the elements as DBObject? I'm also open to suggestions for a better document structure for my purpose -- I'm not married to this structure by any means.
For a bit of perspective, I'm quite well versed in SQL and traditional relational databases, if that helps in terms of explanatory analogies. Also, if I'm butchering the MongoDB terminology, please correct me. Thanks in advance.
So, you can do something like this:
DBCollection coll = db.getCollection("test");
BasicDBObject query = new BasicDBObject("room", "Kitchen");
// optional, limit the fields to only have the lights field
BasicDBObject fields = new BasicDBObject("lights",1).append("_id",false);
DBCursor curs = coll.find(query, fields);
while(curs.hasNext()) {
DBObject o = curs.next();
// shows the whole result document
System.out.println(o.toString());
BasicDBList lights = (BasicDBList) o.get("lights");
// shows the lights array -- this is actually a collection of DBObjects
System.out.println(lights.toString());
// optional: break it into a native java array
BasicDBObject[] lightArr = lights.toArray(new BasicDBObject[0]);
for(BasicDBObject dbObj : lightArr) {
// shows each item from the lights array
System.out.println(dbObj);
}
}
Also, I recommend using the QueryBuilder in the Java driver--it's a bit more concise than creating Queries from DBObjects. Even better, check out Morphia, which is an object mapper that uses the Java driver. It natively supports entity models that have lists in them, and serializes/deserializes them to Mongo without needing to deal with the DBObject stuff.
Look at spring mongo package. A really good way to work with mongo using POJO documents
http://www.springsource.org/spring-data/mongodb
You will not need to perform casting and work with strings
You can use an iterator for the fields
Iterator<DBObject> fields = curs.iterator();
while(fields.hasNext()){
DBObject field = (DBObject) fields.next().get("lights");
System.out.println(field.get("name"));
}
For newer versions, consider the use of the Document. To avoid unchecked casts and linter warnings, along with writing your own loop, use the libary's get(final Object key, final Class<T> clazz) method:
List<Document> comments = posts.get("comments", docClazz)
where docClazz is something that you create once:
final static Class<? extends List> docClazz = new ArrayList<Document().getClass();
References:
http://www.mongodb.org/display/DOCS/Java+Tutorial
Still pretty new to mongo db but I'm trying to update part of an existing document inside a collection... unfortunately, the link above doesn't have an update example.
Essentially, i just want to be able to:
Add new fields to a document
Update existing fields of a document
to a new value
Here's my code (Grails + Groovy + Java + MongoDB + the java driver):
def shape = mongo.shapes.findOne(new BasicDBObject("data", "http://www.foo.com")); // get the document
mongo.shapes.update(new BasicDBObject("_id", shape._id), new BasicDBObject("isProcessed", 0)); // add a new "isProcessed" field set to 0
mongo.shapes.update(new BasicDBObject("_id", shape._id), new BasicDBObject("data", "http://www.bar.com"));
This pretty much clobbers the entire object... I might try just modifying the original shape object and then running the update on that. But until then, does anyone have experience updating just individual fields (rather than the entire document)?
EDIT:
I just tried it and was able to successfully update by sending the entire object across with new and/or updated fields and that works. I wonder if the driver is smart enough to only update the smallest subset of changes or if it's just blindly updating the entire thing? (In the case below, is it just updating the foo field across the wire or the entire shape document?)
Code:
def shape = mongo.shapes.findOne(); // get the first shape to use as a base
shape.removeField("_id"); // remove the id field
shape.put("foo","bar"); // add a new field "foo"
mongo.shapes.insert(shape); // insert the new shape
def shape2 = mongo.shapes.findOne(new BasicDBObject("foo", "bar")); // get the newly inserted shape (and more importantly, it's id)
shape2.put("foo", "bat"); // update the "foo" field to a new value
mongo.shapes.update(new BasicDBObject("_id", shape2._id), shape2); // update the existing document in mongo
I wonder if the driver is smart enough to only update the smallest subset of changes or if it's just blindly updating the entire thing?
No, if you use the "normal" update method, the whole object will be sent over the wire.
I suspect that the database server itself will be clever enough to only update the necessary indexes (and not the ones that did not change), if possible (i.e. the object could be updated in place and did not have to be moved because it grew too much)
What you can do is use the "atomic update modifier" functions. The Java documentation is a bit light on them, but since the driver just transmits the JSON, the stuff from the non-Java tutorials should work, for example:
shapes.update((DBObject)JSON.parse( "{ 'foo' : 'bar'}"),
(DBObject) JSON.parse( "{ '$set' : { 'foo': 'bat'}}") );
Found an example here, which seems to show the usage for the update call. So for your example, I believe something like this should work?
// Find an object
def shape2 = mongo.shapes.findOne( new BasicDBObject( 'foo', 'bar' ) )
// And update the foo field from 'bar' to 'bat'
mongo.shapes.update( shape2, new BasicDBObject( '$set', new BasicDBObject( 'foo', 'bat' ) ) )
Edit
You might be able to use a category to construct your BasicDBObjects in a more groovy way...
Something like this might do it:
class BasicDBObjectMapBuilder {
static String toDbObj( String s ) { s }
static BasicDBObject toDbObj( Map m ) {
m.inject( null ) { r, it -> new BasicDBObject( it.key, it.value.toDbObj() ) }
}
}
use( BasicDBObjectMapBuilder ) {
def shape2 = mongo.shapes.findOne( new BasicDBObject( 'foo', 'bar' ) )
// And update the foo field from 'bar' to 'bat'
mongo.shapes.update( shape2, [ '$set':[ 'foo', 'bat' ] ].toDbObj() )
}
I haven't tested this though...
Edit 2
Actually, BasicDBObject is a Map, so you should be able to do:
mongo.shapes.update( shape2, [ '$set':[ 'foo', 'bat' ] ] as BasicDBObject )
without needing the builder
A lot of the answers on this post are using older versions of the Mongo Java Driver. If you're using a newer version of the Java Driver (v3.0+) then the preferred method seems to be to use the Document object instead of the DBObject interface.
Here is an example:
MongoClient client = new MongoClient();
MongoCollection<Document> fooCollection = client.getDatabase("test").getCollection("foo");
Bson filter = Filters.eq("_id", "123d45678c467bb433c99f99");
Bson updates = Updates.set("isFoo", true);
fooCollection.findOneAndUpdate(filter, updates);
This answer uses the mongo shell, but shows how you can go deep into a JSON object structure to modify a specific field without overwriting the rest.
Given the JSON object in a collection called 'my_collection':
{
"_id" : ObjectId("50fdb2a73f7bc7a5acecc4f8"),
"data" : {
"list" : [ 0, 1, 2, 7, 4, 5 ],
"subobj" : {
"var_a":"valuea",
"var_b":"valueb"
}
}
}
To update 'var_b' , without overwriting anything else:
db.my_collection.update({"_id":"50fdb2a73f7bc7a5acecc4f8"}, { "$set":{"data.subobj.var_b":"new_value"}})
To update the 3rd element in the array 'list' with value '99', without overwriting anything else:
db.my_collection.update({"_id":"50fdb2a73f7bc7a5acecc4f8"}, { "$set":{"data.list.2":"99"} } )
DBCollection dbCollection = db.getCollection("mycollection");
BasicDBObject dbObject = new BasicDBObject();
dbObject.put("_id", "3");
// your update condition - or the query
DBObject newObject = dbCollection.find(dbObject).toArray().get(0);
// I just take the first element. Can iterate through as per your requirement if multiple fields exist
newObject.put("key","value");
//add field, either a new field or any existing field
dbCollection.findAndModify(dbObject, newObject);
Just use the above steps . You can change the details without affecting other items associated to the same key.
// update
WriteResult usr = (WriteResult)mongoOperation.upsert(new Query(Criteria.where("name").is("till")),
Update.update("password", "jk45"), "collection");
System.out.println("updatedUser : " + usr );