We Keep Coding

How to insert a map as attribute value when attribute does not yet exist with DynamoDB?

Could someone please tell me how to set a map as a value for an attribute that does not yet exist. I've been trying an update expression "SET MyAttr.#key = :val", but that only works if the attribute exists and if the value of the attribute is already a map.
I tried using "ADD MyAttr.#key = :val" instead, but then I get:
Invalid UpdateExpression: Syntax error; token: "=", near: "#key = :val"
I'm using Java. Note that I'm using an attempting an UpdateItem request to do this. It just occurred to me that maybe this doesn't work with maps. I just assumed it would because I've always been able to have a new item and/or attribute created with an update request if doesn't already exist (with other attribute value data types like string, number etc).

Let's assume that we have the following Item in our DynamoDB Table:
{
"id": "91c2b60d-c428-403c-be42-8657b4f20669",
"firstName": "John",
"lastName": "Doe"
}
and we want to add custom attributes that we have in a Map:
Map<String, String> customAttributes = new HashMap<>();
customAttributes.put("age", "21");
customAttributes.put("gender", "Male");
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().build();
DynamoDB dynamodb = new DynamoDB(client);
Table table = dynamodb.getTable("users");
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey("id", "91c2b60d-c428-403c-be42-8657b4f20669")
.withUpdateExpression("set #customAttributes = :customAttributes")
.withNameMap(new NameMap().with("#customAttributes", "customAttributes"))
.withValueMap(new ValueMap().withMap(":customAttributes", customAttributes))
.withReturnValues(ReturnValue.ALL_NEW);
UpdateItemOutcome outcome = table.updateItem(updateItemSpec);
System.out.println(outcome.getItem().toJSONPretty());
The result:
{
"id" : "91c2b60d-c428-403c-be42-8657b4f20669",
"firstName" : "John",
"lastName" : "Doe",
"customAttributes" : {
"gender" : "Male",
"age" : "21"
}
}
Now, if we want to add a new attribute to the customAttributes map:
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey("id", "91c2b60d-c428-403c-be42-8657b4f20669")
.withUpdateExpression("set customAttributes.#occupation = :value")
.withNameMap(new NameMap().with("#occupation", "occupation"))
.withValueMap(new ValueMap().withString(":value", "Programmer"))
.withReturnValues(ReturnValue.ALL_NEW);
UpdateItemOutcome outcome = table.updateItem(updateItemSpec);
System.out.println(outcome.getItem().toJSONPretty());
I suggest that you take a look at: DynamoDBMapper, it will save you tons of lines of code.

mongodb mongoTemplate get distinct field with some criteria

My MongoDB json structure is
{
"_id" : "122134231234234",
"name" : "Total_pop",
"description" : "sales category",
"source" : "public",
"dataset" :"d1"
},
{
"_id" : "1123421231234234",
"name" : "Total_pop",
"description" : "sales category",
"source" : "public",
"dataset" :"d1"
},
{
"_id" : "12312342332423343",
"name" : "Total_pop",
"description" : "sales category",
"source" : "private",
"description" : "d1"
}
I need to get collection distinct of dataset where source is public.
I tried this query, and it didn't work:
Criteria criteria = new Criteria();
criteria.where("source").in("public");
query.addCriteria(criteria);
query.fields().include("name");
query.fields().include("description");
query.fields().include("description");
query.fields().include("source"); List list =
mongoTemplate.getCollection("collectionname").distinct("source", query);
Can you please help me out?

For one thing the .getCollection() method returns the basic Driver collection object like so:
DBCollection collection = mongoTemplate.getCollection("collectionName");
So the type of query object might be different from what you are using, but there are also some other things. Namely that .distinct() only returns the "distint" values of the key that you asked for, and doe not return other fields of the document. So you could do:
Criteria criteria = new Criteria();
criteria.where("dataset").is("d1");
Query query = new Query();
query.addCriteria(criteria);
List list = mongoTemplate.getCollection("collectionName")
.distinct("source",query.getQueryObject());
But that is only going to return "sample" as a single element in the list for instance.
If you want the "fields" from a distinct set then use the .aggregate() method instead. With either the "first" occurances of the other field values for the distinct key:
DBCollection colllection = mongoTemplate.getCollection("collectionName");
List<DBObject> pipeline = Arrays.<DBObject>asList(
new BasicDBObject("$match",new BasicDBObject("dataset","d1")),
new BasicDBObject("$group",
new BasicDBObject("_id","$source")
.append("name",new BasicDBObject("$first","$name"))
.append("description", new BasicDBObject("$first","$description"))
)
);
AggregationOutput output = colllection.aggregate(pipeline);
Or the actual "distinct" values of multiple fields, by making them all part of the grouping key:
DBCollection colllection = mongoTemplate.getCollection("collectionName");
List<DBObject> pipeline = Arrays.<DBObject>asList(
new BasicDBObject("$match",new BasicDBObject("dataset","d1")),
new BasicDBObject("$group",
new BasicDBObject("_id",
new BasicDBObject("source","$source")
.append("name","$name")
.append("description","$description")
)
)
);
AggregationOutput output = colllection.aggregate(pipeline);
There are also a direct .aggregate() method on mongoTemplate instances already, which has a number of helper methods to build pipelines. But this should point you in the right direction at least.

As of Spring Data Mongo 2.2.0 MongoTemplate provides a function to retrieve the distinct field with criteria,
Criteria criteria = new Criteria("country").is("IN");
Query query = new Query();
query.addCriteria(criteria);
return mongoTemplate.findDistinct(query,"city",Address.class,String.class);
Which basically finds all the distinct cities in address collection where country is IN.

Add new data to existing collection in mongo in java

I need two to add new items to the existing data in mongo db.
This is mongo db I have the following data.
{
"_id" : ObjectId("53ce11e7d0881d32d9fa935f"),
"name" : "massive riots",
"lastFeachedTime" : "Jul 15, 2014 12:55:27 PM"
}
Here I have to find the data based on name and the I have to add another two items two it.
Here is my code.
DBObject queryObject = new BasicDBObject().append("name", keyword);
if (null == newFetchTime) {
}
DBObject updateObject = new BasicDBObject();
updateObject.put("nextPageToken", nextPageToken);
updateObject.put("prevPageToken", prevPageToken);
Utils utils = new Utils();
DBCollection collection = utils.getStaging().getCollection("test");
collection.update(queryObject, updateObject, true, false);
But I am do update the existing value get removed and the new data get added.
Can any one tell me how to add the items to the existing data in mongo db.

You want the $set operator in your update. This allows the specified fields to be altered without affecting any of the existing fields in the document, unless the specified field exists in which case that field is overwritten:
DBObject update = new BasicDBObject(
"$set", new BasicDBObject()
.append("nextPageToken",nextPageToken)
.append("prevPageToken",prevPageToken)
);
Works out to the equivalent in shell:
{ "$set" : { "nextPageToken" : nextPageToken , "prevPageToken" : prevPageToken }}

Query a document on array elements in MongoDB using Java

I am new to MongoDB. My sample document is
{
"Notification" : [
{
"date_from" : ISODate("2013-07-08T18:30:00Z"),
"date_too" : ISODate("2013-07-30T18:30:00Z"),
"description" : "fdfd",
"url" : "www.adf.com"
},
{
"date_from" : ISODate("2013-07-01T18:30:00Z"),
"date_too" : ISODate("2013-07-30T18:30:00Z"),
"description" : "ddddddddddd",
"url" : "www.pqr.com"
}
],
I am trying to update the Notification whose "url" : "www.adf.com". My Java code to do this is:
BasicDBObject query=new BasicDBObject("url","www.adf.com");
DBCursor f = con.coll.find(query);
It does not search for the document whose "url" is "www.adf.com".

You have a nested document in this case. Your document has a field Notification which is an array storing multiple sub-objects with the field url. To search in a sub-field, you need to use the dot-syntax:
BasicDBObject query=new BasicDBObject("Notification.url","www.adf.com");
This will, however, return the whole document with the whole Notification array. You likely only want the sub-document. To filter this, you need to use the two-argument version of Collection.find.
BasicDBObject query=new BasicDBObject("Notification.url","www.example.com");
BasicDBObject fields=new BasicDBObject("Notification.$", 1);
DBCursor f = con.coll.find(query, fields);
The .$ means "only the first entry of this array which is matched by the find-operator"
This should still return one document with a sub-array Notifications, but this array should only contain the entry where url == "www.example.com".
To traverse this document with Java, do this:
BasicDBList notifications = (BasicDBList) f.next().get("Notification");
BasicDBObject notification = (BasicDBObject) notifications.get(0);
String url = notification.get("url");
By the way: When your database grows you will likely run into performance problems, unless you create an index to speed up this query:
con.coll.ensureIndex(new BasicDBObject("Notification.url", 1));

mongo-db Aggregation query working on mongo shell but not in java program

I am working on a project in which i have a huge collection in inside my mongo db. Now I have to get some details from the mongo db which are pretty much smiliar like SQL group by clause. I successfully executed the query on mongo shell :
> db.votes.aggregate(
... { $group : {
... _id : "vote_post_id",
... votesPerId : { $sum : 1 }
... }}
... );
{
"result" : [
{
"_id" : "vote_post_id",
"votesPerId" : 27371750
}
],
"ok" : 1
}
>
Now while executing the query from a java proggram I am facing follwing error :
Exception in thread "main" com.mongodb.CommandResult$CommandFailure: command
failed [aggregate]: { "serverUsed" : "localhost/127.0.0.1:27017" ,
"errmsg" : "exception: aggregation result exceeds maximum document size (16MB)"
, "code" : 16389 , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:88)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:134)
at com.mongodb.DBCollection.aggregate(DBCollection.java:1311)
at main.Connetion.CheckConnection.GetNoOfVotesForAType(CheckConnection.java:210)
at main.DAO.Votes.VoteDAO.findKeyWord(VoteDAO.java:109)
at main.DAO.Votes.VoteDAO.main(VoteDAO.java:139)
I have debugged my java program and it is producing the exact query what I wrote on mongo shell :
{ "$group" : { "_id" : "$vote_post_id" , "count" : { "$sum" : 1}}}
I have been through a lot of answers for this problem and none seems to solve the issue.
My java code for creating the aggregation function :
// create our pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match", new BasicDBObject("vote_type_id", "1") );
// build the $projection operation
DBObject fields = new BasicDBObject("vote_post_id", 1);
fields.put("_id", 0);
DBObject project = new BasicDBObject("$project", fields );
// Now the $group operation
DBObject groupFields = new BasicDBObject( "_id", "$vote_post_id");
groupFields.put("count", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields);
// run aggregation
AggregationOutput output = votesCollection.aggregate( match, project, group );
System.out.println(output.getCommandResult());

To make your query the same as in shell it would be:
// $group operation
DBObject groupFields = new BasicDBObject( "_id", "vote_post_id");
groupFields.put("votesPerId", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields);
// run aggregation
AggregationOutput output = votesCollection.aggregate( group );
System.out.println(output.getCommandResult());

In your shell query you're doing
_id : "vote_post_id",
but in your Java query you're doing
"_id", "$vote_post_id"
The difference is the dollar sign. $vote_post_id substitutes the value of that field to use as the bucket keys, whereas vote_post_id just uses that literal value (so there's just one bucket).