I've been banging my head against this for an embarrassing amount of time, and appreciate any help. I have what I think is a simple scenario where I'm trying to retrieve and update an entity if it exists, or create a new if it doesn't. There's a parent entity, and that part seems to work fine. I create the child key like this:
Key someKey = KeyFactory.createKey(parent.getKey(), CHILD_KIND, child.getUniqueStringValue());
According to the documents I read doing a get on this would return the entity if it existed:
Entity someChild = datastore.get(someKey);
But it doesn't. Instead I have to do a:
Query query = new Query(CHILD_KIND, someKey);
Entity someChild = datastore.prepare(query).asSingleEntity();
to find it. If I try to log the someKey.toString() value, they look the same during creation as they do during a search, but it still only finds them if I do the query instead of the get. I'm missing something stupid and obvious and searching has gotten me nowhere over the course of about a week, so I appreciate any help.
Thanks,
Mike
Edit:
To clarify how I'm creating the child entity, it's like this:
Key someKey = KeyFactory.createKey(parent.getKey(), CHILD_KIND, child.getUniqueStringValue());
Entity someChild = new Entity(CHILD_KIND, someKey);
someChild.setProperty("desc", child.getUniqueStringValue());
someChild.setProperty("createTime", child.getCreateTime());
// and then a few more values that are timestamps or null
There is a problem in the way you create a child entity. It should be:
Entity someChild = new Entity(CHILD_KIND, parent.getKey());
or
Entity someChild = new Entity(CHILD_KIND, child.getUniqueStringValue(), parent.getKey());
Also, I am not sure why you use child.getUniqueStringValue() instead of a simple Long id, which takes less space.
The problem is that you are confusing Parent_Key with Key.
This line:
Entity someChild = new Entity(CHILD_KIND, someKey);
Creates an Entity of the kind "CHILD_KIND" with parent entity of key "someKey". See this document for details [1].
When you do a simple datastore.get(key), you are using the Entity's own key, not the parent key. So what you need to do is use KeyFactory to get the Entity Key from its Parent Key, Kind, Name. The details can be found here [2] and here [3].
[1] - https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Entity
[2] - https://cloud.google.com/appengine/docs/java/datastore/entities#Java_Retrieving_an_entity
[3] - https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/KeyFactory
Related
Say, I want to save/create new item to the DynamoDb table,
if and only if there is not any existent item already that that would contain the referenceId equal to the same value I set.
In my case I want to create a item with withReferenceId=123 if there is not any other withReferenceId=123 in the table.
the referenceId is not primary key! (I don not want it to be it)
So the code:
val withReferenceIdValue = "123";
val saveExpression = new DynamoDBSaveExpression();
final Map<String, ExpectedAttributeValue> expectedNoReferenceIdFound = new HashMap();
expectedNoReferenceIdFound.put(
"referenceId",
new ExpectedAttributeValue(new AttributeValue().withS(withReferenceIdValue)).withComparisonOperator(ComparisonOperator.NE)
);
saveExpression.setExpected(expectedNoReferenceIdFound);
newItemRecord.setReferenceId(withReferenceId);
this.mapper.save(newItemRecord, saveExpression); // do not fail..
That seems does not work.
I the table has the referenceId=123 already the save() does not fail.
I expected this.mapper.save to fail with exception.
Q: How to make it fail on condition?
I also checked this one where they suggest to add auxiliary table (transaction-state table)..because seems the saveExpression works only for primary/partition key... if so:
not sure why there that limitation. in any case if it is primary key
one can not create duplicated item with the same primary key.. why
creating conditions on first place. 3rd table is too much.. why there
is not just NE to whatever field I want to use. I may create an index
for this filed. not being limited to use only primary key.. that what
I mean
UPDATE:
My table mapping code:
#Data // I use [lombok][2] and it does generate getters and setters.
#DynamoDBTable(tableName = "MyTable")
public class MyTable {
#DynamoDBHashKey(attributeName = "myTableID")
#DynamoDBAutoGeneratedKey
private String myTableID;
#DynamoDBAttribute(attributeName = "referenceId")
private String referenceId;
#DynamoDBAttribute(attributeName = "startTime")
private String startTime;
#DynamoDBAttribute(attributeName = "endTime")
private String endTime;
...
}
Correct me if I'm wrong, but from the:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/dynamodb-dg.pdf
Conditional Writes By default, the DynamoDB write operations (PutItem,
UpdateItem, DeleteItem) are unconditional: each of these operations
will overwrite an existing item that has the specified primary key
the primary key - that makes me thing that the conditional write works ONLY with primary keys
--
Also there is attempt use the transactional way r/w from the db. There is a library. That event has not maven repo: https://github.com/awslabs/dynamodb-transactions
As an alternative seems is the way to use 3rd transaction table with the primary keys that are responsible to tell you whether you are ok to read or write to the table. (ugly) as we replied here: DynamoDBMapper save item only if unique
Another alternative, I guess (by design): it is to design your tables in a way so you use the primary key as your business-key, so you can use it for the conditional writes.
--
Another option: use Aurora :)
--
Another options (investigating): https://aws.amazon.com/blogs/database/building-distributed-locks-with-the-dynamodb-lock-client/ - this I do not like either. because potentially it would create timeouts for others who would want to create new items in this table.
--
Another option: Live with this let duplication happens for the item-creation (not including the primary key). And take care of it as a part of "garbage collection". Depends on the scenario.
I am stuck with geting a persisted object by its id. I am getting an error:
com.google.appengine.api.datastore.EntityNotFoundException: No entity was found matching the key: CNG("T78")
I persist the object as below to the data store:
Key cngKey = KeyFactory.createKey("CNG", jsonCNG.cNGID);
Entity cngEntity = new Entity("CNG", cngKey);
cngEntity.setProperty("cng_name", jsonCNG.cNGName);
cngEntity.setProperty("cng_type", jsonCNG.cNGType);
cngEntity.setProperty("cng_content", cng);
Here cng is a json string. I set the key with a string: cNGID. I am trying to use the same id to get the object.
Key cngKey = KeyFactory.createKey("CNG", "T78")
and end up getting the above error.
The constructor new Entity("CNG", cngKey) is defining the entity with kind and parent key. Then when you try to retrieve it you do not provide parent key: KeyFactory.createKey("CNG", "T78"). This does not work - you must either provide parent key in both places on not.
Note - defining entity parent is used when defining entity groups, which are in turn important when using transactions. You probably did not want that?
Instead you should just use new Entity(cngKey).
You don't seem to have saved it to the datastore.
Datastore.put(cngEntity);
each user can have many questions. questions can only have one user. is the following correct
Key questionKey = KeyFactory.createKey("Questions", userId);
Entity questionEntity = new Entity("Question", questionKey);
questionEntity.setProperty("questionCategory", questionCategory);
...
The given usage is wrong. For the question, you are creating key using kind and userid . This implies the corresponding entity is of Kind="Questions" and id=userid and no parents. This is wrong and you will start getting errors once you have more than 1 question for a user as they will all have the same key.
Ideally what you need is that for a question entity, declare its kind as question and parent as the user as follows :
1, If using manually generated id or name for a question , then :
Key questionKey = KeyFactory.createKey(userkey, "Questions", questionidorname);
2, If using app engine's auto generate id, then no need create key, instead create entity as:
Entity questionEntity = new Entity("Questions",
userkey)
Here userkey is the key of a user entity
List<Key> items = new ArrayList<Key>();
user.setProperty("ItemsList", items);
The code given above does not work. What are the ways in which this can be achieved using Java platform in GAE ?
P.S. Though the following piece of code runs fine -
List<String> items = new ArrayList<String>();
user.setProperty("ItemsList", items);
It should work without any problems. Did you forget to put the item? If you didn't set id for the item, the DataStore will automatically assign the entity an integer numeric ID. You need to put that generated key into the list.
Here is the sample code:
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Entity item = new Entity("Item");
datastore.put(item);
Entity user = new Entity("User");
List<Key> items = new ArrayList<Key>();
items.add(item.getKey());
user.setProperty("ItemsList", items);
datastore.put(user);
Please check the doc: https://developers.google.com/appengine/docs/java/datastore/entities
And, it is better to handle data layer via some third part API like Objectify: http://code.google.com/p/objectify-appengine/
It should work.
Infact, that's one of the ways used to implement many-to-many and many-to-one relationships.
See, for example, http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Multi-Value_Relationship
Key used in the Objectify example is com.googlecode.objectify.Key<T> though.
You cannot store an empty collection in the datastore. It must contain at least one value. Could it be that you missed out...
items.add(item.getKey());
I'm just getting started with MongoDb and I've noticed that I get a lot of duplicate records for entries that I meant to be unique. I would like to know how to use a composite key for my data and I'm looking for information on how to create them. Lastly, I am using Java to access mongo and morphia as my ORM layer so including those in your answers would be awesome.
Morphia: http://code.google.com/p/morphia/
You can use objects for the _id field as well. The _id field is always unique. That way you kind of get a composite primary key:
{ _id : { a : 1, b: 1} }
Just be careful when creating these ids that the order of keys (a and b in the example) matters, if you swap them around, it is considered a different object.
The other possibility is to leave _id alone and create a unique compound index.
db.things.ensureIndex({firstname: 1, lastname: 1}, {unique: true});
//Deprecated since version 3.0.0, is now an alias for db.things.createIndex()
https://docs.mongodb.org/v3.0/reference/method/db.collection.ensureIndex/
You can create Unique Indexes on the fields of the document that you'd want to test uniqueness on. They can be composite as well (called compound key indexes in MongoDB land), as you can see from the documentation. Morphia does have a #Indexed annotation to support indexing at the field level. In addition with morphia you can define compound keys at the class level with the #Indexed annotation.
I just noticed that the question is marked as "java", so you'd want to do something like:
final BasicDBObject id = new BasicDBObject("a", aVal)
.append("b", bVal)
.append("c", cVal);
results = coll.find(new BasicDBObject("_id", id));
I use Morphia too, but have found (that while it works) it generates lots of errors as it tries to marshall the composite key. I use the above when querying to avoid these errors.
My original code (which also works):
final ProbId key = new ProbId(srcText, srcLang, destLang);
final QueryImpl<Probabilities> query = ds.createQuery(Probabilities.class)
.field("id").equal(key);
Probabilities probs = (Probabilities) query.get();
My ProbId class is annotated as #Entity(noClassnameStored = true) and inside the Probabilities class, the id field is #Id ProbId id;
I will try to explain with an example:
Create a table Music
Add Artist as a primary key
Now since artist may have many songs we have to figure out a sort key.
The combination of both will be a composite key.
Meaning, the Artist + SongTitle will be unique.
something like this:
{
"Artist" : {"s" : "David Bowie"},
"SongTitle" : {"s" : "changes"},
"AlbumTitle" : {"s" : "Hunky"},
"Genre" : {"s" : "Rock"},
}
Artist key above is: Partition Key
SongTitle key above is: sort key
The combination of both is always unique or should be unique. Rest are attributes which may vary per record.
Once you have this data structure in place you can easily append and scan as per your custom queries.
Sample Mongo queries for reference:
db.products.insert(json file path)
db.collection.drop(json file path)
db.users.find(json file path)