I am dealing with a ConditionalCheckFailedException and I am not exactly sure which condition is failing the check. When I open up debugger and examine the exception variable, I can't find any useful info.
Below is my Java Dynamo Client code. I am trying to make a conditional write to DynamoDB using DynamoDBSaveExpression when:
The client date in the table comes before the current client date that I am trying to write (stored as EPOCH time)
An entry does not exist in the table (I check for the FEEDBACK_KEY as it is the primary key in my table)
When I write the first entry into the table, it works, but on updates when an entry exists, I get the ConditionalCheckFailedException exception. Any ideas?
final DynamoDBSaveExpression expression = new DynamoDBSaveExpression();
final Map<String, ExpectedAttributeValue> expectedAttributes =
ImmutableMap.<String, ExpectedAttributeValue>builder()
.put(ThemesMessageEligibility.TableKeys.CLIENT_DATE,
new ExpectedAttributeValue()
.withComparisonOperator(ComparisonOperator.LT)
.withValue(new AttributeValue().withN(clientDate)))
.put(ThemesMessageEligibility.TableKeys.FEEDBACK_KEY,
new ExpectedAttributeValue(false))
.build();
expression.setExpected(expectedAttributes);
expression.setConditionalOperator(ConditionalOperator.OR);
// Conditional write if the clientDate is after the dynamo's client Date
try {
dynamoMapper.save(themesFeedbackComponentContainer, expression);
} catch (ConditionalCheckFailedException ex) {
...
}
I would remove the second condition, or change it so that it conditions on the item existing (new ExpectedAttributeValue(true)). UpdateItem will just overwrite the existing item even if it exists, so it seems like the CLIENT_DATE condition is the only one you need.
The API call as written above will only succeed on the first write (that is, when the item does not exist). In retrospect, if you only want the first write to an item to succeed (and fail if the item already exists) the CLIENT_DATE condition is not necessary (as there are no attributes in the existing image to compare to).
Related
I'm new to DynamoDb and I'm struggling to work out how to do this (using the java sdk).
I currently have a table (in mongo) for notifications. The schema is basically as follows (I've simplified it)
id: string
notifiedUsers: [123, 345, 456, 567]
message: "this is a message"
created: 12345678000 (epoch millis)
I wanted to migrate to Dynamodb, but I can't work out the best way to select all notifications that went to a particular user after a certain date?
I gather I can't have an index on a list like notifiedUsers, therefore I can't use a query in this case - is that correct?
I'd prefer not to scan and then filter, there could be a lot of records.
Is there a way to do this using a query or another approach?
EDIT
This is what I'm trying now, it's not working and I'm not sure where to take it (if anywhere).
Condition rangeKeyCondition = new Condition()
.withComparisonOperator(ComparisonOperator.CONTAINS.toString())
.withAttributeValueList(new AttributeValue().withS(userId));
if(startTimestamp != null) {
rangeKeyCondition = rangeKeyCondition.withComparisonOperator(ComparisonOperator.GT.toString())
.withAttributeValueList(new AttributeValue().withS(startTimestamp));
}
NotificationFeedDynamoRecord replyKey = new NotificationFeedDynamoRecord();
replyKey.setId(partitionKey);
DynamoDBQueryExpression<NotificationFeedDynamoRecord> queryExpression = new DynamoDBQueryExpression<NotificationFeedDynamoRecord>()
.withHashKeyValues(replyKey)
.withRangeKeyCondition(NOTIFICATIONS, rangeKeyCondition);
In case anyone else comes across this question, in the end we flattened the schema, so that there is now a record per userId. This has lead to problems because it's not possible with dynamoDb to atomically batch write records. With the original schema we had one record, and could write it atomically ensuring that all users got that notification. Now we cannot be certain, and this is causing pain.
I am trying to use the new Amazon DynamoDB JSON API to add/overwrite key-value pairs in a JSON attribute called "document". Ideally, I would like simply to structure my write calls to send the KV pairs to add to the attribute, and have Dynamo create the attribute if it does not already exist for the given primary key. However if I try this with just a straightforward UpdateItemSpec:
PrimaryKey primaryKey = new PrimaryKey("key_str", "mapKey");
ValueMap valuesMap = new ValueMap().withLong(":a", 1234L).withLong(":b", 1234L);
UpdateItemSpec updateSpec = new UpdateItemSpec().withPrimaryKey(primaryKey).withUpdateExpression("SET document.value1 = :a, document.value2 = :b");
updateSpec.withValueMap(valuesMap);
table.updateItem(updateSpec);
I get com.amazonaws.AmazonServiceException: The document path provided in the update expression is invalid for update, meaning DynamoDB could not find the given attribute named "document" to which to apply the update.
I managed to approximate this functionality with the following series of calls:
try {
// 1. Attempt UpdateItemSpec as if attribute already exists
} catch (AmazonServiceException e) {
// 2. Confirm the exception indicated the attribute was not present, otherwise rethrow it
// 3. Use a put-if-absent request to initialize an empty JSON map at the attribute "document"
// 4. Rerun the UpdateItemSpec call from the above try block
}
This works, but is less than ideal as it will require 3 calls to DynamoDB every time I add a new primary key to the table. I experimented a bit with the attribute_not_exists function that can be used in Update Expressions, but wasn't able to get it to work in the way I want.
Any Dynamo gurus out there have any ideas on how/whether this can be done?
I received an answer from Amazon Support that it is not actually possible to accomplish this with a single call. They did suggest to reduce the number of calls when adding the attribute for a new primary key from 3 to 2, by using the desired JSON map in the put-if-absent request rather than an empty map.
I would like to handle failure of insert to collection (using Java) in order to be sure that my insert was successful. In case that insert fails, I want to perform some fall-back action.
Assuming following code in Java and latest mongo driver (version 2.11.3):
BasicDBObject doc = new BasicDBObject("name", "MongoDB");
WriteResult result = coll.insert(WriteConcern.SAFE, doc);
I am confused by the fact that the insert returns WriteResult and it could throw MongoException. What am I supposed to do in order to safely detect any possible failure of insert? Please, provide code example. And if you can clarify when insert throws exception and when it just returns some error write result. I tried to search in java driver API docs at http://api.mongodb.org/java/2.11.3/ for it; howerever, this infromation is missing there.
WriteResult result;
try {
result = coll.insert(WriteConcern.SAFE, doc);
}
catch(MongoException ex){
logger.warn("Insert failed.", ex);
throw ex;
}
//Shall I check result here for additional errors?
If I should check, what type of error I am able to detect by checking insert result?
You need to take a look at "WriteConcern", it has the all behaviors you need.
You can use it per one write like this:
coll.insert(dbObj, WriteConcern.SAFE);
If you use WriteConcern.SAFE your operation will wait for an acknowledgement from the primary server, so if no exception is raised then you're ok.
Or you can set default behaviour for all write operations when you are creating MongoClient:
MongoClientOptions.Builder builder = new MongoClientOptions.Builder();
builder.writeConcern(WriteConcern.JOURNAL_SAFE);
MongoClient mongoClient = new MongoClient(
new ServerAddress("localhost"), builder.build());
[Based on Colin Morelli's comment] If you don't use a WriteConcern that raises exceptions, you can use the WriteResult.getLastError() to determine if it was successful or not. Similarly, if you use WriteConcern.SAFE, and the write succeeds, WriteResult will have useful information on it such as the number of records that were written.
Here you can read about WriteConcern in general.
I'm fairly new to Amazon's AWS and its API for Java, so I'm not exactly sure what the most efficient method for what I'm trying to do would be. Basically, I'm trying to setup a database that will store a project's ID, it's status, as well as the bucket and location when uploaded to an S3 bucket by a user. What I'm having trouble with is getting a list of all the project IDs that have a status of "ready" under the status attribute. Any projects that are of status "ready" need to have their ID numbers loaded to an array or arraylist for later reference. Any recommendations?
The way to do this is to use the scan API. However, this means dynamo will need to look at every item in your table, and check if its attribute "status" is equal to "ready". The cost of this operation will be large, and will charge you for reading every item in your table.
The code would look something like this:
Condition scanFilterCondition = new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS("ready"));
Map<String, Condition> conditions = new HashMap<String, Condition>();
conditions.put("status", scanFilterCondition);
ScanRequest scanRequest = new ScanRequest()
.withTableName("MasterProductTable")
.withScanFilter(conditions);
ScanResult result = client.scan(scanRequest);
There is a way to make this better, though it requires denormalizing your data. Try keeping a second table with a hash key of "status", and a range key of "project ID". This is in addition to your existing table. This would allow you to use the Query API (scan's much cheaper cousin), and ask it for all items with a hash key of "ready". This will get you a list of the project IDs you need, and you can then get them from the project ID table you already have.
The code for this would look something like:
QueryRequest queryRequest = new QueryRequest()
.withTableName("ProductByStatus")
.withHashKeyValue(new AttributeValue().withS("ready"));
QueryResult result = client.query(queryRequest);
The downside to this approach is you have to update two tables whenever you update the status field, and you have to make sure that you keep them in sync. Dynamo doesn't offer transactionality, so you have to be ready for the case where the update to the master project table succeeds, but your secondary status table doesn't. Or vice-versa.
For further reference: http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/QueryAndScan.html
Using the GeoTools WFS-T plugin, I have created a new row, and after a commit, I have a FeatureId whos .getId() returns an ugly string that looks something like this:
newmy_database:my_table.9223372036854775807
Aside from the fact that the word "new" at the beginning of "my_database" is a surprise, the number in no way reflects the primary key of the new row (which in this case is "23"). Fair enough, I thought this may be some internal numbering system. However, now I want a foreign key in another table to get the primary key of the new row in this one, and I'm not sure how to get the value from this FID. Some places suggest that you can use an FID in a query like this:
Filter filter = filterFactory.id(Collections.singleton(fid));
Query query = new Query(tableName, filter);
SimpleFeatureCollection features = simpleFeatureSource.getFeatures(query);
But this fails at parsing the FID, at the underscore of all places! That underscore was there when the row was created (I had to pass "my_database:my_table" as the table to add the row to).
I'm sure that either there is something wrong with the id, or I'm using it incorrectly somehow. Can anyone shed any light?
It appears as if a couple things are going wrong - and perhaps a bug report is needed.
The FeatureId with "new" at the beginning is a temporary id; that should be replaced with the real result once commit has been called.
There are a number of way to be aware of this:
1) You can listen for a BatchFeatureEvent; this offers the information on "temp id" -> "wfs id"
2) Internally this information is parsed from the Transaction Result returned from your WFS. The result is saved in the WFSTransactionState for you to access. This was before BatchFeatureEvent was invented.
Transaction transaction = new transaction("insert");
try {
SimpleFeatureStore featureStore =
(SimpleFeatureStore) wfs.getFeatureSource( typeName );
featureStore.setTransaction( transaction );
featureStore.addFeatures( DataUtilities.collection( feature ) );
transaction.commit();
// get the final feature id
WFSTransactionState wfsts = (WFSTransactionState) transaction.getState(wfs);
// In this example there is only one fid. Get it.
String result = wfsts.getFids( typeName )[0];
}
finally {
transaction.close();
}
I have updated the documentation with the above example:
http://docs.geotools.org/latest/userguide/library/data/wfs.html