Im trying to delete all items in my table in dynamodb but it does not work.
try {
ScanRequest scanRequest = new ScanRequest().withTableName(table);
ScanResult scanResult = null;
do {
if (Check.nonNull(scanResult)) {
scanRequest.setExclusiveStartKey(scanResult.getLastEvaluatedKey());
}
scanResult = client.scan(scanRequest);
scanResult.getItems().forEach((Item) -> {
String n1 = Item.get("n1").toString();
String n2 = tem.get("n2").toString();
DeleteItemSpec spec = new DeleteItemSpec().withPrimaryKey("n1", n1, "n2", n2);
dynamodb.getTable(table).deleteItem(spec);
});
} while (Check.nonNull(scanResult.getLastEvaluatedKey()));
} catch (Exception e) {
throw new BadRequestException(e);
}
n1 is my Primary partition key
n2 is my Primary sort key
The best approach to delete all the items from DynamoDB is to drop the table and recreate it.
Otherwise, there are lot of read capacity and write capacity units being used which will cost you.
Dropping and recreating the table is the best approach.
PREAMBLE: While a scan operation is expensive, I was needing this answer for initialising a table for a test scenario (low volume). The table was being created by another process and I needed the test scenario on that table, I could therefore not delete and recreate the table.
ANSWER:
given:
DynamoDbClient db
static String TABLE_NAME
static String HASH_KEY
static String SORT_KEY
ScanIterable scanIterable = db.scanPaginator(ScanRequest.builder()
.tableName(TABLE_NAME)
.build());
for(ScanResponse scanResponse:scanIterable){
for( Map<String, AttributeValue> item: scanResponse.items()){
Map<String,AttributeValue> deleteKey = new HashMap<>();
deleteKey.put(HASH_KEY,item.get(HASH_KEY));
deleteKey.put(SORT_KEY,item.get(SORT_KEY));
db.deleteItem(DeleteItemRequest.builder()
.tableName(TRANSACTION_TABLE_NAME)
.key(deleteKey).build());
}
}
To delete all the items from the table first you need to perform scan operation over the table which will results you an scanoutcome. Using the iterator loop over the sacnoutcome with the primary key and it's primary key value.This will be one of the approach to delete all the items from the table. Hope that this code will work you. Thanks
Table table = dynamoDB.getTable(your_table);
ItemCollection<ScanOutcome> deleteoutcome = table.scan();
Iterator<Item> iterator = deleteoutcome.iterator();
while (iterator.hasNext()) {
your_table.deleteItem("PrimaryKey", iterator.next().get("primary key value"));
}
//May be we can make it look generic by reading key schema first as below
String strPartitionKey = null;
String strSortKey = null;
TableDescription description = table.describe();
List<KeySchemaElement> schema = description.getKeySchema();
for (KeySchemaElement element : schema) {
if (element.getKeyType().equalsIgnoreCase("HASH"))
strPartitionKey = element.getAttributeName();
if (element.getKeyType().equalsIgnoreCase("RANGE"))
strSortKey = element.getAttributeName();
}
ItemCollection<ScanOutcome> deleteoutcome = table.scan();
Iterator<Item> iterator = deleteoutcome.iterator();
while (iterator.hasNext()) {
Item next = iterator.next();
if (strSortKey == null && strPartitionKey != null)
table.deleteItem(strPartitionKey, next.get(strPartitionKey));
else if (strPartitionKey != null && strSortKey != null)
table.deleteItem(strPartitionKey, next.get(strPartitionKey), strSortKey, next.get(strSortKey));
}
Related
I am doing pagination in DynamoDb and see that the result not like my expected, but I still don't know where wrong in my code.
I have searched for this document and see that I just need to set evaluated key which I got last evaluated key from last request to pass to next request.
Here is my code :
public QueryResultPage<SbmBookingInfo> getListBooking(String email, int size,
String lastTimeStamp, String lastBookingId, boolean isForward) {
DynamoDBMapper mapper = new DynamoDBMapper(getClient());
Map<String, AttributeValue> expressionAttributeValues = new HashMap<>();
expressionAttributeValues.put(":email", new AttributeValue().withS(email));
if (lastBookingId == null) {
DynamoDBQueryExpression<SbmBookingInfo> queryExpression = new DynamoDBQueryExpression<SbmBookingInfo>()
.withIndexName("practitioner-index").withKeyConditionExpression("email=:email")
.withExpressionAttributeValues(expressionAttributeValues).withConsistentRead(false).withLimit(size);
QueryResultPage<SbmBookingInfo> result = mapper.queryPage(SbmBookingInfo.class, queryExpression);
m_log.info("Last evaluated key " + result.getLastEvaluatedKey());
return result;
} else {
Map<String, AttributeValue> lastEvaluatedKey = new HashMap<>();
lastEvaluatedKey.put("timeStamp", new AttributeValue().withN(lastTimeStamp));
lastEvaluatedKey.put("bookingId", new AttributeValue().withN(lastBookingId));
lastEvaluatedKey.put("email", new AttributeValue().withS(email));
DynamoDBQueryExpression<SbmBookingInfo> queryExpression = new DynamoDBQueryExpression<SbmBookingInfo>().withLimit(size)
.withIndexName("practitioner-index").withKeyConditionExpression("email=:email")
.withExpressionAttributeValues(expressionAttributeValues).withConsistentRead(false)
.withScanIndexForward(isForward).withExclusiveStartKey(lastEvaluatedKey);
m_log.info("Exclusive key" + queryExpression.getExclusiveStartKey());
return mapper.queryPage(SbmBookingInfo.class, queryExpression);
}
}
This table has three key: bookingId is hashkey, index: email, and timeStamp is rangeKey.
I checked this function by making the first request and I receive last evaluated key then I make second request with passing the info that I received but seem like it didn't work.
Please help me at this case. Thanks
I'm using Java and Infusionsofts API to get a list of contacts based on the customer id. I can't figure out a way to do this. I'm using Googles Guava to use multimap but it's producing an error:
org.apache.xmlrpc.common.XmlRpcExtensionException: Serializable objects aren't supported, if isEnabledForExtensions() == false
So now i'm trying hashmap and i'm inserting "Id" as the key and the customer id as the value but there's always one entry in the hashmap.
How can I add to the parameters variable a map that contains:
["Id",11111]
["Id",22322]
["Id",44444]
List parameters = new ArrayList();
parameters.add(APP_ID);
parameters.add(TABLE_NAME);
parameters.add(LIMIT);
parameters.add(pageNumber);
HashMap<String, Integer> map = new HashMap<String, Integer>();
for(int customerId : customerIds){
map.put("Id", customerId);
}
//PROBLEM IS HERE
parameters.add(map);
//THIS IS THE PROBLEM, I NEED TO ADD ["Id", customerId] multiple
//times with the customerId being different but since there's a hashmap
//There's always 1 entry in the map
String[] fields = {"Email","FirstName"};
parameters.add(fields);
Object[] contacts = null;
try{
contacts = ( Object [] ) client.execute("DataService.query",parameters);
}catch(XmlRpcException e){
e.printStackTrace();
}
for (int i = 0; i < contacts.length; i++) {
Map contact = (Map) contacts[i];
System.out.println(contact+"\n\n");
}
I have TreeMap using the Joda DateTime object and is does not seem to be sorting here is the definition:
TreeMap<DateTime, HolderAnswer> dateTimeTreeMap = new TreeMap<DateTime, HolderAnswer>();
I added in the values as follows (I'm just using a generic sql statement here);
//then get previously selected answers to move to the top of the list
String sql = "Select ActionDT, RecID, TextID, Text, Value from Foo";
Cursor c = DataBaseConnector.query(sql);
if (c != null) {
if (c.moveToFirst()) {
do {
HolderAnswer answer = null;
boolean valueAlreadyIn = false;
DateTime dt = formatter.parseDateTime(c.getString(c.getColumnIndex("ActionDT")));
//we will be adding in the options in the next section, setting to null for now.
answer = new HolderAnswer(c.getInt(c.getColumnIndex("RecID")),c.getInt(c.getColumnIndex("TextID")),null,count,c.getString(c.getColumnIndex("Text")));
//////////////////////////////////////////////////////////////
Iterator<Entry<DateTime, HolderAnswer>> it = dateTimeTreeMap.entrySet().iterator();
while (it.hasNext()) {
Entry<DateTime, HolderAnswer> pairs = it.next();
HolderAnswer tempAnswer = (HolderAnswer) pairs.getValue();
DateTime tempDateTime = (DateTime) pairs.getKey();
//if answers match, transfer over options
if (answer.getTextID() == tempAnswer.getTextID()) {
valueAlreadyIn = true;
}
}
if (!valueAlreadyIn) {
dateTimeTreeMap.put(dt,answer);
}
//////////////////////////////////////////////////////////////////
//count++;
} while(c.moveToNext());
c.close();
c = null;
}
}
When I print out the values, they don't seem to be sorted, they come out in no discernable pattern. Even doing:
dateTimeTreeMap.descendingMap();
Does nothing. Am I missing something?
The descendingMap() method is used to return a reverse order view of the mappings contained in this map so it looks like you're forgetting to assign the sorted map to the original one.
dateTimeTreeMap = dateTimeTreeMap.descendingMap();
Is it possible to achieve paging in cassandra through hector API for user defined queries.
If yes, how?
I have added a basic method, rest of things you have to handle. Here, as you can see we have defined page size as 100 rows, each having 10 columns. Now after the first iteration you have to some how store the last key value, which will the starting point for the iteration.
int row_count = 100;
RangeSlicesQuery<UUID, String, Long> rangeSlicesQuery = HFactory
.createRangeSlicesQuery(keyspace, UUIDSerializer.get(), StringSerializer.get(), LongSerializer.get())
.setColumnFamily("Column Family")
.setRange(null, null, false, 10)
.setRowCount(row_count);
UUID last_key = null;
while (true) {
rangeSlicesQuery.setKeys(last_key, null);
System.out.println(" > " + last_key);
QueryResult<OrderedRows<UUID, String, Long>> result = rangeSlicesQuery.execute();
OrderedRows<UUID, String, Long> rows = result.get();
Iterator<Row<UUID, String, Long>> rowsIterator = rows.iterator();
if (last_key != null && rowsIterator != null) rowsIterator.next();
while (rowsIterator.hasNext()) {
Row<UUID, String, Long> row = rowsIterator.next();
last_key = row.getKey();
if (row.getColumnSlice().getColumns().isEmpty()) {
continue;
}
}
}
}
I've a row in db returned by query below. Columns to select include rec, head, amount. I want to sort the rows by head column. I tried Map where string for head and list for other two columns.
I've hit the wall with my non-working code posted below. How would I append another list to list of repeated key. Documentation says it replaces the value for same key whereas I need it appended to the list value. I would be really greatful for any help.
Query q= session.createQuery("select tally_receipt_prefix, tally_receipt_no, tally_head, tally_amount from Tally_table where tally_system_date='"+fmtd_date+"' and tally_dbcr_indicator='DB' and tally_mode='Ca' order by tally_head,tally_receipt_prefix,tally_receipt_no"); System.out.println("query "+q);
List heads=new ArrayList();
for(Iterator it=q.iterate(); it.hasNext(); )
{
Object[] row= (Object[]) it.next();
payincash1=new LinkedHashMap<String, List>();
heads.add((String)row[2]);
List tails = null;
tails=new ArrayList();
tails.add((String)row[0]);
tails.add((String)row[1]);
tails.add((String)row[3]);
System.out.println("heads in dao from iter 1: "+heads);
System.out.println("tails in dao from iter1 on: "+tails);
if(heads.contains((String)row[2])) // for head in temp list
{
System.out.println("in first if");
if(payincash1.containsKey((String)row[2]))
{
System.out.println("map if repeat: "+payincash1);
payincash1.put((String)row[2],tails);
}
}
else
{
System.out.println("map if not repeat: "+payincash1);
payincash1.put((String)row[2], tails);
}
}
Sounds more like you want a list of lists
Something like Map<String, List<List>>
Then you'd end up with something like...
Map<String, List<List>> payincash1 = new LinkedHashMap<String, List<List>>();
heads.add((String) row[2]);
List tails = null;
tails = new ArrayList();
tails.add((String) row[0]);
tails.add((String) row[1]);
tails.add((String) row[3]);
System.out.println("heads in dao from iter 1: " + heads);
System.out.println("tails in dao from iter1 on: " + tails);
List master = payincash1.get((String)row[2]);
if (master == null) {
master = new List();
payincash1.put((String)row[2], master);
}
master.add(tails);
Now, personally, I'd be creating a "data" object that would contain all this information.
public class MyData {
private String rec, head, amount, ??; // Apparently you have another variable I don't know about
public MyData(String rec, String head, String amount, String ??) {
// Initalise...
}
// Setters and getters not inclueded
}
Then you could do something like this...
Map<String, List<MyData>> payincash1 = new LinkedHashMap<String, List<MyData>>();
MyData data = new MyData(row[0], row[1], row[2], row[3]);
List master = payincash1.get((String)row[2]);
if (master == null) {
master = new List<MyData>();
payincash1.put((String)row[2], master);
}
master.add(data);
Which is a little cleaner (IMHO)
From what I understand you need Multimap of guava library.