I have a EntityKind SuggestedInterest.
When I populate that with a key "GrpId" and property "suggestedint".
Now, I need the "suggestedint" value for a requested "GrpId"
So, I write the query as:
String findSuggestedInterest(String grpId)
{
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Filter filter = new FilterPredicate(Entity.KEY_RESERVED_PROPERTY,FilterOperator.EQUAL,grpId);
Query q0 = new Query("SuggestedInterest").setFilter(filter);
PreparedQuery pq0 = datastore.prepare(q0);
Entity result = pq0.asSingleEntity();
return result.getProperty("suggestedint").toString();
}
When I execute this code I get
java.lang.IllegalArgumentException: __key__ filter value must be a Key
The developer docs told to use Entity.KEY_RESERVED_PROPERTY to query on keys, but I guess I misunderstood. What is the correct way to query on key ?
You should pass it a Key instead of String:
Key grpKey = KeyFactory.createKey("SuggestedInterest", grpId)
then use it:
Filter filter =
new FilterPredicate(Entity.KEY_RESERVED_PROPERTY,FilterOperator.EQUAL,grpKey);
Related
I use the google ads API, I get data by query and I want to write the data to a file.
try (GoogleAdsServiceClient googleAdsServiceClient = client.getLatestVersion()
.createGoogleAdsServiceClient()) {
String query = queryData.getQuery();
String customerId = account.toString();
SearchGoogleAdsStreamRequest request = SearchGoogleAdsStreamRequest.newBuilder()
.setCustomerId(customerId)
.setQuery(query)
.build();
for (SearchGoogleAdsStreamResponse searchResponse : stream) {
List<GoogleAdsRow> results = searchResponse.getResultsList();
for (GoogleAdsRow googleAdsRow : results) {
googleAdsRow.getCustomer().getCurrencyCode();
...
I search generic code to go through all the columns and write them and not to go line by line like the following example:
line.append(googleAdsRow.getMetrics().getActiveViewCtr()).append(comma);
line.append(googleAdsRow.getMetrics().getActiveViewImpressions()).append(comma);
line.append(googleAdsRow.getMetrics().getActiveViewMeasurability()).append(comma);
line.append(googleAdsRow.getMetrics().getActiveViewMeasurableCostMicros()).append(comma);
line.append(googleAdsRow.getMetrics().getActiveViewMeasurableImpressions()).append(comma);
line.append(googleAdsRow.getAdGroup().getId()).append(comma);
line.append(googleAdsRow.getAdGroup().getName()).append(comma);
line.append(googleAdsRow.getAdGroup().getStatus()).append(comma);
line.append(googleAdsRow.getSegments().getAdNetworkType()).append(comma);
line.append(googleAdsRow.getMetrics().getAllConversionsFromInteractionsRate()).append(comma);
I thought to convert the googleAdsRow/searchResponse to JSON/map and get column value by key, the key will be the columns name list from YAML (50 columns), but I could not convert the object to JSON.
Example:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
mapper.writeValueAsString(googleAdsRow.getMetrics())
Error:
com.fasterxml.jackson.databind.JsonMappingException: Direct
self-reference leading to cycle (through reference chain:
com.google.ads.googleads.v10.common.Metrics["unknownFields"]->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])
Notes:
The order of the columns that repent is different from the order I requested in the query and I need to list the columns to the file in the order I requested.
Additional columns (which I did not request) are returned in the query, like resourcename, id and I need to filter them.
How to do that in Java? Do you have any ideas for writing a generic code to go through all the columns?
Similar to: I cannot query my dynamodb table from aws lambda due to wrong filterexpression? and DynamoDB update error Invalid UpdateExpression: An expression attribute value used in expression is not defined
I am trying to code a way to query DynamoDB tables using partial matches on Partition Key / Sort Key in Java.
The DynamoDB table I am trying to access has a Partition key of "Type" (A restricted key word in DynamoDB, I know, but not my choice) and a Sort key of "Id". I know the "Type" but not the full Id, so I have researched the Query method using AWS SDK 2.x source code and have implemented as shown below:
DynamoDBClient dynamoDbClient = DynamoDbClient.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(StaticCredentialsProvider.create(awsCredentials))
.build();
String idKey = "wholeIdKey";
String idValue = "partialIdValue";
String typeValue = "typeValue";
Map<String, String> expressionNames = new HashMap<>();
expressionNames.put("#t", "Type");
QueryRequest request = QueryRequest.builder()
.tableName(tableName)
.keyConditionExpression("begins_with ( " + idKey + ", :" + idValue + " )
AND #t = :" + typeValue)
.expressionAttributeNames(expressionNames)
.build();
QueryResponse response = dynamoDbClient.query(request);
However, when I run this code, I get the following error message:
Exception in thread "main" software.amazon.awssdk.services.dynamodb.model.DynamoDbException:
Invalid KeyConditionExpression: An expression attribute value used in expression is not defined; attribute value: :typeValue
It's as if it's not recognizing the fact that I have told the code use the Expression Attribute Names feature to replace the "#t" with "Type" (Which is a reserved keyword in DynamoDB)
Can anyone help?
EDIT: References for code:
https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-dynamodb-src-main-java-com-example-dynamodb-Query.java.html
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ExpressionAttributeNames.html
https://www.javadoc.io/static/software.amazon.awssdk/dynamodb/2.7.14/software/amazon/awssdk/services/dynamodb/model/QueryRequest.html#expressionAttributeNames--
The name is fine, but you're prefixing both values with ':'. That causes a lookup in ExpressionAttributeValues, which you did not provide.
Never try to write dynamic values directly into the query string.
Your expressionAttributeName looks fine, but you forgot to provide a value for :typeValue so dynamoDB cannot know what to look for.
In addition to what you did, you need to add an expressionAttributeValue where you can provide proper values. See documentation here
Fixed Code for whoever wants it in the future (Thanks to #aherve and #MattTimmermans)
DynamoDBClient dynamoDbClient = DynamoDbClient.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(StaticCredentialsProvider.create(awsCredentials))
.build();
String idKey = "wholeIdKey";
String idValue = "partialIdValue";
String typeValue = "typeValue";
String typeKey = "typeKey";
Map<String, String> expressionNames = new HashMap<>();
expressionNames.put("#t", "Type");
expressionNames.put("#i", "Id");
Map<String, AttributeValue> expressionValues = new HashMap<>();
expressionValues.put(":typeName", AttributeValue.builder().s(typeValue).build());
expressionValues.put(":idName", AttributeValue.builder().s(idValue).build());
QueryRequest request = QueryRequest.builder()
.tableName(tableName)
.keyConditionExpression("#t = :typeName AND begins_with ( #i, :idName )")
.expressionAttributeNames(expressionNames)
.expressionAttributeValues(expressionValues)
.build();
response = dynamoDbClient.query(request);
I have a table of users with a primary hash key of userId. each user may/may not have a string attribute called "environment".
I would like to get all the users which have "environment"="xyz" or which do not have the "environment" attribute.
The following code will filter those users with environment=xyz, but how do I filter those items with no environment at all? Dynamo API will not allow to filter on an empty String.
AmazonDynamoDBClient client = DbClientManager.getDynamoDbClient();
ArrayList<AttributeValue> avList = new ArrayList<AttributeValue>();
avList.add(new AttributeValue().withS("xyz"));
Condition scanFilterCondition = new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(avList);
Map<String, Condition> conditions = new HashMap<>();
conditions.put("environment", scanFilterCondition);
ScanRequest scanRequest = new ScanRequest()
.withTableName("users")
.withAttributesToGet(
"userId",
"environment");
.withScanFilter(conditions);
ScanResult result = client.scan(scanRequest);
For now I just dropped the scan filter, and I do the filtering client-side. Bit is there any way to do it server side?
Thanks,
Aliza
Hope I'm not too late.
I've found useful function which you could use in the query. I did not check with ScanRequest but with QueryRequest works as charm.
QueryRequest queryRequest = new QueryRequest()
.withTableName("YouTableName")
queryRequest.setFilterExpression(" attribute_not_exists(yourAttributeName) ")
queryRequest.setExpressionAttributeValues(expressionAttributeValues)
queryRequest.setExclusiveStartKey(ifYouHave)
queryRequest.setSelect('ALL_ATTRIBUTES')
queryRequest.setExpressionAttributeNames(youNames)
attribute_not_exists(yourAttributeName) works with ":aws-sdk:1.11.11"
also you could use attribute_exists(yourAttributeName)
You need to use the NULL ComparisonOperator.
Check out this link: http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Condition.html
NOT_NULL : The attribute exists.
NULL : The attribute does not exist.
Does this work for you?
I my case also, attribute_not_exists(attribute) worked. You can refer to this question:- How to check whether an attribute is not present in dynamoDB filter expression
for more details.
I had tried with the below way and got failed,
Query query = new Query("Users");
List<Entity> results = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(PAGE_SIZE));
for (Entity user : results){
Key key = user.getKey();
System.out.println(key); // here am getting as Users(151),Users(152)...
From here am passing the "Key" as request parameter to a servlet.
In My Servlet,
String keyString = req.getParameter("key");
Key key = KeyFactory.stringToKey(keyString);
datastore.delete(key);
the error says "Cannot parse: Users(151)==" and the line of error is " Key key = KeyFactory.stringToKey(keyString);"
Can anyone suggest me an idea,
Your help will be appreciated.
Instead of passing Key as java objects to servlet, you should use this snippet to serialize
String encodedKey = KeyFactory.keyToString(key)
and this to deserialize
Key key = KeyFactory.stringToKey(encodedKey)
Is this the correct way to set and get a string value using persistance in GAE ?
I'm receiving error :
com.google.appengine.api.datastore.EntityNotFoundException: No entity was found matching the key: jsonString("jsonString")
Adding to store :
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Key key = KeyFactory.createKey("jsonString", "jsonString");
Entity urlEntity = new Entity("jsonString" , key);
urlEntity.setProperty("urlVal", urlVal);
com.google.appengine.api.datastore.Key urlEntityKey = datastore.put(urlEntity);
Returning from store :
Key key = KeyFactory.createKey("jsonString", "jsonString");
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Entity entity = datastore.get(key);
Please review the below code, also you can find more details on how to use the app engine datastore API by reviewing the documentation here. Also review the notes in the Datastore Overview document.
DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Entity e = new Entity("data");
e.setProperty("jsonString", "myValue");
Key key = ds.put(e);
Entity entity = ds.get(key);
System .out.println(entity);