We Keep Coding

AWS SDK Java DynamoDB - Query with Expression Attribute Name - An expression attribute value used in expression is not defined

Similar to: I cannot query my dynamodb table from aws lambda due to wrong filterexpression? and DynamoDB update error Invalid UpdateExpression: An expression attribute value used in expression is not defined
I am trying to code a way to query DynamoDB tables using partial matches on Partition Key / Sort Key in Java.
The DynamoDB table I am trying to access has a Partition key of "Type" (A restricted key word in DynamoDB, I know, but not my choice) and a Sort key of "Id". I know the "Type" but not the full Id, so I have researched the Query method using AWS SDK 2.x source code and have implemented as shown below:
DynamoDBClient dynamoDbClient = DynamoDbClient.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(StaticCredentialsProvider.create(awsCredentials))
.build();
String idKey = "wholeIdKey";
String idValue = "partialIdValue";
String typeValue = "typeValue";
Map<String, String> expressionNames = new HashMap<>();
expressionNames.put("#t", "Type");
QueryRequest request = QueryRequest.builder()
.tableName(tableName)
.keyConditionExpression("begins_with ( " + idKey + ", :" + idValue + " )
AND #t = :" + typeValue)
.expressionAttributeNames(expressionNames)
.build();
QueryResponse response = dynamoDbClient.query(request);
However, when I run this code, I get the following error message:
Exception in thread "main" software.amazon.awssdk.services.dynamodb.model.DynamoDbException:
Invalid KeyConditionExpression: An expression attribute value used in expression is not defined; attribute value: :typeValue
It's as if it's not recognizing the fact that I have told the code use the Expression Attribute Names feature to replace the "#t" with "Type" (Which is a reserved keyword in DynamoDB)
Can anyone help?
EDIT: References for code:
https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-dynamodb-src-main-java-com-example-dynamodb-Query.java.html
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ExpressionAttributeNames.html
https://www.javadoc.io/static/software.amazon.awssdk/dynamodb/2.7.14/software/amazon/awssdk/services/dynamodb/model/QueryRequest.html#expressionAttributeNames--

The name is fine, but you're prefixing both values with ':'. That causes a lookup in ExpressionAttributeValues, which you did not provide.
Never try to write dynamic values directly into the query string.

Your expressionAttributeName looks fine, but you forgot to provide a value for :typeValue so dynamoDB cannot know what to look for.
In addition to what you did, you need to add an expressionAttributeValue where you can provide proper values. See documentation here

Fixed Code for whoever wants it in the future (Thanks to #aherve and #MattTimmermans)
DynamoDBClient dynamoDbClient = DynamoDbClient.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(StaticCredentialsProvider.create(awsCredentials))
.build();
String idKey = "wholeIdKey";
String idValue = "partialIdValue";
String typeValue = "typeValue";
String typeKey = "typeKey";
Map<String, String> expressionNames = new HashMap<>();
expressionNames.put("#t", "Type");
expressionNames.put("#i", "Id");
Map<String, AttributeValue> expressionValues = new HashMap<>();
expressionValues.put(":typeName", AttributeValue.builder().s(typeValue).build());
expressionValues.put(":idName", AttributeValue.builder().s(idValue).build());
QueryRequest request = QueryRequest.builder()
.tableName(tableName)
.keyConditionExpression("#t = :typeName AND begins_with ( #i, :idName )")
.expressionAttributeNames(expressionNames)
.expressionAttributeValues(expressionValues)
.build();
response = dynamoDbClient.query(request);

Mule-Creating dynamic where condition for sql query through DB connector

I need to create dynamic query wherein the where condition will be changed based on the request coming in to mule.The request will be always get with query parameter. Here goes the example:
http://localhost:8084/basePath?name=balwant&age=26 , OR
http://localhost:8084/basePath?name=balwant&age=26&gender=M
Likewise it will be dynamic.Now I need a way by which I can create a query where the WHERE condition will be added based on the query parameters in request.

Haven't tested this, but something like this. Check if the inboundProperty is there and build up the query programatically :
SELECT * FROM USERS WHERE NAME = '#[message.inboundProperties.name]' #[message.inboundProperties.gender !=null ? ' AND GENDER=' + message.inboundProperties.gender] #[message.inboundProperties.age !=null ? ' AND AGE=' + message.inboundProperties.age]

I had this in my mind of using Custom Transformers. So I used java transformer for this.
The logic looks something like this :
public class QueryBuilder extends AbstractMessageTransformer {
#Override
public Object transformMessage(MuleMessage message, String outputEncoding)
throws TransformerException {
System.out.println("Query Params : "
+ message.getInboundProperty("http.query.params").getClass()
.getName());
Map<?, ?> map = message.getInboundProperty("http.query.params");
System.out.println("Map keys : " + map.keySet());
String where = "";
for (Map.Entry<?, ?> entry : map.entrySet()) {
System.out.println(entry.getKey() + "/" + entry.getValue());
where = where+" "+entry.getKey()+"="+"'"+entry.getValue()+"'"+" and";
}
String whereCondition = where.substring(0, where.lastIndexOf(" "));
System.out.println("Where condition is : "+ where.substring(0, where.lastIndexOf(" ")));
return whereCondition;
}}
Now this returns the payload which is string type.
In DB connector, select Query type as Dynamic. After WHERE condition add #[payload].
Cheers

Above query works if the values are available in the message inbound properties. But if you want to build you SQL query with request query param values then you need to use like below (as the query param values will be available under message inbound properties http.query.param from mule 3.6.0 onward)
SELECT * FROM USERS WHERE NAME = '#[message.inboundProperties.'http.query.params'.name]' #[message.inboundProperties.'http.query.params'.gender !=null ? ' AND GENDER=' + message.inboundProperties.'http.query.params'.gender] #[message.inboundProperties.'http.query.params'.age !=null ? ' AND AGE=' + message.inboundProperties.'http.query.params'.age]

Restfb - Using the Batch Request API

This is my FQL. What it does is it receives the post based on post_id, and also returns if post was made by user or page. User or page is returned based on id of the one who posted the post, which is returned by first FQLQuery.
StringBuilder fqlQuery = new StringBuilder();
fqlQuery.append("SELECT "
+ FacebookGraphApi.FQL_STREAM_SELECT
+ " FROM stream WHERE post_id=\"");
fqlQuery.append(rmtId);
fqlQuery.append("\"");
HashMap<String, String> querys = new HashMap<String, String>();
querys.put("messages", fqlQuery.toString());
querys.put(
"users",
"SELECT "
+ FacebookGraphApi.FQL_USER_SELECT
+ " FROM user WHERE uid IN (SELECT actor_id FROM #messages)");
querys.put(
"pages",
"SELECT "
+ FacebookGraphApi.FQL_PAGE_SELECT
+ " FROM page WHERE page_id IN (SELECT actor_id FROM #messages)");
Now what i would like to do is the same using RESTFB Batch Request API
BatchRequest firstRequest = new BatchRequestBuilder(rmtId)).build();
BatchRequest secondRequest = new BatchRequestBuilder(poster_id).build();
List<BatchResponse> batchResponses =
facebookClient.executeBatch(firstRequest,secondRequest);
These are 2 batches that executes, first one returns the post, and it contains "from", that contains poster_id of the one who posted, but i dont know how to put that poster_id from first batch to second batch so it would return poster information.
Any help would be appreciated, thanks.

Just to post the answer,
first request should contain a name method on which u depend on the other request, and search on second request that depends on first ids like this.
?ids={result=name-of-method:$.data.id}

Bing Search API error using odata4j

I am trying to run Bing search API. I used odata4j and tried the code provided here:
How to use Bing search api in Java
ODataConsumer c = ODataConsumers
.newBuilder("https://api.datamarket.azure.com/Bing/Search")
.setClientBehaviors(OClientBehaviors.basicAuth("accountKey", "{your account key here}"))
.build();
OQueryRequest<OEntity> oRequest = c.getEntities("Web")
.custom("Query", "stackoverflow bing api");
Enumerable<OEntity> entities = oRequest.execute();
After I registered in the bing service, I obtained the key and placed it inside the double quotation in the above code. I got the following error:
Exception in thread "main" java.lang.RuntimeException: Expected status OK, found Bad Request. Server response:
Parameter: Query is not of type String
at org.odata4j.jersey.consumer.ODataJerseyClient.doRequest(ODataJerseyClient.java:165)
at org.odata4j.consumer.AbstractODataClient.getEntities(AbstractODataClient.java:69)
at org.odata4j.consumer.ConsumerQueryEntitiesRequest.doRequest(ConsumerQueryEntitiesRequest.java:59)
at org.odata4j.consumer.ConsumerQueryEntitiesRequest.getEntries(ConsumerQueryEntitiesRequest.java:50)
at org.odata4j.consumer.ConsumerQueryEntitiesRequest.execute(ConsumerQueryEntitiesRequest.java:40)
at BingAPI.main(BingAPI.java:20)
Caused by: org.odata4j.exceptions.UnsupportedMediaTypeException: Unknown content type text/plain;charset=utf-8
at org.odata4j.format.FormatParserFactory.getParser(FormatParserFactory.java:78)
at org.odata4j.jersey.consumer.ODataJerseyClient.doRequest(ODataJerseyClient.java:161)
... 5 more
I could not figure out the problem.

All what you need is to set your query like that %27stackoverflow bing api%27
Here is my source code:
ODataConsumer consumer = ODataConsumers
.newBuilder("https://api.datamarket.azure.com/Bing/Search/v1/")
.setClientBehaviors(
OClientBehaviors.basicAuth("accountKey",
"{My Account ID}"))
.build();
System.out.println(consumer.getServiceRootUri() + consumer.toString());
OQueryRequest<OEntity> oQueryRequest = consumer.getEntities("Web")
.custom("Query", "%27stackoverflow%27");
System.out.println("oRequest : " + oQueryRequest);
Enumerable<OEntity> entities = oQueryRequest.execute();
System.out.println(entities.elementAt(0));

You can further try different queries with different filters by keep adding name-value pairs by using .custom("Type of parameter","parameter") of the oQueryrequest object.Say you want to search for indian food but you want only small square images.
ODataConsumer consumer = ODataConsumers
.newBuilder("https://api.datamarket.azure.com/Bing/Search/v1/")
.setClientBehaviors(
OClientBehaviors.basicAuth("accountKey",
"YOUR ACCOUNT KEY"))
.build();
System.out.println(consumer.getServiceRootUri() + consumer.toString());
OQueryRequest<OEntity> oQueryRequest = consumer.getEntities("Image")
.custom("Query", "%27indian food%27");
oQueryRequest.custom("Adult", "%27Moderate%27");
oQueryRequest.custom("ImageFilters", "%27Size:Small+Aspect:Square%27");
System.out.println("oRequest : " + oQueryRequest);
Enumerable<OEntity> entities = oQueryRequest.execute();
int count = 0;
Iterator<OEntity> iter = entities.iterator();
System.out.println(iter.next());

How to exclude retweets from my search query results Options

I'm trying to search tweets using search method in twitter4j. My code is as
follows,
public List<Tweet> searchTweets(Query searchQuery) {
QueryResult queryResult = twitter.search(searchQuery);
return queryResult != null ? queryResult.getTweets() : new
ArrayList<Tweet>(0);
}
How do I exclude retweets from my search query results

I was looking for how to exclude the replies on the query search, so I found this topic.
To exclude retweets, this worked well for me:
Query query = new Query("from:"+twitterAccount + " +exclude:retweets");

I would rather comment on this but I can't yet so I'll post this as an answer. You need to edit the Query by appending parameters to it. The source code for Query.java can be found in the examples folder in the twitter4j folder.
public Query(String query) {
this.query = query;
}
Check out Twitter search (atom) API - exclude retweets. The search basis is different but the concept is the same, which is appending +exclude:retweets to the end of your string query. Try it out and do let me know if it works!

Query query = new Query(yourQuery + " -filter:retweets");
source
PS. I don't know the difference betweeen +exclude:retweets and -filter:retweets; both seem to do the job similarly.

I used a different approach than above. See my code below:
List<Status> tweets = result.getTweets();
for (Status tweet : tweets )
{
if( !tweet.isRetweet() )
// processing ....
}
It works for me. However, there might be efficiency differences between these approaches.

You can use set in the configuration builder the option setIncludeRTsEnabled to false, so is going to remove the retweets.
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.setIncludeRTsEnabled(false);
Twitter twitter = new TwitterFactory(builder.build()).getInstance();