I'm using async client from java library com.azure:azure-cosmos:4.3.0 to connect and query azure cosmosdb(SQL) in direct mode. All the documents in the collection have the following structure
{
"id":"string",
"timestamp":123456789,
"tags":["tag1", "tag2"]
}
What I want to do is find all documents that have any matching tags from a given list of input tags. What is the best way to do this?
What I've tried and not able to make it work -
I inserted four documents to test with and then I ran this query in azure portal - SELECT DISTINCT VALUE CONTAINER_ALIAS FROM CONTAINER_ALIAS JOIN CONTAINER_ALIAS_tags IN CONTAINER_ALIAS.tags WHERE CONTAINER_ALIAS_tags IN ( "tag1", "tag2" ) AND ( CONTAINER_ALIAS.timestamp BETWEEN 1599773389000 AND 1602365389000 ). This gave me 3 documents as response which is right because out of 4 test documents, one of the document doesn't have "tag1" or "tag2"
Then I tried to generate the same query using the client library and it gets to SELECT DISTINCT VALUE CONTAINER_ALIAS FROM CONTAINER_ALIAS JOIN CONTAINER_ALIAS_tags IN CONTAINER_ALIAS.tags WHERE CONTAINER_ALIAS_tags IN ( "tag1", "tag2" ) AND ( CONTAINER_ALIAS.timestamp BETWEEN #timestamp_START AND #timestamp_End )
This query is passed to the database as follows
List<SqlParameters> sqlParameters = Arrays.asList(
new SqlParameter("#timestamp_START", 1599773389000),
new SqlParameter("#timestamp_End", 1602365389000)
);
SqlQuerySpec sqlQuerySpec = new SqlQuerySpec(query, sqlParameters)
long pageSize=20
container.queryItems(sqlQuerySpec, MyCustomEntity.class)
.byPage(continuationToken, pageSize)
.take(1)
.next()
But this only returns 1 document in result. But it does return a continuation token {"lastHash":"","sourceToken":"{\"token\":null,\"range\":\"{\\\"min\\\":\\\"05C1CFFFFFFFF8\\\",\\\"max\\\":\\\"05C1D7FFFFFFFC\\\",\\\"isMinInclusive\\\":true,\\\"isMaxInclusive\\\":false}\"}"}
I have used the client library with WHERE but without JOIN and I was able to make it work properly. The result would include pageSize number of documents and it would return a proper continuation token if there are more documents available. However, when I use join I'm not getting all the results
Other thing I noticed was the RU when I run from azure portal for the above query is around 46 but when I print the logs from the client library the RU is around 2. Not sure if this information is helpful in anyway to figure what I'm doing wrong
Please let me know what I'm doing wrong or is there a better way to achieve this
Related
I have a code which executes a flexible search. When i am calling that code locally to search data it gives expected output but when I try to call it using REST API (Through controller) it gives error as could not translate value expression 'session.currentCountry' but i am not even using session or session country anywhere in flexible search. what can be issue?
here is code
query
select {rr.pk} from {returnrequest as rr join order as r on {r.pk} = {rr.order} join orderstatus as os on {r.status}={os.pk} join basestore as bs on {bs.pk}={r.store} join returntype as rt on {rr.returntype} = {rt.pk} join paymentmode as pm on {pm.pk}={r.paymentmode} join returnstatus as rs on {rs.PK}={rr.status}} where {os.code} NOT in ('PENDING_PAYMENT','ON_VALIDATION', 'CREATED', 'IN_PROGRESS', 'ORDER_SPLIT','CANCELLED') and {rr.creationtime} >= ?returnCreationDateFrom and {rr.creationtime} < ?returnCreationDateTo and {bs.uid} in (?market) and {rt.code} in (?returnType) and {pm.code} = 'SplittedPaymentMode' and {rr.totalRefund} != 0.0 and {rs.code}!='RECEIVED' group by {rr.pk} ";
code
FlexibleSearchQuery query = new FlexibleSearchQuery(flexiQuery.toString());
query.addQueryParameter("market", market);
query.addQueryParameter("returnType", returnType);
query.addQueryParameter("returnCreationDateTo", DateUtils.addDays(returnCreationDateTo, 1));
query.addQueryParameter("returnCreationDateFrom", returnCreationDateFrom);
SearchResult<ReturnRequestModel> results = search(query);
same flow is running properly in local but giving issues in remote instace.
here is error
ERROR PaymentWsController de.hybris.platform.servicelayer.search.exceptions.FlexibleSearchException: could not translate value expression 'session.currentCountry'
19:37:31.834 [hybrisHTTP6] ERROR de.hybris.platform.jalo.flexiblesearch.FlexibleSearch - Flexible search error occured...
This error coming from research restriction. It looks your user hasn't got some session variables and a restriction try to use it. You can check this question answer.
The issue is that you have a SearchRestriction, Search Restriction is a set of rules which is applied on Flexible Search Query in order to limit the search results or to filter search results based on specific conditions.
You can see SearchRestrictions in Backoffice -> System -> Personalization, find the related one that adds session.currentCountry to your flexible search and either disable it or use impersonationService to execute your query inside a context with a site.
Hope this helps
Although others answers on the post are correct I just want to add my answer which helped to solve the issue.
The issue was with search restriction.
I just set current user as admin to bypass the restrictions. i.e.
userService.setCurrentUser(userService.getAdminUser());
I just added it before executing the search and error got resolved.
we can also set user as admin in flexiquery itself so that search restrictions can be avoided.
here is how.
query.setUser(userService.getAdminUser());
so both ways it can be done.
I am trying to use the DynamoDBScanExpression withLimit of 1 using Java aws-sdk version 1.11.140
Even if I use .withLimit(1) i.e.
List<DomainObject> result = mapper.scan(new DynamoDBScanExpression().withLimit(1));
returns me list of all entries i.e. 7. Am I doing something wrong?
P.S. I tried using cli for this query and
aws dynamodb scan --table-name auditlog --limit 1 --endpoint-url http://localhost:8000
returns me just 1 result.
DynamoDBMapper.scan will return a PaginatedScanList - Paginated results are loaded on demand when the user executes an operation that requires them. Some operations, such as size(), must fetch the entire list, but results are lazily fetched page by page when possible.
Hence, The limit parameter set on DynamoDBScanExpression is the maximum number of items to be fetched per page.
So in your case, a PaginatedList is returned and when you do PaginatedList.size it attempts to load all items from Dynamodb, under the hood the items were loaded 1 per page (each page is a fetch request to DynamoDb) till it get's to the end of the PaginatedList.
Since you're only interested in the first result, a good way to get that without fetching all the 7 items from Dynamo would be :
Iterator it = mapper.scan(DomainObject.class, new DynamoDBScanExpression().withLimit(1)).iterator();
if ( it.hasNext() ) {
DomainObject dob = (DomainObject) it.next();
}
With the above code, only the first item will fetched from dynamodb.
The take away is that : The limit parameter in DynamoDBQueryExpression is used in the pagination purpose only. It's a limit on the amount of items per page not a limit on the number of pages that can be requested.
In Neo4J, I want to use the bolt protocol.
I installed the 3.1 version of Neo4J.
I my Java project, that already works well with normal HTTP Rest API of Neo4J, I integrate with Maven the needed drivers and achieve to perform request with BOLT.
The problem is everywhere you make a search about bolt they give example like this one :
MATCH (a:Product) return a.name
But I don't want the name, I want all the data of all product, what ever i know or not before what are these columns, like here:
MATCH (a:Product) return * --> here I retrieve only the ids of nodes
I found there https://github.com/neo4j-contrib/neo4j-jdbc/tree/master/neo4j-jdbc-bolt we can "flatten" the result but it seems to not work or I didn't understand how it works:
GraphDatabase.driver( "bolt://localhost:7687/?flatten=-1", AuthTokens.basic( "neo4j", "......." ) );
I put the ?flatten=-1 at the end of my connection address... but that changed nothing.
Anyone can help? Or confirm it's not possible or not working ?
Thanks
Ok I understood my error, I didn’t dig enough in the object returned. So used to have a JSON formatted response, I didn’t see that I have to search in the StatementResult object to find the wanted object with its properties. In fact Eclipse in the “expressions” shows “in fly” only the ids, but inside the object data are there.
Record oneRecord = rs.next();
String src = oneRecord.get("m").get("source");
That way I can reconstruct my object
How do I retrieve an OrientDB Document/Object or Graph object using its Record ID? (Language: Java)
I'm referring to http://orientdb.com/docs/2.0/orientdb.wiki/Tutorial-Record-ID.html and Vertex.getId() / Edge.getId() methods.
It is just like an SQL query "SELECT * from aTable WHERE ID = 1".
Usage/purpose description: I want to store the generated ID after it is created by OrientDB, and later retrieve the same object using the same ID.
(1) I'd suggest using OrientDB 2.1, and its documentation, e.g. http://orientdb.com/docs/2.1/Tutorial-Record-ID.html
(2) From your post, it's unclear to me whether you need help obtaining the RID from the results of a query, or retrieving an object given its RID, so let me begin by mentioning that the former can be accomplished as illustrated by this example (in the case of an INSERT query):
ODocument result=db.command(new OCommandSQL(<INSERTQUERY>)).execute();
System.out.println(result.field("#rid"));
Going the other way around, there are several approaches. I have verified that the following does work using Version 2.1.8:
OrientGraph graph = new OrientGraph("plocal:PATH_TO_DB", "admin", "admin");
Vertex v = graph.getVertex("#16:0");
An alternative and more generic approach is to construct and execute a SELECT query of the form SELECT FROM :RID, along the lines of this example:
List<ODocument> results = db.query(new OSQLSynchQuery<ODocument>("select from " + rid));
for (ODocument aDoc : results) {
System.out.println(aDoc.field("name"));
}
(3) In practice, it will usually be better to use some other "handle" on OrientDB vertices and edges in Java code, or indeed when using any of the supported programming languages. For example, once one has a vertex as a Java Vertex, as in the "Vertex v" example above, one can usually use it.
I have a Spring Batch project running in Spring Boot that is working perfectly fine. For my reader I'm using JdbcPagingItemReader with a MySqlPagingQueryProvider.
#Bean
public ItemReader<Person> reader(DataSource dataSource) {
MySqlPagingQueryProvider provider = new MySqlPagingQueryProvider()
provider.setSelectClause(ScoringConstants.SCORING_SELECT_STATEMENT)
provider.setFromClause(ScoringConstants.SCORING_FROM_CLAUSE)
provider.setSortKeys("p.id": Order.ASCENDING)
JdbcPagingItemReader<Person> reader = new JdbcPagingItemReader<Person>()
reader.setRowMapper(new PersonRowMapper())
reader.setDataSource(dataSource)
reader.setQueryProvider(provider)
//Setting these caused the exception
reader.setParameterValues(
startDate: new Date() - 31,
endDate: new Date()
)
reader.afterPropertiesSet()
return reader
}
However, when I modified my query with some named parameters to replace previously hard coded date values and set these parameter values on the reader as shown above, I get the following exception on the second page read (the first page works fine because the _id parameter hasn't been made use of by the paging query provider):
org.springframework.dao.InvalidDataAccessApiUsageException: No value supplied for the SQL parameter '_id': No value registered for key '_id'
at org.springframework.jdbc.core.namedparam.NamedParameterUtils.buildValueArray(NamedParameterUtils.java:336)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.getPreparedStatementCreator(NamedParameterJdbcTemplate.java:374)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:192)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:199)
at org.springframework.batch.item.database.JdbcPagingItemReader.doReadPage(JdbcPagingItemReader.java:218)
at org.springframework.batch.item.database.AbstractPagingItemReader.doRead(AbstractPagingItemReader.java:108)
Here is an example of the SQL, which has no WHERE clause by default. One does get created automatically when the second page is read:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p
On the second page, the sql is modified to the following, however it seems that the named parameter for _id didn't get supplied:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p WHERE id > :_id
I'm wondering if I simply can't use the MySqlPagingQueryProvider sort keys together with additional named parameters set in JdbcPagingItemReader. If not, what is the best alternative to solving this problem? I need to be able to supply parameters to the query and also page it (vs. using the cursor). Thank you!
I solved this problem with some intense debugging. It turns out that MySqlPagingQueryProvider utilizes a method getSortKeysWithoutAliases() when it builds up the SQL query to run for the first page and for subsequent pages. It therefore appends and (p.id > :_id) instead of and (p.id > :_p.id). Later on, when the second page sort values are created and stored in JdbcPagingItemReader's startAfterValues field it will use the original "p.id" String specified and eventually put into the named parameter map the pair ("_p.id",10). However, when the reader tries to fill in _id in the query, it doesn't exist because the reader used the non-alias removed key.
Long story short, I had to remove the alias reference when defining my sort keys.
provider.setSortKeys("p.id": Order.ASCENDING)
had to change to in order for everything to work nicely together
provider.setSortKeys("id": Order.ASCENDING)
I had the same issue and got another possible solution.
My table T has a primary key field INTERNAL_ID.
The query in JdbcPagingItemReader was like this:
SELECT INTERNAL_ID, ... FROM T WHERE ... ORDER BY INTERNAL_ID ASC
So, the key is: in some conditions, the query didn't return results, and then, raised the error above No value supplied for...
The solution is:
Check in a Spring Batch decider element if there are rows.
If it is, continue with chunk: reader-processor-writer.
It it's not, go to another step.
Please, note that they are two different scenarios:
At the beginning, there are rows. You get them by paging and finally, there are no more rows. This has no problem and decider trick is not required.
At the beginning, there are no rows. Then, this error raised, and the decider solved it.
Hope this helps.