I am trying to query the google datastore for something like (with pm --> persistanceManager):
String filters = "( field == 'value' || field == 'anotherValue' )";
Query query = pm.newQuery(myType.class, filters);
When I execute - I am getting back: App Engine datastore does not support operator OR.
What's the best approach in people experience for this kind of queries?
Any help appreciated!
Perform multiple queries. The Datastore, like all other databases, isn't able to efficiently execute disjunctions. Unlike other databases, it exposes this difficulty to the user, to make it clear that what you're doing isn't efficient. Your only solution is to execute multiple queries - one for each or - and combine them.
I don't know if GAE's JDO and JPA implementations support this, but using the low-level API, you can use the operator IN for this, in one query.
Query query = new Query("Issue");
List<String> list = Arrays.asList("NEW", "OPEN", "ACCEPTED");
query.addFilter("status", FilterOperator.IN, list);
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
PreparedQuery preparedQuery = datastore.prepare(query);
for (Entity entity : preparedQuery.asIterable()) {
// should iterate over 'NEW', 'OPEN' and 'ACCEPTED' issues
}
According to Google App Engine - Queries and Indexes:
Query Filters
A filter specifies a field name,
an operator, and a value. The value
must be provided by the app; it cannot
refer to another property, or be
calculated in terms of other
properties. The operator can be any of
the following: < <= == >= >
Note: The Java datastore interface does not support the != and
IN filter
operators that are implemented in the
Python datastore interface. (In the
Python interface, these operators are
implemented in the client-side
libraries as multiple datastore
queries; they are not features of the
datastore itself.)
The subject of a filter can be any
object field, including the primary
key and the entity group parent (see
Transactions).
An entity must match all filters to be
a result. In the JDOQL string syntax,
multiple filters are specified
separated by && (logical "and").
Other logical combinations of filters
(logical "or", "not") are not
supported.
Due to the way the App Engine
datastore executes queries, a single
query cannot use inequality filters
(< <= >= >) on more than one
property. Multiple inequality filters
on the same property (such as querying
for a range of values) are permitted.
See Restrictions on Queries.
Basically you're either going to have to restructure your data so that you can find what you're looking for with one condition or multiple "and" conditions or you're going to have to retrieve the data via two (or more) queries and filter/combine it in your code.
Sorry I'm late to the game.. I just ran across your question today.
Another way to "simulate" 'IN' and 'OR' behavior is to use the "low level" Datastore API. The DatastoreService supports a get() method that accepts a collection of Keys and returns a Map of all Entities that matched the passed in Keys. It's an interface, but there's a handy DatastoreServiceFactory available that will dispense a ready-to-use instance.
Unfortunately, Google decided that they don't want to promote this low-level API approach and prefer that developers use JDO or JPA, so there's no documentation available other than the JavaDocs and whatever code samples that you might find when you Google "DatastoreService".
TL
Late breaking News.. at least I'm just getting it. As I was downloading the latest Java SDK for GAE I noticed on the Release Notes that "Issue 29: Expose batch gets" was fixed in the latest release (v1.2.1). Basically it seems that we (I'm looking for the same support it seems) may have a JDO based alternative rather than having to drop down to the "low-level" Datastore API. I've just downloaded the latest Java GAE SDK so I haven't had an opportunity to test anything yet, but I wanted to give you a heads-up ASAP. I'll post anything more I learn after I've had a chance to confirm this "fix".
Please accept my apologies if I've broken StackOverflow etiquette by re-posting my comment as an answer, but I decided to do it for two reasons. Firstly because, even though it's me addressing the same issue again, IMHO this new information appears to provide a completely different "answer" to the problem. And secondly, I was concerned that the comment form might not get your attention before you'd spent a great deal of time looking into the first answer that I provided.
Next time I'll think more carefully before acting.
TL
One way to simplify having to "do it yourself" might be to use parameterized queries:
Query query = pm.newQuery(mytype.class);
query.setFilter("field == autoParam");
query.declareParameters("String autoParam");
List<String> params = myListOfThingsFieldCanBeEqualTo;
Set merged = new HashSet();
for (String f : params) {
merged.addAll(q.execute(f));
}
Contrary to cletus' answer, OR-ing works, in more recent version of App Engine anyway.
Indeed, I found OR-ing not working in App Engine 1.3.0 that I had, but according to Google App Engine - Queries and Indexes (the same source cletus referred to in his answer),
An entity must match all filters to be a result. In the JDOQL string syntax, you can separate multiple filters with || (logical "or") and && (logical "and"), although keep in mind that || can only be employed when the filters it separates all have the same field name. In other words, || is only legal in situations where the filters it separates can be combined into a single contains() filters.
I figured since his answer (and since I last updated my App Engine), App Engine must have been upgraded on this matter.
Update App Engine to 1.3.4, and the OR-ing works! Though with the limitation.
Thanks to cletus anyway:)
You can use the contains method
String filters = "( :values.contains(field) )";
Query query = pm.newQuery(myType.class, filters);
Related
How can i use jpa for query over an object (not an entity)?
For example this simple code:
String [] theList = {a,b,c,d}.
Query q = new Query("Select tl from theList tl")
Reason behind: the queries are dynamically created and executed, but the objects in the from clause of the jpql query aren't necessarily mapped tables. In some cases there are just an Object, So the actual behavior needed is modify the query during execution of the program to meet the criteria, but i don't know how to modify the query.
Edit: I Don't use native queries because of portability of code. It will be the last option.
What you're looking for is called LINQ, and unfortunately (?) it is available only in C#.
However, you can partially emulate it with Stream(s).
A Stream offers basically all the operators you need
.filter() where
.max() max
.sorted() orderby
.limit() limit
.skip() offset
.collect(groupingBy()) group by
And so on. Just give a look at the Javadoc!
I think 'JdbcTemplate' would suffice your requirement.
JdbcTemplate gives you the flexibility to run native queries and map them to a Java class.
However, you'll have to explicitly map your Java class with the column names in the database.
I have solved using joSQL. Is a powerfull opensource tool that allows you to query over java objects using "sql". It is not jpa but satisfied my needs.
Another tool i have seen that do that is called querydsl.
I have a Java, GraphQL, Hibernate, PostgreSQL, QueryDSL application that queries a very large PostgreSQL table with over 275 columns.
I've created a GraphQL schema with the 25 most popular columns as query-able fields. I'd like to add a generic "field" input type that consists of a name (the db column name + "_" + operation (like gte, gt, contains, etc.) and a value (the value the user is searching for).
So when the user (in GraphiQL) enters something like (field:{name:"age_gt", value:"50"}) as a search input to the GraphQL query, I can come up with: "age > 50".
All that works fine, but when it's time to create the Predicate and add it to the whole query ( booleanBuilder.and(new Predicate) ), I cannot figure out how to create a Predicate that just contains a raw String of SQL ("age > 50").
I've created several Predicates the "right" way using my entity POJO tied to Hibernate and the jpa generated "Q" object. But I need the ability to add one or more Predicates that are just a String of SQL. I'm not even sure if the ability exists, the documentation for QueryDSL Predicates is non-existent.
I'm thinking PredicateOperation() might be the answer, but again, no documentation and I cannot find any examples online.
My apologies for not posting code, all my stuff is behind a firewall on a different network so there's no cut and paste to my internet machine.
In Hibernate its possible to inject arbitrary SQL using custom functions or the FUNCTION-function (introduced in JPA 2.1). In QueryDSL its possible to inject arbitrary JPQL/HQL through TemplateExpressions. Combined you get:
Expressions.numberTemplate("FUNCTION('SUM', {0}), x)
However, age > 50 as expression is probably valid JPQL as well, so one can just write:
Expressions.numberTemplate("SUM(age)")
Either way, its probably best to create a visitor that traverses the GraphQL query and creates the proper expression in QueryDSL, as TemplateExpressions are prone to SQL injection.
So I am exploring how to query mongo from java, and I found several different ways of querying this, and I'm not sure if I'm missing some nuance, thus not fully understanding the queries, or they are the same.
So far I found, for java driver v3.2, this:
collection.find().projection(fields(include("x", "y"), excludeId()))
And I've been told this should work:
BasicDBobject query = new BasicDBObject("x", x).append("y", y);//This example may not compile, I haven't tried it, I'm more talking about the idea and concept.
This query would go with a find(), findOne(), distinct(), and so on.
String fields = "averageSpeed";
coll = db.getCollection(strMongoCollection);
coll.find(fields, query));
So, are both right approaches? Or its purpose is deferent
You always have the option of using the old unwieldy Bson objects yourself, but for the 3.2 driver I'd rather go with the Filters and Projections helper classes.
Thus, a simple search with some criteria can be sent as
collection.find(Filters.eq("myfield", "myvalue"))
For selecting certain fields only, you append a projection:
collection.find(Filters.eq("myfield", "myvalue"))
.projection(Projections.include("myfield", "anotherfield"))
Apart from the more elegant code of the new API, the queries do the same as the BasicDBObject-based calls.
I am having some difficulty structuring the exact Elasticsearch query that I am looking for, specifically using the java api.
It seems like if I construct a fieldsearch using the java api, I can only use a single field and a single term. If I use a querystring, it looks like I can apply an entire query to a set of fields. What I want to do is apply a specific query to one field, and another query to a different field.
This is confusing I know. This is the type of query I would like to construct
(name contains "foo" or name contains "bar") AND ( date equals today)
I am really loving Elasticsearch for it's speed and flexibility, but the docs on http://www.elasticsearch.org/ are kind of tough to parse (I noticed "introduction" and "concepts" have no links, but the API section does) If anyone has some good resources on mastering these queries, I'd love to see them. Thanks!
Sounds like a bool query with 2 must clause:
matchQuery("name", "foo bar")
rangeQuery("date").from("2013-02-05").to("2013-02-06")
Does it help?
According to the documentation here (under "unique results") and here, A JDO "default" query result will return a List, and a "unique" specified query returns null if they are empty.
So am I safe to just use .isEmpty() to check that I got rows for a normal query? Likewise for those times that a query is marked "unique", should I just use == null?
I tested and a empty result does indeed return a List with .size() of 0. So it seems my only danger of a NullPointerException lies with a "unique" query.
What other precautions should I take (if any)? Do I cover my bases acceptably with just those two types of checks? Pretty much looking for some best practice advice, and any potential gotcha's. I am pretty novice with both Java and JDO/ORM persistence.
Thanks
Right way ? A List is a List and that has a Java contract (API). JDO doesn't change that. Also the JDO spec (and DN docs) define what form of result comes from a query
http://www.datanucleus.org/products/accessplatform_3_0/jdo/jdoql_result.html