I am writing a query but it always says "No matching index found". I don't know why. My code is as below:
Query query = pm.newQuery(Classified.class);
query.setFilter("emp_Id == emp");
query.setOrdering("upload_date desc");
query.declareParameters("String emp");
List<Classified> results = (List<Classified>)query.execute(session.getAttribute("emp_Id").toString());
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
<datastore-index kind="Classified" ancestor="false">
<property name="emp_Id" direction="asc" />
<property name="category" direction="asc" />
<property name="upload_date" direction="desc" />
</datastore-index>
</datastore-indexes>
I have added the above index, but it did not help.
I believe you need to configure a Datastore Index. There's probably one already generated for you in Eclipse at WEB-INF/appengine-generated/datastore-indexes-auto.xml that you just need to copy to WEB-INF/datastore-indexes.xml and deploy again.
Because this needs to be somewhere on the internet...
I kicked myself when I found this out
The error is you do not have a index matching what the query would like to perform. You can have multiple indexes for each entity.
In the Logcat, error, it will tell you exactly what index to set and what order the elements need to be.
ie, if the error says it wants (it wont be nicely formatted):
<datastore-index kind="Classified" ancestor="false">
<property name="category" direction="desc" />
<property name="upload_date" direction="desc" />
</datastore-index>
then Project -> war -> WEB-INF -> appengine-generated -> datastore-indexes-auto.xml and add exactly that. Then, redeploy the project.
Next go into your Google Cloud Console and look at Datastore -> indexes. It should say that the index is being prepared (This goes quicker if you can kill all apps connected and shut down the instance in the console).
Once this has moved into the list of other indexes, rerun the your application and it wont error out with regards to the index anymore.
Go get it Gentlemen/Ladies
The index you define must hold all possible results in the order they will be returned. Your query asks for a particular emp_Id, ordered by upload_date, but your index is ordered primarily by category.
Try removing the category line from your index definition, or swapping the order of category and upload_date, to make upload_date the primary sort order for the index. If another part of your code relies on the category line, you may have to make two separate indices (which incurs some computational cost).
Edit: see comment below by Nick Johnson re. extra parameters.
I am running into this issue at the moment when doing a single property query such as:
const query = datastore
.createQuery('Emailing_dev')
.filter('status', '=', 'Scheduled')
In my case, I should not be getting any errors, however I get Error 9: No matching index found.
If I defined the single property index twice in the yaml, it works:
indexes:
- kind: Emailing_dev
properties:
- name: status
- name: status
but this for sure must be a bug..!!
Related
I'd like to configure LocalDatastoreServiceTestConfig such that queries will fail if a compound index is needed (e.g., a query with a sort on multiple properties). Is there a way to do this?
I tried new LocalDatastoreServiceTestConfig().setNoIndexAutoGen(true) but it had no effect.
(There is a corresponding way to do this with the Python SDK.)
I assume by "fail" you mean "throw an exception" or something similar.
If so, you should set the autoGenerate attribute in your WEB-INF/datastore-indexes.xml to false.
Example WEB-INF/datastore-indexes.xml:
<datastore-indexes autoGenerate="false">
</datastore-indexes>
Setting autoGenerate to false will make a query that requires a composite index throw an exception.
Example code:
try {
Query q = new Query("Action")
.addSort("encrypter", Query.SortDirection.ASCENDING)
.addSort("requester", Query.SortDirection.ASCENDING)
.addSort("time", Query.SortDirection.DESCENDING);
//...snip...
} catch (Exception e) {
log.severe(e.toString());
}
I tested this and got an exception logged as expected:
SEVERE: com.google.appengine.api.datastore.DatastoreNeedIndexException: Query com.google.appengine.api.datastore.dev.LocalCompositeIndexManager$IndexComponentsO
nlyQuery#f9f81ad3 requires a composite index that is not defined. You must update C:\appengine-java-sdk\dev\core1\war\WEB-INF\datastore-indexes.xml or enable au
toGenerate to have it automatically added.
The suggested index for this query is:
<datastore-index kind="Action" ancestor="false" source="manual">
<property name="encrypter" direction="asc"/>
<property name="requester" direction="asc"/>
<property name="time" direction="desc"/>
</datastore-index>
For more information, see datastore-indexes.xml reference.
Ok, see this picture in Google Cloud:
It said "Below are the composite indexes for this application. These indexes are managed in your app's index configuration file."
And see this following code:
public static long getPropertyMaxIDStronglyConsistent(String entityName, String propertyName, Key parentKey){
// Order alphabetically by property name:
//Key parentKey=KeyFactory.createKey(parentEntityName, parentName);
Query q = new Query(entityName, parentKey).setAncestor(parentKey)
.addSort(propertyName, SortDirection.DESCENDING);
//List<Entity> results = datastore.prepare(q).asList(FetchOptions.Builder.withDefaults());
List<Entity> results = datastore.prepare(q)
.asList(FetchOptions.Builder.withLimit(5));
if(results.size()>0){
Entity e=results.get(0);
long maxID=(Long)e.getProperty(propertyName);
return maxID;
}
else{
return 0;
}
}
Suppose that we are running this function1 getPropertyMaxIDStronglyConsistent("EnglishClass", "ClassNo", KeyFactory.createKey("EnglishClassParent", "EnglishClassParentName")).
What I found out that, the function1 is not gonna work if the kind "EnglishClass" does not appear in Indexes Table with "serving" status.
I didn't know what I did but after I was struggling for a few hours then suddenly the Index "EnglishClass" appeared. When the "EnglishClass" appeared with "serving" status, the app can work as normal without any problem.
My questions are
What are composite indexes?
Why didn't it appear immediately after running the function1?
What does "serving" status mean?
How to force composite indexes to appear in Google Cloud?
Extra:
In the datastore-indexes-auto.xml I have
<datastore-indexes autoGenerate="true">
<datastore-index kind="EnglishClass" ancestor="true" source="auto">
<property name="ClassNo" direction="desc"/>
</datastore-index>
</datastore-indexes>
But it still did not work
Indexes for the App Engine datastore are described in these docs and these docs(Java 7, but the principles are the same for Java 8).
A composite index is an index that comprises of more than one property of a model: for example an index that sorts a model by Model.name, then Model.creationDate. Composite indexes are used by queries that need to access datastore records in the order described by the query.
Some indexes must be declared in the datastore-indexes.xml file - see here
Serving status means that the index is ready for use; when an index is first uploaded App Engine must build the index, and until the index is built queries that use the building index will throw an exception. So it can be helpful to update indexes before deploying the code that requires them.
Configure your app to automatically configure indexes.
Edit: I want to change the question a little :-)
I want to save a SaveGame Objet in a SQL database with Hibernate XML. All i want to is to save it. My problem is taht i cant find a way how to do it. I searched now for 3 hours on the internet and didnot find a solution.
All i want to do is to save a integer array as a normal value into my database. I found during my search i can do this with a blob column. But nowhere on this great internet i found the way how to do it. I simply search for a person that can show me how to do it right. Becasue all my tried failed. And i am frustrated now.
My Class that i want to save looks like this:
public class SaveGame implements IDBObject {
int id;
String username;
int basic_game_id;
List<Integer> route = new ArrayList<Integer>();
/*getter + setter */
The SaveGame table should look like this:
ID | username | basic_game_id | route |
1 | Maxi | 1 | xxxxx |
I hope this time its clear what i want to do. I dont have to save every single array entry. I just want to save the whole array in the route column. And i cant find out how i can do this!
Old Question:
I try to map a Integer Array to my database with Hibernate.
This is how my Class looks
public class SaveGame implements IDBObject {
int id;
String username;
int basic_game_id;
List<Integer> route = new ArrayList<Integer>();
/*getter + setter */
My mapping xml looks like this:
<class name="com.travelsales.base_classes.SaveGame" table="SAVEGAME">
<meta attribute="class-description">
This class contains the Save games.
</meta>
<id name="id" type="int" column="id">
<generator class="native"></generator>
</id>
<property name="basic_game_id" type="int" column="basic_game_id"></property>
<property name="username" type="string" column="username"></property>
<list name="route" column="route">
<element type="int"></element>
</list>
</class>
Well i cant find the mistake.
My problem is that i must have an error in the xml mapping part. If i delete the list everything works fine. I want to save integer array or integer list in my database. And i think i have a mistake in the way i do it.
With this code i get the following error:
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.travelsales.controller.MainController.main(MainController.java:42)
Caused by: org.hibernate.InvalidMappingException: Could not parse mapping document from resource Hibernate Config/mapping.hbm.xml
Not sure what you are trying to achieve, do you want to the route to be stored in separate table (to be searchable, etc) or are you happy to have it in same row as your SaveGame.
If your case is really as simple as you presented I would recommend storing in in the same row to save on fetching the int values from the join.
Map the property to a blob column and all is done. I am not using the xml configuration so dont know the syntax but with JPA annotations it is solved by the code:
#Lob
List<Integer> route;
If your hibernate version do not handle interfaces (you will get error messages), change the property declaration form List to ArrayList (which is serializable, so it can be saved as bytes in the blob column).
Well, After all my trys and after all my study from online Documents. Maybe one day another person will have the same problem I had.
Its very simple i was searching for Hibernate types and I found this sentence:
If the type you have is no Basic Hibernate Type and you only want to save it in 1 Column dont write in the Hibernate XML a type and let hibernate decide which typ to use.
If you make your property like normal with name column and type. Hibernate will try if the type is possible for yoru input in my case a int Array. And Hibernate will answer you that the type is unknown. If you set the type to byte[] hibernate will answer you : Do not know to convert int array to byte array.
The Solution is simple just do not set type. Write the property like this:
<property name="route" column="route"/>
Hibernate will now look up which type you will need and set it for you. And everything works fine, and like you want to do it. On the internet you will find alot of resources for array, set and list. All this things will provide a new Tabel special for you array.
If you want to do this go check the internet its full with, helpfull examples. If you just want to save Objects and Hibernate dont know the type just dont set type in your XML.
I have been trying to use Query.setOrdering(xxxx desc) but AppEngine gives ServerError : Unable to process the request.
Admin Console :
com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found.
The suggested index for this query is:
<datastore-index kind="Chat_messages" ancestor="false" source="manual">
<property name="FROM_TO" direction="asc"/>
<property name="TIME" direction="desc"/>
</datastore-index>
My code in servlet :
Query q=pm.newQuery(Chat_messages.class);
q.setFilter("FROM_TO == email_emailto || FROM_TO == emailto_email");
q.declareParameters("String email_emailto,String emailto_email");
q.setRange(0, 50);
q.setOrdering("TIME desc");
With the Python runtime, the process of running the app locally with the dev server generates the index file.
Are you actually getting an error about not having the correct index? The index that you have put in your datastore-index.xml is a single property index. This means that it is automatically built by Datastore and will not show up in your console.
You can see this page which describes which queries you can perform without needing to upload any indexes.
I've been working with Java JDBC (mainly in desktop apps) for a long time and I'm realizing that many operations that I perform could be improved and simplified. For example:
Hardcoding SQL statements directly into Java is not very practical.
Replacing variables with "?" in JDBC is nice but it would be much better to use real variable names like "USER-NAME" or something like that.
Executing multiple update statements at once would be very cool.
In an effort to improve JDBC, I decided to write my own tool but before I reinvent the wheel I would like to know if there is any Java utility that is able to:
Read and execute .sql scripts, preferably stored inside the JAR of my application.
Define variables in these scripts, preferably with real names and not with the "?" character.
Run query (SELECT) and update (CREATE, INSERT, DELETE, ...) statements from these scripts.
Execute multiple update statements in one single method call. This could let me, for example, run DDL and DML scripts to initialize databases.
I know about JDBC ScriptRunner but it's not complete enough. Is there something better out there? I honestly think such a tool like this would be very useful.
Note: I don't mind to import a lib.
I use Warework just for that. It's a big JAR but I think it does what you're looking for. I'll show you how it works:
1- Create this directory structure in the source folder of your project:
/META-INF/system/statement/sql
2- In the "/META-INF/system" directory, create a file named "pool-service.xml" with this content:
<?xml version="1.0" encoding="UTF-8"?>
<proxy-service xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://repository.warework.com/xsd/proxyservice-1.0.0.xsd">
<clients>
<client name="c3p0-client" connector="com.warework.service.pool.client.connector.C3P0Connector">
<parameter name="driver-class" value="com.mysql.jdbc.Driver" />
<parameter name="jdbc-url" value="jdbc:mysql://host:port/database-name" />
<parameter name="user" value="the-user-name" />
<parameter name="password" value="the-password" />
<parameter name="connect-on-create" value="true" />
</client>
</clients>
</proxy-service>
In this file, replace the values of the parameters with the ones you need to connect with your database. Keep "connect-on-create" equals to "true".
3- Write your .sql scripts in the "/META-INF/system/statement/sql" directory. You can write SELECT statements like this (one statement per file):
find-user.sql
SELECT * FROM HOME_USERS A WHERE A.ID = ${USER_ID}
and UPDATE statements like this (one or multiple statements per file):
create-user.sql
INSERT INTO HOME_USERS (ID, NAME) VALUES (${USER_ID}, ${USER_NAME});
INSERT INTO ACTIVE_USERS (ID) VALUES (${USER_ID});
4- To connect with the database, perform this:
// "Test.class" must be any class of your project (the same project where /META-INF/system directory exists).
// Do not change "full" and "relational-database" strings.
// If you change "system" for "test", then the directory will be /META-INF/test.
RDBMSView ddbb = (RDBMSView) ScopeFactory.createTemplate(Test.class, "full", "system").getObject("relational-database");
// Connect with the database.
ddbb.connect();
5- Run a SELECT statement from a .sql file like this:
// Values for variables in the SELECT statement.
Hashtable values = new Hashtable();
// Set variables to filter the query.
values.put("USER_ID", new Integer(8375));
// Read '/META-INF/system/statement/sql/find-user.sql', replace variables and run.
// -1 values are for pagination (first -1 is the page, second -1 is the max rows per page).
ResultSet result = (ResultSet) ddbb.executeQueryByName("find-user", values, -1, -1);
6- Run UPDATE statements from a .sql file like this:
// Values for variables in the UPDATE statements.
Hashtable values = new Hashtable();
// Set variables for the update statement.
values.put("USER_ID", new Integer(3));
values.put("USER_NAME", "Oompa Loompa");
// Read '/META-INF/system/statement/sql/create-user.sql', replace variables and run.
// ';' is the character that separates each statement.
ddbb.executeUpdateByName("create-user", values, new Character(';'));
The RDBMSView class provides these methods plus connect/disconnect, commit, rollback, ... You can also run statements directly from String objects.
About named parameters here are a couple of solutions I found around
http://www.javaworld.com/javaworld/jw-04-2007/jw-04-jdbc.html?page=1
Named parameters in JDBC
Extracting statements from a script is a quite simple task (but not trivial) if you limit yourself to SELECT/UPDATE/INSERT/DELETE statements. Just split at any ";" character wich isn't between two "'" characters. It becomes a more complex parsing problem if you admit also CREATE TRIGGERS or CREATE PROCEDURE etc. to your script since they have have ";" as part of their sintax.
I personally think it's safer to keep DDL out of the equation and manage database creation with some external tool like Liquibase