We Keep Coding

Is there a way to configure tests for the Java Google App Engine SDK to fail if a compound index is needed?

I'd like to configure LocalDatastoreServiceTestConfig such that queries will fail if a compound index is needed (e.g., a query with a sort on multiple properties). Is there a way to do this?
I tried new LocalDatastoreServiceTestConfig().setNoIndexAutoGen(true) but it had no effect.
(There is a corresponding way to do this with the Python SDK.)

I assume by "fail" you mean "throw an exception" or something similar.
If so, you should set the autoGenerate attribute in your WEB-INF/datastore-indexes.xml to false.
Example WEB-INF/datastore-indexes.xml:
<datastore-indexes autoGenerate="false">
</datastore-indexes>
Setting autoGenerate to false will make a query that requires a composite index throw an exception.
Example code:
try {
Query q = new Query("Action")
.addSort("encrypter", Query.SortDirection.ASCENDING)
.addSort("requester", Query.SortDirection.ASCENDING)
.addSort("time", Query.SortDirection.DESCENDING);
//...snip...
} catch (Exception e) {
log.severe(e.toString());
}
I tested this and got an exception logged as expected:
SEVERE: com.google.appengine.api.datastore.DatastoreNeedIndexException: Query com.google.appengine.api.datastore.dev.LocalCompositeIndexManager$IndexComponentsO
nlyQuery#f9f81ad3 requires a composite index that is not defined. You must update C:\appengine-java-sdk\dev\core1\war\WEB-INF\datastore-indexes.xml or enable au
toGenerate to have it automatically added.
The suggested index for this query is:
<datastore-index kind="Action" ancestor="false" source="manual">
<property name="encrypter" direction="asc"/>
<property name="requester" direction="asc"/>
<property name="time" direction="desc"/>
</datastore-index>
For more information, see datastore-indexes.xml reference.

Can't persist emojis with mysql and hibernate

I have actually found this problem multiple times on Stackoverflow, but the solutions would not help me.
I have a chat module in my android app and want to persist the messages in my server db, which works fine until special characters like emojis appear.
ERROR: Incorrect string value: '\xF0\x9F\x98\x81' for column 'message' at row 1
...
...
Caused by: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\x81' for column 'message' at row 1
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1084)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4232)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4164)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2838)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2082)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2334)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2262)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2246)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:187)
... 23 more
My environment is:
-Mysql 5.6
-Tomcat 8.0.8
-Hibernate 4.3.5
-JDK 1.8.0_05
This is the used table with the column in question, 'message':
These are my properties in the persistence.xml (version 2.1):
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver" />
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost:3306/gamedb?useUnicode=true&characterEncoding=UTF-8" />
<property name="javax.persistence.jdbc.user" value="*********" />
<property name="javax.persistence.jdbc.password" value="**************" />
<property name="hibernate.hbm2ddl.auto" value="update" />
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5Dialect" />
<property name="hibernate.connection.useUnicode" value="true" />
<property name="hibernate.connection.characterEncoding" value="utf8" />
Now I tried the following solutions without effect:
-Change datatype of 'message' from varchar to longtext
-Change collation of 'message' to utf8mb4
-Change collation of table to utf8mb4
-Append url with "?useUnicode=true&characterEncoding=UTF-8"
-Set character-set-server of mysql to utf8mb4
I think the emoji is correctly transmitted to the server, before it persists the message it broadcasts it back to the app and it gets correctly displayed.

I once had the same problem. I don't know a pretty solution but this had worked for me.
After I created the Session object I changed the connection collation by hand:
s.doReturningWork(new ReturningWork<Object>() {
#Override
public Object execute(Connection conn) throws SQLException
{
try(Statement stmt = conn.createStatement()) {
stmt.executeQuery("SET NAMES utf8mb4");
}
return null;
}
});

if you are using hibernate with c3p0, you can use c3p0 config connectionCustomizerClassName whitch you can set to a class doing work with connections c3p0 got.
example:
hibernate.cfg.xml
<property name="hibernate.c3p0.connectionCustomizerClassName">com.hzmoyan.newlyappserver.db.C3p0UseUtf8mb4</property>
C3p0UseUtf8mb4 class
public class C3p0UseUtf8mb4 extends AbstractConnectionCustomizer{
#Override
public void onAcquire(Connection c, String parentDataSourceIdentityToken)
throws Exception {
super.onAcquire(c, parentDataSourceIdentityToken);
try(Statement stmt = c.createStatement()) {
stmt.executeQuery("SET NAMES utf8mb4");
}
}
}

The solution is to use utf8mb4 rather than utf8 in MySQL. The blog post I linked to explains how to do just that.

I just found a nice little hack to get this to work without having to add any code. If you set the validation query to be SET NAMES utf8mb4 then it will execute this when it gets the connection, therefore setting the parameter each time a new connection is retrieved. You also need test on borrow to make this work.
Note I have found that this can sometimes take a few seconds to work after a restart so could have failures if you have records waiting to be processed on startup
So in your application.properties you would add something like
datasource.test-on-borrow=true
datasource.validation-query=SET NAMES utf8mb4

I was able to fix the issue by providing the following in my connection URL;
useUnicode=true&characterEncoding=UTF-8
Example;
jdbc:mysql://localhost/database?useUnicode=true&characterEncoding=UTF-8

Maybe you need to modify hibernate configuration as
<property name="hibernate.connection.characterEncoding" value="utf8mb4" />

If your datasource is org.apache.commons.dbcp.BasicDataSource you can set connectionInitSqls parameter
<bean id="dataSource"
class="org.apache.commons.dbcp.BasicDataSource"
p:driverClassName="${jdbc.driverClassName}"
p:url="${jdbc.databaseurl}"
p:username="${jdbc.username}"
p:password="${jdbc.password}"
p:testOnBorrow="true"
p:maxActive="1000"
p:testWhileIdle="true"
p:validationQuery="SELECT 1"
p:validationQueryTimeout="5">
<property name="connectionInitSqls">
<list>
<value>SET NAMES 'utf8mb4' COLLATE 'utf8mb4_unicode_ci'</value>
</list>
</property>
</bean>

After not receiving further answers to my comments I found an alternative solution: Base64.
Instead of teaching my DB to understand utf8mb4 I encode all critical messages to Base64 before storing them and decode them when retrieving them from the database.
Pro:
- Works great
- Libraries are already given for java and android
Contra:
- Base64 strings take up more space than the pure utf8mb4 strings (33%-36% more)
- May cost some performance encoding and decoding

Hibernate timestamp - millisecond precision

Seems like storing timestamps with millisecond precision is a know issue with hibernate.
My field in the db was initially set as timestamp(3), but I've tried datetime(3) as well...unfortunately, it didn't make any difference.
I've tried using Timestamp and Date classes, and recently I've started using joda-time library. After all those efforts, I still wasn't unable to save timestamps with millisecond accuracy.
My mapping for the class contains following property:
<property name="startTime" column="startTime" type="org.jadira.usertype.dateandtime.joda.PersistentDateTime" length="3" precision="3" />
and I'v custom defined Dialect class
public class MySQLQustomDialect extends MySQL5InnoDBDialect{
protected void registerColumnType(int code, String name) {
if (code == Types.TIMESTAMP) {
super.registerColumnType(code, "TIMESTAMP(3)");
} else {
super.registerColumnType(code, name);
}
}
}
If I enter the data manually into db, hibernate manages to retrieve the sub second part.
Is there any way to solve this issue?

Are you, by any chance, using the MySQL Connector/J JDBC driver with MariaDB 5.5?
Connector/J usually sends the milliseconds part to the server only when it detects that the server is new enough, and this detection checks that the server version is >= 5.6.4. This obviously does not work correctly for MariaDB 5.5.x.
You can see the relevant part of Connector/J source here:
http://bazaar.launchpad.net/~mysql/connectorj/5.1/view/head:/src/com/mysql/jdbc/PreparedStatement.java#L796
Using MariaDB's own JDBC driver (MariaDB Java Client) might help (I haven't tried), but I accidentally discovered that adding useServerPrepStmts=true to the connection string makes this work with Connector/J, too.

Basic HQL statement always fails

I've recently started trying to use Hibernate, but am doing so in Netbeans. This has left me having to use this example project to try and get me up and running.
Unfortunately, at the step "Enumerating Film Titles and Retrieving Actors Using an HQL Query" my HQL queries do not give results and instead fail, with the exception:
org.hibernate.exception.SQLGrammarException: could not execute query
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'from limit 100' at line 1
It seems that the HQL isn't generating proper MySQL statements but I can't seem to see why, as I've followed the example to the letter thus far.
I am attempting to connect to a local MySQL database named 'sakila', with the following details:
jdbc:mysql://localhost:3306/sakila
which seems to work correctly as I am able to browse the tables from within Netbeans no problem.
My hibernate.cfg.xml is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.dialect">org.hibernate.dialect.MySQLInnoDBDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/sakila</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.show_sql">true</property>
<property name="transaction.factory_class">org.hibernate.transaction.JDBCTransactionFactory</property>
<property name="hibernate.current_session_context_class">thread</property>
<property name="hibernate.query.factory_class">org.hibernate.hql.classic.ClassicQueryTranslatorFactory</property>
<mapping resource="dvdrental/Language.hbm.xml"/>
<mapping resource="dvdrental/FilmActor.hbm.xml"/>
<mapping resource="dvdrental/FilmCategory.hbm.xml"/>
<mapping resource="dvdrental/Category.hbm.xml"/>
<mapping resource="dvdrental/Film.hbm.xml"/>
<mapping resource="dvdrental/Actor.hbm.xml"/>
</session-factory>
</hibernate-configuration>
When I'm using the HQL Query window, the SQL it seems to generate only ever says 'select from ' which is obviously wrong, but I can't see why this is being caused?
Edit 1: The HQL Query tab showing my input

Completely stupid reason- I hadn't even thought about the fact I hadn't built the project, so the example code from the bottom of the example ran properly (cause it came pre-compiled). Works no problem now.

I think the main cause of the error is as stated earlier from User, the U in User should be uppercase/capitalized as hibernate uses the POJO. I tried recompiling the code and it works correctly for the sample Swing application: https://netbeans.org/kb/docs/java/hibernate-java-se.html.
However the HQL Query editor still gives this error. It is strange but seems like this editor is not reliable.

The error message 'from limit 100' from your posted exception tells me, that you forgot to add Film after from. Please provide the HQL query that throws the exception.
BTW: the examples in the example project are bad practice. You should never use inline arguments in HQL queries:
from Film as film where film.filmId between '"+startID+"' and '"+endID+"'
You should use query parameters:
Query q = session.createQuery("from Film as film where film.filmId between :startID and :endID");
q.setParameter("startID", startID);
q.setParameter("endID", endID);

When I'm using the HQL Query window, the SQL it seems to generate only ever says 'select from ' which is obviously wrong, but I can't see why this is being caused?
A simple
from Film
is valid HQL and should work. Could you try without the following line:
<property name="hibernate.query.factory_class">org.hibernate.hql.classic.ClassicQueryTranslatorFactory</property>
Not sure this will solve the problem but there is no reason to use the "old" parser anyway.
That replaces the problem with a new one I'm afraid. Now the SQL tab of the HQL pane reports 'Invalid Query', and if I run the query I get org.hibernate.hql.ast.QuerySyntaxException. Any further pointers?
As I said, I was not sure this would solve the problem. What I would do is write a "simple" JUnit test to validate the setup is working. But an obvious bug like this in NetBeans HQL Query window is unlikely, IMO.

Completely stupid reason- I hadn't even thought about the fact I hadn't built the project, so the example code from the bottom of the example ran properly (cause it came pre-compiled). Works no problem now.
I experienced exactly the same problem but I think your stated solution is not correct. Even your project is not compiled HQL will show you results.
One should be careful of Table name in hibernate.cfg.xml file
as it should be same in query (case sensitive) also for example in your case it has table name as "Actor" so
from Actor will work, not from actor

Oracle quartz selectWithLockSQL value?

I have this in my spring config for quartz....
<prop key="org.quartz.jobStore.selectWithLockSQL">SELECT *
FROM {0} LOCKS WITH (UPDLOCK ROWLOCK)
WHERE LOCK_NAME = ?</prop>`
That works fine for MS SQL Server, but I don't know what to use for Oracle.
Can anyone help me out please?

so what are you trying to do there? the syntax in Oracle to lock a row is:
SELECT * FROM <table_name> WHERE <condition> FOR UPDATE;