I am doing some big queries on my database with Hibernate and I sometimes hit timeouts. I would like to avoid setting the timeout manually on every Query or Criteria.
Is there any property I can give to my Hibernate configuration that would set an acceptable default for all queries I run?
If not, how can I set a default timeout value on Hibernate queries?
JPA 2 defines the javax.persistence.query.timeout hint to specify default timeout in milliseconds. Hibernate 3.5 (currently still in beta) will support this hint.
See also https://hibernate.atlassian.net/browse/HHH-4662
JDBC has this mechanism named Query Timeout, you can invoke setQueryTime method of java.sql.Statement object to enable this setting.
Hibernate cannot do this in unified way.
If your application retrive JDBC connection vi java.sql.DataSource, the question can be resolved easily.
we can create a DateSourceWrapper to proxy Connnection which do setQueryTimeout for every Statement it created.
The example code is easy to read, I use some spring util classes to help this.
public class QueryTimeoutConfiguredDataSource extends DelegatingDataSource {
private int queryTimeout;
public QueryTimeoutConfiguredDataSource(DataSource dataSource) {
super(dataSource);
}
// override this method to proxy created connection
#Override
public Connection getConnection() throws SQLException {
return proxyWithQueryTimeout(super.getConnection());
}
// override this method to proxy created connection
#Override
public Connection getConnection(String username, String password) throws SQLException {
return proxyWithQueryTimeout(super.getConnection(username, password));
}
private Connection proxyWithQueryTimeout(final Connection connection) {
return proxy(connection, new InvocationHandler() {
//All the Statement instances are created here, we can do something
//If the return is instance of Statement object, we set query timeout to it
#Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
Object object = method.invoke(connection, args);
if (object instanceof Statement) {
((Statement) object).setQueryTimeout(queryTimeout);
}
return object;
});
}
private Connection proxy(Connection connection, InvocationHandler invocationHandler) {
return (Connection) Proxy.newProxyInstance(
connection.getClass().getClassLoader(),
ClassUtils.getAllInterfaces(connection),
invocationHandler);
}
public void setQueryTimeout(int queryTimeout) {
this.queryTimeout = queryTimeout;
}
}
Now we can use this QueryTimeoutConfiguredDataSource to wrapper your exists DataSource to set Query Timeout for every Statement transparently!
Spring config file:
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource">
<bean class="com.stackoverflow.QueryTimeoutConfiguredDataSource">
<constructor-arg ref="dataSource"/>
<property name="queryTimeout" value="1" />
</bean>
</property>
</bean>
Here are a few ways:
Use a factory or base class method to create all queries and set the timeout before returning the Query object
Create your own version of org.hibernate.loader.Loader and set the timeout in doQuery
Use AOP, e.g. Spring, to return a proxy for Session; add advice to it that wraps the createQuery method and sets the timeout on the Query object before returning it
Yes, you can do that.
As I explained in this article, all you need to do is to pass the JPA query hint as a global property:
<property
name="javax.persistence.query.timeout"
value="1000"
/>
Now, when executing a JPQL query that will timeout after 1 second:
List<Post> posts = entityManager
.createQuery(
"select p " +
"from Post p " +
"where function('1 >= ALL ( SELECT 1 FROM pg_locks, pg_sleep(2) ) --',) is ''", Post.class)
.getResultList();
Hibernate will throw a query timeout exception:
SELECT p.id AS id1_0_,
p.title AS title2_0_
FROM post p
WHERE 1 >= ALL (
SELECT 1
FROM pg_locks, pg_sleep(2)
) --()=''
-- SQL Error: 0, SQLState: 57014
-- ERROR: canceling statement due to user request
For more details about setting a timeout interval for Hibernate queries, check out this article.
For setting global timeout values at query level - Add the below to config file.
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource"></property>
<property name="queryTimeout" value="60"></property>
</bean>
For setting global timeout values at transaction(INSERT/UPDATE) level - Add the below to config file.
<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="myEmf" />
<property name="dataSource" ref="dataSource" />
<property name="defaultTimeout" value="60" />
<property name="jpaDialect">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaDialect" />
</property>
</bean>
Related
I'm using a hibernate interceptor to intercept queries and alter them before they are sent to the postgresql database.
The change made on the queries is specific to every connected user(the interceptor is getting the user's information from his session).
The problem is, since i'm using spring along with hibernate, the interceptor is a singleton and made at the sessionFactory level, so it's not thread-safe.
This is the configuration of spring related to hibernate :
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation" value="classpath:hibernate.cfg.xml"/>
<property name="entityInterceptor">
<ref bean="myEntityInterceptor"/>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">${jdbc.dialect}</prop>
<prop key="hibernate.show_sql">true</prop>
</props>
</property>
</bean>
<bean id="myEntityInterceptor" class="dao.generic.HibernateInterceptor"/>
And the interceptor class :
#Component
public class HibernateInterceptor extends EmptyInterceptor {
private static final long serialVersionUID = 1L;
final static Logger logger = Logger.getLogger(HibernateInterceptor.class);
#Override
public String onPrepareStatement(String sql) {
String ps = super.onPrepareStatement(sql);
if (SecurityContextHolder.getContext().getAuthentication() != null) {
UserDetails ud = (UserDetails) SecurityContextHolder
.getContext().getAuthentication().getDetails();
ps = ps.replaceAll("dynamic.", ud.getDb() + ".");
}
return ps;
}
}
So I'm lookinf for a way to make this interceptor thread-safe by attaching a seperate instance for every client session and not only one instance used by all the users.
Any help will be appreciated..
First, your code seems thread safe to me, so I'm not sure if you really need this, but in case you do, you can set the interceptor to be an instance per session instead of a shared instance by creating it at Session level on HibernateTransactionManager by setting the property entityInterceptorBeanName
From Spring Doc (HibernateTransactionManager.setEntityInterceptorBeanName):
Typically used for prototype interceptors, i.e. a new interceptor instance per session.
So, make sure your interceptor bean is scoped as prototype
Take a look at this post
SingleTon objects can be treated as thread safe, as long as the object does not hold any state information (i.e., instance variables with getters and setters).
So, here in this example, HibernateInterceptor object is thread safe.
I have a web application running under Tomcat 7 using Spring with c3po as the connection pool manager. I have also used dbcp and have the same result.
I initiate a long running single threaded process which makes a large number of database calls using jdbcTemplate.update(), etc, in various dao's. As each of these updates is simple and independent, no transaction manager is being used.
For some reason, I am running out of connections. What appears to be happening is that each dao is holding onto its own connection and not returning it to the pool.
Is this normal behaviour? I had expected that the connection was tied to the jdbcTemplate.update() and released back as soon as this had finished.
...
In the context file...
<bean id="enquiryDataSource" destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${enquiry.drivername}"/>
<property name="url" value="${enquiry.jdbc}"/>
<property name="username" value="${enquiry.username}"/>
<property name="password" value="${enquiry.password}"/>
<property name="maxWait" value="30000"/>
<property name="maxActive" value="50"/>
</bean>
In a typical dao constructor...
#Autowired
public XXXCountryDao(#Qualifier("enquiryDataSource") DataSource dataSource,
#Qualifier("sqlUpdaterFactoryImpl") SqlUpdaterFactory sqlUpdaterFactory, #Qualifier("sqlFormatterFactoryImpl") SqlFormatterFactory sqlFormatterFactory) {
super("Country", dataSource, sqlUpdaterFactory, sqlFormatterFactory);
// ...other constructor stuff
}
All dao's inherit from...
public abstract class AbstractFileProcessorDao<ImportRecType, QueryRecType> extends JdbcDaoSupport {
// ...
}
In a typical dao method...
protected boolean runUpdateToSqlDatabase(Map<String, Object> values, Map<String, Object> whereValues) {
if (values.isEmpty())
return true;
String sql = updateUpdaterServer.getSql(values, whereValues);
if (logger.isDebugEnabled())
logger.debug("Server SQL -> " + sql);
getJdbcTemplate().update(sql);
return false;
}
Please check your application for "rogue" calls to DataSource#getConnection (you can use your IDE to search for method references). Connection leaks are usually caused by obtaining a connection which is then never closed via Connection#close.
When working with Spring's JdbcTemplate all JDBC resource handling (opening / closing connections, statements, result sets) is done automatically. But with legacy code you never know.
I have a Spring application that currently executes some queries utilizing stored procedures. The configuration is something like this:
Datasource:
<bean id="dataSource" destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.ReplicationDriver"/>
<property name="url" value="jdbc:mysql:replication://master,slave1,slave2/db?allowMultiQueries=true"/>
<property name="username" value="${db.dbusername}"/>
<property name="password" value="${db.dbpassword}"/>
<property name="defaultReadOnly" value="true"/>
</bean>
<bean id="jdbcDeviceDAO" class="dao.jdbc.JdbcDeviceDAO">
<property name="dataSource" ref="dataSource"/>
</bean>
DAO:
public class JdbcDeviceDAO implements DeviceDAO {
// ...
public void setDataSource(DataSource dataSource) {
this.jdbcTemplate = new JdbcTemplate(dataSource);
this.procGetCombinedDeviceRouting = new SimpleJdbcCall(jdbcTemplate)
.withProcedureName("get_combined_device_routing");
// ...
}
public CombinedDeviceRouting getCombinedDeviceRouting(String deviceName, String deviceNameType) {
SqlParameterSource in = createParameters(deviceName, deviceNameType);
Map<String, Object> results = this.procGetCombinedDeviceRouting.execute(in);
return extractResults(results);
}
Now when I call getCombinedDeviceRouting(...) it fails with the following exception:
org.springframework.dao.TransientDataAccessResourceException: CallableStatementCallback; SQL [{call get_combined_device_routing()}]; Connection is read-only. Queries leading to data modification are not allowed; nested exception is java.sql.SQLException: Connection is read-only. Queries leading to data modification are not allowed
I know the connection is read-only and I need it to be that way so the queries are load-balanced between slave hosts. But the stored procedure is actually read only, it's just a lot of SELECT statements, in fact I tried adding READS SQL DATA to its definition but it didn't work.
Finally I came to the point of reading the mysql's connector code and I found this:
protected boolean checkReadOnlySafeStatement() throws SQLException {
synchronized (checkClosed().getConnectionMutex()) {
return this.firstCharOfStmt == 'S' || !this.connection.isReadOnly();
}
}
It sounds naive, but is the connector checking whether my statement is read-only by just matching the first character with 'S'?
If this is the case, it seems like there's no way of calling a stored procedure on a slave host, because the statement starts with 'C' (CALL ...).
Does anyone know if there's a workaround for this problem? Or maybe I'm wrong assuming this first character check?
It appears as though this is a bug with the driver I had a look at the code to see if there is an easy extension point, but it looks like you'd have to extend a lot of classes to affect this behaviour :(
I'm using hibernate 3.2.7 (same problem on 3.2.5) with spring 3.0.1, all deployed on weblogic 10.3 and with an Oracle 10g database. I'm using JTA transaction management and the transaction is distributed (it is actually started and ended in another application, this code is just in between).
The configuration used by hibernate is declared in my persistence.xml and is the following:
<property name="hibernate.dialect" value="org.hibernate.dialect.Oracle10gDialect"/>
<property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.WeblogicTransactionManagerLookup"/>
<property name="hibernate.query.factory_class" value="org.hibernate.hql.classic.ClassicQueryTranslatorFactory"/>
<property name="hibernate.current_session_context_class" value="jta"/>
<property name="hibernate.connection.release_mode" value="auto"/>
The spring configuration regarding the transaction manager is the following:
<!-- Instructs Spring to perfrom declarative transaction managemenet on annotated classes -->
<tx:annotation-driven transaction-manager="txManager" proxy-target-class="true"/>
<!-- Data about transact manager and session factory -->
<bean id="txManager" class="org.springframework.transaction.jta.WebLogicJtaTransactionManager">
<property name="transactionManagerName" value="javax.transaction.TransactionManager"/>
<property name="defaultTimeout" value="${app.transaction.timeOut}"/>
</bean>
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<!-- persistence unit is missing jta data source so that application server is not
creating EntitiyManagerFactory, spring will create its own LocalContainerEntityManagerFactoryBean overriding data source-->
<property name="dataSource" ref="myDataSource"/>
<!-- specific properties like jpa provider and jpa provider properties are in persistance unit -->
<property name="persistenceUnitName" value="my.persistence.unit"/>
</bean>
<!-- define data source in application server -->
<jee:jndi-lookup id="myDataSource" jndi-name="${db.jndiName}"/>
I'm using a generic CrudDao with an update method that looks like this:
public void update(Object entity) {
//entityManager injected by #PersistenceContext
entityManager.merge(entity);
entityManager.flush();
}
public Object getById(Object id, Class entityClass) throws PersistenceException{
return (Object)entityManager.find(entityClass, id);
}
UPDATED: added the getById method.
The code that does not work as expected looks like this:
MyObject myObj = getMyObjectThroughSomeOneToManyRelation(idOne, idOther);
// till now was null
myObj.setSomeDateAttr(someDate);
genericDao.update(myObj);
MyObject myObjFromDB = genericDao.getById(myObj.getId(), MyObject.class);
The result is that if I print myObj.getSomeDateAttr() it returns me the value of someDate, if I print myObjFromDB.getSomeDateAttr() it still has null.
I've tried changing the update method to:
org.hibernate.Session s = (org.hibernate.Session) entityManager.getDelegate();
s.evict(entity);
s.update(entity);
s.flush();
And it still doesn't work.
When turning on the show_sql flag of hibernate I don't see any update occurring when doing flush nor when I query the entity manager for the object with the same id. The selects are all visible.
UPDATE:
At the end of the transaction the update is actually called and everything is written to the db. So my problem is "just" during the transaction.
I'm afraid the problem may be linked with the configuration of the transaction manager on spring and on hibernate.
Hope that someone can help me as I have already lost a day and a half with no luck.
You need to look at the hibernate merge behaviour closely. As per documentation
if there is a persistent instance with the same identifier currently
associated with the session, copy the state of the given object onto
the persistent instance
if there is no persistent instance currently associated with the session, try to load it from the database, or create a new persistent instance
the persistent instance is returned
the given instance does not become associated with the session, it
remains detached
As per your statement on the sql queries in log, it look like
MyObject myObj = getMyObjectThroughSomeOneToManyRelation(idOne, idOther); returning the persistent object but when you modify it(becomes dirty) and call merge method, new state is copied to the current persistent object in session. If you see third point merge returns persistent object which is actually new manageable persistent object which you need to use in subsequent operations.
When you call find method hibernate returns the persistent object in session and not maneagable persistent object thats why you dont find the changes in object return by find.
To fix your problem change the reurn type of update method
public Object update(Object entity) {
//entityManager injected by #PersistenceContext
return entityManager.merge(entity);
}
and in service you need to use as below
MyObject myObj = getMyObjectThroughSomeOneToManyRelation(idOne, idOther);
// till now was null
myObj.setSomeDateAttr(someDate);
//You can use myObj as well instead myNewObj
MyObject myNewObj= genericDao.update(myObj);
//No need to call get
//MyObject myObjFromDB = genericDao.getById(myObj.getId(), MyObject.class);
System.out.println("Updated value:"+myNewObj.getSomeDateAttr());
Have a look at this artical as well.
I have a Jersey resource that access the database. Basically it opens a database connection in the initialization of the resource. Performs queries on the resource's methods.
I have observed that when I do not use #Singleton, the database is being open at each request. And we know opening a connection is really expensive right?
So my question is, should I specify that the resource be singleton or is it really better to keep it at per request especially when the resource is connecting to the database?
My resource code looks like this:
//Use #Singleton here or not?
#Path(/myservice/)
public class MyResource {
private ResponseGenerator responser;
private Log logger = LogFactory.getLog(MyResource.class);
public MyResource() {
responser = new ResponseGenerator();
}
#GET
#Path("/clients")
public String getClients() {
logger.info("GETTING LIST OF CLIENTS");
return responser.returnClients();
}
...
// some more methods
...
}
And I connect to the database using a code similar to this:
public class ResponseGenerator {
private Connection conn;
private PreparedStatement prepStmt;
private ResultSet rs;
public ResponseGenerator(){
Class.forName("org.h2.Driver");
conn = DriverManager.getConnection("jdbc:h2:testdb");
}
public String returnClients(){
String result;
try{
prepStmt = conn.prepareStatement("SELECT * FROM hosts");
rs = prepStmt.executeQuery();
...
//do some processing here
...
} catch (SQLException se){
logger.warn("Some message");
} finally {
rs.close();
prepStmt.close();
// should I also close the connection here (in every method) if I stick to per request
// and add getting of connection at the start of every method
// conn.close();
}
return result
}
...
// some more methods
...
}
Some comments on best practices for the code will also be helpful.
Rather than thinking about making the resource a singleton, focus more on managing backend, service type objects like your ResponseGenerator class as singletons, which obviously shouldn't be instantiated every request.
Making the resource a singleton as well is one way of managing ResponseGenerator as a singleton, but it's not the only or necessarily the best way, see Access external objects in Jersey Resource class and How to wire in a collaborator into a Jersey resource? for ways to inject this into non-singleton resources.
Note that your ResponseGenerator class would need work before it would function as a singleton, whether injected into a per-request resource or instantiated in a singleton resource. It's not thread safe, and you would open a single connection on startup and reuse it across requests, which won't work, you should use a connection pool to do the heavy lifting of efficiently + safely reusing connections across requests.
Some comments on best practices for the code will also be helpful.
You'll get better responses on http://codereview.stackexchange.com,
but:
ResponseGenerator is a poor name for a class (just about everything in a web application is a response generator).
don't use String as the return type of your service and object, use proper typed objects (eg it sounds like you're returning a java.util.List of something).
Don't swallow your SQLException, bubble it up to allow Jersey to generate a 5xx series response code in your resource.
Use final member variables.
Your log object should be static.
You best option is to use a framework like Spring with Jersey which I outlined in a similar post. The only difference is that instead of injecting a service bean you would inject a pooled DataSource and this can easily be configured using c3p0.
Example applicationContext.xml, notice the "scope" is set to prototype which is equivalent to a singleton in Spring parlance.
<bean id="pooledDataSource" scope="prototype" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="jdbcUrl" value="${jpa.url}" />
<property name="user" value="${jpa.username}" />
<property name="password" value="${jpa.password}" />
<property name="initialPoolSize" value="1" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="3" />
<property name="idleConnectionTestPeriod" value="500" />
<property name="acquireIncrement" value="1" />
<property name="maxStatements" value="50" />
<property name="numHelperThreads" value="1" />
</bean>
In your MyResource.java you would simply add the following and Spring would inject it appropriately.
private DataSource pooledDataSource;
public void setPooledDataSource(DataSource pooledDataSource) {
this.pooledDataSource = pooledDataSource;
}
Then you could change your ResponseGenerator to accept the DataSource and use this to query the database.