Hibernate is 1000 times slower than sql query - java

I have this setup
#Table(name ="A")
EntityA {
Long ID;
List<EntityB> children;
}
#Table(name ="B")
EntityB {
Long ID;
EntityA parent;
EntityC grandchild;
}
#Table(name ="C")
EntityC {
Long ID;
}
The SQL query is this (I omitted irrelevant details):
select top 300 from A where ... and ID in (select parent from B where ... and grandchild in (select ID from C where ...)) order by ...
The sql query in direct database or through Hibernate (3.5) SQL runs 1000 faster than using Criteria or HQL to express this.
The SQL generated is identical from HQL and Criteria and the SQL I posted there.
[EDIT]: Correction - the sql was not identical. I didn't try the Hibernate style parameter setting on the management studio side because I did not realize this until later - see my answer.
If I separate out the subqueries into separate queries, then it is fast again.
I tried
removing all mappings of child, parent, ect.. and just use Long Id references - same thing, so its not a fetching, lazy,eager related.
using joins instead of subqueries, and got the same slow behaviour with all combinations of fetching and loading.
setting a projection on ID instead of retrieving entities, so there is no object conversion - still slow
I looked at Hibernate code and it is doing something astounding. It has a loop through all 300 results that end up hitting the database.
private List doQuery(
final SessionImplementor session,
final QueryParameters queryParameters,
final boolean returnProxies) throws SQLException, HibernateException {
final RowSelection selection = queryParameters.getRowSelection();
final int maxRows = hasMaxRows( selection ) ?
selection.getMaxRows().intValue() :
Integer.MAX_VALUE;
final int entitySpan = getEntityPersisters().length;
final ArrayList hydratedObjects = entitySpan == 0 ? null : new ArrayList( entitySpan * 10 );
final PreparedStatement st = prepareQueryStatement( queryParameters, false, session );
final ResultSet rs = getResultSet( st, queryParameters.hasAutoDiscoverScalarTypes(), queryParameters.isCallable(), selection, session );
// would be great to move all this below here into another method that could also be used
// from the new scrolling stuff.
//
// Would need to change the way the max-row stuff is handled (i.e. behind an interface) so
// that I could do the control breaking at the means to know when to stop
final EntityKey optionalObjectKey = getOptionalObjectKey( queryParameters, session );
final LockMode[] lockModesArray = getLockModes( queryParameters.getLockOptions() );
final boolean createSubselects = isSubselectLoadingEnabled();
final List subselectResultKeys = createSubselects ? new ArrayList() : null;
final List results = new ArrayList();
try {
handleEmptyCollections( queryParameters.getCollectionKeys(), rs, session );
EntityKey[] keys = new EntityKey[entitySpan]; //we can reuse it for each row
if ( log.isTraceEnabled() ) log.trace( "processing result set" );
int count;
for ( count = 0; count < maxRows && rs.next(); count++ ) {
if ( log.isTraceEnabled() ) log.debug("result set row: " + count);
Object result = getRowFromResultSet(
rs,
session,
queryParameters,
lockModesArray,
optionalObjectKey,
hydratedObjects,
keys,
returnProxies
);
results.add( result );
if ( createSubselects ) {
subselectResultKeys.add(keys);
keys = new EntityKey[entitySpan]; //can't reuse in this case
}
}
if ( log.isTraceEnabled() ) {
log.trace( "done processing result set (" + count + " rows)" );
}
}
finally {
session.getBatcher().closeQueryStatement( st, rs );
}
initializeEntitiesAndCollections( hydratedObjects, rs, session, queryParameters.isReadOnly( session ) );
if ( createSubselects ) createSubselects( subselectResultKeys, queryParameters, session );
return results; //getResultList(results);
}
In this code
final ResultSet rs = getResultSet( st, queryParameters.hasAutoDiscoverScalarTypes(), queryParameters.isCallable(), selection, session );
it hits the database with the full SQL, but there are no results collected anywhere.
Then it proceeds to go through this loop
for ( count = 0; count < maxRows && rs.next(); count++ ) {
Where for every one of the expected 300 results, it ends up hitting the database to get the actual result.
This seems insane, since it should already have all the results after 1 query. Hibernate logs do not show any additional SQL being issued during all that time.
Anyone have any insight? The only option I have is to go to native SQL query through Hibernate.

I finally managed to get to the bottom of this. The problem was being caused by Hibernate setting the parameters separately from the actual SQL query that involved subqueries. So native SQL or not, the performance will be slow if this is done. For example this will be slow:
String sql = some sql that has named parameter = :value
SQLQuery sqlQuery = session.createSQLQuery(sql);
sqlQuery.setParameter ("value", someValue);
List<Object[]> list = (List<Object[]>)sqlQuery.list();
And this will be fast
String sql = some native sql where parameter = 'actualValue'
SQLQuery sqlQuery = session.createSQLQuery(sql);
List<Object[]> list = (List<Object[]>)sqlQuery.list();
It seems that for some reason with letting Hibernate take care of the parameters it ends up getting stuck in the resultSet fetching. This is probably because the underlying query on the database is taking much longer being parameterized. I ended up writing the equivalent of Hibernate Criteria and Restrictions code that sets the parameters directly as above.

We noticed a similar behaviour in our system.
And also encountered that writing the query with hardcoded parameters instead of using setParameter() would fixed the issue.
We are using MS SQL Server and after further investigation we noticed the the root cause of our issue is a default configuration of the sql server driver that transmits the query parameters as unicode. This lead to our indices being ignored since they were based on the ascii values on the queried columns.
The solution was to setup this property in the jdbc url : sendStringParametersAsUnicode=false
More details can be found here.
https://stackoverflow.com/a/32867579

Related

Execute multiple queries with parameters on a single connection using jdbc template

I need to execute multiple queries via jdbc template in a single connection.
I am using SQL server with the mssql jdbc driver. Here is the query:
IF OBJECT_ID('tempdb..#temp') IS NULL
SELECT * INTO #temp FROM ( SELECT * FROM "complex query here" ) a
DECLARE #page_size INT
SET #page_size = ?
SELECT *
FROM ( SELECT TOP(#page_size) * FROM #temp
WHERE custom_column > ?
ORDER BY custom_column ) inner_query
(MULTIPLE JOINES HERE)
ORDER BY custom_column
The query simply puts the whole result from the complex query to the temp table if the table does not exist and selects pages from the generated temp table.
Since the temp tables have connection scope, i need to execute the whole paging with multiple queries in a single connection. This is the code i have tried:
try (Connection connection = dataSource.getConnection()) {
while (true) {
List<CustomObject> customObjects = customRepository.
.getCustomObjectsPage(connection, pageSize, minimumValueInWhere);
// business logic here..
if (customObjects.size() < pageSize) {
break;
}
minimumValueInWhere = customObjects.get(customObjects.size() - 1).getId().toString();
}
}
and getCustomObjectsPage():
public List<CustomObject> getCustomObjectsPage(Connection connection, int pageSize,
String parameter) {
JdbcTemplate singleConnectionJdbcTemplate = new JdbcTemplate(
new SingleConnectionDataSource(connection, true));
try {
return singleConnectionJdbcTemplate
.query("the query from above", new Object[]{pageSize, parameter},
JdbcTemplateMapperFactory.newInstance()
.newResultSetExtractor(CustomObject.class)); // the root entity.
}
}
But for some reasons the queries are not executed in a single connection, and if i hardcode the
parameters to the query instead of passing them to the jdbc template it works perfectly.
How to make this work independently of the parameters ?

select scope_identity() using createSQLQuery in Hibernate

I am forced to use createSQLQuery to insert values into tables with an Identity column (the first column and the primary key) using hibernate. Using hibernate classes are not an option since the tables are created on the fly for each customer that is added to the system. I have run the query and it successfully inserts into the table. I then execute a "select scope_identity()" and it always returns null. "select ##Identity" works but that is not guaranteed to be the correct one. I have also tried to append "select scope_identity()" to the insert query. Then I tried query.list() and query.uniqueResult() both of which throw the hibernate exception of "No Results ..."
Session session = DatabaseEngine.getSessionFactory().openSession();
String queryString = "insert into table1 (dataid) values (1)"
SQLQuery query = session.createSQLQuery(insertQueryString);
query.executeUpdate();
query = session.createSQLQuery("select scope_identity()");
BigDecimal entryID = (BigDecimal)query.uniqueResult();
The simple example table is defined as follows:
"CREATE TABLE table1 (EntryID int identity(1,1) NOT NULL," +
"DataID int default 0 NOT NULL, " +
"PRIMARY KEY (EntryID))";
Is there a way I am missing to use scope_identity() with createSQLQuery?
Actually the SQLServerDialect class used by Hibernate uses the same "scope_identity()" too.
The reason why it's not working is because you need to execute those in the same statement or stored procedure.
If you execute the scope_identity() call in a separate statement, SQL Server will not be able to give you last inserted identity value.
You cannot do it with the SQLQuery, even Hibernate uses JDBC to accomplish this task. I wrote a test on GitHub to emulate this and it works like this:
Session session = entityManager.unwrap(Session.class);
final AtomicLong resultHolder = new AtomicLong();
session.doWork(connection -> {
try(PreparedStatement statement = connection.prepareStatement("INSERT INTO post VALUES (?) select scope_identity() ") ) {
statement.setString(1, "abc");
if ( !statement.execute() ) {
while ( !statement.getMoreResults() && statement.getUpdateCount() != -1 ) {
// do nothing until we hit the resultset
}
}
try (ResultSet rs = statement.getResultSet()) {
if(rs.next()) {
resultHolder.set(rs.getLong(1));
}
}
}
});
assertNotNull(resultHolder.get());
The code uses Java 8 lambdas instead of anonymous classes, but you can easily port it to Java 1.7 too.

Ebean query using setDistinct() does not work

I'm using an ebean query in the play! framework to find a list of records based on a distinct column. It seems like a pretty simple query but the problem is the ebean method setDistinct(true) isn't actually setting the query to distinct.
My query is:
List<Song> allSongs = Song.find.select("artistName").setDistinct(true).findList();
In my results I get duplicate artist names.
From what I've seen I believe this is the correct syntax but I could be wrong. I'd appreciate any help. Thank you.
I just faced the same issue out of the blue and can not figure it out. As hfs said its been fixed in a later version but if you are stuck for a while you can use
findSet()
So in your example use
List<Song> allSongs = Song.find.select("artistName").setDistinct(true).findSet();
According to issue #158: Add support for using setDistinct (by excluding id property from generated sql) on the Ebean bug tracker, the problem is that an ID column is added to the beginning of the select query implicitly. That makes the distinct keyword act on the ID column, which will always be distinct.
This is supposed to be fixed in Ebean 4.1.2.
As an alternative you can use a native SQL query (SqlQuery).
The mechanism is described here:
https://ebean-orm.github.io/apidocs/com/avaje/ebean/SqlQuery.html
This is from the documentation:
public interface SqlQuery
extends Serializable
Query object for performing native SQL queries that return SqlRow's.
Firstly note that you can use your own sql queries with entity beans by using the SqlSelect annotation. This should be your first approach when wanting to use your own SQL queries.
If ORM Mapping is too tight and constraining for your problem then SqlQuery could be a good approach.
The returned SqlRow objects are similar to a LinkedHashMap with some type conversion support added.
// its typically a good idea to use a named query
// and put the sql in the orm.xml instead of in your code
String sql = "select id, name from customer where name like :name and status_code = :status";
SqlQuery sqlQuery = Ebean.createSqlQuery(sql);
sqlQuery.setParameter("name", "Acme%");
sqlQuery.setParameter("status", "ACTIVE");
// execute the query returning a List of MapBean objects
List<SqlRow> list = sqlQuery.findList();
i have a solution for it:-
RawSql rawSql = RawSqlBuilder
.parse("SELECT distinct CASE WHEN PARENT_EQUIPMENT_NUMBER IS NULL THEN EQUIPMENT_NUMBER ELSE PARENT_EQUIPMENT_NUMBER END AS PARENT_EQUIPMENT_NUMBER " +
"FROM TOOLS_DETAILS").create();
Query<ToolsDetail> query = Ebean.find(ToolsDetail.class);
ExpressionList<ToolsDetail> expressionList = query.setRawSql(rawSql).where();//ToolsDetail.find.where();
if (StringUtils.isNotBlank(sortBy)) {
if (StringUtils.isNotBlank(sortMode) && sortMode.equals("descending")) {
expressionList.setOrderBy("LPAD("+sortBy+", 20) "+"desc");
//expressionList.orderBy().asc(sortBy);
}else if (StringUtils.isNotBlank(sortMode) && sortMode.equals("ascending")) {
expressionList.setOrderBy("LPAD("+sortBy+", 20) "+"asc");
// expressionList.orderBy().asc(sortBy);
} else {
expressionList.setOrderBy("LPAD("+sortBy+", 20) "+"desc");
}
}
if (StringUtils.isNotBlank(fullTextSearch)) {
fullTextSearch = fullTextSearch.replaceAll("\\*","%");
expressionList.disjunction()
.ilike("customerSerialNumber", fullTextSearch)
.ilike("organizationalReference", fullTextSearch)
.ilike("costCentre", fullTextSearch)
.ilike("inventoryKey", fullTextSearch)
.ilike("toolType", fullTextSearch);
}
//add filters for date range
String fromContractStartdate = Controller.request().getQueryString("fm_contract_start_date_from");
String toContractStartdate = Controller.request().getQueryString("fm_contract_start_date_to");
String fromContractEndtdate = Controller.request().getQueryString("fm_contract_end_date_from");
String toContractEnddate = Controller.request().getQueryString("fm_contract_end_date_to");
if(StringUtils.isNotBlank(fromContractStartdate) && StringUtils.isNotBlank(toContractStartdate))
{
Date fromSqlStartDate=new Date(AppUtils.convertStringToDate(fromContractStartdate).getTime());
Date toSqlStartDate=new Date(AppUtils.convertStringToDate(toContractStartdate).getTime());
expressionList.between("fmContractStartDate",fromSqlStartDate,toSqlStartDate);
}if(StringUtils.isNotBlank(fromContractEndtdate) && StringUtils.isNotBlank(toContractEnddate))
{
Date fromSqlEndDate=new Date(AppUtils.convertStringToDate(fromContractEndtdate).getTime());
Date toSqlEndDate=new Date(AppUtils.convertStringToDate(toContractEnddate).getTime());
expressionList.between("fmContractEndDate",fromSqlEndDate,toSqlEndDate);
}
PagedList pagedList = ToolsQueryFilter.getFilter().applyFilters(expressionList).findPagedList(pageNo-1, pageSize);
ToolsListCount toolsListCount = new ToolsListCount();
toolsListCount.setList(pagedList.getList());
toolsListCount.setCount(pagedList.getTotalRowCount());
return toolsListCount;

Hibernate StoredProcedure with joins and Temporary table

I have a MySQL StoredProcedure(EmployeeAbsentReport Procedure)for Given Two Dateranges. which is Successfully running on MySql Commandprompt.
How can I run this Stored Procedure using Hibernate,I found one Example on How to call Stored Procedure Example,But in the Example program
he worked with a single table.
Procedure:
DELIMITER $$
DROP PROCEDURE IF EXISTS `AbsentReportproc`$$
CREATE DEFINER=`root`#`localhost` PROCEDURE `AbsentReportproc`(IN _fromdate DATETIME, IN _todate DATETIME)
BEGIN
CREATE TEMPORARY TABLE daterange25 (dte DATE);
SET #counter := 0;
WHILE (#counter < DATEDIFF(DATE(_todate), DATE(_fromdate))) DO
INSERT INTO daterange25 VALUES (DATE_ADD(_fromdate, INTERVAL #counter:=#counter + 1 DAY));
END WHILE;
SELECT tp.EMPCODE,tp.NAME,tp.DEPARTMENT, Group_Concat(d.dte order by d.dte SEPARATOR '\n')AbsentDate, COUNT(tp.EMPCODE) Totalnoofabsentdates
FROM Master tp
JOIN daterange25 d
LEFT JOIN Transactions tpt ON (tp.EMPCODE = tpt.empcode) AND DATE(S_DateTime) = d.dte
WHERE tpt.empcode IS NULL
GROUP BY tp.EMPCODE;
DROP TABLE daterange25;
END$$
DELIMITER ;
how to call a Stored Procedure using Hibernate with joins(Including a Temporary Table)and writing sql-query in XML mapping file?
Following the common framework method which i have used in one of my project to call the Stored Procedure of MySQL database :
#Transactional(propagation = Propagation.REQUIRED, readOnly = false)
public <T> List<T> executeSP(final String spName,
final String[] parameterNames, final String[] parameterValues,
final String[] resultColumnNames, final int offset, final int size,
final Class<T> returnType) {
validateParams(parameterNames, parameterValues);
return (List<T>) hibernateTemplate
.execute(new HibernateCallback<List<T>>() {
#SuppressWarnings("unchecked")
public List<T> doInHibernate(final Session session) {
SQLQuery query = session.createSQLQuery("call "
+ spName);
if (returnType != null) {
query.addEntity(returnType);
}
setQueryParams(parameterNames, parameterValues, query);
setResultColumnNames(resultColumnNames, query);
if (offset >= 0) {
query.setFirstResult(offset);
}
if (size > 0) {
query.setMaxResults(size);
}
return query.list();
}
});
}
Next, i would suggest that please make the Temporary Table join into the Stored Procedure only, instead of make it join with SP, if execute in SP then it is better from performance point of view also.

hibernate java select queries

i am new to this and today i tried to play hibernate with a method that returns the result of selected row...if is selected then it can return the result in int.. here is my method
public int validateSub(String slave, String source, String table){
Session session = HibernateUtil.getSessionFactory().openSession();
session.beginTransaction();
Query q = session.createQuery("from Subscribers where slave = :slave AND source = :source AND tbl = :tbl");
q.setParameter("slave", slave);
q.setParameter("source", source);
q.setParameter("tbl", table);
int result = q.executeUpdate();
return result;
}
from this method i tried to validate the 3 values that i get from the Subscribers table but at the end i tried to compile having this error
Exception in thread "Thread-0" org.hibernate.hql.QueryExecutionRequestException: Not supported for select queries [from com.datadistributor.main.Subscribers where slave = :slave AND source = :source AND tbl = :tbl]
You can have a look at the below links that how executeUpdate works, one is from the hibernate docs and other the java docs for JPA which defines when the exception is thrown by the method
http://docs.oracle.com/javaee/6/api/javax/persistence/Query.html#executeUpdate()
https://docs.jboss.org/hibernate/orm/3.2/api/org/hibernate/Query.html#executeUpdate()
Alternatively you can use
List list = query.list();
int count = list != null ? list.size() : 0;
return count;
you are running a select query, Eventhough you are not using the select keyword here hibernate will add that as part of the generated SQL.
what you need to do to avoid the exception is the say
q.list();
now, this will return a List (here is the documentation).
if you are trying to get the size of the elements you can say
Query q = session.createQuery("select count(s) from Subscribers s where slave = :slave AND source = :source AND tbl = :tbl");
Long countOfRecords = (Long)q.list().get(0);
you can execute update statements as well in HQL, it follows a similar structure as SQL (except with object and properties).
Hope this helps.
here you want to select record so it is posible without select key word
sessionFactory sesionfatory;
ArrayList list = (ArrayList)sessionfactory.getCurruntSession().find(from table where name LIKE "xyz");
long size = list.get(0);
I also happened to make the same mistake today.
Your SQL statement is not correct.
You can try:
DELETE from Subscribers WHERE slave = :slave AND source
Try this:
int result = q.list().size();

Categories