We Keep Coding

Execute multiple queries with parameters on a single connection using jdbc template

I need to execute multiple queries via jdbc template in a single connection.
I am using SQL server with the mssql jdbc driver. Here is the query:
IF OBJECT_ID('tempdb..#temp') IS NULL
SELECT * INTO #temp FROM ( SELECT * FROM "complex query here" ) a
DECLARE #page_size INT
SET #page_size = ?
SELECT *
FROM ( SELECT TOP(#page_size) * FROM #temp
WHERE custom_column > ?
ORDER BY custom_column ) inner_query
(MULTIPLE JOINES HERE)
ORDER BY custom_column
The query simply puts the whole result from the complex query to the temp table if the table does not exist and selects pages from the generated temp table.
Since the temp tables have connection scope, i need to execute the whole paging with multiple queries in a single connection. This is the code i have tried:
try (Connection connection = dataSource.getConnection()) {
while (true) {
List<CustomObject> customObjects = customRepository.
.getCustomObjectsPage(connection, pageSize, minimumValueInWhere);
// business logic here..
if (customObjects.size() < pageSize) {
break;
}
minimumValueInWhere = customObjects.get(customObjects.size() - 1).getId().toString();
}
}
and getCustomObjectsPage():
public List<CustomObject> getCustomObjectsPage(Connection connection, int pageSize,
String parameter) {
JdbcTemplate singleConnectionJdbcTemplate = new JdbcTemplate(
new SingleConnectionDataSource(connection, true));
try {
return singleConnectionJdbcTemplate
.query("the query from above", new Object[]{pageSize, parameter},
JdbcTemplateMapperFactory.newInstance()
.newResultSetExtractor(CustomObject.class)); // the root entity.
}
}
But for some reasons the queries are not executed in a single connection, and if i hardcode the
parameters to the query instead of passing them to the jdbc template it works perfectly.
How to make this work independently of the parameters ?

How can I update the result set without removing the previous result from it

I am using jdbc to retrieve data from database. I have four different queries for that, however, the result of first query is used to get the data of second and fourth query. But, the resultset, gets updated as i run other queries. So is there any way that i can keep the resultset and add new results in it.
Here is my code:
class GetData{
String toDate;
String fromDate;
GetData(String d1,String d2) throws ClassNotFoundException, SQLException, ParseException, TransformerException, ParserConfigurationException {
toDate=d1;
fromDate=d2;
Connection connection= null;
ResultSet resultset= null;
String customerquery="SELECT o.ordernumber,o.orderdate,o.customernumber,c.customername,c.addressLine1,c.postalCode,c.city,c.country from orders o join customers c on o.customernumber=c.customernumber where orderdate between ? and ?";
String orderdetailquery="SELECT orderNumber,productCode,quantityOrdered,priceEach,orderLineNumber,(quantityOrdered * priceEach) as total FROM orderdetails where ordernumber=?";
String productsquery="SELECT productName,productLine,productVendor FROM products where productcode=?";
String employeequery="SELECT c.salesRepEmployeeNumber,e.firstname,e.lastname,o.officecode,o.city from customers c join employees e on c.salesRepEmployeeNumber = e.employeeNumber join offices o on e.officecode=o.officecode where c.customernumber=?";
Class.forName("com.mysql.cj.jdbc.Driver");
connection=DriverManager.getConnection("jdbc:mysql://localhost:3306/classicmodels","root","root");
if(d1!=null || d2!=null) {
PreparedStatement p1= connection.prepareStatement(customerquery);
p1.setString(1, toDate);
p1.setString(2,fromDate);
resultset= p1.executeQuery();
PreparedStatement p2= connection.prepareStatement(orderdetailquery);
while(resultset.next()) {
p2.setString(1, resultset.getString("orderNumber"));
}
resultset=p2.executeQuery();
PreparedStatement p3= connection.prepareStatement(productsquery);
while (resultset.next()) {
p3.setString(1, resultset.getString("productcode"));
}
resultset=p3.executeQuery();
PreparedStatement p4=connection.prepareStatement(employeequery);
while(resultset.next()) {
p4.setString(1, resultset.getString("customernumber"));
}
resultset=p4.executeQuery();
resultset.close();
connection.close();
}
}
I am trying to use the result of first query that contains the column customernumber to fetch the data. While, executing the code a error comes that customernumber column not found. So, how can I use the result of first query in other queries. Also, i am trying to get all the result of queries in one resultset as I am trying to create a xml out of it by using DOM.

You can put everything in a single query. using joins.
select o.ordernumber,o.orderdate
, o.customernumber,c.customername
, c.addressLine1,c.postalCode,c.city,c.country
, t1.productCode, t1.quantityOrdered, t1.priceEach, t1.orderLineNumber,(t1.quantityOrdered * t1. priceEach) as total
, t2.productName, t2.productLine, t2.productVendor
, c.salesRepEmployeeNumber,t4.firstname,t4.lastname,t3.officecode,t3.city
from orders o
join customers c on o.customernumber=c.customernumber
left join orderdetails t1 on t1.orderNumber = o.orderNumber
left join products t2 on t2.productCode = t1.productCode
left join offices t3 on t3.offiecode = c.customernumber
left join employees t4 on t3.officecode = t4.officecode

ResultSet is getting changed because you are using the same reference (object) every time you are executing the sql statement, hence it is overriding the old result, so if you want to deal with previously returned ResultSet, you can create new ResultSet instance to use, and also you can create a Bean class to set the elements and make it List type to keep on adding the results as per your logic.

Create a POJO to store required properties retrieved from different queries.
The resultset is getting updated because the same reference variable is being used to assign the resultset of subsequent queries.
If you want to use the same variable, you can follow like this -
create a POJO (which can be used to create a xml out of it by using DOM).
get result of first query in resultset.
populate relevant properties of POJO from this resultset.
re-use the resultset to store result of next queries.
When you need to use previous results, get them from the POJO to be used as parameters in subsequent queries.

How to fetch list of tables in database using HibernateDaoSupport?

Here I have added my code. Issue occurs in try block while I am trying fetch list of tables.
Database is MySql
Exception is : java.lang.IllegalArgumentException: node to traverse cannot be null!
public class DBOptimizationDAO extends HibernateDaoSupport {
private static final Log log = LogFactory.getLog(DBOptimizationDAO.class);
public void optimizeAdapter(String year)
{
List<com.ecw.adapterservice.beans.TransactionInbound> transactionInboundList = null;
StringBuilder queries = new StringBuilder();
try {
transactionInboundList = (List<com.ecw.adapterservice.beans.TransactionInbound>)super.getHibernateTemplate().find("from TransactionInbound where inboundTimestamp < '" + year+ "-01-01'order by 1 desc limit 2");
// Check if archive table exist or not
List<Object> inboundObj = getHibernateTemplate().find("SHOW TABLES LIKE transaction_outbound");
List<Object> outboundObj = getHibernateTemplate().find("SHOW TABLES LIKE 'transaction_outbound_archive'");

The HibernateTemplate::find expects a HQL query in the string parameter and you are passing a native statement. You can do native stuff (queries, statements, etc) using the Session object returned by HibernateTemplate::getSession. To pass a native select query you then have Session::createSQLQuery
BUT do you really want to rely on database specific code to do this? There is a more elegant way to do it by using DatabaseMetaData::getTables. See this answer. And you can get an instance of DatabaseMetaData from a callback method of your HibernateTemplate.
Try this:
Session session = getHibernateTemplate().getSessionFactory().getCurrentSession();
if(!session instaceof SessionImpl){
//handle this, maybe throw an exception
}
else {
Connection con = (SessionImpl)session.connection();
...
}

How to solve this sql string in Dbutils ？

public List<Hero> list() throws SQLException {
return list(0, Short.MAX_VALUE);
}
public List<Hero> list(int start, int count) throws SQLException{
Connection c = this.getConnection();
QueryRunner runner = new QueryRunner();
String sql = "select * from hero order by id desc limit ?,? ";
Object params[] = {"start", "count" };
List<Hero> heros = (List<Hero>) runner.query(c,sql,new BeanListHandler(Hero.class),params);
DbUtils.closeQuietly(c);
return heros;
}
Before that I have imported Dbutils JAR that I need, like org.apache.commons.dbutils.handlers.BeanListHandler and
org.apache.commons.dbutils.QueryRunner
But after running my project, it goes wrongs whose message is :
You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near ''start','count'' at line 1 Query: select * from hero order by id
desc limit ?,? Parameters: [start, count]
I know that something wrong in program, but I dont want to find all, I just want to find part of my table using limit ?.? (I only know this sql sentence can do that).
Could u please help me?

You're passing strings as your parameters, so it's running your SQL limit literally as LIMIT 'start','count'
Try this instead:
Object params[] = {start, count };
So that you're building a parameter array of your actual int values (now autoboxed to Integer)

Hibernate is 1000 times slower than sql query

I have this setup
#Table(name ="A")
EntityA {
Long ID;
List<EntityB> children;
}
#Table(name ="B")
EntityB {
Long ID;
EntityA parent;
EntityC grandchild;
}
#Table(name ="C")
EntityC {
Long ID;
}
The SQL query is this (I omitted irrelevant details):
select top 300 from A where ... and ID in (select parent from B where ... and grandchild in (select ID from C where ...)) order by ...
The sql query in direct database or through Hibernate (3.5) SQL runs 1000 faster than using Criteria or HQL to express this.
The SQL generated is identical from HQL and Criteria and the SQL I posted there.
[EDIT]: Correction - the sql was not identical. I didn't try the Hibernate style parameter setting on the management studio side because I did not realize this until later - see my answer.
If I separate out the subqueries into separate queries, then it is fast again.
I tried
removing all mappings of child, parent, ect.. and just use Long Id references - same thing, so its not a fetching, lazy,eager related.
using joins instead of subqueries, and got the same slow behaviour with all combinations of fetching and loading.
setting a projection on ID instead of retrieving entities, so there is no object conversion - still slow
I looked at Hibernate code and it is doing something astounding. It has a loop through all 300 results that end up hitting the database.
private List doQuery(
final SessionImplementor session,
final QueryParameters queryParameters,
final boolean returnProxies) throws SQLException, HibernateException {
final RowSelection selection = queryParameters.getRowSelection();
final int maxRows = hasMaxRows( selection ) ?
selection.getMaxRows().intValue() :
Integer.MAX_VALUE;
final int entitySpan = getEntityPersisters().length;
final ArrayList hydratedObjects = entitySpan == 0 ? null : new ArrayList( entitySpan * 10 );
final PreparedStatement st = prepareQueryStatement( queryParameters, false, session );
final ResultSet rs = getResultSet( st, queryParameters.hasAutoDiscoverScalarTypes(), queryParameters.isCallable(), selection, session );
// would be great to move all this below here into another method that could also be used
// from the new scrolling stuff.
//
// Would need to change the way the max-row stuff is handled (i.e. behind an interface) so
// that I could do the control breaking at the means to know when to stop
final EntityKey optionalObjectKey = getOptionalObjectKey( queryParameters, session );
final LockMode[] lockModesArray = getLockModes( queryParameters.getLockOptions() );
final boolean createSubselects = isSubselectLoadingEnabled();
final List subselectResultKeys = createSubselects ? new ArrayList() : null;
final List results = new ArrayList();
try {
handleEmptyCollections( queryParameters.getCollectionKeys(), rs, session );
EntityKey[] keys = new EntityKey[entitySpan]; //we can reuse it for each row
if ( log.isTraceEnabled() ) log.trace( "processing result set" );
int count;
for ( count = 0; count < maxRows && rs.next(); count++ ) {
if ( log.isTraceEnabled() ) log.debug("result set row: " + count);
Object result = getRowFromResultSet(
rs,
session,
queryParameters,
lockModesArray,
optionalObjectKey,
hydratedObjects,
keys,
returnProxies
);
results.add( result );
if ( createSubselects ) {
subselectResultKeys.add(keys);
keys = new EntityKey[entitySpan]; //can't reuse in this case
}
}
if ( log.isTraceEnabled() ) {
log.trace( "done processing result set (" + count + " rows)" );
}
}
finally {
session.getBatcher().closeQueryStatement( st, rs );
}
initializeEntitiesAndCollections( hydratedObjects, rs, session, queryParameters.isReadOnly( session ) );
if ( createSubselects ) createSubselects( subselectResultKeys, queryParameters, session );
return results; //getResultList(results);
}
In this code
final ResultSet rs = getResultSet( st, queryParameters.hasAutoDiscoverScalarTypes(), queryParameters.isCallable(), selection, session );
it hits the database with the full SQL, but there are no results collected anywhere.
Then it proceeds to go through this loop
for ( count = 0; count < maxRows && rs.next(); count++ ) {
Where for every one of the expected 300 results, it ends up hitting the database to get the actual result.
This seems insane, since it should already have all the results after 1 query. Hibernate logs do not show any additional SQL being issued during all that time.
Anyone have any insight? The only option I have is to go to native SQL query through Hibernate.

I finally managed to get to the bottom of this. The problem was being caused by Hibernate setting the parameters separately from the actual SQL query that involved subqueries. So native SQL or not, the performance will be slow if this is done. For example this will be slow:
String sql = some sql that has named parameter = :value
SQLQuery sqlQuery = session.createSQLQuery(sql);
sqlQuery.setParameter ("value", someValue);
List<Object[]> list = (List<Object[]>)sqlQuery.list();
And this will be fast
String sql = some native sql where parameter = 'actualValue'
SQLQuery sqlQuery = session.createSQLQuery(sql);
List<Object[]> list = (List<Object[]>)sqlQuery.list();
It seems that for some reason with letting Hibernate take care of the parameters it ends up getting stuck in the resultSet fetching. This is probably because the underlying query on the database is taking much longer being parameterized. I ended up writing the equivalent of Hibernate Criteria and Restrictions code that sets the parameters directly as above.

We noticed a similar behaviour in our system.
And also encountered that writing the query with hardcoded parameters instead of using setParameter() would fixed the issue.
We are using MS SQL Server and after further investigation we noticed the the root cause of our issue is a default configuration of the sql server driver that transmits the query parameters as unicode. This lead to our indices being ignored since they were based on the ascii values on the queried columns.
The solution was to setup this property in the jdbc url : sendStringParametersAsUnicode=false
More details can be found here.
https://stackoverflow.com/a/32867579