Spring Batch Paging with sortKeys and parameter values - java

I have a Spring Batch project running in Spring Boot that is working perfectly fine. For my reader I'm using JdbcPagingItemReader with a MySqlPagingQueryProvider.
#Bean
public ItemReader<Person> reader(DataSource dataSource) {
MySqlPagingQueryProvider provider = new MySqlPagingQueryProvider()
provider.setSelectClause(ScoringConstants.SCORING_SELECT_STATEMENT)
provider.setFromClause(ScoringConstants.SCORING_FROM_CLAUSE)
provider.setSortKeys("p.id": Order.ASCENDING)
JdbcPagingItemReader<Person> reader = new JdbcPagingItemReader<Person>()
reader.setRowMapper(new PersonRowMapper())
reader.setDataSource(dataSource)
reader.setQueryProvider(provider)
//Setting these caused the exception
reader.setParameterValues(
startDate: new Date() - 31,
endDate: new Date()
)
reader.afterPropertiesSet()
return reader
}
However, when I modified my query with some named parameters to replace previously hard coded date values and set these parameter values on the reader as shown above, I get the following exception on the second page read (the first page works fine because the _id parameter hasn't been made use of by the paging query provider):
org.springframework.dao.InvalidDataAccessApiUsageException: No value supplied for the SQL parameter '_id': No value registered for key '_id'
at org.springframework.jdbc.core.namedparam.NamedParameterUtils.buildValueArray(NamedParameterUtils.java:336)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.getPreparedStatementCreator(NamedParameterJdbcTemplate.java:374)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:192)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:199)
at org.springframework.batch.item.database.JdbcPagingItemReader.doReadPage(JdbcPagingItemReader.java:218)
at org.springframework.batch.item.database.AbstractPagingItemReader.doRead(AbstractPagingItemReader.java:108)
Here is an example of the SQL, which has no WHERE clause by default. One does get created automatically when the second page is read:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p
On the second page, the sql is modified to the following, however it seems that the named parameter for _id didn't get supplied:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p WHERE id > :_id
I'm wondering if I simply can't use the MySqlPagingQueryProvider sort keys together with additional named parameters set in JdbcPagingItemReader. If not, what is the best alternative to solving this problem? I need to be able to supply parameters to the query and also page it (vs. using the cursor). Thank you!

I solved this problem with some intense debugging. It turns out that MySqlPagingQueryProvider utilizes a method getSortKeysWithoutAliases() when it builds up the SQL query to run for the first page and for subsequent pages. It therefore appends and (p.id > :_id) instead of and (p.id > :_p.id). Later on, when the second page sort values are created and stored in JdbcPagingItemReader's startAfterValues field it will use the original "p.id" String specified and eventually put into the named parameter map the pair ("_p.id",10). However, when the reader tries to fill in _id in the query, it doesn't exist because the reader used the non-alias removed key.
Long story short, I had to remove the alias reference when defining my sort keys.
provider.setSortKeys("p.id": Order.ASCENDING)
had to change to in order for everything to work nicely together
provider.setSortKeys("id": Order.ASCENDING)

I had the same issue and got another possible solution.
My table T has a primary key field INTERNAL_ID.
The query in JdbcPagingItemReader was like this:
SELECT INTERNAL_ID, ... FROM T WHERE ... ORDER BY INTERNAL_ID ASC
So, the key is: in some conditions, the query didn't return results, and then, raised the error above No value supplied for...
The solution is:
Check in a Spring Batch decider element if there are rows.
If it is, continue with chunk: reader-processor-writer.
It it's not, go to another step.
Please, note that they are two different scenarios:
At the beginning, there are rows. You get them by paging and finally, there are no more rows. This has no problem and decider trick is not required.
At the beginning, there are no rows. Then, this error raised, and the decider solved it.
Hope this helps.

Related

Retrieve generated keys from multiple queries in single Spring JDBC Update

I am using a single Spring JDBC update to make an update to two tables in my Postgres database. My SQL query is as follows:
UPDATE accounts SET last_transaction_amount = :transaction_amount WHERE acct_num = :acct_num; INSERT INTO transactions (transaction_amout) VALUES (:transaction_amount);
Using NamedParameterJdbcTemplate#update, I have no issue executing this query and achieving the expected results.
The transactions table generates a sequential transaction identifier, and I want to return this to my application.
I've tried passing a GeneratedKeyHolder in the update call. This is returning the error "A result was returned when none was expected". Docs link.
I've tried passing a GeneratedKeyHolder and array of column names (new String[] {"transaction_id"}). This is returning the error that the column doesn't exist. Note this method call does work to return the transaction id when I only pass the INSERT query without the preceding UPDATE query. Docs link.
How can I retrieve the generated key? Thank you!
You seem to be looking for the RETURNING clause. Assuming that the serial number is called transaction_id:
INSERT INTO transactions (transaction_amout)
VALUES (:transaction_amount)
RETURNING transaction_id;

Checking if table exist or not

I am retrieving data from database using jdbc. In my code I am using 3-4 tables to get data. But sometimes if table is not present in database my code gives exception. How to handle this situation. I want my code to continue working for other tables even if one table is not present. Please help.
I have wrote a code like this
sql="select * from table"
now Result set and all.
If table is not present in database it give exception that no such table. I want to handle it. In this code I cannot take tables which are already present in advance . I want to check here itself if table is there or not.
Please do not mark it as a duplicate question. The link you shared doesnot give me required answer as in that question they are executing queries in database not through JDBC code
For Sybase ASE the easiest/quickest method would consist of querying the sysobjects table in the database where you expect the (user-defined) table to reside:
select 1 from sysobjects where name = 'table-name' and type = 'U'
if a record is returned => table exists
if no record is returned => table does not exist
How you use the (above) query is up to you ...
return a 0/1-row result set to your client
assign a value to a #variable
place in a if [not] exists(...) construct
use in a case statement
If you know for a fact that there won't be any other object types (eg, proc, trigger, view, UDF) in the database with the name in question then you could also use the object_id() function, eg:
select object_id('table-name')
if you receive a number => the object exists
if you receive a NULL => the object does not exist
While object_id() will obtain an object's id from the sysobjects table, it does not check for the object type, eg, the (above) query will return a number if there's a stored proc named 'table-name'.
As with the select/sysobjects query, how you use the function call in your code is up to you (eg, result set, populate #variable, if [not] exists() construct, case statement).
So, addressing the additional details provided in the comments ...
Assuming you're submitting a single batch that needs to determine table existence prior to running the desired query(s):
-- if table exists, run query(s); obviously if table does not exist then query(s) is not run
if exists(select 1 from sysobjects where name = 'table-name' and type = 'U')
begin
execute("select * from table-name")
end
execute() is required to keep the optimizer from generating an error that the table does not exist, ie, the query is not parsed/compiled unless the execute() is actually invoked
If your application can be written to use multiple batches, something like the following should also work:
# application specific code; I don't work with java but the gist of the operation would be ...
run-query-in-db("select 1 from sysobjects where name = 'table-name' and type = 'U'")
if-query-returns-a-row
then
run-query-in-db("select * from table-name")
fi
This is the way of checking if the table exists and drop it:
IF EXISTS (
SELECT 1
FROM sysobjects
WHERE name = 'a_table'
AND type = 'U'
)
DROP TABLE a_table
GO
And this is how to check if a table exists and create it.
IF NOT EXISTS (
SELECT 1
FROM sysobjects
WHERE name = 'a_table'
AND type = 'U'
)
EXECUTE("CREATE TABLE a_table (
col1 int not null,
col2 int null
)")
GO
(They are different because in table-drop a temporary table gets created, so if you try to create a new one you will get an exception that it already exists)
Before running the query which has some risk in table not existing, run the following sql query and check if the number of results is >= 1. if it is >= 1 then you are safe to execute the normal query. otherwise, do something to handle this situation.
SELECT count(*)
FROM information_schema.TABLES
WHERE (TABLE_SCHEMA = 'your_db_name') AND (TABLE_NAME = 'name_of_table')
I am no expert in Sybase but take a look at this,
exec sp_tables '%', '%', 'master', "'TABLE'"
Sybase Admin

jpql left join fetch not returning results for like

In a spring mvc app using hibernate and MySQL, I have written the following query method to return a list of names with patients:
#SuppressWarnings("unchecked")
public Collection<Person> findPersonByLastName(String ln) throws DataAccessException{
Query query = this.em.createQuery("SELECT DISTINCT pers FROM rimPerson pers left join fetch pers.names nm WHERE nm.family LIKE :lnm");
query.setParameter("lnm", ln);
return query.getResultList();
}
This is producing the following hibernate sql:
Hibernate:
select distinct
person0_.hppid as hppid1_340_0_,
names1_.HJID as HJID1_89_1_,
person0_2_.classCode_HJID as classCod2_339_0_,
person0_1_.administrativeGenderCode_HJID as administ2_341_0_,
person0_1_.birthTime_HJID as birthTim3_341_0_,
names1_.DELIMITER_ as DELIMITE2_89_1_,
names1_.FAMILY as FAMILY3_89_1_,
names1_.named_entity_hppid as named5_89_1_,
names1_.SUFFIX as SUFFIX4_89_1_,
names1_.name_entity_HJID as name9_340_0__,
names1_.HJID as HJID1_89_0__
from
rim_person person0_ inner join rim_living_subject person0_1_ on person0_.hppid=person0_1_.hppid
inner join rim_entity person0_2_ on person0_.hppid=person0_2_.hppid
inner join rim_infrastructure_root person0_3_ on person0_.hppid=person0_3_.hppid
left outer join EN names1_ on person0_.hppid=names1_.name_entity_HJID
where names1_.FAMILY like ?
When I call the above jpql method with the following command, it returns zero results:
this.myappService.findPersonByLastName("");
I also get zero results when I cut and past the above generated hibernate code into the MySQL command line client and replace ? with ''.
If, however, I remove the where names1_.FAMILY like ? from the hibernate generated sql above and place the shortened sql into the MySQL command line client, I get four results, eachof which has a value for the lastname field.
How can I change the jpql so that it generates a hibernate query that returns the four results when `` is passed as the empty string parameter? I want the result set to include every result when the user gives empty input, but to give filtered results when the user types in any given text input.
The typical reason that like fails to do what you think it ought to do is to forget to put a wildcard in the pattern string. For example, if you want to match all user names that begin with 'Code' you must do something like name like 'Code%', NOT name like 'Code'. You can control exactly what your predicate matches with careful placement of %s in your string.
Try this to see all entities no matter what the value in family:
this.myappService.findPersonByLastName("%");
It is kinda cheesy to have the caller of findPersionByLastName have to put in the % wildcard. A better implementation is to have the caller specify which last name they are looking for, and then have the code that constructs the query put the wildcard in the right place. When you are looking for last names, you might do something like this:
query.setParameter("lnm", "%" + ln);
That would match anything that ends with the parameter that is passed to the method.

Hibernate returns list of nulls although executed SQL returns values

I'm using hibernate as an ORMapper. I want to execute an actually rather simple hql query:
SELECT a
FROM Foo a
WHERE a.status = :A0status
ORDER BY a.bookingTypeCode ASC,
a.priority ASC
This hql query is then converted into a sql query which looks something like this:
select a.*
from Foo a
where a.status='A'
order by a.bookingtypecode ASC,
a.priority ASC
When I execute the sql on the oracle database using the Oracle SQL Developer I get 17 rows returned. However, when I execute the hql query (using the list method of a Query I get a list of 17 elements that are all null. Although the number of elements is correct, not a single one of the elements is actually loaded.
This is the way I create and execute my query:
// the hql query is stored in the hqlQuery variable;
// the parameter are stored in a Map<String, Object> called params
Query hQuery = hibSession.createQuery(hqlQuery);
for (Entry<String, Object> param : params.entrySet()) {
String key = param.getKey();
Object value = param.getValue();
hQuery.setParameter(key, value);
}
List<?> result = hQuery.list();
Does anyone know what might be the problem here?
Update 1
I've recently upgrade from hibernate 3.2 to 4.3.5. Before the upgrade everything worked fine. After the upgrade I get this error.
I've set the Log level of hibernate to TRACE and found the problem. It was actually a mapping/logic/database error. The primary key consisted of two columns (according to the entity class) and one of these columns was nullable. However a primary key can never be nullable. Therefore hibernate always returned null.
If you have not set a custom (and buggy) ResultTransformer, my second best guess is that your debugger is lying to you. Does you code actually receives a list of null?
Also make sure to test with the code you are showing is. Too many times, people simplify things and the devil is in the details.
This error is happening to me. MySQL query browser works, but in hibernate of 7 columns and only one column always came with all null fields. I checked all the ids and they were not null. The error was in the construction of SQL Native. I had to change the way of writing it. Ai worked.
SELECT c.idContratoEmprestimo as idContratoEmprestimo,
c.dtOperacao as dataOperacao,
p.cpf as cpf,
p.nome as nome,
(Select count(p2.idParcelaEmprestimo) from EMP_PARCELA p2 where p2.valorPago > 0 and p2.dtPagamento is not null
and p2.idContratoEmprestimo = c.idContratoEmprestimo and p2.mesCompetencia <= '2014-08-01') as parcelasPagas, c.numeroParcelas as numeroParcelas,
pe.valorPago as valorParcela
FROM EMP_CONTRATO c inner join TB_PARTICIPANTE_DADOS_PLANO AS pp on pp.idParticipantePlano = c.idParticipantePlano
inner join TB_PARTICIPANTE as p on p.id = pp.idParticipante
inner join TB_PARTICIPANTE_INSTITUIDOR as pi on pi.PARTICIPANTE_ID = p.id
inner join EMP_PARCELA as pe on pe.idContratoEmprestimo = c.idContratoEmprestimo
where c.dtInicioContrato <= '2014-08-01' and pi.INSTITUIDOR_ID = 1
and c.avaliado is true
and pe.mesCompetencia = '2014-08-01'
and c.deferido is true
and c.dtQuitacao is null
and c.dtExclusao is null
and pe.valorPago is not null
group by c.idContratoEmprestimo
order by p.nome

Hibernate query not returning correct value

So in my database, I have 3 rows, two rows have defaultFlag as 0 and one is set to 1, now in my processing am updating defaultProperty of one object to 1 from 0 but am not saving this object yet.
Before saving I need to query database and find if any row has defaultFlag set or not, there would be only 1 default set.
So before doing update am running query to find if default is set and i get 2 values out, note here if i go and check in db then there is only 1 row with default set but query gives me two result because this.object default property has changed from 0 to 1 but note that this object is not yet saved in database.
I am really confused here as to why hibernate query is returning 2 when there is one row with default set in database and other object whose default property has changed but it is not saved.
Any thoughts would be helpful. I can provide query if need be.
Update
Following suggestions, I added session.clear() to before running the query.
session.clear();
String sql = "SELECT * FROM BANKACCOUNTS WHERE PARTYID = :partyId AND CURRENCYID = :currencySymbol AND ISDEFAULTBANKACCOUNT= :defaultbankAccount";
SQLQuery q = session.createSQLQuery(sql);
q.addEntity(BankAccount.class);
q.setParameter("partyId", partyId);
q.setParameter("currencySymbol", currencySymbol);
q.setParameter("defaultbankAccount", 1);
return q.uniqueResult();
and it returns 1 row in result as expected but now am getting
nested exception is org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session exception
Either query which row has the "default flag" set before you start changing it, or query for a list of rows with default flag set & clear all except the one you're trying to set.
Very easy, stop mucking about with your "brittle" current approach which will break in the face of concurrency or if data is ever in an inconsistent state. Use a reliable approach instead, which will always set the data to a valid state.
protected void makeAccountDefault (BankAccount acc) {
// find & clear any existing 'Default Accounts', other than specified.
//
String sql = "SELECT * FROM BANKACCOUNTS WHERE PARTYID = :partyId AND CURRENCYID = :currencySymbol AND ISDEFAULTBANKACCOUNT= :defaultbankAccount";
SQLQuery q = session.createSQLQuery(sql);
q.addEntity(BankAccount.class);
q.setParameter("partyId", partyId);
q.setParameter("currencySymbol", currencySymbol);
q.setParameter("defaultbankAccount", 1);
//
List<BackAccount> existingDefaults = q.list();
for (BankAccount existing : existingDefaults) {
if (! existing.equals( acc))
existing.setDefaultBankAccount( false);
}
// set the specified Account as Default.
acc.setDefaultBankAccount( true);
// done.
}
This is how you write proper code, do it simple & reliable. Never make or depend on weak assumptions about the reliability of data or internal state, always read & process "beforehand state" before you do the operation, just implement your code clean & right and it will serve you well.
I think that your second query won't be executed at all because the entity is already in the first level cache.
As your transaction is not yet commited, you don't see the changes in the underlying database.
(this is only a guess)
That's only a guess because you're not giving many details, but I suppose that you perform your myObject.setMyDefaultProperty(1) while your session is open.
In this case, be careful that you don't need to actually perform a session.update(myObject) to save the change. It is the nominal case when database update is transparently done by hibernate.
So, in fact, I think that your change is saved... (but not commited, of course, thus not seen when you check in db)
To verify this, you should enable the hibernate.show_sql option. You will see if an Update statement is triggered (I advise to always enable this option in development phase anyway)

Categories