We Keep Coding

Stream very large table via spring-data-jdbc Resultset.TYPE_FORWARD_ONLY READ_ONLY

Hi I would like to stream a very large table spring-data-jdbc. For this purpose
I have set my connection to READ_ONLY I have declared in my repository a method that looks in the following way:
PackageRepository extends Repository<Package,String> {
Stream<Package> findAll();
}
My expectation here would be that the resultset would be of type FORWARD_ONLY and this method will not block indefinatly untill all results are recieved from the database.
Here I would make a comparison with Spring Data JPA where the Stream methods are not blocking and the content of the database is fetched in portions depending on the fetch size.
Have I missed some configuration ? How can I achieve this behaviour with spring-data-jdbc ?
UPDATE: I will put the question in a different form. How can I achieve with spring-data-jdbs the equivalent of:
template.query(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection con) throws SQLException {
PreparedStatement statement = con.prepareStatement("select * from MYTABLE with UR",ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
statement.setFetchSize(150000);
return statement;
}
}, new RowCallbackHandler() {
#Override
public void processRow(ResultSet rs) throws SQLException {
// do my processing here
}
});

Just adding thesetFetchSize(Integer.MIN_VALUE) before querying, the queryForStream indeed gives us a stream which load records one by one rather than eagerly load all records into memroy in one shot.
namedTemplate.getJdbcTemplate().setFetchSize(Integer.MIN_VALUE);
Stream<LargeEntity> entities = namedTemplate.queryForStream(sql, params, rowMapper);
dependencies:
spring framework 5.3+
mysql-connector-java 8.0.x (or mariadb-java-client 2.7.x)

How to assign all the '?' with object's data in JDBC template?

I am new in spring boot.
I have created on method in spring boot, please see the below function:
public ArrayList<ShipmentDetailsVO> getShipmentStatus(
ShipmentDetailsVO shpmntpert) {
return jdbcTemplate.query("select * from SELECT_SEARCH_DETAILS(?,?,?,?,?,?,?,?)",new ResultSetExtractor<ArrayList<ShipmentDetailsVO>>(){
#Override
public ArrayList<ShipmentDetailsVO> extractData(ResultSet rs) throws SQLException,
DataAccessException {
shipmentDao = new ArrayList<ShipmentDetailsVO>();
while(rs.next()) {
shipmentDetDaoObj =new ShipmentDetailsVO();
shipmentDetDaoObj.setContractNumber(rs.getString(1));
System.out.println("hello" + rs.getString(1));
shipmentDetDaoObj.setOrderNumber(rs.getString(2));
System.out.println(rs.getString(2));
shipmentDetDaoObj.setShipmentNumber(rs.getString(3));
shipmentDetDaoObj.setShipmentControlNo(rs.getString(4));
shipmentDetDaoObj.setStatusCode(rs.getString(5));
shipmentDetDaoObj.setStatusDateStr(rs.getString(6));
shipmentDetDaoObj.setLastUpdatedtStr(rs.getString(7));
shipmentDetDaoObj.setResendFlag(false);
shipmentDetDaoObj.setSourceSystem(rs.getString(8));
shipmentDetDaoObj.setDestinationSystem(rs.getString(9));
shipmentDetDaoObj.setRfid(rs.getString(10));
shipmentDetDaoObj.setUid(rs.getString(11));
shipmentDetDaoObj.setShipmentSeqId(rs.getString(12));
shipmentDao.add(shipmentDetDaoObj);
}
return shipmentDao;
}
});
}
I don't have any idea how to assign all the '?' with object's(shpmntpert) data. Can any one kindly help on the same.

From JdbcTemplate documentation
You can find one of the query the signature
public <T> T query(String sql, Object[] args, ResultSetExtractor<T> rse)
throws DataAccessException
With an explanation of the interesting parameters
sql - SQL query to execute
args - arguments to bind to the query
So you can see that the array is used to bing the values. That means that for a query like
select * from table where name = ? and weight= ?
you need to generate an array with those values in the correct order
new Object[]{name, weight}
An other signature show you that they provide some setter for PreparedStatement named PreparedStatementSetter.
public <T> T query(String sql,
PreparedStatementSetter pss,
ResultSetExtractor<T> rse)
throws DataAccessException
Where the PreparedStatementSetter interface have only one methods to implements
void setValues(PreparedStatement ps)
That will let you set the values like you want fron this setter. You could pass the instance to an implementation of this interface or directly in the code like
final MyInstance m = new MyInstance();
jdbcTemplate.query("select * from table where name = ? and weight= ?",
new PreparedStatementSetter(){
#Override
public void setValues(PreparedStatement ps){
ps.setString(1, m.getName);
ps.setInt(2, m.getWeight);
}
}, myResultSetExtractor);
Note : This is a quick reading of the documentation knowing that I never used this API. But from what I have read, this should work just fine.

Custom mapper class is not getting the result?

I am writing a DAO application and I want to get zone data. I have written my own mapper class. I am unable to get values. It is throwing an exception.
Mapper class
public class zoneMapper implements RowMapper {
#Override
public Object mapRow(ResultSet resultSet, int rowNum) throws SQLException {
ZoneBean zone=new ZoneBean();
zone.setZoneId(resultSet.getInt("ZONE_ID"));
zone.setZoneName(resultSet.getString("ZONE_NAME"));
return zone;
}
dao call
List<ZoneBean> zoneList=new ArrayList<ZoneBean>();
try {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
zoneList = (List<ZoneBean>) jdbcTemplate.queryForObject(
queriesConstants.GET_ZONES, new zoneMapper());
}catch(Exception e){
System.out.println("getZones "+e.getMessage());
//logger.error("getZones "+e.getMessage());
}
**Exception is**
getZones Incorrect result size: expected 1, actual 17

getZones Incorrect result size: expected 1, actual 17
You retrieve multiple elements when your execute your query.
public <T> T queryForObject(String sql, RowMapper<T> rowMapper) is therefore not suitable for your need because it executes a query given static SQL, mapping a single result row to a Java object via a RowMapper.
You should use rather this method :
public <T> List<T> query(String sql, RowMapper<T> rowMapper)which executes a query given static SQL, mapping each row to a Java object via a RowMapper.

Boolean value not getting mapped properly in Hibernate with MySQL

I am trying to map the result of an exists query (which returns TRUE/FALSE) from a MySQL database to a POJO via resultSetTransformer. I would hope the result of this exists query can get mapped to a boolean but it does not and throws the below error:
org.hibernate.PropertyAccessException: IllegalArgumentException
occurred while calling setter of TestBean.value
The cause of this exception is shown as:
java.lang.IllegalArgumentException: argument type mismatch
My sample class:
public class TestHibernate {
private static SessionFactory sessionFactory = HibernateUtil.getSessionFactory();
public static void main(String[] args) throws ParseException {
try {
Query query = sessionFactory.openSession().createSQLQuery(
"SELECT " +
"EXISTS (SELECT * FROM A WHERE id = 3) AS value"
);
query.setResultTransformer(Transformers.aliasToBean(TestBean.class));
List<TestBean> beanList = (List<TestBean>) query.list();
} catch (Exception e) {
System.out.println(e);
}
}
}
The POJO:
public class TestBean {
private boolean value;
public boolean isValue() {
return value;
}
public void setValue(boolean value) {
this.value = value;
}
}
Am I missing something or is it a bug with Hibernate or MySQL JDBC Driver?
Hibernate version: 3.2.6GA
MySQL JDBC Driver: mysql-connector-java-5.1.2

Hibernate has a built-in "yes_no" type that would do what you want. It maps to a CHAR(1) column in the database.
Basic mapping:
<property name="some_flag" type="yes_no"/>
Annotation mapping (Hibernate extensions):
#Type(type="yes_no")
public boolean getFlag();

Your problem may be caused by the case mapping of the selected column, q.v. my solution below. I also parametrized your WHERE clause to avoid SQL injection.
Query query = session.createSQLQuery("select exists(select * from A where id = :id) as value");
.setParameter("id", "3");
.addScalar("value")
.setResultTransformer( Transformers.aliasToBean(TestBean.class))
List result = query.list();
TestBean theBean = (TestBean)result.get(0);

The transform of the result query can be explicitly set each parameter to the corresponding model datatype using hibernate addScalar() method. Please find the solution below.
Query query = sessionFactory.openSession().createSQLQuery(""+
"select " +
" exists(select * from A where id = 3) as value"
).addScalar("value", BooleanType.INSTANCE);
This will resolve to set to the Boolean value.

I know this is old answer, I tried to resolve this coz answer from here not worked for me.
with Addition to Answer from #anil bk, I overloaded a setter method accepting String as argument. Now It worked as expected.
public void setPriority(String priority) {
this.priority = "true".equals(priority);
}
Here is my answer

spring jdbc and composite primary keys

Is there a way in spring jdbc to return a composite primary key when a row is inserted.
This composite primary key is made up of values from separate sequences
Any help is greatly appreciated
Regards
Damien

Here is a full example (tested on PostgreSQL 8.4):
My table:
CREATE TABLE test
(
id serial NOT NULL,
otherid serial NOT NULL,
val text,
CONSTRAINT test_pkey PRIMARY KEY (id, otherid)
)
This is how you get keys back:
public void doStuff() {
KeyHolder keyHolder = new GeneratedKeyHolder();
jdbcTemplate.update(
new PreparedStatementCreator() {
public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
PreparedStatement ps = connection.prepareStatement("insert into test(val) values (?)", Statement.RETURN_GENERATED_KEYS);
ps.setInt(1, 42);
return ps;
}
},
keyHolder);
keyHolder.getKeys().get("id");
keyHolder.getKeys().get("otherid");
}
Now, if you want to get your composite key as an instance of some class directly from keyHolder, it is not simple.
JdbcTemplate uses ColumnMapRowMapper to map generated keys (generated keys are returned as result set, at least on PostgreSQL. It actually returns the whole row as if you were executing select on the row you just inserted). Same ColumnMapRowMapper is used in number of other places in JdbcTemplate.
The only possible point of extension here is KeyHolder itself. Here is what you can do:
public void doStuff() {
CompositeKeyHolder keyHolder = new CompositeKeyHolder();
... same code here ...
keyHolder.getCompositeKey();
}
class CompositeKeyHolder extends GeneratedKeyHolder {
private boolean converted;
public CompositeKey getCompositeKey() {
return new CompositeKey((Integer)this.getKeys().get("id"), (Integer)this.getKeys().get("otherid"));
}
}
class CompositeKey {
private Integer id;
private Integer otherId;
CompositeKey(Integer id, Integer otherId) {
this.id = id;
this.otherId = otherId;
}
public Integer getId() {
return id;
}
public Integer getOtherId() {
return otherId;
}
}

Here is the basic idea for a single key. The long id at the end is the key. If you have multiple sequences, I would recommend just using two separate statements to get each generated key.
JdbcTemplate template = getJdbcTemplate();
KeyHolder keyHolder = new GeneratedKeyHolder();
template.update(
new PreparedStatementCreator() {
public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
PreparedStatement ps = connection.prepareStatement(...);
return ps;
}
},
keyHolder);
long id = keyHolder.getKey().longValue();

What database server are you using? MySQL only allows one auto_increment field per table and I'd imagine this is often the case, but without knowing your setup it's hard to say. Assuming there is only one auto_generated field in your table, your INSERT would have had to be aware of the value going into the second PK field. Robert's code should work for retrieving the generated key value, and the cleanest solution would probably be to perform a SELECT after the fact using this generated key and the value which you had a hold of already.

I think what you need is GeneratedKeyHolder.getKeys(). Code would look like this example, except you will have to call
keyHolder.getKeys()
instead of
keyHolder.getKey()