I need to fetch many records from an RDBMS in Java (10-20k) my target system expects them to be available as Java List. So I want to implement my code as "Virtual list" where I actually only fetch the records I actually need. I expect SQL like
SELECT * FROM CUSTOMER WHERE COUNTRY="Moldovia"
as parameter and just return what is requested. Most likely the data is requested in batches of 50. Any hints how to do that?
Unless you expect your clients to randomly access the data, you're probably better off returning an Iterator. Also, take a look at ResultSet.setFetchSize: http://java.sun.com/javase/6/docs/api/java/sql/ResultSet.html#setFetchSize(int)
So something like:
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Iterator;
public class FooResultSetIterator implements Iterator<Foo>
{
private final ResultSet resultSet;
private boolean hasNext;
FooResultSetIterator(final ResultSet resultSet, final int fetchSize) throws SQLException
{
this.resultSet = resultSet;
this.resultSet.setFetchSize(fetchSize);
this.hasNext = resultSet.next();
}
#Override
public boolean hasNext()
{
return hasNext;
}
#Override
public Foo next()
{
final Foo foo = new Foo(resultSet);
try
{
this.hasNext = resultSet.next();
}
catch (final SQLException e)
{
throw new RuntimeException(e);
}
return foo;
}
#Override
public void remove()
{
throw new UnsupportedOperationException("Cannot remove items from a ResultSetIterator");
}
}
class Foo
{
public Foo(ResultSet resultSet)
{
// TODO Auto-generated constructor stub
}
}
Use OFFSET and LIMIT in your query:
SELECT * FROM CUSTOMER WHERE
COUNTRY="Moldovia" LIMIT 50 OFFSET 50
Assuming of course that your SQL dialect allows it. The example will return rows 51-100.
Related
I'm using the foundationDB SQL parser (https://github.com/FoundationDB/sql-parser) to parse a query inside Java, but I'm not very familiar with the visitor design pattern, that is used by the parser to consume a query.
I would like to send a query to the parser like this: "SELECT a, b FROM c WHERE d > 5" and get as a result:
all the fields names in the SELECT clause (accomplished)
the table name into the FROM clause (accomplished)
the columns names, operand and expression into the WHERE clause
Thats the code I'm implementing:
#Override
public QueryDescription parse() throws StandardException {
SQLParser parser = new SQLParser();
StatementNode stmt = parser.parseStatement(sql);
Visitor v = new Visitor() {
#Override
public boolean visitChildrenFirst(Visitable arg0) {
// TODO Auto-generated method stub
return false;
}
#Override
public Visitable visit(Visitable arg0) throws StandardException {
// Temporary stores the QueryDescription parameters
StatementEnum se = null;
String fromTable = null;
String[] fields = null;
if(arg0 instanceof CursorNode) {
CursorNode cn = (CursorNode) arg0;
// print out what statement is been declared in sql query
System.out.println("Statement: " + cn.statementToString());
// temporarly stores the statement
String statement = cn.statementToString();
// creates the right StatementEnum
if(statement == "CREATE TABLE") {
se = StatementEnum.CREATE_TABLE;
} else if(statement == "INSERT") {
se = StatementEnum.INSERT;
} else if(statement == "SELECT") {
se = StatementEnum.SELECT;
} else if(statement == "DROP TABLE") {
se = StatementEnum.DROP_TABLE;
}
}
description = new QueryDescription(se, fromTable, fields);
return arg0;
}
#Override
public boolean stopTraversal() { return false; }
#Override
public boolean skipChildren(Visitable arg0) throws StandardException { return false; }
};
stmt.accept(v);
// TODO remove, only for debug purpose
stmt.treePrint();
return description;
}
And that's the QueryDescription class code:
public class QueryDescription {
/* Member variables: */
private QueryTypeEnum queryType;
private StatementEnum statement;
private String fromTable;
private String[] fields;
/* Constructors: */
/**
*
* #param statement
* #param fromTable
* #param fields
*/
public QueryDescription(StatementEnum statement, String fromTable, String[] fields) {
this.statement = statement;
this.fromTable = fromTable;
this.fields = fields;
}
/* Methods: */
/**
* Analyze which type of query is the one passed by parameter and assigns the right queryTypeEnum
*/
public void assignType() {
switch(statement) {
case CREATE_TABLE:
break;
case SELECT:
if(fields[0] == "allFields")
queryType = QueryTypeEnum.DUMP;
else {
// TODO risolvere questione del WHERE
queryType = QueryTypeEnum.SELECT_FROM;
}
break;
case UPDATE:
break;
case INSERT:
break;
case DROP_TABLE:
break;
}
}
/* Getters and Setter: */
/**
*
* #return the queryType
*/
public QueryTypeEnum getQueryType() {
return queryType;
}
/**
*
* #return the statement
*/
public StatementEnum getStatement() {
return statement;
}
/**
*
* #return the from table
*/
public String getFromTable() {
return fromTable;
}
/**
*
* #return the fields
*/
public String[] getFields() {
return fields;
}
}
Your code does not show what the QueryDescription class does, but I can guess.
In dealing with the where clause, you are looking for three types of nodes:
BinaryLogicalOperatorNode - This has the AND, OR, IS operators that separate the individual clauses in the WHERE clause.
BinaryOperatorNode - This has the individual >, <, and other operations.
The constants and column operators.
In your example, you would visit a BinaryOperatorNode, with a type of LT, and the two children would be ConstantNode with a value of 5, and a ColumnReference node with the value of "d".
Note: the parser does not connect the Column references to the tables in the tables list. This is a separate step in the query processing. So you will have a ColumnReference node, but no link to which table the column is referencing. The reason is the parser does not have enough information to correctly link the "d" column to the correct table.
This is enough to process the simple query you gave in the example. Obviously queries can become much more complex.
The one node I would add to the list of checks in the InListOperatorNode which handles WHERE d IN (1,2,3,4).
EDIT to add:
Keep in mind the Vistor#visit() method gets called for each and every node in the tree created by the query parser. The visit method will need to check, and set your variables properly, for FromBaseTable, ConstantNode, ColumnReference.
Visitor v = new Visitor() {
List<String> fromTable = new ArrayList<String>();
List<String> fields = new ArrayList<String>();
// other Visitor methods go here, not copied for conciseness.
#Override
public Visitable visit(Visitable arg0) throws StandardException {
// other code from your visit() method goes here
//
if (arg0 instanceof FromBaseTable) {
FromBaseTable table = (FromBaseTable)arg0;
fromTable.append(table.getTableName());
} else if (arg0 instanceof ColumnReference) {
ColumnReference column = (ColumnReference) arg0;
fields.append(column.getColumnName())
}
// Remove the call to create QueryDescription
}
public QueryDescription getQueryDescription() {
return new QueryDescription(se, fromTable, fields)
}
}
Then in your main line of code you call:
stmt.accept(v);
QueryDescription description = v.getQueryDescription();
Now if you have other parts of the query you are interested in, you need to add those nodes to the visit method, and capture the part of the node (names, values, etc) you are interested in.
I want to display list of record from database to output text field. I'm having problem with method with brings record from database. It is causing an infinite loop as it call in constructor of managed bean class. Here is the code.
Constructor of managed bean class:
public InterViewDto() throws SQLException {
User u = getCurrentUser();
InterviewDao d = new InterviewDao();
List<InterViewDto> dao1 = d.getCall(u.getEmailAddress());
setDto(dao1);
}
Method bringing record from database :
public List<InterViewDto> getCall(String email) throws SQLException {
System.out.print("fyc");
List<InterViewDto> list = new ArrayList<InterViewDto>();
String job = null;
boolean exists = false;
Connection c = null;
try {
c = openConnection();
String query_check = "SELECT * FROM interviewcall WHERE useremail = '"+email+"' ";
Statement st = c.createStatement();
ResultSet rs = st.executeQuery(query_check);
while (rs.next()) {
InterViewDto dto = new InterViewDto();
dto.setDate( rs.getDate("time"));
dto.setJobtitle( rs.getString("jobtitle"));
dto.setJobtitle( rs.getString("useremail"));
list.add(dto);
System.out.print(list.get(0).getJobtitle());
} rs.close();
} catch (Exception e) {
System.out.println(e);
} finally {
c.close();
}
return list;
}
You have a circular dependency. Your constructor for the DTO reaches out to the database, which in turn creates a new DTO to represent the data loaded from the database, which goes to the database and back and forth until you overflow the call stack.
Quite simply, you have merged two complementary design approaches.
Either your InterViewDto constructor loads data from the DAO or the DAO constructs a new InterViewDto object. Pick one or the other.
In my opinion, it makes more sense for the DAO to create the DTO objects. If you want the DTO to delegate to the DAO as a matter of convenience, consider a static method.
public class InterViewDto {
public InterViewDto() {
}
...
public static fromCurrentUser() {
return new InterviewDao().getCall(getCurrentUser().getEmailAddress());
}
}
Then change your constructor to be empty.
I'm implementing a method that does something like:
...
try {
myPojo.setProperty("foo");
myService.execute(myPojo);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
...
If some exception is thrown by my service from this try block on pojo property will have the new value. Is there some way to start a kind of transaction for pojo changes and roll it back if something goes wrong?
Something like:
PojoTransaction pt = startPojoTransaction();
transactionedPojo = pt.handleByTransaction(myPojo);
try {
transactionedPojo.setProperty("foo");
myService.execute(transactionedPojo);
pt.commit;
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
Or something similar...
Take a look at the Memento Pattern, it includes a Java example.
http://en.wikipedia.org/wiki/Memento_pattern
I toyed around with the idea, this is far from perfect, just a simple proof of concept. There are pitfalls in this implementation:
It only tries to call a parameterless constructor of the given source
object to create the target-copy, would need some logic to select a correct constructor (or only support Cloneables?)
Only copies fields declared in the class, not from superclasses (this problem can be solved walking through the inheritance tree and copying any superclass fields)
If the fields are complex types, only the references are copied to target-object, so any changes to them will not be transactional, as both the source and target share the same instance (solvable by recursively creating copies of nested objects and copying their values, requires walking through the entire object-graph, starting from source, and then doing it vice-versa on commit-time)
But, improving from here, I believe it could become very usable. Here's the POC:
import java.lang.reflect.Field;
import org.junit.Assert;
import org.junit.Test;
public class PojoTransactionTest
{
public static class PojoTransaction<T>
{
/**
* This is the original (unmodified) object
*/
private T source;
/**
* This is the object modified by within the transaction
*/
private T target;
/**
* Creates a new transaction for the given source object
* #param source Source object to modify transactionally
*/
public PojoTransaction(T source)
{
try
{
this.source = source;
this.target = (T)source.getClass().newInstance(); //Note: this only supports parameterless constructors
copyState(source, target);
}
catch(Exception e)
{
throw new RuntimeException("Failed to create PojoTransaction", e);
}
}
/**
* Copies state (member fields) from object to another
* #param from Object to copy from
* #param to Object to copy to
* #throws IllegalAccessException
*/
private void copyState(T from, T to) throws IllegalAccessException
{
//Copy internal state to target, note that this will NOT copy fields from superclasses
for(Field f : from.getClass().getDeclaredFields())
{
f.setAccessible(true);
f.set(to, f.get(from));
}
}
/**
* Returns the transaction target object, this is the one you should modify during transaction
* #return Target object
*/
public T getTransactionTarget()
{
return target;
}
/**
* Copies the changes from target object back to original object
*/
public void commit()
{
try
{
copyState(target, source);
}
catch(Exception e)
{
throw new RuntimeException("Failed to change state of original object", e);
}
}
}
public static class TestData
{
private String strValue = "TEST";
private int intValue = 1;
private float floatValue = 3.1415f;
public String getStrValue()
{
return strValue;
}
public void setStrValue(String strValue)
{
this.strValue = strValue;
}
public int getIntValue()
{
return intValue;
}
public void setIntValue(int intValue)
{
this.intValue = intValue;
}
public float getFloatValue()
{
return floatValue;
}
public void setFloatValue(float floatValue)
{
this.floatValue = floatValue;
}
}
#Test
public void testTransaction()
{
//Create some test data
TestData orig = new TestData();
//Create transaction for the test data, get the "transaction target"-object from transaction
PojoTransaction<TestData> tx = new PojoTransaction<TestData>(orig);
TestData target = tx.getTransactionTarget();
target.setFloatValue(1.0f);
target.setIntValue(5);
target.setStrValue("Another string");
//Original object is still at the original values
Assert.assertEquals(1, orig.getIntValue());
Assert.assertEquals(3.1415f, orig.getFloatValue(), 0.001f);
Assert.assertEquals("TEST", orig.getStrValue());
//Commit transaction
tx.commit();
//The "orig"-object should now have the changes made to "transaction target"-object
Assert.assertEquals(5, orig.getIntValue());
Assert.assertEquals(1.0f, orig.getFloatValue(), 0.001f);
Assert.assertEquals("Another string", orig.getStrValue());
}
}
The question is a bit vague, but it sounds like you are wrestling with the basic design pattern for transaction management. You would benefit greatly from the experience that has gone into the production of the pattern used here:
http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/transaction.html
Perhaps Spring Transaction management would suit you well for your project anyway.
NOTE: Please ignore my use of MultivaluedMap instead of multiple vargs String...args.
Is there a standard way in java of doing this?
What I have is a resource, that is returned from a remote server. But before each query, the remote connection must be open, and after the returns are returned - it must be closed.
So a natural way of doing this is something like:
Connection c = config.configureConnection();
c.open(); //open
List<Car> cars;
try{
cars = c.getCars();
}finally{
c.close(); //close
}
Now I want to implement something that operates on the level of the resources themselves, without worrying about connection, for example:
List<Car> cars = new CarResource().all(); //opens and closes connection
The way I am currently doing it is by having one abstract class, AbstractQueriable call abstract methods query(String ...args) and query(int id), which any class extending it must implement.
The AbstractQuerieable implements the Queriable interface, which makes it expose the three public methods filter(String ...args), all() and get(int id) - which are the public facing methods.
Here is the Queriable interface:
public interface Queriable <T>{
public T get(String id);
/** Simply returns all resources */
public Collection<T> all();
public Collection<T> filter(MultivaluedMap<String, String> args);
}
here is the AbstractQueriable class that implements it:
public abstract class AbstractQueriable<T> implements Queriable<T> {
#Override
public final T get(String id) {
setup();
try {
return query(id);
} finally {
cleanup();
}
}
#Override
public final Collection<T> filter(MultivaluedMap<String, String> args) {
setup();
try {
return query(args);
} finally {
cleanup();
}
}
/**
* Returns all resources.
*
* This is a convenience method that is equivalent to passing an empty
* arguments list to the filter function.
*
* #return The collection of all resources if possible
*/
#Override
public final Collection<T> all() {
return filter(null);
}
/**
* Queries for a resource by id.
*
* #param id
* id of the resource to return
* #return
*/
protected abstract T query(String id);
/**
* Queries for a resource by given arguments.
*
* #param args
* Map of arguments, where each key is the argument name, and the
* corresponing values are the values
* #return The collection of resources found
*/
protected abstract Collection<T> query(MultivaluedMap<String, String> args);
private void cleanup() {
Repository.close();
}
private void setup() {
Repository.open();
}
and finally my resource, which I want to use in the code, must extend the AbstractQueriable class, for example (please note that the details of these methods are not important):
public class CarRepositoryResource extends AbstractQueriable<Car> {
#Override
protected Car query(String id) {
MultivaluedMap<String, String> params = new MultivaluedMapImpl();
params.add("CarID", id);
// Delegate the query to the parametarized version
Collection<cars> cars = query(params);
if (cars == null || cars.size() == 0) {
throw new WebApplicationException(Response.Status.NOT_FOUND);
}
if (cars.size() > 1) {
throw new WebApplicationException(Response.Status.NOT_FOUND);
}
return cars.iterator().next();
}
#Override
protected Collection<Car> query(MultivaluedMap<String, String> params) {
Collection<Car> cars = new ArrayList<Car>();
Response response = Repository.getConnection().doQuery("Car");
while (response.next()) {
Returned returned = response.getResult();
if (returned != null) {
cars.add(returned);
}
}
return cars;
}
}
which finally, I can use in my code:
Collection<Car> cars = new CarRepositoryResource().all();
//... display cars to the client etc...
There are a few things I don't like about this kind of setup:
I must instantiate a new instance of my "CarRepositoryResource" every time I do a query.
The method names "query", while internal and private, are still confusing and clunky.
I am not sure if there is a better pattern or framework out there.
The connection that I am using does not support/implement the JDBC api and is not sql-based.
You could use a variation of the (in)famous Open session in view pattern.
Basically it comes down to this:
Define a "context" in which connections are available
(usually the request in web applications)
Handle (possibly lazy) initialization and release of a connection when entering/exiting the context
Code your methods taking for granted they will only be used inside such a context
It is not difficult to implement (storing the connection in a static ThreadLocal to make it thread safe) and will definitely spare a few open/close calls (performance-wise that could be a big gain, depending on how heavy your connection is).
The context class could look something like (consider this pseudo-code);
public class MyContext{
private static final
ThreadLocal<Connection> connection = new ThreadLocal<Connection>();
public static void enter() {
connection.set(initializeConnection());
// this is eager initialization
// if you think it will often the case that no connection is actually
// required inside a context, you can defer the actual initialization
// until the first call to get()
}
public static void exit() {
try { connection.close(); }
catch(Throwable t) { /* panic! */ }
finally { connection.set(null); }
}
public static Connection get() {
Connection c = connection.get();
if (c == null) throw new IllegalStateException("blah blah");
return c;
}
}
Then you would use connections like this:
MyContext.enter();
try {
// connections are available here:
// anything that calls MyContext.get()
// gets (the same) valid connection instance
} finally {
MyContext.exit();
}
This block can be put wherever you want (in webapps it usually wraps the processing of each request) - from the main method if you are coding a simple case when you want a single connection available for the whole lifespan of your application, to the finest methods in your API.
You might want to take a look at fluent interfaces (with an interesting example here) and its "Builder" pattern.
You would query like this:
cars().in(DB).where(id().isEqualTo(1234));
This way you can hide the connection/disconnection code in the outermost cars() method, for example.
I've got a class that implements Iterator with a ResultSet as a data member. Essentially the class looks like this:
public class A implements Iterator{
private ResultSet entities;
...
public Object next(){
entities.next();
return new Entity(entities.getString...etc....)
}
public boolean hasNext(){
//what to do?
}
...
}
How can I check if the ResultSet has another row so I can create a valid hasNext method since ResultSet has no hasNext defined itself? I was thinking doing SELECT COUNT(*) FROM... query to get the count and managing that number to see if there's another row but I'd like to avoid this.
This is a bad idea. This approach requires that the connection is open the whole time until the last row is read, and outside the DAO layer you never know when it will happen, and you also seem to leave the resultset open and risk resource leaks and application crashes in the case the connection times out. You don't want to have that.
The normal JDBC practice is that you acquire Connection, Statement and ResultSet in the shortest possible scope. The normal practice is also that you map multiple rows into a List or maybe a Map and guess what, they do have an Iterator.
public List<Data> list() throws SQLException {
List<Data> list = new ArrayList<Data>();
try (
Connection connection = database.getConnection();
Statement statement = connection.createStatement("SELECT id, name, value FROM data");
ResultSet resultSet = statement.executeQuery();
) {
while (resultSet.next()) {
list.add(map(resultSet));
}
}
return list;
}
private Data map(ResultSet resultSet) throws SQLException {
Data data = new Data();
data.setId(resultSet.getLong("id"));
data.setName(resultSet.getString("name"));
data.setValue(resultSet.getInteger("value"));
return data;
}
And use it as below:
List<Data> list = dataDAO.list();
int count = list.size(); // Easy as that.
Iterator<Data> iterator = list.iterator(); // There is your Iterator.
Do not pass expensive DB resources outside the DAO layer like you initially wanted to do. For more basic examples of normal JDBC practices and the DAO pattern you may find this article useful.
You can get out of this pickle by performing a look-ahead in the hasNext() and remembering that you did a lookup to prevent consuming too many records, something like:
public class A implements Iterator{
private ResultSet entities;
private boolean didNext = false;
private boolean hasNext = false;
...
public Object next(){
if (!didNext) {
entities.next();
}
didNext = false;
return new Entity(entities.getString...etc....)
}
public boolean hasNext(){
if (!didNext) {
hasNext = entities.next();
didNext = true;
}
return hasNext;
}
...
}
ResultSet has an 'isLast()' method that might suit your needs. The JavaDoc says it is quite expensive though since it has to read ahead. There is a good chance it is caching the look-ahead value like the others suggest trying.
You can use ResultSetIterator, just put your ResultSet in the constructor.
ResultSet rs = ...
ResultSetIterator = new ResultSetIterator(rs);
One option is the ResultSetIterator from the Apache DBUtils project.
BalusC rightly points out the the various concerns in doing this. You need to be very careful to properly handle the connection/resultset lifecycle. Fortunately, the DBUtils project also has solutions for safely working with resultsets.
If BalusC's solution is impractical for you (e.g. you are processing large datasets that can't all fit in memory) you might want to give it a shot.
public class A implements Iterator<Entity>
{
private final ResultSet entities;
// Not required if ResultSet.isLast() is supported
private boolean hasNextChecked, hasNext;
. . .
public boolean hasNext()
{
if (hasNextChecked)
return hasNext;
hasNext = entities.next();
hasNextChecked = true;
return hasNext;
// You may also use !ResultSet.isLast()
// but support for this method is optional
}
public Entity next()
{
if (!hasNext())
throw new NoSuchElementException();
Entity entity = new Entity(entities.getString...etc....)
// Not required if ResultSet.isLast() is supported
hasNextChecked = false;
return entity;
}
}
Its not a really bad idea in the cases where you need it, it's just that you often do not need it.
If you do need to do something like, say, stream your entire database.... you could pre-fetch the next row - if the fetch fails your hasNext is false.
Here is what I used:
/**
* #author Ian Pojman <pojman#gmail.com>
*/
public abstract class LookaheadIterator<T> implements Iterator<T> {
/** The predetermined "next" object retrieved from the wrapped iterator, can be null. */
protected T next;
/**
* Implement the hasNext policy of this iterator.
* Returns true of the getNext() policy returns a new item.
*/
public boolean hasNext()
{
if (next != null)
{
return true;
}
// we havent done it already, so go find the next thing...
if (!doesHaveNext())
{
return false;
}
return getNext();
}
/** by default we can return true, since our logic does not rely on hasNext() - it prefetches the next */
protected boolean doesHaveNext() {
return true;
}
/**
* Fetch the next item
* #return false if the next item is null.
*/
protected boolean getNext()
{
next = loadNext();
return next!=null;
}
/**
* Subclasses implement the 'get next item' functionality by implementing this method. Implementations return null when they have no more.
* #return Null if there is no next.
*/
protected abstract T loadNext();
/**
* Return the next item from the wrapped iterator.
*/
public T next()
{
if (!hasNext())
{
throw new NoSuchElementException();
}
T result = next;
next = null;
return result;
}
/**
* Not implemented.
* #throws UnsupportedOperationException
*/
public void remove()
{
throw new UnsupportedOperationException();
}
}
then:
this.lookaheadIterator = new LookaheadIterator<T>() {
#Override
protected T loadNext() {
try {
if (!resultSet.next()) {
return null;
}
// process your result set - I use a Spring JDBC RowMapper
return rowMapper.mapRow(resultSet, resultSet.getRow());
} catch (SQLException e) {
throw new IllegalStateException("Error reading from database", e);
}
}
};
}
I agree with BalusC. Allowing an Iterator to escape from your DAO method is going to make it difficult to close any Connection resources. You will be forced to know about the connection lifecycle outside of your DAO, which leads to cumbersome code and potential connection leaks.
However, one choice that I've used is to pass a Function or Procedure type into the DAO method. Basically, pass in some sort of callback interface that will take each row in your result set.
For example, maybe something like this:
public class MyDao {
public void iterateResults(Procedure<ResultSet> proc, Object... params)
throws Exception {
Connection c = getConnection();
try {
Statement s = c.createStatement(query);
ResultSet rs = s.executeQuery();
while (rs.next()) {
proc.execute(rs);
}
} finally {
// close other resources too
c.close();
}
}
}
public interface Procedure<T> {
void execute(T t) throws Exception;
}
public class ResultSetOutputStreamProcedure implements Procedure<ResultSet> {
private final OutputStream outputStream;
public ResultSetOutputStreamProcedure(OutputStream outputStream) {
this.outputStream = outputStream;
}
#Override
public void execute(ResultSet rs) throws SQLException {
MyBean bean = getMyBeanFromResultSet(rs);
writeMyBeanToOutputStream(bean);
}
}
In this way, you keep your database connection resources inside your DAO, which is proper. But, you are not necessarily required to fill a Collection if memory is a concern.
Hope this helps.
You could try the following:
public class A implements Iterator {
private ResultSet entities;
private Entity nextEntity;
...
public Object next() {
Entity tempEntity;
if ( !nextEntity ) {
entities.next();
tempEntity = new Entity( entities.getString...etc....)
} else {
tempEntity = nextEntity;
}
entities.next();
nextEntity = new Entity( entities.getString...ext....)
return tempEntity;
}
public boolean hasNext() {
return nextEntity ? true : false;
}
}
This code caches the next entity, and hasNext() returns true, if the cached entity is valid, otherwise it returns false.
There are a couple of things you could do depending on what you want your class A. If the major use case is to go through every single result then perhaps its best to preload all the Entity objects and throw away the ResultSet.
If however you don't want to do that you could use the next() and previous() method of ResultSet
public boolean hasNext(){
boolean next = entities.next();
if(next) {
//reset the cursor back to its previous position
entities.previous();
}
}
You do have to be careful to make sure that you arent currently reading from the ResultSet, but, if your Entity class is a proper POJO (or at least properly disconnected from ResultSet then this should be a fine approach.
Here's my iterator that wraps a ResultSet. The rows are returned in the form a Map. I hope you'll find it helpful. The strategy is that I always bring one element in advance.
public class ResultSetIterator implements Iterator<Map<String,Object>> {
private ResultSet result;
private ResultSetMetaData meta;
private boolean hasNext;
public ResultSetIterator( ResultSet result ) throws SQLException {
this.result = result;
meta = result.getMetaData();
hasNext = result.next();
}
#Override
public boolean hasNext() {
return hasNext;
}
#Override
public Map<String, Object> next() {
if (! hasNext) {
throw new NoSuchElementException();
}
try {
Map<String,Object> next = new LinkedHashMap<>();
for (int i = 1; i <= meta.getColumnCount(); i++) {
String column = meta.getColumnName(i);
Object value = result.getObject(i);
next.put(column,value);
}
hasNext = result.next();
return next;
}
catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
}
entities.next returns false if there are no more rows, so you could just get that return value and set a member variable to keep track of the status for hasNext().
But to make that work you would also have to have some sort of init method that reads the first entity and caches it in the class. Then when calling next you would need to return the previously cached value and cache the next value, etc...
Iterators are problematic for traversing ResultSets for reasons mentioned above but Iterator like behaviour with all the required semantics for handling errors and closing resources is available with reactive sequences (Observables) in RxJava. Observables are like iterators but include the notions of subscriptions and their cancellations and error handling.
The project rxjava-jdbc has implementations of Observables for jdbc operations including traversals of ResultSets with proper closure of resources, error handling and the ability to cancel the traversal as required (unsubscribe).
Do you expect most of the data in your result set to actually be used? If so, pre-cache it. It's quite trivial using eg Spring
List<Map<String,Object>> rows = jdbcTemplate.queryForList(sql);
return rows.iterator();
Adjust to suit your taste.
I think there's enough decry over why it's a really bad idea to use ResultSet in an Iterator (in short, ResultSet maintains an active connection to DB and not closing it ASAP can lead to problems).
But in a different situation, if you're getting ResultSet (rs) and are going to iterate over the elements, but you also wanted to do something before the iteration like this:
if (rs.hasNext()) { //This method doesn't exist
//do something ONCE, *IF* there are elements in the RS
}
while (rs.next()) {
//do something repeatedly for each element
}
You can achieve the same effect by writing it like this instead:
if (rs.next()) {
//do something ONCE, *IF* there are elements in the RS
do {
//do something repeatedly for each element
} while (rs.next());
}
It can be done like this:
public boolean hasNext() {
...
return !entities.isLast();
...
}
It sounds like you are stuck between either providing an inefficient implementation of hasNext or throwing an exception stating that you do not support the operation.
Unfortunately there are times when you implement an interface and you don't need all of the members. In that case I would suggest that you throw an exception in that member that you will not or cannot support and document that member on your type as an unsupported operation.