I had a class for a db connection like that (simplified):
public class SQL {
private static SQL instance = null;
private static String ADRESS;
private SQL() {
try {
// Settings
settings = Settings.getInstance();
ADDRESS = settings.getAdress;
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql://" + ADRESS + "...");
}
catch (SQLException e) {
throw e;
}
}
public static SQL getInstance() {
if(instance == null) {
instance = new SQL();
}
return instance;
}
public String select(...) {
// SELECT Code
}
public String update(...) {
// UPDATE Code
}
public String insert(...) {
// INSERT Code
}
}
But now I need two variants of SQL and they mainly differ only in the settings (different databases). So I wanted to make SQL to an abstract class and overwrite only the constructor in the two inherited classes.
But as far as I could find out it's not possible to have an abstract private constructor!?
So how can I change the class SQL to an abstract class then?
Thanks, I hope someone understands the problen :)
I would use something like this:
public static SQL getOrMakeInstance1() {
if(instance1 == null) {
instance1 = new SQL(pram11,param12,param13);
}
return instance1;
}
public static SQL getOrMakeInstance2() {
if(instance2 == null) {
instance2 = new SQL(pram21,param22,param23);
}
return instance2;
}
-maybe is a factory pattern, I don't know the names from design patterns :)
I hope it helps.
I'm not sure you need more than 1 class. What if you changed your class's constructor to take a Settings instance?
public SQL(Settings settings){
ADDRESS = settings.getAddress();
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql://" + ADRESS + "...");
}
Then what you make different Setting instances
Settings settings = ...
SQL sql = new SQL(settings);
Settings differentSettings =...
SQL differentSql = new SQL(differentSettings);
The problem that I'm running into is that after I make an AutoIndex index a certain property, I can add a key/value pair and the index won't show that it's there. I'm relatively new to neo4j so my concept of what this class is supposed to do might be wrong. The test code creates an impermanent graph database, instantiates my data service class with it, and then creates a user. When the data service class is instantiated, that's when the properties are added to the AutoIndex. You can see inside the createUser() function, I've printed out the user that I've just created and should be in the AutoIndex, but it prints null.
This is the code I'm testing:
#Before
public void setup() throws IOException {
graphDb = new ImpermanentGraphDatabase();
Transaction tx = graphDb.beginTx();
try {
nA = graphDb.createNode();
nB = graphDb.createNode();
packetA = graphDb.createNode();
packetB = graphDb.createNode();
dataService = new DataServiceImpl(graphDb);
tx.success();
}
finally
{
tx.finish();
}
}
/****************** Test User Creation Functionality *******************/
#Test
public void createUser() throws ExistsException {
Transaction tx = graphDb.beginTx();
try {
UserWrapper user = (UserWrapper) dataService.createUser(BigInteger.valueOf(1));
tx.success();
}
finally {
tx.finish();
}
}
Here's the code in DataServiceImpl:
/**
* Node property keys that should be auto-indexed.
*/
private static final Set<String> NODE_KEYS_INDEXABLE = new HashSet<String>(
Arrays.asList(new String[] { UserWrapper.KEY_IDENTIFIER }));
/**
* Relationship property keys that should be auto-index.
*/
private static final Set<String> REL_KEYS_INDEXABLE = new HashSet<String>(
Arrays.asList(new String[] { SentWrapper.TIME_PROP }));
private void initIndices() {
/* Get the auto-indexers */
AutoIndexer<Node> nodeAutoIndexer = this.graphDb.index()
.getNodeAutoIndexer();
RelationshipAutoIndexer relAutoIndexer = this.graphDb.index()
.getRelationshipAutoIndexer();
this.updateIndexProperties(nodeAutoIndexer,
DataServiceImpl.NODE_KEYS_INDEXABLE);
this.nodeIndex = nodeAutoIndexer.getAutoIndex();
this.updateIndexProperties(relAutoIndexer,
DataServiceImpl.REL_KEYS_INDEXABLE);
this.relIndex = relAutoIndexer.getAutoIndex();
}
/**
* Sets the indexed properties of an {#link AutoIndexer} to the specified
* set, removing old properties and adding new ones.
*
* #param autoIndexer
* the AutoIndexer to update.
* #param properties
* the properties to be indexed.
* #return autoIndexer, this given AutoIndexer (useful for chaining calls.)
*/
private <T extends PropertyContainer> AutoIndexer<T> updateIndexProperties(
AutoIndexer<T> autoIndexer, Set<String> properties) {
Set<String> indexedProps = autoIndexer.getAutoIndexedProperties();
// Remove unneeded properties.
for (String prop : difference(indexedProps, properties)) {
autoIndexer.stopAutoIndexingProperty(prop);
}
// Add new properties.
for (String prop : difference(properties, indexedProps)) {
autoIndexer.startAutoIndexingProperty(prop);
}
// Enable the index, if needed.
if (!autoIndexer.isEnabled()) {
autoIndexer.setEnabled(true);
}
return autoIndexer;
}
public User createUser(BigInteger identifier) throws ExistsException {
// Verify that user doesn't already exist.
if (this.nodeIndex.get(UserWrapper.KEY_IDENTIFIER, identifier).getSingle() != null) {
throw new ExistsException("User with identifier '" + identifier.toString() + "' already exists.");
}
// Create new user.
final Node userNode = graphDb.createNode();
final User user = new UserWrapper(userNode);
user.setIdentifier(identifier);
Node userNode2 = this.nodeIndex.get(UserWrapper.KEY_IDENTIFIER, identifier).getSingle();
System.out.println(userNode2);
userParent.getNode().createRelationshipTo(userNode, NodeRelationships.PARENT);
return user;
}
Here's the code in UserWrapper:
/**
* Mapping to neo4j key for the identifier property.
*/
//Changed to public in order to test in Test class
static final String KEY_IDENTIFIER = "identifier";
#Override
public void setIdentifier(BigInteger newIdentity) {
neo4jNode.setProperty(KEY_IDENTIFIER, newIdentity.toByteArray());
}
Where do you add the second user? By running the test twice? Then ImpermanentGraphDatabase will have deleted all data (as it is intended for testing) before the second run. The indexing happens at commit time, and aggregates all changes during the transaction, that's why you don't see it within the tx (userNode2). If you'd like to you can add this check to the test (see below).
From your code it is also not visible where you call the initIndices, could you please indicate the place? Please also verify that the auto-indexer is indexing the correct properties.
Try to change your test to, then the second call should throw the exception:
#Test(expected = ExistsException.class)
public void createUser() throws ExistsException {
Transaction tx = graphDb.beginTx();
try {
UserWrapper user = (UserWrapper) dataService.createUser(BigInteger.valueOf(1));
tx.success();
}
finally {
tx.finish();
}
Node userNode2 = this.graphDb.index().getNodeAutoIndexer().getAutoIndex().get(UserWrapper.KEY_IDENTIFIER, identifier).getSingle();
assertNotNull(userNode2);
Transaction tx = graphDb.beginTx();
try {
UserWrapper user = (UserWrapper) dataService.createUser(BigInteger.valueOf(1));
tx.success();
}
finally {
tx.finish();
}
}
I found the answer to this question indirectly from the Michael's answer. I realized that using assertNotNull(userNode2); failed if it was directly after dataService.createUser() but passed if it was after the try finally block. Thus, if I wanted to use the node I created, i would need to create another try finally block after the one that creates the node to use this. I believe this is because the user isn't added to the index until tx.success() is called, although that is just a guess.
NOTE: Please ignore my use of MultivaluedMap instead of multiple vargs String...args.
Is there a standard way in java of doing this?
What I have is a resource, that is returned from a remote server. But before each query, the remote connection must be open, and after the returns are returned - it must be closed.
So a natural way of doing this is something like:
Connection c = config.configureConnection();
c.open(); //open
List<Car> cars;
try{
cars = c.getCars();
}finally{
c.close(); //close
}
Now I want to implement something that operates on the level of the resources themselves, without worrying about connection, for example:
List<Car> cars = new CarResource().all(); //opens and closes connection
The way I am currently doing it is by having one abstract class, AbstractQueriable call abstract methods query(String ...args) and query(int id), which any class extending it must implement.
The AbstractQuerieable implements the Queriable interface, which makes it expose the three public methods filter(String ...args), all() and get(int id) - which are the public facing methods.
Here is the Queriable interface:
public interface Queriable <T>{
public T get(String id);
/** Simply returns all resources */
public Collection<T> all();
public Collection<T> filter(MultivaluedMap<String, String> args);
}
here is the AbstractQueriable class that implements it:
public abstract class AbstractQueriable<T> implements Queriable<T> {
#Override
public final T get(String id) {
setup();
try {
return query(id);
} finally {
cleanup();
}
}
#Override
public final Collection<T> filter(MultivaluedMap<String, String> args) {
setup();
try {
return query(args);
} finally {
cleanup();
}
}
/**
* Returns all resources.
*
* This is a convenience method that is equivalent to passing an empty
* arguments list to the filter function.
*
* #return The collection of all resources if possible
*/
#Override
public final Collection<T> all() {
return filter(null);
}
/**
* Queries for a resource by id.
*
* #param id
* id of the resource to return
* #return
*/
protected abstract T query(String id);
/**
* Queries for a resource by given arguments.
*
* #param args
* Map of arguments, where each key is the argument name, and the
* corresponing values are the values
* #return The collection of resources found
*/
protected abstract Collection<T> query(MultivaluedMap<String, String> args);
private void cleanup() {
Repository.close();
}
private void setup() {
Repository.open();
}
and finally my resource, which I want to use in the code, must extend the AbstractQueriable class, for example (please note that the details of these methods are not important):
public class CarRepositoryResource extends AbstractQueriable<Car> {
#Override
protected Car query(String id) {
MultivaluedMap<String, String> params = new MultivaluedMapImpl();
params.add("CarID", id);
// Delegate the query to the parametarized version
Collection<cars> cars = query(params);
if (cars == null || cars.size() == 0) {
throw new WebApplicationException(Response.Status.NOT_FOUND);
}
if (cars.size() > 1) {
throw new WebApplicationException(Response.Status.NOT_FOUND);
}
return cars.iterator().next();
}
#Override
protected Collection<Car> query(MultivaluedMap<String, String> params) {
Collection<Car> cars = new ArrayList<Car>();
Response response = Repository.getConnection().doQuery("Car");
while (response.next()) {
Returned returned = response.getResult();
if (returned != null) {
cars.add(returned);
}
}
return cars;
}
}
which finally, I can use in my code:
Collection<Car> cars = new CarRepositoryResource().all();
//... display cars to the client etc...
There are a few things I don't like about this kind of setup:
I must instantiate a new instance of my "CarRepositoryResource" every time I do a query.
The method names "query", while internal and private, are still confusing and clunky.
I am not sure if there is a better pattern or framework out there.
The connection that I am using does not support/implement the JDBC api and is not sql-based.
You could use a variation of the (in)famous Open session in view pattern.
Basically it comes down to this:
Define a "context" in which connections are available
(usually the request in web applications)
Handle (possibly lazy) initialization and release of a connection when entering/exiting the context
Code your methods taking for granted they will only be used inside such a context
It is not difficult to implement (storing the connection in a static ThreadLocal to make it thread safe) and will definitely spare a few open/close calls (performance-wise that could be a big gain, depending on how heavy your connection is).
The context class could look something like (consider this pseudo-code);
public class MyContext{
private static final
ThreadLocal<Connection> connection = new ThreadLocal<Connection>();
public static void enter() {
connection.set(initializeConnection());
// this is eager initialization
// if you think it will often the case that no connection is actually
// required inside a context, you can defer the actual initialization
// until the first call to get()
}
public static void exit() {
try { connection.close(); }
catch(Throwable t) { /* panic! */ }
finally { connection.set(null); }
}
public static Connection get() {
Connection c = connection.get();
if (c == null) throw new IllegalStateException("blah blah");
return c;
}
}
Then you would use connections like this:
MyContext.enter();
try {
// connections are available here:
// anything that calls MyContext.get()
// gets (the same) valid connection instance
} finally {
MyContext.exit();
}
This block can be put wherever you want (in webapps it usually wraps the processing of each request) - from the main method if you are coding a simple case when you want a single connection available for the whole lifespan of your application, to the finest methods in your API.
You might want to take a look at fluent interfaces (with an interesting example here) and its "Builder" pattern.
You would query like this:
cars().in(DB).where(id().isEqualTo(1234));
This way you can hide the connection/disconnection code in the outermost cars() method, for example.
I've got a class that implements Iterator with a ResultSet as a data member. Essentially the class looks like this:
public class A implements Iterator{
private ResultSet entities;
...
public Object next(){
entities.next();
return new Entity(entities.getString...etc....)
}
public boolean hasNext(){
//what to do?
}
...
}
How can I check if the ResultSet has another row so I can create a valid hasNext method since ResultSet has no hasNext defined itself? I was thinking doing SELECT COUNT(*) FROM... query to get the count and managing that number to see if there's another row but I'd like to avoid this.
This is a bad idea. This approach requires that the connection is open the whole time until the last row is read, and outside the DAO layer you never know when it will happen, and you also seem to leave the resultset open and risk resource leaks and application crashes in the case the connection times out. You don't want to have that.
The normal JDBC practice is that you acquire Connection, Statement and ResultSet in the shortest possible scope. The normal practice is also that you map multiple rows into a List or maybe a Map and guess what, they do have an Iterator.
public List<Data> list() throws SQLException {
List<Data> list = new ArrayList<Data>();
try (
Connection connection = database.getConnection();
Statement statement = connection.createStatement("SELECT id, name, value FROM data");
ResultSet resultSet = statement.executeQuery();
) {
while (resultSet.next()) {
list.add(map(resultSet));
}
}
return list;
}
private Data map(ResultSet resultSet) throws SQLException {
Data data = new Data();
data.setId(resultSet.getLong("id"));
data.setName(resultSet.getString("name"));
data.setValue(resultSet.getInteger("value"));
return data;
}
And use it as below:
List<Data> list = dataDAO.list();
int count = list.size(); // Easy as that.
Iterator<Data> iterator = list.iterator(); // There is your Iterator.
Do not pass expensive DB resources outside the DAO layer like you initially wanted to do. For more basic examples of normal JDBC practices and the DAO pattern you may find this article useful.
You can get out of this pickle by performing a look-ahead in the hasNext() and remembering that you did a lookup to prevent consuming too many records, something like:
public class A implements Iterator{
private ResultSet entities;
private boolean didNext = false;
private boolean hasNext = false;
...
public Object next(){
if (!didNext) {
entities.next();
}
didNext = false;
return new Entity(entities.getString...etc....)
}
public boolean hasNext(){
if (!didNext) {
hasNext = entities.next();
didNext = true;
}
return hasNext;
}
...
}
ResultSet has an 'isLast()' method that might suit your needs. The JavaDoc says it is quite expensive though since it has to read ahead. There is a good chance it is caching the look-ahead value like the others suggest trying.
You can use ResultSetIterator, just put your ResultSet in the constructor.
ResultSet rs = ...
ResultSetIterator = new ResultSetIterator(rs);
One option is the ResultSetIterator from the Apache DBUtils project.
BalusC rightly points out the the various concerns in doing this. You need to be very careful to properly handle the connection/resultset lifecycle. Fortunately, the DBUtils project also has solutions for safely working with resultsets.
If BalusC's solution is impractical for you (e.g. you are processing large datasets that can't all fit in memory) you might want to give it a shot.
public class A implements Iterator<Entity>
{
private final ResultSet entities;
// Not required if ResultSet.isLast() is supported
private boolean hasNextChecked, hasNext;
. . .
public boolean hasNext()
{
if (hasNextChecked)
return hasNext;
hasNext = entities.next();
hasNextChecked = true;
return hasNext;
// You may also use !ResultSet.isLast()
// but support for this method is optional
}
public Entity next()
{
if (!hasNext())
throw new NoSuchElementException();
Entity entity = new Entity(entities.getString...etc....)
// Not required if ResultSet.isLast() is supported
hasNextChecked = false;
return entity;
}
}
Its not a really bad idea in the cases where you need it, it's just that you often do not need it.
If you do need to do something like, say, stream your entire database.... you could pre-fetch the next row - if the fetch fails your hasNext is false.
Here is what I used:
/**
* #author Ian Pojman <pojman#gmail.com>
*/
public abstract class LookaheadIterator<T> implements Iterator<T> {
/** The predetermined "next" object retrieved from the wrapped iterator, can be null. */
protected T next;
/**
* Implement the hasNext policy of this iterator.
* Returns true of the getNext() policy returns a new item.
*/
public boolean hasNext()
{
if (next != null)
{
return true;
}
// we havent done it already, so go find the next thing...
if (!doesHaveNext())
{
return false;
}
return getNext();
}
/** by default we can return true, since our logic does not rely on hasNext() - it prefetches the next */
protected boolean doesHaveNext() {
return true;
}
/**
* Fetch the next item
* #return false if the next item is null.
*/
protected boolean getNext()
{
next = loadNext();
return next!=null;
}
/**
* Subclasses implement the 'get next item' functionality by implementing this method. Implementations return null when they have no more.
* #return Null if there is no next.
*/
protected abstract T loadNext();
/**
* Return the next item from the wrapped iterator.
*/
public T next()
{
if (!hasNext())
{
throw new NoSuchElementException();
}
T result = next;
next = null;
return result;
}
/**
* Not implemented.
* #throws UnsupportedOperationException
*/
public void remove()
{
throw new UnsupportedOperationException();
}
}
then:
this.lookaheadIterator = new LookaheadIterator<T>() {
#Override
protected T loadNext() {
try {
if (!resultSet.next()) {
return null;
}
// process your result set - I use a Spring JDBC RowMapper
return rowMapper.mapRow(resultSet, resultSet.getRow());
} catch (SQLException e) {
throw new IllegalStateException("Error reading from database", e);
}
}
};
}
I agree with BalusC. Allowing an Iterator to escape from your DAO method is going to make it difficult to close any Connection resources. You will be forced to know about the connection lifecycle outside of your DAO, which leads to cumbersome code and potential connection leaks.
However, one choice that I've used is to pass a Function or Procedure type into the DAO method. Basically, pass in some sort of callback interface that will take each row in your result set.
For example, maybe something like this:
public class MyDao {
public void iterateResults(Procedure<ResultSet> proc, Object... params)
throws Exception {
Connection c = getConnection();
try {
Statement s = c.createStatement(query);
ResultSet rs = s.executeQuery();
while (rs.next()) {
proc.execute(rs);
}
} finally {
// close other resources too
c.close();
}
}
}
public interface Procedure<T> {
void execute(T t) throws Exception;
}
public class ResultSetOutputStreamProcedure implements Procedure<ResultSet> {
private final OutputStream outputStream;
public ResultSetOutputStreamProcedure(OutputStream outputStream) {
this.outputStream = outputStream;
}
#Override
public void execute(ResultSet rs) throws SQLException {
MyBean bean = getMyBeanFromResultSet(rs);
writeMyBeanToOutputStream(bean);
}
}
In this way, you keep your database connection resources inside your DAO, which is proper. But, you are not necessarily required to fill a Collection if memory is a concern.
Hope this helps.
You could try the following:
public class A implements Iterator {
private ResultSet entities;
private Entity nextEntity;
...
public Object next() {
Entity tempEntity;
if ( !nextEntity ) {
entities.next();
tempEntity = new Entity( entities.getString...etc....)
} else {
tempEntity = nextEntity;
}
entities.next();
nextEntity = new Entity( entities.getString...ext....)
return tempEntity;
}
public boolean hasNext() {
return nextEntity ? true : false;
}
}
This code caches the next entity, and hasNext() returns true, if the cached entity is valid, otherwise it returns false.
There are a couple of things you could do depending on what you want your class A. If the major use case is to go through every single result then perhaps its best to preload all the Entity objects and throw away the ResultSet.
If however you don't want to do that you could use the next() and previous() method of ResultSet
public boolean hasNext(){
boolean next = entities.next();
if(next) {
//reset the cursor back to its previous position
entities.previous();
}
}
You do have to be careful to make sure that you arent currently reading from the ResultSet, but, if your Entity class is a proper POJO (or at least properly disconnected from ResultSet then this should be a fine approach.
Here's my iterator that wraps a ResultSet. The rows are returned in the form a Map. I hope you'll find it helpful. The strategy is that I always bring one element in advance.
public class ResultSetIterator implements Iterator<Map<String,Object>> {
private ResultSet result;
private ResultSetMetaData meta;
private boolean hasNext;
public ResultSetIterator( ResultSet result ) throws SQLException {
this.result = result;
meta = result.getMetaData();
hasNext = result.next();
}
#Override
public boolean hasNext() {
return hasNext;
}
#Override
public Map<String, Object> next() {
if (! hasNext) {
throw new NoSuchElementException();
}
try {
Map<String,Object> next = new LinkedHashMap<>();
for (int i = 1; i <= meta.getColumnCount(); i++) {
String column = meta.getColumnName(i);
Object value = result.getObject(i);
next.put(column,value);
}
hasNext = result.next();
return next;
}
catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
}
entities.next returns false if there are no more rows, so you could just get that return value and set a member variable to keep track of the status for hasNext().
But to make that work you would also have to have some sort of init method that reads the first entity and caches it in the class. Then when calling next you would need to return the previously cached value and cache the next value, etc...
Iterators are problematic for traversing ResultSets for reasons mentioned above but Iterator like behaviour with all the required semantics for handling errors and closing resources is available with reactive sequences (Observables) in RxJava. Observables are like iterators but include the notions of subscriptions and their cancellations and error handling.
The project rxjava-jdbc has implementations of Observables for jdbc operations including traversals of ResultSets with proper closure of resources, error handling and the ability to cancel the traversal as required (unsubscribe).
Do you expect most of the data in your result set to actually be used? If so, pre-cache it. It's quite trivial using eg Spring
List<Map<String,Object>> rows = jdbcTemplate.queryForList(sql);
return rows.iterator();
Adjust to suit your taste.
I think there's enough decry over why it's a really bad idea to use ResultSet in an Iterator (in short, ResultSet maintains an active connection to DB and not closing it ASAP can lead to problems).
But in a different situation, if you're getting ResultSet (rs) and are going to iterate over the elements, but you also wanted to do something before the iteration like this:
if (rs.hasNext()) { //This method doesn't exist
//do something ONCE, *IF* there are elements in the RS
}
while (rs.next()) {
//do something repeatedly for each element
}
You can achieve the same effect by writing it like this instead:
if (rs.next()) {
//do something ONCE, *IF* there are elements in the RS
do {
//do something repeatedly for each element
} while (rs.next());
}
It can be done like this:
public boolean hasNext() {
...
return !entities.isLast();
...
}
It sounds like you are stuck between either providing an inefficient implementation of hasNext or throwing an exception stating that you do not support the operation.
Unfortunately there are times when you implement an interface and you don't need all of the members. In that case I would suggest that you throw an exception in that member that you will not or cannot support and document that member on your type as an unsupported operation.
I need to fetch many records from an RDBMS in Java (10-20k) my target system expects them to be available as Java List. So I want to implement my code as "Virtual list" where I actually only fetch the records I actually need. I expect SQL like
SELECT * FROM CUSTOMER WHERE COUNTRY="Moldovia"
as parameter and just return what is requested. Most likely the data is requested in batches of 50. Any hints how to do that?
Unless you expect your clients to randomly access the data, you're probably better off returning an Iterator. Also, take a look at ResultSet.setFetchSize: http://java.sun.com/javase/6/docs/api/java/sql/ResultSet.html#setFetchSize(int)
So something like:
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Iterator;
public class FooResultSetIterator implements Iterator<Foo>
{
private final ResultSet resultSet;
private boolean hasNext;
FooResultSetIterator(final ResultSet resultSet, final int fetchSize) throws SQLException
{
this.resultSet = resultSet;
this.resultSet.setFetchSize(fetchSize);
this.hasNext = resultSet.next();
}
#Override
public boolean hasNext()
{
return hasNext;
}
#Override
public Foo next()
{
final Foo foo = new Foo(resultSet);
try
{
this.hasNext = resultSet.next();
}
catch (final SQLException e)
{
throw new RuntimeException(e);
}
return foo;
}
#Override
public void remove()
{
throw new UnsupportedOperationException("Cannot remove items from a ResultSetIterator");
}
}
class Foo
{
public Foo(ResultSet resultSet)
{
// TODO Auto-generated constructor stub
}
}
Use OFFSET and LIMIT in your query:
SELECT * FROM CUSTOMER WHERE
COUNTRY="Moldovia" LIMIT 50 OFFSET 50
Assuming of course that your SQL dialect allows it. The example will return rows 51-100.