We Keep Coding

SQL, JDBC "Invalid column index" error with simple query

I'm new to STRUTS and JDBC, my application tries to connect to a simple DB that has 3 tables, right now all is doing is trying to query 1 table that only stores "first, last names and a Id field"
System.out.println("-------- Oracle JDBC Connection Testing ------");
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
} catch (ClassNotFoundException e) {
System.out.println("Where is your Oracle JDBC Driver?");
e.printStackTrace();
return null;
}
System.out.println("Oracle JDBC Driver Registered!");
try {
connection =
DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:xe","david","changeit");
} catch (SQLException e) {
System.out.println("Connection Failed! Check output console");
e.printStackTrace();
return null;
}
if (connection != null) {
System.out.println("You made it, take control your database now!");
} else {
System.out.println("Failed to make connection!");
}
where I would like to get the result of 1 column if a match occurs:
String sql = "SELECT S_ID FROM Students WHERE firstname=? AND lastname=?";
PreparedStatement ps = connection.prepareStatement(sql);
ps.setString(1, firstname);
ps.setString(2, lastname);
rs = ps.executeQuery();
while (rs.next()) {
studentid = rs.getString(1);
ret = SUCCESS;
}
} catch (Exception e) { ...
As far as I can tell the connection is made,
the SQL query
Select s_id from Students where firstname='first' and lastname='last';
when run on SQL Dev. works and gives me a single result.
I don't really get a stack trace the code just jumps from right before the 'while (rs.next()) {..' directly into the finally block
} catch (Exception e) {
e.printStackTrace();
ret = ERROR;
} finally {
if (connection != null) {
try {
connection.close();
} catch (Exception e) {
}
}
}

I'm not sure how Oracle drivers work. But below statement is what i see on Oracle site. Are you getting a non empty resultset ?
As you are not getting a nullpointerexception on .next(), i'm wondering if Oracle drivers return an empty ResultSet, which may lead to this problem.
http://docs.oracle.com/cd/B28359_01/java.111/b31224/getsta.htm
In case of a standard JDBC driver, if the SQL string being executed
does not return a ResultSet object, then the executeQuery method
throws a SQLException exception. In case of an Oracle JDBC driver, the
executeQuery method does not throw a SQLException exception even if
the SQL string being executed does not return a ResultSet object.

Like I said I'm new at using this.
The problem was that my schema didn't have the CONNECT role assigned to it.
Solution log in as 'SYSTEM' and grant the role to my schema
grant connect to MY_SCHEMA;

Returning a ResultSet

I am trying to create a method from where I can query my database and retrieve a whole table.
Currently, it works just fine if I use the data inside the method. However, I want the method to return the results.
I'm getting a java.sql.SQLException: Operation not allowed after ResultSet closed on the current code.
How can I achieve this?
public ResultSet select() {
con = null;
st = null;
rs = null;
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM biler");
/*
if (rs.next()) {
System.out.println(rs.getString("model"));
}*/
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
return rs;
}

You should never pass a ResultSet around through public methods. This is prone to resource leaking because you're forced to keep the statement and the connection open. Closing them would implicitly close the result set. But keeping them open would cause them to dangle around and cause the DB to run out of resources when there are too many of them open.
Map it to a collection of Javabeans like so and return it instead:
public List<Biler> list() throws SQLException {
Connection connection = null;
PreparedStatement statement = null;
ResultSet resultSet = null;
List<Biler> bilers = new ArrayList<Biler>();
try {
connection = database.getConnection();
statement = connection.prepareStatement("SELECT id, name, value FROM Biler");
resultSet = statement.executeQuery();
while (resultSet.next()) {
Biler biler = new Biler();
biler.setId(resultSet.getLong("id"));
biler.setName(resultSet.getString("name"));
biler.setValue(resultSet.getInt("value"));
bilers.add(biler);
}
} finally {
if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
if (statement != null) try { statement.close(); } catch (SQLException ignore) {}
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return bilers;
}
Or, if you're on Java 7 already, just make use of try-with-resources statement which will auto-close those resources:
public List<Biler> list() throws SQLException {
List<Biler> bilers = new ArrayList<Biler>();
try (
Connection connection = database.getConnection();
PreparedStatement statement = connection.prepareStatement("SELECT id, name, value FROM Biler");
ResultSet resultSet = statement.executeQuery();
) {
while (resultSet.next()) {
Biler biler = new Biler();
biler.setId(resultSet.getLong("id"));
biler.setName(resultSet.getString("name"));
biler.setValue(resultSet.getInt("value"));
bilers.add(biler);
}
}
return bilers;
}
By the way, you should not be declaring the Connection, Statement and ResultSet as instance variables at all (major threadsafety problem!), nor be swallowing the SQLException at that point at all (the caller will have no clue that a problem occurred), nor be closing the resources in the same try (if e.g. result set close throws an exception, then statement and connection are still open). All those issues are fixed in the above code snippets.

If you don't know what you want of the ResultSet on retrieving time I suggest mapping the complete thing into a map like this:
List<Map<String, Object>> resultList = new ArrayList<Map<String, Object>>();
Map<String, Object> row = null;
ResultSetMetaData metaData = rs.getMetaData();
Integer columnCount = metaData.getColumnCount();
while (rs.next()) {
row = new HashMap<String, Object>();
for (int i = 1; i <= columnCount; i++) {
row.put(metaData.getColumnName(i), rs.getObject(i));
}
resultList.add(row);
}
So basically you have the same thing as the ResultSet then (without the ResultSetMetaData).

Well, you do call rs.close() in your finally-block.
That's basically a good idea, as you should close all your resources (connections, statements, result sets, ...).
But you must close them after you use them.
There are at least three possible solutions:
don't close the resultset (and connection, ...) and require the caller to call a separate "close" method.
This basically means that now the caller needs to remember to call close and doesn't really make things easier.
let the caller pass in a class that gets passed the resultset and call that within your method
This works, but can become slightly verbose, as you'll need a subclass of some interface (possibly as an anonymous inner class) for each block of code you want to execute on the resultset.
The interface looked like this:
public interface ResultSetConsumer<T> {
public T consume(ResultSet rs);
}
and your select method looked like this:
public <T> List<T> select(String query, ResultSetConsumer<T> consumer) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery(query);
List<T> result = new ArrayList<T>();
while (rs.next()) {
result.add(consumer.consume(rs));
}
} catch (SQLException ex) {
// logging
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
return rs;
}
do all the work inside the select method and return some List as a result.
This is probably the most widely used one: iterate over the resultset and convert the data into custom data in your own DTOs and return those.

As everyone before me said its a bad idea to pass the result set. If you are using Connection pool library like c3p0 then you can safely user CachedRowSet and its implementation CachedRowSetImpl. Using this you can close the connection. It will only use connection when required. Here is snippet from the java doc:
A CachedRowSet object is a disconnected rowset, which means that it makes use of a connection to its data source only briefly. It connects to its data source while it is reading data to populate itself with rows and again while it is propagating changes back to its underlying data source. The rest of the time, a CachedRowSet object is disconnected, including while its data is being modified. Being disconnected makes a RowSet object much leaner and therefore much easier to pass to another component. For example, a disconnected RowSet object can be serialized and passed over the wire to a thin client such as a personal digital assistant (PDA).
Here is the code snippet for querying and returning ResultSet:
public ResultSet getContent(String queryStr) {
Connection conn = null;
Statement stmt = null;
ResultSet resultSet = null;
CachedRowSetImpl crs = null;
try {
Connection conn = dataSource.getConnection();
stmt = conn.createStatement();
resultSet = stmt.executeQuery(queryStr);
crs = new CachedRowSetImpl();
crs.populate(resultSet);
} catch (SQLException e) {
throw new IllegalStateException("Unable to execute query: " + queryStr, e);
}finally {
try {
if (resultSet != null) {
resultSet.close();
}
if (stmt != null) {
stmt.close();
}
if (conn != null) {
conn.close();
}
} catch (SQLException e) {
LOGGER.error("Ignored", e);
}
}
return crs;
}
Here is the snippet for creating data source using c3p0:
ComboPooledDataSource cpds = new ComboPooledDataSource();
try {
cpds.setDriverClass("<driver class>"); //loads the jdbc driver
} catch (PropertyVetoException e) {
e.printStackTrace();
return;
}
cpds.setJdbcUrl("jdbc:<url>");
cpds.setMinPoolSize(5);
cpds.setAcquireIncrement(5);
cpds.setMaxPoolSize(20);
javax.sql.DataSource dataSource = cpds;

You can use the CachedRowSet object that is just for what you want:
public CachedRowSetImpl select(String url, String user, String password) {
CachedRowSetImpl crs = null;
try (Connection con = DriverManager.getConnection(url, user, password);
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("SELECT * FROM biler");) {
crs = new CachedRowSetImpl();
crs.populate(rs);
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
return crs;
}
You can read the documentation here:
https://docs.oracle.com/javase/7/docs/api/javax/sql/rowset/CachedRowSet.html

You're closing the ResultSet and consequently you can't use it anymore.
In order to return the contents of the table, you'll have to iterate through the ResultSet and build a per-row representation (in a List, perhaps?). Presumably each row represents some entity, and I would create such an entity for each row.
while (rs.next()) {
list.add(new Entity(rs));
}
return list;
The alternative is to provide some callback object, and your ResultSet iteration would call on that object for each ResultSet row. That way you don't need to build an object representing the whole table (which may be a problem if it's sizable)
while (rs.next()) {
client.processResultSet(rs);
}
I would be reluctant to let clients close the result set/statement/connection. These need to be managed carefully to avoid resource leaks, and you're much better off handling this in one place (preferably close to where you open them!).
Note: You can use Apache Commons DbUtils.closeQuietly() to simply and reliably close the connect/statement/resultset tuple (handling nulls and exceptions properly)

It is bad practice to return result set ,secondly you are already closing it so after closing it you can not use it anymore.
I would suggest using Java 7 with multiple resource in try block will helpful you as suggested above.
If you want entire table result ,you should return its output rather than resultSet.

Assuming you can afford storing the entire result in memory, you may simply return some table-like structure. Using Tablesaw for instance, simply do
Table t = Table.read().db(rows);
with rows a standard java.sql.ResultSet. For details see here. Tablesaw becomes especially useful if you intend to slice-and-dice your data further as it gives you Pandas-like functionality.

Handling all exceptions when executing SQL in Java

There are many steps involved in executing one SQL statement in Java:
Create connection
Create statement
Execute statement, create resultset
Close resultset
Close statement
Close connection
At each of these steps SQLException can be thrown. If we to handle all exception and release all the resources correctly, the code will will look like this with 4 levels of TRY stacked on the top of each other.
try {
Connection connection = dataSource.getConnection();
try {
PreparedStatement statement = connection.prepareStatement("SELECT 1 FROM myTable");
try {
ResultSet result = statement.executeQuery();
try {
if (result.next()) {
Integer theOne = result.getInt(1);
}
}
finally {
result.close();
}
}
finally {
statement.close();
}
}
finally {
connection.close();
}
}
catch (SQLException e) {
// Handle exception
}
Can you propose a better (shorter) way to execute a statement while still release all the consumed resources?

If you are using Java 7, the try with resources statement will shorten this quite a bit, and make it more maintainable:
try (Connection conn = ds.getConnection(); PreparedStatement ps = conn.prepareStatement(queryString); ResultSet rs = ps.execute()) {
} catch (SQLException e) {
//Log the error somehow
}
Note that closing the connection closes all associated Statements and ResultSets.

Check out Apache Commons DbUtils, and in particular the closeQuietly() method. It will handle the connection/statement/result set closing correctly, including the cases where one or more are null.
An alternative is Spring JdbcTemplate, which abstracts a lot of work away from you, and you handle your database queries in a much more functional fashion. You simply provide a class as a callback to be called on for every row of a ResultSet. It'll handle iteration, exception handling and the correct closing of resources.

I create a utility class with static methods I can call:
package persistence;
// add imports.
public final class DatabaseUtils {
// similar for the others Connection and Statement
public static void close(ResultSet rs) {
try {
if (rs != null) {
rs.close();
}
} catch (Exception e) {
LOGGER.error("Failed to close ResultSet", e);
}
}
}
So your code would be:
Integer theOne = null;
Connection connection = null;
PreparedStatement statment = null;
ResultSet result = null;
try {
connection = dataSource.getConnection();
statement = connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
while (result.next()) {
theOne = result.getInt(1);
}
} catch (SQLException e) {
// do something
} finally {
DatabaseUtils.close(result);
DatabaseUtils.close(statement);
DatabaseUtils.close(connection);
}
return theOne;
I'd recommend instantiating the Connection outside this method and passing it in. You can handle transactions better that way.

Connection connection = null;
PreparedStatement statement = null;
ResultSet result = null;
try {
connection = dataSource.getConnection();
statement = connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
if (result.next()) {
Integer theOne = result.getInt(1);
}
}
catch (SQLException e) { /* log error */ }
finally {
if (result != null) try { result.close(); } catch (Exception e) {/*log error or ignore*/}
if (statement != null) try { statement.close(); } catch (Exception e) {/*log error or ignore*/}
if (connection != null) try { connection.close(); } catch (Exception e) {/*log error or ignore*/}
}

Just close the Connection, this releases all resources*. You don't need to close Statement and ResultSet.
*just make sure you don't have any active transactions.

Your code can be shortened and written in this way...
Connection connection = dataSource.getConnection();
PreparedStatement statement = null;
ResultSet result = null;
try {
statement= connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
if (result.next()) {
Integer theOne = result.getInt(1);
}
} catch (SQLException e) {
// Handle exception
} finally {
if(result != null) result.close();
if(statement != null) statement.close();
if(connection != null) connection.close();
}

Class not found loading JDBC org.postgresql.Driver

I'm working on a web project and I recently installed postgres 9.1.1
The postgresql server is up and running. I can connect via psql as usual and everything is loaded and properly saved from a dump of the db I made from 8.5.
So I also downloaded the JDBC4 driver for 9.1 postgres version here:
http://jdbc.postgresql.org/download/postgresql-jdbc-9.1-901.src.tar.gz
I added it to the java build path using the project properties via eclipse.
This is the code I use to provide db connection to other classes (i.e. it's a singleton, I get a new connection only if the existing is either closed or null, from one object at a time only)
public abstract class DBConnection {
private static Connection connection = null;
public static void connect() {
try {
if (connection == null) {
String host = "127.0.0.1";
String database = "xxxxx";
String username = "xxxxx";
String password = "xxxxx";
String url = "jdbc:postgresql://" + host + "/" + database;
String driverJDBC = "org.postgresql.Driver";
Class.forName(driverJDBC);
connection = DriverManager.getConnection(url, username,
password); //line firing the class not found exception
} else if (connection.isClosed()) {
connection = null;
connect();
}
} catch (SQLException e) {
e.printStackTrace(System.err);
} catch (Exception e) {
e.printStackTrace(System.err);
}
}
public static void disconnect() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
Logger.getLogger(DBConnection.class.getName()).log(
Level.SEVERE, null, e);
}
}
}
public static Connection getConnection() {
try {
if (connection != null && !connection.isClosed()) {
return connection;
} else {
connect();
return connection;
}
} catch (SQLException e) {
Logger.getLogger(DBConnection.class.getName()).log(Level.SEVERE,
null, e);
return null;
}
}
#Override
public void finalize() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
Logger.getLogger(DBConnection.class.getName()).log(
Level.SEVERE, null, e);
}
}
}
}
As I wrote in the title when I run the project and a class asks for a connection to this class I always get a Class Not Found Exception, Since it apparently can't load the org.postgresql.Driver.class The driver is located in a subfolder of the project ~/lib/org.postgresql-9.1-901.jdbc4.jar and as I said added to the build path via eclipse project properties.
I'm also providing a sample query to let see the usual behavior of my classes to access the DBConnection:
public static final User validateUserCredentials(String id, String pswd) {
Connection connection = DBConnection.getConnection();
Logger.getLogger(Credentials.class.getName()).log(Level.SEVERE, (connection!=null)?"connection not null":"connection null");
Statement stmt = null;
Logger.getLogger(Home.class.getName()).log(Level.SEVERE, "validating credentials for user: username : " + id + " password : " + pswd);
String sql = "Select * from fuser where id = '" + id + "'";
ResultSet resultset = null;
try {
stmt = connection.createStatement();
resultset = stmt.executeQuery(sql);
Logger.getLogger(Credentials.class.getName())
.log(Level.SEVERE, sql);
resultset.next();
String password = resultset.getString("pswd");
if (pswd.equals(password))
return new User(id, pswd);
} catch (SQLException ex) {
Logger.getLogger(Credentials.class.getName()).log(Level.SEVERE,
null, ex);
} finally {
if (stmt != null)
stmt = null;
if (resultset != null)
resultset = null;
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
}
connection = null;
}
}
return null;
}

I'm working on a web project and I recently installed postgres 9.1.1
...
I added it to the java build path using the project properties via eclipse.
That's the wrong way. That JAR has to be dropped straight in /WEB-INF/lib folder of the web project without fiddling with the Build Path in the project's properties. That folder is standard part of webapp's runtime classpath.
Unrelated to the concrete problem: you've a major design flaw in your DBConnection class. You've declared Connection as static which essentially makes your connection not threadsafe. Use a connection pool and never assign the Connection (nor Statement nor ResultSet) as a class/instance variable. They should be created and closed in the very same try-finally block as where you're executing the query. Further you've there also a SQL injection hole. Use PreparedStatement instead of concatenating user-controlled variables in the SQL string.
See also:
JDBC MySql connection pooling practices to avoid exhausted connection pool
Get database connection from a connection pool
Am I Using JDBC Connection Pooling?

Add this dependency in your pom:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1203-jdbc4</version>
</dependency>

The first thing I'd do is unpack the jar and confirm that the driver is really in there as org.postgresql.Driver. I notice when looking at jarfinder and related sites that there isn't a Postgres 9.x jar containing org.postgresql.Driver.

Which should I close first, the PreparedStatement or the Connection?

When using a PreparedStatement in JDBC, should I close the PreparedStatement first or the Connection first? I just saw a code sample in which the Connection is closed first, but it seems to me more logical to close the PreparedStatement first.
Is there a standard, accepted way to do this? Does it matter? Does closing the Connection also cause the PreparedStatement to be closed, since the PreparedStatement is directly related to the Connection object?

The statement. I would expect you to close (in order)
the result set
the statement
the connection
(and check for nulls along the way!)
i.e. close in reverse order to the opening sequence.
If you use Spring JdbcTemplate (or similar) then that will look after this for you. Alternatively you can use Apache Commons DbUtils and DbUtils.close() or DbUtils.closeQuietly().

The following procedures should be done (in order)
The ResultSet
The PreparedStatement
The Connection.
Also, it's advisable to close all JDBC related objects in the finally close to guarantee closure.
//Do the following when dealing with JDBC. This is how I've implemented my JDBC transactions through DAO....
Connection conn = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
conn = ....
ps = conn.prepareStatement(...);
//Populate PreparedStatement
rs = ps.executeQuery();
} catch (/*All relevant exceptions such as SQLException*/Exception e) {
logger.error("Damn, stupid exception: " , e);
} finally {
if (rs != null) {
try {
rs.close();
rs = null;
} catch (SQLException e) {
logger.error(e.getMessage(), e.fillInStackTrace());
}
}
if (ps != null) {
try {
ps.close();
ps = null;
} catch (SQLException e) {
logger.error(e.getMessage(), e.fillInStackTrace());
}
}
try {
if (conn!= null && !conn.isClosed()){
if (!conn.getAutoCommit()) {
conn.commit();
conn.setAutoCommit(true);
}
conn.close();
conn= null;
}
} catch (SQLException sqle) {
logger.error(sqle.getMessage(), sqle.fillInStackTrace());
}
}
You can see I've checked if my objects are null and for connection, check first if the connection is not autocommited. Many people fail to check it and realise that the transaction hasn't been committed to DB.