We Keep Coding

SQLite Query With Parameters Not Working in Java

I have a program that selects from a database given a table and column string.
public void selectAllFrom(String table, String column){
String sql = "SELECT ? FROM ?";
try (Connection conn = this.connect();
PreparedStatement pstmt = conn.prepareStatement(sql)){
pstmt.setString(1, column);
pstmt.setString(2, table);
ResultSet rs = pstmt.executeQuery();
while (rs.next()){
System.out.println(rs.getString(column));
}
} catch (SQLException e){
System.out.println(" select didn't work");
System.out.println(e.getMessage());
}
}
For some reason it is not working and it is going right to catch
Here is the connect() function as well:
private Connection connect(){
Connection conn = null;
// SQLite connection string
String url = "jdbc:sqlite:C:/sqlite/db/chinook.db";
try{
// creates connection to the database
conn = DriverManager.getConnection(url);
System.out.println("Connection to SQLite has been established");
} catch (SQLException e){
System.out.println(e.getMessage());
System.out.println("Connection didn't work");
}
return conn;
}
I know the problem is not with the database because I'm able to run other select queries without parameters. It is the parameters that are giving me the problem. Can anyone tell what the problem is?

A table or column name can't be used as a parameter to PreparedStatement. It must be hard coded.
String sql = "SELECT " + column + " FROM " + table;
You should reconsider the design so as to make these two constant and parameterize the column values.

? is a place holder to indicate a bind variable. When a SQL statement is executed, database first checks syntax, and validates the objects being referenced, columns and access permission for specified objects (i.e metadata about objects) and confirms that all are in place and valid. This stage is called parsing.
Post parsing, it substitutes bind variables to query and then proceeds for actual fetch of results.
Bind variables can be substituted in any place in query to replace an actual hard coded data/strings, but not the query constructs them selves. It means
You can not use bind variables for keywords of sql query (ex: SELECT, UPDATE etc.)
You can not use bind variables for objects or their attributes (i.e table names, column names, functions, procedures etc.)
You can use them only in place of a otherwise hard coded data.
ex: SELECT FIRST_NAME, LAST_NAME, 'N' IS_DELETED FROM USER_DATA WHERE COUNTRY ='CANADA' AND VERIFIED_USER='YES'
In above sample query, 'N','CANADA' and 'YES' are the only strings which can be replaced by a bind variable, not any other word.
Using bind variable is best practice of coding. It improves query performance (when used with large no. of queries in tuned database products like Oracle or MSSQL) and also protects your code against sql injection attacks.
Constructing query by concatenating strings (especially data part of query) is never recommended way. You can still construct a query by concatenation for other parts like table name or column name as long as those strings are not directly taken from input.
Below example is acceptable:
query = "Select transaction_id, transaction_date from ";
if (isHistorical(reportType)
{ query = query + "HISTORY_TRANSACTIONS" ;}
else
{query = query + "PRESENT_TRANSACTIONS" ; }
recommended practice is to use
String query_present = "SELECT transaction_id, transaction_date from PRESENT_TRANSACTIONS";
String query_historical = "SELECT transaction_id, transaction_date from HISTORY_TRANSACTIONS";
if (isHisotrical(reportType))
{
ps.executeQuery(query_historical);
}else{
ps.executeQuery(query_present);
}

How to add rows to a table after the columns were altered? (SQLite, JAVA)

I am trying to add rows to a table. Initially, I had this code but it creates an error that there are 8 columns (additional columns were altered, initially I only had 2 columns) but I am only adding 2 values:
PreparedStatement addDate =
con.prepareStatement("insert into contributions values(?,?)");
addDate.setString(2, string);
addDate.execute();
}
And then I tried this:
public void addDateToContributionsTable(String string) throws ClassNotFoundException, SQLException {
if(con == null) {
// get connection
getConnection();
}
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("Select * from contributions");
ResultSetMetaData rsmd = rs.getMetaData();
int columnsNumber = rsmd.getColumnCount();
StringBuilder sb = new StringBuilder();
for(int i= 0; i < columnsNumber; i++){
sb.append("?");
sb.append(",");
}
System.out.println(sb.toString());
System.out.println("insert into contributions values('"+sb.toString()+"')");
PreparedStatement addDate = con
.prepareStatement("insert into contributions values('"+sb.toString()+"')");
addDate.setString(2, string);
addDate.execute();
}
But I am still having the same error:
[SQLITE_ERROR] SQL error or missing database (table contributions has 8 columns but 1 values were supplied)
Any help would be appreciated. :)

The values don't know where they should be inserted, so I suggest writing the column names for each value.
reparedStatement addDate =
con.prepareStatement("insert into contributions (COLUMN_NAMES) values (?,?)");
addDate.setString(2, string);
addDate.execute();
}

You added multiple bind parameters in the SQL, but you only ever bound one of them:
addDate.setString(2, string);
You need to call setXXX() per bind parameter.
If you are really sure you want to use the same value, you can instead use named parameters instead, then you won't have to call setXXX() multiple times. You can refer to this:
Named parameters in JDBC
Also, questionMarksList and StringBuilder sb are doing the same thing over two loops.
Edit
If your SQL has 2 question marks, you have 2 bind parameters, then you need to set two bind parameters.
e.g. For SQL with 3 bind parameters:
INSERT INTO MYTABLE VALUES(?, ?, ?)
You need to provide 3 values:
addDate.setString(1, "String1"); // Bind to first question mark
addDate.setString(2, "String2"); // etc.
addDate.setString(3, "String3");

Thanks to all that responded to my question. The problem was with the number of ? I had on my statement every time I increase or decrease the column numbers. The only problem with my statement is having this expression '' . When I changed it to "+questionMarks+" instead of '"+questionMarks+"', it worked. Does not matter how many setXXX() method I use as long as the number of ? are the same with the number of columns on the table, it will work. Having the code below, I did not encounter any errors anymore.
PreparedStatement addmembers = con
.prepareStatement("insert into membersdata values "+questionMarks+"");
addmembers.setString(2, name);
addmembers.setString(3, accounts);
addmembers.setString(4, email);
addmembers.execute();

How to get all the rows affected by a UPDATE query in JDBC?

I have an assignment where I need to update records using a PreparedStatement. Once the record have been updated as we know update query return count, i.e., number of row affected.
However, instead of the count I want the rows that were affected by update query in response, or at least a list of id values for the rows that were affected.
This my update query.
UPDATE User_Information uInfo SET address = uInfo.contact_number || uInfo.address where uInfo.user_id between ? AND ?;
Normally it will return count of row affected but in my case query should return the ids of row or all the row affected.
I have used the returning function of PostgreSQL it is working but is not useful for me in that case.

i have used returning function of PostgreSQL but is not useful for me
It should be. Perhaps you were just using it wrong. This code works for me:
sql = "UPDATE table1 SET customer = customer || 'X' WHERE customer LIKE 'ba%' RETURNING id";
try (PreparedStatement s = conn.prepareStatement(sql)) {
s.execute(); // perform the UPDATE
try (ResultSet rs = s.getResultSet()) {
// loop through rows from the RETURNING clause
while (rs.next()) {
System.out.println(rs.getInt("id")); // print the "id" value of the updated row
}
}
}
The documentation indicates that we can also use RETURNING * if we want the ResultSet to include the entire updated row.
Update:
As #CraigRinger suggests in his comment, the PostgreSQL JDBC driver does actually support .getGeneratedKeys() for UPDATE statements too, so this code worked for me as well:
sql = "UPDATE table1 SET customer = customer || 'X' WHERE customer LIKE 'ba%'";
try (PreparedStatement s = conn.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) {
s.execute(); // perform the UPDATE
try (ResultSet rs = s.getGeneratedKeys()) {
while (rs.next()) {
System.out.println(rs.getInt(1)); // print the "id" value of the updated row
}
}
}
Thanks, Craig!

You might be able to use JDBC's support for getting generated keys. See the Connection.prepareStatement(String sql, int[] columnIndexes) API method, then use Statement.getGeneratedKeys() to access the results.
The spec says "the driver will ignore the array if the SQL statement is not an INSERT statement" but I think PostgreSQL's JDBC driver will actually honour your request with other statement types too.
e.g.
PreparedStatement s = conn.prepareStatement(sql, new String[] {'id'})
s.executeUpdate();
ResultSet rs = s.getGeneratedKeys();
Otherwise, use RETURNING, as Gord Thompson describes.

There are two way of doing it
1. by passing an array of column name or index of column prepareStatement
i.e conn.prepareStatement(sql, new String[] {'id','uname'})
and
2. by using Statement.RETURN_GENERATED_KEYS in prepareStatement.
My code is for this i.e as per my requirement i have developed my code you can have a look for better idea.
private static final String UPDATE_USER_QUERY= "UPDATE User_Information uInfo SET address = uInfo.contact_number || uInfo.address where uInfo.user_id between ? AND ?;";
//pst = connection.prepareStatement(UPDATE_USER_QUERY,columnNames);
pst = connection.prepareStatement(UPDATE_USER_QUERY,Statement.RETURN_GENERATED_KEYS);
ResultSet rst = pst.getGeneratedKeys();
List<UserInformation> userInformationList = new ArrayList<UserInformation>();
UserInformation userInformation;
while (rst.next()){
userInformation = new UserInformation();
userInformation.setUserId(rst.getLong("user_id"));
userInformation.setUserName(rst.getString("user_name"));
userInformation.setUserLName(rst.getString("user_lName"));
userInformation.setAddress(rst.getString("address"));
userInformation.setContactNumber(rst.getLong("contact_number"));
userInformationList.add(userInformation);
}
That think i need to achieve in this case.
Hope so this will help you a lot.

how Can I get the full query that a PreparedStatement is about to execute? [duplicate]

I have a general Java method with the following method signature:
private static ResultSet runSQLResultSet(String sql, Object... queryParams)
It opens a connection, builds a PreparedStatement using the sql statement and the parameters in the queryParams variable length array, runs it, caches the ResultSet (in a CachedRowSetImpl), closes the connection, and returns the cached result set.
I have exception handling in the method that logs errors. I log the sql statement as part of the log since it's very helpful for debugging. My problem is that logging the String variable sql logs the template statement with ?'s instead of actual values. I want to log the actual statement that was executed (or tried to execute).
So... Is there any way to get the actual SQL statement that will be run by a PreparedStatement? (Without building it myself. If I can't find a way to access the PreparedStatement's SQL, I'll probably end up building it myself in my catches.)

Using prepared statements, there is no "SQL query" :
You have a statement, containing placeholders
it is sent to the DB server
and prepared there
which means the SQL statement is "analysed", parsed, some data-structure representing it is prepared in memory
And, then, you have bound variables
which are sent to the server
and the prepared statement is executed -- working on those data
But there is no re-construction of an actual real SQL query -- neither on the Java side, nor on the database side.
So, there is no way to get the prepared statement's SQL -- as there is no such SQL.
For debugging purpose, the solutions are either to :
Ouput the code of the statement, with the placeholders and the list of data
Or to "build" some SQL query "by hand".

It's nowhere definied in the JDBC API contract, but if you're lucky, the JDBC driver in question may return the complete SQL by just calling PreparedStatement#toString(). I.e.
System.out.println(preparedStatement);
At least MySQL 5.x and PostgreSQL 8.x JDBC drivers support it. However, most other JDBC drivers doesn't support it. If you have such one, then your best bet is using Log4jdbc or P6Spy.
Alternatively, you can also write a generic function which takes a Connection, a SQL string and the statement values and returns a PreparedStatement after logging the SQL string and the values. Kickoff example:
public static PreparedStatement prepareStatement(Connection connection, String sql, Object... values) throws SQLException {
PreparedStatement preparedStatement = connection.prepareStatement(sql);
for (int i = 0; i < values.length; i++) {
preparedStatement.setObject(i + 1, values[i]);
}
logger.debug(sql + " " + Arrays.asList(values));
return preparedStatement;
}
and use it as
try {
connection = database.getConnection();
preparedStatement = prepareStatement(connection, SQL, values);
resultSet = preparedStatement.executeQuery();
// ...
Another alternative is to implement a custom PreparedStatement which wraps (decorates) the real PreparedStatement on construction and overrides all the methods so that it calls the methods of the real PreparedStatement and collects the values in all the setXXX() methods and lazily constructs the "actual" SQL string whenever one of the executeXXX() methods is called (quite a work, but most IDE's provides autogenerators for decorator methods, Eclipse does). Finally just use it instead. That's also basically what P6Spy and consorts already do under the hoods.

I'm using Java 8, JDBC driver with MySQL connector v. 5.1.31.
I may get real SQL string using this method:
// 1. make connection somehow, it's conn variable
// 2. make prepered statement template
PreparedStatement stmt = conn.prepareStatement(
"INSERT INTO oc_manufacturer" +
" SET" +
" manufacturer_id = ?," +
" name = ?," +
" sort_order=0;"
);
// 3. fill template
stmt.setInt(1, 23);
stmt.setString(2, 'Google');
// 4. print sql string
System.out.println(((JDBC4PreparedStatement)stmt).asSql());
So it returns smth like this:
INSERT INTO oc_manufacturer SET manufacturer_id = 23, name = 'Google', sort_order=0;

If you're executing the query and expecting a ResultSet (you are in this scenario, at least) then you can simply call ResultSet's getStatement() like so:
ResultSet rs = pstmt.executeQuery();
String executedQuery = rs.getStatement().toString();
The variable executedQuery will contain the statement that was used to create the ResultSet.
Now, I realize this question is quite old, but I hope this helps someone..

I've extracted my sql from PreparedStatement using preparedStatement.toString() In my case toString() returns String like this:
org.hsqldb.jdbc.JDBCPreparedStatement#7098b907[sql=[INSERT INTO
TABLE_NAME(COLUMN_NAME, COLUMN_NAME, COLUMN_NAME) VALUES(?, ?, ?)],
parameters=[[value], [value], [value]]]
Now I've created a method (Java 8), which is using regex to extract both query and values and put them into map:
private Map<String, String> extractSql(PreparedStatement preparedStatement) {
Map<String, String> extractedParameters = new HashMap<>();
Pattern pattern = Pattern.compile(".*\\[sql=\\[(.*)],\\sparameters=\\[(.*)]].*");
Matcher matcher = pattern.matcher(preparedStatement.toString());
while (matcher.find()) {
extractedParameters.put("query", matcher.group(1));
extractedParameters.put("values", Stream.of(matcher.group(2).split(","))
.map(line -> line.replaceAll("(\\[|])", ""))
.collect(Collectors.joining(", ")));
}
return extractedParameters;
}
This method returns map where we have key-value pairs:
"query" -> "INSERT INTO TABLE_NAME(COLUMN_NAME, COLUMN_NAME, COLUMN_NAME) VALUES(?, ?, ?)"
"values" -> "value, value, value"
Now - if you want values as list you can just simply use:
List<String> values = Stream.of(yourExtractedParametersMap.get("values").split(","))
.collect(Collectors.toList());
If your preparedStatement.toString() is different than in my case it's just a matter of "adjusting" regex.

Using PostgreSQL 9.6.x with official Java driver 42.2.4:
...myPreparedStatement.execute...
myPreparedStatement.toString()
Will show the SQL with the ? already replaced, which is what I was looking for.
Just added this answer to cover the postgres case.
I would never have thought it could be so simple.

Code Snippet to convert SQL PreparedStaments with the list of arguments. It works for me
/**
*
* formatQuery Utility function which will convert SQL
*
* #param sql
* #param arguments
* #return
*/
public static String formatQuery(final String sql, Object... arguments) {
if (arguments != null && arguments.length <= 0) {
return sql;
}
String query = sql;
int count = 0;
while (query.matches("(.*)\\?(.*)")) {
query = query.replaceFirst("\\?", "{" + count + "}");
count++;
}
String formatedString = java.text.MessageFormat.format(query, arguments);
return formatedString;
}

Very late :) but you can get the original SQL from an OraclePreparedStatementWrapper by
((OraclePreparedStatementWrapper) preparedStatement).getOriginalSql();

I implemented the following code for printing SQL from PrepareStatement
public void printSqlStatement(PreparedStatement preparedStatement, String sql) throws SQLException{
String[] sqlArrya= new String[preparedStatement.getParameterMetaData().getParameterCount()];
try {
Pattern pattern = Pattern.compile("\\?");
Matcher matcher = pattern.matcher(sql);
StringBuffer sb = new StringBuffer();
int indx = 1; // Parameter begin with index 1
while (matcher.find()) {
matcher.appendReplacement(sb,String.valueOf(sqlArrya[indx]));
}
matcher.appendTail(sb);
System.out.println("Executing Query [" + sb.toString() + "] with Database[" + "] ...");
} catch (Exception ex) {
System.out.println("Executing Query [" + sql + "] with Database[" + "] ...");
}
}

If you're using MySQL you can log the queries using MySQL's query log. I don't know if other vendors provide this feature, but chances are they do.

Simply function:
public static String getSQL (Statement stmt){
String tempSQL = stmt.toString();
//please cut everything before sql from statement
//javadb...:
int i1 = tempSQL.indexOf(":")+2;
tempSQL = tempSQL.substring(i1);
return tempSQL;
}
It's fine aswell for preparedStatement.

I'm using Oralce 11g and couldn't manage to get the final SQL from the PreparedStatement. After reading #Pascal MARTIN answer I understand why.
I just abandonned the idea of using PreparedStatement and used a simple text formatter which fitted my needs. Here's my example:
//I jump to the point after connexion has been made ...
java.sql.Statement stmt = cnx.createStatement();
String sqlTemplate = "SELECT * FROM Users WHERE Id IN ({0})";
String sqlInParam = "21,34,3434,32"; //some random ids
String sqlFinalSql = java.text.MesssageFormat(sqlTemplate,sqlInParam);
System.out.println("SQL : " + sqlFinalSql);
rsRes = stmt.executeQuery(sqlFinalSql);
You figure out the sqlInParam can be built dynamically in a (for,while) loop I just made it plain simple to get to the point of using the MessageFormat class to serve as a string template formater for the SQL query.

You can try to use javaagent to print SQL:
public class Main {
private static final String mybatisPath = "org.apache.ibatis.executor.statement.PreparedStatementHandler";
private static final String mybatisMethod = "parameterize";
private static final String sqlPath = "java.sql.Statement";
public static void premain(String arg, Instrumentation instrumentation) {
instrumentation.addTransformer(new ClassFileTransformer() {
#Override
public byte[] transform(
ClassLoader loader,
String className,
Class<?> classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
if (!mybatisPath.replaceAll("\\.", "/").equals(className)) {
return null;
}
ClassPool pool = new ClassPool();
pool.appendClassPath(new LoaderClassPath(loader));
pool.appendSystemPath();
try {
CtClass ctClass = pool.get(mybatisPath);
CtMethod method = ctClass.getDeclaredMethod(mybatisMethod, new CtClass[]{pool.get(sqlPath)});
method.insertAfter("cn.wjhub.Main#printSQL($1)");
return ctClass.toBytecode();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
});
}
/**
* printSQL
*
* #param statement statement
*/
private void printSQL(Statement statement) {
String sqlSource = statement.toString();
System.out.println(sqlSource);
}
}

To do this you need a JDBC Connection and/or driver that supports logging the sql at a low level.
Take a look at log4jdbc

Why do I need a connection to create PreparedStatements?

I would like to use prepared statements, for many reasons.
But, I would like to create a method that looks like this:
/* This opens a connection, executes the query, and closes the connection */
public static void executeNonQuery(String queryString);
In other words, I want my application logic to only have to formulate the queries and feed in parameters, but not deal with connections & statements. However, PreparedStatements are created from a connection object, so I am currently forced into preparing the query string using String.format() - butt ugly and dangerous.
Is there a way to do what I want without using String.format()?

Why do I need a connection to create PreparedStatements ?
Because the statements are prepared on per-connection basis in most RDBMS's.
Prepared statements are in fact cached execution plans that don't take you permissions, encodings, collation settings etc. into account.
All this is done during query parsing.
Is there a way to do what I want without using String.format()
Don't see why you need String.format() here.
You can implement your query as a class, create a connection and prepare the query in the class constructor and then execute it in a method.
A parametrized query typically looks like this:
SELECT *
FROM table
WHERE col1 = ?
AND col2 = ?
, where the bound parameters will be substituted for ?'s during the query execution.
If you want a static method:
Create a static connection handle.
Create a static hash table of prepared queries using the parametrized query text as a key, and the handle to the prepared query as a value.
Whenever you want to execute a query, find its handle (or create it if it wasn't found) and use to to bind the parameters and execute the query.

Why not have your "application" logic use a data layer which you create which can present that kind of interface method?
Your data layer can then handle creating connections, preparing statements, etc., all within that executeNonQuery method.
I think that if you are attempting to merge the parameters in your query/statement yourself into a String, then you are shooting yourself in the foot and actually not using the parameter functionality of PreparedStatements. Not sure why you would want to do this.
You might also want to look into using an API such as Spring, which has a series of JdbcTemplate classes that can abstract all of the connection handling away from you, but still allow you to work with parameters in a Map.

You probably want something like the DbUtils package in the Apache Commons libraries: [http://commons.apache.org/dbutils/index.html][1]
The QueryRunner class lets you execute sql statements without having to manually create PreparedStatements, or even have an open connection for that matter. From the examples page:
QueryRunner run = new QueryRunner( dataSource );
try
{
// Create an object array to hold the values to insert
Object[] insertParams = {"John Doe", new Double( 1.82 )};
// Execute the SQL update statement and return the number of
// inserts that were made
int inserts = run.update( "INSERT INTO Person (name,height) VALUES (?,?)",
insertParams );
// Now it's time to rise to the occation...
Object[] updateParams = {new Double( 2.05 ), "John Doe"};
int updates = run.update( "UPDATE Person SET height=? WHERE name=?",
updateParams );
}
catch(SQLException sqle) {
// Handle it
}
So it basically handles the creation of prepared statements transparently, and the only thing you really need to know is a DataSource. This also works just as well for non-update/insert statements, i.e. plain-vanilla select queries, and the ability to create ResultSetHandlers gives you the power to convert a ResultSet into something like a fully-prepared bean, or a Map with the keys being the column names, and the values being the actual row values. Very useful for when you can't implement a whole ORM solution.

I abstract out all of the JDBC stuff by having a class I call QueryRunner that has an execute method that takes the sql, a List of objects that represent the parameters, and an object that will process the ResultSet. If you use the setObject method from JDBC to set your parameters it will figure out the appropriate DB types to use based on the underlying object. Here is a portion of my code. I've got another method that wraps this one and get's the connection.
public void executeNoCommit(Connection conn,
String sql,
List params,
ResultSetProcessor processor) throws SQLException {
PreparedStatement stmt = null;
ResultSet rs = null;
int updateCount = 0;
Iterator it;
int paramIndex = 1;
boolean query;
try {
stmt = conn.prepareStatement(sql);
if (params != null) {
it = params.iterator();
while (it.hasNext()) {
stmt.setObject(paramIndex, it.next());
paramIndex++;
}
}
query = stmt.execute();
if (query) {
rs = stmt.getResultSet();
}
else {
updateCount = stmt.getUpdateCount();
}
processor.process(rs, updateCount);
}
finally {
if (rs != null) {
try {
rs.close();
}
catch (SQLException e) {
log.error(e);
}
}
if (stmt != null) {
try {
stmt.close();
}
catch (SQLException e) {
log.error(e);
}
}
}
}