We Keep Coding

GemfireXD - PreparedStatement setArray for String(VARCHAR) array not working

I have some JDBC code as follows where I am using gemfireXD as DB.
I am trying to parallelize the execution of a procedure on a list of inputs. The size of the list I am passing as the parameter to the procedure is not fixed and it is determined after the execution of the query in the code below. This list can be too big in size.
Code:
private static void executeProc (Connection cxn) throws SQLException {
Statement stmt = null;
try {
stmt = cxn.createStatement();
stmt.execute("select distinct field1 from Table1");
ResultSet rs = stmt.getResultSet();
List<String> fieldList = new ArrayList<String> ();
while (rs.next()) {
fieldList.add(rs.getString("field1"));
}
Array fieldArray = cxn.createArrayOf("VARCHAR", fieldList.toArray(new String[fieldList.size()]));
CallableStatement callableStmt = cxn.prepareCall("{CALL procedure1() ON TABLE Table1 WHERE field1 IN (?)}");
callableStmt.setArray(1, fieldArray);
callableStmt.execute();
} catch (SQLException e) {
e.printStackTrace();
}
}
Error:
while executing the code it is giving below runtime error.
Feature not Implemented : createArrayOf(String, Object[])
Question:
The gemfireXD documentation says Data Aware procedure can be executed passing a list using setArray method. But when I am working on actual API, the setAaary() method is expecting an Array Object instead of a List.
GemfireXD document gives an example of such procedure call in its documentation (1.4.0) which supports setArray() method. The direct example from the documentation is as follows: -
// GemFire XD data-aware procedure invocation.
CallableStatement callableStmt = connection.prepareCall("{CALL order_credit_check() ON TABLE Orders WHERE customerID IN (?)}");
callableStmt.setArray(1, <-list of customer IDs->);
How do I call preparedStatement.setArray to set the parameter in the query?
Could any one please suggest any other implementation approch to achieve the desired result?

GemFire XD does not support Array data type so createArrayOf method will not work.
For the kind of query in you question where you want to use in-list in the where clause, you may have to set each individual value as a parameter
CallableStatement callableStmt = cxn.prepareCall("{CALL procedure1() ON TABLE Table1 WHERE field1 IN (?, ?, ?)}");
callableStmt.setInt(1, 1);
callableStmt.setInt(2, 2);
callableStmt.setInt(3, 3);
....

how Can I get the full query that a PreparedStatement is about to execute? [duplicate]

I have a general Java method with the following method signature:
private static ResultSet runSQLResultSet(String sql, Object... queryParams)
It opens a connection, builds a PreparedStatement using the sql statement and the parameters in the queryParams variable length array, runs it, caches the ResultSet (in a CachedRowSetImpl), closes the connection, and returns the cached result set.
I have exception handling in the method that logs errors. I log the sql statement as part of the log since it's very helpful for debugging. My problem is that logging the String variable sql logs the template statement with ?'s instead of actual values. I want to log the actual statement that was executed (or tried to execute).
So... Is there any way to get the actual SQL statement that will be run by a PreparedStatement? (Without building it myself. If I can't find a way to access the PreparedStatement's SQL, I'll probably end up building it myself in my catches.)

Using prepared statements, there is no "SQL query" :
You have a statement, containing placeholders
it is sent to the DB server
and prepared there
which means the SQL statement is "analysed", parsed, some data-structure representing it is prepared in memory
And, then, you have bound variables
which are sent to the server
and the prepared statement is executed -- working on those data
But there is no re-construction of an actual real SQL query -- neither on the Java side, nor on the database side.
So, there is no way to get the prepared statement's SQL -- as there is no such SQL.
For debugging purpose, the solutions are either to :
Ouput the code of the statement, with the placeholders and the list of data
Or to "build" some SQL query "by hand".

It's nowhere definied in the JDBC API contract, but if you're lucky, the JDBC driver in question may return the complete SQL by just calling PreparedStatement#toString(). I.e.
System.out.println(preparedStatement);
At least MySQL 5.x and PostgreSQL 8.x JDBC drivers support it. However, most other JDBC drivers doesn't support it. If you have such one, then your best bet is using Log4jdbc or P6Spy.
Alternatively, you can also write a generic function which takes a Connection, a SQL string and the statement values and returns a PreparedStatement after logging the SQL string and the values. Kickoff example:
public static PreparedStatement prepareStatement(Connection connection, String sql, Object... values) throws SQLException {
PreparedStatement preparedStatement = connection.prepareStatement(sql);
for (int i = 0; i < values.length; i++) {
preparedStatement.setObject(i + 1, values[i]);
}
logger.debug(sql + " " + Arrays.asList(values));
return preparedStatement;
}
and use it as
try {
connection = database.getConnection();
preparedStatement = prepareStatement(connection, SQL, values);
resultSet = preparedStatement.executeQuery();
// ...
Another alternative is to implement a custom PreparedStatement which wraps (decorates) the real PreparedStatement on construction and overrides all the methods so that it calls the methods of the real PreparedStatement and collects the values in all the setXXX() methods and lazily constructs the "actual" SQL string whenever one of the executeXXX() methods is called (quite a work, but most IDE's provides autogenerators for decorator methods, Eclipse does). Finally just use it instead. That's also basically what P6Spy and consorts already do under the hoods.

I'm using Java 8, JDBC driver with MySQL connector v. 5.1.31.
I may get real SQL string using this method:
// 1. make connection somehow, it's conn variable
// 2. make prepered statement template
PreparedStatement stmt = conn.prepareStatement(
"INSERT INTO oc_manufacturer" +
" SET" +
" manufacturer_id = ?," +
" name = ?," +
" sort_order=0;"
);
// 3. fill template
stmt.setInt(1, 23);
stmt.setString(2, 'Google');
// 4. print sql string
System.out.println(((JDBC4PreparedStatement)stmt).asSql());
So it returns smth like this:
INSERT INTO oc_manufacturer SET manufacturer_id = 23, name = 'Google', sort_order=0;

If you're executing the query and expecting a ResultSet (you are in this scenario, at least) then you can simply call ResultSet's getStatement() like so:
ResultSet rs = pstmt.executeQuery();
String executedQuery = rs.getStatement().toString();
The variable executedQuery will contain the statement that was used to create the ResultSet.
Now, I realize this question is quite old, but I hope this helps someone..

I've extracted my sql from PreparedStatement using preparedStatement.toString() In my case toString() returns String like this:
org.hsqldb.jdbc.JDBCPreparedStatement#7098b907[sql=[INSERT INTO
TABLE_NAME(COLUMN_NAME, COLUMN_NAME, COLUMN_NAME) VALUES(?, ?, ?)],
parameters=[[value], [value], [value]]]
Now I've created a method (Java 8), which is using regex to extract both query and values and put them into map:
private Map<String, String> extractSql(PreparedStatement preparedStatement) {
Map<String, String> extractedParameters = new HashMap<>();
Pattern pattern = Pattern.compile(".*\\[sql=\\[(.*)],\\sparameters=\\[(.*)]].*");
Matcher matcher = pattern.matcher(preparedStatement.toString());
while (matcher.find()) {
extractedParameters.put("query", matcher.group(1));
extractedParameters.put("values", Stream.of(matcher.group(2).split(","))
.map(line -> line.replaceAll("(\\[|])", ""))
.collect(Collectors.joining(", ")));
}
return extractedParameters;
}
This method returns map where we have key-value pairs:
"query" -> "INSERT INTO TABLE_NAME(COLUMN_NAME, COLUMN_NAME, COLUMN_NAME) VALUES(?, ?, ?)"
"values" -> "value, value, value"
Now - if you want values as list you can just simply use:
List<String> values = Stream.of(yourExtractedParametersMap.get("values").split(","))
.collect(Collectors.toList());
If your preparedStatement.toString() is different than in my case it's just a matter of "adjusting" regex.

Using PostgreSQL 9.6.x with official Java driver 42.2.4:
...myPreparedStatement.execute...
myPreparedStatement.toString()
Will show the SQL with the ? already replaced, which is what I was looking for.
Just added this answer to cover the postgres case.
I would never have thought it could be so simple.

Code Snippet to convert SQL PreparedStaments with the list of arguments. It works for me
/**
*
* formatQuery Utility function which will convert SQL
*
* #param sql
* #param arguments
* #return
*/
public static String formatQuery(final String sql, Object... arguments) {
if (arguments != null && arguments.length <= 0) {
return sql;
}
String query = sql;
int count = 0;
while (query.matches("(.*)\\?(.*)")) {
query = query.replaceFirst("\\?", "{" + count + "}");
count++;
}
String formatedString = java.text.MessageFormat.format(query, arguments);
return formatedString;
}

Very late :) but you can get the original SQL from an OraclePreparedStatementWrapper by
((OraclePreparedStatementWrapper) preparedStatement).getOriginalSql();

I implemented the following code for printing SQL from PrepareStatement
public void printSqlStatement(PreparedStatement preparedStatement, String sql) throws SQLException{
String[] sqlArrya= new String[preparedStatement.getParameterMetaData().getParameterCount()];
try {
Pattern pattern = Pattern.compile("\\?");
Matcher matcher = pattern.matcher(sql);
StringBuffer sb = new StringBuffer();
int indx = 1; // Parameter begin with index 1
while (matcher.find()) {
matcher.appendReplacement(sb,String.valueOf(sqlArrya[indx]));
}
matcher.appendTail(sb);
System.out.println("Executing Query [" + sb.toString() + "] with Database[" + "] ...");
} catch (Exception ex) {
System.out.println("Executing Query [" + sql + "] with Database[" + "] ...");
}
}

If you're using MySQL you can log the queries using MySQL's query log. I don't know if other vendors provide this feature, but chances are they do.

Simply function:
public static String getSQL (Statement stmt){
String tempSQL = stmt.toString();
//please cut everything before sql from statement
//javadb...:
int i1 = tempSQL.indexOf(":")+2;
tempSQL = tempSQL.substring(i1);
return tempSQL;
}
It's fine aswell for preparedStatement.

I'm using Oralce 11g and couldn't manage to get the final SQL from the PreparedStatement. After reading #Pascal MARTIN answer I understand why.
I just abandonned the idea of using PreparedStatement and used a simple text formatter which fitted my needs. Here's my example:
//I jump to the point after connexion has been made ...
java.sql.Statement stmt = cnx.createStatement();
String sqlTemplate = "SELECT * FROM Users WHERE Id IN ({0})";
String sqlInParam = "21,34,3434,32"; //some random ids
String sqlFinalSql = java.text.MesssageFormat(sqlTemplate,sqlInParam);
System.out.println("SQL : " + sqlFinalSql);
rsRes = stmt.executeQuery(sqlFinalSql);
You figure out the sqlInParam can be built dynamically in a (for,while) loop I just made it plain simple to get to the point of using the MessageFormat class to serve as a string template formater for the SQL query.

You can try to use javaagent to print SQL:
public class Main {
private static final String mybatisPath = "org.apache.ibatis.executor.statement.PreparedStatementHandler";
private static final String mybatisMethod = "parameterize";
private static final String sqlPath = "java.sql.Statement";
public static void premain(String arg, Instrumentation instrumentation) {
instrumentation.addTransformer(new ClassFileTransformer() {
#Override
public byte[] transform(
ClassLoader loader,
String className,
Class<?> classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
if (!mybatisPath.replaceAll("\\.", "/").equals(className)) {
return null;
}
ClassPool pool = new ClassPool();
pool.appendClassPath(new LoaderClassPath(loader));
pool.appendSystemPath();
try {
CtClass ctClass = pool.get(mybatisPath);
CtMethod method = ctClass.getDeclaredMethod(mybatisMethod, new CtClass[]{pool.get(sqlPath)});
method.insertAfter("cn.wjhub.Main#printSQL($1)");
return ctClass.toBytecode();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
});
}
/**
* printSQL
*
* #param statement statement
*/
private void printSQL(Statement statement) {
String sqlSource = statement.toString();
System.out.println(sqlSource);
}
}

To do this you need a JDBC Connection and/or driver that supports logging the sql at a low level.
Take a look at log4jdbc

Calling PL/SQL package code in a Java Program

I am trying to call a procedure defined with a PL/SQL package in a Java program.
I am aware one can call stored procedures using connection.prepareCall in Jdbc. But there is very little information out there on how to call a procedure within a package.
I am at a stage in development where i am still considering what db framework to use. Just wondering what are the pros and cons of using JDBC for PLSQL ? For this usecase are there better alternatives to JDBC ?

Follow the simple steps below:
public static final String SOME_NAME = "{call schema_name.org_name_pkg.return_something(?,?)}"; // Change the schema name,packagename,and procedure name.
// Simple JDBC Connection Pooling
// Here I am passing param companyId which is IN param to stored procedure which will return me some value.
Connection conn = null;
CallableStatement stmt = null;
ResultSet rset = null;
try {
conn = DriverManager.getConnection("jdbc:mysql://hostname:port/dbname","username", "password");
stmt = conn.prepareCall(SOME_NAME);//We have declared this at the very top
stmt.setString(1, companyid);//Passing CompanyID here
stmt.registerOutParameter(2, OracleTypes.CURSOR);//Refcursor selects the row based upon query results provided in Package.
stmt.execute();
rset = (ResultSet) stmt.getObject(2);
while (rset.next()) {
String orgId=rset.getString("RPT_ORG_ID");
// When using refcursor easy to get the value just by using Column name
String orgName=rset.getString("RPT_ORG_NAME");
// Some Logic based what do you want to do with the data returned back from query
} catch (Exception e) {
LOGGER.error("Error extracting ", e);
} finally {
DBUtils.cleanUp(conn, stmt, rset);
}
// Clean and close you connection

Java INSERT to SQL Database with JDBC

Just looking for some small help here. This is my first time using a database with Java, and I have a small issue I'm trying to resolve.
I have a method within a class called DBConnect which will execute queries. I'd like to insert this List into my database.
List<String> data = new ArrayList();
data.add(name);
data.add(bank);
data.add(pin);
data.add(email);
data.add(pass);
data.add(phone);
data.add(paypal_email);
data.add(paypal_pass);
data.add(IPV4Assistant.getExternalIPAddress());
data.add(crypto);
data.add("1");
data.add(dob);
DBConnect.executeQuery();
I suppose I'd start creating the query string with
String insert = ("INSERT INTO Client_Data (card_number,card_pin,client_dob,crypto_currency_address,email,email_password,id,ip_address,name,paypal_email,paypal_password,phone_number) VALUES
The above fields being the columns I'm trying to insert into, and Client_Data being my table.
How do I go about formatting the fields in my list to query properly?
After Values I believe the format is ('data','data','data').
Could anybody experienced with JDBC please assist me?
Thank you.

I would use PreparedStatements to insert the values into your table.
/*
* Code
* I am assuming that you have a Connection object named conn.
* This is just a simple example
*/
try(
PreparedStatement ps = conn.prepareStatement(
"insert into yourTable(field1, field2, field3) values (?,?,?)"
) {
/*
* The question marks are placeholders for the values you will insert.
*/
ps.setString(1, "abc");
ps.setInt(2, 123);
ps.setDouble(3, 3.1416);
ps.execute(); // The insert is executed here
} catch(SQLException e) {
// Your exception handling code
}
If you need to insert values into your table using a loop, you may also execute the inserts as a batch:
/*
* Code
*/
try(
PreparedStatement ps = conn.prepareStatement(
"insert into yourTable(field1, field2, field3) values (?,?,?)"
) {
for(int i = 0; i < 10; i++) {
ps.setString(1, "abc");
ps.setInt(2, 123 * i);
ps.setDouble(3, 3.1416);
ps.addBatch(); // The insert is added to a batch, pending for execution
}
ps.executeBatch(); // All the inserts added to the batch are executed.
} catch(SQLException e) {
// Your exception handling code
}
Reference:
The Java Tutorials: JDBC - Using Prepared Statements (You may also want to read the full JDBC tutorial)
Java API reference for the PreparedStatement class

Basically, you should be trying to use PreparedStatement, there are a number of very good reasons for this, but in your case, it's the simplest way to bind the values from your List to the Statement
For example, you could start by defining the insert statement as a constant, this isn't required, but for the example, it made it easier...
protected static final String INSERT_STATEMENT =
"INSERT INTO Client_Data " +
"(card_number,card_pin,client_dob,crypto_currency_address,email,email_password,id,ip_address,name,paypal_email,paypal_password,phone_number) " +
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?)";
Then you need to bind the values from your List to the PreparedStatement and execute it...
List<String> data = new ArrayList();
data.add(name);
data.add(bank);
data.add(pin);
data.add(email);
data.add(pass);
data.add(phone);
data.add(paypal_email);
data.add(paypal_pass);
data.add(IPV4Assistant.getExternalIPAddress());
data.add(crypto);
data.add("1");
data.add(dob);
// Replace with your own connection management, just here for
// example reasons
try (Connection con = DriverManager.getConnection(url)) {
try (PreparedStatement stmt = con.prepareStatement(INSERT_STATEMENT)) {
for (int index = 0; index < data.size(); index++) {
stmt.setObject(index + 1, data.get(index));
int rows = stmt.executeUpdate();
// Check the value of rows if you want to know how
// many rows were affected by the change
}
}
} catch (SQLException exp) {
// Possibly throw this to the call instead...
exp.printStackTrace();
}
I assume, you'll be passing the List as an parameter to some method.
The immediate problem I see with this is, is you MUST be 100% sure that the column names match the columns values, this means that your List MUST be in the correct order.
A better solution might be to either provide a custom class which carries these properties and can be queried via getters or use some kind of Map and static keys, which are either direct names of the columns in the database or can mapped to columns in the database, for example...
public static final String CLIENT_NAME = "name";
//... Other column names/keys...
//...
Map<String, Object> clientData = new HashMap<String, Object>();
clientData.put(CLIENT_NAME, name);
//...
stmt.setObject(CLIENT_NAME, clientData.get(CLIENT_NAME));
You should also avoid inserting String into columns which have different data type requirements (such as Date, TimeStamp and/or numbers). Instead, you should be trying to use the correct JDBC mapping types where possible
Take a look at Using Prepared Statements for more details

Why do I need a connection to create PreparedStatements?

I would like to use prepared statements, for many reasons.
But, I would like to create a method that looks like this:
/* This opens a connection, executes the query, and closes the connection */
public static void executeNonQuery(String queryString);
In other words, I want my application logic to only have to formulate the queries and feed in parameters, but not deal with connections & statements. However, PreparedStatements are created from a connection object, so I am currently forced into preparing the query string using String.format() - butt ugly and dangerous.
Is there a way to do what I want without using String.format()?

Why do I need a connection to create PreparedStatements ?
Because the statements are prepared on per-connection basis in most RDBMS's.
Prepared statements are in fact cached execution plans that don't take you permissions, encodings, collation settings etc. into account.
All this is done during query parsing.
Is there a way to do what I want without using String.format()
Don't see why you need String.format() here.
You can implement your query as a class, create a connection and prepare the query in the class constructor and then execute it in a method.
A parametrized query typically looks like this:
SELECT *
FROM table
WHERE col1 = ?
AND col2 = ?
, where the bound parameters will be substituted for ?'s during the query execution.
If you want a static method:
Create a static connection handle.
Create a static hash table of prepared queries using the parametrized query text as a key, and the handle to the prepared query as a value.
Whenever you want to execute a query, find its handle (or create it if it wasn't found) and use to to bind the parameters and execute the query.

Why not have your "application" logic use a data layer which you create which can present that kind of interface method?
Your data layer can then handle creating connections, preparing statements, etc., all within that executeNonQuery method.
I think that if you are attempting to merge the parameters in your query/statement yourself into a String, then you are shooting yourself in the foot and actually not using the parameter functionality of PreparedStatements. Not sure why you would want to do this.
You might also want to look into using an API such as Spring, which has a series of JdbcTemplate classes that can abstract all of the connection handling away from you, but still allow you to work with parameters in a Map.

You probably want something like the DbUtils package in the Apache Commons libraries: [http://commons.apache.org/dbutils/index.html][1]
The QueryRunner class lets you execute sql statements without having to manually create PreparedStatements, or even have an open connection for that matter. From the examples page:
QueryRunner run = new QueryRunner( dataSource );
try
{
// Create an object array to hold the values to insert
Object[] insertParams = {"John Doe", new Double( 1.82 )};
// Execute the SQL update statement and return the number of
// inserts that were made
int inserts = run.update( "INSERT INTO Person (name,height) VALUES (?,?)",
insertParams );
// Now it's time to rise to the occation...
Object[] updateParams = {new Double( 2.05 ), "John Doe"};
int updates = run.update( "UPDATE Person SET height=? WHERE name=?",
updateParams );
}
catch(SQLException sqle) {
// Handle it
}
So it basically handles the creation of prepared statements transparently, and the only thing you really need to know is a DataSource. This also works just as well for non-update/insert statements, i.e. plain-vanilla select queries, and the ability to create ResultSetHandlers gives you the power to convert a ResultSet into something like a fully-prepared bean, or a Map with the keys being the column names, and the values being the actual row values. Very useful for when you can't implement a whole ORM solution.

I abstract out all of the JDBC stuff by having a class I call QueryRunner that has an execute method that takes the sql, a List of objects that represent the parameters, and an object that will process the ResultSet. If you use the setObject method from JDBC to set your parameters it will figure out the appropriate DB types to use based on the underlying object. Here is a portion of my code. I've got another method that wraps this one and get's the connection.
public void executeNoCommit(Connection conn,
String sql,
List params,
ResultSetProcessor processor) throws SQLException {
PreparedStatement stmt = null;
ResultSet rs = null;
int updateCount = 0;
Iterator it;
int paramIndex = 1;
boolean query;
try {
stmt = conn.prepareStatement(sql);
if (params != null) {
it = params.iterator();
while (it.hasNext()) {
stmt.setObject(paramIndex, it.next());
paramIndex++;
}
}
query = stmt.execute();
if (query) {
rs = stmt.getResultSet();
}
else {
updateCount = stmt.getUpdateCount();
}
processor.process(rs, updateCount);
}
finally {
if (rs != null) {
try {
rs.close();
}
catch (SQLException e) {
log.error(e);
}
}
if (stmt != null) {
try {
stmt.close();
}
catch (SQLException e) {
log.error(e);
}
}
}
}