We Keep Coding

Using JDBC to call a PL/SQL stored procedure with custom type input parameter, all fields are null

I'm using JDBC with createStruct() to call a stored procedure on an Oracle database that accepts a custom type as a parameter. The stored procedure inserts the custom type fields into a table and when I SELECT from the table later I see that all the fields that I tried to insert are NULL.
The custom type looks like this:
type record_rec as object (owner_id varchar2 (7),
target_id VARCHAR2 (8),
IP VARCHAR2 (15),
PREFIX varchar2 (7),
port varchar2 (4),
description VARCHAR2 (35),
cost_id varchar2(10))
The stored procedure looks like this:
package body "PKG_RECORDS"
IS
procedure P_ADD_RECORD (p_target_id in out VARCHAR2,
p_record_rec in record_rec)
is
l_target_id targets.target_id%TYPE;
BEGIN
Insert into targets (target_id,
owner_id,
IP,
description,
prefix,
start_date,
end_date,
cost_id,
port,
server_name,
server_code)
values (f_sequence ('TARGETS'),
p_record_rec.owner_id,
p_record_rec.ip,
p_record_rec.description,
p_record_rec.prefix,
sysdate,
to_date ('01-JAN-2050'),
p_record_rec.cost_id,
p_record_rec.port,
'test-server',
'51')
returning target_id
into p_target_id;
END;
END PKG_RECORDS;
My Java code looks something like this:
try (Connection con = m_dataSource.getConnection()) {
ArrayList<String> ids = new ArrayList<>();
CallableStatement call = con.prepareCall("{call PKG_RECORDS.P_ADD_RECORD(?,?)}");
for (Record r : records) {
call.registerOutParameter("p_target_id", Types.VARCHAR);
call.setObject("p_record_rec",
con.createStruct("SCHEME_ADM.RECORD_REC", new Object[] {
r.getTarget_id(),
null, // will be populated by SP
t.getIp(),
t.getPrefix(),
t.getPort(),
t.getDescription(),
t.getCost_id()
}), Types.STRUCT);
call.execute();
ids.add(call.getString("p_target_id"));
}
return new QueryRunner().query(con,
"SELECT * from TARGETS_V WHERE TARGET_ID IN ("+
ids.stream().map(s -> "?").collect(Collectors.joining(",")) +
")",
new BeanListHandler<Record>(Record.class),
ids.toArray(new Object[] {})
).stream()
.collect(Collectors.toList());
} catch (SQLException e) {
throw new DataAccessException(e.getMessage());
}
Notes:
* That last part is using Apache Commons db-utils - I love their bean stream operations.
* The connection is using C3P0 connection pool - could that be related?
* Just to make it clear - its not that the bean processor populates null values into the Record bean fields - if I use an SQL explorer to load the table (or view) directly, I can see that the fields in the database are indeed set to NULL.
There are no SQLExceptions when the process runs, or any other notice that something is wrong.
Any ideas what to check?
[Update]
After reading on Oracle Objects and SQLData mappings, I rewrote the code to use SQLData.
The Record class now implements SQLData and it's writeSQL() method looks like this:
#Override
public void writeSQL(SQLOutput stream) throws SQLException {
stream.writeString(owner_id);
stream.writeString(target_id);
stream.writeString(Objects.isNull(ip) ? "0" : ip); // weird, but as specified
stream.writeString(prefix);
stream.writeString(String.valueOf(port));
stream.writeString(description);
stream.writeString(cost_id);
}
Then at the start of the calling code, I've added:
con.getTypeMap().put("SCHEME_ADM.RECORD_REC", Record.class);
And instead of using createStruct(), the setObject() call now looks simply like this:
call.setObject("p_record_rec", t, Types.STRUCT)
But the result is the same - no errors and all the passed values are read as NULL. I've traced through the writeSQL() implementation and I can see that it is called and all values are passed correctly into the Oracle code. I've tried to use Types.JAVA_OBJECT in the setObject() call, and got an error: Invalid column type.
[Update 2]
Bordering on insane helplessness I've implemented the OracleData pattern:
public class Record implements SQLData, OracleData, OracleDataFactory {
...
#Override
public Object toJDBCObject(Connection conn) throws SQLException {
return conn.createStruct(getSQLTypeName(), new Object[] {
Objects.isNull(owner_id) ? "" : owner_id,
Objects.isNull(record_id) ? "" : record_id,
Objects.isNull(ip) ? "0" : ip,
Objects.isNull(prefix) ? "" : prefix,
String.valueOf(port),
Objects.isNull(description) ? "" : description,
Objects.isNull(cost_id) ? "" : cost_id
});
}
#Override
public OracleData create(Object jdbcValue, int sqltype) throws SQLException {
if (Objects.isNull(jdbcValue)) return null;
LinkedList<Object> attr = new LinkedList<>(Arrays.asList(((OracleStruct)jdbcValue).getAttributes()));
Record r = new Record();
r.setOwner_id(attr.removeFirst().toString());
r.setRecord_id(attr.removeFirst().toString());
r.setIp(attr.removeFirst().toString());
r.setPrefix(attr.removeFirst().toString());
r.setPort(Integer.parseInt(attr.removeFirst().toString()));
r.setDescription(attr.removeFirst().toString());
r.setCost_id(attr.removeFirst().toString());
return r;
}
public static OracleDataFactory getOracleDataFactory() {
return new Record();
}
Calling code:
...
// unwrap the Oracle object from C3P0 (standard JDBCv4 API)
OracleCallableStatement ops = call.unwrap(OracleCallableStatement.class);
// I'm not sure why I even need to do this - it looks exactly like
// the standard JDBC code
for (Records r : records) {
ops.registerOutParameter(1, Types.VARCHAR);
ops.setObject(2, t);
ops.execute();
ids.add(ops.getString(1));
}
...
And again, same result - no errors, a record is created in the table, with all provided values are null. I've traced through the code and the toJDBCObject() method is called correctly and does pass the values correctly in to createStruct().

Found the problem. Annoyingly, its about character encoding.
If in the toJDBCObject() implementation, I run getAttributes() on the created struct, the resulting Object[] array has all fields set as "???". Which is weird and looks like a character set transcoding failure (although it looks weird for that too - has three question marks for all fields regardless of value length, including empty string values).
According to Oracle's JDBC developer guide, "Globalization Support":
The basic Java Archive (JAR) file ojdbc7.jar, contains all the necessary classes to provide complete globalization support for:
Oracle character sets for CHAR, VARCHAR, LONGVARCHAR, or CLOB data that is not being retrieved or inserted as a data member of an Oracle object or collection type.
CHAR or VARCHAR data members of object and collection for the character sets US7ASCII, WE8DEC, WE8ISO8859P1, WE8MSWIN1252, and UTF8.
To use any other character sets in CHAR or VARCHAR data members of objects or collections, you must include orai18n.jar in the CLASSPATH environment variable:
ORACLE_HOME/jlib/orai18n.jar
And my setup was using the character set "WE8ISO8859P9" (I have no idea why, what it means, or even if it is selected by the client or the server - I just dumped the STRUCT object created by the OracleData API implementation and it was there somewhere).
So when Oracle says that it does not "provide complete globalization support", they mean "all character fields will be silently converted to NULL". Hmpph.
Anyway, adding orai18n.jar to the CLASSPATH indeed fixed the problem, and now records are added correctly to the database.

jOOQ - multiple bindings

I have a map of binding values:
final Map<String, Object> values = ...;
Before executing the query, I loop through the binding parameters and bind its value as following:
final ResultQuery<Record> q = ...;
for (final Param p : q.getParams().values()) {
if (p.getParamName() != null) {
q.bind(p.getParamName(), values.get(p.getParamName()));
}
}
When the same binding is used multiple times, this seems to fail:
final ResultQuery<Record> q = create.select().from(DSL.table("my_table"))
.where((DSL.field("identifier").eq(DSL.param("binding"))
.and(DSL.field("identifier").eq(DSL.param("binding")))));
... code above ...
create.fetch(q);
In the generated query, only one of the bindings is filled in. The other is null.
Is there an alternative approach? The same binding parameter can only be used once, or is this a bug?
(I know this query doesn't make much sense, it is only for demonstrating the problem.)
The issue was probably solved if we could do the following instead, but it is not possible since getParams() returns Param<?> and not Param<Object>:
for (final Param p : q.getParams().values()) {
if (p.getParamName() != null) {
p.bind(values.get(p.getParamName()));
}
}
update - The statement above is incorrect, since getParams() returns a Map<String, Param> and not Map<String, Collection<Param>>. It still woud be useful to bind a Param directly though.

This is a bug in jOOQ 3.5.2 (#4056).
Another workaround apart from the one you've found would be to make sure that both instances of "binding" are in fact the same, you should probably externalise the bind value:
Param<Object> binding = DSL.param("binding");
final ResultQuery<Record> q = create.select().from(DSL.table("my_table"))
.where((DSL.field("identifier").eq(binding)
.and(DSL.field("identifier").eq(binding))));
Now, you'll explicitly create a single bind value that is used two times in the AST.

Using a NamedParamterJdbcTemplate object to specify a mysql field and value to query on

I am currently in the process of learning the Java Spring Framework, and I am having difficulty understanding why the following query is failing to return any results from the database.
I am ultimately trying to create a where method in my OffersDAO class that allows my to query on a specific field, for a specific value.
public List<Offer> where(String field, String value){
MapSqlParameterSource params = new MapSqlParameterSource();
params.addValue("field", field);
params.addValue("value", value);
String sql = "select * from offers where :field = :value";
return jdbc.query(sql, params, new RowMapper<Offer>(){
public Offer mapRow(ResultSet rs, int arg1) throws SQLException {
Offer offer = new Offer();
offer.setId(rs.getInt("id"));
offer.setName(rs.getString("name"));
offer.setText(rs.getString("text"));
offer.setEmail(rs.getString("email"));
return offer;
}
});
}
I am able to successfully query the database for results when I specify the field explicitly, as follows:
String sql = "select * from offers where name = :value";
Obviously there is something wrong with specifying the field name dynamically. My guess is it is most likely due to the fact that the field key is being inserted as a mysql string (with ''), when in fact mysql expects a column name for the :field placeholder.
My questions are as follows:
Is there a way to accomplish what I am attempting to do above, using the jdbc NamedParameterJdbcTemplate class?
If I cannot accomplish the above, by what means can I?
Thank you
Edit: No exceptions are thrown. In the case when I am attempting to supply the column name, a empty result set is returned.

You can't specify the field name in a parameter - only the field value. Since you know the DB schema when you're writing the code, this shouldn't be much of a problem.

What about include all possible fields in the filter but restricting their usage by field name param. Like this:
select * from offers where
('name'=:field and name = :value)
OR
('field2'=:field and field2 = :value)
OR
('field3'=:field and field3 = :value)

I don't know how You can implement it with spring (I mean use variable column names) but I can suggest to use the following principle.
Keep your query like template:
String sql = "select * from offers where ##field = :value";
And every time before execution replace ##value parameter with the column You want.
And then You are gone.

Java create PreparedStatement with no SQL content

Is it possible to create a PreparedStatement in java without setting the initial SQL query?
Example code:
#Override
public List<AccountBean> search(AccountConstraint... c) {
if (c.length == 0) {
throw new IllegalArgumentException("dao.AccountDAO.search: c.length == 0");
}
try {
List<AccountBean> beans = new ArrayList<>();
for (AccountConstraint ac : c) {
PreparedStatement ps = connection.prepareStatement(null);
QueryBuilder queryBuilder = new QueryBuilder(ps, "SELECT * FROM accounts");
queryBuilder.add(ac.getAccountIdConstraint());
queryBuilder.add(ac.getUsernameConstraint());
queryBuilder.add(ac.getPasswordConstraint());
queryBuilder.add(ac.getEmailConstraint());
//INSERT QUERY INTO PS
ResultSet rs = ps.executeQuery();
while (rs.next()) {
beans.add(new AccountBean(rs));
}
}
return beans;
} catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
The trick is in QueryBuilder, this class is responsible for building parts of a query based on the initial SELECT part and then adds respective WHERE and AND clauses.
However to ensure that all data is safe, the actual arguments must also be put in the PreparedStatement, hence why it is being passed to the QueryBuilder.
Every QueryBuilder.add() adds some arguments into the PreparedStatement and appends a specific string to the end of the query.
I think some workarounds are possible, such as instead of giving a PreparedStatement to the QueryBuilder you would give a List<Object> and then you would write a custom function that puts them in the PreparedStatement later on.
But what are your thoughts, suggestions on this?
Regards.
Solution added
Few key changes first:
QueryBuilder now implements the Builder pattern properly.
QueryBuilder.add() accepts multiple Constraints at once.
AccountConstraint can give an array that gives all Constraints now.
#Override
public List<AccountBean> search(AccountConstraint... c) {
if (c.length == 0) {
throw new IllegalArgumentException("dao.AccountDAO.search: c.length == 0");
}
try {
List<AccountBean> beans = new ArrayList<>();
for (AccountConstraint ac : c) {
try (PreparedStatement ps = new QueryBuilder("SELECT * FROM accounts").add(ac.getConstraints()).build();ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
beans.add(new AccountBean(rs));
}
}
}
return beans;
} catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
ps. I get two statements in one try{ } because of the try-with-resources.

Preparing a statement means compiling it so you can efficiently execute it many times with different arguments. So, no it does not make sense to compile a query before it is defined.
As I understand, you want to use the Java compiler to assist you in dynamically defining the query. Why don't you just create the prepared statement in a compile() method, thus, as the result of your builder. Also, your code becomes more readable and more resembles a declarative query if you use the builder pattern such that each call to add() returns this. Then you can write your query like this:
PreparedStatement ps = new QueryBuilder()
.select("*")
.from("accounts")
.where()
.add(yourConstraint())
...
.compile();
However, you must create the prepared statement before the loop. Otherwise, if you keep a reference to the builder and call compile() in your loop you will get a new prepared statement on every call. So you won't get the benefit of reusing a precompiled query. In the loop you only assign values to the variables in your prepared statement.

You can't modify the prepared statement via the API after you crate it. You can't create it without an SQL statement either.
Why not create the query separately and then bind the parameters? You can use a Map to hold the parameter placeholders and their values so they can be set to the prepared statement.
Although I'd just use the Spring's JDBC templates to get the same thing done more quickly.

How to improve your SQL query builder
If you look at how popular query builders like jOOQ and others do it, the idea is that you separate your concerns more thoroughly. You should have:
An expression tree representation of your SQL statement (and ideally that doesn't directly operate on strings)
A way to construct that expression tree conveniently, e.g. by using a DSL
Some sort of execution lifecycle management that generates the SQL string, prepares the statement, binds the variables, etc.
Or in code (jOOQ example, but this could also apply to your own query builder):
Result<?> result =
// This constructs the expression tree through the jOOQ DSL
ctx.selectFrom(ACCOUNTS)
.where(ac.getAccountIdConstraint())
.and(ac.getUsernameConstraint())
.and(ac.getPasswordConstraint())
.and(ac.getEmailConstraint())
// This internally creates a PreparedStatement, binds variables, executes it, and maps results
.fetch();
Of course, your AccountConstraint.getXYZConstraint() methods would not return SQL string snippets, but again expression tree elements. In the case of jOOQ, this would be a Condition
(Disclaimer: I work for the vendor of jOOQ)
How to improve your SQL performance
I've noticed that you run N queries for N AccountConstraint values, and you mix the results in a way that it doesn't matter which AccountConstraint value produced which AccountBean. I strongly suggest you move that loop into the generated SQL query, as you're going to get much faster results on pretty much every database. I've blogged about this here.

How to set list of parameters on prepared statement? [duplicate]

This question already has answers here:
PreparedStatement IN clause alternatives?
(33 answers)
Closed 3 years ago.
i have a list of names e.g.:
List<String> names = ...
names.add('charles');
...
and a statement:
PreparedStatement stmt =
conn.prepareStatement('select * from person where name in ( ? )');
how to do the following:
stmt.setParameterList(1,names);
Is there a workaround? can someone explain why this method is missing?
using: java, postgresql, jdbc3

This question is very old, but nobody has suggested using setArray
This answer might help https://stackoverflow.com/a/10240302/573057

There's no clean way to do this simply by setting a list on the PreparedStatement that I know of.
Write code that constructs the SQL statement (or better replaces a single ? or similar token) with the appropriate number of questions marks (the same number as in your list) and then iterate over your list setting the parameter for each.

this method is missing due to type erasure the parameter type of the List is lost at runtime. Therefore the need to add several methods arires: setIntParameters, setLongParameters, setObjectParameters, etc

For postgres 9 I have used this approach:
jdbcTemplate.query(getEmployeeReport(), new PreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps) throws SQLException {
ps.setTimestamp(1, new java.sql.Timestamp(from.getTime()));
ps.setTimestamp(2, new java.sql.Timestamp(to.getTime()));
StringBuilder ids = new StringBuilder();
for (int i = 0; i < branchIds.length; i++) {
ids.append(branchIds[i]);
if (i < branchIds.length - 1) {
ids.append(",");
}
}
// third param is inside IN clause
// Use Types.OTHER avoid type check while executing query
ps.setObject(3, ids.toString(), **Types.OTHER**);
}
}, new PersonalReportMapper());

In case the questions' meaning is to set several params in a single call...
Because the type validation is already defined in a higher level, I think the only need is for setObject(...).
Thus, a utility method can be used:
public static void addParams(PreparedStatement preparedStatement, Object... params) throws SQLException {
for (int i = 0; i < params.length; i++) {
Object param = params[i];
preparedStatement.setObject(i+1, param);
}
}
Usage:
SqlUtils.addParams(preparedStatement, 1, '2', 3d);
Feel free converting this to a Java 8 lambda :)

I was reviewing code this morning and one of my colleagues had a different approach, just pass the parameter using setString("name1','name2','name3").
Note: I skipped the single quote at the beginning and end because these are going to be added by the setString.

After examining various solutions in different forums and not finding a good solution, I feel the below hack I came up with, is the easiest to follow and code. Note however that this doesn't use prepared query but gets the work done anyway:
Example: Suppose you have a list of parameters to pass in the 'IN' clause. Just put a dummy String inside the 'IN' clause, say, "PARAM" do denote the list of parameters that will be coming in the place of this dummy String.
select * from TABLE_A where ATTR IN (PARAM);
You can collect all the parameters into a single String variable in your Java code. This can be done as follows:
String param1 = "X";
String param2 = "Y";
String param1 = param1.append(",").append(param2);
You can append all your parameters separated by commas into a single String variable, 'param1', in our case.
After collecting all the parameters into a single String you can just replace the dummy text in your query, i.e., "PARAM" in this case, with the parameter String, i.e., param1. Here is what you need to do:
String query = query.replaceFirst("PARAM",param1); where we have the value of query as
query = "select * from TABLE_A where ATTR IN (PARAM)";
You can now execute your query using the executeQuery() method. Just make sure that you don't have the word "PARAM" in your query anywhere. You can use a combination of special characters and alphabets instead of the word "PARAM" in order to make sure that there is no possibility of such a word coming in the query. Hope you got the solution.

Other method :
public void setValues(PreparedStatement ps) throws SQLException {
// first param inside IN clause with myList values
ps.setObject(1 , myList.toArray(), 2003); // 2003=array in java.sql.Types
}