I'm using JOOQ with plain/raw SQL, so that means i'm not using any code generation or the fluid DSL thingy.
The following code works:
Connection connection = ...;
DSLContext context = DSL.using(connection, ...);
String sql = "select * from mytable t where (t.id = ?)";
String id = ...; //
Result<Record> result = context.fetch(sql, id);
Now let's say i have a query with multiple parameters like this:
String sql = "select * from mytable t where (t.id = ?) " +
"and (t.is_active = ?) and (t.total > ?)";
How do i use a named parameter with these types of queries?
I'm thinking something like :
String sql = "select * from mytable t where (t.id = :id) " +
"and (t.is_active = :is_active) and (t.total > :total)";
ResultQuery<Record> rq = context.resultQuery(sql);
rq.getParam("id").setValue(...);
rq.getParam("is_active").setValue(...);
rq.getParam("total").setValue(...);
Result<Record> result = rq.fetch();
But the above code doesn't work (for obvious reasons). Thanks in advance.
jOOQ currently doesn't support executing SQL with named parameters. You can use jOOQ to render named parameters if you're executing the query with another API, such as Spring JDBC. For more information, consider the manual:
http://www.jooq.org/doc/latest/manual/sql-building/bind-values/named-parameters
But the plain SQL templating API allows for re-using templates, e.g.
String sql = "select * "
+ "from mytable t "
+ "where t.id = {0} or (t.id != {0} and t.name = {1})";
ResultQuery<Record> q = ctx.resultQuery(sql, val(1), val("A"));
This way, you can at least re-use values several times.
Because Lukas said that this feature is not available, I thought I'll code a 'good enough' solution.
The idea is that the variable name like :name has to be replaced with the {0} at all places and the rest is done by JOOQ. I thought this is the easiest way of doing it. (Replacing variables with their proper form, like handling data types is definitely a lot of work.)
I merited some ideas from this other StackOverflow answer and then created this gist in Kotlin (it would have been too long in Java otherwise).
The current gist looks like this now:
import org.jooq.DSLContext
import org.jooq.Record
import org.jooq.ResultQuery
import org.jooq.impl.DSL
object SqlJooqBindVariableOrganizer {
data class Processed(
val statement: String,
val originalStatement: String,
val variables: List<Pair<String, Any>>,
) {
fun toResultQuery(context: DSLContext): ResultQuery<Record> {
return context.resultQuery(
statement,
*variables.map { DSL.`val`(it.second) }.toTypedArray(),
)
}
}
private fun extractBindVariableLocations(
statement: String,
): Map<String, List<IntRange>> {
// https://stackoverflow.com/a/20644736/4420543
// https://gist.github.com/ruseel/e10bd3fee3c2b165044317f5378c7446
// not sure about this regex, I haven't used colon inside string to test it out
return Regex("(?<!')(:[\\w]*)(?!')")
.findAll(statement)
.map { result ->
val variableName = result.value.substringAfter(":")
val range = result.range
variableName to range
}
.groupBy(
{ it.first },
{ it.second }
)
}
fun createStatement(
statement: String,
vararg variables: Pair<String, Any>,
): Processed {
return createStatement(statement, variables.toList())
}
fun createStatement(
statement: String,
variables: List<Pair<String, Any>>,
): Processed {
val locations = extractBindVariableLocations(statement)
val notProvidedKeys = locations.keys.subtract(variables.map { it.first })
if (notProvidedKeys.isNotEmpty()) {
throw RuntimeException("Some variables are not provided:\n"
+ notProvidedKeys.joinToString()
)
}
val relevantVariables = variables
// there may be more variables provided, so filter this
.filter { it.first in locations.keys }
// these locations should have the same order as the variables
// so it is important to know the proper order of the indices
val variableNameToIndex = relevantVariables
.mapIndexed { index, variable -> variable.first to index }
.associateBy({ it.first }, { it.second })
val variableNameReplacements = locations
.flatMap { (variableName, ranges) ->
ranges.map { range -> variableName to range }
}
// the replacements have to be done in a reversed order,
// as the replaced string is not equal length
.sortedByDescending { it.second.first }
// replace :name with {0}
val processedStatement = variableNameReplacements
.fold(statement) { statementSoFar, (variableName, range) ->
// index has to exist, we just checked it
val index = variableNameToIndex[variableName]!!
statementSoFar.replaceRange(range, "{$index}")
}
return Processed(
statement = processedStatement,
originalStatement = statement,
variables = relevantVariables,
)
}
}
Related
I have a List<String> of categories and for each category, I want to add them to my WHERE clause by combining with AND operator like: SELECT question_id FROM question WHERE category = categ1 AND category = categ2 AND category = ...
Since the size of the categories list is changing, I cannot do something like this:
String sql = "SELECT question_id FROM question WHERE category = ? AND category = ?";
jdbcTemplate.query(sql, stringMapper, "categ1", "categ2");
How can I achieve what I want?
Either check if JDBC Template from Spring handle that for you using a syntax which could be something like (from the doc, I don't think it does)
SELECT question_id FROM question WHERE category in (?...)
Or write your own query with the problems that may arise:
List<Object> parameters = new ArrayList<>(categories.size());
StringBuilder sb = new StringBuilde("SELECT question_id FROM question WHERE 1=1");
if (!categories.isEmpty()) {
if (categories.size() == 1) {
sb.append(" and category = ?");
} else {
sb.append(" and category in ");
sb.append(categories.stream()
.map(ignored -> "?")
.collect(joining(", ", "(", ")")));
sb.append(")");
}
parameters.addAll(categories);
}
Object[] paramArray = parameters.toArray();
jdbcTemplate.query(sb.toString(), stringMapper, paramArray);
Notes:
some security/quality tool may report SQL issues because you are writing a dynamic SQL.
Oracle put a limit on 1000 elements per IN. You would have to partition categories per group of 1000 (or less).
I used a stream() in a more or less strange fashion in order to generate the "?". If you use commons-lang3, you can replace it by "(" + StringUtils.repeat("?", ", ", categories.size()) + ")" (the example in the javadoc was probably done with this kind of use).
if you only have category as single criteria, you may probably remove the 1=1 as well as the and.
I believe this may work for you:
// The SQL Query
String sql = "SELECT question_id FROM question";
// Create the WHERE clause based on the number of items in List...
StringBuilder whereClause = new StringBuilder(" WHERE ");
StringBuilder ps = new StringBuilder("");
for (int i = 0; i < categories.size(); i++) {
if (!ps.toString().isEmpty()) {
ps.append(" AND ");
}
ps.append("category = ?");
}
whereClause.append(ps.toString()).append(";");
//Append the WHERE clause string to the SQL query string
sql = sql + whereClause.toString();
//System.out.println(sql);
/* Convert the categories List to an Object[] Array so as to
pass in as varArgs to the jdbcTemplate.query() method. */
Object[] psArgs = categories.toArray(new Object[categories.size()]);
jdbcTemplate.query(sql, stringMapper, psArgs);
In some E2E tests I'm faced with a problem. Let's say, I have the following JPQL query:
Query query = entityManager.createQuery(
" select d from Document d left join d.audit da " +
" where " +
" (cast(:startDate as java.time.ZonedDateTime)) is null " +
" or truncate_for_minutes(da.dateCreate, 'UTC') >= " +
" truncate_for_minutes(:startDate, 'UTC')")
.setParameter("startDate", ZonedDateTime.now());
In the query string I use named parameter startDate which can be null. The query above works. But if I pass null, the following exception is thrown:
Caused by: org.postgresql.util.PSQLException:
ERROR: cannot cast type bytea to timestamp without time zone
Without type casting the following exception is thrown:
Caused by: org.postgresql.util.PSQLException:
ERROR: could not determine data type of parameter $1
Without check for null the following exception is thrown:
Caused by: org.postgresql.util.PSQLException:
ERROR: function pg_catalog.timezone(unknown, bytea) does not exist
No function matches the given name and argument types.
You might need to add explicit type casts.
I use this query in Spring Data repository by using #Query string. Function truncate_for_minute(...) - is just a small customization for the PostgreSQL function date_trunc(...).
I know that I can implement custom repository and build query string dynamically, but why I can't check for null ZonedDateTime in JPQL string? Maybe there is a way to do it?
My environment:
Java 11
PostgreSQL 11.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 8.3.0) 8.3.0, 64-bit
Hibernate 5.3.7 Final
Another solution is to use Criteria API and build the query dynamically.
#Repository
#RequiredArgsConstructor
public class CustomDocumentRepositoryImpl implements CustomDocumentRegistry {
private final EntityManager em;
#Override
public Page<Document> findDocumentsForExpertByFilter(SearchDocumentCriteria criteria,
Pageable pageable) {
final String AUDIT_TABLE = "...";
final String USER_TABLE = "...";
final String ID_FIELD = "id";
final String FIRST_NAME_FIELD = "...";
final String LAST_NAME_FIELD = "...";
final String MIDDLE_NAME_FIELD = "...";
final String WORKSTATION_FIELD = "...";
final String DATE_CREATE_FIELD = "...";
final String LIKE_MASK = "%%%s%%";
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Document> query = cb.createQuery(Document.class);
Root<Document> root = query.from(Document.class);
Path<ZonedDateTime> dateCreatePath =
root.get(AUDIT_TABLE).get(DATE_CREATE_FIELD);
Path<String> lastNamePath =
root.get(AUDIT_TABLE).get(USER_TABLE).get(LAST_NAME_FIELD);
Path<String> firstNamePath =
root.get(AUDIT_TABLE).get(USER_TABLE).get(FIRST_NAME_FIELD);
Path<String> middleNamePath =
root.get(AUDIT_TABLE).get(USER_TABLE).get(MIDDLE_NAME_FIELD);
root.fetch(AUDIT_TABLE, JoinType.LEFT)
.fetch(USER_TABLE, JoinType.LEFT);
Predicate documentIdsPredicate;
List<Long> documentIds = criteria.getIds();
if (isNull(documentIds) || documentIds.isEmpty()) {
documentIdsPredicate = cb.isNotNull(root.get(ID_FIELD));
} else {
documentIdsPredicate = root.get(ID_FIELD).in(criteria.getIds());
}
Predicate startDatePredicate;
ZonedDateTime startDate = criteria.getStartDate();
if (isNull(startDate)) {
startDatePredicate = cb.isNotNull(dateCreatePath);
} else {
startDatePredicate = cb.greaterThanOrEqualTo(dateCreatePath, startDate);
}
Predicate endDatePredicate;
ZonedDateTime endDate = criteria.getEndDate();
if (isNull(endDate)) {
endDatePredicate = cb.isNotNull(dateCreatePath);
} else {
endDatePredicate = cb.lessThanOrEqualTo(dateCreatePath, endDate);
}
Predicate lastNamePredicate = cb.like(cb.upper(lastNamePath),
format(LIKE_MASK, criteria.getLastName().toUpperCase()));
Predicate firstNamePredicate = cb.like(cb.upper(firstNamePath),
format(LIKE_MASK, criteria.getFirstName().toUpperCase()));
Predicate middleNamePredicate = cb.like(cb.upper(middleNamePath),
format(LIKE_MASK, criteria.getMiddleName().toUpperCase()));
Predicate fullNamePredicate =
cb.and(lastNamePredicate, firstNamePredicate, middleNamePredicate);
Predicate compositePredicate = cb.and(
fullNamePredicate,
documentIdsPredicate,
startDatePredicate,
endDatePredicate
);
query.where(compositePredicate);
Query limitedQuery = em.createQuery(query
.orderBy(cb.desc(root.get(AUDIT_TABLE).get(DATE_CREATE_FIELD))))
.setFirstResult(nonNull(criteria.getSize()) ?
criteria.getPage() * criteria.getSize() :
criteria.getPage());
if (nonNull(criteria.getSize())) {
limitedQuery.setMaxResults(criteria.getSize());
}
List<Document> documents = limitedQuery.getResultList();
return new PageImpl<>(documents, pageable, criteria.getSize());
}
}
Generates the following SQL:
select
document0_.id as id1_3_0_,
user1_.id as id1_13_1_,
document0_.created_dt as created_2_3_0_,
document0_.updated_dt as updated_3_3_0_,
document0_.created_user_id as created_6_3_0_,
document0_.updated_user_id as updated_7_3_0_,
document0_.name as name4_3_0_,
user1_.first_name as first_na2_13_1_,
user1_.last_name as last_nam3_13_1_,
user1_.middle_name as middle_n5_13_1_
from
some_scheme.document document0_
left outer join
some_scheme.s_user user1_
on document0_.created_user_id=user1_.id cross
join
some_scheme.s_user user2_
where
document0_.created_user_id=user2_.id
and (
upper(user2_.last_name) like '%LASTNAME%'
)
and (
upper(user2_.first_name) like '%FIRSTNAME%'
)
and (
upper(user2_.middle_name) like '%MIDDLENAME%'
)
and (
document0_.id in (
2 , 1
)
)
and document0_.created_dt>=...
and document0_.created_dt<=...
order by
document0_.created_dt desc limit 10;
In my case, the problem was as follows. I registered customization of SQL function date_trunc:
public class CustomSqlFunction implements MetadataBuilderContributor {
#Override
public void contribute(MetadataBuilder metadataBuilder) {
metadataBuilder.applySqlFunction(
"truncate_for_minutes",
new SQLFunctionTemplate(
StandardBasicTypes.TIMESTAMP,
"date_trunc('minute', (?1 AT TIME ZONE ?2))"
)
);
}
}
If change the StandardBasicTypes.TIMESTAMP to ZonedDateTime.INSTANCE and pass ZonedDateTime in one parameter, then the comparison in the JPQL query does not cause errors:
public class CustomSqlFunction implements MetadataBuilderContributor {
#Override
public void contribute(MetadataBuilder metadataBuilder) {
metadataBuilder.applySqlFunction(
"truncate_for_minutes",
new SQLFunctionTemplate(
ZonedDateTimeType.INSTANCE,
"date_trunc('minute', ?1)"
)
);
}
}
I am trying to retrieve the data from Table TABLE_A, Table Table_A contain Column BB_NUM.
I need to write the expression in JPA BooleanExpression,
Needed to generate below SQL Query from JPA predicate logic:
**Select * from TABLE_A where BB_NUM like 'GA22'OR BB_NUM like 'GA33';**
And this GA22 GA33 are comes dynamically, So this where condition is dynamically added.
Below code have write to dynamically create where condition in predicate expression, but this code is not working,
boolean firstTime = true;
str = "GA22GA33";
BooleanExpression ent = null;
for (String array : str.split("GA")) {
if (StringUtils.isNotBlank(array)) {
if (firstTime) {
ent = qTABLE_A.BB_NUM.containsIgnoreCase("GA" + array.trim());
firstTime = false;
} else {
ent.or(qTABLE_A.BB_NUM.containsIgnoreCase("GA" + array.trim()));
}
}
}
expressions.add(ent);
Above code is works like SQL,
**Select * from TABLE_A where BB_NUM like 'GA22';**
need to work
**Select * from TABLE_A where BB_NUM like 'GA22' or BB_NUM like 'GA33';**
Try it like:
else {
ent = ent.or(qTABLE_A.BB_NUM.containsIgnoreCase("GA" + array.trim()));
}
Also you can substitute if (firstTime) with if (ent == null)
I am querying a table in Postgres to see if a row with primary key values match those in a given Map exists. If there are no results, I want to insert a new row. I can do this just fine when I go the standard JDBC route of creating a prepared statement and explicitly setting all the column values, but I want a more generic approach with the help of Groovy's Sql class.
This is my insert statement (the primary key columns are the first three):
insert into MY_TABLE
(contract_month, contract_year, publish_date, open, high, low)
values
(10, 2015, 2015-09-15 00:00:00.0, 46.65, 46.9, 45.98)
The issue is that I'm getting a NullPointerException from the Postgres driver as soon as I insert a row:
Exception in thread "main" java.lang.NullPointerException
at org.postgresql.core.v3.SimpleParameterList.setNull(SimpleParameterList.java:142)
at org.postgresql.jdbc2.AbstractJdbc2Statement.setNull(AbstractJdbc2Statement.java:1259)
at org.postgresql.jdbc3.AbstractJdbc3Statement.setNull(AbstractJdbc3Statement.java:1499)
at org.postgresql.jdbc4.AbstractJdbc4Statement.setNull(AbstractJdbc4Statement.java:85)
at org.postgresql.jdbc2.AbstractJdbc2Statement.setObject(AbstractJdbc2Statement.java:2092)
at org.postgresql.jdbc3g.AbstractJdbc3gStatement.setObject(AbstractJdbc3gStatement.java:38)
at org.postgresql.jdbc4.AbstractJdbc4Statement.setObject(AbstractJdbc4Statement.java:48)
Digging into it, I find that an object called ProtoConnection is null in the org.postgresql.core.v3.SimpleParameterList class. What am I doing wrong?
This is the wrapping code which creates the Sql object:
private Main() {
final def headers = new HashMap<Integer, String>()
final def record = new HashMap<String, Object>()
Sql.withInstance(/* my db */) { final Sql sql ->
sql.withBatch {
new URL(/* my url */).withReader {
it.readLines().each {
if (headers.isEmpty()) {
setHeaders it, headers
}
else {
processRecord it, headers, record
storeRecord sql, record
}
}
}
}
}
}
And this is the database logic. The error occurs at the very last line:
def storeRecord = { final Sql sql, final Map<String, Object> record ->
final def connection = sql.connection
final def metadata = connection.metaData
final def columns = metadata.getColumns(null, null, MY_TABLE, null).with {
final def result = []
while (it.next()) {
result << it.getString(4)
}
result
}
final def primaryKeys = metadata.getPrimaryKeys(null, null, MY_TABLE).with {
final def result = []
while (it.next()) {
result << it.getString(4)
}
result
}
final def whereClause = primaryKeys.collect { final String pk ->
"$pk = ?.$pk"
}
final def select = """
select ${columns.join(', ')}
from MY_TABLE
where ${whereClause.join(' and ')}
"""
final def row = sql.firstRow(select, record)
if (row) {
println "Updating"
// Not implemented yet
}
else {
println "Inserting record: $record"
final def insert = """
insert into MY_TABLE
(${columns.findAll { null != record[it] }.join(", ")})
values
(${columns.findAll { null != record[it] }.collect { record[it] }.join(", ")})
"""
println insert
sql.executeInsert(insert, record)
}
There were a few things I was doing wrong with my code. First of all, I wasn't converting the string objects in the record map into the appropriate JDBC types. This explains why I was able to insert records via the standard prepared statement route, as I originally did the conversion there, and overlooked it when transitioning to Groovy's API.
After fixing this, I then got an error that the hstore extension was not supported. This was simply solved by executing create extension hstore in Postgres.
Finally, I received an index out of bounds error. This was due to a silly bug in my code where I was setting the values in my insert statement, rather than letting the API fill in the parameters. You can see this fix in the final line of the insert statement now:
final def insert = """
insert into MY_TABLE
(${columns.join(', ')})
values
(${columns.collect { "?.$it" }.join(', ')})
"""
sql.executeInsert(insert, record)
How I can make one dynamic query in Spring MVC with all my all my parameters from URL?
For exemple:/example?parameter1=test¶meter2=apple;fruit¶meter3=Park
For that link I want to have a query like that: SELECT * FROM news WHERE parameter1 = test AND parameter2 = apple AND fruits AND parameter3 = Park
In in the same time if I have a URL like that /example?parameter1=test¶meter2=apple
I don't want to create a new query in some one like SELECT * FROM news WHERE parameter1 = test AND parameter2 = apple;
More exactly, how can I create a dynamic query how change automatic in function of parameters from URL?
Its not good idea to pass dynamic data to query which will surely cause you SQL injection attacks.
You can use BeanPropRowMapper (http://www.mkyong.com/spring/spring-jdbctemplate-querying-examples/) and prepared statement kind of approach (Named parameters http://www.studytrails.com/frameworks/spring/spring-jdbc-template-named-params.jsp) in Spring JDBC to achieve similar but typed query. ( Your Java Bean getter and setter are automatically called, query is updated from that)
Don't write dynamic query ( keep it strict typed)
Check this if you can use it..
public StringBuilder prepareQuery(String tblName, String url)
{
StringBuilder query = new StringBuilder("Select * from " + tblName + " Where ");
String[] params = url.split("&");
for (int i = 0; i < params.length; i++ )
{
String str = params[i];
System.out.println(str.contains(";"));
if (i == 0)
{
query.append(str.replaceAll(";", " AND "));
}
else
{
query.append(" AND " + str.replaceAll(";", " AND "));
}
}
return query;
}