H2: "data conversion error" in WHERE clause - java

I am using the H2 database (v1.3.170) for JUnit testing in my project. Our production environment uses Oracle DB so we have to do lot conversions of the DDLs exported from Oracle to make them working with H2. One of the issues we are facing right now is following:
select * from table_country c where c.code > 0 AND 'USA' = c.name(+)
This query works fine in Oracle, but fails in H2 with below stack trace:
org.h2.jdbc.JdbcSQLException: Data conversion error converting "USA"; SQL statement:
select * from table_country c where c.code>0 AND 'USA' = c.name(+) [22018-170]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:329)
at org.h2.message.DbException.get(DbException.java:158)
at org.h2.value.Value.convertTo(Value.java:852)
at org.h2.value.Value.getBoolean(Value.java:373)
at org.h2.expression.ConditionAndOr.optimize(ConditionAndOr.java:188)
at org.h2.command.dml.Select.prepare(Select.java:802)
at org.h2.command.Parser.prepareCommand(Parser.java:218)
at org.h2.engine.Session.prepareLocal(Session.java:414)
at org.h2.server.TcpServerThread.process(TcpServerThread.java:253)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:149)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException
at java.math.BigDecimal.<init>(Unknown Source)
at java.math.BigDecimal.<init>(Unknown Source)
at org.h2.value.Value.convertTo(Value.java:801)
... 8 more
I tried removing the (+) - it worked:
select * from table_country c where c.code > 0 AND 'USA' = c.name
But as I said, I don't want to change this in production (want the Oracle => H2 conversion to be transparent).
Please suggest what could be the issue and a better way to solve this?

(+) syntax for outer joins is deprecated in Oracle. Stop using it.
In your case an "outer join" doesn't really makes sense anyway because you are not joining.
You probably want something like this (which works reliably across all DBMS)
select *
from table_country c
where c.code > 0
and (c.name = 'USA' or c.name is null);
A final word: using a different DBMS for testing and production makes your tests worthless. There are too many subtle differences between the DBMS that you simply can't cover when using different DBMS.

Related

Jooq Multiset - SQL shows the word multiset thorwing syntax error - Postgres

I am using Jooq version 3.16.6 with Java 11 and Spring Boot 2.6.6 and (PostgreSQL) 14.1
The issue i am having is with multiset , the non multiset query using the old join method works fine . However when using multiset and examining the sql i can see the word multiset actually in the sql query hence it thorws a syntax error
var result =
dslContext.select(USERS.FIRST_NAME,USER_LEVELS.USER_LEVEL_NAME)
.from(USERS)
.join(USER_LEVELS)
.on(USERS.USER_LEVEL_ID.eq(USER_LEVELS.USER_LEVEL_ID))
.where(USERS.USERNAME.eq("email#email.com"))
.fetch();
The above is the standard join method works fine
The below is the multiset version
var result =
dslContext.select(USERS.FIRST_NAME,
multiset(select(USER_LEVELS.USER_LEVEL_NAME)
.from(USER_LEVELS)
.where(USER_LEVELS.USER_LEVEL_ID.eq(USERS.USER_LEVEL_ID)))
)
.from(USERS)
.where(USERS.USERNAME.eq("email#email.com"))
.getSQL();
I am using the getSQL to see the sql it is running which comes out as below
select "public"."users"."first_name",
multiset(select "public"."user_levels"."user_level_name"
from "public"."user_levels" where "public"."user_levels"."user_level_id" = "public"."users"."user_level_id") from "public"."users" where "public"."users"."username" = ?
The issue is obviously the multiset being in the sql which causes a syntax error.
I am at a loss as to why this is , unless its some setting for my postgres (which i don't think)
Any ideas ?
You probably haven't configured your SQLDialect correctly, e.g.
spring.jooq.sql-dialect=Postgres
See also: Spring Boot JOOQ sql dialect not picked up from application.properties

General syntax error raised from CONNECT BY query in Informix when using quoted table identifiers

When running the following query on an Informix database, the database reports a general syntax error (without any indication with respect to what causes the problem). The same query runs perfectly on CUBRID or Oracle databases, both of which also support the CONNECT BY syntax:
select
lower(connect_by_root "t_directory"."name"),
connect_by_isleaf,
connect_by_iscycle,
substr(
sys_connect_by_path(lower("t_directory"."name"), '/'),
2) "dir"
from "t_directory"
start with "t_directory"."parent_id" is null
connect by nocycle prior "t_directory"."id" = "t_directory"."parent_id"
order siblings by lower("t_directory"."name") asc
The database I'm using is a Developer Edition of Informix 12.10 on Windows. I'm running the query from a JDBC driver with the following connection URL (to allow for quoted table identifiers):
jdbc:informix-sqli://localhost:9092/test:INFORMIXSERVER=ol_informix;DELIMIDENT=y
The exact issue here is the fact that prior doesn't accept quoted table identifiers, although quoted column identifiers seem to be fine. This query runs perfectly well:
select
lower(connect_by_root "t_directory"."name"),
connect_by_isleaf,
connect_by_iscycle,
substr(
sys_connect_by_path(lower("t_directory"."name"), '/'),
2) "dir"
from "t_directory"
start with "t_directory"."parent_id" is null
connect by nocycle prior t_directory."id" = "t_directory"."parent_id"
order siblings by lower("t_directory"."name") asc
... with the difference being:
-- Bad:
connect by nocycle prior "t_directory"."id" = "t_directory"."parent_id"
-- Good:
connect by nocycle prior t_directory."id" = "t_directory"."parent_id"

Firebird SQL Query with # (at) sign - How to run the query in JDBC (Jaybird)?

I have an application that uses Firebird. The application performs a long list of queries e.g. each time you list your items. I want to take out these queries, and run them in my own Java application (so I can manipulate the list, display it, and so on.)
The problem is... there is a debug option in the application where you can see what kind of queries does your application run. Some of the original queries got # signs. If I run a query with an # in it, I get an error. If I take out that part of the query, everything runs and works "as expected". No errors, like a charm.
Detailed error message:
Error code: -104
Token unknown - line 8, column 32
We use IntelliJ IDEA which automatically applies escape characters when needed.
Such a part from the original query:
SELECT TBL4487."Id" "database.id",
TBL4487."Code" "database.code",
TBL4487."Name" "database.name",
TBL4487."Barcode" "database.barcode",
TBL4488."Name" "Datagroup",
TBL4489."Name" "Mey",
(SELECT FIRST 1 TBL4494."Price" / (CASE
WHEN (TBL4487."GrossPrices" = #Param4495) THEN 1
ELSE (TBL4492."Rate" + 100) / 100
END) "productprice.price"
FROM "ProductPrice" TBL4494
WHERE (TBL4494."Product" = TBL4487."Id") AND (TBL4494."PriceCategory" = #Param4497) AND (TBL4494."ValidFrom" <= #Param4498) AND (TBL4494."Currency" = #Param4499) AND (TBL4494."QuantityUnit" = TBL4487."QuantityUnit")
ORDER BY TBL4494."ValidFrom" DESC) "xyz",
(SELECT FIRST 1 TBL4500."Price" / (CASE
WHEN (TBL4487."GrossPrices" = #Param4501) THEN 1
ELSE (TBL4492."Rate" + 100) / 100
The question is.. how could I run this query? How do I replace the # symbol?
You can't run this query directly with Jaybird. These #ParamXXXX seem to be placeholders in the query for parameters. However Firebird nor Jaybird supports this type of placeholders (they only support ? as placeholder in DSQL).
To execute this with Jaybird, you will need to replace each instance of the #ParamXXXX either with a ? and set the right value for each placeholder in a PreparedStatement, or with the actual value in the query text itself.
The Firebird .NET provider does support #....-style placeholders (it translates them to Firebird-style ? placeholders), so you could try to use C#/.NET instead if you don't want to do replacing yourself.
Full disclosure: I am the developer of Jaybird

JPA (Hibernate) Native Query for Prepared Statement SLOW

Having strange performance issue using Hibernate 3.3.2GA behind JPA (and the rest of the Hibernate packages included in JBoss 5.)
I'm using Native Query, and assembling SQL into a prepared statement.
EntityManager em = getEntityManager(MY_DS);
final Query query = em.createNativeQuery(fullSql, entity.getClass());
The SQL has a lot of joins, but is actually very basic, with a single parameter. Like:
SELECT field1, field2, field3 FROM entity left join entity2 on... left join entity3 on
WHERE stringId like ?
and the query runs in under a second on MSSQL Studio.
If I add
query.setParameter(0, "ABC123%");
The query will pause for 9 seconds
2012-01-20 14:36:21 - TRACE: - AbstractBatcher.getPreparedStatement:(484) | preparing statement
2012-01-20 14:36:21 - TRACE: - StringType.nullSafeSet:(133) | binding 'ABC123%' to parameter: 1
2012-01-20 14:36:30 - DEBUG: - AbstractBatcher.logOpenResults:(382) | about to open ResultSet (open ResultSets: 0, globally: 0)
However, if I just replace the "?" with the value (making it not a Prepared Statement, but just a straight SQL query.
fullSql = fullSql.replace("?", "'ABC123%'");
the query will complete in less that a second.
I would really prefer to us a Prepared Statement (the input for the parameters is being extracted from user data) to prevent injection attacks.
Tracing down the slow point in the code, I arrived deep within the jtds-1.2.2 package. The offending line seems to be SharedSocket line 841 "getIn().readFully(hdrBuf);" Nothing really obvious there though...
private byte[] readPacket(byte buffer[])
throws IOException {
//
// Read rest of header
try {
getIn().readFully(hdrBuf);
} catch (EOFException e) {
throw new IOException("DB server closed connection.");
}
Arrived to through this stack...
at net.sourceforge.jtds.jdbc.SharedSocket.readPacket(SharedSocket.java:841)
at net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket(SharedSocket.java:722)
at net.sourceforge.jtds.jdbc.ResponseStream.getPacket(ResponseStream.java:466)
at net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java:103)
at net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java:88)
at net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3928)
at net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1045)
at net.sourceforge.jtds.jdbc.TdsCore.microsoftPrepare(TdsCore.java:1178)
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.prepareSQL(ConnectionJDBC2.java:657)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeQuery(JtdsPreparedStatement.java:776)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1808)
at org.hibernate.loader.Loader.doQuery(Loader.java:697)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:259)
at org.hibernate.loader.Loader.doList(Loader.java:2228)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2125)
at org.hibernate.loader.Loader.list(Loader.java:2120)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:312)
at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1722)
at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165)
at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:175)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:67)
I'll leave this question and answer out here in case anyone has the same issue in the future.
The issue is in the way the JTDS drivers send the parameter strings to MSSQL. Apparently Java will attempt to send the parameters Unicode by default, and MSSQL will translate it to Ascii. Why that takes 9 seconds, I do not know.
Lot's of references to this out there, but nothing that helped my till I was able to isolate that it was an issue with the driver to MSSQL connection.
This link was helpful:
[http://server.pramati.com/blog/2010/06/02/perfissues-jdbcdrivers-mssqlserver/]
This is the string using the Microsoft driver.
jdbc:sqlserver://localhost\SQLEXPRESS;
DatabaseName=TESTDB;
sendStringParametersAsUnicode=false
You just need to get the sendStringParametersAsUnicode=false passed to your driver URL setup and you are good.
Check the query plans that SQL server is producing. Prepared statements can be especially problematic.
Let me explain...
If you do this:
SELECT field1, field2, field3 FROM entity left join entity2 on... left join entity3 on
WHERE stringId like 'ABC123%';
and you have an index on "stringId" SQL server knows it can use it.
However if you do this:
SELECT field1, field2, field3 FROM entity left join entity2 on... left join entity3 on
WHERE stringId like ?;
SQL server doesn't know it can use the index when it creates the prepared statement (as you could fill in the parameter with '%ABC123' instead of 'ABC123%') and thus may choose a completely different query plan.
And another answer for people potentially using Oracle with a similar Unicode problem...
Check to make sure someone hasn't set the property oracle.jdbc.defaultNChar=true
This is sometimes done to resolve unicode problems but it means all columns are treated as nvarchars. If you have an index on a varchar column, it won't be used because oracle has to use a function to convert the character encoding.

PreparedStatement and Oracle 10g bug

I have a big but INTERMITTENT problem with a bug in Oracle 10g when we call some SQL within a Java web application. We can't quickly patch or upgrade to 11g - which seems to be the first 'stupid' oracle support response. There is a work around, but I am having trouble doing this within PreparedStatements within my Java code.
The actual error is:
ORA-00600: internal error code, arguments: [kcblasm_1]
The bug is: Oracle Bug 12419392
The work around is running
alter session set "_hash_join_enabled" = FALSE;
before we run our bug-inducing SQL. However, traditionally a PreparedStatement takes in one single piece of SQL:
PreparedStatement stmt = con.prepareSelect("sql statement2");
Is it possible to have one PreparedStatement call that looks like this:
PreparedStatement stmt = con.prepareSelect("sql statement1; sql statement2;");
Or is this possible just by running a series of sequential PreparedStatements one after the other?
Not the best time to be getting this with Xmas looming and reduced support etc. etc., so I really hope someone can help. Thanks.
Edit: #jonearles asked for the code, so here it is, if it's on any use. Probably very specific to our project, but someone might spot the glaring bug-inducing issue:
SELECT DISTINCT qm.validator_id,
qm.QM_ID,
u.EMAIL,
qm.creation_dt,
qm.emailed,
qm.valid,
qm.resolved,
qm.new_obs_id,
o.*,
nests.*,
s.*,
l.*,
latc.TENKM
FROM query_man qm,
obs o,
obs_aux_aon nests,
sub s,
location l,
l_atlas_tetrad_coverage latc,
users u
WHERE qm.OBS_ID = o.OBS_ID
AND o.SUB_ID = s.SUB_ID
AND u.user_id = qm.user_id
AND o.obs_id = nests.obs_id(+)
AND s.LOC_ID = l.LOC_ID
AND latc.ATLAS_REGION = 'NKNE'
AND (LENGTH (l.gridref) = 6
AND (SUBSTR(l.gridref,1,3)
|| SUBSTR(l.gridref,5,1)) = latc.TENKM
OR LENGTH (l.gridref) = 4
AND l.gridref = latc.TENKM)
AND qm.RESOLVED IS NULL
ORDER BY latc.tenkm,
l.tetrad
OK. The answer to my primary question is NO, you can't create a PreparedStatement like so:
PreparedStatement stmt = con.prepareSelect("sql statement1; sql statement2;");
Running individual statements to alter session temporarily for one bit of SQL did work, but agreed seems awful and also unacceptably slowed response. Options seem to be patch or upgrade, or look into the no_use_hash hint (which I think will be slow too). Will look at code.

Categories