We Keep Coding

Hibernate paging query returns no records with DB2/400 dialect

I am using Vaadin Flow together with Spring-data-jpa and Hibernate with DB2/400 dialect. I am trying to use paging queries as my dataset could be very large. I have Hibernate logging on so that I can see the statements that Hibernate is executing. It works for the first page as the Hibernate query just asks for the first 50 records. However for the second page it asks for 100 records and filters out the first 50 but the query does not return any results. A slightly simplified version of the Hibernate generated query is:
select * from (
select inner2_.*, rownumber() over(order by order of inner2_) as rownumber_ from (
select * from flxalll1 flxalll1x0_ where upper(flxalll1x0_.aoukey) like upper('%te%') fetch first 100 rows only
) as inner2_
) as inner1_ where rownumber_ > 50 order by rownumber_;
I have run this myself using the IBMi Run SQL Script tool and no results are returned. However, if I just do the inner two selects:
select inner2_.*, rownumber() over(order by order of inner2_) as rownumber_ from (
select * from flxalll1 flxalll1x0_ where upper(flxalll1x0_.aoukey) like upper('%te%') fetch first 100 rows only
) as inner2_ ;
I get the expects list of result, though of course all of them and not just the last 50. I have done some more experimentation and discovered that (unsurprisingly) this works:
select * from (
select * from flxalll1
);
in that it lists all the records, but this:
select * from (
select * from (
select * from flxalll1
)
);
produces no records.
Obviously there is no sense in that but I'm wondering if there is a problem with DB2/400 in that it won't do a select with two nested sub-selects, or something like that, and is that the reason why my original query does not return any records?
Solving the problem could be tricky but for now at least I am just trying to work out where the problem lies.

I have now solved my problem by not using the (IBMi) logical as mentioned in my comment above but by allowing spring data jpa to do the UNION. I created an abstract super class for my 10 tables and changed their Java class declarations to extend it.
I am indebted to this post and to the answer by Patrice Blanchardie:
Union tables with spring data jpa

jOOQ - how to use .whereNotExists() with conditional update?

I have a following jOOQ query, originally composed with the help of Lukas Eder in this question.
create.insertInto(DATA,DATA.TICKER,DATA.OPEN,DATA.HIGH,DATA.LOW,DATA.CLOSE,DATA.DATE)
.select(
select(
val(dailyData.getTicker()),
val(dailyData.getOpen()),
val(dailyData.getHigh()),
val(dailyData.getLow()),
val(dailyData.getClose()),
val(dailyData.getDate())
)
.whereNotExists(
selectOne()
.from(DATA)
.where(DATA.DATE.eq(dailyData.getDate()))
)
).execute();
This query works properly. In addition, I would like to modify to accomplish the following feat, but I am not certain it is actually doable. In simple english:
"Insert the row if a row with the same 'date' column doesn't already exist in the table. If it exists AND 'realtime close' column is true, update the 'close', otherwise do nothing."
The first part is already covered by the existing query, but the second part with if...update... is not and that's what I need help with.

In plain PostgreSQL, you would write this query as follows:
INSERT INTO data (ticker, open, high, low, close, date)
VALUES (:ticker, :open, :high, :low, :close, :date)
ON CONFLICT (date)
DO UPDATE SET close = false WHERE close
This translates to the following jOOQ query:
DSL.using(configuration)
.insertInto(DATA)
.columns(
DATA.TICKER,
DATA.OPEN,
DATA.HIGH,
DATA.LOW,
DATA.CLOSE,
DATA.DATE)
.values(
dailyData.getTicker(),
dailyData.getOpen(),
dailyData.getHigh(),
dailyData.getLow(),
dailyData.getClose(),
dailyData.getDate())
.onConflict()
.doUpdate()
.set(DATA.CLOSE, inline(false))
.where(DATA.CLOSE)
.execute();

DAO: diffrence between InMemory implementation and Database Implementation

I'm confusing with implementation of CRUD methods for DAODatabase (for Oracle 11 xe).
The problem is that the "U"-method (update) in case of storing in generally to a Map collection inserts a new element or renews it (key-value data like ID:AbstractBusinessObject) in a Map collection. And you don't care about it, when you write something like myHashMap.add(element). This method (update) is widely used in project's business logic.
Obviously, in case of using Oracle I must care about both inserting and renewing of existing elements. But I'm stucked to choose the way how to implement it:
There is no intrinsic function for so-called UPSERT in Oracle (at least in xe11g r2 version). However, I can emulate necessary function by SQL-query like this:
INSERT INTO mytable (id1, t1)
SELECT 11, 'x1' FROM DUAL
WHERE NOT EXISTS (SELECT id1 FROM mytble WHERE id1 = 11);
UPDATE mytable SET t1 = 'x1' WHERE id1 = 11;
(src:http://stackoverflow.com/a/21310345/2938167)
By using this kind of query (first - insert, second - update) I presume that the data mostly will be inserted not updated (at least it will be rather rare). (May it be not optimal for concurrency?).
Ok, it is possible. But at this point I'm confusing to decide:
-- should I write an SQL function (with approriate arguments of course) for this and call it via Java
-- or should I simply handle a serie of queries for preparedStatements and do them via .executeUpdate/.executeQuery? Should I handle the whole UPSERT SQL code for one preparedStatment or split it into several SQL-queries and prepared statements inside one method's body? (I'm using Tomcat's pool of connections and I pass a connection instance via static method getConnection() to each method implementation in DAODatabase) ?
Is there another possibility to solve the UPSERT quest?

The equivalent to your UPSERT statement would seem to be to use MERGE:
MERGE INTO mytable d
USING ( SELECT 11 AS id, 'x1' AS t1 FROM DUAL ) s
ON ( d.id = s.id )
WHEN NOT MATCHED THEN
INSERT ( d.id, d.t1 ) VALUES ( s.id, s.t1 )
WHEN MATCHED THEN
UPDATE SET d.t1 = s.t1;
You could also use (or wrap in a procedure):
DECLARE
p_id MYTABLE.ID%TYPE := 11;
p_t1 MYTABLE.T1%TYPE := 'x1';
BEGIN
UPDATE mytable
SET t1 = p_t1
WHERE id = p_id;
IF SQL%ROWCOUNT = 0 THEN
INSERT INTO mytable ( id, t1 ) VALUES ( p_id, p_t1 );
END IF;
END;
/
However, when you are handling a CRUD request - if you are doing a Create action then it should be represented by an INSERT (and if something already exists then you ought to throw the equivalent of the HTTP status code 400 Bad Request or 409 Conflict, as appropriate) and if you are doing an Update action it should be represented by an UPDATE (and if nothing is there to update then return the equivalent error to 404 Not Found.
So, while MERGE fits your description I don't think it is representative of a RESTful action as you ought to be separating the actions to their appropriate end-points rather than combining then into a joint action.

JPA Criteria "locate" ignoring "from" parameter

I have a table SASDOSSIERS with one column apentrytext which has one value 6.8.3 Dossiers "A" , SAS, Fürsorgeleistungen an Auslandschweizer which I need to select after the last ,: Fürsorgeleistungen an Auslandschweizer. If I perform the following query in sql, the select is correct:
SELECT distinct TRIM(SUBSTR(apentrytext,INSTR(apentrytext,',',-1)+1)) from SASDOSSIERS;
Whereas, if I try to use the equivalent in jpa critera
cq.multiselect(
cb.trim(
cb.substring(
sasdossier.get(Sasdossier_.apentrytext),
cb.sum(
cb.locate(sasdossier.get(Sasdossier_.apentrytext), cb.literal(","), cb.literal(Integer.valueOf(-1))),
1
)
)
)
).distinct(true);
It ignores totally what I put in the from parameter (I have tried with many different values, eg:-2, -100, 1, 2, 100) and it seems to ignore what I put there, the result is always the same. Moreover, the query generated by hibernate shows that the argument is, indeed, ignored:
SELECT DISTINCT trim(BOTH FROM substr(sasdossier0_.apentrytext, instr(sasdossier0_.apentrytext, ?) + 1)) AS col_0_0_
FROM ACCESS_DB.sasdossiers sasdossier0_
WHERE sasdossier0_.apentrytext IS NOT NULL
It seems like a legit bug, but I cuouldn't find anybody that has the same error before, so, am I doing something wrong? should I report the error?
Thank you.

Postgresql - ERROR : tuple concurrently updated

The following query is performed concurrently by two threads logged in with two different users:
WITH raw_stat AS (
SELECT
host(client_addr) as client_addr,
pid ,
usename
FROM
pg_stat_activity
WHERE
usename = current_user
)
INSERT INTO my_stat(id, client_addr, pid, usename)
SELECT
nextval('mystat_sequence'), t.client_addr, t.pid, t.usename
FROM (
SELECT
client_addr, pid, usename
FROM
raw_stat s
WHERE
NOT EXISTS (
SELECT
NULL
FROM
my_stat u
WHERE
current_date = u.creation
AND
s.pid = u.pid
AND
s.client_addr = u.client_addr
AND
s.usename = u.usename
)
) t;
From time to time, I get the following error:
tuple concurrently updated
I can't figure out what throw this error and why this error is thrown. Can you shed a light ?
Here is the sql definition of the table mystat.
mystats.sql
CREATE TABLE mystat
(
id bigint NOT NULL,
creation date NOT NULL DEFAULT current_date,
client_addr text NOT NULL,
pid integer NOT NULL,
usename name NOT NULL,
CONSTRAINT mystat_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
);

This isn't really an answer - so much as maybe helping someone else who stumbles on this error.
In my case, I was trying to be fancy and encapsulate the creation of all my functions within one function.
Something like
CREATE OR REPLACE FUNCTION main_func()
BEGIN
CREATE OR REPLACE FUNCTION child_func1()
BEGIN
END
CREATE OR REPLACE FUNCTION child_func1()
BEGIN
END
main func stuff...
END
For whatever reason, I could call this function no problem from inside pgAdmin. And I could call it as much as I wanted from Java -> MyBatis.
However, as soon as I started calling the function from two different threads, I got the error from the OP: ERROR : tuple concurrently updated
The fix was, simply take those child functions out of the main function, and maintain them separately.
Looking back on it, it's a pretty bad idea to be creating functions as a result of calling a function. However, the idea was to 'encapsulate' all the functionality together.
Hope this helps someone.

If the pg hackers threads are anything to go by, the error kicks in when the same row is concurrently being updated by competing transactions. In your case it's likely due to the not exists() clause, which can potentially yield true and two competing inserts of the same tuple.
To work around it, you'd want to either use more robust locking (e.g. a predicate lock), serializable isolation level, or place the needed logic in an upsert statement (can be done using a function with an exception block).

From the docs(https://www.postgresql.org/docs/current/functions-sequence.html) from Postgres, Because sequences are non-transactional, changes made by setval are not undone if the transaction rolls back.
It means that you need to update provide thread safety by yourself using transaction so running the query inside transaction might fix your problem.

I manage to solve my problem by changing my query to this one:
INSERT INTO my_stat(id, client_addr, pid, usename)
SELECT
nextval('mystat_sequence'), client_addr, pid, usename
FROM (
SELECT
host(client_addr) as client_addr,
pid ,
usename
FROM
pg_stat_activity
WHERE
usename = current_user
) s
WHERE
NOT EXISTS (
SELECT
NULL
FROM
my_stat u
WHERE
current_date = u.creation
AND
s.pid = u.pid
AND
s.client_addr = u.client_addr
AND
s.usename = u.usename
);
I think something happened under the hood right from the Postgresql internals but I can't figure out what ...