I'm currently using Jooq for a project, but I need a way to ignore duplicate keys on insert.
I've got an array of objects I want to write into a table but if they already exist determined by a composite unique index on START_TS and EVENT_TYPE I want the insert to silently fail.
My Code looks something like this:
InsertValuesStep<MyRecord> query = fac.insertInto(MY_REC,
MY_REC.START_TS,
MY_REC.STOP_TS,
MY_REC.EVENT_DATA,
MY_REC.EVENT_TYPE,
MY_REC.PUBLISHED_TS,
MY_REC.MY_ID
);
for(int i=0;i<recs.length;i++)
{
MyClass evt = recs[i];
query.values(
new java.sql.Date(evt.startTS.getTime()),
(evt.stopTS == null) ? null : new java.sql.Date(evt.stopTS.getTime()),
evt.eventData,
evt.type.name(),
date,
id)
}
query.execute();
A solution like this would be ideal: https://stackoverflow.com/a/4920619/416338
I figure I need to add something like:
.onDuplicateKeyUpdate().set(MY_REC.EVENT_TYPE,MY_REC.EVENT_TYPE);
But whatever I add it still seems to throw an error on duplicates.
Support for MySQL's INSERT IGNORE INTO syntax is on the roadmap for jOOQ 2.3.0. This had been discussed recently on the jOOQ user group. This syntax will be simulated in all other SQL dialects that support the SQL MERGE statement.
In the mean time, as a workaround, you could try to insert one record at a time
Related
Using Jooq, I am trying to fetch from a table by id first, if no matches found, then fetch by handle again.
And I want all fields of the returned rows, not just one.
Field<?> firstMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.ID.eq(id))
.asfield(); // This is wrong, because it supports only one field, but above we selected Tables.MY_TABLE.fields(), which is plural.
Field<?> secondMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.HANDLE.eq(handle))
.asfield(); // Same as above.
dslContext.select(DSL.coalesce(firstMatch, secondMatch))
.fetchInto(MyClass.class);
Due to the mistake mentioned above in the code, the following error occurs:
Can only use single-column ResultProviderQuery as a field
I am wondering how to make firstMatch and secondMatch two lists of fields, instead of two fields?
I tried
Field<?>[] secondMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.HANDLE.eq(handle))
.fields();
but the following error occurred in the line containing DSL.coalesce
Type interface org.jooq.Field is not supported in dialect DEFAULT
Thanks in advance!
This sounds much more like something you'd do with a simple OR?
dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.eq(id))
// The ne(id) part might not be required...
.or(MY_TABLE.ID.ne(id).and(MY_TABLE.HANDLE.eq(handle))
.fetchInto(MyClass.class);
If the two result sets should be completely exclusive, then you can do this:
dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.eq(id))
.or(MY_TABLE.HANDLE.eq(handle).and(notExists(
selectFrom(MY_TABLE).where(MY_TABLE.ID.eq(id))
)))
.fetchInto(MyClass.class);
If on your database product, a query using OR doesn't perform well, you can write an equivalent query with UNION ALL, which might perform better.
So i have a project where we use springBoot and PostgreSQL 10 with PostGis and hibernate.spatial for spatial queries. Everything works fine so far.
A new requirement is to find entities, which start-end dates overlap the start-end dates of the query in any possible way (ranges might be enclosing, start-overlap, inbetween, end-overlap).
In PostgreSQL theres the Overlaps operator that seems a pretty good fit for the job.
When trying to use it within my JPA-Query for an Entity "Sth" like this..
select sth from Sth sth where 1=1 and (sth.start, sth.end) overlaps (:begin, :end)
// set begin and end params..
i get one of..
antlr.NoViableAltException: unexpected token: overlaps
antlr.NoViableAltException: unexpected AST node: (
org.postgresql.util.PSQLException: FEHLER: rt_raster_from_wkb: wkb size (5) < min size (61)
Is it possible to use overlaps for dates with JPA without writing a native query?
So seems like there are three things you need to do to make it work.
Its probably not possible to use overlaps as an operator, but luckily it seems it can also be used as a function: overlaps(start1, end1, start2, end2)
Overlaps is not mapped by any of the hibernate-core PostgreSQL[NN]Dialects. But it is mapped by hibernate-spatial PostgisPG[NN]Dialects, where it maps to the st_overlaps-function for spatial overlapping. So you need to use your own custom Dialect that registers the overlaps function with an alias like so:
public class PostgisDialect extends org.hibernate.spatial.dialect.postgis.PostgisPG95Dialect {
public PostgisDialect() {
super();
registerFunction("dateoverlaps", new StandardSQLFunction("overlaps", StandardBasicTypes.BOOLEAN));
}
}
and specify it as your spring.jpa.properties.hibernate.dialect (or spring.jpa.database-platform 🤷♂️).
Your JPA query must include a = true like this:
select sth from Sth sth where 1=1 and dateoverlaps(sth.start, sth.end, :begin, :end) = true
You might addtionally enhance this by using the coalesce function to handle null-values, i.e.
select sth from Sth sth where 1=1 and dateoverlaps(coalesce(sth.start, sth.plannedStart), coalesce(sth.end, '3999-01-01'), :begin, :end) = true
which uses plannedStart when start is null and a long-in-the-future-date when end is null / open.
Any idea on how I could define the following jOOQ query with less repetition?
I am using jOOQ 3.11.4.
db.insertInto(ACCOUNT,
ACCOUNT.ACCOUNT_ID,
ACCOUNT.EMAIL,
ACCOUNT.FIRST_NAME,
ACCOUNT.LAST_NAME,
ACCOUNT.IS_ADMIN,
ACCOUNT.PASSWORD)
.values(account.accountId,
account.email,
account.firstName,
account.lastName,
account.isAdmin,
account.password)
.onConflict(ACCOUNT.ACCOUNT_ID)
.doUpdate()
.set(ACCOUNT.EMAIL, account.email)
.set(ACCOUNT.FIRST_NAME, account.firstName)
.set(ACCOUNT.LAST_NAME, account.lastName)
.set(ACCOUNT.IS_ADMIN, account.isAdmin)
.set(ACCOUNT.PASSWORD, account.password)
.returning(
ACCOUNT.ACCOUNT_ID,
ACCOUNT.EMAIL,
ACCOUNT.FIRST_NAME,
ACCOUNT.LAST_NAME,
ACCOUNT.IS_ADMIN,
ACCOUNT.PASSWORD
)
.fetchOne()
(I turns out my question is mostly code, and StackOverflow does not let me post it as is, without adding more details, which I do not think is necessary for my question, but nevertheless, they want me to post some more text, which I am doing right now by typing this message, and I hope you did not have to read to the end.)
Since you're passing all the columns to the insert statement, you might write this instead:
// Create an AccountRecord that contains your POJO data
Record rec = db.newRecord(ACCOUNT);
rec.from(account);
// Don't pass the columns to the insert statement explicitly
db.insertInto(ACCOUNT)
// But pass the record to the set method. It will use all the changed values
.set(rec)
// Use the MySQL syntax, which can be emulated on PostgreSQL using ON CONFLICT
.onDuplicateKeyUpdate()
// But pass the record to the set method again
.set(rec)
// Don't specify any columns to the returning clause. It will take all the ACCOUNT columns
.returning()
.fetchOne();
In MySQL, if you specify ON DUPLICATE KEY UPDATE and a row is inserted that would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an UPDATE of the old row is performed. For example, if column a is declared as UNIQUE and contains the value 1, the following two statements have identical effect:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
UPDATE table SET c=c+1 WHERE a=1;
I don't believe I've come across anything of the like in T-SQL. Does SQL Server offer anything comparable to MySQL's ON DUPLICATE KEY UPDATE?
I was surprised that none of the answers on this page contained an example of an actual query, so here you go:
A more complex example of inserting data and then handling duplicate
MERGE
INTO MyBigDB.dbo.METER_DATA WITH (HOLDLOCK) AS target
USING (SELECT
77748 AS rtu_id
,'12B096876' AS meter_id
,56112 AS meter_reading
,'20150602 00:20:11' AS time_local) AS source
(rtu_id, meter_id, meter_reading, time_local)
ON (target.rtu_id = source.rtu_id
AND target.time_local = source.time_local)
WHEN MATCHED
THEN UPDATE
SET meter_id = '12B096876'
,meter_reading = 56112
WHEN NOT MATCHED
THEN INSERT (rtu_id, meter_id, meter_reading, time_local)
VALUES (77748, '12B096876', 56112, '20150602 00:20:11');
There's no DUPLICATE KEY UPDATE equivalent, but MERGE and WHEN MATCHED might work for you
Inserting, Updating, and Deleting Data by Using MERGE
You can try the other way around. It does the same thing more or less.
UPDATE tablename
SET field1 = 'Test1',
field2 = 'Test2'
WHERE id = 1
IF ##ROWCOUNT = 0
INSERT INTO tablename
(id,
field1,
field2)
VALUES (1,
'Test1',
'Test2')
SQL Server 2008 has this feature, as part of TSQL.
See documentation on MERGE statement here - http://msdn.microsoft.com/en-us/library/bb510625.aspx
SQL server 2000 onwards has a concept of instead of triggers, which can accomplish the wanted functionality - although there will be a nasty trigger hiding behind the scenes.
Check the section "Insert or update?"
http://msdn.microsoft.com/en-us/library/aa224818(SQL.80).aspx
Using the GeoTools WFS-T plugin, I have created a new row, and after a commit, I have a FeatureId whos .getId() returns an ugly string that looks something like this:
newmy_database:my_table.9223372036854775807
Aside from the fact that the word "new" at the beginning of "my_database" is a surprise, the number in no way reflects the primary key of the new row (which in this case is "23"). Fair enough, I thought this may be some internal numbering system. However, now I want a foreign key in another table to get the primary key of the new row in this one, and I'm not sure how to get the value from this FID. Some places suggest that you can use an FID in a query like this:
Filter filter = filterFactory.id(Collections.singleton(fid));
Query query = new Query(tableName, filter);
SimpleFeatureCollection features = simpleFeatureSource.getFeatures(query);
But this fails at parsing the FID, at the underscore of all places! That underscore was there when the row was created (I had to pass "my_database:my_table" as the table to add the row to).
I'm sure that either there is something wrong with the id, or I'm using it incorrectly somehow. Can anyone shed any light?
It appears as if a couple things are going wrong - and perhaps a bug report is needed.
The FeatureId with "new" at the beginning is a temporary id; that should be replaced with the real result once commit has been called.
There are a number of way to be aware of this:
1) You can listen for a BatchFeatureEvent; this offers the information on "temp id" -> "wfs id"
2) Internally this information is parsed from the Transaction Result returned from your WFS. The result is saved in the WFSTransactionState for you to access. This was before BatchFeatureEvent was invented.
Transaction transaction = new transaction("insert");
try {
SimpleFeatureStore featureStore =
(SimpleFeatureStore) wfs.getFeatureSource( typeName );
featureStore.setTransaction( transaction );
featureStore.addFeatures( DataUtilities.collection( feature ) );
transaction.commit();
// get the final feature id
WFSTransactionState wfsts = (WFSTransactionState) transaction.getState(wfs);
// In this example there is only one fid. Get it.
String result = wfsts.getFids( typeName )[0];
}
finally {
transaction.close();
}
I have updated the documentation with the above example:
http://docs.geotools.org/latest/userguide/library/data/wfs.html