I use tomcat for my server side with Mysql Server 5.5.
i use Spring framework for the database connectivity.
I would like to be able to insert a row to a table using simpleJdbcInsert. if the insert fails because of a duplicate i want it to replace the duplicated row.
is there a way to do that simpleJdbcInsert or should I just use jdbcTemplate and create my on query with "ON DUPLICATE" statement?
thanks
logically, duplication occurs when there is a double primary key.
so, if you are using the traditional JDBC or even Hibernate, then you should check if the same primary key value already exists, before you insert the new one.
but if the primary key is not set yet, or will be set by the DBMS, there will be different problem.
in JDBC, again you need to do the old style manual checking by querying before inserting the new one, but
in Hibernate, you just need to update it. Hibernate will create the new if there is no duplicate, but will replace if there is a duplicate,.
in JDBC, again you need to do the old style manual checking by querying before inserting the new one, but
That's incorrect - because there is a time gap between the checking of existing row and the actual insert, so somebody in parallel thread can insert the row with the same key simultaneously and you'll obtain the DuplicateKeyException (if we talk about spring). So, you have to handle this exception or just use the honest SQL insert into ... on duplicate key ....
Related
I'm not sure exactly where the error is coming from, unfortunately, but I have a guess and would like to know the best way to solve it.
Problem
Suppose we have the following table in the database
ID
Field A
Field B
Field C
1
A
C
Something
2
B
C
Something else
And we have two unique indexes on the table
Unique-Index1 (ID)
Unique-Index2 (FieldA, FieldB)
Now I am loading both entities
Session session = ...();
Transaction tx = session.beginTransaction();
TestTable dataset1 = (TestTable) session.get(TestTable.class, 1);
TestTable dataset2 = (TestTable) session.get(TestTable.class, 2);
And now I want to do something like this
update testtable set fielda = 'B' where id = 1;
update testtable set fielda = 'A' where id = 2;
So at the end the unique key is not violated, but after the first statement, the unique index is violated.
In my JAVA application it looks like this
dataset1.setFieldA("B");
dataset2.setFieldA("A");
session.saveOrUpdate(dataset1);
session.saveOrUpdate(dataset2);
tx.commit();
After executing the application I get the following exception
Could not execute JDBC batch update
Unfortunately, the error is not really meaningful. Also, I don't get any information whether it might be a duplicate or not. But if I delete the unique index, it works. So my guess is that it is because of that.
Used frameworks / systems
Java 17 SE application, using Hibernate 3.2 (very old version) with the legacy mapping XML files (so still without annotations). The database is an IBM Informix database.
The database model, as well as the indexes are not generated by Java, but by regular SQL scripts.
I can't change anything about the versions of Hibernate or the database either, unfortunately. Also I cannot influence how the index was created. This all happens outside the application.
Idea
The only idea I had was to first change all records that need to be changed to fictitious values and then set the correct values again. But that would mean that two update statements are triggered per record, right?
Something like this:
dataset1.setFieldA("XXX");
dataset2.setFieldA("YYY");
session.saveOrUpdate(dataset1);
session.saveOrUpdate(dataset2);
dataset1.setFieldA("B");
dataset2.setFieldA("A");
session.saveOrUpdate(dataset1);
session.saveOrUpdate(dataset2);
tx.commit();
However, I am not even sure if I need to commit the transaction. Maybe a flush or something similar is enough, but the solution is not really nice. I can kind of understand the problem, but I would also have thought that this would be legitimate within a transaction then - only at the end of the transaction the constraints have to be correct.
Many greetings and thanks for your help,
Hauke
You have two options. Either you configure the unique constraint to be "deferrable" and also mark it as "initially deferred" so that the constraint is only enforced at transaction commit time, or you delete and re-insert the entries.
I would suggest you to use the first option if your database supports this. You didn't specify which database you are using, but PostgreSQL supports it. You'd only have to run alter table test_table alter constraint your_unique_constraint deferrable initially deferred.
I am using Hibernate with MSSQL server writing the software that integrates with an existing database. There is an instead of insert trigger on the table that I need to insert into and it messes up ##Identity, which means on Hibernate's save I can't get the id of inserted row. I can't control the trigger (can't modify it). I saw this question, but it involves procedures, which my trigger does not have, so I thought my question is different enough. I can't post the whole trigger, but hopefully I can post enough to get the point across:
CREATE TRIGGER TrigName ON TableName
INSTEAD OF INSERT
AS
SET XACT_ABORT ON
BEGIN TRANSACTION
-- several DECLARE, SET statements
-- a couple of inserts into other tables for business logic
-- plain T-SQL statements without procedures or functions
...
-- this is the actual insert that i need to perform
-- to be honest, I don't quite understand how INSERTED table
-- was filled with all necessary columns by this point, but for now
-- I accept it as is (I am no SQL pro...)
INSERT INTO ClientTable (<columns>)
SELECT <same columns> from INSERTED
-- a couple of UPDATE queries to unrelated tables
...
COMMIT TRANSACTION;
I was wondering if there is a reliable way to get the id of the row being inserted? One solution I thought of and tried to make is to install an on insert trigger on the same table that writes the newly inserted row into a new table I added to the db. I'd use that table as a queue. After transaction commit in Hibernate I could go into that table and run a select with the info I just inserted (I still have access to it from the same method scope), and I can get the id and finally remove that row. This is a bulky solution, but best I can come up with so far.
Would really appreciate some help. I can't modify existing triggers and procedures, but I can add something to the db if it absolutely does not affect existing logic (like that new table and a on insert trigger).
To sum up: I need to find a way to get the ID of the row I just inserted with Hibernate's save call. Because of that instead of insert trigger, hibernate always returns identity=0. I need to find a way to get that ID because I need to do the insert in a few other tables during one transaction.
I think I found an answer for my question. To reply to #SeanLange's comment: I can't actually edit insert code - it's done by another application and inquiry to change that will take too long (or won't happen - it's a legacy application). What I did is insert another trigger on insert on the same table. Since I know the order of operations in the existing instead of insert trigger I can see that the last insert operation will be in the table I want so that means my on insert trigger will fire right after that. In the scope of that trigger I have access to inserted table out of which I pull out the id.
CREATE TRIGGER Client_OnInsert ON myClientTable
FOR INSERT
AS
BEGIN
DECLARE #ID int;
SET #ID = (select ClientID from inserted);
INSERT INTO ModClient (modClientId)
OUTPUT #ID
VALUES (#ID);
END
GO
Then in Hibernate (since I can't use save() anymore), I use a NativeQuery to do this insert. I set parameters and run the list() method of NativeQuery, which returns a List where the first and only argument is the id I want.
This is a bulky way, I know. If there is anything that's really bad that will stand out to people - please let me know. I would really appreciate some feedback on this. However, I wanted to post this answer as a potential answer that worked so far, but it does not mean it's very good. For this solution to work I did have to create another small table ModClient, which I will have to use as a temp id storage for this exact purpose.
Is there a way to tell Hibernate to first check if the current primary key generated by a Table Generator is usable or outdated?
I have an application which uses hibernate to create new entries in several tables in my database, but sometimes these generated values are outdated and already used. This happens because this database is used by quite a few applications and scripts, and some of these use the "select MAX(ID)+1"-Keygeneration"strategy". It is not really an option to change all other components to use the table generator (although it would solve the problem), so I have to make sure that the values I get from the table generator are really usable.
Is there any way to tell Hibernate to check the validity of the generated values before it tries to insert a new record into the database (and throw a ConstraintViolationException)?
Or, alternatively, is there a way to manually update the generator tables before hibernate uses them to generate new Ids?
The obvious way would be to run a native query like UPDATE pk_generator SET value=(SELECT MAX(ID)+1 from members) WHERE column='members'
When you save a object with saveOrUpdate() the objects id field will get updated with the auto generated id if it was a create operation. So that it will never conflict with id which was already generated and used.
We have a stateless ejb which persists some data in an object oriented database. Unfortunately, today our persistence object does not have a unique key due to some unknown reason and altering the PO is also not possible today.
So we decided to synchronize the code. Then we check if there is an object already persisted with the name(what we consider should be unique). Then we decide to persist or not.
Later we realized that the code is deployed on a cluster which has three jboss instances.
Can anyone please suggest an idea which does not allow to persist objects with the same name.
If you have a single database behind the JBoss cluster you can just apply a unique contraint to the column for example (I am assuming its an SQL database):
ALTER TABLE your_table ADD CONSTRAINT unique_name UNIQUE (column_name);
Then in the application code you may want to catch the SQL exception and let the user know they need to try again or whatever.
Update:
If you cannot alter the DB schema then you can achieve the same result by performing a SELECT query before insert to check for duplicate entries, if you are worried about 2 inserts happening at the same time you can look at applying a WRITE_LOCK to the row in question
I have a webservice in java that receives a list of information to be inserted or updated in a database. I don't know which one is to insert or update.
Which one is the best approach to abtain better performance results:
Iterate over the list(a object list, with the table pk on it), try to insert the entry on Database. If the insert failed, run a update
Try to load the entry from database. if the results retrieved update, if not insert the entry.
another option? tell me about it :)
In first calls, i believe that most of the entries will be new bd entries, but there will be a saturation point that most of the entries will be to update.
I'm talking about a DB table that could reach over 100 million entries in a mature form.
What will be your approach? Performance is my most important goal.
If your database supports MERGE, I would have thought that was most efficient (and treats all the data as a single set).
See:
http://www.oracle.com/technology/products/oracle9i/daily/Aug24.html
https://web.archive.org/web/1/http://blogs.techrepublic%2ecom%2ecom/datacenter/?p=194
If performance is your goal then first get rid of the word iterate from your vocabulary! learn to do things in sets.
If you need to update or insert, always do the update first. Otherwise it is easy to find yourself updating the record you just inserted by accident. If you are doing this it helps to have an identifier you can look at to see if the record exists. If the identifier exists, then do the update otherwise do the insert.
The important thing is to understand the balance or ratio between the number of inserts versus the number of updates on the list you receive. IMHO you should implement an abstract strategy that says "persists this on database". Then create concrete strategies that (for example):
checks for primary key, if zero records are found does the insert, else updates
Does the update and, if fails, does the insert.
others
And then pull the strategy to use (the class fully qualified name for example) from a configuration file. This way you can switch from one strategy to another easily. If it is feasible, could be depending on your domain, you can put an heuristic that selects the best strategy based on the input entities on the set.
MySQL supports this:
INSERT INTO foo
SET bar='baz', howmanybars=1
ON DUPLICATE KEY UPDATE howmanybars=howmanybars+1
Option 2 is not going to be the most efficient. The database will already be making this check for you when you do the actual insert or update in order to enforce the primary key. By making this check yourself you are incurring the overhead of a table lookup twice as well as an extra round trip from your Java code. Choose which case is the most likely and code optimistically.
Expanding on option 1, you can use a stored procedure to handle the insert/update. This example with PostgreSQL syntax assumes the insert is the normal case.
CREATE FUNCTION insert_or_update(_id INTEGER, _col1 INTEGER) RETURNS void
AS $$
BEGIN
INSERT INTO
my_table (id, col1)
SELECT
_id, _col1;
EXCEPTION WHEN unique_violation THEN
UPDATE
my_table
SET
col1 = _col1
WHERE
id = _id;
END;
END;
$$
LANGUAGE plpgsql;
You could also make the update the normal case and then check the number of rows affected by the update statement to determine if the row is actually new and you need to do an insert.
As alluded to in some other answers, the most efficient way to handle this operation is in one batch:
Take all of the rows passed to the web service and bulk insert them into a temporary table
Update rows in the mater table from the temp table
Insert new rows in the master table from the temp table
Dispose of the temp table
The type of temporary table to use and most efficient way to manage it will depend on the database you are using.