We Keep Coding

Why is the row id increased by one (or more) even after one (or more) ConstraintViolationException? [duplicate]

I have got a table with auto increment primary key. This table is meant to store millions of records and I don't need to delete anything for now. The problem is, when new rows are getting inserted, because of some error, the auto increment key is leaving some gaps in the auto increment ids.. For example, after 5, the next id is 8, leaving the gap of 6 and 7. Result of this is when I count the rows, it results 28000, but the max id is 58000. What can be the reason? I am not deleting anything. And how can I fix this issue.
P.S. I am using insert ignore while inserting records so that it doesn't give error when I try to insert duplicate entry in unique column.

This is by design and will always happen.
Why?
Let's take 2 overlapping transaction that are doing INSERTs
Transaction 1 does an INSERT, gets the value (let's say 42), does more work
Transaction 2 does an INSERT, gets the value 43, does more work
Then
Transaction 1 fails. Rolls back. 42 stays unused
Transaction 2 completes with 43
If consecutive values were guaranteed, every transaction would have to happen one after the other. Not very scalable.
Also see Do Inserted Records Always Receive Contiguous Identity Values (SQL Server but same principle applies)

You can create a trigger to handle the auto increment as:
CREATE DEFINER=`root`#`localhost` TRIGGER `mytable_before_insert` BEFORE INSERT ON `mytable` FOR EACH ROW
BEGIN
SET NEW.id = (SELECT IFNULL(MAX(id), 0) + 1 FROM mytable);;
END

This is a problem in the InnoDB, the storage engine of MySQL.
It really isn't a problem as when you check the docs on “AUTO_INCREMENT Handling in InnoDB” it basically says InnoDB uses a special table to do the auto increments at startup
And the query it uses is something like
SELECT MAX(ai_col) FROM t FOR UPDATE;
This improves concurrency without really having an affect on your data.
To not have this use MyISAM instead of InnoDB as storage engine

Perhaps (I haven't tested this) a solution is to set innodb_autoinc_lock_mode to 0.
According to http://dev.mysql.com/doc/refman/5.7/en/innodb-auto-increment-handling.html this might make things a bit slower (if you perform inserts of multiple rows in a single query) but should remove gaps.

You can try insert like :
insert ignore into table select (select max(id)+1 from table), "value1", "value2" ;
This will try
insert new data with last unused id (not autoincrement)
if in unique fields duplicate entry found ignore it
else insert new data normally
( but this method not support to update fields if duplicate entry found )

Error in Script File If I Restart Identifyer Column After Insert

Is it possible to restart the ID column of an HSQLDB after rows were inserted? Can I even set it to restart at a value lower than existing IDs in the table?
The Situation
I have a simple Java program which connects to a HSQLDB like so:
DriverManager.getConnection("jdbc:hsqldb:file:" + hsqldbPath, "", "");
This gives me an HsqlException when executing the following script (this is an excerpt, the complete script for HSQLDB 2.2.4 can be found here):
SET SCHEMA PUBLIC
CREATE MEMORY TABLE PUBLIC.MAP(
ID BIGINT GENERATED BY DEFAULT AS IDENTITY(START WITH 0) NOT NULL PRIMARY KEY,
FOO VARCHAR(16) NOT NULL)
ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 1
// [...]
SET SCHEMA PUBLIC
INSERT INTO MAP VALUES(1,'Foo')
INSERT INTO MAP VALUES(2,'Bar')
ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 42
The message is:
HsqlException: error in script file: ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 42
The exception goes away when I move the RESTART-command before the INSERTs. The documentation gives no hint as to why that would be necessary.
I will eventually have to make this work on version 2.2.4 but have the same problem with the current version 2.3.2.
Background
What I am trying to do here is to recreate a situation which apparently occurred in production: An unlucky interaction with the database (I don't know what exactly happened) seems to have caused newly inserted rows to collide with existing ones because they were issued the same IDs. I want to create a test replicating the scenario in order to write a proper fix.

The .script file of the database follows a predefined order for the statements. This shouldn't be altered if it is edited and only certain manual changes are allowed (see the guide for details).
You can execute the ALTER TABLE statement via JDBC at the start of your test instead of inserting it in the script.
If IDENTITY values for the PRIMARY KEY collide, you will get an exception when you insert the values.
The actual fix for a problem like this is to RESTART WITH the max value in the primary key column plus one.

I think SEQUENCES are much more flexiblee than IDENTITY. The IDENTITY generator disabled JDBC batching, by the way.
But if you use SEQUENCE identifiers, you must pay attention to the hilo optimizers as well, because identifier are generated by Hibernate using a sequence value as a base calculation starting point.
With a SEQUENCE the restart goes like this:
ALTER SEQUENCE my_seqeunce RESTART WITH 105;

hibernate auditions generating same rev id in for multiple insert in one table

I am using hibernate auditions with hibernate version 3.5. while it is working fine when I try to insert single record in one table in one transation but the problem is when a "BATCH" runs
and it inserts multiple records in one table in single transation then single "rev id" is being generated for audit tables which causes "integrityconstraintsviolation".
As it is the normal behavior of hibernate that all the insert queries are fired at the end of tansaction(When it flushes) but at this time only one query is being fired for "rev id" generation.
select hibernate_sequence.nextval from dual;
Kindly tell me whether is it a bug in audit or i am missing something ??
Thanks in advance

A revid always span modifications in multiple tables!
Yes, the inserts are called at the end of the transaction, but: If you use optimistic transaction isolation, a transaction can read the uncommitted state of another transactions that are currently active but not yet committed.
If you become a integrityconstraintsviolation the column rev_id is unfortunatelly configured "unique". This is a wrong database schema! Correct the schema by removing the Uniqueness of the column rev_id!
The rev_id-Column in the global-hibernate-relationtable must be unique because it is a Primary-Key! (the global-hibernate-relationtable usually contains also a timestamp).

TX-row lock contention : Inserting Duplicate values for Unique Key

We are getting a TX-row lock contention error while trying to insert data
It happens while running a job which processes an xml with almost 10000 records, inserting the data into a table
We are having a unique key constraint on one of the columns in the table, and in the request we are getting duplicate values. This is causing the locks and thereby the job is taking more time.
We are using hibernate and spring. The DAO method we use is hibernate template's 'save' annotated with Spring Transaction Manager's #Transactional
Any suggestions please?

It's not clear whether you're getting locking problems or errors.
"TX-row lock contention" is an event indicating that two sessions are trying to insert the same value into a primary or unique constraint column set -- an error is not raised until the first one commits, then the second one gets the error. So you definitely have multiple sessions inserting rows. If you just had one session then you'd receive the error immediately, with no "TX-row lock contention" event being raised.
Suggestions:
Insert into a temporary table without the constraint, then load to the real table using logic that eliminates the duplicates
Eliminate the duplicates as part of the read of the XML.
Use Oracle's error logging syntax -- example here. http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_9014.htm#SQLRF55004

Hibernate Delete Error: Batch Update Returned Unexpected Row Count

I wrote this method below that is suppose to delete a member record from the database. But when I use it in my servlet it returns an error.
MemberDao Class
public static void deleteMember(Member member) {
Session hibernateSession = HibernateUtil.getSessionFactory().getCurrentSession();
Transaction tx = hibernateSession.beginTransaction();
hibernateSession.delete(member);
tx.commit();
}
Controller Part
if(delete != null) {
HttpSession httpSession = request.getSession();
Member member = (Member) httpSession.getAttribute("member");
MemberDao.deleteMember(member);
nextPage = "ledenlijst.jsp";
}
HTTP Status 500
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
Sometimes it even throws this error when I try to execute the page multiple times.
org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update
Does anybody know what exactly is causing these errors?

The error can be caused by several things. I'm not taking the credit for it, found it here.
Flushing the data before committing the object may lead to clear all
object pending for persist.
If object has primary key which is auto generated and you are
forcing an assigned key
if you are cleaning the object before committing the object to
database.
Zero or Incorrect ID: If you set the ID to zero or something else,
Hibernate will try to update instead of insert.
Object is Stale: Hibernate caches objects from the session. If the
object was modified, and Hibernate doesn’t know about it, it will
throw this exception — note the StaleStateException
Also look at this answer by beny23 which gives a few further hints to find the problem.
In your hibernate configuration, set hibernate.show_sql to true. This should show you the SQL that is executed and causes the problem.
Set the log levels for Spring and Hibernate to DEBUG, again this will give you a better idea as to which line causes the problem.
Create a unit test which replicates the problem without configuring a transaction manager in Spring. This should give you a better idea of the offending line of code.

The exception
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
use to be thrown when Hibernate notice that the entity he wants to flush to the database isn't exactly as it was at the beginning of the transaction.
I described more in details two different use cases that happen to me here.

In my case this exception was caused by wrong entity mapping. There were no cascade for relation, and referenced child entity wasn't saved before trying to reference it from parent. Changing it to
#OneToMany(cascade = CascadeType.ALL)
fixed the issue.
Surely best way to find cause of this exception is setting show_sql and DEBUG level for logs - it will stop just at the sql that caused the problem.

I was facing same issue.
The code was working in the testing environment. But it was not working in staging environment.
org.hibernate.jdbc.BatchedTooManyRowsAffectedException: Batch update returned unexpected row count from update [0]; actual row count: 3; expected: 1
The problem was the table had single entry for each primary key in testing DB table. But in staging DB there was multiple entry for same primary key. ( Problem is in staging DB the table didn't had any primary key constraints also there was multiple entry.)
So every time on update operation it gets failed. It tries to update single record and expect to get update count as 1. But since there was 3 records in the table for the same primary key, The result update count finds 3. Since expected update count and actual result update count didn't match, It throws exception and rolls back.
After the I removed all the records which have duplicate primary key and added primary key constraints. It is working fine.

this is the solution for my case, maybe it will help you!
Actually it was a conversion problem between a database field type (timestamp on postgreSQL) and his equivalent property type (Calendar) on hibernate xml file.
When Hibernate did this update request, it didn't retrieve the row because the request interrogate with a bad convert Calendar value.
So I simply replaced property type "Calendar" in "Date" in Hibernate xml file and the problem was fixed.

i recently experienced this and what happened was that i used the update method and it was throwing an exception because there was no existing record. I changed the method to saveOrUpdate. It worked.

I experienced this same issue with hibernate/JPA 2.1 when using memcached as the secondary cache. You would get the above exception along with a StaleStateException. The resolution was different than what has been noted previously.
I noticed that if you have an operation that interleaves deletes and selects (finds) from within the same table and transaction, hibernate can become overwhelmed and report that stale state exception. It would only occur for us during production as multiple identical operations on different entities would occur on the same table. You would see the system timeout and toss exceptions.
The solution is to simply be more efficient. Rather than interleaving in a loop, attempt to resolve which items need to be read and do so, preferably in one operation. Then perform the delete in a separate operation. Again, all in the same transaction but don't pepper hibernate with read/delete/read/delete operations.
This is much faster and reduces the housekeeping load on Hibernate considerably. The problem went away. This occurs when you are using a secondary cache and won't occur otherwise as the load will be on the database for resolution without a secondary cache. That's another issue.

I have had this problem,
I checked my code, there weren't any problems but when i checked my data I found out, I had two entities with the same id!
So, the flush() could not work, because it works one by one for batch updates and it found 2 rows. Therefore, it didn't update and threw exception, but it was my problem. I don't know if it works fine for you or not!