I've been searching for a way to generate IDs without actually persisting entities in Hibernate. The reason for this, is that I want to manually insert new database rows. The ideal situation would be to use the auto increment that is present in MySQL, but that is not possible due to the fact that I'm using InheritanceType.TABLE_PER_CLASS. (It is not possible to switch strategies for either inheritance mapping or sequence generation, since the project is already quite mature.)
The conclusion from the research I've done is (please correct me if I'm wrong):
Hibernate reserves a block of IDs by increasing next_val by 1. The range of IDs Hibernate can assign to entities without having to query and update the sequence table again, is (using the old value of next_val) next_val * allocationSize ... (next_val + 1) * allocationSize
My observations from playing with the conclusion:
Hibernate generates IDs that have a value far smaller than next_val * allocationSize (which is in conflict with the conclusion) - in fact, it seems that Hibernate uses IDs up to next_val * allocationSize (but not quite: the value of next_val is 350781 whereas the highest ID in the database is 350744, with an allocationSize of 20)
Sometimes, the value of next_val is smaller than the highest ID in the database
This leaves me with the only option of delegating the ID generation process to Hibernate, since it is unclear to me how IDs are generated. Besides the fact that it is unclear to me, I would like to have a solution that works for different ID generation strategies.
So the approach I want to take is to generate a bunch of ids from java code, which are then 'reserved' for me (the backing table containing the next_val gets updated), so I can use them in my INSERT queries.
This would look something like this:
Given the following definition:
#Entity
#Table(name="datamodel")
#Inheritance(strategy=InheritanceType.TABLE_PER_CLASS)
public abstract class DataModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator="model_sequence")
#SequenceGenerator(
name="model_sequence",
sequenceName="model_sequence",
allocationSize=20
)
#Column(name="id", nullable=false, unique=true, length=11)
private int id;
}
Generate IDs in a way like this:
// This will be the ID generator, a
// long-lived object that generates ids
IDGenerator gen = new IDGenerator();
gen.initialize(DataModel.class, ... /* provide a serviceRegistry, sessionFactory, or anything else that might be needed to initialize this object */);
// This will generate the next ID based
// on the parameters passed to #SequenceGenerator
// and the values that are present in the DB.
int nextId = gen.generateNextId(/* provide a session, transaction, or anything else that might be needed for accessing the DB */);
The following posts have helped me get a bit further:
Hibernate, #SequenceGenerator and allocationSize
How to set up #Id field properly when database table is already populated JPA
https://vladmihalcea.com/from-jpa-to-hibernates-legacy-and-enhanced-identifier-generators/
And my current attempt is stuck at this point:
URL configUrl = this.getClass().getClassLoader().getResource("hibernate.main.cfg.xml");
Configuration configuration = new Configuration().configure(configUrl);
ServiceRegistry serviceRegistry = new StandardServiceRegistryBuilder().applySettings(configuration.getProperties()).build();
SessionFactorysessionFactory = configuration.buildSessionFactory(serviceRegistry);
Properties generatorProperties = new Properties();
// Properties would normally be read from annotations
// that are present on the entity class
generatorProperties.put("name", "model_sequence");
generatorProperties.put("sequence_name", "model_sequence");
generatorProperties.put("allocation_size", 20);
sequenceGenerator = new SequenceStyleGenerator();
sequenceGenerator.configure(LongType.INSTANCE, generatorProperties, serviceRegistry);
// Without the line below, the queries that Hibernate uses to read
// and alter the sequence table are not initialized (they are null)
// and Hibernate throws an exception
sequenceGenerator.registerExportables( database?? /* This requires a Database object, but how to provide it? */);
// In order to generate a new ID, do the following:
Session session = sessionFactory.getCurrentSession();
Serializable id = sequenceGenerator.generate((SessionImplementor)session, entity?? /* This requires an entity, but that is exactly what I'm trying to omit. */);
My considerations on this approach:
As I understand, Hibernate reserves a block of IDs (rather than just 1 ID). This implies that the block of IDs is maintained in-memory in the SequenceStyleGenerator instance. This would cause two blocks to be present simultaneously: one for the running web application, and one for my ID generation application.
It might be problematic when the sequence table is not locked, in which case both instances may simultaneously read the same value from the table and end up using the same block of IDs.
Do any of you have an idea on how to do this?
Related
I am trying to persist a simple class using Spring, with hibernate/JPA and a PostgreSQL database.
The ID column of the table is a UUID, which I want to generate in code, not in the database.
This should be straightforward since hibernate and postgres have good support for UUIDs.
Each time I create a new instance and write it with save(), I get the following error:
o.h.j.JdbcSQLIntegrityConstraintViolationException: NULL not allowed for column "ID"; SQL statement: INSERT INTO DOODAHS (fieldA, fieldB) VALUES $1, $2) ...
This error indicates that it's expecting the ID column to be auto-populated (with some default value) when a row is inserted.
The class looks like this:
#lombok.Data
#lombok.AllArgsConstructor
#org.springframework.data.relational.core.mapping.Table("doodahs")
public class Doodah {
#org.springframework.data.annotation.Id
#javax.persistence.GeneratedValue(generator = "UUID")
#org.hibernate.annotations.GenericGenerator(name="UUID", strategy = "uuid2")
#javax.persistence.Column(nullable = false, unique = true)
private UUID id;
//... other fields
Things I have tried:
Annotate the field with #javax.persistence.Id (in addition to existing spring Id)
Annotate the field with #org.hibernate.annotations.Type(type = "pg-uuid")
Create the UUID myself - results in Spring complaining that it can't find the row with that id.
Specify strategy = "org.hibernate.id.UUIDGenerator"
Annotate class with #Entity
Replace spring #Id annotation with #javax.persistence.Id
I've seen useful answers here, here and here but none have worked so far.
NB the persistence is being handled by a class which looks like this:
#org.springframework.stereotype.Repository
public interface DoodahRepository extends CrudRepository<Doodah, UUID> ;
The DDL for the table is like this:
CREATE TABLE DOODAHS(id UUID not null, fieldA VARCHAR(10), fieldB VARCHAR(10));
Update
Thanks to Sve Kamenska, with whose help I finally got it working eventually. I ditched the JPA approach - and note that we are using R2DBC, not JDBC, so the answer didn't work straight away. Several sources (here, here, here, here, here and here) indicate that there is no auto Id generation for R2DBC. So you have to add a callback Bean to set your Id manually.
I updated the class as follows:
#Table("doodahs")
public class Doodah {
#org.springframework.data.annotation.Id
private UUID id;
I also added a Bean as follows:
#Bean
BeforeConvertCallback<Doodah> beforeConvertCallback() {
return (d, row, table) -> {
if (d.getId() == null){
d.id = UUID.randomUUID();
}
return Mono.just(d);
};
}
When a new object (with id = null, and isNew = true) is passed to the save() method, the callback method is invoked, and it sets the id.
Initially I tried using BeforeSaveCallback but it was being called too late in the process, resulting in the following exception:
JdbcSQLIntegrityConstraintViolationException: NULL not allowed for column "ID"....
Update
There are, at least, 2 types of Spring Data: JPA and JDBC.
The issue happens because you are mixing the 2 of them.
So, in order to fix, there are 2 solutions.
Solution 1 - Use Spring Data JDBC only.
Pom.xml dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
Generate ID.
Spring Data JDBC assumes that ID is generated on database level (like we already figured that out from log). If you try to save an entity with pre-defined id, Spring will assume that it is existing entity and will try to find it in the database and update. That is why you got this error in your attempt #3.
In order to generate UUID, you can:
Leave it to DB (it looks like Postgre allows to do it)
or Fill it in BeforeSaveCallback (more details here https://spring.io/blog/2021/09/09/spring-data-jdbc-how-to-use-custom-id-generation)
#Bean BeforeSaveCallback<Doodah> beforeSaveCallback() {
return (doodah, mutableAggregateChange) -> {
if (doodah.id == null) {
doodah.id = UUID.randomUUID();
}
return doodah;
};
}
Solution 2 - Use Spring Data JPA only
Pom.xml dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
Generate ID.
Here you can, actually, use the approach with the UUID auto-generation, like you wanted to do initially
Use javax.persistence #Entity annotation instead of springdata #Table on the class-level
and Use #javax.persistence.Id and #javax.persistence.GeneratedValue with all defaults on id-field.
#javax.persistence.Id
#javax.persistence.GeneratedValue
private UUID id;
Other notes:
Specification of generator and strategy is not required, since it will generate based on the type of the id field (UUID in this case).
Specification of Column(nullable = false, unique = true) is not required either, since putting #Id annotation already assumes these constraints.
Initial answer before update
The main question: how do you save the entity? As id-generation is handled by JPA provider, Hibernate in this case. It is done during save method of em or repository. In order to create entities and ids Hibernate is looking for javax.persistence annotations, while you have Spring-specific, so I am wandering how do you save them.
And another question here: the error you provided INSERT INTO DOODAHS (fieldA, fieldB) VALUES $1, $2 shows that there is no id field in the insert-query at all. Did you just simplified the error-message and removed ID from it? Or this is original error and your code does not even "see" field ID? In that case the issue in not related to the id-generation, but rather is related to the question why your code does not see this field.
I am trying to migrate one of our services to Spring Boot 2.0.3.
While most of the tests are fine, one of them fails with error:
Caused by: org.h2.jdbc.JdbcSQLException: Sequence "HIBERNATE_SEQUENCE" not found; SQL statement:
call next value for hibernate_sequence [90036-197]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.command.Parser.readSequence(Parser.java:5970)
at org.h2.command.Parser.readTerm(Parser.java:3131)
at org.h2.command.Parser.readFactor(Parser.java:2587)
This is really confusing because all teh entities rely on the same generation id mechanism:
#GeneratedValue(strategy = GenerationType.AUTO)
It's a repository test and the repository itself is very straight-forward:
#Repository
public interface OrderDetailsRepository extends JpaRepository<OrderDetails, Long> {
OrderDetails findFirstByOrderIdOrderByIdDesc(String orderId);
}
What can possible go wrong here?
PS: And, yes, there is both orderId and Id field present in the entity.
When you choose #GeneratedValue(strategy = GenerationType.AUTO) Hibernate selects a generation strategy based on the database-specific dialect.
The problem in your case is hibernate can't find the HIBERNATE_SEQUENCE and thus can't create a new object for the sequence. Try adding a sequence like this and it should solve the problem, but could lead to inconsistencies with the data...
CREATE TABLE CUSTOMER(
id int primary key,
);
CREATE SEQUENCE HIBERNATE_SEQUENCE START WITH 1 INCREMENT BY 1;
I would suggest using the GenerationType.SEQUENCEand try to recreate your id pattern with your custom db sequence. You can read more about the GenerationType's
here
I encountered the same issue when written sample code for spring boot with h2. please find the details below of my findings.
In your entity class sequence is not given and check your table as well i.e. have you given AUTO_INCREMENT for primary key?
Please follow as below.
1. Check your ddl once and set auto_increment for primary key (see below for id)
CREATE TABLE EMPLOYEES (
id INT AUTO_INCREMENT PRIMARY KEY,
first_name VARCHAR(250),
last_name VARCHAR(250),
email VARCHAR(250) DEFAULT NULL
);
Check your entity class and update primary key as below
public class Employee {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
Please make a note that GenerationType is given IDENTITY you can give AUTO as well. Also if you are using h2 in-memory DB and table inserted few records while boot-start (if dml file available in resource) then hibernate insertion may give unique constraint because sequence 1,2,3..(depends on how many records inserted while startup) may have already used and as I said above hibernate will generate the sequence from 1 and will increment by 1 for every new insertion. So I would suggest don't insert records while boot startup better to insert programmatically.
For your learning you can use as given above but if it may use in production then better to implement your own logic to generate the sequence.
I had similar problem. If I understand things correctly It went down like this.
Before Spring upgrade I used AUTO - but it actually opted by default to IDENTITY strategy. I had auto incrementing PKs defined like this:
id BIGINT AUTO_INCREMENT PRIMARY KEY
Everything was fine.
With spring upgrade I had to specify H2 dialect:
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
I've read that if you use Hibernate as your persistence provider, it selects a generation strategy based on the database specific dialect. For H2 it probably opted for global sequence (that's what AUTO should mean according to JPA spec) - and it didn't find the sequence.
Solution is of course create the sequence (as suggested above) or manually override to originally auto selected IDENTITY.
CREATE SEQUENCE HIBERNATE_SEQUENCE START WITH 1 INCREMENT BY 1;
#GeneratedValue(strategy = GenerationType.IDENTITY)
I believe that root cause is that meaning of AUTO is/was not consistent/well defined/understood in time. Probably original 'auto' switch to IDENTITY was basically a bug.
I have a Spring based application which uses JPA 2.1 for the persistence layer.
I use #TableGenerator and #GeneratedValue annotations to handle my primary key id generation.
These are the annotations:
#TableGenerator(name = "user_gen", table = "id_gen", pkColumnName = "gen_name", valueColumnName = "gen_val", allocationSize = 1)
…
#GeneratedValue(strategy = GenerationType.TABLE, generator = "user_gen")
I use this entity to insert records in my user table.
It was all working good.
Now I have a new problem. An external application needs to insert records in my user tables. This causes primary key violation at my end.
Is there any option in JPA which will make my id_gen table's gen_val value updated based on the max id of my user table?
[I could not find such a solution in my research]
Other thoughts to fix the issue are also welcome.
You might make it work with custom id generator where you would check for maximum id before each insert, but I wouldn't recommend it. In my opinion, this would be best handled on database level.
How is that application generating id's? They have to get it from somewhere, so why not why not handle id in that step. Best option would be if it can use the same id_gen mechanism for this. Several ideas:
After inserting the data, they could just call your stored procedure which would take care of synchronizing ids
They could call your stored procedure prior to insert, which would return them the id for the new row
They could call your stored procedure which would insert the data instead of them inserting it directly, and you would handle the id
I have searched on the web before posting this here, but I'm in an important project and I don't have any more time to waste with this. Well, here's the deal:
My tables in the database(SqlServer) are created automatically by Hibernate. So, I have an Entity and this entity was mapped before with Id annotation, like this:
#Id
#GeneratedValue(strategy= GenerationType.AUTO)
private int id;
Hibernate created a table dbo.Parcela inside sqlServer, but we had to change the way we generate the id, Because sometimes we receive the id number and we want that id saved on our database. So our Entity now is like this:
#Id
private Integer id;
Things work fine when we run the program for the first time, but we have some customers that already have their databases created with old Id mapping and we cannot create a new table, with the new Id mapping. So when I'm trying to insert a new record I get this message:
SEVERE: Cannot insert explicit value for identity column
in table 'Parcela' when IDENTITY_INSERT is set to OFF.
Any help would be appreciate.
Thanks
So you want your surrogate keys generated by the database, except when they were already generated by the customer. How are you going to avoid collisions, if the database wants to set id=12345, but a customer-imported entry with that id already exists?
The short answer to your question is: don't do this. I don't want to go into the old natural key vs surrogate key debate, this has been done already for example here. And google "codd surrogate keys" to learn how to properly use them. All i want to say is: if you use surrogate keys, then have your database generate them, and treat everything from outside as additional lookup key. That's the sane way.
The long answer is: if you really want to do this, and if you really know what you're doing, you can implement your own IdGenerator class. In JPA for example, you could annotate your id:
#Id
#GenericGenerator(name = "customId", strategy = "com.example.CustomIdGenerator", parameters = { #Parameter(name = "sequence", value = "SEQ_IDGENERATOR") })
#GeneratedValue(generator = "customId")
#Column(name = "ID", unique = true, nullable = false)
private Integer id;
Your customIdGenerator would then extend SequenceGenerator:
public class CustomIdGenerator extends SequenceGenerator {
public Serializable generate(SessionImplementor session, Object obj) {
// return o.getId() if obj.getId() is not null
// newId = SEQ_IDGENERATOR.nextval
// while (newId is already in db) newId = SEQ_IDGENERATOR.nextval
// return newId
}
}
and your database would provide SEQ_IDGENERATOR. Id would no longer be an autogenerated field but simply
create table foo( id integer not null primary key, ...);
But again: you don't want to do this. You want your surrogate keys to be irrelevant to the outside world and handled by the database.
How did you have Hibernate create the schema for the DB? If you used hbm2ddl perhaps adding
<property name="hibernate.hbm2ddl.auto" value="update"/>
to your persistence.xml or setting hbm2ddl.auto to "update" may have Hibernate automatically update the db schema on redeploy, having it fix the insertion problem.
Of course it won't help you in cases when you try inserting an already existing id, but i guess you know it :)
I have my entity class mapped like below:
#Entity
#Audited
#Table(name="messages_locale")
public class Locale {
#Id
#GeneratedValue
#Getter #Setter //Project Lombok's annotations, equal to generated getter and setter method
private int id;
(...)
I create clean new database ,and properties:
< prop key="hibernate.hbm2ddl.auto" >create < /prop>
WHY THE HELL (Sorry, almost two days wasted on this bug) after created database, i got a sequence in my postgres db?:
CREATE SEQUENCE hibernate_sequence
INCREMENT 1
MINVALUE 1
MAXVALUE 9223372036854775807
START 2
CACHE 1;
ALTER TABLE hibernate_sequence
OWNER TO postgres;
I dont want to have a sequence, I want to have just auto increment auto generated values..
I think the accepted answer from Petar is not correct, or not correct any longer. The auto-increment in Postgres is handled through SERIAL pseudo type, that’s correct. However, the mapping that Petar gives will result in the following DDL generated by Hibernate 5.1:
CREATE SEQUENCE users_id_seq START 1 INCREMENT 50;
CREATE TABLE … (
id INT8 NOT NULL,
…
);
This is not using SERIAL, but a Hibernate managed sequence. It is not owned by the table and no default value has been set. Of course, DDL generation is a feature that many people do not use in production (but many take the generated code as a template).
If you hand-write your DDL and actually used SERIAL, then using GenerationType.SEQUENCE may even conflict with the database behaviour. The correct way to map Hibernate with Postgres’ preferred ID strategy is using GenerationType.IDENTITY. Incidentally, the code is also much shorter and more readable:
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
Long id;
In PostgreSQL auto-increment is handled using the SERIAL pseudo type. You use this type when you execute CREATE TABLE.
Now to the point - this SERIAL pseudo type creates a sequence.
Autoincrement in PostgreSQL is handled using the created sequence. The default value of the id column becomes - nextval('your_sequence_name').
In Hibernate for an User entity:
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "users_seq_gen")
#SequenceGenerator(name = "users_seq_gen", sequenceName = "users_id_seq")
public Long getId() {
return id;
}
Read here:
http://www.postgresql.org/docs/8.4/static/datatype-numeric.html#DATATYPE-SERIAL
http://www.neilconway.org/docs/sequences/