I have the next problem
POJO
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long idUser;
#Email
#Column(length = 64, unique = true)
private String email;
When the email constrains fail, the next generated entity doesn't follow the previous id.
So, it goes like
1
2, (email constraint fails), 4. Jpa skipped number 3....
This is normal for many databases; to support concurrent transactions, underlying sequence numbers may be incremented by transactions that are later rolled back, but the database doesn't attempt to "fill in" the missing values, since it would be prohibitive to keep track of them.
Related
I have a Client entity with orgId and clientId as a composite key. When I have to insert a new client object, I have to generate clientId id sequentially for each orgId, so to do that, I am generating clientId by maintaining the last clientId of every orgId in a separate table, and selecting, adding 1, and updating it.
#Entity
#Table(name = "ftb_client")
public class Client implements Serializable {
#Id
#JoinColumn(name = "ORG_ID")
protected String orgId;
#Id
#Column(name = "CLIENT_ID")
protected int clientId;
#Column(name = "CLIENT_NAME_ENG")
private String clientNameEng;
//....
}
#Entity
#Table
public class MySeq implements Serializable {
#Id
protected String orgId;
private int lastClientId;
//....
}
public Long getNewClientId(String orgId) {
MySeq mySeq = getSession()
.createQuery("from MySeq where orgId = :orgId", MySeq.class)
.setParameter("orgId", orgId)
.setLockMode(LockModeType.PESSIMISTIC_WRITE)
.uniqueResult();
mySeq.setLastClientId(mySeq.getLastClientId() + 1);
return mySeq.getLastClientId();
}
But, this leads to duplicate id generation if there are thousands of concurrent requests. So, to make it thread-safe I have to use Pessimistic locking, so that multiple requests do not generate the same clientId. But now, the problem is that lock doesn't get released until the transaction ends and concurrent requests keep pending for a long time.
So instead of using a lock, if I could use a separate sequence per orgId then I could make the id generation concurrent too. I want to manually execute the sequence generator by determining the sequence name in the runtime by doing something like client_sequence_[orgId] and execute it to generate the id.
And I also want to make it database-independent, or at least for Oracle, MySQL, and Postgres.
I want to know if it is possible or is there any other approach?
It doesn't matter if you use PESSIMISTIC_WRITE or not, a lock will be acquired anyway if you update the entity. The difference is that the lock is acquired eagerly in the case you describe here which prevents lost writes.
Usually, this is solved by creating a separate transaction for the sequence increment. To improve performance, you should increment by a batching factor i.e. 10 and keep 10 values in a queue in-memory to serve from. When the queue is empty, you ask for another 10 values etc.
Hibernate implements this behind the scenes with the org.hibernate.id.enhanced.TableGenerator along with org.hibernate.id.enhanced.PooledOptimizer. So if you know the sequences that you need upfront, I would recommend you use these tools for that purpose. You can also do something similar though yourself if you like.
I have an entity with string id:
#Table
#Entity
public class Stock {
#Id
#Column(nullable = false, length = 64)
private String index;
#Column(nullable = false)
private Integer price;
}
And JpaRepository for it:
public interface StockRepository extends JpaRepository<Stock, String> {
}
When I call stockRepository::findAll, I have N + 1 problem:
logs are simplified
select s.index, s.price from stock s
select s.index, s.price from stock s where s.index = ?
The last line from the quote calls about 5K times (the size of the table). Also, when I update prices, I do next:
stockRepository.save(listOfStocksWithUpdatedPrices);
In logs I have N inserts.
I haven't seen similar behavior when id was numeric.
P.S. set id's type to numeric is not the best solution in my case.
UPDATE1:
I forgot to mention that there is also Trade class that has many-to-many relation with Stock:
#Table
#Entity
public class Trade {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Integer id;
#Column
#Enumerated(EnumType.STRING)
private TradeType type;
#Column
#Enumerated(EnumType.STRING)
private TradeState state;
#MapKey(name = "index")
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(name = "trade_stock",
joinColumns = { #JoinColumn(name = "id", referencedColumnName = "id") },
inverseJoinColumns = { #JoinColumn(name = "stock_index", referencedColumnName = "index") })
private Map<String, Stock> stocks = new HashMap<>();
}
UPDATE2:
I added many-to-many relation for the Stock side:
#ManyToMany(cascade = CascadeType.ALL, mappedBy = "stocks") //lazy by default
Set<Trade> trades = new HashSet<>();
But now it left joins trades (but they're lazy), and all trade's collections (they are lazy too). However, generated Stock::toString method throws LazyInitializationException exception.
Related answer: JPA eager fetch does not join
You basically need to set #Fetch(FetchMode.JOIN), because fetch = FetchType.EAGER just specifies that the relationship will be loaded, not how.
Also what might help with your problem is
#BatchSize annotation, which specifies how many lazy collections will be loaded, when the first one is requested. For example, if you have 100 trades in memory (with stocks not initializes) #BatchSize(size=50) will make sure that only 2 queries will be used. Effectively changing n+1 to (n+1)/50.
https://docs.jboss.org/hibernate/orm/4.3/javadocs/org/hibernate/annotations/BatchSize.html
Regarding inserts, you may want to set
hibernate.jdbc.batch_size property and set order_inserts and order_updates to true as well.
https://vladmihalcea.com/how-to-batch-insert-and-update-statements-with-hibernate/
However, generated Stock::toString method throws
LazyInitializationException exception.
Okay, from this I am assuming you have generated toString() (and most likely equals() and hashcode() methods) using either Lombok or an IDE generator based on all fields of your class.
Do not override equals() hashcode() and toString() in this way in a JPA environment as it has the potential to (a) trigger the exception you have seen if toString() accesses a lazily loaded collection outside of a transaction and (b) trigger the loading of extremely large volumes of data when used within a transaction. Write a sensible to String that does not involve associations and implement equals() and hashcode() using (a) some business key if one is available, (b) the ID (being aware if possible issues with this approach or (c) do not override them at all.
So firstly, remove these generated methods and see if that improves things a bit.
With regards to the inserts, I do notice one thing that is often overlooked in JPA. I don't know what Database you use, but you have to be careful with
#GeneratedValue(strategy = GenerationType.AUTO)
For MySQL I think all JPA implementations map to an auto_incremented field, and once you know how JPA works, this has two implication.
Every insert will consist of two queries. First the insert and then a select query (LAST_INSERT_ID for MySQL) to get the generated primary key.
It also prevents any batch query optimization, because each query needs to be done in it's own insert.
If you insert a large number of objects, and you want good performance, I would recommend using table generated sequences, where you let JPA pre-allocate IDs in large chunks, this also allows the SQL driver do batch Insert into (...) VALUES(...) optimizations.
Another recommendation (not everyone agrees with me on this one). Personally I never use ManyToMany, I always decompose it into OneToMany and ManyToOne with the join table as a real entity. I like the added control it gives over cascading and fetch, and you avoid some of the ManyToMany traps that exist with bi-directional relations.
I have code that uses Hibernate to write to an Oracle table. It uses a SequenceGenerator to generate unique id's. Say I have id's 1 through 40 in the database. What happens is that if any users are deleted from the table, it leaves a gap (say, id=24) in the id's in the table. Then, when a new user is created, the new user's id is set by Hibernate to 24.
Now there is a problem because the immediate next new user gets an id=25, which causes a UniqueConstraint exception.
Is there something I'm doing wrong? How do I make Hibernate stop generating sequence values that already exist in the table?
#Entity
#Table(name="User")
public class User {
#Id
#SequenceGenerator(name="UserGen", sequenceName="UserSeq")
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="UserGen")
private Integer id;
#Column(length=64, unique=true)
private String username;
...
Here is the sequence info in Oracle:
CREATED 31-OCT-16
LAST_DDL_TIME 31-OCT-16
SEQUENCE_OWNER USERSERVICE
SEQUENCE_NAME USERSEQ
MIN_VALUE 1
MAX_VALUE 9999999999999999999999999999
INCREMENT_BY 1
CYCLE_FLAG N
ORDER_FLAG N
CACHE_SIZE 20
LAST_NUMBER 81
PARTITION_COUNT
SESSION_FLAG N
KEEP_VALUE N
You have to define your SequenceGenerator to have the same allocationSize as the INCREMENT_BY value of your sequence.
#SequenceGenerator(name="UserGen", sequenceName="UserSeq", allocationSize = 1)
I've faced this problem before (in PostgreSQL) and eventually I just changed it to #GeneratedValue( strategy = GenerationType.IDENTITY ) and removed the SequenceGenerators entirely, since they were already used as default value on insert when not specified. It's a slight performance boost when your sequence increment size is 1, because with the SequenceGenerator Hibernate calls the sequence manually, using one extra query that it can spare.
In my web application I have multiple scheduled services which work on same entities ( like article, customer...etc). If I run a single service at time I've no problem, but when I run two services I get an error because the primary key unique constraint is violated.
As primary key I use a generated Long value:
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "ID")
public Long getId() {
return id;
}
Every service read data from a txt file, create its entities and every 20 entities execute a flush on hibernate session followed by a clear. Only at the end of the execution of the service the session is committed.
How can I solve?
Oracle supports only sequences for generated keys. Add a sequence to your database:
CREATE SEQUENCE ARTICLE_SEQ;
Change your annotations to:
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="articleSequence")
#SequenceGenerator(name="articleSequence", sequenceName="ARTICLE_SEQ",allocationSize=1)
It's best to use a separate sequence for each table/type.
Create sequence in DB and use it like #GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SEQ_NAME").
I have the following Model:
public class Parameter extends Model {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
public long id;
}
The records created by save() look like this:
play=> select id from parameter;
id
----
1
2
3
4
5
21
(6 rows)
Is there a way that I can tell JPA to always increment the id by exactly one, instead of randomly jumping to 21?
What you are observing (jump to 21) is likely to be an implmentation detail of Ebean prior to 4.0.5. Ebean is using a 'fetch ahead' mechanism on the sequence. After 4.0.5 Ebean switched to use Postgres SERIAL by default (so by default you won't see this with Ebean/Postgres after 4.0.5). Refer: https://github.com/ebean-orm/avaje-ebeanorm/issues/97
That all said - 'incrementing EXACTLY by 1' is something you generally want to avoid as it becomes a concurrency issue (avoid making the autoincrement ID transactional as that introduces contention). When you need 'increment exactly by one' semantics (like cheque numbers etc) then you can look to create in batch in advance.
You should be able to achieve this by using #TableGenerator and allocationSize=1. An example code something like the following. You need to create a separate table to store the PK, additional work, but more portable than #GeneratedValue.
#TableGenerator(name="MAIN_PK_GEN", table="pk_gen", pkColumnName="GEN_NAME", valueColumnName="GEN_VALUE", pkColumnValue="MAIN_PK_GEN", allocationSize=1)
#Id #GeneratedValue(strategy=GenerationType.TABLE, generator="MAIN_PK_GEN" )
#Basic(optional = false)
#Column(name = "PK")
private Integer pk;
(But using allocationSize=1 may not be efficient)
See this tutorial for step by step explanation.
If you are using Oracle don't forget to add SequenceGenerator with an allocationSize of 1 :
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "mysequence")
#SequenceGenerator(name = "mysequence", sequenceName = "mysequence", allocationSize = 1)
private Long id;