I am using Hibernate 3.0 in my application with Postgres database. It is a monitoring application and gathers data every minute. So we have thousands of rows in some tables every month.
Currently i am using sequence for generating Id in hibernate. Is there any better option according to this scenario?
Any suggestion will be appreciated.
IMHO sequence is the best approach because it gives you more flexibility although you may also use identity (auto-increment) column. I think it postgres it is called serial and there is also a way to store ids in sepearate table. To address these 3 approach you may use
appropriately :
#GeneratedValue(strategy=GenerationType.TABLE)
#GeneratedValue(strategy=GenerationType.SEQUENCE)
#GeneratedValue(strategy=GenerationType.IDENTITY)
As for your previous question whether it is good to use single sequence for all tables. I wouldn't recommend this approach becasue db must assert that all sequence numbers are unique that is why each sequence generated value needs to be synchronized by the db server. If you have single sequence per db it may cause performace issues when multiple requests from multiple tables asks for next id value. I would rather recommend to have single sequence per table.
While I am not sure if there is a better alternative than using a sequence, I am pretty sure that you would want to look at using StatelessSession if this is just for gathering data. You can get rid of all the overhead for e.g 1st level cache, transactional write-behind etc
Related
I would like to have a persistent, distributed counter. My idea is to use a database sequence. Only sequence. I do not want to have a table, because I will not populate the table. I just need a sequence of unique numbers.
I don't want to use naive select mys-seq.nextval from dual (or org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer) because I would like to use sequence caching ability - I do not want to hit the database every time I need a new number.
I guess I should use org.hibernate.id.enhanced.SequenceStyleGenerator, but I cannot find any example of how to use it "standalone", without entity.
Unfortunately, all examples I found describes how to configure entity id generation with the sequence.
PS. I have the Spring Boot app.
I found simple solution for my problem: I can treat each number from database sequence as a range of numbers to use. For example, if sequence returns 5, it means that reserved range for my counter is 5000 - 5999.
In that solution I will hit database once for thousand numbers.
Initially I thought I have to utilize internal database level sequence number caching, but the same result I can achieve with trivial application level caching
I am writing an application which will be deployed on n number of nodes. The applications entity classes used the SEQUENCE generation strategy to generate the primary keys. Since, there would be bulk inserts; we shall be giving an allocation size as well.
The concern is when the application will be deployed on n nodes and if two nodes simultaneously requests next sequence from the defined sequence in database:
Wouldn't there be any race condition?
Or is that sequence also has some light weight locking mechanism to serve the requests sequentially, as it happens in IDENTITY strategy?
Or sequence is not the right solution to this problem?
Kindly help. Thank you !
Think of Sequence as a table with one column storing an integer representing the current id. Each time you insert a new entry, the next operations happen in a transaction:
The current value from SEQUENCE table is read
That value is assigned as ID to the new entry
The value from SEQUENCE is incremented
To answer your questions
The concurrency issues are addressed by the database.
Since inserts happen in a transaction (both simple and bulk inserts), the consistency on ID generation is enforced by the database engine via transactions (by the isolation level of the transaction to be more precise). Make sure your database engine supports transactions.
Sequence is the right solution, assuming your database engine supports transactions.
If the result set is large, then having the entire result set in memory (server cache e.g. hazelcast) will not be feasible. With large result sets, you cannot afford to have them in memory. In such case, you have to fetch a chunk of data at a time (query based paging). The down side of using query based paging, is that there will be multiple calls to the database for multiple page requests.
Can anyone suggest how to implement a hybrid approach of it.
I haven't put any sample code here since I think the question is more about a logic instead of specific code. Still if you need sample code I can put it.
Thanks in advance.
The most effective solution is to use the primary key as a paging criterion.This enables us to rely of first class constructs like a between range query which is simple for the RDBMS to optimize, the primary key of the queried entity will most likely be indexed already.
Retrieving data using a range query on the primary key is a two-step process. First one have to retrieve the collection of primary-keys, followed by a step to generate the intervals to properly identify a proper subset of the data,followed by the actual queries against the data.
This approach is almost as fast as the brute-force version. The memory consumption is about one tenth. By selecting the appropriate page-size for this implementation, you may alter the ratio between execution time and memory consumption. This version is also stateless, it does not keep references to resources like the ScrollableResults version does, nor does it strain the database like the version using setFirstResult/setMaxResult.
Effective pagination using Hibernate
I need to insert or update records based on whether record already exist.
I am using JdbcBatchItemWriter to write the records. but if the record with the primary key already exist, I should update it...
So one solution is:
To make two seperate lists one for insert and one for update(note: I have to check in my processor every time whether the record already exist and thus add the record in one of the list), and have two different JdbcBatchItemWriter instances in my writer, for example:
JdbcBatchItemWriter<insertList> insertWriter;
JdbcBatchItemWriter<updateList> updateWriter;
Is there any other way to switch between the queries in writer based on record already exist at the time of batch update...i.e.
just one
JdbcBatchItemWriter<mylist> allWriter...and
allWriter.write(mylistallitems);
I am thinking of using a merge query...but are there any performance issues?
Having two different lists may be a better option since if you have a different persistence mechanism in the future, you needn't redesign your app. You may want to have a single query to get all existing primary keys from DB, store it a Collection holder, and refer to it in the processor.
A brief search on SO on 'Oracle Merge Performance' indicates multiple instances of performance issues due to different factors & maybe slower than crafted insert/update SQLs.
Also if you are receiving the complete data again (for updates), you may want to consider the truncate-insert approach [delete before insertions by adding a listener]
I am developing an application that support multiple databases and hibernate fulfilling that requirement.
Now the issue is in primary auto generate key. some databases support auto increment and some required sequence for increment the identity. to solve this issue the use the following strategy
strategy = GenerationType.TABLE (javax.persistence)
This is fulfilling my requirement.
in this post, a user comment that
its always better to use increment or sequence instead of table generation if you need the ids to be in sequence
If I use the auto increment or sequence, it means it required some changes # annotation level, when I move one database to another (extra burden)
update me , it is really better to use increment or sequence instead of table generation or it is just a statement?
Auto increment drawbacks: You don't know the id until the transaction has committed (which can be a problem in JPA, since some EntityManager operations rely on Id's). Not all databases support auto increment fields.
Sequence drawbacks: Not all databases have sequences.
Table drawbacks: Id's are not necessarily consecutive.
Since it is very unlikely that you run out of Id's, using Table generation remains a good option. You can even tweak the id allocation size in order to use more consecutive id's (default size is 50):
#TableGenerator(name="myGenerator", allocationSize=1)
However, this will result in at least two queries to the id allocation table for each insert: one to step the value of the latest id, and one to retrieve it.