We Keep Coding

Multiple sequence number generator for a single column while saving to the postgresql db

I have a column (certificate number) in DB, wherein I have to save the value of certificate number for 3 different products, each products series starts from a different number (1st product series starts from 001, 2nd product starts with 2000 and so on so forth), I have to update the same column in the table from the previously saved certificate number of that particular product series.is there any way I can achieve this using spring boot, JPA, with PostgreSQL?
Thanks in advance.

It's quite easy to do with PostgreSQL SEQUENCE.
First you create a set of required sequences, i.e.
CREATE SEQUENCE product1 START 1; // You may need to define max value as well
CREATE SEQUENCE product2 START 2000;
etc.
Then you can use nextval(sequence_name) in your SQL statements, i.e.
INSERT INTO table_name VALUES (nextval('product1'), 'bla-bla');
You can find more info here https://www.postgresql.org/docs/13/sql-createsequence.html
If you want to use JPA, then you need to look at two annotations.
In order to create a sequence for an entity you use
#SequenceGenerator(name="product2", initialValue=2000)
and on the field definition you use
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="product2")
Edit: I see the issue you may have with JPA, you can't choose which sequence to use for an operation.

How to return no matched row in Pentaho Data Inegration (Kettle)?

I look for a solution to perform SSIS lookup in Pentaho Data Integration.
I'll try to explain with an exemple :
I have two tables A and B.
Here , data in table A :
1
2
3
4
5
Here , data in table B:
3
4
5
6
7
After my process :
All rows in A and not in B ==> will be insert to B
All rows in B and not in A ==> will be deleted to A
So , here my final Table B :
3
4
5
1
2
someone can help me please ?

There is indeed a step that does this, but it doesn't do it alone. It's the Merge rows(diff) step and it has some requirements. In your case, A is the "compare" table and B is the "reference" table.
First of all, both inputs (rows from A and B in your case, Dev and Prod in mine) need to be sorted by a key value. In the step you specify the key fields to match on, and then the value fields to compare. The step adds a field to the output (by default called 'flagfield'). After comparing each row, this field is given one of four values: "new", "changed", "deleted", or "identical". Note in my example below I have explicit sort steps. That's because the sorting scheme of my database is not compatible with PDI's, and for this step to work, your data must be in PDI's sort order. You may not need these.
You can follow this with a Synchronize after merge step to apply the identified changes. In this step you specify the flagfield and the values that correspond to insert, update, and delete. FYI these are specified on the "Advanced" tab, and they must be filled out for the step to work.
For a very small table like your example, I would favor just a truncate and full load with a Table output step, but if your tables are large and the number of changes relatively small (<= ~25%) and replication is not available, this step is usually the way to go.

In Pentaho direct step is not availble. There are so many ways to do these.
=> Writing sql's to achieve your solution. If you write sql's execution speed also faster.
=> Using filter step also you can acheive.
Thank you.

Remove records from table which id does not exist in a list, keep existing and insert those not in the table

I have been thinking a way to do what the topic suggests:
Assume that I am given an array list of integers, e.g. 1, 3, 5, and in a table like the following I have certain records existing:
categoryid userid
1 2
2 2
3 2
Which categoryid and userid together form the primary key. And with the array list given by user with id = 2, I want to remove records with categoryid that are not in the list, which is 2. And then keep those are in the list, which is 3, finally insert those are not in the table yet, i.e. 5. And the table after the operation will looks like the following:
categoryid userid
1 2
3 2
5 2
Is there better way to do this other than simply remove all records with userid = 2, and insert new records according to the array list?
Thanks a lot!

As you said, you want to do two main operations DELETE and INSERT. you cannot do these in one query, so you want at least two queries. in the other hand you can write your sql query for delete and insert in a smart way that all deletions is done in just one query (by using OR for different values) and also insertion in one batch query.

How to fill a closure table using JPA?

I am trying to model a hierarchy of objects (actually, domain groups) in a database. I decided to use a closure table, so that I can gain high flexibility in querying the hierarchy. Basically, my schema looks something like this:
CREATE TABLE group (
id INT -- primary key
... -- other fields here
)
CREATE TABLE groupHierarchy (
idAncestor INT,
idGroup INT,
hierarchyLevel INT
)
So, when a group with an id of 1 contains a group with an id of 2, which in turn contains a group with an id of 3, I will need to have following rows in the groupHierarchy table.
idAncestor idGroup hierarchyLevel
1 1 0
2 2 0
3 3 0
1 2 1
2 3 1
1 3 2
I am also OK with not having the rows with the hierarchyLevels of 0 (self - reference).
Now I would like to have an JPA entity that would map to the group table. My question is - what would be a good way to manage the groupHierarchy table?
What I already considered is:
1) Having the group hierarchy mapped as an element collection, like :
#ElementCollection
#JoinTable(name = "groupHierarchy")
#MapKeyJoinColumn(name = "idAncestor")
#Column(name = "hierarchyLevel")
Map<Group, Integer> ancestors;
This would require handling the hierarchy entirely in the application, and I am afraid that this may become very complex.
2) Make the application unaware of the hierarchyLevel column and handle it in the database using a trigger (when a record is added, check if the parent already has ancestors and if so, add any other required rows. This is also where the hierarchyLevel of 0 would come in handy). It seems to me that the database trigger would be simpler, but I'm not sure if that would be good for the overall readability.
Can anyone suggest other options? Or maybe point to any pros or cons of the solutions I have mentioned?

May I suggest you using JpaTreeDao? I think it's complete and very well documented. I'm going to try to port the closure tables code to a groovy implementation...

How to manage consecutive column values in table rows

A little presentation for what I want to do:
Consider the case where different people from a firm get, once a year, an all expenses paid trip to somewhere. There may be 1000 persons that could qualify for the trip but only 16 places are available.
Each of this 16 spots has an associated index which must be from 1 to 16. The ones on the reservation have index starting from 17.
The first 16 persons that apply get a definite spot on the trip. The rest end up on the reservation list. If one of the first 16 persons cancels, the first person with a reservation gets his place and all the indexes are renumbered to compensate for the person that canceled.
All of this is managed in a Java web app with an Oracle DB.
Now, my problem:
I have to manage the index in a correct way (all sequential, no duplicate indexes), with possible hundreds of people that simultaneously apply for the trip.
When inserting a record in the table for the trip, the way of getting the index is by
SELECT MAX(INDEX_NR) + 1 AS NEXT_INDEX_NR FROM TABLE
and using this as the new index (this is done Java side and then a new query to insert the record). It is obvious why we have multiple spots or reservations with the same index. So, we get, let’s say, 19 people on the trip because 4 of them have index 10, for example.
How can I manage this? I have been thinking of 3 ways so far:
Use an isolation level of Serializable for the DB transactions (don’t like this one);
Insert a record with no INDEX_NR and then have a trigger manage the things… in some way (never worked with triggers before);
Each record also has a UPDATED column. Could I use this in some way? (note that I can’t lose the INDEX_NR since other parts of the app make use of it).
Is there a best way to do this?

Why make it complicated ?
Just insert all reservations as they are entered and insert a timestamp of when they resevered a spot.
Then in you query just use the timestamp to sort them.
There is offcourse the chance that there are people that reserved a spot at the very same millisecond then just use a random method to assign order.

Why do you need to explicitly store the index? Instead you could store each person's order (which never changes) along with an active flag. In your example if person #16 pulls out you simply mark them as inactive.
To compute whether a person qualifies for the trip you simply count the number of active people with order less than that person:
select count(*)
from CompetitionEntry
where PersonOrder < 16
and Active = 1
This approach removes the need for bulk updates to the database (you only ever update one row) and hence mostly mitigates your problem of transactional integrity.

Another way would be to explicitly lock a record on another table on the select.
-- Initial Setup
CREATE TABLE NUMBER_SOURCE (ID NUMBER(4));
INSERT INTO NUMBER_SOURCE(ID) VALUES 0;
-- Your regular code
SELECT ID AS NEXT_INDEX_NR FROM NUMBER_SOURCE FOR UPDATE; -- lock!
UPDATE NUMBER_SOURCE SET ID = ID + 1;
INSERT INTO TABLE ....
COMMIT; -- releases lock!
No other transaction will be able to perform the query on the table NUMBER_SOURCE until the commit (or rollback).

When adding people to the table, give them an ID in such a way that the ID is ascending in the order in which they were added. This can be a timestamp.
Select all the records from the table which qualify, order by ID, and update their INDEX_NR
Select * from table where INDEX_NR <= 16 order by INDEX_NR
Step #2 seems complicated but it's actually quite simple:
update (
select *
from TABLE
where ...
order by ID
)
set INDEX_NR = INDEXSEQ.NEXTVAL
Don't forget to reset the sequence to 1.

Calculate your index in runtime:
CREATE OR REPLACE VIEW v_person
AS
SELECT id, name, ROW_NUMBER() OVER (ORDER BY id) AS index_rn
FROM t_person
CREATE OR REPLACE TRIGGER trg_person_ii
INSTEAD OF INSERT ON v_person
BEGIN
INSERT
INTO t_person (id, name)
VALUES (:new.id, :new.name);
END;
CREATE OR REPLACE TRIGGER trg_person_iu
INSTEAD OF UPDATE ON v_person
BEGIN
UPDATE t_person
SET id = :new.id,
name = :new.name
WHERE id = :old.id;
END;
CREATE OR REPLACE TRIGGER trg_person_id
INSTEAD OF DELETE ON v_person
BEGIN
DELETE
FROM t_person
WHERE id = :old.id;
END;
INSERT
INTO v_person
VALUES (1, 'test', 1)
SELECT *
FROM v_person
--
id name index_rn
1 test 1
INSERT
INTO v_person
VALUES (2, 'test 2', 1)
SELECT *
FROM v_person
--
id name index_rn
1 test 1
2 test 2 2
DELETE
FROM v_person
WHERE id = 1
SELECT *
FROM v_person
--
id name index_rn
2 test 2 1

"I have to manage the index in a correct way (all sequential, no duplicate indexes), with possible hundreds of people that simultaneously apply for the trip.
When inserting a record in the table for the trip, the way of getting the index is by
SELECT MAX(INDEX_NR) + 1 AS NEXT_INDEX_NR FROM TABLE
and using this as the new index (this is done Java side and then a new query to insert the record). It is obvious why we have multiple spots or reservations with the same index."
Yeah. Oracle's MVCC ("snapshot isolation") used incorrectly by someone who shouldn't have been in IT to begin with.
Really, Peter is right. Your index number is, or rather should be, a sort of "ranking number" on the ordered timestamps that he mentions (this holds a requirement that the DBMS can guarantee that any timestamp value appears only once in the entire database).
You say you are concerned with "regression bugs". I say "Why do you need to be concerned with "regression bugs" in an application that is DEMONSTRABLY beyond curing ?". Because your bosses paid a lot of money for the crap they've been given and you don't want to be the pianist that gets shot for bringing the message ?

The solution depends on what you have under your control. I assume that you can change both database and Java code, but refrain from modifying the database scheme since you had to adapt too much Java code otherwise.
A cheap solution might be to add a uniqueness constraint on the pair (trip_id, index_nr) or just on index_nr if there is just one trip. Additionally add a check contraint check(index_nr > 0) - unless index_nr is already unsigned. Everything else is then done in Java: When inserting a new applicant as described by you, you have to add code catching the exception when someone else got inserted concurrently. If some record is updated or deleted, you either have to live with holes between sequence numbers (by selecting the 16 candidates with the lowest index_nr as shown by Quassnoi in his view) or fill them up by hand (similarily to what Aaron suggested) after every update/delete.
If index_nr is mostly used in the application as read-only, a better solution might be to combine the answers of Peter and Quassnoi: Use either a time stamp (automatically inserted by the database by defining the current time as default) or an auto-incremented integer (as default inserted by the database) as value stored in the table. And use a view (like the one defined by Quassnoi) to access the table and the automatically calculated index_nr from Java. But also define both constraints like for the cheap solution.