We Keep Coding

Flyway - auto increment id not working with test data in PostgreSQL

Before I added Flyway to my project, I could run POST request and the new user was created successfully with ID = 1, next one ID = 2 etc.
Then I added Flyway to create tables and insert some test data by V1_init.sql:
create table "user"(
id int8 not null,
username varchar(255),
);
insert into "user" values (1, 'user1');
insert into "user" values (2, 'user2');
insert into "user" values (3, 'user3');
Table is created. Users are inserted.
Trying to run POST request -> error 500
org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "organisation_pkey" Key (id)=(1) already exists.
So my app should add new user with ID=4 but it looks like it can't recognize that there are 3 users already added.
I'm using GenericEntity:
#Getter
#Setter
#MappedSuperclass
public abstract class GenericEntity<ID extends Serializable> implements Serializable {
#Id
#GeneratedValue
protected ID id;
}
application.properties:
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/my-app
spring.datasource.username=user
spring.datasource.password=user
spring.jpa.hibernate.ddl-auto=update
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.format_sql=true
I tried to use all strategies #GeneratedValue, changing spring.jpa.hibernate.ddl-auto, adding users in init.sql without id (not working)
but still no positive effects. Any ideas what could be wrong?

You seem to have only a half understanding of what you're doing...
I tried to use all strategies #GeneratedValue
You don't need to randomly try strategies, you need to pick the one that matches your current database design.
changing spring.jpa.hibernate.ddl-auto
This is dangerous and you should set it to "none", given that you are using flyway.
adding users in init.sql without id (not working)
This will only work if postgresql is set up to automatically generate ids (which is easiest through a sequence).
From your code, it does not look like that is the case.
what could be wrong?
JPA's #GeneratedValue is capable of ensuring that values are generated when it is responsible for creating rows (that means when you pass EntityManager#persist). It does not and can not know about your flyway scripts where you bypass JPA to insert rows manually.
Furthermore, let's look at #GeneratedValue's strategy property. The strategy you choose will influence how JPA generates IDs. There are only a few options: TABLE, SEQUENCE, IDENTITY and AUTO. Since you did not explicitly specify a strategy, you are currently using the default, which is AUTO. This is not recommended because it is not explicit, and now it's hard to say what your code is doing.
Under the TABLE and SEQUENCE strategies, JPA will do an interaction with the database in order to generate an ID value. In those cases, JPA is responsible for generating the value, though it will rely on the database to do so. Unsurprisingly, the former will use a table (this is rare, btw, but also the only strategy that is guaranteed to work on all RDBMS) and the latter will use a sequence (far more common and supported by practically every commercially relevant RDBMS).
With IDENTITY, JPA will not attempt to generate a key at all, because this strategy assumes that the DB will generate an ID value on its own. The responsibility is thus delegated to the database entirely. This is great for databases that have their own auto-increment mechanism.
Postgres does not really have an auto-increment system but it has some nice syntactic sugar that nearly makes it work like it: the serial "datatype". If you specify the datatype of a column as "serial", it will in fact be created with datatype int, but postgresql will also create a sequence and tie the default value of the ID column to the sequence's next value generator.
In your case, JPA is most likely using either SEQUENCE or TABLE. Since your DDL setting is set to "update", Hibernate will have generated a table or sequence behind your back. You should check your database with something like pgAdmin to verify which it is, but I'd put my money on a sequence (so I'm assuming it's using the SEQUENCE strategy).
Because you haven't specified a #SequenceGenerator, a default will be used which, AFAIK, will start from 1.
Then when JPA tries to insert a new row, it will call that sequence to generate an ID value. It will get the next value of the sequence, which will be 1. This will conflict with the IDs you manually entered in flyway.
My recommended solution would be to:
redefine your postgresql data type from int8 to "serial" (which is actually int + a sequence + sets up default value linking the ID column to the sequence so that postgres will automatically generate an ID if you don't explicitly specify one - careful, also don't specify null, just don't specify the ID column in the insert statement at all!)
explicitly set the generator strategy to IDENTITY on the JPA side
update your flyway scripts to insert users without explicit ID value (this will ensure that the test data advance the sequence, so that when JPA uses that same sequence later, it will not generate a conflicting ID)
I'd say there are alternative solutions, but other than using the TABLE strategy or generating keys in memory (both things which you should avoid), there isn't really a viable alternative because it will boil down to using a sequence anyway. I suppose it's possible to manually specify the sequence, forego the default value on the id field, call the sequence manually in your insert statements, and map the sequence explicitly in JPA... but I don't see why you'd make things hard on yourself.

Why do we add new String[]{"ID"} when fetching keyholder value in NamedParameterJDBCTemplate

Why do we add new String[]{"ID"} when fetching keyholder value in NamedParameterJDBCTemplate?
My question is based on the answer of this question:
Is there a way to extract primary key(or ROWID) using NamedParameterJdbcTemplate and GeneratedKeyHolder?
My question may sound stupid but I do need a clarification on this.
I know that, the solution for the exception when trying to fetch the PK from DB using Keyholder: The generated key is not of a supported numeric type. Unable to cast [oracle.sql.ROWID] to [java.lang.Number]
My question why do we cast ID to a string array and then fetch its long value? Can't we directly get a long value?
For the code snippet refer to the answer in the above link. My question is based on the answer.

In the default situation, Spring's JdbcTemplate will use JDBC's Statement.RETURN_GENERATED_KEYS option. The problem is that the Oracle JDBC driver will not return the value of an id column for that option, but instead return a 'row id' (basically an internal pointer to the row, not the id value in that row). This is not suitable for most use cases, as it would require you to explicitly query the row using the row id to obtain the actual id value, and specifically in the implementation of Spring's KeyHolder it results in the mentioned error, because a java.sql.RowId implementation is not a long (nor a subclass of java.lang.Number).
Fortunately, JDBC offers additional methods to obtain generated columns: by explicitly specifying the column names (or column indexes) of the column(s) you want to have. That is what new String[] {"ID"} does. It instructs JdbcTemplate to pass this array of column names to the JDBC driver, and the driver - assuming it supports this feature - will return the specified column(s), that is if your table actually has a column with the name ID.
In other words, this does not "cast ID to a string array and then fetch its long value", it will retrieve the value of the column ID instead of the Oracle JDBC driver returning the 'row id' of the inserted row.
The question you link to is specifically about Oracle (which defaults to returning a row id). The reason the Oracle JDBC driver returns a row id is because until recently Oracle did not have identity columns, and the only way to have identity-like behaviour was to use custom triggers to generate IDs. This makes it impossible to determine if a table has generated columns, and if so which column(s). Instead the implementation decision was to return the row id unless explicit columns were specified.
Although this problem is specific to Oracle, other drivers may have different problems that may need to be solved using the same solution: for example some drivers (e.g. PostgreSQL and Firebird (Jaybird, the Firebird JDBC driver I maintain)) will default to returning all columns, and if your ID column is not the first column, this may result in similar errors because the Spring GeneratedKeyHolder.getKey() method assumes the generated id is held in the first column of the generated keys result set.

INSERT..RETURNING is not working in JOOQ

I have a MariaDB database and I'm trying to insert a row in my table users. It has a generated id and I want to get it after insert. I have seen this but it's not working for me:
public Integer addNewUser(String name) {
Record record = context.insertInto(table("users"), field("name"))
.values(name)
.returning(field("id"))
.fetchOne();
return record.into(Integer.class);
}
New row is inserted but record is always null. I'm not using JOOQ code generation.

This is a known limitation in jOOQ 3.9: https://github.com/jOOQ/jOOQ/issues/2943
You currently cannot use the RETURNING clause in jOOQ when using plain SQL tables and fields (as opposed to generated ones), because jOOQ needs to know the identity column name to bind to JDBC (in most databases), and that meta data is not available from your table("users") object.
Unfortunately, passing the ID column to the RETURNING clause isn't sufficient, because there's no guarantee that this is the identity column. You might also pass several columns to the RETURNING clause, in case of which jOOQ wouldn't know which one would be the identity column.

Hibernate Unique Indices with condition [duplicate]

I need to use these unique constraints in PostgreSQL
CREATE UNIQUE INDEX favorites_3col_uni_idx
ON favorites (user_id, menu_id, recipe_id)
WHERE menu_id IS NOT NULL;
CREATE UNIQUE INDEX favorites_2col_uni_idx
ON favorites (user_id, recipe_id)
WHERE menu_id IS NULL;
The first one I annotate in JPA:
#Table(uniqueConstraints= {
#UniqueConstraint(name="favorites_3col_uni_idx", columnNames = {"user_id", "menu_id", "recipe_id"})
})
But, ¿it is possible to annotate in JPA the second unique index?
Thx.

You appear to want to create partial indexes (CREATE INDEX ... ON ... WHERE) using JPA constraint definitions.
These are fairly PostgreSQL specific, and aren't specified by JPA. You will need to use native syntax to create them. I don't believe JPA offers any features for index definition.
You cannot use a unique constraint for this purpose because unique partial indexes are not unique constraints. Partial unique indexes cannot be created with CONSTRAINT constraint_name UNIQUE(columns) in PostgreSQL. It's only an implementation detail that PostgreSQL creates a unique index for a unique constraint at all.
See:
Specifying an Index (Non-Unique Key) Using JPA
JPA: defining an index column
Some JPA providers offer extension annotations specific to that JPA provider that add features for running native DDL scripts, defining indexes with annoations, etc. Since you haven't mentioned which JPA provider you are using I can't tell you more. Here's the documentation for EclipseLink index DDL; this will not work if you are using Hibernate, OpenJPA, or something other than EclipseLink.
A JPA standard workaround is to check for the presence of those indexes during startup by querying pg_catalog.pg_index. If you don't find them, use an EntityManager native query to send the appropriate native SQL CREATE UNIQUE INDEX commands. A #Startup #Singleton bean is useful for this sort of task if you're using EJB3.1. See the PostgreSQL documentation for the structure of pg_catalog.pg_index. To just check if an index of a given name exists, run:
SELECT EXISTS(
SELECT 1
FROM pg_index
WHERE indexrelid = 'public.indexname'::regclass
);
Note that the above query does nothing to verify it's the index you expect, but you can do that with some additional checks. Just examine the contents of pg_index after creating the index so you know what to test for. I don't recommend trying to check for any particular value of indpred; just make sure it isn't null.

HSQLDB 2.2 skipping certain entities and not creating tables for them

I am working with Hibernate 3.5.5 and HSQLDB 2.2.9. I use JPA semantics to define Entities and have around 10 entities.
For some unknown reason, the HSQLDB doesn't seem to be picking three entities as I don't find the corresponding tables in the HSQLDB explorer and any reference to those tables result in SQL exception user lacks privilege or object not found: <TableName>.
All the three entities are in the same format as the other ones and as follows:
#Entity
#Table(name = "<TABLE_NAME>")
public class TableName
{
// private members;
// protected default constructor for the ORM
#Id
#Column(name = <PRIMARY_KEY>)
// public getter for Primary Key member.
#Column(name = <COLUMN_NAME>)
// public getters for all other members;
// protected setters
}
Did anyone face a similar issue where HSQLDB is skipping some entities?
At least, it is consistently skipping the same three entities and I am not able to figure out what is different in those from the rest. All these entities are generated using the JPA facet in Eclipse so there is no difference as such among these.
Edit 1:
My persistence.xml simply has this:
<persistence-unit name="fssPersistenceUnit" transaction-type="RESOURCE_LOCAL">
</persistence-unit>
and HSQLDB 2.0 used to pick all the entities annotated with #Entity, whereas HSQLDB 2.2.9 doesn't pick all and leaves three entities as mentioned earlier.
Edit 2:
After enabling the logging of SQL statements, I see that the CREATE TABLE statements for these three tables are rolled back. I am not able to get any further information on what was wrong with these create table statements. Also, the CREATE TABLE statement is truncated when printed.

The Hibernate dialect for HSQLDB was updated in Hibernate 3.6 and later. It is not known if 3.5.x works well with HSQLDB 2.2.x
In any case, you can use a file: database and add the hsqldb.sqllog=2 property to the database URL. This will produce a log of all the executed SQL statements. You can then check for the statements for the missing tables.
See the Guide:
http://hsqldb.org/doc/2.0/guide/dbproperties-chapt.html

Well, I'm guessing you don't actually have this exact text in your code:
#Entity
#Table(name = "<TABLE_NAME>")
public class TableName
(with an actual <TABLE_NAME> string), but that you wrote that just as a generic/pseudo code example. (because if you'd actually use the string "<TABLE_NAME>" it would be an invalid string for hibernate and it won't create the schema).
Can you tell us the actual table and column names that you use on the entities which do not have their tables created by Hibernate? In some cases the underlying db (HSQLDB in your case) has some reserved words which it will not allow for table and/or names, and it might fail silently if that's the case (I've stumbled upon this with PostgreSQL and MySQL, where a Hibernate generated schema would work in MySQL but would silently fail to create certain tables in PostgreSQL because a certain table name was not "liked" by the latter).
The same can apply to field/column names.