We Keep Coding

Passing null Values in String Format from iBatis into an Integer value in Postgres DB

I am transferring data from 1 table (MySQL) to another (Postgres) using iBatis.
Table1(String Name, INTEGER Age, DATE dob, String Note)
The code I am using to INSERT data through Ibatis is:
<insert id="insertData" parameterType="TransferBean" >
INSERT
INTO ${base.tableName}(${base.columns})
VALUES(
${base.columnValueStr}
)
</insert>
Where columns is the list of the columns, and columnValueStr is a list of the values being passed in a comma separated format.
The base query created by iBatis being passed to Postgres is :
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES("Ayush",NULL,"Sample")
However Postgres is throwing the following error:
column \"Age\" is of type smallint but expression is of type character varying\n Hint: You will need to rewrite or cast the expression.
My guess is that Postgres is reading NULL as 'null'. I have tried passing 0 (which works), and '' (does not work) based on Column Name, but its not a generic and graceful solution.
Filtering is not possible according to the type of the column.
I need help to know if there is a workaround I can make at the query or even JAVA level which would pass the NULL as a proper null.
P.S. I have tried inserting null values into SmallInt from Postgres IDE, and that works fine.

You need to check if a variable = NULL, replace it with text NULL, as the NULL might be rendered as BLANK
make sure it is
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES('Ayush',NULL,'Sample','')
not the following: (which is not working for SQL)
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES('Ayush',,'Sample',)
Since you are transferring data from MySQL to PostgreSQL, have a try: www.topnew.net/sidu/ which might be easier for your task. Simply expert from one database, and import to another. As SIDU supports both MySQL and PostgreSQL

Spring Batch Paging with sortKeys and parameter values

I have a Spring Batch project running in Spring Boot that is working perfectly fine. For my reader I'm using JdbcPagingItemReader with a MySqlPagingQueryProvider.
#Bean
public ItemReader<Person> reader(DataSource dataSource) {
MySqlPagingQueryProvider provider = new MySqlPagingQueryProvider()
provider.setSelectClause(ScoringConstants.SCORING_SELECT_STATEMENT)
provider.setFromClause(ScoringConstants.SCORING_FROM_CLAUSE)
provider.setSortKeys("p.id": Order.ASCENDING)
JdbcPagingItemReader<Person> reader = new JdbcPagingItemReader<Person>()
reader.setRowMapper(new PersonRowMapper())
reader.setDataSource(dataSource)
reader.setQueryProvider(provider)
//Setting these caused the exception
reader.setParameterValues(
startDate: new Date() - 31,
endDate: new Date()
)
reader.afterPropertiesSet()
return reader
}
However, when I modified my query with some named parameters to replace previously hard coded date values and set these parameter values on the reader as shown above, I get the following exception on the second page read (the first page works fine because the _id parameter hasn't been made use of by the paging query provider):
org.springframework.dao.InvalidDataAccessApiUsageException: No value supplied for the SQL parameter '_id': No value registered for key '_id'
at org.springframework.jdbc.core.namedparam.NamedParameterUtils.buildValueArray(NamedParameterUtils.java:336)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.getPreparedStatementCreator(NamedParameterJdbcTemplate.java:374)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:192)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:199)
at org.springframework.batch.item.database.JdbcPagingItemReader.doReadPage(JdbcPagingItemReader.java:218)
at org.springframework.batch.item.database.AbstractPagingItemReader.doRead(AbstractPagingItemReader.java:108)
Here is an example of the SQL, which has no WHERE clause by default. One does get created automatically when the second page is read:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p
On the second page, the sql is modified to the following, however it seems that the named parameter for _id didn't get supplied:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p WHERE id > :_id
I'm wondering if I simply can't use the MySqlPagingQueryProvider sort keys together with additional named parameters set in JdbcPagingItemReader. If not, what is the best alternative to solving this problem? I need to be able to supply parameters to the query and also page it (vs. using the cursor). Thank you!

I solved this problem with some intense debugging. It turns out that MySqlPagingQueryProvider utilizes a method getSortKeysWithoutAliases() when it builds up the SQL query to run for the first page and for subsequent pages. It therefore appends and (p.id > :_id) instead of and (p.id > :_p.id). Later on, when the second page sort values are created and stored in JdbcPagingItemReader's startAfterValues field it will use the original "p.id" String specified and eventually put into the named parameter map the pair ("_p.id",10). However, when the reader tries to fill in _id in the query, it doesn't exist because the reader used the non-alias removed key.
Long story short, I had to remove the alias reference when defining my sort keys.
provider.setSortKeys("p.id": Order.ASCENDING)
had to change to in order for everything to work nicely together
provider.setSortKeys("id": Order.ASCENDING)

I had the same issue and got another possible solution.
My table T has a primary key field INTERNAL_ID.
The query in JdbcPagingItemReader was like this:
SELECT INTERNAL_ID, ... FROM T WHERE ... ORDER BY INTERNAL_ID ASC
So, the key is: in some conditions, the query didn't return results, and then, raised the error above No value supplied for...
The solution is:
Check in a Spring Batch decider element if there are rows.
If it is, continue with chunk: reader-processor-writer.
It it's not, go to another step.
Please, note that they are two different scenarios:
At the beginning, there are rows. You get them by paging and finally, there are no more rows. This has no problem and decider trick is not required.
At the beginning, there are no rows. Then, this error raised, and the decider solved it.
Hope this helps.

How can I get an id for a new record in a generic way? (JOOQ 3.4 with Postgres)

With jooq 3.4 I can't figure out how to do this (with Postgresql):
Query query = dsl.insertInto(TABLE)
.set(TABLE.ID, Sequences.TABLE_ID_SEQ.nextval());
but in a case when I don't know which is the exact table, something like this:
TableImpl<?> tableImpl;
Query query = dsl.insertInto(tableImpl)
.set(tableImpl.getIdentity(), tableImpl.getIdentity().getSequence().nextval());
Is it somehow possible?
I tried this:
dsl.insertInto(tableImpl)
.set(DSL.field("id"),
tableImpl.getSchema().getSequence("table_id_seq").nextval())
This works but I still don't know how to get the sequence name from the TableImpl object.
Is there a solution for this? Or is there a problem with my approach?
In plain SQL I would do this:
insert into table_A (id) VALUES nextval('table_A_id_seq');
insert into table_B (table_A_id, some_val) VALUES (currval('table_A_id_seq'), some_val);
So I need the value or a reference to that id for later use of the id that was generated for the inserted record as default, but I don't want to set any other values.

jOOQ currently doesn't have any means of associating a table with its implicitly used sequence for the identity column. The reason for this is that the sequence is generated when the table is created, but it isn't formally connected to that table.
Usually, you don't have to explicitly set the serial value of a column in a PostgreSQL database. It is generated automatically on insert. In terms of DDL, this means:
CREATE TABLE tablename (
colname SERIAL
);
is equivalent to specifying:
CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
The above is taken from:
http://www.postgresql.org/docs/9.3/static/datatype-numeric.html#DATATYPE-SERIAL
In other words, just leave out the ID values from the INSERT statements.
"Empty" INSERT statements
Note that if you want to create an "empty" INSERT statement, i.e. a statement where you pass no values at all, generating a new column with a generated ID, you can use the DEFAULT VALUES clause.
With SQL
INSERT INTO tablename DEFAULT VALUES
With jOOQ
DSL.using(configuration)
.insertInto(TABLENAME)
.defaultValues()
.execute();
Returning IDs
Note that PostgreSQL has native support for an INSERT .. RETURNING clause, which is also supported by jOOQ:
With SQL
INSERT INTO tablename (...) VALUES (...) RETURNING ID
With jOOQ
DSL.using(configuration)
.insertInto(TABLENAME, ...)
.values(...)
.returning(TABLENAME.ID)
.fetchOne();

HQL does not ignore values with blank string in column

I am trying to setup a query for my application to pull only values from a table that have a specific column set. Mostly this column will be null, but if you edit and save the item on the application end without putting anything in this field, then it saves a blank string to that database field.
I have tried the TSQL query:
SELECT * from TABLE where COLUMN is not NULL AND COLUMN != ''
This query returns the results I need, but when I run the same query in HQL:
SELECT OBJECT from TABLE where COLUMN is not NULL and COLUMN <> ''
Then it still contains the values that have a blank string in that column. I have tried this using HQL with the operators <> and !=, and have also tried converting it to a criteria object using Restrictions.ne("column","") but nothing seems to provide the result I need.
I tried Length as in the comments, but had no luck. With the length in the query hibernate generates the full query as so. the time_clock_id column is the one that i'm having the problem with. Hibernate is set to SQLServerDialect
select timezone0_.time_zone_id as time1_368_, timezone0_.version as version368_, timezone0_.modification_timestamp as modifica3_368_, timezone0_.time_offset as time4_368_, timezone0_.modification_user as modifica5_368_, timezone0_.name as name368_, timezone0_.description as descript7_368_, timezone0_.active as active368_, timezone0_.time_clock_id as time9_368_ from time_zone timezone0_ where timezone0_.active=1 and (timezone0_.time_clock_id is not null) and len(timezone0_.time_clock_id)>0

Rookie Mistake. There was another place within my action class where I was using a different query to build the select list in the application. This was resulting in the list being overwritten with all values instead of those that use blank. After snipping this duplication I can use the operator column <> '' and I am getting the correct results

java web service working with PostgreSQL database

my code is written in java, and I am really new to java, so i hope my explanations are correct:
i have a java written web service that works with a data base.
the data base types can be PostgreSQL and mysql.
so my webservice works with the JDBC connection for both data bases.
one of my data base tables is table urls,
for postgressql it is created like this:
CREATE TABLE urls (
id serial NOT NULL primary key,
url text not null,
type integer not null);
for mysql it is creates like this:
CREATE TABLE IF NOT EXISTS URLS (
id INTEGER primary key NOT NULL AUTO_INCREMENT,
url varchar (1600) NOT NULL,
type INTEGER NOT NULL );
when I try inserting data to this table I use an entity called urls:
this entity has:
private BigDecimal id;
private String url;
private BigInteger type;
when I try to insert values to the urls table I assign values to the urls entity, while leaving the id as NULL since it is AUTO_INCREMENT for mysql and serial for postgres.
the query works for my sql, but fails for postgress.
in the postgres server log I can see the following error:
null value in column "id" violates not-null constraint
cause I sends NULL as the id value.
I found out that in order for the query to work I should use this query:
INSERT INTO URLS(ID, TYPE, URL) VALUES(DEFAULT, 1, 'DED'); or this one:
INSERT INTO URLS(TYPE, URL) VALUES(1, 'DED'); or this one:
instead of this one, that I use:
INSERT INTO URLS(ID, TYPE, URL) VALUES(NULL, 1, 'DED');
so my question is,
how do I assign the DEFAULT value to a BigDecimal value in java ?
is removing the id from my entity is the right way to go ?
how can I make sure that any changes I do to my code wont harm the mysql or any other data base that I will use ?

If you specify the column name in the insert query then postgres does not take the default value. So you should use your second insert query.
INSERT INTO URLS(TYPE, URL) VALUES(1, 'DED');
This syntax is correct for both postgres and MySQL.
This should resolve your question (1) and (3). For (2) DO NOT delete the id field from your entity. This id is going to be your link to the database row for a specific object of the entity.

1 - I think it is proper to use Long or long types instead of BigDecimal for id fields.
2 - Yes it generally helps, but it lowers portability. BTW, using an ORM framework like Hibernate may be a good choice.
3 - Integration testing usually helps and you may want to adopt TDD style development.

When using this statement:
INSERT INTO URLS(ID, TYPE, URL) VALUES(NULL, 1, 'DED');
you are telling the database that you want to insert a NULL value into the column ID and Postgres will do just that. Unlike MySQL, PostgreSQL will never implicitely replace a value that you supply with something totally different (actually all DBMS except MySQL work that way - unless there is some trigger involved of course).
So the only solution to is to actually use an INSERT that does not supply a value for the ID column. That should work on MySQL as well.