I am trying to setup a query for my application to pull only values from a table that have a specific column set. Mostly this column will be null, but if you edit and save the item on the application end without putting anything in this field, then it saves a blank string to that database field.
I have tried the TSQL query:
SELECT * from TABLE where COLUMN is not NULL AND COLUMN != ''
This query returns the results I need, but when I run the same query in HQL:
SELECT OBJECT from TABLE where COLUMN is not NULL and COLUMN <> ''
Then it still contains the values that have a blank string in that column. I have tried this using HQL with the operators <> and !=, and have also tried converting it to a criteria object using Restrictions.ne("column","") but nothing seems to provide the result I need.
I tried Length as in the comments, but had no luck. With the length in the query hibernate generates the full query as so. the time_clock_id column is the one that i'm having the problem with. Hibernate is set to SQLServerDialect
select timezone0_.time_zone_id as time1_368_, timezone0_.version as version368_, timezone0_.modification_timestamp as modifica3_368_, timezone0_.time_offset as time4_368_, timezone0_.modification_user as modifica5_368_, timezone0_.name as name368_, timezone0_.description as descript7_368_, timezone0_.active as active368_, timezone0_.time_clock_id as time9_368_ from time_zone timezone0_ where timezone0_.active=1 and (timezone0_.time_clock_id is not null) and len(timezone0_.time_clock_id)>0
Rookie Mistake. There was another place within my action class where I was using a different query to build the select list in the application. This was resulting in the list being overwritten with all values instead of those that use blank. After snipping this duplication I can use the operator column <> '' and I am getting the correct results
Related
I am transferring data from 1 table (MySQL) to another (Postgres) using iBatis.
Table1(String Name, INTEGER Age, DATE dob, String Note)
The code I am using to INSERT data through Ibatis is:
<insert id="insertData" parameterType="TransferBean" >
INSERT
INTO ${base.tableName}(${base.columns})
VALUES(
${base.columnValueStr}
)
</insert>
Where columns is the list of the columns, and columnValueStr is a list of the values being passed in a comma separated format.
The base query created by iBatis being passed to Postgres is :
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES("Ayush",NULL,"Sample")
However Postgres is throwing the following error:
column \"Age\" is of type smallint but expression is of type character varying\n Hint: You will need to rewrite or cast the expression.
My guess is that Postgres is reading NULL as 'null'. I have tried passing 0 (which works), and '' (does not work) based on Column Name, but its not a generic and graceful solution.
Filtering is not possible according to the type of the column.
I need help to know if there is a workaround I can make at the query or even JAVA level which would pass the NULL as a proper null.
P.S. I have tried inserting null values into SmallInt from Postgres IDE, and that works fine.
You need to check if a variable = NULL, replace it with text NULL, as the NULL might be rendered as BLANK
make sure it is
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES('Ayush',NULL,'Sample','')
not the following: (which is not working for SQL)
INSERT INTO TABLE2(Name,Age,Dob,Note) VALUES('Ayush',,'Sample',)
Since you are transferring data from MySQL to PostgreSQL, have a try: www.topnew.net/sidu/ which might be easier for your task. Simply expert from one database, and import to another. As SIDU supports both MySQL and PostgreSQL
I have a Spring Batch project running in Spring Boot that is working perfectly fine. For my reader I'm using JdbcPagingItemReader with a MySqlPagingQueryProvider.
#Bean
public ItemReader<Person> reader(DataSource dataSource) {
MySqlPagingQueryProvider provider = new MySqlPagingQueryProvider()
provider.setSelectClause(ScoringConstants.SCORING_SELECT_STATEMENT)
provider.setFromClause(ScoringConstants.SCORING_FROM_CLAUSE)
provider.setSortKeys("p.id": Order.ASCENDING)
JdbcPagingItemReader<Person> reader = new JdbcPagingItemReader<Person>()
reader.setRowMapper(new PersonRowMapper())
reader.setDataSource(dataSource)
reader.setQueryProvider(provider)
//Setting these caused the exception
reader.setParameterValues(
startDate: new Date() - 31,
endDate: new Date()
)
reader.afterPropertiesSet()
return reader
}
However, when I modified my query with some named parameters to replace previously hard coded date values and set these parameter values on the reader as shown above, I get the following exception on the second page read (the first page works fine because the _id parameter hasn't been made use of by the paging query provider):
org.springframework.dao.InvalidDataAccessApiUsageException: No value supplied for the SQL parameter '_id': No value registered for key '_id'
at org.springframework.jdbc.core.namedparam.NamedParameterUtils.buildValueArray(NamedParameterUtils.java:336)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.getPreparedStatementCreator(NamedParameterJdbcTemplate.java:374)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:192)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.query(NamedParameterJdbcTemplate.java:199)
at org.springframework.batch.item.database.JdbcPagingItemReader.doReadPage(JdbcPagingItemReader.java:218)
at org.springframework.batch.item.database.AbstractPagingItemReader.doRead(AbstractPagingItemReader.java:108)
Here is an example of the SQL, which has no WHERE clause by default. One does get created automatically when the second page is read:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p
On the second page, the sql is modified to the following, however it seems that the named parameter for _id didn't get supplied:
select *, (select id from family f where date_created between :startDate and :endDate and f.creator_id = p.id) from person p WHERE id > :_id
I'm wondering if I simply can't use the MySqlPagingQueryProvider sort keys together with additional named parameters set in JdbcPagingItemReader. If not, what is the best alternative to solving this problem? I need to be able to supply parameters to the query and also page it (vs. using the cursor). Thank you!
I solved this problem with some intense debugging. It turns out that MySqlPagingQueryProvider utilizes a method getSortKeysWithoutAliases() when it builds up the SQL query to run for the first page and for subsequent pages. It therefore appends and (p.id > :_id) instead of and (p.id > :_p.id). Later on, when the second page sort values are created and stored in JdbcPagingItemReader's startAfterValues field it will use the original "p.id" String specified and eventually put into the named parameter map the pair ("_p.id",10). However, when the reader tries to fill in _id in the query, it doesn't exist because the reader used the non-alias removed key.
Long story short, I had to remove the alias reference when defining my sort keys.
provider.setSortKeys("p.id": Order.ASCENDING)
had to change to in order for everything to work nicely together
provider.setSortKeys("id": Order.ASCENDING)
I had the same issue and got another possible solution.
My table T has a primary key field INTERNAL_ID.
The query in JdbcPagingItemReader was like this:
SELECT INTERNAL_ID, ... FROM T WHERE ... ORDER BY INTERNAL_ID ASC
So, the key is: in some conditions, the query didn't return results, and then, raised the error above No value supplied for...
The solution is:
Check in a Spring Batch decider element if there are rows.
If it is, continue with chunk: reader-processor-writer.
It it's not, go to another step.
Please, note that they are two different scenarios:
At the beginning, there are rows. You get them by paging and finally, there are no more rows. This has no problem and decider trick is not required.
At the beginning, there are no rows. Then, this error raised, and the decider solved it.
Hope this helps.
With jooq 3.4 I can't figure out how to do this (with Postgresql):
Query query = dsl.insertInto(TABLE)
.set(TABLE.ID, Sequences.TABLE_ID_SEQ.nextval());
but in a case when I don't know which is the exact table, something like this:
TableImpl<?> tableImpl;
Query query = dsl.insertInto(tableImpl)
.set(tableImpl.getIdentity(), tableImpl.getIdentity().getSequence().nextval());
Is it somehow possible?
I tried this:
dsl.insertInto(tableImpl)
.set(DSL.field("id"),
tableImpl.getSchema().getSequence("table_id_seq").nextval())
This works but I still don't know how to get the sequence name from the TableImpl object.
Is there a solution for this? Or is there a problem with my approach?
In plain SQL I would do this:
insert into table_A (id) VALUES nextval('table_A_id_seq');
insert into table_B (table_A_id, some_val) VALUES (currval('table_A_id_seq'), some_val);
So I need the value or a reference to that id for later use of the id that was generated for the inserted record as default, but I don't want to set any other values.
jOOQ currently doesn't have any means of associating a table with its implicitly used sequence for the identity column. The reason for this is that the sequence is generated when the table is created, but it isn't formally connected to that table.
Usually, you don't have to explicitly set the serial value of a column in a PostgreSQL database. It is generated automatically on insert. In terms of DDL, this means:
CREATE TABLE tablename (
colname SERIAL
);
is equivalent to specifying:
CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
The above is taken from:
http://www.postgresql.org/docs/9.3/static/datatype-numeric.html#DATATYPE-SERIAL
In other words, just leave out the ID values from the INSERT statements.
"Empty" INSERT statements
Note that if you want to create an "empty" INSERT statement, i.e. a statement where you pass no values at all, generating a new column with a generated ID, you can use the DEFAULT VALUES clause.
With SQL
INSERT INTO tablename DEFAULT VALUES
With jOOQ
DSL.using(configuration)
.insertInto(TABLENAME)
.defaultValues()
.execute();
Returning IDs
Note that PostgreSQL has native support for an INSERT .. RETURNING clause, which is also supported by jOOQ:
With SQL
INSERT INTO tablename (...) VALUES (...) RETURNING ID
With jOOQ
DSL.using(configuration)
.insertInto(TABLENAME, ...)
.values(...)
.returning(TABLENAME.ID)
.fetchOne();
In MySQL, if you specify ON DUPLICATE KEY UPDATE and a row is inserted that would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an UPDATE of the old row is performed. For example, if column a is declared as UNIQUE and contains the value 1, the following two statements have identical effect:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
UPDATE table SET c=c+1 WHERE a=1;
I don't believe I've come across anything of the like in T-SQL. Does SQL Server offer anything comparable to MySQL's ON DUPLICATE KEY UPDATE?
I was surprised that none of the answers on this page contained an example of an actual query, so here you go:
A more complex example of inserting data and then handling duplicate
MERGE
INTO MyBigDB.dbo.METER_DATA WITH (HOLDLOCK) AS target
USING (SELECT
77748 AS rtu_id
,'12B096876' AS meter_id
,56112 AS meter_reading
,'20150602 00:20:11' AS time_local) AS source
(rtu_id, meter_id, meter_reading, time_local)
ON (target.rtu_id = source.rtu_id
AND target.time_local = source.time_local)
WHEN MATCHED
THEN UPDATE
SET meter_id = '12B096876'
,meter_reading = 56112
WHEN NOT MATCHED
THEN INSERT (rtu_id, meter_id, meter_reading, time_local)
VALUES (77748, '12B096876', 56112, '20150602 00:20:11');
There's no DUPLICATE KEY UPDATE equivalent, but MERGE and WHEN MATCHED might work for you
Inserting, Updating, and Deleting Data by Using MERGE
You can try the other way around. It does the same thing more or less.
UPDATE tablename
SET field1 = 'Test1',
field2 = 'Test2'
WHERE id = 1
IF ##ROWCOUNT = 0
INSERT INTO tablename
(id,
field1,
field2)
VALUES (1,
'Test1',
'Test2')
SQL Server 2008 has this feature, as part of TSQL.
See documentation on MERGE statement here - http://msdn.microsoft.com/en-us/library/bb510625.aspx
SQL server 2000 onwards has a concept of instead of triggers, which can accomplish the wanted functionality - although there will be a nasty trigger hiding behind the scenes.
Check the section "Insert or update?"
http://msdn.microsoft.com/en-us/library/aa224818(SQL.80).aspx
If I have a SQL table with columns:
NR_A, NR_B, NR_C, NR_D, R_A, R_B, R_C
and on runtime, I add columns following the column's sequence such that the next column above would be R_D followed by R_E.
My problem is I need to reset the values of columns that starts with R_ (labeled that way to indicate that it is resettable) back to 0 each time I re-run my script . NR_ columns btw are fixed, so it is simpler to just say something like:
UPDATE table set col = 0 where column name starts with 'NR_'
I know that is not a valid SQL but I think its the best way to state my problem.
Any thoughts?
EDIT: btw, I use postgres (if that would help) and java.
SQL doesn't support dynamically named columns or tables--your options are:
statically define column references
use dynamic SQL to generate & execute the query/queries
Java PreparedStatements do not insulate you from this--they have the same issue, just in Java.
Are you sure you have to add columns during normal operations? Dynamic datamodels are most of the time a realy bad idea. You will see locking and performance problems.
If you need a dynamic datamodel, take a look at key-value storage. PostgreSQL also has the extension hstore, check the contrib.
If you don't have many columns and you don't expect the schema to change, just list them explicitly.
UPDATE table SET NR_A=0;
UPDATE table SET NR_B=0;
UPDATE table SET NR_C=0;
UPDATE table SET NR_D=0;
Otherwise, a simple php script could dynamically build and execute your query:
<?php
$db = pg_connect("host=localhost port=5432 user=postgres password=mypass dbname=mydb");
if(!$db) die("Failed to connect");
$reset_cols = ["A","B","C","D"];
foreach ($col in $reset_cols) {
$sql = "UPDATE my_table SET NR_" . $col . "=0";
pg_query($db,$sql);
}
?>
You could also lookup table's columns in Postgresql by querying the information schema columns tables, but you'll likely need to write a plpgsql function to loop over the query results (one row per table column starting with "NR_").
if you rather using sql query script, you should try to get the all column based on given tablename.
maybe you could try this query to get all column based on given tablename to use in your query.
SELECT attname FROM
pg_attribute, pg_type
WHERE typname = 'tablename' --your table name
AND attrelid = typrelid
AND attname NOT IN ('cmin', 'cmax', 'ctid', 'oid', 'tableoid', 'xmin', 'xmax')
--note that this attname is sys column
the query would return all column with given tablename except system column