We Keep Coding

How to do INSERT OR IGNORE with H2 database? [duplicate]

I am working on a java plugin interfacing with an H2 database. What I really want is an "Insert Ignore" statement; however, I'm aware that H2 doesn't support this. I am also aware of Merge, but this is really not what I want, if the record exists I don't want to change it.
What I am considering is to just run the insert and let the duplicate key exception happen. However, I don't want this to fill my log file. The DB call happens in an imported class that I can't change. So my questions are:
Is this a reasonable thing to do? I'm not one for letting errors happen, but this seems like the best way in this case (it should not happen all that much).
How can I keep this exception from hitting my log file? If there isn't a way to block exceptions down the stack, can I redirect the output of the stack trace that is output?
Thanks.

One solution is to use:
insert into test
select 1, 'Hello' from dual
where not exists(select * from test where id = 1)
This should work for all databases (except for the dual part; you may need to create your own dummy table with one row).
To disable logging exceptions, append ;trace_level_file=0 to the database URL:
jdbc:h2:~/test;trace_level_file=0
or run the SQL statement:
set trace_level_file 0

Hibernate produce different SQL for every query

I've just tested my application under the profiler and found out that sql strings use about 30% of my memory! This is bizarre.
There are a lot of strings like this stored in app memory. This is SQL queries generated by hibernate, note the different numbers and trailing underscores:
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=? for update
select avatardata0_.Id as Id4347_0_,...... where avatardata0_.Id=? for update
Here is the part I can't understand. Why does hibernate have to generate different sql strings with different identifiers like "Id4305_0_" for each query? Why can't it use one query string for all identical queries? Is this some kind of trick to bypass query caching?
I would greatly appreciate if someone would describe me why it happening and how to avoid such resource wasting.
UPDATE
Ok. I found it. I was wrong assuming memory leak, It was my fault. Hibernate is working as intended.
My app created 121(!) SessionFactories in 10 threads, they produced about 2300 instances of SingleTableEntityPersisters. And each SingleTableEntityPersister generates about 15 SQL queries with different identifiers. Hibernate was forced to generate about 345.000 different SQL queries. Everything is fine, nothing weird :)

There is a logic behind the query string that hibernate generates. Its primary aim is to get unique aliases for tables and columns names.
From your query,
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=?
avatardata0_ ==> avatardata is the alias of the table and 0_ is appended to indicate it is the first table in the query. So if it were the second table(or Entity) in the query it should have been shown as avatardata1_. It uses the same logic for the column aliases.
So, this way all the possible conflicts are avoided.
You are seeing theses queries because you have turns on the show_sql flag the configuration. This is intended for the debugging of queries. Once you application started working you are supposed turn it off.
Read more on the API docs here.
I am not much aware of the memory consumption part, but you repeat your tests with the above flag turned off and see if there is any improvement.

Assuming you are using sql server, you might want to check the parameter type declaration for '?', making sure the declaration results in the same, fixed length declaration every time.
Dynamic length parameters would result in separate execution plans for each query. This could possibly comsume a lot of resources. What we see as the same procedure, get's interpreted by sql server as a different query, rendering a separate execution plan.
Thus,
exec myprocedure #p1 varchar(3)='foo'
and
exec myprocedure #p1 varchar(6)='foobar'
would result in different plans. Simply by the fact that the declarations of #p1, differ in size.
There is a lot to know about this behaviour. If the above applies to you, I would recommend you read up on 'parameter sniffing'.

No... you can generate you common query inside the hibernate. The logic behind is to mapping with table and fetch the record from there. It is used common query for all the database. Please create a common query like that :
Example :
select t.Id as Id4305_0_,...... from t where t.Id=?

H2 Java Insert ignore - allow exception

I am working on a java plugin interfacing with an H2 database. What I really want is an "Insert Ignore" statement; however, I'm aware that H2 doesn't support this. I am also aware of Merge, but this is really not what I want, if the record exists I don't want to change it.
What I am considering is to just run the insert and let the duplicate key exception happen. However, I don't want this to fill my log file. The DB call happens in an imported class that I can't change. So my questions are:
Is this a reasonable thing to do? I'm not one for letting errors happen, but this seems like the best way in this case (it should not happen all that much).
How can I keep this exception from hitting my log file? If there isn't a way to block exceptions down the stack, can I redirect the output of the stack trace that is output?
Thanks.

One solution is to use:
insert into test
select 1, 'Hello' from dual
where not exists(select * from test where id = 1)
This should work for all databases (except for the dual part; you may need to create your own dummy table with one row).
To disable logging exceptions, append ;trace_level_file=0 to the database URL:
jdbc:h2:~/test;trace_level_file=0
or run the SQL statement:
set trace_level_file 0

big insert query fails Hibernate\Postgresql

I'm trying to find out the root cause of failure in existing system. I don't know much about it, but looks like the issue is in inserting big row into Postregsql via Hibernate.
It fails to insert record w/ TEXT field which is about 50-100k size.
Should not be an issue for postgresql itself. But I guess there might be some settings\parameters in hibernate which can affect it. Any suggestion for the search direction?

First I try to look at the exception,
if it's in your local machine or a
server log, to get more clues. Since
you say it's when inserting a row,
maybe you know where it's happening.
Try inserting a row where the text
field has only a few bytes to see if
that works. Maybe the connection is
slow and inserting more than 50k
causes a timeout followed by a
rollback.
Also check out if that insertion
belongs to a much larger transaction
or it's executing on a smaller one.
Try doing that insertion in plain jdbc (just temporarily) to see if that works and rule out connection issues.
If the problem is not in the connection then you can start tweaking Hibernate parameters. Maybe disabling the 2nd cache. The stack exception or a debugging session will be helpful to know what parameters to change.

Verfying a database is as you expect it it be

I've been writing a java app on my machine and it works perfectly using the DB I set up, but when I install it on site it blows up because the DB is slightly different.
So I'm in the process of writing some code to verify that:
A: I've got the DB details correct
B: The database has all the Tables I expect and they have the right columns.
I've got A down but I've got no idea where to start with B, any suggestions?
Target DB is for the current client is Oracle, but the app can be configured to run on SQL Server as well. So a generic solution would be appreciated, but is not nessisary as I'm sure I can figure out how to do one from the other.

You'll want to query the information_schema of the database, here are some examples for Oracle, every platform I am aware of has something similar.
http://www.alberton.info/oracle_meta_info.html

You might be able to use a database migration tool like LiquiBase for this -- most of these tools have some way of checking the database. I don't have first hand experience using it so it's a guess.

I use DbUnit to test databases. It is a Java based solution, that integrates well with Junit. It is possible to use it with almost no Java. I havent used it in exactly the same situation as you described, but it should be close enough to work.

Most generic solution would be to execute queries with select clause having the expected coulmns and from clause having table names, within try catch block. You can put where clause as 1=2 so as not to fetch any data. If query executed without throwing exception then you have got the expected table and columns.

The slightly different piece might be better handled by scripting the creation of the database in the first place. A automated process gives you a better chance of making the two identical.
Another point worth making is that you minimize your risk by making your devl and prod environments identical - same database schema and vendor for both. Change the circumstances that make the two different.
Lastly, you don't say what is "slightly" different, but sometimes these are unavoidable (e.g. Oracle uses sequences, SQL Server uses identities). Maybe Hibernate can help you to switch between vendors more reliably. It abstracts details in such a way that changing databases can mean modifying a single value in a configuration file.

What you need to have is basically Unit Tests for your database. "A column must exist named FOOBAR, the type must be Integer. No foreign keys may exist etc."
This is doable with plain JUnit and JDBC (ask the table for its meta-data) as you may want to ensure that you are absolutely certain what is being done which may be harder when using e.g. dbUnit.

You can check for the presence of tables, columns, views, etc. using these tables in Oracle
USER_TABLES
USER_VIEWS
USER_PROCEDURE
(or for everything)
USER_OBJECTS WHERE OBJECT_TYPE = '??'
To keep going... USER_TAB_COLS for table columns
Regards
K

I use MigrateDB for this. It lets you build queries that do things like check for the existence of given tables, columns, rows, indexes, etc. for a given database and use those as "tests." If a test fails, it triggers an "action" (which is just another query that knows how to remedy the problem.)
MigrateDB supports multiple database platforms (you can specify the "check for table existence query" for each platform, for example), completely configurable tests (you can make your own up), comes with fairly complete Oracle tests, and can be run in "audit only" mode so that it only tells you what the differences are.
It's a nice, robust solution.

If you're using plain JDBC, you should try utilizing this method: DatabaseMetadata.getTables and other similar methods available in the metadata class.