I am working on a java plugin interfacing with an H2 database. What I really want is an "Insert Ignore" statement; however, I'm aware that H2 doesn't support this. I am also aware of Merge, but this is really not what I want, if the record exists I don't want to change it.
What I am considering is to just run the insert and let the duplicate key exception happen. However, I don't want this to fill my log file. The DB call happens in an imported class that I can't change. So my questions are:
Is this a reasonable thing to do? I'm not one for letting errors happen, but this seems like the best way in this case (it should not happen all that much).
How can I keep this exception from hitting my log file? If there isn't a way to block exceptions down the stack, can I redirect the output of the stack trace that is output?
Thanks.
One solution is to use:
insert into test
select 1, 'Hello' from dual
where not exists(select * from test where id = 1)
This should work for all databases (except for the dual part; you may need to create your own dummy table with one row).
To disable logging exceptions, append ;trace_level_file=0 to the database URL:
jdbc:h2:~/test;trace_level_file=0
or run the SQL statement:
set trace_level_file 0
Related
I have a ton of raw html files that I'm parsing and inserting to a MySQL database via a connection in Java.
I'm using "REPLACE INTO" statements and this method:
public void migrate(SomeThread thread) throws Exception{
PreparedStatement threadStatement = SQL.prepareStatement(threadQuery);
thread.prepareThreadStatement(threadStatement);
threadStatement.executeUpdate();
threadStatement.close();
for(SomeThread.Post P : thread.threadPosts){
PreparedStatement postStatement = SQL.prepareStatement(postQuery);
P.preparePostStatement(postStatement);
postStatement.executeUpdate();
postStatement.close();
}
}
I am running 3 separate instances of my program each in its own command prompt, with their own separate directory of htmls to parse and commit.
I'm using HeidiSQL to monitor the database and a funny thing is happening where I'll see that I have 500,000 rows in a table at one point for example, then I'll close HeidiSQL and check back later to find that I now have 440,000 rows. The same thing occurs for the two tables that I'm using.
Both of my tables use a primary key called "id", each of their ID's have their own domain but it's possible their values overlap and are overwriting each other? I'm not sure if this could be an issue because I'd think SQL would differentiate between the table's "local" id values.
Otherwise I was thinking it could be that since I'm running 3 separate instances that each have their connection to the DB, some kind of magic is happening where right as one row is being committed, the execution swaps to another commit statement, displaces the table, then back to the first commit and then some more magic that causes the database to roll back the number of rows collected.
I'm pretty new to SQL so I'm not too sure where to start, if somebody has an idea about what the heck is going on and could point me in the right direction I'd really appreciate it.
Thanks
You might want to use INSERT INTO instead of REPLACE INTO.
Data doesn't disappear.
Here are some tips:
Do you have another thread running that actually deletes entries?
Do other people have access to the database?
Not sure what HeidiSQL may do. To exclude that possibility maybe use MySQL Workbench instead.
Yeah now that I run a COUNT(*) query against my tables I see that all my rows are in fact there.
Most likely the heidiSQL summary page is just a very rough estimate.
Thanks for the suggestion to use workbench pete, I will try it and see if it is better than Heidi as Heidi is freezing up on me on a regular basis.
I'm running a Java application with Spring and I am getting an error on one of my insert statements. My error is:
nested exception is java.sql.SQLIntegrityConstraintViolationException:
ORA-01400: cannot insert NULL into ("MY_SCHEMA"."VALIDATION_RESULT"."RESULT_SEQ")
For all the database guys, is there ever a scenario that Oracle would return null from a nextval call? What about if multiple threads are calling it simultaneously?
For any Spring developers, we're using
org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer
to handle the sequence. We use the nextLongValue method.
My gut here is telling me that Oracle isn't giving me a null nextval. From everything I've already searched for, that seems impossible. Can anyone confirm?
Confirmed. They do not return NULL. You get an error message.
Oracle sequences actually generate a block of "nextval" objects so threads can quickly access them. You can alter the sequences to create larger readahead numbers of values if it is performance obstacle. The only possibility is if Oracle is seriously broken. Get your DBA to look in the alert log. Errors like ORA-06nn errors are a DBA's nightmare and are the only thing I am aware of that actually breaks objects like sequences.
In this case the DB and probably the DBA, too, are close to DOA. This kind of thing happens once in a career.
I would suspect your code first. Or someone tinking with the sequences - like doing something stupid with ALTER SEQUENCE. i.e., restarting the sequence from one and breaking table constraints. It is also easy to get things screwed up sequence-wise when you export only table from database DEV -> import to database TEST, because the other metadata needs to be brought over as well.
Recently my team have get a situation in which some records in our shared test database disappear with no clear reason. Because it's a shared database (which is utilized by so many teams), so that we can't track down if it's a programming mistake or someone just run a bad sql script.
So that I'm looking for a way to notify (at database level) when a row of a specific table A get deleted. I have looked at the Postgres TRIGGER, but it failed to give me the specific sql that cause the deletion.
Is there anyway I can log the sql statement which cause the deletion of some rows in table A?
You could use something like this.
It allows you to create a special triggers for PostgreSQL tables, that log all the changes to the chosen tables.
This triggers can log the query, that cause the change (via current_query()).
Using this as a base you can add more fields/information to log.
You would do this to the actual postgres config files:
http://www.postgresql.org/docs/9.0/static/runtime-config-logging.html
log_statement (enum)
Controls which SQL statements are logged. Valid values are none (off), ddl, mod, and all (all statements). ddl logs all data
definition statements, such as CREATE, ALTER, and DROP statements. mod
logs all ddl statements, plus data-modifying statements such as
INSERT, UPDATE, DELETE, TRUNCATE, and COPY FROM. PREPARE, EXECUTE, and
EXPLAIN ANALYZE statements are also logged if their contained command
is of an appropriate type. For clients using extended query protocol,
logging occurs when an Execute message is received, and values of the
Bind parameters are included (with any embedded single-quote marks
doubled).
The default is none. Only superusers can change this setting.
You want either ddl or all to be the selection. This is what you need to alter:
In your data/postgresql.conf file, change the log_statement setting to 'all'. Further the following may also need to be validated:
1) make sure you have turned on the log_destination variable
2) make sure you turn on the logging_collector
3) also make sure that pg_log actually exists relative to your data directory, and that the postgres user can write to it.
taken from here
I have a list of strings that contain valid SQL expressions.
I need to execute only those that do not modify the database.
What would be the best way to do this? Just doing something like:
if(sqlQuery.contains("DELETE")){
//don't execute this
}
seems like a bad hack
Update:
I'll make this more specific.
I already have a list of SQL queries that are allowed. I want to make sure only these are executed.
What would be the best way to match against these?
The easiest and best (most comprehensive) way to do this is to create a read-only user and only connect to the database with that user. In SQLServer, the easiest way to do this is to create the user and add them to the built-in "db_datareader" role. This will only allow SELECTs.
And you have to worry about more than just DELETEs, INSERTs or UPDATEs. You also have to be careful about calling any stored procedures, so to be safe you'd also want to remove execute rights, ALTER rights, GRANT rights, etc...
EDIT:
Just execute this...
CREATE LOGIN [user] WITH PASSWORD='password', DEFAULT_DATABASE=[your_db], CHECK_POLICY=OFF
GO
CREATE USER [user] FOR LOGIN [user]
EXEC sp_addrolemember N'db_datareader', N'your_db'
GO
DELETE is not the only SQL instruction that might modify your database; INSERT will definitely do so, and UPDATE might (depending on your exact query). So just analysing the Strings might be a hard way of doing this.
As long as performance is not really an issue, you could start a transaction, run your instructions one by one, check the number of affected rows for each of them, and finally rollback your transaction. Afterwards, you only run those statements that affected 0 rows.
Besides, check your database documentation: some RDBMS-es (like Oracle) don't support rollback of DDL statements like ALTER TABLE, DROP TABLE and the like...
I don't think there's a bulletproof way of preventing the alteration of records by simply checking the content of the given SQL. For example, you might have a field, which has the value "update" and some user is trying to query all rows which contain this value, yet the SQL would not be executed, since it contains a "blacklisted" string.
I guess the only safe way would be to execute the SQL's with an user, who has no rights to alter records at all.
I am working on a java plugin interfacing with an H2 database. What I really want is an "Insert Ignore" statement; however, I'm aware that H2 doesn't support this. I am also aware of Merge, but this is really not what I want, if the record exists I don't want to change it.
What I am considering is to just run the insert and let the duplicate key exception happen. However, I don't want this to fill my log file. The DB call happens in an imported class that I can't change. So my questions are:
Is this a reasonable thing to do? I'm not one for letting errors happen, but this seems like the best way in this case (it should not happen all that much).
How can I keep this exception from hitting my log file? If there isn't a way to block exceptions down the stack, can I redirect the output of the stack trace that is output?
Thanks.
One solution is to use:
insert into test
select 1, 'Hello' from dual
where not exists(select * from test where id = 1)
This should work for all databases (except for the dual part; you may need to create your own dummy table with one row).
To disable logging exceptions, append ;trace_level_file=0 to the database URL:
jdbc:h2:~/test;trace_level_file=0
or run the SQL statement:
set trace_level_file 0