I have an oracle query which i am trying to execute using jdbc. Following is the query.
insert into bd_vehicles_temp select * from bd_vehicles_temp_1
table bd_vehicles_temp_1 contains about 7000-10000 rows. If a primary key in bd_vehicles_temp_1 is already present in bd_vehicles_temp i get an SQLException : Unique key constraint.
the line of exception is offcourse pstmt.executeUpdate() in my code. Is there a way to pinpoint the row in bd_vehicles_temp_1 which causes exception.
Or do i have to do loop through rows in bd_vehicles_temp_1 and insert each row one by one ?
Thanks in Advance !
The only way (I know of) to find out which row causes the problem is to use Oracle's "log errors into" feature. That way the insert won't throw an exception and any row violating any constraint will be written into the error table specified.
To do that you first need to create a log table that holds the rejected rows:
EXECUTE DBMS_ERRLOG.CREATE_ERROR_LOG('BD_VEHICLES_TEMP');
That will create a table named ERR$_BD_VEHICLES_TEMP
Then run change your statement to this:
insert into bd_vehicles_temp
select *
from bd_vehicles_temp_1
LOG ERRORS REJECT LIMIT UNLIMITED;
The statement will continue even if a row fails to validate the constraints. After the statement is finished you can check the contents of the table ERR$_BD_VEHICLES_TEMP for the rows that violated a constraint including the error message and the values.
(Edit): If you want to stop at the first error (and see that in the log table), then leave out the REJECT LIMIT UNLIMITED clause.
More details are in the manual:
for DBMS_ERRLOG http://docs.oracle.com/cd/B28359_01/appdev.111/b28419/d_errlog.htm#CEGBBABI
for the LOG ERRORS INTO clause: http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_9014.htm#BGBEIACB
If you know the column that can cause the exception you can use (Oracle specific)
SELECT col FROM bd_vehicles_temp
INTERSECT
SELECT col FROM bd_vehicles_temp_1;
to identify all rows that are in both tables.
You are trying to insert into a table right? You should be using pstmt.executeUpdate() instead of pstmt.execute(). If there are already existing records in your table, then its better to delete all rows and add again if this statement is executed more than once.
Related
I would like to do a real time reading from mysql.
The idea is simple. I use the binary log to trigger the select statement.
Meanwhile I'd like to read only the new rows on every change.
And currently I just consider insert.
So when someone do
insert into sometable(uid,somecolumn) values(uid,something)
My code will be triggered and do
select from sometable where uid=uid
Of course I have already written down which columns are the primary key because it seems no information from binlog.
I cannot find a tool to analysis mysql insert statement. So I use the regex to find out which column equals which value, then extract primary keys.
BUT the real problems what will happen if I do
Insert into `table` (`col`) values (select 0 as `col` from `dummy`);
How can I find out the col=0?
Is it impossible that make a select statement that select the new changed rows, triggered by the insert statement?
In a TRIGGER, you have access to the OLD and NEW values. With them, you can write code (in the TRIGGER) to log, for example, just the changes. Something like...
IF NEW.col1 != OLD.col1 THEN INSERT INTO LOG ...; END;
IF NEW.col2 != OLD.col2 THEN INSERT INTO LOG ...; END;
Is it possible to restart the ID column of an HSQLDB after rows were inserted? Can I even set it to restart at a value lower than existing IDs in the table?
The Situation
I have a simple Java program which connects to a HSQLDB like so:
DriverManager.getConnection("jdbc:hsqldb:file:" + hsqldbPath, "", "");
This gives me an HsqlException when executing the following script (this is an excerpt, the complete script for HSQLDB 2.2.4 can be found here):
SET SCHEMA PUBLIC
CREATE MEMORY TABLE PUBLIC.MAP(
ID BIGINT GENERATED BY DEFAULT AS IDENTITY(START WITH 0) NOT NULL PRIMARY KEY,
FOO VARCHAR(16) NOT NULL)
ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 1
// [...]
SET SCHEMA PUBLIC
INSERT INTO MAP VALUES(1,'Foo')
INSERT INTO MAP VALUES(2,'Bar')
ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 42
The message is:
HsqlException: error in script file: ALTER TABLE PUBLIC.MAP ALTER COLUMN ID RESTART WITH 42
The exception goes away when I move the RESTART-command before the INSERTs. The documentation gives no hint as to why that would be necessary.
I will eventually have to make this work on version 2.2.4 but have the same problem with the current version 2.3.2.
Background
What I am trying to do here is to recreate a situation which apparently occurred in production: An unlucky interaction with the database (I don't know what exactly happened) seems to have caused newly inserted rows to collide with existing ones because they were issued the same IDs. I want to create a test replicating the scenario in order to write a proper fix.
The .script file of the database follows a predefined order for the statements. This shouldn't be altered if it is edited and only certain manual changes are allowed (see the guide for details).
You can execute the ALTER TABLE statement via JDBC at the start of your test instead of inserting it in the script.
If IDENTITY values for the PRIMARY KEY collide, you will get an exception when you insert the values.
The actual fix for a problem like this is to RESTART WITH the max value in the primary key column plus one.
I think SEQUENCES are much more flexiblee than IDENTITY. The IDENTITY generator disabled JDBC batching, by the way.
But if you use SEQUENCE identifiers, you must pay attention to the hilo optimizers as well, because identifier are generated by Hibernate using a sequence value as a base calculation starting point.
With a SEQUENCE the restart goes like this:
ALTER SEQUENCE my_seqeunce RESTART WITH 105;
I am implementing application specific data import feature from one database to another.
I have a CSV file containing say 10000 rows. These rows need to be inserted/updated into database.
I am using mysql database and inserting from Java.
There might be the case, where couple of rows may present in database that means those need to be updated. If not present in database, those need to be inserted.
One possible solution is that, I can read one by one line, check the entry in database and build insert/update queries accordingly. But this process may take much time to create update/insert queries and execute them in database. Some times my CSV file may have millions of records.
Is there any other faster way to achieve this feature?
I don't know how you determine "is already present", but if it's any kind of database level constraint (probably on a primary key?) you can make use of the REPLACE INTO statement, which will create a record unless it gets an error in which case it'll update the record that prevents it from being inserted.
It works just like INSERT basically:
REPLACE INTO table ( id, field1, field2 )
VALUES ( 1, 'value1', 'value'2 )
If a row with ID 1 exists, it's updated with these values; otherwise it's created.
Given that you're using MySQL you could use the INSERT ... ON DUPLICATE KEY UPDATE ... statement, which functions similarly to the SQL standard MERGE statement. MYSQL doc reference here and general Wikipedia reference to SQL MERGE functionality here. The statement would look something like
INSERT INTO MY_TABLE
(PRIMARY_KEY_COL, COL2, COL3, COL4)
VALUES
(1, 2, 3, 4)
ON DUPLICATE KEY
UPDATE COL2 = 2,
COL3 = 3,
COL4 = 4
In this example I'm assuming that PRIMARY_KEY_COL is a primary or unique key on MY_TABLE. If the INSERT statement would fail due to a duplicate value on the primary or unique key then the UPDATE clause is executed. Also note (on the MySQL doc page) that there are some gotcha's associated with auto-increment columns on an InnoDB table.
Share and enjoy.
Do you need to do this often or just once in a while?
I need to load csv files from time to time to a database for analysis and I created a SSIS-Datasolution with a Data Flow task which loads the csv-File into a table on the SQL Server.
For more infos look at this blog
http://blog.sqlauthority.com/2011/05/12/sql-server-import-csv-file-into-database-table-using-ssis/
Add a stored procedure in SQL for inserting. In the stored procedure use a try catch block to do the insert. If the insert fails do an update. Then you can simply call this method from your program.
Alternatively:
UPDATE Table1 SET (...) WHERE Column1='SomeValue'
IF ##ROWCOUNT=0
INSERT INTO Table1 VALUES (...)
When i'm trying to delete a last row in a table using a PreparedStatement (I'm using a MySQL database), the row wasn't been deleted, I tried to use a DELETE FROM... command and a TRUNCATE command (using the executeUpdate() command), but none of those commands deleted that last row, What should I do, in order to be able to delete that last row?
this is the command i have written:
String sqlDelete = "DELETE FROM free_time WHERE therapist_id=? AND from=? AND to=? AND date=?";
And I have checked that the parameters that i'm sending to the Prepared Statement are correct, but still, the row isn't been deleted.
Thanks in advanced.
Last row doesn't mean anything in a relational database. You don't need to know how the rows are stored.
You should be using a WHERE clause to identify what you need to DELETE.
I wouldn't recommend TRUNCATE.
You are using key words, and obviously suppressing exceptions ;).
String sqlDelete = "DELETE FROM free_time WHERE therapist_id=? AND `from`=? AND `to`=? AND `date`=?";
I have a program that has 2 threads running, and each thread has its own database JDBC connection, and they will access/modify the same database table A like below. Table A only has 2 columns (id, name), and the primary key is the combination of id and name.
statement stmt;
// first delete it if the record has exist in table
stmt.addBatch("delete from A where id='arg_id' and name='arg_name';");
// then insert it to table
stmt.addBatch("insert into A values (arg_id, arg_name);");
stmt.executeBatch();
The 2 threads maybe insert the same data to the table, and i got the following exception,
java.sql.BatchUpdateException: Duplicate entry '0001-joey' for key 1
at com.mysql.jdbc.Statement.executeBatch(Statement.java:708)
at com.mchange.v2.c3p0.impl.NewProxyStatement.executeBatch(NewProxyStatement.java:743)
at proc.Worker.norD(NW.java:450)
Do you have any idea how I can fix this issue? Thank you.
Regards,
Joey
Why not introduce a simple optimistic locking mecanism on the database?
Add a version column and track the version number when performing delete or update transactions.
Your table would look like
create table test(
id int not null primary key,
name varchar,
rowversion int default = 0);
Every time you retrieve a row you should retrieve the row version so you can do
update test set name='new name' rowversion=rowversion+1 where id=id and rowversion=retrieved row version;
The same with delete
delete from test where id=id and rowversion=retrievedRowVersion;
This is a simple mechanism that will exploit the dbms concurency management features. Check this link for more information on optimistic locking http://en.wikipedia.org/wiki/Optimistic_concurrency_control#Examples
This is obviously only a very simple implementation of concurency management but your problem has to take these into account.
Also for the double insert the fact that your transaction is rejected is good that means that no duplicate keys are inserted. You should just handle the Exception and notify the user of what happen.
Wrap both statements in a transaction:
BEGIN;
DELETE FROM a WHERE ...;
INSERT INTO a VALUES (...);
COMMIT;
Note that as long as the table consists of only the primary key, this conflict arises only when the table is unmodified at the end; I presume you want to add more columns, in which case you should use the UPDATE ... WHERE syntax to change values.
Are you using any kind of synchronization? First you will need to wrap the code that modifies the table in:
synchronized(obj)
{
// code
}
where obj is an object that both threads can access.
I don't know the exact semantics of your table modifications, but if they both insert ids, you will also need to hold a "global" id and atomically increment it in each thread, such that they don't both get the same value.