The join is done on the primary key column of both these tables.
I have a doubt if I should fire a select query before the update or will this query be a good alternative?(in terms of performance)
order item table
order_item_id
order_id
quantity
unit_price
shipping_price
business_id
workflow_id
delivery_id
item_id
Orders table
billing_address_id
shipping_address_id
payment_mode
total_price
shipping_price
customer_id
order_id
Following is the query I fire from my Java service (using jdbc) :
UPDATE order_items t1
INNER
JOIN Orders t2
ON t2.order_id = t1.order_id
SET t1.workflow_id = ?
WHERE t1.order_item_id = ?
and t2.order_id = ?
and t2.customer_id = ?
and t1.delivery_id = ?
UPDATE : Adding show create table order_items
'CREATE TABLE `order_items` (
`order_item_id` int(20) NOT NULL AUTO_INCREMENT,
`quantity` int(10) unsigned NOT NULL,
`unit_price` int(10) unsigned NOT NULL,
`shipping_price` int(10) unsigned NOT NULL,
`pickup_date` datetime DEFAULT NULL,
`create_TS` datetime DEFAULT CURRENT_TIMESTAMP,
`update_TS` datetime DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
`business_id` int(10) NOT NULL,
`order_id` int(11) NOT NULL,
`item_id` int(10) unsigned NOT NULL,
`delivery_id` int(11) NOT NULL,
`workflow_id` int(11) DEFAULT NULL,
PRIMARY KEY (`order_item_id`),
KEY `fk_business_id` (`business_id`),
KEY `fk_order_id` (`order_id`),
KEY `fk_item_id` (`item_id`),
KEY `fk_delivery_id` (`delivery_id`),
CONSTRAINT `fk_business_id` FOREIGN KEY (`business_id`) REFERENCES `business` (`MID`),
CONSTRAINT `fk_delivery_id` FOREIGN KEY (`delivery_id`) REFERENCES `delivery_mode` (`delivery_id`),
CONSTRAINT `fk_item_id` FOREIGN KEY (`item_id`) REFERENCES `item_business` (`item_id`),
CONSTRAINT `fk_order_id` FOREIGN KEY (`order_id`) REFERENCES `Orders` (`order_id`)
)
Talking in theory
You should have the minimum set of data before you do the join, so the join will actually be performed only on the data you need, and that is the case even with the update that is internally a special select and "write this data on the select"
Talking in practice
One of the job of any dbms is to perform an agressive level of optimization using database algebra and other stuff, so most of the time the time you spend in optimizing your query is actually futile because your dbms will perform the same level of optimization
So what
I would try to have the table the slimmest as possible but without getting too crazy, I performed on a aws db2.micro machine an update query on like 100k rows and it took it like 4 seconds, so in my opinion, try and see if you're getting the real result you need.
tl;dr just try and see if the speed increase
Related
I'm using Spring boot along with Hibernate. I've only recently started using Java so I'm not quite good at it.
I have a OneToMany unidirectional relationship with a join table.
RssUrl Table
CREATE TABLE `rss_url` (
`id` BIGINT(19) NOT NULL,
`created_at` DATETIME(6) NOT NULL,
`updated_at` DATETIME(6) NULL DEFAULT NULL,
`url` VARCHAR(255) NULL DEFAULT NULL COLLATE 'utf8mb4_unicode_ci',
PRIMARY KEY (`id`) USING BTREE
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
;
User Table
CREATE TABLE `user` (
`id` BIGINT(19) NOT NULL,
`created_at` DATETIME(6) NOT NULL,
`updated_at` DATETIME(6) NULL DEFAULT NULL,
`email` VARCHAR(255) NULL DEFAULT NULL COLLATE 'utf8mb4_unicode_ci',
`is_active` BIT(1) NULL DEFAULT NULL,
`password` VARCHAR(255) NULL DEFAULT NULL COLLATE 'utf8mb4_unicode_ci',
PRIMARY KEY (`id`) USING BTREE,
UNIQUE INDEX `email_unique_constraint_1873213` (`email`) USING BTREE
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
;
Join table
CREATE TABLE `user_rssurl` (
`user_id` BIGINT(19) NOT NULL,
`rss_url_id` BIGINT(19) NOT NULL,
UNIQUE INDEX `UK_paw8syp4ru29oqh7430u2c7vl` (`rss_url_id`) USING BTREE,
INDEX `FK7gp98smro2y75g1026ut00jsf` (`user_id`) USING BTREE,
CONSTRAINT `FK7gp98smro2y75g1026ut00jsf` FOREIGN KEY (`user_id`) REFERENCES `javadb`.`user` (`id`) ON UPDATE NO ACTION ON DELETE NO ACTION,
CONSTRAINT `FKpxci6mn3aayjdpkdigcpekrsy` FOREIGN KEY (`rss_url_id`) REFERENCES `javadb`.`rss_url` (`id`) ON UPDATE NO ACTION ON DELETE NO ACTION
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
;
I'm trying to translate this SQL into HQL
SELECT COUNT(u.id)
FROM `user` u
LEFT JOIN user_rssurl urs ON urs.user_id = u.id
LEFT JOIN rss_url ru ON urs.rss_url_id = ru.id
WHERE u.id = ?
AND urs.rss_url_id = ?
It's basically checking if a url is already associated with a user. I'm sure it's not the most efficient sql out there but it gets the job done(I'm more than happy to hear how I can improve it, but that's besides the point).
So far I got this going for me, which is really not a lot tbh.
#Repository
public interface UserRepository extends JpaRepository<User, Long> {
#Query("SELECT COUNT(u.id) FROM User u LEFT JOIN user_rssurl urs on urs.user_id = u.id LEFT JOIN RssUrl ru on urs.rss_url_id = ru.id WHERE u.id = ?1 AND urs.rss_url_id = ?2")
Integer findInRelation(Long uid, Long rssUrlId);
}
I've also considered alternative methods, such as letting the database error out and catching the exception but that' just seems wrong.
This question has 2 parts:
What is the correct HQL for the above query?
Is there an easier way to do this? I come from a PHP background. Using ORMs like Doctrine made things a lot easier. One could in fact fetch related entities with considerably less hassle. I find it hard to believe that Hibernate doesn't have a better way to do this.
I'd love to google it myself but quite frankly I don't know what to google exactly.
In actual, java jpa is not friendly with join table query; in there, I can give you two methed only for refer:
you can split this query into three base query to complete you question, i know this method is not good;
you can define an entity as a join entity, then use #OneToOne or #ManyToOne anaotations to reflect the relation;
I aslo has the 3 suggestion, not use jpa but use mybatis, in mybatis, you can direct use your sql lile what you write when query with many table;
I am looking for a way to define a set of columns as unique and then insert a new entry into the table or update the row if the columns aren't unique. I have done some research and found ways to do it, but I couldn't find anything that is compatible with both MySQL and SQLite.
Say I have the following table:
CREATE TABLE `users` (
`id` INT NOT NULL AUTO_INCREMENT,
`uuid` VARCHAR ( 64 ) NOT NULL,
`name` VARCHAR ( 32 ) NOT NULL,
`date` BIGINT NULL,
PRIMARY KEY ( id )
);
I want uuid and date to be unique so that there can be multiple entries for one uuid or one date, but not for one combination of uuid and date. What I initially wanted to do is set the primary key to those:
PRIMARY KEY ( uuid, date )
However, for some reason I won't be able to use null values for date when doing this.
I have also found something about a constraint, but I am not sure if this works:
CONSTRAINT user UNIQUE ( `uuid`, `date` )
Now I want to insert a new row into this table, but update the existing row if a row with the same uuid and date already exists. I have found a few ways but they are either not doing what I want or not compatible with both MySQL and SQLite:
INSERT INTO ... ON DUPLICATE KEY doesn't work with SQLite
REPLACE INTO will delete anything I don't specify instead of updating
I have been doing research for quite a while but I couldn't find a solution that worked for me. Any help appreciated.
SQLite solution (same principle should apply in mysql)
You could simply add a UNIQUE index (at least for SQLite for which this is for) so you could have :-
DROP TABLE IF EXISTS `users`;
CREATE TABLE IF NOT EXISTS `users` (
`id` INTEGER, //<<<<<<<<<< See notes below
`uuid` VARCHAR ( 64 ) NOT NULL,
`name` VARCHAR ( 32 ) NOT NULL,
`date` BIGINT NULL,
PRIMARY KEY ( `id` )
);
CREATE UNIQUE INDEX IF NOT EXISTS uuid_date ON `users` (`uuid`,`date`); //<<<<<<<<<<
Note AUTO_INCREMENT results in a failure for SQLite as it's not a keyword, the correct keyword in SQlite is AUTOINCREMENT. However, it's been omitted as it's probably not required as INTEGER PRIMARY KEY (or the implicit by specifiying PRIMARY KEY (id)) will result in a uniqiue id being automatically generated if no value is supplied for the column when inserting.
SQLite requires INTEGER, not INT, for the automatically generated id. NOT NULL and also UNIQUE are implied so no need to specify them.
Here's two sets of example inserts each duplicating the uuid/date combination thus updating instead of inserting and also inserting with same uuid but different date and vice-versa :-
INSERT OR REPLACE INTO `users` VALUES(null,'Fred01234567','Fred Bloggs the 1st','20180101');
INSERT OR REPLACE INTO `users` VALUES(null,'Fred01234567','Fred Bloggs the 2nd','20180101'); -- <<<< DUPLICATE
INSERT OR REPLACE INTO `users` VALUES(null,'Fred99999999','Fred Bloggs the 2nd','20180101'); -- <<<< different uuid same date
INSERT OR REPLACE INTO `users` VALUES(null,'Fred01234567','Fred Bloggs the 2nd','99999999'); -- <<<< same uuid different date
INSERT OR REPLACE INTO `users` (`uuid`,'name','date') VALUES('Fred76543210','Fred NotBloggs the 1st','20180202');
INSERT OR REPLACE INTO `users` (`uuid`,'name','date') VALUES('Fred76543210','Fred NotBloggs the 1st','20180202');
INSERT OR REPLACE INTO `users` (`uuid`,'name','date') VALUES('Fred99999999','Fred NotBloggs the 1st','20180202');
INSERT OR REPLACE INTO `users` (`uuid`,'name','date') VALUES('Fred76543210','Fred NotBloggs the 1st','99999999');
SELECT * FROM `users`;
Results are :-
I have been googling for a few hours and did some testing with both MySQL and SQLite and I think I found a working solution.
To make the combination of uuid and date unique, I have added a unique constraint to the table. To insert a new row or 'update' an existing row, I am using REPLACE INTO ... SELECT.
To create the table:
CREATE TABLE IF NOT EXISTS `users` (
`id` INT NOT NULL AUTO_INCREMENT, // use INTEGER NOT NULL for SQLite
`uuid` VARCHAR ( 64 ) NOT NULL,
`name` VARCHAR ( 32 ) NOT NULL,
`date` BIGINT NULL,
CONSTRAINT `uuid_date` UNIQUE ( `uuid`, `date` ),
PRIMARY KEY ( `id` )
);
The CONSTRAINT will make sure the combination of uuid and date is always unique.
For inserting data, I use REPLACE INTO ... SELECT, where I enter all (new) values in the SELECT query and enter the column names for all columns I haven't specified a value for, to ensure it will keep their values intact rather than deleting them when the existing row is replaced.
REPLACE INTO `users`
SELECT `id`, `uuid`, ?, `date`
FROM `users` WHERE `uuid` = ? AND `date` = ?;
Of course, because in this case there are no columns that can be lost when using a normal REPLACE INTO, so I could also use:
REPLACE INTO `users` ( `uuid`, `name`, `date` ) VALUES ( ?, ?, ? );
However, the REPLACE INTO ... SELECT query can be useful when I have a table with more columns and there are actually columns that can be lost when not selecting them.
Sorry about all the comments. Here is how I achieved what I think you are going for. You are going to lose the id as Primary Key on your users table. You will also have to stage your insert variables in a table. But you will get your end results. Sorry I do not have a better solution.
DROP TABLE users;
DROP TABLE users2;
CREATE TABLE `users` (
`uuid` VARCHAR ( 64 ) NOT NULL,
`name` VARCHAR ( 32 ) NOT NULL,
`date` BIGINT NULL
);
ALTER TABLE `users` ADD PRIMARY KEY(`uuid`,`date`);
INSERT INTO users (`name`,`uuid`,`date`) SELECT '','123','2018-04-01';
CREATE TABLE `users2` (
`uuid` VARCHAR ( 64 ) NOT NULL,
`name` VARCHAR ( 32 ) NOT NULL,
`date` BIGINT NULL
);
INSERT INTO users2 (`name`,`uuid`,`date`) SELECT 'brad','123','2018-04-01';
REPLACE INTO users SELECT `uuid`,`name`,`date` FROM users2 GROUP BY `uuid`,`date`;
SELECT * FROM users;`
I want to add the database from my jform and there's a column which will be auto incremented, like when i click done, the data will be inserted and a column receipt_no will have a value 1. Next time I click done then this value should be 2 and so on.
So the problem is, i have created a table with receipt_no as the primary key and auto increment, so what should be my query in java, to add the data correctly in the table.
String sql = "insert into table_name values('"++"',...)";
Can you help me in this query?
Step 1: Creating table in MySQL
CREATE TABLE `user_master` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`Firstname` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Step 2: Insert record
INSERT INTO user_master (`Firstname`) values('Vicky');
Step 3: Fetch record
SELECT * FROM user_master;
I can't comment so there is an answer to the comment you posted in your question:
If your table is
CREATE TABLE users(
id INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
firstname VARCHAR(25) NOT NULL,
lastname VARCHAR(25) NOT NULL,
);
You can simply auto_increment the primary by not giving it on your SQL request:
INSERT INTO users(firstname, lastname) VALUES('Steve', 'Jobs');
Java don't have to generate auto increment, it is SQL job :)
I was working with UIs where the user will click the add button to add employees, but when I do it, it gives me an error like this
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`finalpayroll`.`personal_info`, CONSTRAINT `personal_info_ibfk_1`
How would I fix this?? I know I am using a parent key, and its foreign key is the User, and also take note that the parent key has already a data, but it seems my query won't work, why is that? I am using a foreign key with delete cascade and on update cascade so that when I delete a data, all of the child table rows will be deleted, vice versa. here's my key for adding or inserting statements
public void addEmployee(Personal p ,Contact c,Employee e) {
Connection conn = Jdbc.dbConn();
Statement statement = null;
String insert1 = "INSERT INTO personal_info (`First_Name`, `Middle_Initial`, `Last_Name`, `Date_Of_Birth`, `Marital_Status`, `Beneficiaries`) VALUES ('"+p.getFirstName()+"', '"+p.getMiddleInitial()+"'" +
" , '"+p.getLastName()+"', '"+p.getDateOfBirth()+"', '"+p.getMaritalStatus()+"', '"+p.getBeneficiaries()+"')";
try {
statement = conn.createStatement();
statement.executeUpdate(insert1);
statement.close();
conn.close();
JOptionPane.showMessageDialog(null, "Employee Added!!");
} catch(Exception ex) {
ex.printStackTrace();
}
}
Users table:
CREATE TABLE `users` (
`idusers` int(11) NOT NULL AUTO_INCREMENT,
`emp_id` varchar(45) DEFAULT NULL,
`emp_pass` varchar(45) DEFAULT NULL,
PRIMARY KEY (`idusers`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1
Personal_info table:
CREATE TABLE `personal_info` (
`idpersonal_info` int(11) NOT NULL AUTO_INCREMENT,
`First_Name` varchar(45) DEFAULT NULL,
`Middle_Initial` varchar(45) DEFAULT NULL,
`Last_Name` varchar(45) DEFAULT NULL,
`Date_Of_Birth` varchar(45) DEFAULT NULL,
`Marital_Status` varchar(45) DEFAULT NULL,
`Beneficiaries` varchar(45) DEFAULT NULL,
PRIMARY KEY (`idpersonal_info`),
CONSTRAINT `personal_info_ibfk_1`
FOREIGN KEY (`idpersonal_info`)
REFERENCES `users` (`idusers`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
You are trying to insert a record with 6 fields: First_Name, Middle_Initial, Last_Name, Date_Of_Birth, Marital_Status and Beneficiaries. Your schema is currently unknown but none of these fields seem to be a candidate foreign key to id of User table you mentioned. Thus I think there is a default value for that foreign key column and that default value is missing in User table.
Needless to say, you shouldn't have a default value for a foreign key of any table..
I am adding these information regarding your questions in comments and update on your question:
A foreign key is a link between a child table and parent table, personal_info and users tables in your case respectively. Child table's foreign key column must reference to a key value in parent table which means that for every value in child table's FK column, there must be a value in parent table's linked column.
Now, in your case when you try to insert a new personal_info record MySQL assigns a idpersonal_info to it, since you defined it as auto increment. But since there is a link to users table, MySQL searchs for the new idpersonal_info to be inserted in users table's idusers column. And as you are getting this exception, you surely don't have that value in the users table.
You can change your table structure as follows:
CREATE TABLE `personal_info` (
`idpersonal_info` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
... OTHER FIELD DEFINITIONS,
PRIMARY KEY (`idpersonal_info`),
CONSTRAINT `user_id_fk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`idusers`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB
And your query will need to include user_id field as well. So it will be something like this:
INSERT INTO personal_info
(`user_id`, `First_Name`, `Middle_Initial`, `Last_Name`, `Date_Of_Birth`, `Marital_Status`, `Beneficiaries`)
VALUES ( .... SET YOUR VALUES HERE. DON'T FORGET TO SET A VALID USER_ID
Looks like in your Personal_Info table you have a column called "finalpayroll", that points to a column in another table (a foreign key) and it's required (not nullable). In your insert you're not giving it a value. So what you could do is make that column nullable.
Or could be the other way around as #Konstantin Naryshkin is saying
What the error means is that you are trying to insert a value into a column with a foreign key a value that is not in the remote table.
I assume that there is a user column that we are not seeing. Since you are not explicitly setting the value, I assume that it is getting a default. The default value is not in the parent table.
i have two tables where in the first one i have 14 millions and in the second one i have 1.5 million of data.
So i wonder how could i transfer this data to another table to be normalized ?
And how do i convert some type to another, for example: i have a field called 'year' but its type is varchar, but i want it an integer instead, how do i do that ?
I thought about do this using JDBC in a loop while from java, but i think this is not effeciently.
// 1.5 million of data
CREATE TABLE dbo.directorsmovies
(
movieid INT NULL,
directorid INT NULL,
dname VARCHAR (500) NULL,
addition VARCHAR (1000) NULL
)
//14 million of data
CREATE TABLE dbo.movies
(
movieid VARCHAR (20) NULL,
title VARCHAR (400) NULL,
mvyear VARCHAR (100) NULL,
actorid VARCHAR (20) NULL,
actorname VARCHAR (250) NULL,
sex CHAR (1) NULL,
as_character VARCHAR (1500) NULL,
languages VARCHAR (1500) NULL,
genres VARCHAR (100) NULL
)
And this is my new tables:
DROP TABLE actor
CREATE TABLE actor (
id INT PRIMARY KEY IDENTITY,
name VARCHAR(200) NOT NULL,
sex VARCHAR(1) NOT NULL
)
DROP TABLE actor_character
CREATE TABLE actor_character(
id INT PRIMARY KEY IDENTITY,
character VARCHAR(100)
)
DROP TABLE director
CREATE TABLE director(
id INT PRIMARY KEY IDENTITY,
name VARCHAR(200) NOT NULL,
addition VARCHAR(150)
)
DROP TABLE movie
CREATE TABLE movie(
id INT PRIMARY KEY IDENTITY,
title VARCHAR(200) NOT NULL,
year INT
)
DROP TABLE language
CREATE TABLE language(
id INT PRIMARY KEY IDENTITY,
language VARCHAR (100) NOT NULL
)
DROP TABLE genre
CREATE TABLE genre(
id INT PRIMARY KEY IDENTITY,
genre VARCHAR(100) NOT NULL
)
DROP TABLE director_movie
CREATE TABLE director_movie(
idDirector INT,
idMovie INT,
CONSTRAINT fk_director_movie_1 FOREIGN KEY (idDirector) REFERENCES director(id),
CONSTRAINT fk_director_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
CONSTRAINT pk_director_movie PRIMARY KEY(idDirector,idMovie)
)
DROP TABLE genre_movie
CREATE TABLE genre_movie(
idGenre INT,
idMovie INT,
CONSTRAINT fk_genre_movie_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
CONSTRAINT fk_genre_movie_2 FOREIGN KEY (idGenre) REFERENCES genre(id),
CONSTRAINT pk_genre_movie PRIMARY KEY (idMovie, idGenre)
)
DROP TABLE language_movie
CREATE TABLE language_movie(
idLanguage INT,
idMovie INT,
CONSTRAINT fk_language_movie_1 FOREIGN KEY (idLanguage) REFERENCES language(id),
CONSTRAINT fk_language_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
CONSTRAINT pk_language_movie PRIMARY KEY (idLanguage, idMovie)
)
DROP TABLE movie_actor
CREATE TABLE movie_actor(
idMovie INT,
idActor INT,
CONSTRAINT fk_movie_actor_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
CONSTRAINT fk_movie_actor_2 FOREIGN KEY (idActor) REFERENCES actor(id),
CONSTRAINT pk_movie_actor PRIMARY KEY (idMovie,idActor)
)
UPDATE:
I'm using SQL Server 2008.
Sorry guys i forgot to mention that are different databases :
The not normalized is call disciplinedb and the my normalized call imdb.
Best regards,
Valter Henrique.
If both tables are in the same database, then the most efficient transfer is to do it all within the database, preferably by sending a SQL statement to be executed there.
Any movement of data from the d/b server to somewhere else and then back to the d/b server is to be avoided unless there is a reason it can only be transformed off-server. If the destination is different server, then this is much less of an issue.
Though my tables were dwarfs compared to yours, I got over this kind of problem once with stored procedures. For MySQL, below is a simplified (and untested) essence of my script, but something similar should work with all major SQL bases.
First you should just add a new integer year column (int_year in example) and then iterate over all rows using the procedure below:
DROP PROCEDURE IF EXISTS move_data;
CREATE PROCEDURE move_data()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE orig_id INT DEFAULT 0;
DECLARE orig_year VARCHAR DEFAULT "";
DECLARE cur1 CURSOR FOR SELECT id, year FROM table1;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
PREPARE stmt FROM "UPDATE table1 SET int_year = ? WHERE id = ?";
read_loop: LOOP
FETCH cur1 INTO orig_id, orig_year;
IF done THEN
LEAVE read_loop;
END IF;
SET #year= orig_year;
SET #id = orig_id;
EXECUTE stmt USING #orig_year, #id;
END LOOP;
CLOSE cur1;
END;
And to start the procedure, just CALL move_data().
The above SQL has two major ideas to speed it up:
Use CURSORS to iterate over a large table
Use PREPARED statement to quickly execute pre-known commands
PS. for my case this speeded things up from ages to seconds, though in your case it can still take a considerable amount of time. So it would be probably best to execute from command line, not some web interface (e.g. PhpMyAdmin).
I just recently did this for ~150 Gb of data. I used a pair of merge statements for each table. The first merge statement said "if it's not in the destination table, copy it there" and the second said "if it's in the destination table, delete it from the source". I put both in a while loop and only did 10000 rows in each operation at a time. Keeping it on the server (and not transferring it through a client) is going to be a huge boon for performance. Give it a shot!