I want some advice on some concurrency issues regarding jdbc, i basically need to update a value and then retrieve that value using a update then a select, I'm assuming by turning auto commit off no other transaction can access this table, hence other transactions won't be able to perform update and select queries until this has been committed.
Below is some example code. Do you think this will work and does any one else have a better solution to implementing this?
int newVal=-1;
con.setAutoCommit(false);
PreparedStatement statement = con.prepareStatement("UPDATE atable SET val=val+1 WHERE id=?");
statement.setInt(1, id);
int result = statement.executeUpdate();
if (result != 1) {
throw new SQLException("Nothing updated");
} else {
statement = con.prepareStatement("SELECT val FROM atable WHERE id=?");
statement.setInt(1, id);
ResultSet resultSet = statement.executeQuery();
if (resultSet.next()) {
newVal = resultSet.getInt("val");
}
}
statement.close();
con.commit();
con.setAutoCommit(true);
Thanks.
Assuming you use some form of data source, you may configure there if you want transactionality and the isolation level. But to be explicit:
try(Connection con = ds.getConnection()){
con.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
con.setAutoCommit(false);
//...
} catch(SQLException sqle) {
throw new MyModelException(e)
}
Now, you could trigger pesimistic locking by updating a version (or timestamp) field in your table. This will trigger a lock in the database (most likely at the record level):
try(PreparedStatement pStm = con.prepareStatement("update atable set version=version+1")){
pStm.executeUpdate();
}
At this point, if another user is trying to update the same record simultaneously, this connection will either wait or timeout, so you must be ready for both things. The record will not be unlocked until your transaction ends (commit or rollback).
Then, you can safely select and update whatever you want and be sure that nobody else is touching your record as you process your data. If anybody else tries they will be put on wait until you finish (or they will timeout depending on connection configuration).
Alternatively you could use optimistic locking. In this case you read your record, do modifications to it, but in the update you make sure nobody else has changed it since you read it by checking that the version/timestamp field is the same as the one you orginally read. In this case you must be prepared to retry a transaction (or abort it alltogether) if you realize you have stale/outdated data.
i.e. update atable set afield=? where id=? and version=1
If the number of rows affected is 0, then you know that is probable that the record was updated between your read and your update and the record is no longer in version 1.
Setting autocommit=false on your connection will not prevent other connections/threads from changing the row in the database! It will only disable automatic commits after each JDBC operation on that specific connection.
You will need to lock the row, eg. with select ... for update to prevent other transactions against the row, and also you will need to do your selects and updates within a single transaction.
Cheers,
Related
I have to execute multiple insert queries using JDBC for which I am trying to execute batch statement. Everything works fine in my code but when i try to see values in the table, the table is empty.
Here is the code :
SessionImpl sessionImpl = (SessionImpl) getSessionFactory().openSession();
Connection conn = (Connection) sessionImpl.connection();
Statement statement = (Statement) conn.createStatement();
for (String query : queries) {
statement.addBatch(query);
}
statement.executeBatch();
statement.close();
conn.close();
And the
List<String> queries
contains insert queries like:
insert into demo values (null,'Sharmzad','10006','http://demo.com','3 Results','some values','$44.00','10006P2','No Ratings','No Reviews','Egypt','Duration: 8 hours','tour','Day Cruises');
And the table structure is like:
create table demo ( ID INTEGER PRIMARY KEY AUTO_INCREMENT,supplierName varchar(200),supplierId varchar(200),supplierUrl varchar(200),totalActivities varchar(200),activityName varchar(200),activityPrice varchar(200),tourCode varchar(200),starRating varchar(200),totalReviews varchar(200),geography varchar(200),duration varchar(200),category varchar(200),subCategory varchar(200));
No exception is thrown anywhere but no value is inserted. Can someone explain?
Most JDBC drivers use autocommit, but some of them do not. If you don't know, you should use either .setAutoCommit(true) before the transaction or .commit() after it..
Could be a transaction issue. Perhaps you're not committing your transaction? If so, then it is normal not to see anything in the database.
You can check if this is the case by running a client in READ_UNCOMMITTED transaction mode, right after .executeBatch(); (but before close()) and see if there are any rows.
You don't should assign a value to ID add supply all the others columns name
insert into demo
(
supplierName
,supplierId
,supplierUrl
,totalActivities
,activityName
,activityPrice
,tourCode
,starRating
,totalReviews
,geography
,duration
,category
,subCategory
)
values (
'Sharmzad'
,'10006'
,'http://demo.com'
,'3 Results'
,'some values'
,'$44.00'
,'10006P2'
,'No Ratings'
,'No Reviews'
,'Egypt'
,'Duration: 8 hours
','tour'
,'Day Cruises'
);
and add commit to your code
Need help in JDBC transaction control mechanism in JAVA.
Issue:
There are certain stored procedures in our Sybase DB that needs to be run on Unchained mode. Since we are updating our data on two different databases (unfortunately, both Sybase) we need to be able to rollback all our previous transactions, if there is any failure.
But running with Unchained Mode (Auto commit - on) is not helping us with the rollbacks as some of the SPs have already committed the transactions.
Connection connection = getConnection();
PreparedStatement ps = null;
try{
String sql = getQuery(); // SQL Chained Mode
ps = connection.prepareStatement(sql);
ps.executeUpdate(); //Step 1
.
.
sql = getTransctionQuery(); // SQL Unchained Mode
connection.setAutoCommit(true); //Step 2
ps = connection.prepareStatement(sql);
ps.executeUpdate();
connection.setAutoCommit(false);
.
.
sql = getQuery(); // SQL Chained Mode
ps = connection.prepareStatement(sql);
ps.executeUpdate(); //Step 3 This step fails.
connection.commit();
}catch(){
connection.rollback(); //Doesn’t rollback step 1 and understandably step 2.
}
finally{
connection.close(); //cleanup code
}
We would ideally like to rollback both step 1 and step 2 effectively if 3 fails.
Current Solution:
Our idea is to reinvent the wheel and write our own version of rollback (by deleting inserted records and reverting the updated values, from Java).
Need effective solution
Since this solution is effort intensive and not fool proof we would like to know if there are any other better solutions.
Thanks
You need to perform an explicit BEGIN TRANSACTION statement. Otherwise, every DML is a transaction by itself which you cannot control. Obviously autocommit must be off as well.
question background:
1.database is neo4j 2.3.1, driver using jdbc;
2.db connection initialized as a class member, default is auto-commit(not changed);
To avoid insert duplicates, i query before insert. after program stopped, found duplicates. why?
code:
String query = "CREATE (n:LABEL {name:'jack'})";
System.out.println(query);
Statement stmt = dbConnection.createStatement();
stmt.executeUpdate(query);
stmt.close();
Use MERGE + unique constraints instead
How do you "check"
You would have to check in the same tx and also take a write lock
after debugging i found that for neo4j-jdbc(v2.1.4), the default db connection transaction level is TRANSACTION_NONE, then i set it to TRANSACTION_READ_COMMITTED, above issue disappeared. so i think that TRANSACTION_READ_COMMITTED will force the previous insert committed, though this is not the recommended way. for isolation level refer to:Difference between read commit and repeatable read
I am using MySQL Database. The following piece creates a record and gets the id from the created record:
insertStmt = connection
.prepareStatement("INSERT INTO bugs (summary, status, report_date) VALUES (?, ?, ? )");
//...
insertStmt.executeUpdate();
idQuery = connection.prepareStatement("SELECT LAST_INSERT_ID()");
rs = idQuery.executeQuery();
if (rs != null) {
rs.next();
return new Long(rs.getLong(1)).toString();
}
Now, if two threads execute this and their execution is interleaved, say, the first thread inserts the record followed by the insertion by the second thread, after which the first thread calls last_insert_id() which will be incorrect for this thread as the second thread has already inserted a record.
This might be overcome using synchronization, however. Is there a way we can execute the two statements in a single database call?
LAST_INSERT_ID works per-connection, and as your question states you can have a race condition if two statements in two threads use the same connection.
You have two ways around this:
1: Use a separate connection per thread (not easy, but this is really the best option for scaling and sense; use connection pooling)
2: Use the form of executeUpdate that records the auto-generated key in the same API call, allowing you to read it back later using getGeneratedKeys so that you don't have to use LAST_INSERT_ID in a second query, so avoiding the race condition. There's a similar form of prepareStatement that you can use in prepared statements.
Option 2 is probably what you want in the short term. The link in option 2 goes straight to that API. This link is a MySQL article outlining how to use it.
According to https://dev.mysql.com/doc/refman/5.7/en/connector-j-reference-configuration-properties.html, you should be able to add ?allowMultiQueries=true to your JDBC connection string. Then you would be able to pass multiple statements, separated by semicolons, in Statement#execute(String sql) calls.
Edit: or, use a stored procedure that does what you want. Or, as you said, synchronize the Java code.
You can try using a Multiquery, combined the Insert and the Select Last_INSERT_ID() in the same string.
1) prepare the connection for using the multiquery:
"jdbc:mysql://"+host+"/"+database+"?allowMultiQueries=true"
2) Combine The Insert Query with the Select:
multiQuerySqlString="INSERT INTO bugs (summary, status, report_date) VALUES (1, 2, 3 ); SELECT LAST_INSERT_ID()"
3) esecute the query and expecting multiple result sets:
boolean isResultSet = statement.execute();
ResultSet res = statement.getResultSet();
if isResultSet = statement.getMoreResults();
// Second ReulstSet object
res = cs.getResultSet();
I hope it works
If you have to do this all on a single connection you can ask the driver to return the generated ID:
insertStmt = connection.prepareStatement("...",PreparedStatement.RETURN_GENERATED_KEYS );
insertStmt.executeUpdate();
ResultSet rs = insertStatement.getGeneratedKeys();
Long id = null;
if (rs != null)
{
rs.next();
id = rs.getLong(1);
}
connection.commit();
return id;
Depending on the driver you might need a different prepareStatement() call that takes the column names as the second parameter:
insertStmt = connection.prepareStatement("INSERT ", new String[] {"ID"});
But even in with the above code you should be doing the concurrent inserts on different physical connections to be able to properly control your transactions.
I work at a gaming cybercafe, and we've got a system here (smartlaunch) which keeps track of game licenses. I've written a program which interfaces with this system (actually, with it's backend MySQL database). The program is meant to be run on a client PC and (1) query the database to select an unused license from the pool available, then (2) mark this license as in use by the client PC.
The problem is, I've got a concurrency bug. The program is meant to be launched simultaneously on multiple machines, and when this happens, some machines often try and acquire the same license. I think that this is because steps (1) and (2) are not synchronised, i.e. one program determines that license #5 is available and selects it, but before it can mark #5 as in use another copy of the program on another PC tries to grab that same license.
I've tried to solve this problem by using transactions and table locking, but it doesn't seem to make any difference - Am I doing this right? Here follows the code in question:
public LicenseKey Acquire() throws SmartLaunchException, SQLException {
Connection conn = SmartLaunchDB.getConnection();
int PCID = SmartLaunchDB.getCurrentPCID();
conn.createStatement().execute("LOCK TABLE `licensekeys` WRITE");
String sql = "SELECT * FROM `licensekeys` WHERE `InUseByPC` = 0 AND LicenseSetupID = ? ORDER BY `ID` DESC LIMIT 1";
PreparedStatement statement = conn.prepareStatement(sql);
statement.setInt(1, this.id);
ResultSet results = statement.executeQuery();
if (results.next()) {
int licenseID = results.getInt("ID");
sql = "UPDATE `licensekeys` SET `InUseByPC` = ? WHERE `ID` = ?";
statement = conn.prepareStatement(sql);
statement.setInt(1, PCID);
statement.setInt(2, licenseID);
statement.executeUpdate();
statement.close();
conn.commit();
conn.createStatement().execute("UNLOCK TABLES");
return new LicenseKey(results.getInt("ID"), this, results.getString("LicenseKey"), results.getInt("LicenseKeyType"));
} else {
throw new SmartLaunchException("All licenses of type " + this.name + "are in use");
}
}
You must do two things:
Wrap your code in a transaction (to avoid autocommit releasing locks immediately)
Use SELECT ... FOR UPDATE and mysql will give you the lock you need (released on commit)
SELECT ... FOR UPDATE is better than LOCK TABLE as it can possibly get by with row-level locking, instead of automatically locking the whole table
According to the online manual, the correct syntax for locking is:
LOCK TABLES ...
and you have
LOCK TABLE ...
but you don't have any error checking. Hence you're probably failing to get the lock and it's silently ignoring that.
FWIW, I'd put your cleanup code (UNLOCK TABLES, conn.commit(), etc) in a finally block to ensure that you always clean up properly in the event of an exception.
As it is, you appear to be potentially leaking database connection handles, and never releasing the lock if there's no free license.
I would like to suggest just doing an update statement and checking how many rows where updated. i will write it out in psudo code.
int uniqueId = SmartLaunchDB.getCurrentPCID();;
int updatedRows = execute('UPDATE `licensekeys` SET `InUseByPC` = uniqueId WHERE `InUseByPC` NOT null LIMIT1')
if (updatedRows == 1)
SUCCESS
else
FAIL
If it succeeds you can then get the licence key/ID by doing a select.
As is so often the case, OP is an idiot. The code I posted was actually working, but I've just discovered a duplicate row in the database - I guess someone entered the same license twice by mistake. This led me to believe that a concurrency bug I had fixed (by introducing table locks) was still unfixed.
Thanks for the general advice, I've introduced better exception handling to this method.