At several points in the documentation of GridDB the auto-commit feature is disabled and instead there are manual commits. I did not manage to find any explanation for this behaviour.
It seems that it needs to be disabled when deleting a row from a GridDB container, but not for example when adding a row. In the later case there seems to be little difference between it being active or not. Though of course one has to commit manually at least once if it is disabled for changes to actually be reflected by the database.
So what does auto-commit exactly do, when is it committing changes automatically? When is there a need to or an advantage from disabling auto-commit?
These are the functions I am talking about:
Java:
col.setAutoCommit(false); col.commit();
PHP:
col->set_auto_commit(false); col->commit();
Auto commit allows GridDB to determine when is best to commit resulting in good performance but also allows other clients to fetch stale data.
I disable auto commit and commit manually every time for a singular write or any number deletes or updates but leave auto commit on when writing a stream of data.
Related
Simple question, but.. just want to be sure. If database connection is set to conn.setAutoCommit(false);, after st.executeQuery(...); - do we need to conn.commit();? Of course, if some changes were made like st.execute(...); - conn.commit(); is needed. I'm using HSQLDB.
In addition to committing updates, commit also releases locks, so it is useful to call commit after executing queries even if no updates were made (though this may be true only for specific databases, and you didn't specify which DB you are using).
I'm not getting a clear idea of why autocommit is by default false in Hibernate when we have the Transaction management apis provided.
I have three questions
why autocommit mode is not recommended by Hibernate?
What happens when we use autocommit = true and then use the Hibernate Transaction apis for transaction management ?
When using spring declarative transaction management how #Transactional(readonly = true) will help the read only code (Hibernate code) we write?
I will answer one by one Starting with (2) as i don't know much about (1)
(2): autocommit=true means by default all queries get commited. In such case If
if there is a #Transactional on a method, it overrides the autocommit and encloses all queries into a single transaction, thus overriding the autocommit
if there is a #Transactional method that calls other #Transactional annotated methods, the outer most annotation should override the inner annotaions and create a larger transaction, thus annotations also override eachother.
(3): In DBs like Oracal/Mysql a read only transaction can be translated READ_ONLY level which provides no dirty reads, no unrepeatable reads but doesn’t allow any updates. That means the flush mode will be set as FlushMode.NEVER in the current Hibernate Session preventing the session from commiting the transaction. Even setReadOnly(true) will be called on the JDBC Connection which ensure that you cannot call session.flsuh() even to flush session manually.
since Spring doesn't do persistence itself, it cannot specify what readOnly should exactly mean. This attribute is only a hint to the provider, the behavior depends on, in this case, Hibernate.
In regards of (1):
Suppose, you own a company and have 1000 employees. You maintain a database table to track if an employee was paid at the end of the month already or not.
So, here we are, the end of the month and pay day. Payroll sends you an excel file
with 600 names who has just received their compensation for the last month. So you log on to your computer and start your java app and select the excel to save all 600 records in your database. It usually takes 2 minutes, but now it fails at 1 minute and 23 second. What is your expectation now? Do you expect a partially uploaded file with gosh'knows how many records, or nothing at all? Your autocommit will drive it: If autocommit=false then you can give a go to upload the whole file again and again, but if autocommit=true you might need to tweak around your input data first to remove some records to prevent duplicates in your database.
I hope, my simplified example helps you to better understand it, but really the purpose is to ensure either everything from a batch is saved (it includes every write operation, like insert/update/delete) in the database or nothing at all in case an error occurs at anytime during the process. In the real life in most of the cases you and your organisation will expect complete data sets rather than partial data set in the database. Understanding it is key and anybody who advises to 'use autocommit=true because it is safe' should be avoided by miles. This is a key concept and one of the foundation of data management.
The fundamental source of my questionning came from this observation. When I use Hibernate and make any query, I get the following in the MySQL logs:
SET autocommit=0
insert into SimpleNamedEntity (name, version) values (null, 0)
commit
SET autocommit=1
Now, I did some research (refs below, would have added more, but I'm not 'good' enough it seems :-)) and this seems to be a fairly classic problem. I did a number of tests on various levels of architecture (mysql config, jdbc, connection pool, hibernate) to get a better idea of how things work, and I ended up more confused as a result, so here are a few questions:
Can someone confirm if autocommit does or doesn't have a negative impact on performance? The main reason I see for this is defensive behavior to prevent unwanted uncommitted transactions that would happen if people forgot to commit after statement execution. Is there another reason?
The way I see it, it seems to me that you will want autocommit to off when doing large dependant inserts-updates-deletes as this helps ensure data integrity. If all I want to do are selects, then it's useless and I'd probably want to set it to true to avoid some long-transaction lock-type side-effects. Does that make sense?
I noticed that whatever the mysql autocommit setting, the jdbc connection is always to true and the only way for me to control that (have it false by default) is to add init-connect="SET AUTOCOMMIT=0" on the mysql server'. But, the strange part is once I start using c3p0 with that setting, c3p0 decides to manually override that and forces the set autocommit to 1. There's even a rollback in this case I find really suspect. Why does c3p0 force-set the flag? Why does it rollback when it doesn't normally? Is there a way to tell c3p0 not to do this?
19 Query SHOW COLLATION
19 Query SELECT ##session.autocommit
19 Query SET NAMES utf8mb4
19 Query SET character_set_results = NULL
19 Query SET sql_mode='STRICT_TRANS_TABLES'
19 Query SELECT ##session.tx_isolation
19 Query rollback
19 Query SET autocommit=1
I was trying to 'clean up' the autocommit statements I was having in the example above. The closest I got was by setting autocommit to off on the mysql server. But even then, hibernate feels compelled to at a minimum do one set autocommit = 0 statement at the beginning of a transaction. Why would it need to do that? To 'reset' transaction state? Is there a configuration path I can use somewhere in the stack to prevent hibernate from doing this?
References:
Java JDBC ref
Article on the problem
(edit - clarification)
I am re-reading my question and can understand how complicated it all sounds. I'm sorry I wasn't clear, I'll try to clarify. I want transactions. I get the InnoDB part, READ_COMMITED. I also understand autocommit is essentially a transaction per sql statement. Here's what I don't get:
hibernate does 2 calls to the DB it doesn't need to do in my first example. I don't need it to toggle between autocommit = false and autocommit = true. I expect it to stay at false all the time but however I configure it, it still wants to make set autocommit calls automatically. I don't get why it does that.
c3p0 exhibits somewhat similar behavior. It does set autocommit automagically for me when I don't want it to, and I also don't understand why it does that.
You need to decide if you are doing transactions or not.
It is important to understand how SQL servers work, there is no such thing as no transactions. What autoCommit=true does it make each statement a transaction in its own right, you don't need to use an explicit BEGIN TRANSACTION and COMMIT around each statement (the SQL server implies this behavior).
Yes... it is important that you do SELECTs inside transaction. That fact you suggest it is not needed indicates you do not yet fully appreciate and understand the point of relational integrity.
Hibernate expects relational integrity to exist, if you read some data now, then runs some Java code, then use the data you previously read to access more data from SQL (for example using Hibernates lazy loading feature) then it is important that data still exists isn't it. Otherwise what should your Java code do now? throw the user an error?
This is what using transactions and say the default READ_COMMITTED Transaction Isolation level would provide and this is the minimum level you should start your quest to grok (learn) SQL.
So are you doing transactions or not. The question is do you want consistent, reliable, repeatible behavior or do you fast but sometimes crazy ? (this is not to say transactions slow down performance, that is a common misconception, only transactions that have conflicts with each other might lower performance, working on large data sets in an open transaction increase the possibility to lower performance).
Re autoCommit=true|false affecting performance. I think you are asking the wrong question.
Re databases locking up due to long transactions, this is usually a code/code-design bug, sometimes it is a design issue about how to do about doing things in SQL. Many small data-set transactions is good, one big large data-set is bad.
.
In order to get transactional support in MySQL you should ensure all your tables are InnoDB type using "CREATE TABLE tableName ( ... ) ENGINE=InnoDB;".
You should keep the SQL driver, pooled connection handle (via c3p0) with autoCommit=false (this should be the default).
You should ensure the default Transaction Isolation Level is (at least) READ_COMMITTED.
Don't forget to use the correct Hibernate dialect for InnoDB (check the hibernate startup logging output).
Now learn how to use that. If you hit a problem it is because you are doing something wrong and don't fully understand it all. If you feel you need to lower the isolation level, stop using transactions or need autoCommit=true then you should seriously hire a professional consulant to look at the project and should not consider turning it off until you understand why and when you can turn it off.
.
Update after your edit
The default autoCommit setting is a property of the driver and/or the connection pooler. So the suggestion to mess with hibernate setting isn't where I'd start.
By default hibernate checks what it gets and modifies it if necessary. Then puts is back to how it was before returning the Connection to the connection pooling (some broken connection poolers don't check/reset this so hibernate does this to improve its inter-operation within a larger system).
So if it is assistance with setting up the connection pooler (c3p0) then maybe you should start by posting the details of how you configured it ? The version you are using, also an example of the JDBC connection string in use and also the MySQL driver jar you are using.
Also what is your deployment scenario ? standalone J2SE ? web-app WAR ? app-server ?
Can you write some test code to get a Connection and there is API to ask it what the autoCommit mode it. http://download.oracle.com/javase/6/docs/api/java/sql/Connection.html#getAutoCommit%28%29 Your goal is to then adjust your configuration so that you always get 'false' back from this API.
The MySQL driver documentation http://dev.mysql.com/doc/refman/5.1/en/connector-j-reference-configuration-properties.html there is some reference to autoCommit in there.
You also want to try and run "mysqladmin variables" on your server to get back the server configuration, it maybe possible to set it globally (which seems like a silly thing to do IMHO).
In short you need to find the source of where it is being set on, since that is an unusual default. Then you can remove it.
Suppose I have a table that contains valid data. I would like to modify this data in some way, but I'd like to make sure that if any errors occur with the modification, the table isn't changed and the method returns something to that effect.
For instance, (this is kind of a dumb example, but it illustrates the point so bear with me) suppose I want to edit all the entries in a "name" column so that they are properly capitalized. For some reason, I want either ALL of the names to have proper capitalization, or NONE of them to have proper capitalization (and the starting state of the table is that NONE of them do).
Is there an already-implemented way to run a batch update on the table and be assured that, if any one of the updates fails, all changes are rolled back and the table remains unchanged?
I can think of a few ways to do this by hand (though suggestions are welcomed), but it'd be nice if there was some method I could use that would function this way. I looked at the java.sql.statement.executeBatch() command, but I'm not convinced by the documentation that my table wouldn't be changed if it failed in some manner.
I hit this one too when starting with JDBC - it seemed to fly in the face of what I understood about databases and ACID guarantees.
Before you begin, be sure that your MySQL storage engine supports transactions. MyISAM doesn't support transactions, but InnoDB does.
Then be sure to disable JDBC autoCommit - Connection.setAutoCommit(false), or the JDBC will run each statement as a separate trasaction. The commit will be an all or nothing affair - there will be no partial changes.
Then you run your various update statements, and finally call Connection.commit() to commit the transaction.
See the Sun Tutorial for more details about JDBC transactions.
Using a batch does not change the ACID guarantees - you're either transacted or you're not! - batching is more about collecting multiple statements together for improved performance.
Have gone through hibernate api specification on FlushMode but didn't get the exact difference. So please help.
If flush mode is 'AUTO' before firing any query hibernate will check if there are any tables to be updated. If so, flush will be done otherwise no. If flush mode is 'ALWAYS', flush will happen even if there are no tables to be updated.
Check source of , org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(AutoFlushEvent)
Always means that before any query is run on a collection or such the query is rerun against the database. With auto I am assuming there is some "magic" under the hoods that knows most data doesn't change that often so you don't always have to flush. It also affects how often might happen during a transaction. I say might because some sources say setting the flushmode is only a hint to hibernate - but see this thread for some discussion...
http://forum.springsource.org/archive/index.php/t-14044.html