My application sometimes can lost connection to MySQL database.
I think that good solution will be schedule some timer to try to recconect after some time.
How better it can be done? May be separate thread that try to connecto to db? Or exist the stardard practices ?
Thanks.
JDBC is a great way to start building a java database application, but managing object mappings and connections / transactions can very rapidly lead to alot of boiler plate and rewriting of logic which has already been written many times by many programmers.
It is expected that you should lose/close a connection normally, unless you have a high throughput application, in which case you might keep several connections alive (this is known as connection pooling).
There are essentially 3 "high-level" approaches to maintaining efficient connections and transactions :
1) The simplest solution is to check when you are reusing a connection to make sure it is valid, or reopen it every time.
2) A more sophisticated solution is to use a connection pooling mechanism, such as the apache http://commons.apache.org/dbcp/ dbcp library.
3) Finally, in my opinion, the most maintainable solution is to use a JDBC framework like ibatis/hibernate which will , providing you a simple , declarative interface for managing object relational mapping / transactions / database state ---- while also transparently maintaining connection logic for you.
ALSO : If object relational mapping is not your thing, then you can use a framework such as DBUtils by apache to manage querying and connections, without the heavy weight data mapping stuff getting in the way.
JDBC is a simple API to abstract the operations of different database system. It makes something uniform, such different native types to java type.
However, the lost connection is another big issue. Using the connection pool library is better than writing a new one by yourself. It is too many details to implement a connection pool from scratch without bug.
Consider to use the mature library:
Commons-DBCP
bonecp
etc
Commons DBCP is based on the Commons Pool.
You should understand the configurable options both of them.
bonecp is another new connection pool, no lock is its advantage.
Validated SQL String is important to check a connection is dead or alive.
Lost connection checking enable by the validation string set.
Here is the dbcp configuration page:
http://commons.apache.org/dbcp/configuration.html
It says:
NOTE - for a true value to have any effect, the validationQuery
parameter must be set to a non-null string.
For example:
dataSource.setValidationQuery(isDBOracle() ? "select 1 from dual" : "select 1");
dataSource.setTestWhileIdle(true);
dataSource.setTestOnReturn(true);
dataSource.setRemoveAbandoned(true);
dataSource.setRemoveAbandonedTimeout(60 * 3 /* 3 mins */);
dataSource.setMaxIdle(30);
dataSource.setMaxWait(1000 * 20 /* 20 secs*/);
Remind: If you use the Common DBCP in weblogic, don't forget the older Commons library in the server, it will drive your application use the different version. The prefer-web-inf-classes setting will help you.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I'm working on a small project in Java and I have got DatabaseController class which contains many methods to insert/select/delete from my database.
my question is, related to setting up the connection in regards to best software engineering practices.
Should I establish the database connection in DatabaseController constructor so it's opened once and left open Or..
Should I place a method to connect to the database before my insert/delete/select methods?
I understand that both might have cons and pros .. but I'd like to know what's best for small and large scale projects?
DataSource
Firstly, it is best to put access to your database behind a DataSource interface. This gives you the power to change your strategy for getting a database connection without changing all the many places in your code where you use those database connections. See Oracle Tutorial.
Your JDBC driver should provide an implementation of DataSource. For example, consider the Postgres JDBC driver from https://jdbc.postgresql.org. That driver supplies two implementations: One for direct connections to the database, and the other for pooling connections. By using the DataSource as a façade, you could switch between either of these, or even use a 3rd party connection pooling solution, in one place in your code, without changing the many places you call DataSource::getConnection().
A third kind of implementation you might want to substitute in would be for distributed transactions.
Another example of swapping the concrete class behind the interface: Perhaps you decide to switch JDBC drivers. Perhaps for your Postgres database, you decide to switch from the jdbc.postgresql.org driver to the pgjdbc-ng driver. You want to be able to make this configuration change in one place, without breaking all your existing app code.
Here is an example of using a concrete implementation behind the DataSource interface.
public javax.sql.DataSource obtainDataSource() {
org.postgresql.ds.PGSimpleDataSource dataSource = new PGSimpleDataSource() ;
dataSource("AcmeInvoices database data source") ;
source.setServerName( "localhost" ) ;
source.setDatabaseName( "invoicing" ) ;
source.setUser( "Scott" ) ;
source.setPassword( "tiger" ) ;
return dataSource ; // Returning a reference to the object of this concrete class `PGSimpleDataSource` as an object of the interface `DataSource`.
}
The methods you see being called in that code snippet above will vary dramatically by the capabilities of your database and of your JDBC driver. For example, if you are using encrypted connections to the database, you will have additional methods to call for configuring the encryption keys.
You set up this DataSource object early on, likely as part of your app launch. Keep a reference someplace. Or keep several: you may have different connection types (different user name & passwords) for different parts of your app where different security or different user roles are necessary.
Pass the DataSource object around your codebase to the many methods that insert/select/delete with your database. In each method, they call myDataSource.getConnection(), then go on to execute their SQL. When that method finishes its work, it should close the connection by calling Connection::close. Even better, use try-with-resources to automatically close the connection.
If the connection came from a connection pool, the connection is not really closed. Rather, the connection is returned to the pool. This is another example of the power of using an interface, here Connection. The concrete class behind the interface can choose how to respond to invoking the close method — and that concrete class can be swapped out later without breaking the calling code.
Open connections
Should I establish the database connection in DatabaseController constructor so it's opened once and left open
No, it is not generally a good idea to keep a connection open with no further work to be done.
Basically you would be creating your own connection pool. And connection pooling is not a simple matter. There are various issues to consider in pooling, and you are unlikely to think of them much less be competent to solving them. All the connection pooling tools I have seen have each had their share of bugs, issues, and limitations. So do not take on this task yourself. If you do choose to go with pooling, find a robust well-worn existing implementation. And study its documentation thoroughly to be sure you understand how to configure and use the pool properly.
Frankly, I stopped using connection pooling myself. The various risks and complications are not worth it in my opinion for systems with a lower number of less frequent number of users/connections. For example, look at the default time-out set in the various pooling solutions where the pooled connections are closed after some period of no use. If your activity in the pool is mostly lower than that time-out, then you effectively won't be benefiting from the pool. Also, I have found that most of the claims about opening a connection to the database as expensive are exaggerated, especially for a local database on the same machine.
Examples
Should I place a method to connect to the database before my insert/delete/select methods
Here are some example scenarios showing how you should get-and-close the database connection close to where you use the connection for some specific database task.
In your login screen where you authenticate the user by looking up users and groups stored in your database, when the user clicks the Login button, your code should retrieve a reference to the DataSource object, call getConnection, do the user-and-groups lookups, then immediately close the connection.
When the user goes on to lookup overdue invoices, do the same. Retrieve the DataSource, call getConnection, run your SQL to find any overdue invoices, load that invoice data into memory as a cache, call Connection::close to close the connection to the database, and proceed to build your GUI to display the cached invoice data.
When the user clicks a button to take some action on one of those invoices, do the same process. Retrieve the DataSource, call getConnection, run your SQL to fetch the particular invoice row for the desired primary key value, retrieve additional invoice fields and related rows such as invoice-items, call Connection::close to close the connection to the database, and proceed with your action such as generating an email containing the cached invoice details and its invoice-item rows. Notice how you might use the connection multiple times within this action method, first to hit the invoice table, then again to hit the invoice-item table. When finally all the immediate work at hand is complete, close the connection.
Remember, in each of these several times where you got a connection and closed a connection, if pooled:
You are getting a connection that may or may not be new.
“Closing” returns the connection to the pool rather than actually closing it.
The pool may close the connection at some point, and at some point perhaps replace it with a fresh connection. But that is transparent to all your user-authentication code and invoice lookup code.
We have a central database for accounts. It contains login information, and a field called database profile. The database profile indicates what database connection should be used for the account. For example, we would have Profile1, Profile2, Profile3... ProfileN
If a user is indicated to have Profile1, they would be using a different database than a user who is indicated to be a part of Profile2.
My understanding of JPA is that you would need a new EntityManagerFactory for each Profile (Persistence Unit), even though the databases all have the same schema, just different connection information. So if we ended up having 100 profiles we would have 100 Entity Manager Factories, which doesn't seem ideal.
I've looked into the EntityManagerFactory, and it doesn't seem to let you change the database connection options.
Is my only option to have N EntityManagerFactory's, and if so would their be be any major consequences to this (such as bad performance)?
Thanks for any advice.
The kinds of things you're talking about are getting out of the scope of the JPA abstraction. Things like specific connection management are generally more provider specific. For example, with Hibernate SessionFactory you can certainly create new sessions from an aribtraty JDBC connection. There would be pitfalls to consider such as ID generation schemes (you'll probably have to use sequences generated in the DB), and you're basically hooped for L2 caching, but with careful programming it could be made to work.
Just use javax.persistence.Persistence#createEntityMananagerFactory(String,Map), and provide in the Map the connection parameters. Cache the EMF, and use the connections judiciously, don't mix n match object from the different EMFs.
If you are using spring then I know there is a way to dynamically switch the DataSource. Find more information here
We have a Java J2EE application that was using individual web service calls for each database row insert/update. That turned out to be WAY too slow. They have brought me in to "quickly" fix it. I plan to convert all the web service calls to plain JDBC. To do that, I need to get a JDBC connection from the pool and then use it in multiple different methods. I need to use the same JDBC connection in multiple DAOs to string it all together into a single database transaction. I can explicitly pass around the JDBC connection to each DAO that needs it, but that would require me to change a LOT of method signatures, plus a LOT of unit tests (which goes against the "quickly" part).
I am trying to come up with a good way put the JDBC connection somewhere and then just grab it in the methods that need it without having to explicitly pass it around everywhere. We can't use Spring, JPA, or Hibernate on this project because the support team won't support those technologies. I can put the JDBC connection into an EJB, but I am not sure how reliable that would be. I could create a custom Singleton to manage database connections for each user (session?), but I would have to be careful about thread safety. If anyone has tried to do something like this before, I would appreciate some advice.
You could use a ThreadLocal. Have the entry point set it up and than the DAOs
class ConnectionUtil {
public static final ThreadLocal<Connection> connection = new ThreadLocal<Connection>();
}
public Return method(Args arg) {
ConnectionUtil.connection.set(newConnection());
try {
...
} finally {
ConnectionUtil.connection.remove();
}
}
Pretty ugly, but that seems to be what your boss wants.
Use Apache Commons DBCP. It's the Connection Pool project from Apache, and what's used internally in many engines.
We have done that before (5 years ago or so on an IBM WebSphere).
We wrote an own pool and stored the jdbc-connections in a hashtable with the sessionID. The only pitfall was to close the connection on sessionend and return it to the pool (we did that with a sessionlistener). If one user session connects only to one jdbc-connection, the thread safety is inherited. So the singleton approach does definitly work.
Our performance gain was awful.
Additionally what I have done for DAOs in this is not to pass the connection in the method signature but in the Constructor instead. Then I hold the object that originally creates the connection responsible to close it.
In our application when we create the Datasource, we select the
Database Name DB2
Driver: BEA Type 4 XA DB2
But what i know is, there are only 4 types of Driver. Then what is Type 4 XA driver?
From this blog entry.
An XA transaction, in the most general
terms, is a "global transaction" that
may span multiple resources.
That is, a transaction running across (say) 2 databases. So, for example, insertions can be managed across those 2 databases and committed/rolled back atomically.
The "type 4" refers to a native Java JDBC driver converting directly into the database protocol. See here for more details
Type 4: All Native Java
XA: stands for Extensible Architecture which is refered mostly for a 2-phase-commit protocol - see wikipedia. Short: A standard protocol for a global transaction between one transaction coordinator and several transaction managers. Sometimes they are also called transaction monitors. It's pretty slow so should avoid it if you don't really need it. But well, at our customer we mostly need it :(
Major advantage of XA is that it can access multiple databases in one connection/transaction.
It seems to me that introducing an ORM tool is supposed to make your architecture cleaner, but for efficiency I've found myself bypassing it and iterating over a JDBC Result Set on occasion. This leads to an uncoordinated tangle of artifacts instead of a cleaner architecture.
Is this because I'm applying the tool in an invalid Context, or is it deeper than that?
When can/should you go whole hog with the ORM approach?
Any insight would be greatly appreciated.
A little of background:
In my environment I have about 50 client computers and 1 reasonably powerful SQL Server.
I have a desktop application in which all 50 clients are accessing the data at all times.
The project's Data Model has gone through a number of reorganizations for various reasons including clarity, efficiency, etc.
My Data Model's history
JDBC calls directly
DAO + POJO without relations between Pojos (basically wrapping the JDBC).
Added Relations between POJOs implementing Lazy Loading, but just hiding the inter-DAO calls
Jumped onto the Hibernate bandwagon after seeing how "simple" it made data access (it made inter POJO relations trivial) and because it could decrease the number of round trips to the database when working with many related entities.
Since it was a desktop application keeping Sessions open long term was a nightmare so it ended up causing a whole lot of issues
Stepped back to a partial DAO/Hibernate approach that allows me to make direct JDBC calls behind the DAO curtain while at the same time using Hibernate.
Hibernate makes more sense when your application works on object graphs, which are persisted in the RDBMS. Instead, if your application logic works on a 2-D matrix of data, fetching those via direct JDBC works better. Although Hibernate is written on top of JDBC, it has capabilities which might be non-trivial to implement in JDBC. For eg:
Say, the user views a row in the UI and changes some of the values and you want to fire an update query for only those columns that did indeed change.
To avoid getting into deadlocks you need to maintain a global order for SQLs in a transaction. Getting this right JDBC might not be easy
Easily setting up optimistic locking. When you use JDBC, you need to remember to have this in every update query.
Batch updates, lazy materialization of collections etc might also be non-trivial to implement in JDBC.
(I say "might be non-trivial", because it of course can be done - and you might be a super hacker:)
Hibernate lets you fire your own SQL queries also, in case you need to.
Hope this helps you to decide.
PS: Keeping the Session open on a remote desktop client and running into trouble is really not Hibernate's problem - you would run into the same issue if you keep the Connection to the DB open for long.