I'm writing a DB connection pool in Java. It's just a class that holds a bunch of connections and gives them out.
Do I need some kind of destructor method so that the DB connections will be closed when an instance of my class goes out of scope?
Edit:
This is for learning purposes only. I would definitely use a reliable, open source connection pool in production use. I really just want to understand the memory management implications.
If you are trying to do this for learning purposes then what you are doing is fine. Download the source code for one of the below mentioned libraries and browse through their code. That would help you in understanding how are they handling the life-cycle of a connection.
But if you want to use it as part of a production application then I'd highly recommend to use one of the below
BoneCP
c3p0
Just off the top of my head, say you have an object, your key. Implement a unique identity for your key. hashCode() is a good place to start. When you create your object, place the object in a WeakReference and associate it to a ReferenceQueue. Now use the hashCode as the key to a Map<int, Connection>. When your object goes out of scope, go to the ReferenceQueue to retrieve the object, use the hashCode to get the Connection from the Map and close it.
See tutorial on references here
The correct way to do it is to wrap the connection instance in your own class which implements java.sql.Connection, then give that "Connection" out through the pool. You will be forced to implement all of the methods which you can happily pass on to the "real" connection. Except for one - close(). In this method you should return the connection to the pool without closing it.
You can't have an object returned to the pool by just letting it go out of scope. You can put something in finalize() which returns it to the pool, but doing this is dangerous. Finalize() only runs once for any object, so if you put something in there to "catch" the object and prevent it from being gc'd, the finalize() will never be called on it again.
Related
well, I've been thinking of making database requests a little faster by keeping the connection to the database open as long as the object is being used. So I was thinking of opening the connection in the constructor of that class.
Now the question is, how can I close the connection after I stopped using? I have to call close() somewhere, don't I?
I've been reading about the finalize() method, but people seemed to be skeptical about usage of this method anywhere at all. I'd expect it to have something like a destructor, but Java doesn't have that, so?
So could anyone provide me with a solution? Thanks in advance.
I would suggest that you rather implement database connection pooling if the application would allow it. With connection pooling a pool of connections would be created and stay connected to the database. Your application would then grab a open/unused connection from the pool use it and return it to the pool.
This would allow you to acquire connections faster and you won't have to modify your classes too much. Database connection pooling is a great technique if you need to scale your application.
The other benefit is that your database connection pool will be managed by some sort of driver which will take care of opening connections, keeping them open, growing the pool if required and also shrinking the pool when extra connections are not used for a certain amount of time. This would be similar to the code you are trying to implement in the constructor and finalization methods.
Generally speaking you aqquire a database connection only when needed and release it as soon as possible.
I would recommend to make your class an implementor of java.io.Closeable. According to this interface you will have to implement void close() throws IOException, which all clients of the class will call, because it's a good practice to close Closeable classes after use.
this question is for either android or java .
suppose i have an instance of a class (even a thread) .
i wish that in order to make it fool proof against forgetting to dispose/close the instance (and avoid any possible memory leaks) , if there are no more references to this instance , it will automatically call a specific method , disposing/closing itself (in case of a thread , it will probably interrupt itself) .
is it possible to enforce such a thing?
if needed , i don't mind that such a thing will occur only during GC .
if there are no more references to this instance , it will
automatically call a specific method , disposing/closing itself (in
case of a thread , it will probably interrupt itself)
finalize() does what you describe here. You very rarely see it used though, and there are some pitfalls when using it. As you cant control garbage collection, you cant be sure when finalize will be run, if ever! From the api:
the usual purpose of finalize, however, is to perform cleanup actions
before the object is irrevocably discarded. For example, the finalize
method for an object that represents an input/output connection might
perform explicit I/O transactions to break the connection before the
object is permanently discarded.
You cant enforce garbage collection. You can only suggest the JVM to do so using System.gc(), but it is not guaranteed that it will be done.
I know this is very late but I hope it might help someone someday.
You can receive such events this by using the using the library that i am developing called gcRadar. It provides events when an object is orphaned and after the actual garbage collection of the object.
Any suggestions for improvements in the library are welcome.
There is no way to access the references held by the VM. As Zavior suggested, the only way to know for sure that an object, or an "island" of objects is inaccessible, is to use the finalize method.
Please note that you will only get notified during GC runs. So it does not really help closing/disposing resources that are still referenced but should be closed. If you want to do that as well and do not want to use such constructs as try/catch/finally, you should write a manager class for the resources.
With any of the possibillities, including a manager class, you will not get a "bulletproof" way to consolidate your resources. Being careful is the best solution IMHO.
EDIT:
I have found this thread that may be useful.
I would like to implement a ORM-style system that can save updates to POJOs when they are no longer reachable by the caller.
I thought the reference classes could do it, but they seem to only enqueue the reference after the object has been cleared (I was hoping it was when they were able to be collected), so once enqueued the .get() method will always return null.
I could use a finalizer but last time I checked those were questionable (Not guaranteed to run promptly or run at all)--I believe a combination of finalizers and runShutdownHook() would work but that's getting into fairly swampy territory.
Is there another path I'm not thinking besides the obligatory "Just have the caller call .save() when he's done"?
Are you just trying to avoid having to call save() on every POJO that you modify?
This can be done reliably using a persistence session object, like this:
Open a new session object.
Load objects via the session object. The session object maintains references to all the objects it loads.
Make any changes to the loaded objects. It is not necessary to call a save method on updated objects.
Close the session object. The session saves all of its objects. It might even be fancy enough to keep a copy of clean loaded data, compare all of its objects to the clean data, and save only the ones that have been modified.
And if you don't want to pass session objects through your code, you can take things a step further with the Unit of Work pattern, associating a session object to the current thread:
Start a unit of work. This creates a session object behind the scenes and associates it with the current thread.
Load objects. Whenever an object is loaded, your ORM automatically associates it with a session object based on the current thread.
Make any changes to the loaded objects. It is not necessary to call a save method on updated objects.
Complete the unit of work. This closes the session object, saving all the objects.
This fixes several problems with a reachability based solution:
You are not relying on nondeterministic garbage collections, which may have a long time between runs, or not run at all.
All objects modified in one operation are saved together. If you rely on reachability, different objects modified in the same operation can become unreachable at different times, meaning your modifications can be saved to the database in bits-and-pieces.
Rollback is much easier - just give your session object a rollback() method. With a reachability solution, you would need to remember to call rollback() on every modified POJO if an operation fails, which is really the same as your original problem.
Perhaps see http://nhibernate.info/doc/patternsandpractices/nhibernate-and-the-unit-of-work-pattern.html or research the Unit of Work pattern and emulate some of those ideas.
Use the Observer Pattern do build a ClearanceManager and some Destroyables.
IDestroyable is an interface, which is used for the observers it contains the method public void destroy()
The ClearanceManager is the Subject of the Observerpattern. Maybe use Singleton here to ensure you have just one ClearanceManager object in your application.
Use a Set internaly inside the ClearanceManager (not a List to ensure Objects can just be added once)
support an addDestroyable(IDestroyable destoryable) method (and maybe a removeDestroyable one).
During Runtime the Classes for which you need some destructor emulation, can register them self at the ClearenceManager. ClearenceManager.getInstance().addDestroyable(this);
The ClearanceManager has an doClearance() method, which should just be called at the end of the Main method. It iterates threw the private Set and calls destroy() on every IDestroyable object.
Doing it this way you can emulate destructors, without using them, because using destructors you are losing control about the existance of myabe needed object.
You do not know when overwriting finalize, when it is called.
Maybe, if you do not want to call doClearance() in your Main method you can use here, but just here, a real destructor finalize(). Because there are references in the ClearenceManager to the needed Objects, they will not be destroyed first. But maybe mhh, if there are cross references .... better do not use finalize, use doClearance() and have fun with it :)
I think you are barking up the wrong tree here.
All of Java's finalizer and Reference mechanisms based on reachability depend on the garbage collector to determine whether the respective objects are reachable. So if you use any of the Reference mechanisms for some kind of finalization, you run into much the same issues that make finalize a bad idea.
It is technically possible to implement your own mechanisms for doing reachability; e.g. by implementing your own application-specific reference counting. However, it is likely to be expensive, fragile, and make your code look horrible. (Reference counting in Java is likely to be messier and more fragile than in C++, because you can't overload reference assignment operators to ensure that reference counts are adjusted transparently. So every reference assignment needs to be wrapped in a method call.) So I'd say that doing your own reachability analysis is a bad idea.
So, to be practical you need to either:
rethink your design so that you don't do things based on reachability, or
live with the consequences of using finalize.
The first option is clearly the best, IMO.
maybe you can subclass PhantomReference, and store necessary data in it for final actions.
I am trying to log the creation and destruction of database connections in our application using c3p0's ConnectionCustomizer. In it, I have some code that looks like this:
log(C3P0Registry.getPooledDataSources())
I'm running into deadlocks. I'm discovering that c3p0 has at least a couple of objects in its library that use synchronized methods, and don't seem to specify their intended lock ordering. When I log the connections, I'm holding a lock on C3P0Registry and eventually PoolBackedDataSource (simply creating a list of the datasources is accessing the hashcode causing a lock).
Shutting down the connection provider (calling C3P0ConnectionProvider.close()) causes the locks to be called in the opposite order. But while the child datasources are being shut down, my logging is being triggered. The result is a deadlock.
It seems like both calls I am making into the c3p0 library are valid, expected calls:
C3P0ConnectionProvider.close()
C3P0Registry.getPooledDataSources()
It also seems like (unless explicitly stated in the documentation) it should be the library's responsibility to manage it's own locking strategy. (I don't say this to blame anyone.. just to confirm my understanding of best practices)
How should I deal with this issue? Since c3p0 uses synchronized methods rather than a more modern mechanism, I can't really test the locks.
From my DataSource closing code, I could first grab the C3P0Registry lock before closing the DataSource. I would be guessing at the correct lock order, which I don't know if I feel comfortable with.
I don't think I could reverse the lock order for the logging call. I need the C3P0Registry to get the list of DataSources, so I couldn't lock the DataSources without first locking C3P0Registry to get references to them.
Another solution, of course is to provide another, higher level lock above everything c3p0. In the case of a connection pool, that seems to defeat the point.
For now, I'm rolling back my logging. Thanks for any help.
I dont know how to fix the locking issue, but i think you should take a step back here and think about the original problem.
"I am trying to log the creation and destruction of database connections in our application ..."
I would recommend the following.
Create a class and make it implement javax.sql.DataSource.
Create a field of the same type and delegate all methods to it.
In the getConnection() method return your own Connection class wrapping around
java.sql.Connection and so on.
Then wrap this class around your original data source.
In your classes you can now simply create a logger and log all actions you want to see in your log.
I have a Java program consisting of about 15 methods. And, these methods get invoked very frequently during the exeuction of the program. At the moment, I am creating a new connection in every method and invoking statements on them (Database is setup on another machine on the network).
What I would like to know is: Should I create only one connection in the main method and pass it as an argument to all the methods that require a connection object since it would significantly reduce the number of connections object in the program, instead of creating and closing connections very frequently in every method.
I suspect I am not using the resources very efficiently with the current design, and there is a lot of scope for improvement, considering that this program might grow a lot in the future.
Yes, you should consider re-using connections rather than creating a new one each time. The usual procedure is:
make some guess as to how many simultaneous connections your database can sensibly handle (e.g. start with 2 or 3 per CPU on the database machine until you find out that this is too few or too many-- it'll tend to depend on how disk-bound your queries are)
create a pool of this many connections: essentially a class that you can ask for "the next free connection" at the beginning of each method and then "pass back" to the pool at the end of each method
your getFreeConnection() method needs to return a free connection if one is available, else either (1) create a new one, up to the maximum number of connections you've decided to permit, or (2) if the maximum are already created, wait for one to become free
I'd recommend the Semaphore class to manage the connections; I actually have a short article on my web site on managing a resource pool with a Semaphore with an example I think you could adapt to your purpose
A couple of practical considerations:
For optimum performance, you need to be careful not to "hog" a connection while you're not actually using it to run a query. If you take a connection from the pool once and then pass it to various methods, you need to make sure you're not accidentally doing this.
Don't forget to return your connections to the pool! (try/finally is your friend here...)
On many systems, you can't keep connections open 'forever': the O/S will close them after some maximum time. So in your 'return a connection to the pool' method, you'll need to think about 'retiring' connections that have been around for a long time (build in some mechanism for remembering, e.g. by having a wrapper object around an actual JDBC Connection object that you can use to store metrics such as this)
You may want to consider using prepared statements.
Over time, you'll probably need to tweak the connection pool size
You can either pass in the connection or better yet use something like Jakarta Database Connection Pooling.
http://commons.apache.org/dbcp/
You should use a connection pool for that.
That way you could ask for the connection and release it when you are finish with it and return it to the pool
If another thread wants a new connection and that one is in use, a new one could be created. If no other thread is using a connection the same could be re-used.
This way you can leave your app somehow the way it is ( and not passing the connection all around ) and still use the resources properly.
Unfortunately first class ConnectionPools are not very easy to use in standalone applications ( they are the default in application servers ) Probably a microcontainer ( such as Sping ) or a good framework ( such as Hibernate ) could let you use one.
They are no too hard to code one from the scratch though.
:)
This google search will help you to find more about how to use one.
Skim through
Many JDBC drivers do connection pooling for you, so there is little advantage doing additional pooling in this case. I suggest you check the documentation for you JDBC driver.
Another approach to connection pools is to
Have one connection for all database access with synchronised access. This doesn't allow concurrency but is very simple.
Store the connections in a ThreadLocal variable (override initialValue()) This works well if there is a small fixed number of threads.
Otherwise, I would suggest using a connection pool.
If your application is single-threaded, or does all its database operations from a single thread, it's ok to use a single connection. Assuming you don't need multiple connections for any other reason, this would be by far the simplest implementation.
Depending on your driver, it may also be feasible to share a connection between threads - this would be ok too, if you trust your driver not to lie about its thread-safety. See your driver documentation for more info.
Typically the objects below "Connection" cannot safely be used from multiple threads, so it's generally not advisable to share ResultSet, Statement objects etc between threads - by far the best policy is to use them in the same thread which created them; this is normally easy because those objects are not generally kept for too long.