We Keep Coding

JDBC kinds of connection

I'm creating app for library management with Java and MySQL ( JDBC to connect with DB ) , and I have a problem , I checked a lot of topics, books, and websites but I didn't find good answer for me. Is it the good way to deal with connections ? I think that one connection for entire app is good option in this case. My idea is that in every function in every class when I need to use Connection object , these functions will need a connection parameter. In main class I'll call manager object 'Man' for example and to every constructor etc I'll pass Man.getMyConn() as this parameter and call Man.close() when Main frame will be closed . Is it bad idea ? Maybe I should use singleton pattern or connection pool ?
Sorry for my English , I'm still learning.
public class manager {
private Connection myConn;
public manager() throws Exception {
Properties props = new Properties();
props.load(new FileInputStream("app.properties"));
String user = props.getProperty("user");
String password = props.getProperty("password");
String dburl = props.getProperty("dburl");
myConn = DriverManager.getConnection(dburl, user, password);
System.out.println("DB connection successful to: " + dburl);
}
public Connection getMyConn() {
return myConn;
}
//close class etc.
}

Usually not. Further answer depends on type of the application. If you're making web application then you should definitely go with connection pool. If you're making e.g. desktop application (where only one user can access it at the time), then you can open and close connection upon each request.

I have working applications that do it your way. As #Branislav says, it's not adequate if you want to do multiple concurrent queries. There's also a danger that the connection to the database might be lost, and you would need to restart your application to get a new one, unless you write code to catch that and recreate the connection.
Using a singleton would be overcomplicated. Having a getConnection() method (as you have done) is very important as it means you can easily change your code to use a pool later if you find you need to.

How does the PCFMessageAgent handle disconnecting?

I'm trying to get an understanding of how the PCFMessageAgent manages connections to ensure I'm cleaning up everything properly.
For example, at what point is the connection disconnected once I've created the object and I'm done with it?
PCFMessageAgent agent = new PCFMessageAgent ("localhost", 1414, "CLIENT");
The example in the documentation does not show any disconnect call, but I want to be sure I'm not assuming anything.
If it matters, I'm currently on MQ version 7.5, but it doesn't appear this has changed much.

Always cleanup your stuff.
If you connect it, then make sure you disconnect from it
If you open it, then make sure you close it
i.e.
if (agent != null)
agent.disconnect();

How to avoid big delays on concurrent database queries?

I am writing a small program, which is going to be launched on Apache web-server (not Tomcat) through CGI in respond to a POST request.
The program does the following:
read the xml, sent via http in request
execute a stored procedure in a database with the data extracted from the xml
return the result of the stored procedure as the respond to the POST request
The database is Oracle. I use jdbc OCI to access it.
Class.forName("oracle.jdbc.OracleDriver");
String dbCS = "jdbc:oracle:oci:#//ip:port/service"
Connection conn = DriverManager.getConnection(dbCS, dbUserId, dbPwd);
CallableStatement cs = conn.prepareCall("{ call ? := my_pkg.my_sp(?,?,?,?)}");
cs.registerOutParameter(pReturnValue, OracleTypes.NUMBER);
cs.setInt("p1", p1);
cs.setString("p2", p2);
cs.setString("p3", p3);
cs.registerOutParameter("p_out", Types.VARCHAR);
try {
cs.executeQuery();
return cs.getString(pReqResponse);
} finally {
try {
cs.close();
} catch (SQLException ex) {
//....
}
}
While doing a single request, it worked fine (the whole programm finished in 2 sec.). However, if I tryed to send multiple POST requests at once, I got all of them stuck for some amount of time, depending on the quantity of requests (it's approximately, 10 sec. for 10 req., 15 sec. for 15 req.).
I tried to estimate, which part of code gave the delay. It appeared to be two lines:
Connection conn = DriverManager.getConnection(dbConnectionString, dbUserId, dbPwd);
CallableStatement cs = conn.prepareCall("{ call ? := my_pkg.my_sp(?,?,?,?)}");
The execution itself finished almost immediatelly.
Why is this so?
P.S.: I experimented the same on Windows7. Of course, it wasn't launched from a web server, but just as a simple console process. It also has to read the xml from a file on a hard drive. All concurrently launched instances of the programm finished in a second all together.
What prevents it from working as fast on Linux through Apache?
Based on comments
I tried to set poolling properties for my connection but all in vain. I tried the following:
While specifying UserId and Password in the url
jdbc:oracle:oci:login/password#//ip:port/service
I tried to set the connection properties:
Properties p = new Properties();
p.setProperty("Pooling", "true");
p.setProperty("Min Pool Size", "1");
p.setProperty("Max Pool Size", "10");
p.setProperty("Incr Pool Size", "4");
Connection conn = DriverManager.getConnection(dbConnectionString, p);
I tried to use OCI Connection Pooling:
OracleOCIConnectionPool cpool = new OracleOCIConnectionPool();
cpool.setUser("user");
cpool.setPassword("pwd");
cpool.setURL(dbConnectionString);
Properties p = new Properties();
p.put(OracleOCIConnectionPool.CONNPOOL_MIN_LIMIT, "1");
p.put(OracleOCIConnectionPool.CONNPOOL_MAX_LIMIT, "5");
p.put(OracleOCIConnectionPool.CONNPOOL_INCREMENT, "2");
p.put(OracleOCIConnectionPool.CONNPOOL_TIMEOUT, "10");
p.put(OracleOCIConnectionPool.CONNPOOL_NOWAIT, "true");
cpool.setPoolConfig(p);
Connection conn = (OracleOCIConnection) cpool.getConnection();
I tried to use the apache DBCP component:
basicDataSource = new BasicDataSource();
basicDataSource.setUsername("user");
basicDataSource.setPassword("pwd");
basicDataSource.setDriverClassName("oracle.jdbc.OracleDriver");
basicDataSource.setUrl(dbConnectionString);
Connection conn = basicDataSource.getConnection();
The behaviour remained the same, i.e. a big delay on getConnection in all concurrent requests.
All these attempts seem to try to solve some other problem to me, as in my case all connections are established from separate processes, and it looks unobvious to manage connections from one pool among different processes (am I mistaken here??).
What options do I have? Or probably did I do anything wrong?
Also I should say, I am quite new to java in general, so I may be missing some basic things..
Could this be an OS or web-server issue? Probably something should be setup there, not in code...?
Also I tried to use thin client instead of oci. However it worked even more weirdly: the first request finished in a second, while the second delayed for a minute.
Poor concurrency with Oracle JDBC drivers states a problem similar to mine.
In the end we found out that processes, launched by Apache through CGI, occupied all 100% of CPU (and a lion share of memory), so they simply did not have enough resources. Unfortunately I do not know, why a very simple and basic programm (reading an xml and establishing one connection to DB to execute a stored procedure) launched simultanuosly only 20 times, eats all resources.
However the solution appeared to be very obvious indeed. I've refactored it to a java web application using servlets, we deployed it on Apache Tomcat, and MAGIC....it started working as expected, without any visible effect on resources.

I think the problem is with cgi. When you make a cgi request, it starts a new cpu process to handle the request. Each new request is also in a new JVM, so connection pooling is not an option.
Even so, it should be quicker than that to get a connection. Maybe in Oracle itself there are config options governing the number of concurrent connections you can have, but I'm no Oracle expert.

"Lazy initialization" of jdbc connections from jndi datasource/connection pool: feasibility

I have a main controller servlet in which i instantiate a datasource. The servlet opens and closes the connections. Mainly, the servlet instantiates a command from the application using the "factory pattern". here is some code to explain:
public void init() throws ServletException {
super.init();
try {
datasource =(DataSource) getServletContext().getAttribute("DBCPool");
}
catch (Exception e) {
}
}
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
//some code...
Connection connection = null;
if(cmd.mightNeedLazyLoadingAConnection)
{
connection = null;
}
else
connection = getConnection();//where getConnection is a method: datasource.getconnection();
//now a command (a java class) is instantied, to which the "null" CONNECTION obj is passed as parameter
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,connection);
//some code
//Then wherever there is catch exception i close() the connection
// and it is always closed in finally
finally {
if(connection!=null)
connection.close()
}
}
Now , this has a problem for the first case, ie connection=null, as it does never close the connection in the "finally" part (explained why in Update below).
"connection=null" is for cases where commands might not need to open a db connection because the data it is seeking for are cached in an identity map.
I tried to pass the "Connection" obj as a "null" parameter in the .execute(tsk,connection); and then in the corresponding java class to open a connection if needed
--> it did open the connection inside the command, however when process goes back to servlet : "Connection" is null as thus not closed.
What can i do to make the "Connection" obj's value get updated so that when back in servlet it is not "Null" anymore and i'd be able to close it?
I generally prefer to use a controller servlet that opens/close db connections, so what would be the best way to deal this kind of scenario where you have to do some sort of "lazy loading" a db connection from the pool and at the same time keep the opens/close of db connection assigned to the servlet?
Update (to explain further):
say i have a command : X.java
this command might/might not need a db connection (depends if the data searched for are in the identity map or not)
The system that i would like to have is:
(1)"client request"
(2)---> "Servlet": command.execute(connection)//where connection = null for now
(3) ---> "Command X": Do i need to go to database or record is in identity map?
(3.a) Case where it is needed to go to the database:
(3.a.1)connection = datasource.getconnection
(3.a.2) go get the data
(4)--->back to servlet: close "connection" in "Servlet"
Right now it is working until (3.a.2), but once back in (4) it appears that connection is still "null" and thus the code:
finally {
if(connection!=null)
connection.close()
}
Does not work (doesn't close the connection) and thus the db pool get drained like that.
How could connection - which starts as "null" and changes inside command "X"- get "globaly" updated to the its new value, and not only "updated" inside the scope of command "X"?
SOLUTION(S)
In case you are encountering the same scenario, you can chose of these 2 solutions:
You can Either use LazyConnectionDataSourceProxy, as mentionned by #Ryan Stewart for a "clean abstraction" and more professional solution
Or if you'd like use my solution described below (Basically i implemented a class similar to "LazyConnectionDataSourceProxy" but it is not as clean, it has less abstraction of details than "LazyConnectionDataSourceProxy")
My personal solution, Details:
I created a "Helper" class, which constructor takes the "datasource" as parameter
This helper class has methods to: "Lazy get" connection from pool,"close" connection
This class is instantiated in the servlet, and it gets a connection from the pool Only if needed throughout the application.
This is the code i added/modified in the servlet:
Connection connection = null;
if(cmd.mightNeedLazyLoadingAConnection)
{
helper hp = new helper(datasource);
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,hp);
}
else
{
connection = getConnection();
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,connection);
}
Then say in a command "X" , a db connection is needed i do:
Connection connection = hp.LazyGet();//Now got a connection from the pool
And this way, when proccess flow is back to the servlet level, i can :
Close
rollback
commit
etc..
All on this hp object of the helper class.
What benefits do i get from this:
I limit all database open / close / commit / rollback in one place, ie the Servlet, which is responsible of executing commands.
Having 3 cases: never needs db / always needs db / might need db thus now i decreased calls to the database by 1/3 , which is quite a lot knowing that database call grows exponentially with new features and new users registrations.
It might not be the Cleanest workaround, but between this way and having an additional "unnecessary" 1/3 database calls, it surely is better. Or just use LazyConnectionDataSourceProxy if you want a Tested, abstract and clean method.

Use a LazyConnectionDataSourceProxy. Then just get a "connection" every time, but a real connection is opened only when you actually do something that requires one. Thus you obey the "create/destroy" wisdom which Hiro2k pointed out because the connection's lifecycle is completely managed by your servlet.

In your specific case the only way to do it would be to return the connection. Java doesn't have any pass by reference semantics that could help you, unlike C where you could pass in the reference to the connection and then set it within the method.
I don't recommend your method return the connection, instead remember this simple rule and everything will work as you expect:
The object that creates it, is responsible for destroying it.
If what you want to do is not instantiate a connection for commands that don't require one, then add a method to your command interface that simply returns if needs one.
Command command = cmdFactory.getInstance().getCommand(Cmd);
if(command.requiresConnections){
connection = getConnection();
}
command.execute(tsk,connection);

Java outgoing TCP connection failover based on multiple DNS results

If I make a connection using new Socket("unit.domain.com", 100) and the unit.domain.com DNS record has multiple IP addresses in the A record.. In the event of a failed connection, Does Java automatically connect to one of the other addresses in the list like the browser does? or does that have to be implemented manually?

No!
Creating a socket via new Socket(String, int) results in a resolving like that
addr = InetAddress.getByName(hostname);
which is a shortcut for
return InetAddress.getAllByName(host)[0];
The address resolution is performed in the Socket c-tor.
If you have to reconnect (failover) use the result returned by InetAddress.getAllByName(host), randomize (or use round-robin) and connect to the necessary addresses.
Edit: also if you are going to need to connect with some likely failure, you'd be better off using connect method of the Socket class with a timeout. Also make sure you close even failed sockets (and esp. channels) since they may leak a FD on *Nix.