We Keep Coding

Should the connection be closed after each query in a long time transaction?

We are currently dealing with the function, that has to work partially with the database, and partially with a service, whice operations are time-consuming. So, generally speaking, here is a transactional method, that has a code like this:
Connection conn = null;
try {
conn = getConnection(); // This I get from the connection pool
Employee emp = queryDatabase(id);
// Point A - ??
Response resp = makeLongTimeServiceCall(emp);
// Point B - ??
anotherQueryDatabase(resp);
} catch (Exception e) {
throw e; // And this also rolls back the transaction
} finally {
// If conn is not closed, close it, so there would be no leaks
}
So the big question is - should I close the connection at point A and then get it again from the connection pool at point B, so that other servlets could use that connection while I interact with the service? My guess is that I should, but will this hurt my transaction?

In many circumstances: yes, closing and reopening the connection sounds good. However you need to understand the implication of doing them in two separate transactions (by closing and re-opening the connection you are inherently doing them in separate transaction).
If another user invokes an operation that changes the state of your data at Point B will the end result still be correct?

If you don't have a transaction you can close the connection and ask for a new connection when needed.
Remember that if you are using a connection pool closing a connection will not really close the connection, but only flag it as reusable by other threads.

Database connections shouldn't be left open. Open the connections just when you need to execute a query and close it as early as possible. So my solution is, YES.

if your application doesn't close connection properly may lead to some issues like the
connection pool maxing out.
Applications Not Properly Closing Connections:
When writing an application to use the WebSphere Application Server(WAS) datasource, the best way is fallowing get/use/close pattern.
1.get
-This is when the application makes a request to get a connection from the pool.
The application will first lookup the datasource name and then do a getConnection() call.
2.use
-This is when the application executes a query and waits for a response
3.close
-This is the final stage when the application has received the response from the database and is now done using that connection.
-The application should call close() in a finally block to close out the connection and return it to the free pool.
If your application does not follow this pattern, you may see connections staying open for long periods of time. This is especially seen when the application does not close the connection properly. After the response is received from the database, the application should be calling close in a finally block. If the application does not explicitly close the connection, that connection will stay open forever in WAS until that server is restarted. This is known as a connection leak. To resolve this type of issue, you will have to modify your application to close every connection
for further information: https://www.ibm.com/support/pages/common-reasons-why-connections-stay-open-long-period-time-websphere-application-server

java.sql.SQLException: Listener refused the connection with the following error: ORA-12519, TNS:no appropriate service handler found

I am passing Resultset object to each thread. Each thread is connecting to the database and inserting data. Untill thread 110 it is working fine. After it crosses 111 thread it throws the above exception.
I am using oracle 11g.
My sample Thread code is:
class MyThreadClass implements Runnable
{
public Connection connection;
public Statement statement2;
public ResultSet rs2;
public String cookie;
public MyThreadClass(ResultSet rs1)
{
rs2=rs1;
}
public void run()
{
try
{
cookie=rs2.getString("COOKIE");
driver = "oracle.jdbc.driver.OracleDriver";
url = "jdbc:oracle:thin:#127.0.0.1:1521:xx";
/* connection
statement2.executeUpdate("INSERT INTO visit_header VALUES ('"+cookie+"')");
}
I am not getting how to handle this exception.

Your multi-threaded application is opening too many Connections/Sessions. Hence, the listener is dropping and blocking new connections for a while.
Check your DB resource usage first:
SELECT * FROM v$resource_limit WHERE resource_name IN ('processes','sessions');
Check to see if your MAX_UTILIZATION for either your Processes or Sessions is getting too close to the LIMIT_VALUE. If yes, you should either:
Use DB Connection pooling to share Connection objects between threads. Or,
Increase the number of processes/sessions that Oracle can service simultaneously.
Actually, Connection Pooling (#1) should always be done. An application cannot scale up otherwise. Check Apache Commons DBCP for details. For #2, open a new SQL*Plus session as SYSTEM and run:
ALTER system SET processes=<n-as-per-number-of-threads> scope=spfile;
to increase backend concurrency. Then RESTART the Database. IMPORTANT!

I guess the database just don't accept more connections from your host. If I understand your question right you are making maybe 100 threads which each connects to the database in short time. Maybe you don't even close the connection correctly, or the accesses are lasting so long that a huge amount of connections are opened. The database have a limit to which it accepts connections.
You should definitely reduce the number of connections by some clever technique. Maybe reduce the number of concurrent threads and/or use a connection pool.

Try this solution at your end. It worked for me.
Close the connection in try/catch block and just after closing the connection,
write-
Thread.sleep(1000);
In this case you can write it as-
finally {
try {
if (conn != null && !conn.isClosed())
{
conn.close();
Thread.sleep(1000);
}
}
catch (SQLException e) {
e.printStackTrace();}
}

Each request from user call a new getConnection()?

The database connection is get like below
public Connection getDBConection(){
Context context = new InitialContext();
DataSource dataSource = (javax.sql.DataSource) context.lookup("java:myDataSource");
Connection conn = dataSource.getConnection();
}
For a userA, is each database request should call getDBConnection() once; but no need to control all request use the same connection?
That is, if userA has three database request, then userA should call getDBConnection() three times, and call Connection.closed() after used in each request?
If the userA call getDBConnection() three times (that is, call dataSource.getConnection() three times), is three connection created? Or it is unknown and controlled by weblogic?
I feel very chaos, is it true that there should be one new connection for one database request? or just call DataSource.getConnection() for each database request and the number of new connection created is controlled by web server, no need to think how many connection is actually created.

Every time you call DataSource.getConnection, the data source will retrieve a connection for you. It should be true that the returned connection is not being actively used by anyone else, but it is not necessarily a brand-new connection.
For example, if you use a connection pool, which is a very common practice, then when you call Connection.close, the connection is not actually closed, but instead returns to a pool of available connections. Then, when you call DataSource.getConnection, the connection pool will see if it has any spare connections lying around that it hasn't already handed out. If so, it will typically test that they haven't gone stale (usually by executing a very quick query against a dummy table). If not, it will return the existing connection to the caller. But if the connection is stale, then the connection pool will retrieve a truly new connection from the underlying database driver, and return that instead.
Typically, connection pools have a maximum number of real connections that they will keep at any one time (say, 50). If your application tries to request more than 50 simultaneous connections, DataSource.getConnection will throw an exception. Or in some implementations, it will block for a while until one becomes available, and then throw an exception after that time expires. For a sample implementation, have a look at Apache Commons DBCP.
Hopefully that answers your question!

is public Connection getInstance(String databasename); a form of connection pooling?

In a javase database application, I process a lot of short-lived object (say accounting documents like bills etc.). Processing each object consists in opening a connection to a database and looking up for some data. Not all objects are looked up on the same database, but i select a specific database according to some object property, so i'll end up having several connections opened.
What i actually need is no more than one connection for each database.
So I've done something like this:
public MyPool {
Map<String, Connection> activeConnections = new TreeMap<String, Connection>();
public Connection getConnection(String database_name) throws SQLException {
if (activeConnections.containsKey(database_name)) {
return activeConnections.get(database_name);
}
//Retrive the configuration data from a configuration object
ConnectionConfig c = Configuration.getConnectionConfig(database_name);
Connection connection = DriverManager.getConnection(c.url, c.user, c.password);
return connection;
}
The questions are:
1) Since I see around a lot of pooling libraries, DBCP, c3p0 and others: what is the point of all those libraries, what do they add to a "basic" approach like this?
Tutorials like this doesn't help much in responding this question, since the basic solution exposed here fits perfectly their definition of connection pooling.
2) This is a piece of code that will be "exposed" to other developers, that in turn may develop procedures to retrieve data from databases with different structures, probably getting connections from this "pool object".
Is it correct, in the docs and in the code, referring to it as a "pool", or is it something different, so that calling "pool" would be misleading?

Your code isn't a connection pool implementation in the colloquial use of the term since each datasource only has a single physical connection. The concept behind object pooling (in this case, the object is a connection) is that some objects require overhead to configure. In the case of a connection pool, as you know, a database connection must be opened before you can talk to the database.
The difference here is that your code isn't thread safe for a concurrent environment like the popular connection pool implementations you've mentioned. Applications running in high concurrency circumstances like the web shouldn't need to absorb the overhead of establishing a connection on each request. Instead, a pool of open connections is maintained and when the request has finished working on the connection, it is returned to the pool for subsequent requests to make use of.
This is required because connections are stateful. You can't have multiple requests sharing the same connection at the same time and guarantee any sort of reasonable transaction semantics.

Use BoneCP and wrap the connection pool like this:
Do not try to create your own connection pool, that is what BoneCP or any number of other very good and well tested pools are for.
object ConnectionPool {
Class.forName("[ENTER DRIVER]")
private val connstring = [ENTER YOUR STRING]
private var cp : BoneCP = _
createConnectionPool() //upon init create the cp
/**
* Create a new connection pool
*/
def createConnectionPool() = {
if(cp == null) {
try {
val config = new BoneCPConfig()
config.setJdbcUrl(connstring)
config.setMaxConnectionsPerPartition(3)
config.setMinConnectionsPerPartition(1)
config.setPartitionCount(1)
cp = new BoneCP(config)
}
catch {
case e: SQLException => e.printStackTrace()
}
}
}
def getConnection () = { cp.getConnection }

Queue method or individual db connection?

I have socket connection which will send data into a queue via databaseQueue.add(message);. Next the the DatabaseProcessor class which is started as thread during the start where single database connection will be made. The connection will keep taking the message via databaseQueue.take(); and process. The good part about this part everything is that just one database connection is made. The problem arises when suddenly there is a surge of data. So another method is that for each data received I will open and close method. So based your experiences for heavy loads which is the best way to go here?
Some snippet of my codes.
class ConnectionHandler implements Runnable {
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}
// gets data from an inbound connection and queues it for databse update
public void run() {
databaseQueue.add(message); // put to db queue
}
}
class DatabaseProcessor implements Runnable {
public void run()
{
// open database connection
createConnection();
while (true)
{
message = databaseQueue.take(); // keep taking message from the queue add by connectionhandler and here I will have a number of queries to run in terms of select,insert and updates.
}
}
void createConnection()
{
System.out.println("Crerate Connection");
connCreated = new Date();
try
{
dbconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test1?"+"user=user1&password=*******");
dbconn.setAutoCommit(false);
}
catch(Throwable ex)
{
ex.printStackTrace(System.out);
}
}
}
public void main()
{
new Thread(new DatabaseProcessor()).start(); //calls the DatabaseProcessor
//initiate the socket
}

As far as I understand you are managing a client-server Socket connection in which you send and receive message through a queue. If I also got it right, you are creating a thread for each new message on the queue.
Considering that there will be plenty of messages being sent and read I recommend you to declare your method(s) in your threads Synchronized so that you won't need to open and close streaming each time a data is received (refer to your second approach here). Synchronized Methods are usually the best way to handle surge of common data which can be modified by threads at the same time.

You can use connection pooling to get the best of both worlds: you are not limited to a single thread, and you also do not need to open connections for each request. Have a look at Apache DBCP.

This approach is fine. Except you can create an a DB pool using c3pO. Also use threadPool executor for miantaining your thread pool.