We Keep Coding

Why does setNetworkTimeout cause a timeout exception on queries with function?

I have a connection to a PostgreSQL DB (using the PostgreSQL JDBC Driver), and I set the network timeout on it (using the setNetworkTimeout method), and there is something weird about it.
When I use the connection for simple queries like select * from table, which takes a lot of time, it works fine (waits for the query to return a result). But when I use the connection for queries which use functions (like select max(a) from table), which also take a lot of time, it throws an exception, as a result of a timeout.
example code:
// Queries which takes more than 5 seconds
String bigQuery = "select * from data.bigtable tb1 inner join data.bigtable tb2 on tb1.a like '%a%'";
String bigQueryWithFunction = "select max(tb1.a) from data.bigtable tb1 inner join data.bigtable tb2 on tb1.a like '%a%'";
// Creating a connection with 5 seconds network timeout
Connection con = source.getConnection();
con.setNetworkTimeout(Executors.newSingleThreadExecutor(), 5000);
con.setAutoCommit(false);
Statement st2 = con.createStatement();
st2.execute(bigQueryWithFunction); // This line DOES throws an exception
st2.execute(bigQuery); // This line DOES NOT throws an exception
(Ignore the logic of the queries.)
Can someone explain to me why it happens?

PostgresSQL streams the result rows to the client as soon as they become available.
In your first query, the first result row will be returned quite soon, even though it takes the query a long time to finish. The JDBC driver collects the results and waits until the query is done, but the network connection won't be idle for any longer time.
The second query takes about as long to complete as the first one, but it cannot return its first (and only) result row until all result rows from the join have been calculated. So there is a long idle time on the network connection, which causes the timeout.

Sql Server data extraction (java jdbc): process hangs during retrieving rows

I need to extract data from a remote Sql server database. I am using the mssql jdbc driver.
I noticed that often dwhen retrieving rows from the database the process suddenly hangs, giving no errors. It remains simply stuck and no more rows are processed.
The code to read from the database is the following:
String connectionUrl = "jdbc:sqlserver://10.10.10.28:1433;databaseName=MYDB;user=MYUSER;password=MYPWD;selectMethod=direct;sendStringParametersAsUnicode=false;responseBuffering=adaptive;";
String query = "SELECT * FROM MYTABLE";
try (Connection sourceConnection = DriverManager.getConnection(connectionUrl);
Statement stmt = sourceConnection.createStatement(SQLServerResultSet.TYPE_SS_SERVER_CURSOR_FORWARD_ONLY, SQLServerResultSet.CONCUR_READ_ONLY) ) {
stmt.setFetchSize(100);
resultSet = stmt.executeQuery(query);
while (resultSet.next()) {
// Often, after retrieving some rows, process remains stuck here
}
}
Usually the connection is established correctly, some rows are fetched, than at some point the process can become stuck in retrieving the next rows batch, giving no errors and not processing any new row. This happens some times, other times it completes succesfully.
AFAIK the only reason I can see is that at some point a connection problem occurs with the remote machine, but shouldn't I be notified of this from the driver?
I am not sure how I should handle these type of situations...is there anything I can do on my side to let the process complete even if there is a temporary connection problem with the remote server (of course if the connection is not recoverable there is nothing I can do)?
As another test, instead of the java jdbc driver I've tried the bcp utility to extract data from the remote database and even with this native utility I can observe the same problem: sometimes it completes succesfully, other times it retrieves some rows (say 20000) and then becomes stuck, no errors and no more rows processed.

Understanding JDBC Timeout Variables and replication

I am using JDBC driver to connect to mySql from my java code (read client).
Driver = com.mysql.jdbc.Driver
JdbcUrl = jdbc:mysql://<<IpOftheDb>>/<<DbSchema Name>>?autoReconnect=true&connectTimeout=5000&socketTimeout=10000
In case the database is down ( machine hosting the db is up but the mysqld process is not running) , it takes some time to get the exception , The exception is
"com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:Could not create connection to database server. Attempted reconnect 3 times. Giving up."
In the statement above socketTimeout is 10 sec . Now if I bring up the db with 10 sec as SocketTimeout I get the response correctly.
But If i reduce it to one sec and am executing the query I get the same exception.
"com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:Could not create connection to database server. Attempted reconnect 3 times. Giving up."
But connectTimeout doesnt change anything. Can someone explain me what socketTimeout and connectTimeout means.
Also , If we are setting up replication and specifying the 2nd database as failover i.e.
my connection string changes to
jdbc:mysql://<<PrimaryDbIP>>,<<SecondaryDbIp>>/<<DbSchema>>?useTimezone=true
&serverTimezone=UTC&useLegacyDatetimeCode=false
&failOverReadOnly=false&autoReconnect=true&maxReconnects=3
&initialTimeout=5000&connectTimeout=6000&socketTimeout=6000
&queriesBeforeRetryMaster=50&secondsBeforeRetryMaster=30
I see that if primary is down then I get the response from secondary (failover Db) .
Now when client executes a query , does it go to primary database and then wait for socketTimeout (or whatever) and then goes to Secondary or it goes to Seconday before timeout occurs.
Moreover, the second time when the same connection Object is used , does it go directly to the secondary or again the above process is repeated .
I tried find some documentation which explains this but couldnt get .
Hopefully , someone can help here explaining the various timeout parameters and their usefulness.

Apache Tomcat JDBC Connection Pool bad performance on batch \ bulk inserts

I have recently incorporated the Apache Tomcat JDBC Connection Pool to my application (using MySQL DB). I tried using Apache DBCP before, but didn't like its results, and the tomcat implementation seemed to fit my needs even though I run a standalone java application and don't use tomcat at all.
Recently, I encountered a huge performance problem when executing batch (aka bulk ) insert queries.
I have a flow in which I insert ~2500 records to a table in a batched fashion. It takes forever when using the jdbc connection pool, compared to a few seconds when reverting back to opening a connection for each query (no pooling).
I wrote a small application that inserts 30 rows to the same table. It takes 12 seconds when pooling, and ~ 800 millis when not pooling.
Prior to using the connection pool, I used com.mysql.jdbc.jdbc2.optional.MysqlDataSource as my DataSource. The connection was configured with the following line:
dataSource.setRewriteBatchedStatements(true);
I'm quite sure that this is the core difference between the two approaches, but couldn't find an equivalent parameter in jdbc-pool.

MySql JDBC driver does not support batch operations. RewriteBatchedStatement is the best that you can get. Here the code from mysql PreparedStatement.java:
try {
statementBegins();
clearWarnings();
if (!this.batchHasPlainStatements && this.connection.getRewriteBatchedStatements()) {
if (canRewriteAsMultiValueInsertAtSqlLevel()) {
return executeBatchedInserts(batchTimeout);
}
if (this.connection.versionMeetsMinimum(4, 1, 0) && !this.batchHasPlainStatements && this.batchedArgs != null
&& this.batchedArgs.size() > 3 /* cost of option setting rt-wise */) {
return executePreparedBatchAsMultiStatement(batchTimeout);
}
}
return executeBatchSerially(batchTimeout);
} finally {
this.statementExecuting.set(false);
clearBatch();
}
It is one of the reason why I do not like MySql and prefer Postgres
EDIT:
You should combine connection pool, batch operation, and RewriteBatchedStatement option. You can set RewriteBatchedStatement option through jdbc url parameter: jdbc:mysql://localhost:3307/mydb?rewriteBatchedStatements=true

SQL execution time much slower in a Tomcat Servlet than in a normal Java program

For inexplicable reasons however, this morning the performance increased for two of my Queries that used to be slow. I have no idea why.
I have no authority over the server, maybe someone changed something.
The problem is no more.
In a nutshell:
s.executeQuery(sql) runs extremely slowly within a tomcat servlet on server
Same query runs fine without servlet (simple java program) on the same machine
Not all queries are slow within the servlet. Only a few bigger ones do
Same servlet runs fast on another machine
UPDATES
Please read the updates below the text !
I have a servlet that executes SQL requests and sends back the results via JSON. For some reason, some requests take a huge amount of time to execute, but when I run them in any Oracle SQL Client, they are executed in no time.
I am talking about a difference of 1 second vs 5 minutes for the same SQL (that is not that complex).
How can this be explained ?
Is there a way to improve the performance of a java based SQL request ?
I am using the traditional way of executing queries:
java.sql.Connection conn = null;
java.sql.Statement s = null;
ResultSet rs = null;
String dbDriver = "oracle.jdbc.driver.OracleDriver";
String dbConnectionString = "jdbc:oracle:thin:#" + dbHost + ":" + dbPort + ":" + dbSid;
Class.forName(dbDriver).newInstance();
conn = DriverManager.getConnection(dbConnectionString, dbUser, dbPass);
s = conn.createStatement();
s.setQueryTimeout(9999);
rs = s.executeQuery(newStatement);
ResultSetMetaData rsmd = rs.getMetaData();
// Get the results
while (rs.next()) {
// collect the results
}
// close connections
I tried with ojdbc14 and ojdbc6 but there was no difference.
UPDATE 1:
I tried the same SQL in a local Java project (not a servlet) on my client machine, and I get the results immediately. So I assume the problem is coming from my servlet or the tomcat configuration ?
UPDATE 2:
The culprit is indeed rs = s.executeQuery(mySql); I tried to use preparedStatement instead, but there is no difference.
UPDATE 3:
I created a new Servlet running on a local Tomcat and the Query comes back fast. The problem is therefore coming from my production server or Tomcat config. Any ideas what config items could affect this ?
UPDATE 4:
I tried the same code in a normal java program instead of a servlet (still on the same server) and the results are coming fast. Ergo the problem comes from the Servlet itself (or Tomcat ?). Still don't know what to do, but I narrowed it down :)
UPDATE 5:
Jstack shows the following (It starts where my servlet is, I cut the rest)
"http-8080-3" daemon prio=3 tid=0x00eabc00 nid=0x2e runnable [0xaa9ee000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at oracle.net.ns.Packet.receive(Packet.java:311)
at oracle.net.ns.DataPacket.receive(DataPacket.java:105)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:305)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:249)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:171)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:89)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79)
at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:429)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:397)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:30)
at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:762)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:925)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1104)
at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1309)
- locked <0xe7198808> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:422)
So i am Stuck at java.net.SocketInputStream.socketRead0(Native Method) ?

In some cases (not sure if this applies to yours) setting fetchSize on the Statement object yields great performance improvements. It depends on the size of the resultSet that is being fetched.
Try playing with it by setting it to something bigger than default 10 for Oracle (see this link).
See Statement.setFetchSize.

Given your symptoms, I believe that your issue is not with your SQL client code and you are in fact looking at issues with your server. The stack shows that your client is waiting for a response. This tallies with the fact that you can run the client without any problem in a separate process.
So what you probably need to look at is systemic reasons why the SQL server is running slowly and how that may be tied to Tomcat. My experience in cases like this is its usually the disk, so I'd be inclined to check whether you are paging due to a lack of RAM when Tomcat is loaded, or suffering from much higher disk ops due to a reduced disk cache. Assuming you are running on a UNIX variant, I'd have a look at vmstat and iostat for a working and broken case to eliminate such issues.

For inexplicable reasons however, this morning the performance increased and my problem is no more. I have no idea why. I have no authority over the server, maybe someone changed something.

Since your thread is waiting on socket read, which means is waiting for a response from the database server I would :
Check database performance, make sure not the instance nor the query is getting impacted at some point in time during the day?
Check your network latencies between Java and DB Servers. Same as above. Probably traceroute?

Since you have not put the query, I can give you a scenario where it is possible. If you use a function in your query like to_char etc. then your table indexes wouldn't be used while executing query via JDBC but will work fine you run it in console. I don't exactly know why but there's something with JDBC driver. I had the exact same issue in db2 and I resolved it removing the use of functions.
Other scenario could be that a huge no of records is being fetched and proper batching is not implemented.