I need to extract data from a remote Sql server database. I am using the mssql jdbc driver.
I noticed that often dwhen retrieving rows from the database the process suddenly hangs, giving no errors. It remains simply stuck and no more rows are processed.
The code to read from the database is the following:
String connectionUrl = "jdbc:sqlserver://10.10.10.28:1433;databaseName=MYDB;user=MYUSER;password=MYPWD;selectMethod=direct;sendStringParametersAsUnicode=false;responseBuffering=adaptive;";
String query = "SELECT * FROM MYTABLE";
try (Connection sourceConnection = DriverManager.getConnection(connectionUrl);
Statement stmt = sourceConnection.createStatement(SQLServerResultSet.TYPE_SS_SERVER_CURSOR_FORWARD_ONLY, SQLServerResultSet.CONCUR_READ_ONLY) ) {
stmt.setFetchSize(100);
resultSet = stmt.executeQuery(query);
while (resultSet.next()) {
// Often, after retrieving some rows, process remains stuck here
}
}
Usually the connection is established correctly, some rows are fetched, than at some point the process can become stuck in retrieving the next rows batch, giving no errors and not processing any new row. This happens some times, other times it completes succesfully.
AFAIK the only reason I can see is that at some point a connection problem occurs with the remote machine, but shouldn't I be notified of this from the driver?
I am not sure how I should handle these type of situations...is there anything I can do on my side to let the process complete even if there is a temporary connection problem with the remote server (of course if the connection is not recoverable there is nothing I can do)?
As another test, instead of the java jdbc driver I've tried the bcp utility to extract data from the remote database and even with this native utility I can observe the same problem: sometimes it completes succesfully, other times it retrieves some rows (say 20000) and then becomes stuck, no errors and no more rows processed.
Related
I have a connection to a PostgreSQL DB (using the PostgreSQL JDBC Driver), and I set the network timeout on it (using the setNetworkTimeout method), and there is something weird about it.
When I use the connection for simple queries like select * from table, which takes a lot of time, it works fine (waits for the query to return a result). But when I use the connection for queries which use functions (like select max(a) from table), which also take a lot of time, it throws an exception, as a result of a timeout.
example code:
// Queries which takes more than 5 seconds
String bigQuery = "select * from data.bigtable tb1 inner join data.bigtable tb2 on tb1.a like '%a%'";
String bigQueryWithFunction = "select max(tb1.a) from data.bigtable tb1 inner join data.bigtable tb2 on tb1.a like '%a%'";
// Creating a connection with 5 seconds network timeout
Connection con = source.getConnection();
con.setNetworkTimeout(Executors.newSingleThreadExecutor(), 5000);
con.setAutoCommit(false);
Statement st2 = con.createStatement();
st2.execute(bigQueryWithFunction); // This line DOES throws an exception
st2.execute(bigQuery); // This line DOES NOT throws an exception
(Ignore the logic of the queries.)
Can someone explain to me why it happens?
PostgresSQL streams the result rows to the client as soon as they become available.
In your first query, the first result row will be returned quite soon, even though it takes the query a long time to finish. The JDBC driver collects the results and waits until the query is done, but the network connection won't be idle for any longer time.
The second query takes about as long to complete as the first one, but it cannot return its first (and only) result row until all result rows from the join have been calculated. So there is a long idle time on the network connection, which causes the timeout.
I have the following java code fragment:
Connection conn = DriverManager.getConnection(connString, user, password);
Statement stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);
stmt.setMaxRows(100);
stmt.setQueryTimeout(2);
try {
stmt.execute("SELECT SLEEP(100)"); // (1)
} finally {
stmt.close();
conn.close();
}
It is designed to simulate a long running query that will time out. However, it raises an exception that is not the expected one:
Streaming result set com.mysql.jdbc.RowDataDynamic# is still
active. No statements may be issued when any streaming result sets are
open and in use on a given connection. Ensure that you have called
.close() on any active streaming result sets before attempting more
queries. java.sql.SQLException: Streaming result set
com.mysql.jdbc.RowDataDynamic# is still active. No
statements may be issued when any streaming result sets are open and
in use on a given connection. Ensure that you have called .close() on
any active streaming result sets before attempting more queries.
Looking closer, it appears the error happens as the close method attempts to reset the MaxRows to its default value. However, if I replace line (1) with
stmt.executeQuery("SELECT SLEEP(100)");
The proper exception, com.mysql.jdbc.exceptions.MySQLTimeoutException is raised.
I would like to run execute, stream the results, have a limit on the number of rows, and not have this error crop up. Any suggestions?
I have a code in Java for insert many sql lines in database from a text file.
I have the connection programmed with setAutoCommit(false), then at the end if none error happen (detected having all methods throws Throwable), I send the commit.
The task normally take 30 minutes.
It works very good in cable connection, but in a wifi connection it never reach the end because the connection is lost sometimes, for a short time.
For solve it. I programmed two things: The lines of the text file is converted to a serialized object that have all the lines in a ArrayList, and I created other serialized object that have a int index, that save the index of the last line inserted succesfully.
Then, in the program I do this:
charge in memory the object.lines, from the serialized object.
charge in memory the object.index, from the serialized object.
pseudo code:
loop:
index = sum 1 to the object.index
line = object.getLine(index)
insert line
if error continue (or goto loop)
send commit
if error continue (or goto loop)
object.index = index
serialize object
in this way I have a backup of the lines that are succesfully commited to the database, and I can continue the job in other time. If I have a connection problem in a line, I can try insert again the line.
If i have a connection problem, i wait 1 minute. The connection is recovered but is reseted automatically, not by me.
Then for example in a lines like this:
INSERT INTO my_table1 (id) VALUES (sq_mytable1_id.NEXTVAL);
//success
//connection lost
//connection reset
INSERT INTO my_table2 (id) VALUES (sq_mytable1_id.CURRVAL);
//error, sq_mytable1_id.CURRVAL is not in session.
I get a ORA-08002 exception because, the connection was reset, I can get the sq_mytable1_id.CURRVAL from the session.
Please, you can give me ideas of how programm a batch sql inserter tolerant to connection downs in wi-fi ?
I think serialize the connection, but I cannot: oracle.jdbc.driver.T4CConnection is not serializable.
I'm trying to run a simple Android application that has a button click event which connects to database and retrieves a user from Clients table depending on the text input.
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection connection = DriverManager.getConnection("jdbc:sqlserver://ServerUrl;databaseName=DBNAME;selectMethod=cursor;user=USER;password=*****");
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery("SELECT * FROM Clients WHERE IMEI = "+imei+"");
However, when I run the project with a breakpoint on the line that begins with "Statement" I notice that the code does not event arrive at that line.
I tried surrounding the connection line with try-catch but it throws no exception, it simply does nothing. The database is hosted so it's not on localhost.
Please keep in mind that I'm very new to Android development and do not burn me if I'm missing something very simple :)
I am trying to connect to MSSQL server 2008 on my localhost, but I am getting Errors
WARNING: ConnectionID:2 Prelogin error: host 127.0.0.1 port 1434 Error reading prelogin response: Connection reset
this error repeats like 20 times very quickly, then i get
com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
I suspect this is some wrong configuration of server. I can connect to external servers with no issues. I can also connect via management studio with no problems.
Please help me, I am getting sick of this issue :|
1) TCP protocol for MSSQL 2008 is disabled by default. You need to enable it from SQL Configuration Manager (in the same [Start], Programs folder as MSSQL):
http://msdn.microsoft.com/en-us/library/ms190425%28v=sql.105%29.aspx
2) To use TCP connections (even on localhost!) you need to allow MSSQL in Windows Firewall:
http://technet.microsoft.com/en-us/library/cc646023.aspx
3) You don't necessarily need to use TCP/IP in order to access MSSQL:
http://msdn.microsoft.com/en-us/library/ms187892%28v=sql.105%29.aspx
'Hope that helps!
To connect to MSSQL Server from a Java application, you need to use the JDBC API. The JDBC API provides classes and methods that connect to the database, load the appropriate driver, send SQL queries, retrieve results etc.
HOW TO CONNECT TO THE DATABASE A ‘Connection’ object represents a connection with a database. To establish the connection, use the method ‘DriverManager.getConnection’. This method takes a string containing a URL which represents the database we are trying to connect to. Below is the sample code for establishing a connection:
private String DATABASE_URL = "jdbc:odbc:embedded_sql_app"; // establish connection to database
Connection connection = DriverManager.getConnection( DATABASE_URL,"sa","123" );
Detailed discussion about the Database URL and how to create it can be found in the resource provided at the end of this post.
QUERYING THE DATABASE The JDBC API provides three interfaces for sending SQL statements to the database, and corresponding methods in the ‘Connection’ interface create instances of them. 1. Statement - created by the ‘Connection.createStatement’ methods. A ‘Statement’ object is used for sending SQL statements with no parameters.
2. PreparedStatement - created by the ‘Connection.prepareStatement methods’. A ‘PreparedStatement’ object is used for precompiled SQL statements. These can take one or more parameters as input arguments (IN parameters).
3. CallableStatement - created by the ‘Connection.prepareCall’ methods. ‘CallableStatement’ objects are used to execute SQL stored procedures from Java database applications.
RETRIEVING THE RESULT A ‘ResultSet ‘is a Java object that contains the results of executing a SQL query. The data stored in a ‘ResultSet’ object is retrieved through a set of get methods that allows access to the various columns of the current row. The ‘ResultSet.next’ method is used to move to the next row of the ‘ResultSet’, making it the current row. The following code fragment executes a query that returns a collection of rows, with column ‘a’ as an ‘int’, column ‘b’ as a ‘String’, and column ‘c’ as a ‘float’:
java.sql.Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("SELECT a, b, c FROM Table1");
while (rs.next()) { // retrieve and print the values for the current row
int i = rs.getInt("a");
String s = rs.getString("b");
float f = rs.getFloat("c");
System.out.println("ROW = " + i + " " + s + " " + f); }
This is just a brief introduction on how to interact with a database from Java. For more details on the items discussed above as well as information on passing parameters, executing stored procedures etc. please refer to the following resource: ( http://www.shahriarnk.com/Shahriar-N-K-Research-Embedding-SQL-in-C-Sharp-Java.html#Shahriar_N_Embedding_SQL_in_Java ) Here, you will also find information on how to interact with a database programmatically; i.e. without using SQL. Hope you find this useful.