Retry Database connection Dropwizard - java

How do I retry to connect to my database if the connection does not work for the first time?
I am using jdbi3 for my database connection
public static void main(String[] args) throws Exception {
startApp(args);
}
private static void startApp(String[] args) throws Exception {
try {
new Application().run(args);
} catch(SQLException ex) {
System.out.println("Could not connect to database.");
System.out.println("Try reconnecting...");
TimeUnit.SECONDS.sleep(2);
startApp(args);
}
}
I implemented this since new Application().run(args) is throwing the SQLException if there was no connection but the exception never got caught.

This will eventually cause a stack overflow. Better:
while (true) {
try (
new Application().run(args);
return;
} catch (SQLException e) {
continue;
}
}
But that's not all: This will retry on any SQL exception. So, if, say, the db is up but the first thing the app does is, say, create a table, but that table already exists, this will just end up re-trying and failing, forever, at a 2 second interval, and that's quite yucky. I suggest checking which -actual- SQLException is thrown if the DB is down (presumably, that's why you want to retry: Wait for the DB to return) and catching ONLY that one. Note ethat SQLException has more methods than most exception types do, such as .getSQLState(), which you should check to figure out if the error is in fact 'cannot connect to DB engine', vs some other problem.
Furthermore, it is likely that Application's run() method is catching all exceptions, and will log them, which means your attempt to catch them isn't going to do anything. In this case, you'd have to edit the code of run().

Related

AssertThrows failing the test even after the right exception is thrown

#Test
void testConnectToInvalidUrl() {
model.setCredentials("cst8288", "8288");
assertThrows(SQLException.class,() -> model.connectTo("jdbc:mysql://localhost:3306/redditreader?useUnicode=true&serverTimezone=UTC&useSSL=falses"));
}
I'm trying to test this method which connects to a MySQL database in JUnit by entering a wrong url as it will throw an SQLException, which it does, but the test fails regardless. Here is the actual method:
public void connectTo(String url) {
try {
connection = DriverManager.getConnection(url, User, Pass);
} catch (SQLException e) {
e.printStackTrace();
}
}
The Exception thrown :
java.sql.SQLException: The connection property 'useSSL' only accepts values of the form: 'true', 'false', 'yes' or 'no'. The value 'falses' is not in this set.
I know its the extra s at the end, but since its a SQLException, shouldn't assertThrows make the test pass?

Test an Oracle DB connection in Java, with no username/password

I know this is very similar to questions already answered, but there is a slight variation.
I have a list of connections in my production connection setup. The process is to start with the first and keep trying till I get a connection. I would like to be able to run a task that used this same list as its input, but did just enough to show which of the connections will be used by the application. To avoid our security team getting all upset, this would have to be done without the username/password.
Is it possible?
Below answer may be helpful to you. getErrorCode() method in SQLException returns 1017 value on authentication failure. So you can iterate through list of connections and invoke validateConnection.
I'm using dummy username and password here (I don't see any other option)
Replace host, port and SID values.
public static void main(String[] args) {
String connString = "jdbc:oracle:thin:#host:port:SID";
System.out.println(validateConnection(connString));
}
public static boolean validateConnection(String connString) {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection con = DriverManager.getConnection(connString, "x", "y");
} catch (SQLException sqle) {
if (sqle.getErrorCode() == 1017)
return true;
} catch (Exception e) {
e.printStackTrace();
}
return false;
}

Difference between Exception and SQLException

Can someone explain the difference between catching an Exception and catching an SQLException? I know that SQLException will print out more information if you choose to print out the exception errors, but is there anything else?
try {
//code
} catch(Exception ex) {
//code
}
And
try {
//code
} catch(SQLException ex) {
//code
}
What are the benefits and differences of using Exception and SQLException in the catch block?
This is not the only difference.
Catching Exception is dangerous because it also catches all RuntimeExceptions (therefore unchecked exceptions), and that include niceties such as NullPointerException etc which are clear programmer errors. Don't do that!
Also, Exception is a class like any other, so you can subclass it and add constructors/methods of yours. For instance, SQLException has a .getErrorCode() method which Exception does not have. If you only catch Exception, you cannot access this method.
In general, catching the "more precise" exception first is the best. For instance, with the new (in Java 7...) file API, you can easily distinguish between filesystem level errors and other I/O errors, since FileSystemException extends IOException:
try {
something();
} catch (FileSystemException e) {
// fs level error
} catch (IOException e) {
// I/O error
}
It's all about the hierarchy,when you are talking about the catching the exception.
Technically speaking, Exception - is the super class which catches each and every exception.
If you are writing something related to SQL in the try block and you know it may even throw SQL Exception.
Then you may declare it this way as well.
try
{
}catch(SQLException ex)
{
Do this,when SQL Exception is caught.
}
catch(Exception ex)
{
Generic Exception - works for all
}
SQLException inherits from Exception, so SQLException will contain more (and more specific) information than Exception (which is intended to apply generally to all exceptions).
You can also have multiple catch clauses; so you can first try to catch the SQLException, but if it's not a SQLException, then you can just catch the general Exception.
In general, you shouldn't catch exceptions unless you intend to handle them in some way. You can have a top-level exception handler that catches any exceptions that bubble up to the top of the call stack, so that your program doesn't crash on unhandled exceptions.
A - Explanation
SQLException is a subtype of java.lang.Exception and also it is implementing the Iterable<Throwable> class. Programmers prefer throwing different subtypes of Exception class because on some higher level, they want to catch the exact sub-Exception class so that they can be sure that that specific Exception is thrown on some exact scenario. Thus, they can know the exact source of Exception.
B - Example
Consider you have written a method that throws multiple exceptions. Let's say, you take a json String and parse it, then persist it on the database. Consider the following method;
public boolean persistCustomer(String jsonString) throws SQLException, IOException {
Connection conn = getConnection();
PreparedStatement preparedStatement = null;
ObjectMapper objectMapper = new ObjectMapper();
try {
Customer customer = objectMapper.readValue(jsonString, Customer.class);
preparedStatement = conn.prepareStatement(PERSIST_CUSTOMER);
preparedStatement.setString (1, customer.getName());
preparedStatement.setInt (2, customer.getAge());
preparedStatement.setBoolean (3, customer.getIsRegular());
preparedStatement.executeUpdate();
return true;
} catch (IOException e) {
throw e;
} finally {
try {
if (preparedStatement != null)
preparedStatement.close();
if (conn != null)
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
In this method, we are converting JSON into a Customer class and also we persist customer class to the database.
The following lines throw SQLException;
preparedStatement = conn.prepareStatement(PERSIST_CUSTOMER);
preparedStatement.setString (1, customer.getName());
preparedStatement.setInt (2, customer.getAge());
preparedStatement.setBoolean (3, customer.getIsRegular());
preparedStatement.executeUpdate();
prepareStatement(), setters and executeUpdate() methods, all of them throwing SQLException's. But also, the line that which we convert JSON in a String into a Customer object, also throws several Exceptions other than SQLException.
Customer customer = objectMapper.readValue(jsonString, Customer.class);
readValue() method throws JsonParseException, JsonMappingException and also IOException. All of them can be catched using an IOException because the JSON related exceptions extend IOException.
I'm going to provide two different examples so that it will be obvious to understand why we need different types of Exceptions.
C - Bad Practice: Using Exception To Catch All Exceptions
public class BadPracticeExample {
public static void main(String[] args) {
MySQLUtil dbUtil = new MySQLUtil();
String jsonString = "{\"name\":\"Levent\",\"age\":31,\"isRegular\":true}";
try {
dbUtil.persistCustomer(jsonString);
} catch (Exception e) {
System.out.println("A problem occured");
}
}
}
As you can see, it catches the Exception but what are we going to do if we need special exception handling for two different sources of problems? persistCustomer can throw either IOException or an SQLException and what if we need to do different set of tasks to handle those problems? I want to send an email to the database admin when an SQLException occurs and I want to continue when a JSON parsing problem occurs, on the case that an IOException is catched?
In this scenario you can't do that. Here is the output of the code snippet above and we are only sure that an Exception occured but we don't have any idea about what is the source of it;
A problem occured
D - Good Practice Example I: SQL Exception catched
public class GoodPracticeExample {
public static void main(String[] args) {
MySQLUtil dbUtil = new MySQLUtil();
String jsonString = "{\"name\":\"Levent\",\"age\":31,\"isRegular\":true}";
try {
dbUtil.persistCustomer(jsonString);
} catch (SQLException e) {
System.out.println("SQL Exception catched, SQL State : " + e.getSQLState());
System.out.println("Error Code : " + e.getErrorCode());
System.out.println("Error Message : " + e.getMessage());
} catch (IOException e) {
System.out.println("Cannot parse JSON : " + jsonString);
System.out.println("Error Message : " + e.getMessage());
}
}
}
As you can see, we catch for both, JSON and SQL problem and in this example, submethod is trying to persist DB where there is no table. The output is as below;
SQL Exception catched, SQL State : 42000
Error Code : 1142
Error Message : INSERT command denied to user 'levent'#'example.com' for table 'CUSTOMER'
So we have catched SQL Exception and we have all parameters we need to send an alarm email. We can add additional handler or utility methods on the SQLException catch block.
D - Good Practice Example II: IOExceptoin catched on Parsing Error
public class GoodPracticeExample {
public static void main(String[] args) {
MySQLUtil dbUtil = new MySQLUtil();
String jsonString = "{\"Zname\":\"Levent\",\"age\":31,\"isRegular\":true}";
try {
dbUtil.persistCustomer(jsonString);
} catch (SQLException e) {
System.out.println("SQL Exception catched, SQL State : " + e.getSQLState());
System.out.println("Error Code : " + e.getErrorCode());
System.out.println("Error Message : " + e.getMessage());
} catch (IOException e) {
System.out.println("Cannot parse JSON : " + jsonString);
System.out.println("Error Message : " + e.getMessage());
}
}
}
If you've noticed, I"ve corrupted the JSON to cause an IOException. Now in the json string, instead of "name", "Zname" is written which will cause Jackson Parser to fail. Let's checkout the output of this code.
Cannot parse JSON : {"Zname":"Levent","age":31,"isRegular":true}
Error Message : Unrecognized field "Zname" (class com.divilioglu.db.utils$Customer), not marked as ignorable (3 known properties: "isRegular", "name", "age"])
at [Source: (String)"{"Zname":"Levent","age":31,"isRegular":true}"; line: 1, column: 11] (through reference chain: com.divilioglu.db.utils.MySQLUtil$Customer["Zname"])
As you can see, we catched the specific scenario and we are sure, this comes from the line in dbUtil.persistCustomer() method which can be seen below;
Customer customer = objectMapper.readValue(jsonString, Customer.class);
E - Conclusion
So as it is a best practice to create new Exceptions by extending existing Exception classes. While writing your code at first, you may think that it is an overkill and you won't need additional Exception classes, but you will need them when you need distinguish the source of the problem and handle them independently.
In this example demonstrated above, I can independently catch IOException and SQLException and the sources of both Exceptions are coming from the same method. I want to distinguish both so that I can handle them independently. You cannot have that flexibility if you just wrap all the Exceptions with the base Exception class.
Exception is a standard class from which every exceptions inherit.
SQLException is a class that inherits from Exception and that is designed specifically for database(SQL) exceptions.
By doing
try {
// Your code here
} catch (Exception e) {
// Catching here
}
You are catching every type of exception possible... But then, you might not be able to know how to react to a specific exception.
but by doing
try {
// Your code here
} catch (SQLException e) {
// Catching here
}
You know that the exception happened while working on a database and it helps you know how to react to the exception.
As you see SQLException extends exception. So that's the only difference really. When you are catching exception then you will catch ALL exceptions (which is bad). But when you are catching SQLException then you catch only that(which is good because that is what you are seeking).
If an exception in between the try and catch blocks is thrown that is not a SQL Exception (these will typically only come from database-related code), for example a Null Pointer Exception, the Exception catch will catch it but the SQLException will not.
SQLException is an Exception so you are just getting a more specific exception.
According to Oracle's javadocs, this specific information you get is:
a string describing the error. This is used as the Java Exception
message, available via the method getMessage.
a "SQLstate" string, which follows either the XOPEN SQLstate
conventions or the SQL:2003 conventions. The values of the SQLState
string are described in the appropriate spec. The DatabaseMetaData
method getSQLStateType can be used to discover whether the driver
returns the XOPEN type or the SQL:2003 type.
an integer error code that is specific to each vendor. Normally this
will be the actual error code returned by the underlying database.
a chain to a next Exception. This can be used to provide additional
error information.
the causal relationship, if any for this SQLException.
SQLException is a specialized exception derived from Exception.
If you catch Exception, all exception shall get caught. Even undesirable exceptions.
If you catch only its specialiazation, the SQLException, only the SQLException itself or its derived shall get caught.
One shall catch only exceptions one can handle or wishes to handle, and let the others bubble up.
For further reference, please take a look at the following:
Exception
SQLException
SQL exception is a frequent error while working in Java Database Connectivity (JDBC).Its related to accessing or setting column in your SQL Query using prepared statement.
SQLException is a derived from Exception and contains more specific information related to accessing or setting column in your SQL query, while exception is usually more general.

How can you tell if a database is down or unavailable?

I am using a Spring DefaultMessageListenerContainer to consume messages from a queue. The messages are then saved to an Oracle database.
When the database goes down, I throw an exception out of the onMessage method and that leaves the message on the queue to be reprocessed. Below you can see that on a DataAccessResourceFailureException and CannotCreateTransactionException exception, I throw the exception out of the method, which puts it back on the queue. The other exceptions do not save the message; they correspond to data problems and such.
public void onMessage(javax.jms.Message mqMessage) {
...get the message blah, blah, blah
try {
this.theService.doMessage(tmaticMessage, theHandler);
} catch (DataAccessResourceFailureException e) {
this.slowDown(mqMessage);
throw e;
} catch (CannotCreateTransactionException e) {
this.slowDown(mqMessage);
throw e;
} catch (DataAccessException e) {
...
} catch (TmUnusableMessageException e) {
...
} catch (Exception e) {
...
}
}
Reading the Spring docs, I discovered that DataAccessResourceFailureException should be thrown "... when a resource fails completely: for example, if we can't connect to a database using JDBC." The problem is that I just did a test where I had the DBA take the database down and got a new exception: CannotCreateTransactionException. So that is one more exception that can be thrown. I am wondering if there are others.
I am using Spring Connections and getHibernateTemplate() to make my calls. Here is the question. How do I know what exceptions can be thrown when a database goes down?
Maybe the complexity is caused by various way you can 'take a database down'. For example:
deleting a table
deleting entire database
disabling a db user account
shutting down the database server
All can be considered as 'taking a database down', but each could cause a different exception being thrown
If you browse through following sections in spring javadoc, there are lists of exceptions that could be thrown:
http://static.springsource.org/spring/docs/2.0.x/api/org/springframework/dao/package-frame.html
http://static.springsource.org/spring/docs/2.0.x/api/org/springframework/transaction/package-frame.html

Storm Crashing after 23 hours

Hello all I have a basic Storm application set up where it receives a stream of tweets and stores them in a MySQL database. The application works great for the first ~23 hours or so then it starts giving the following error:
SQL Exception
SQL State: 08003
After it does this a few times it dies. I'm using the standard JBDC connector to connect to the database from Java. The code for the functions for storing and setting up the DB connection are as follows:
private String _db="";
private Connection conn = null;
private PreparedStatement pst = null;
public ArchiveBolt(String db){
_db = db;
}
private void setupConnection() {
//Connect to the database
try {
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql://localhost:8889/twitter_recording", "root", "root");
} catch (Exception e){
e.printStackTrace();
}
}
public void execute(Tuple tuple, BasicOutputCollector collector) {
Status s = (Status) tuple.getValue(0);
//setup the connection on the first run through or if the connection got closed down
try {
setupConnection();
} catch (Exception e) {
// TODO: handle exception
System.out.println(e.toString());
}
try {
pst = conn.prepareStatement("INSERT INTO " + _db + " (tweet)" +
"VALUES (?);");
pst.setString(1, s.toString());
//execute the SQL
pst.executeUpdate();
} catch (SQLException ex) {
// handle any errors
System.out.println("SQLException: " + ex.getMessage());
System.out.println("SQLState: " + ex.getSQLState());
System.out.println("VendorError: " + ex.getErrorCode());
if(ex.getSQLState().equals("08003")){
setupConnection();
}
} finally {
try {
conn.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
After it became apparent that it was crashing because of a 08003 error I decided that if it threw that error it should retry the set up of the connection, however that didn't help either. Could anyone point me in the right direction for solving this issue?
After it became apparent that it was crashing because of a 08003 error I decided that if it threw that error it should retry the set up of the connection, however that didn't help either. Could anyone point me in the right direction for solving this issue?
There are basically two problems here that need to be solved:
Why are the connections getting lost in the first place?
Why isn't your attempt to reconnect succeeding?
For the first problem, you should take a look at the MySQL logs to see if there are any indications there. And also, check for SQL exceptions immediately prior to the (repeated) "state 080003" exceptions. The latter are simply telling you that the connection has died previously.
My guess is that the problem is one of the following:
The MySQL server has timed out the connection due to inactivity. You can change the connection timeout in the MySQL configs if this is the problem.
Your application may be slowly leaking JDBC connections.
For the second problem, the general approach is correct, but your code doesn't match the description. In fact, it looks like it is always trying to establish a new database connection each time your execute method is called. This renders the reconnection call in your exception handler pointless. (OTOH, the code show signs that someone has been "beating on it" to try to get it to work ... and that could well be part of the problem.)
I would check that setupConnection is being called when it needs to be, and look for any exception that might be thrown. In addition, you should make sure that you explicitly close() the dead connection object ... and rethink / recode your connection management so that it doesn't leak.
For the record, there is a connection URL parameter called "autoReconnect" that in the distant past used to "deal" with lost connections. Unfortunately, the original implementation was unsafe, so they effectively disabled it; see this Question for details: Why does autoReconnect=true not seem to work?

Categories