We Keep Coding

Spring + Hibernate+ HikariCP: how to handle DB connection while doing long running REST call?

I have a project running on Spring Boot 1.3.8, Hikari CP 2.6.1 and Hibernate (Spring ORM 4.2.8). The code on service layer looks like this:
public void doStuff() {
A a = dao.findByWhatever();
if (a.hasProperty()) {
B b = restService.doRemoteRequestWithRetries(); // May take long time
}
a.setProp(b.getSomethig());
dao.save(b);
}
Hikari configuration has this: spring.datasource.leakDetectionThreshold=2000.
The problem is that external REST service is quite slow and often takes 2+ seconds to respond, as a result we see a lot of java.lang.Exception: Apparent connection leak detected which are nothing else but false negatives, though the problem can be clearly seen: we hold DB connection for the time we executing rest request.
The question would be: how to properly decouple DB and REST stuff? Or how to tell hibernate to release connection in between? So that we return DB connection to pool while waiting for REST response.
I have tried setting hibernate.connection.release_mode=AFTER_TRANSACTION and it kind of helps, at least we do not have connection leak exceptions. The only problem is that our tests started showing this:
2018-04-17 15:48:03.438 WARN 94029 --- [ main] o.s.orm.jpa.vendor.HibernateJpaDialect : JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect's prepareConnection flag to false`
The tests are using injected DAO to insert records in DB and later check them via application API. They are not annotated with #Transactional and the list of listeners looks like this:
#TestExecutionListeners({
DependencyInjectionTestExecutionListener.class,
TransactionalTestExecutionListener.class,
TransactionDbUnitTestExecutionListener.class
})
Any ideas what could be the problem with tests?

In the code
public void doStuff() {
A a = dao.findByWhatever();
if (a.hasProperty()) {
B b = restService.doRemoteRequestWithRetries(); // May take long time
}
a.setProp(b.getSomethig());
dao.save(b);
}
I see three tasks here - fetching entity A, connecting to remote service and updating entity A. And all these are in same transaction, so the underlying connection will be held till the method is complete.
So the idea is to split the tasks one and three into separate transactions, there by allowing the connection to be releases before making the call to remote service.
Basically, with spring boot you need to add spring.jpa.open-in-view=false. This will not register OpenEntityManagerInViewInterceptor and thus entityManager (in-turn connection) is not bound to the current thread/request.
Subsequently, split the three tasks into separate methods with #Transactional. This helps us bind the entityManager to the transaction scope and releasing connection at end of transaction method.
NOTE: And do ensure that there isn't any transaction started/in progress before (i.e., caller - like Controller etc) calling these methods. Else the purpose is defeated and these new #Transactional methods will run in the same transaction as before.
So the high-level approach could look like below:
In spring boot application.properties add property spring.jpa.open-in-view=false.
Next you need to split doStuff method into three methods in new service class. Intent is to ensure they use different transactions.
First method with #Transactionalwill call A a = dao.findByWhatever();`.
Second method makes remote call.
Third method with #Transactionalwill call rest of the code with JPA merge or hibernate saveOrUpdate on objecta`.
Now Autowired this new service in your current code and call the 3 methods.

Spring HealthIndicator - Timeout

We do use Spring Boot Health Checks in our application. On one of the checked applications it looks like the DB cannot answer in a timely manner. We are using the DataSourceHealthIndicator, and this is answering after several seconds with an exception which is fine, but takes different timeframes to come back.
Could we set a timeout on this HealthIndicator (and most probably on others as well), so that the HealthIndicator reports back after say 2s in the latest with an error, if no connection to the DB can be established?
Or could we ask the Datasource if there are still any open connections available?
I know, that we should fix the problem with these connections and we are already working on this one, but a HealthCheck for something like this would be nice as well.

You could disable the default db health indicator in your application.properties
management.health.db.enabled=false
and implement custom HealthIndicator that has desired behavior, as described here.
In your custom HealthIndicator implemetation you could use a different JdbcTemplatethat will have desired timeout value of 2 seconds, something like this:
JdbcTemplate jdbcTemplate = new JdbcTemplate(datasource);
jdbcTemplate.setQueryTimeout(2);
jdbcTemplate.execute(...);
If execute call throws an exception, your indicator should return Health.down(), otherwise Health.up() should be returned.

How can I validate my jdbc connection whether the data base is up running without executing sql query

I'm working with spring jdbc, In my case I have to execute a query and refresh cache only when data base is up running for this I'm not finding any method appropriate to call on Connection object.

If you're using Spring JdbcTemplate you could perhaps use:
public <T> T execute(ConnectionCallback<T> action) throws DataAccessException
Execute a JDBC data access operation, implemented as callback action
working on a JDBC Connection. This allows for implementing arbitrary
data access operations, within Spring's managed JDBC environment: that
is, participating in Spring-managed transactions and converting JDBC
SQLExceptions into Spring's DataAccessException hierarchy.
The callback action can return a result object, for example a domain object or a collection of domain objects.
jdbcTemplate.
You could execute an action once the connection is established such a simple query perhaps.
Let me know if that helps.

Testing the database connection with spring and hibernate

I'm currently working on a java application. It's a standalone client with Spring and Hibernate. Also C3P0.
In the previous version we used a standard user(hardcoded in the configuration file) for the database connection but now we changed it so that every user has to provide his own credentials.
The beans with the code for the database are basically created on-demand.
I changed the XML-files and added a postprocessor which sets the credentials as well as some connection settings. It looks similar to this now:
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext();
configurer = new PropertyPlaceholderConfigurer();
// properties are retrieved from a private method
configurer.setProperties(getProperties());
context.addBeanFactoryPostProcessor(configurer);
context.setConfigLocations(new String[] { "first.xml","second.xml" });
context.refresh();
return context.getBean("myClass", MyClass.class);
This all works as expected but now we reach the part where I'm currently stuck.
I want to provide a test functionality for the connection, so that the user can click a button and then is told if his credentials were accepted or not.
My first idea was to do a simple select on the database. Sifting through the logs however, I noticed that Spring tries to connect to the database during the refresh() (or rather the instantiation of the bean) anyway. I can see exceptions in the logs, for example: java.sql.SQLException: Connections could not be acquired from the underlying database!
Unfortunately, Spring doesn't seem to actually care. The exceptions are logged away but refresh() finishes and is not throwing any exceptions because of this. I had hoped that I could simply catch the exception and then I would know that the connection failed.
I could just do the select as planned, but I want to limit the connection attempts as much as possible, because the database server will block the user after several attempts. Even permanently if there are to many attempts(already had some fun with that, before I changed the settings for the connection pool).
My searches regarding this problem came up with practically nothing. Is there a way to get the exception somehow? Or does Spring provide an API of sorts that would tell me about the connection error during the instantiation/refresh?
Failing that, any ideas for an alternative approach? Preferably one that needs only a single attempt to determine if a connection is possible.
Edit: For anyone interested: I went with the suggestion from Santosh and implemented a connection test in JDBC.
Unfortunately there seems to be no easy way to make use of the database errors/exceptions encountered during the bean instantiation.

The kind of functionality you are looking for would be very tricky to accomplish using spring+hibernate.
The connection properties are set at the session-factory level and if credentials are incorrect, the session-factory is not instantiated.
Quoting #Bozo from his answer here.
What you can do is extend LocalSessionFactoryBean and override the
getObject() method, and make it return a proxy (via
java.lang.reflect.Proxy or CGLIB / javassist), in case the
sessionFactory is null. That way a SessionFactory will be injected.
The proxy should hold a reference to a bare SessionFactory, which
would initially be null. Whenever the proxy is asked to connect, if
the sessionFacotry is still null, you call the buildSessionFactory()
(of the LocalSessionFactoryBean) and delegate to it. Otherwise throw
an exception. (Then of course map your new factory bean instead of the
current)
There is also a simple and rudimentary approach wherein before creating ClassPathXmlApplicationContext, simply try to obtain a connection using raw JDBC calls. If that succeed then proceed or else give use appropriate message.
You can limit the connection attempts here as you are in full control.

MongoDB configuration in a java web app

I'm looking for some advice on the proper way to set up mongoDB for my web application that runs with java.
From the mongoDB tutorial, i understand that I should have only one instance of the Mongo class.
The Mongo class is designed to be thread safe and shared among threads. Typically you create only 1 instance for a given DB cluster and use it across your app.
So I've got a singleton provider for this (I'm using guice for injection)
#Singleton
public class MongoProvider implements Provider<Mongo> {
private Mongo mongo;
public Mongo get() {
if (mongo == null)
mongo = new Mongo("localhost", 27017);
return mongo;
}
}
And whenever I have to work with mongo in my webapp i inject the provider and get the same instance of mongo.
public class MyService {
private Provider<Mongo> mongoProvider;
#Inject
private MyService(Provider<Mongo> mongoProvider) {
this.mongoProvider = mongoProvider;
}
public void execute() {
DB db = mongoProvider.get().getDB("mydatabase");
DBCollection coll = db.getCollection("mycollection");
// Do stuff in collection
...
}
}
What I find weird is that everytime i access my database, i get logs like this from mongo :
[initandlisten] connection accepted from 192.168.1.33:54297 #15
[initandlisten] connection accepted from 192.168.1.33:54299 #16
So far, I haven't had any problems but I'm wondering if it's good practice and if I won't run into any problems when the number of connections accepted gets too high.
Should I also have only one instance of the DB object for my entire app ?
Do I have to configure MongoDB differently to automatically close the connections after some time ? Or do I have to close connections manually ? I've read something about using the close() method on Mongo but I'm not sure when or if to call it.
Thank you for you advice.

This is good practice. Each instance of Mongo manages a connection pool, so you will see multiple connections in the mongod logs, one for each connection in the pool. The default pool size is 10, but that can be configures using the connectionsPerHost field in MongoOptions.
Mongo instances also maintain a cache of DB instances, so you don't have to worry about maintaining those as singletons yourself.
You do not have to configure Mongo to automatically close connections. You can call Mongo#close at the appropriate time to close all the sockets in the connection pool.

Founded something like this om MondoDB site:
"The Java MongoDB driver is thread safe. If you are using in a web serving environment, for example, you should create a single MongoClient instance, and you can use it in every request. The MongoClient object maintains an internal pool of connections to the database (default pool size of 10). For every request to the DB (find, insert, etc) the Java thread will obtain a connection from the pool, execute the operation, and release the connection. This means the connection (socket) used may be different each time."
And from FAQ from MongoSite which I think completely anwsers on you question.
http://docs.mongodb.org/manual/faq/developers/#why-does-mongodb-log-so-many-connection-accepted-events