We do use Spring Boot Health Checks in our application. On one of the checked applications it looks like the DB cannot answer in a timely manner. We are using the DataSourceHealthIndicator, and this is answering after several seconds with an exception which is fine, but takes different timeframes to come back.
Could we set a timeout on this HealthIndicator (and most probably on others as well), so that the HealthIndicator reports back after say 2s in the latest with an error, if no connection to the DB can be established?
Or could we ask the Datasource if there are still any open connections available?
I know, that we should fix the problem with these connections and we are already working on this one, but a HealthCheck for something like this would be nice as well.
You could disable the default db health indicator in your application.properties
management.health.db.enabled=false
and implement custom HealthIndicator that has desired behavior, as described here.
In your custom HealthIndicator implemetation you could use a different JdbcTemplatethat will have desired timeout value of 2 seconds, something like this:
JdbcTemplate jdbcTemplate = new JdbcTemplate(datasource);
jdbcTemplate.setQueryTimeout(2);
jdbcTemplate.execute(...);
If execute call throws an exception, your indicator should return Health.down(), otherwise Health.up() should be returned.
Related
I want to find the actual java class that serves the Spring Actuator endpoint (/actuator).
It's similar to this question in a way, but that person wanted to call it via a network HTTP call. Ideally, I can call it within the JVM to save on the cost of setting up an HTTP connection.
The reason for this is because we have 2 metrics frameworks in our system. We have a legacy metrics framework built on OpenCensus and we migrated to Spring Actuator (Prometheus metrics based on Micrometer). I think the Spring one is better but I didn't realize how much my company built infrastructure around the old one. For example, we leverage internal libraries that use OpenCensus. Infra team is depending on Opencensus-based metrics from our app. So the idea is to try to merge and report both sets of metrics.
I want to create my own metrics endpoint that pulls in data from Opencensus's endpoint and Actuator's endpoint. I could make an HTTP call to each, but I'd rather call them within the JVM to save on resources and reduce latency.
Or perhaps I'm thinking about it wrong. Should I simply be using MeterRegistry.forEachMeter() in my endpoint?
In any case, I thought if I found the Spring Actuator endpoint, I can see an example of how they're doing it and mimic the implementation even if I don't call it directly.
Bonus: I'll need to track down the Opencensus handler that serves its endpoint too and will probably make another post for that, but if you know the answer to that as well, please share!
I figured it out and posting this for anyone else interested.
The key finding: The MeterRegistry that is #Autowired is actually a PrometheusMeterRegistry if you enable the prometheus metrics.
Once you cast it into a PrometheusMeterRegistry, you can call its .scrape() method to return the exact same metrics printout you would when you hit the http endpoint.
I also need to get the same info from OpenCensus and I found a way to do that too.
Here's the snippet of code for getting metrics from both frameworks
Enumeration<MetricFamilySamples> openCensusSamples = CollectorRegistry.defaultRegistry.filteredMetricFamilySamples(ImmutableSet.of());
StringWriter writer = new StringWriter();
TextFormat.write004(writer, openCensusSamples);
String openCensusMetrics = writer.toString();
PrometheusMeterRegistry registry = (PrometheusMeterRegistry) meterRegistry;
String micrometerMetrics = registry.scrape();
return openCensusMetrics.concat(micrometerMetrics);
I found out another interesting way of doing this.
The other answer I gave but it has one issue. It contains duplicate results. When I looked into it, I realized that both OpenCensus and Micrometer were reporting the same result.
Turns out that the PrometheusScrapeEndpoint implementation uses the same CollectorRegistry that OpenCensus does so the both sets of metrics were being added to the same registry.
You just need to make sure to provide these beans
#PostConstruct
public void openCensusStats() {
PrometheusStatsCollector.createAndRegister();
}
#Bean
public CollectorRegistry collectorRegistry() {
return CollectorRegistry.defaultRegistry;
}
I was wondering if there is a specific way to override database connection timeout in the properties file in my Java web project? I am using Hibernate, Spring, and MySQL DB. I have tried several different property fields and reduced the timeout time to 1 millsecond, yet the connection is still completed with transactions still being processed properly.
These are the property fields I have used to no avail...
spring.jpa.properties.javax.persistence.query.timeout=1
spring.jdbc.template.query-timeout=1
hibernate.c3p0.timeout=1
Is hibernate overriding this timeout value or am I just setting it improperly?
Thanks in advance!
Assuming that you're using Spring Boot you can try:
spring.transaction.defaultTimeout=1
This property sets defaultTimeout for transactions to 1 second.
(Looking at the source code of TransactionDefinition it seems that it is not possible to use anything more precise than seconds.)
See also: TransactionProperties
javax.persistence.query.timeout
This is a hint for Query. It is supposed to work if you use it like this:
entityManager.createQuery("select e from SampleEntity e")
.setHint(QueryHints.SPEC_HINT_TIMEOUT, 1)
.getResultList();
See also QueryHints
spring.jdbc.template.query-timeout
Remember that according to the JdbcTemplate#setQueryTimeout javadoc:
Any timeout specified here will be overridden by the remaining transaction timeout when executing within a transaction that has a timeout specified at the transaction level.
hibernate.c3p0.timeout
I suspect that this property specifies timeout for getting from the connection pool, not for a query execution
So I currently have a spring batch process that has a composite skip policy implemented for a few custom exception types. So the issue that I am now running into is the fact that I don't always just want to skip when I get an exception.
For some database related exceptions I would like to retry a few times and then if it still fails move on and skip the record. Unfortunately I don't see a way to do this.
I tried implementing my own RetryPolicy but the only option for canRetry is true or false (rather than false I would like to throw my skippable exception).
So am I missing something here or is this not really functionality that spring batch has?
Thanks
From a StepBuilderFactory, you can do that:
stepBuilder.reader(reader).writer(writer).faultTolerant().retryPolicy(retryPolicy).skipPolicy(skipPolicy).
And yes it is working. I had the same issue and after test, I see that my items are retried following my RetryPolicy and then they are skipped following my SkipPolicy.
I want to retry MySql queries in cases of deadlock which will be removed if I can simply retry.
I use apache tomcat jdbc pool library for mysql connection pooling and it has concept of JdbcInterceptor objects which can be attached to existing Connection object. I wonder how to use these interceptors to do a query retry. How can I know if query has thrown an exception and I should retry?
I am thinking of doing something like this but I am confused in getting the right retry logic:
public class JdbQueryRetryInterceptor extends JdbcInterceptor {
private final static Logger logger = LoggerFactory
.getLogger(JdbQueryRetryInterceptor.class);
#Override
public void reset(ConnectionPool parent, PooledConnection con) {
}
}
And I have added this interceptor in the pool properties:
p.setJdbcInterceptors("com.test.mysql.JdbQueryRetryInterceptor");
I am not sure if it can be done at interceptor level however since tomcat docs say that it can be done so I want to go with this approach.
http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
Ability to configure custom interceptors. This allows you to write custom interceptors to enhance the functionality. You can use
interceptors to gather query stats, cache session states, reconnect
the connection upon failures, retry queries, cache query results, and
so on. Your options are endless and the interceptors are dynamic, not
tied to a JDK version of a java.sql/javax.sql interface.
I'm currently working on a java application. It's a standalone client with Spring and Hibernate. Also C3P0.
In the previous version we used a standard user(hardcoded in the configuration file) for the database connection but now we changed it so that every user has to provide his own credentials.
The beans with the code for the database are basically created on-demand.
I changed the XML-files and added a postprocessor which sets the credentials as well as some connection settings. It looks similar to this now:
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext();
configurer = new PropertyPlaceholderConfigurer();
// properties are retrieved from a private method
configurer.setProperties(getProperties());
context.addBeanFactoryPostProcessor(configurer);
context.setConfigLocations(new String[] { "first.xml","second.xml" });
context.refresh();
return context.getBean("myClass", MyClass.class);
This all works as expected but now we reach the part where I'm currently stuck.
I want to provide a test functionality for the connection, so that the user can click a button and then is told if his credentials were accepted or not.
My first idea was to do a simple select on the database. Sifting through the logs however, I noticed that Spring tries to connect to the database during the refresh() (or rather the instantiation of the bean) anyway. I can see exceptions in the logs, for example: java.sql.SQLException: Connections could not be acquired from the underlying database!
Unfortunately, Spring doesn't seem to actually care. The exceptions are logged away but refresh() finishes and is not throwing any exceptions because of this. I had hoped that I could simply catch the exception and then I would know that the connection failed.
I could just do the select as planned, but I want to limit the connection attempts as much as possible, because the database server will block the user after several attempts. Even permanently if there are to many attempts(already had some fun with that, before I changed the settings for the connection pool).
My searches regarding this problem came up with practically nothing. Is there a way to get the exception somehow? Or does Spring provide an API of sorts that would tell me about the connection error during the instantiation/refresh?
Failing that, any ideas for an alternative approach? Preferably one that needs only a single attempt to determine if a connection is possible.
Edit: For anyone interested: I went with the suggestion from Santosh and implemented a connection test in JDBC.
Unfortunately there seems to be no easy way to make use of the database errors/exceptions encountered during the bean instantiation.
The kind of functionality you are looking for would be very tricky to accomplish using spring+hibernate.
The connection properties are set at the session-factory level and if credentials are incorrect, the session-factory is not instantiated.
Quoting #Bozo from his answer here.
What you can do is extend LocalSessionFactoryBean and override the
getObject() method, and make it return a proxy (via
java.lang.reflect.Proxy or CGLIB / javassist), in case the
sessionFactory is null. That way a SessionFactory will be injected.
The proxy should hold a reference to a bare SessionFactory, which
would initially be null. Whenever the proxy is asked to connect, if
the sessionFacotry is still null, you call the buildSessionFactory()
(of the LocalSessionFactoryBean) and delegate to it. Otherwise throw
an exception. (Then of course map your new factory bean instead of the
current)
There is also a simple and rudimentary approach wherein before creating ClassPathXmlApplicationContext, simply try to obtain a connection using raw JDBC calls. If that succeed then proceed or else give use appropriate message.
You can limit the connection attempts here as you are in full control.