Spring Boot monitor JDBC Pool activities - java

I'm facing with a frustrating problem with jdbc
org.apache.tomcat.jdbc.pool.PoolExhaustedException: [...] Timeout: Pool empty. Unable to fetch a connection in 30 seconds, none available[size:4; busy:4; idle:0; lastwait:30000]
I found several solutions here about how to fix it but this is not the point.
I'd like to see from the logs WHO is creating these connections and not releasing.
Do you know some spring boot configuration or logback setup to show the event where a jdbc connection is taken from or returned to the pool?
Thanks

If following Paul's recommendation doesn't help, my next step would be to add:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Then use the /trace and /metrics endpoints to figure out whats happening.

Related

Unable to make JDBC Connection ["jdbc:mysql://localhost:3306/practiceDB"]

I am new to hibernate. While running my first program I am facing 2 errors.
Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment] and
Unable to make JDBC Connection ["jdbc:mysql://localhost:3306/practiceDB"]
Dependencies in pom.xml
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.6.8.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/mysql/mysql-connector-java -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.29</version>
</dependency>
JAVA code
Configuration cfg=new Configuration();
cfg.configure();
SessionFactory factory=cfg.buildSessionFactory();
System.out.println(factory);
By default, Hibernate uses its internal database connection pool library. That means it keeps a database connection open to be reused later. And MySQL database server has a timeout value for each connection (default is 8 hours or 28,800 seconds). So if a connection has been idle longer than this timeout value, it will be dropped by the server.
Therefore, when the Java database application has been idle longer than MySQL server’s connection timeout value, and the end user tries to connect again, Hibernate reuses the idle connection which was already dropped by the server, hence JDBCConnectionExceptionis thrown.
Expiring and/or testing connection validity before use in your application:
This requires changing or re-structuring the existing code, which is difficult to implement. Furthermore, opening a database connection is an expensive operation so it’s not optimal to open and close database connection for every user’s request - that’s why database connection pooling comes into play.
EDIT
As #Reg mentioned in a comment below my post, the following approach is only applicable when also using the spring-boot-starter-data-jpa dependency, so if you are not bound to use hibernate directly, feel free to include the dependency and follow the approach down below.
Also, a guide that is worth reading that also helped me a lot Introduction to Spring Data JPA
When using Spring you don't always need to write a configuration class for your database connection (atleast if you only have one single datasource).
You can simply configure your datasource via your application.properties or application.yaml file.
Try adding this to your application.properties file:
spring.datasource.url=jdbc:mysql://localhost:3306/practiceDB
spring.datasource.username=yourUsername
spring.datasource.password=yourPassword
Or adding this to your application.yaml file:
spring:
datasource:
url: jdbc:mysql://localhost:3306/practiceDB
username: yourUsername
password: yourPassword
This should allow you to establish a connection to your datasource.

Spring Boot Application using Oracle - ORA-01000: maximum open cursors exceeded - error happening during load testing

Am working on spring-boot application. While performing load testing we get error
"ORA-01000:maximum open cursors exceeded"
We have following entries for spring boot and jdbc in the pom.xml file
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.6.RELEASE</version>
</parent>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.2.0.3</version>
</dependency>
To fix the issue I tried changing the "ojdbc" from 6 to 8.
<dependency>
<groupId>com.oracle.jdbc</groupId>
<artifactId>ojdbc8</artifactId>
<version>12.2.0.1</version>
</dependency>
but I still got the same issue.
All our backend calls using either "jdbcTemplate" or "namedParameterJdbcTemplate".
The error is happening during load testing.
Our Oracle DB cursor limit is 1000.
Load test run game the error after 1hr 55mins and failure rate our our service was 0.5%
the number of users/threads is set to 25.
Looking for some suggestions to fix this issue.
My first suggestion would be to upgrade the update Spring Library including Spring JDBC and JPA version to latest. The earlier spring JPA version was not closing the cursors properly.
This problem occurs commonly because of not cleaning up resultsets, statements or connections. Each ResultSet that you create, uses a cursor on the backend. If you never close the ResultSet, the Statement that created it or the Connection that was used for the statement, those cursors never get closed. As you are using a connection pool, the connections are never physically closed, thus the cursors are never closed either.
What causes the Solve ora-01000 maximum open cursors exceeded java.sql.SQLException problem in JDBC java-
Not closing the JDBC Statement object can cause maximum open cursors
exceeded java.sql.SQLException,
Not closing the JDBC PreparedStatement object can cause maximum open
cursors exceeded java.sql.SQLException,
Not closing the JDBC CallableStatement object can cause maximum open
cursors exceeded java.sql.SQLException,
Not closing the JDBC ResultSet object and Not closing the JDBC
Connections object can cause maximum open cursors exceeded
java.sql.SQLException
Does Spring JdbcTemplate Close Connections? … Not Always.
Decent developers usually know that they have to try/catch/finally to ensure they clean up connections, file handles, or any number of things. But then, for Java, you hear “just use JdbcTemplate! it does all this boilerplate for you!”. I had, for the longest time, assumed that JdbcTemplate would clean up connections in addition to results sets. In fact, you’ll see this online a lot. But be careful! This does not appear to be the case, or if it is, it is at least data source dependent… and that actually makes sense if you think about their purpose.
When you don't have a Spring managed transaction then yes the JdbcTemplate will call the close() method on the Connection. However if there was already a connection available due to Springs transaction management closing the connection will be handled by Springs transaction support, which in turn also will call close() on the Connection.
EDIT:
Workaround:
Increase the maximum open cursor in the database by executing the following SQL command on the database:
ALTER SYSTEM SET open_cursors = 1000 SCOPE=BOTH;
This example sets the maximum open cursors to 1000. Change this value as required.
Resolution:
Update the Oracle JDBC Driver to the latest version (12.2.0.1)
There is a new version of the Oracle JDBC driver which fixes the cursor leak.
Okay, I finally got an answer. My project was EXACTLY same as yours(Spring boot 2.1.6.RELEASE and ojdbc8 12.2.0.1), (additionally Spring data jpa,) and upgrading version to 2.2.5.RELEASE didn't work for me.
I had the same exception occurs when every time I call the same function using certain amount of data(or above) - I tried to get rid of this error by upgrading ojdbc version to 19.6.0.0, adding spring config spring.jdbc.getParameterType.ignore, and so on. None of them works for me.
Finally what I found was the return type of jpa repository method. Originally my method was like this :
Stream<MyEntity> findByMyColumn(MyColumnType myColumnValue);
note that the return type of this repository method is java.util.Stream. The reason why I get this return type as stream is that this method is used only when the streaming is required.
And everytime I execute same procedure, the stack trace contains this method and I feel weird.
So I changed the type to List, and the error is GONE!!!
List<MyEntity> findByMyColumn(MyColumnType myColumnValue);
I think that the Stream is holding cursors even if the stream ended. Obviously it's a bug on Spring boot or ojdbc, but none of them seems to recognize or have willing to fix this problem.
Hope that helps to your situation, too.
ok, this took long for me to come back and post.
So for me this issue got resolbved just by having following entry in pom.xml
<dependency>
<groupId>com.oracle.jdbc</groupId>
<artifactId>ojdbc8</artifactId>
<version>12.2.0.1</version>
</dependency>
and for sometime when it didnt work we realized there was another ojdbc7 which was getting picked up by the environment and it was getting introduced due to some other dependency. So once we suppressed that the ojdbc8 got picked up and our issue got resolved.
Upgrading to the latest oracle jdbc driver solved the problem. We upgraded to : version (12.2.0.1)

Unknown Error with thread blocked on Hazelcast

I'm using the version 3.8.1 of Vertx with Java 11.05 and I configured the verticles to use Hazelcast.
Everything works fine and the application runs without any issues. However, I begin to receive the error a thread has been blocked on Hazelcast.
I'm checking the error logs for the verticles and there are not blocked threads in the code or any other issues.
The issue only happens when the application is under load test. Also, the issue not seems to affect the services.
The services are responding without any problems.
Does anyone have the same issue before?
Thanks to tsegismont.
I find a solution to this issue. The original conversation is on google forums. But the answer is to upgrade the version of Hazelcast.
The version that solves the issue is 3.11.
I used this POM call for fix the Hazelcast library.
<!-- specify Hazelcast version default 3.10.5 -->
<!-- https://mvnrepository.com/artifact/com.hazelcast/hazelcast -->
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>3.11.7</version>
</dependency>
If you keep having issues, the problem could be in Hazelcast as something referred to brain split syndrome on the cluster.
https://community.pega.com/knowledgebase/articles/system-administration/split-brain-syndrome-and-cluster-fracturing-faqs
Also, Hazelcast has some specific configuration for the cluster that can help to solve the issue.
Kind regards,
Juan

How to exchange signals between applications?

I have two unconnected applications. One is the main app that performs the business logic and CRUD on database.
A 2nd app periodically rebuilds a database cache (long running taks). I want to send a signal to the main app when the rebuild starts, and when it's finished, as the main app should take specific actions while rebuilding takes place.
How could I achive this best using spring-boot?
using spring-boot you can use jms simply by adding active-mq dependencies.
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-broker</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-pool</artifactId>
</dependency>
in yml config you would start the amq jms broker in one application by not specifying broker-url at all because spring.activemq.in-memory property defaults to true (http://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html)
or configuring it like this:
activemq:
broker-url: failover:(vm://localhost:61616?connectionTimeout=3000)
and connect to it from the other application like this
activemq:
broker-url: failover:(tcp://machineoftheotherapplication:61616?connectionTimeout=3000)
You might need to consider if you need your messages to use persistent reliable delivery, meaning if you send a message and the other application is not running it would get the message after it start up again.
If your application works with http requests you could just add a special controller which would process the requests from the second app.
Another option would be a JMX request.
However, security should be considered.

H2 Console Cant see tables created by JAVA

I have downloaded the H2 console from http://www.h2database.com/html/download.html
and I have configured the URL in my jdbc.properties file
to jdbc:h2:c:/data/Messaging.
I am using the same URL in the file to connect to the database but I cannot see the tables;
I can only see the Information Schema and when I try to select * from tables in it I cannot see the tables neither.
Does anybody have any idea what could be wrong?
One tricky thing is that the H2 console will not give you an error if you try to connect to a JDBC URL that doesn't exist. It will instead create a new database at that URL! To connect to the in memory DB, use this JDBC URL (http://localhost:8080/h2-console is the default console):
jdbc:h2:mem:testdb
If you were to enter something like jdbc:h2:~/test then a test.mv file would be created under your home directory. But your application would still be using the in memory database.
The console is available if you have the h2 dependency in your pom, and also the spring developer tools dependency. If you don't have the tools dependency, then you can also see it by having the h2 dependency and adding the following to your application.properties file:
spring.h2.console.enabled=true #not needed if you have spring-boot-devtools dependency
If you want the db as a file, and not in memory, add the following to applications.properties:
spring.datasource.url=jdbc:h2:~/test_db #You will see the file in your home directory.
H2 isn't meant for persisted data, but if you want to persist for testing purposes, then add:
spring.jpa.hibernate.ddl-auto = update
Then start up the app, and at the console, use this JDBC URL:
jdbc:h2:~/test_db
In case you were wondering, I only have 1 entry in application.properties (for the database file) and here are my dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
</dependency>
This is how you enable memory enable database using h2 module.
You need to ensure the following things
You had class that has #Entity annotations.
you need to enable the following in application.properties file spring.h2.console.enabled=true
Run Spring Boot and enter the following URL localhost:8080/h2-console
This will show a connection screen. Enter the following changes in the JDBC URL: -> jdbc:h2:mem:testdb
Hit the connection button.
Based on your question, it doesn't look like you fell victim to this particular pitfall, but this thread ended up helping me nail down the issue, so I am recording the solution here for posterity since it may help others with the same problem.
I also found that when I tried to open my database with the H2 console that I got what appeared to be a blank H2 database (basically, just an INFORMATION_SCHEMA table). While double-checking that I got the name of the DB correct (mydb.mv.db), I discovered that the H2 console had created a second database file, mydb.mv.db.mv.db. Odd.
It turns out that the H2 Console expects you to omit the suffix .mv.db from the filename. Since I hadn't, it was looking for mydb.mv.db.mv.db. Changing the JDBC string to jdbc:h2:mydb solved the problem and I could then open the file from the H2 Console.
Had the same Problem.
This solved it for me:
Why is my embedded h2 program writing to a .mv.db file
Just added ;MV_STORE=FALSE and ;MVCC=FALSE to the jdbc url and everything worked just fine.
I have used the below and I see my table get created.
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_ON_EXIT=FALSE
spring.h2.console.enabled=true
spring.h2.console.path=/h2console
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.driverClassName=org.h2.Driver
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.H2Dialect
You can also avoid this problem by using the same version between H2 console and your Java code.
This is how I solved this same issue here.
Add Annotation #EntityScan("packageName") in main class
I finally figured out how to view my default in-memory H2 database tables. I'm sharing this solution because I think the process must have changed recently, due to security concerns. It is no longer possible to use the default jdbc:h2:mem:testdb to gain access to the db, nor to set a name for entry via the application.properties line spring.datasource.name=yourdbnamehere.
As of this writing, when the application starts, there is a console message similar to the following:
2021-04-26 21:16:44.424 INFO 23142 --- [ restartedMain] o.s.b.a.h2.H2ConsoleAutoConfiguration : H2 console available at '/h2-console'. Database available at 'jdbc:h2:mem:6da7bb82-4008-4396-b825-09e44c17388b'
I'm using Spring Tool Suite 4. Other IDEs may format this message differently.
The key part is the last section of this string, where the database address is specified. The identifier for the database now changes with each boot. (Maybe there is a way to suppress this, but I don't know it.)
I copied the address jdbc:h2:mem:whatever-long-random-name, and pasted it into the h2-console Login Form field labeled "JDBC URL". Once in, hitting or clicking Connect will bring up the database schema and console.
The tutorial I'm working through is Dan Vega's "Springit" application, from his Getting Started With Spring Boot 2.0 course. I'm including this info as a search breadcrumb for anyone following. This course may be a couple years old now, but Vega does a much better-than-average job of explaining, so it remains worthwhile. However, tech marches on, and I assume the need for enhanced security has grown to the point where it's no longer safe to allow an easily guessable database name.

Categories