I am currently working with an app which using two different DB(different instance).
DB A is totally under A's project, however DB B is under the other project.(I am managing these via gcloud app engine.)
What is my problem :
DB B always disconnected if no request more than few hours with below error message.
{"timestamp":1555464776769,"status":500,"error":"Internal Server Error","exception":"org.springframework.transaction.CannotCreateTransactionException","message":"Could not open JPA EntityManager for transaction; nested exception is javax.persistence.PersistenceException: com.mysql.cj.jdbc.exceptions.CommunicationsException: The last packet successfully received from the server was 43,738,243 milliseconds ago. The last packet sent successfully to the server was 43,738,243 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.","path":"/client/getAllCompany"}
To resolve this issue, I tried.
1) add 'autoReconnect=true' at application.properties
api.datasource.url = jdbc:mysql://google/projectB?cloudSqlInstance=projectB:australia-southeast1:projectB&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false&autoReconnect=true
2) add below config at application.properties.
spring.datasource.tomcat.test-while-idle=true
spring.datasource.tomcat.time-between-eviction-runs-millis=3600000
spring.datasource.tomcat.min-evictable-idle-time-millis=7200000
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.validation-query=SELECT 1
(My project doesn't have web.xml file)
If i re-deploy this project, i can access data from DB B as well.
How can I config to prevent killed the connectivity with DB B?
Wish to listen advice. Thank you in advanced.
HibernateConfig Code for DB B
#Bean(name = "apiDataSource")
#ConfigurationProperties(prefix = "api.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "apiEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean apiEntityManagerFactory(
EntityManagerFactoryBuilder builder, #Qualifier("apiDataSource") DataSource dataSource
) {
return builder.dataSource(dataSource).packages("com.workspez.api.entity").persistenceUnit("api").build();
}
#Bean(name = "apiTransactionManager")
public PlatformTransactionManager apiTransactionManager(
#Qualifier("apiEntityManagerFactory") EntityManagerFactory apiEntityManagerFactory
) {
return new JpaTransactionManager(apiEntityManagerFactory);
}
#Bean(name = "apiJdbc")
public NamedParameterJdbcTemplate apiJdbcTemplate() {
return new NamedParameterJdbcTemplate(dataSource());
}
Related
I have Spring Boot application with implemented RESTControllers. In the RESTController I specified /clients/ endpoint (PUT method) to update client resource.
In my database (PostgreSQL) I have implemented several triggers on the client table to record history of the changes to the client_history table. The problem is that I need to insert author (user_id) of the changes to the client_history table.
This question (https://stackoverflow.com/a/13172964) led me to use GUC inside my trigger function.
select current_setting('custom.application_user_id'))
Now I need to execute sql query set custom.application_user_id = {user_id}; to set user_id when Hibernate opens new session. I have tried to use Hibernate Interceptors to do this, but this solution does not work.
Could someone give me an example how to subscribe to the event when session is created?
P.S. Also I have silly question: Does every HTTP requests received by the application creates (obtains from connection pool) new connection?
After research, I found this trick.
For me i have two datasource in my project (ds1, ds2)
spring:
datasource:
ds1:
url: jdbc:oracle:thin:${DATABASE_USERNAME:E4A5}/${DATABASE_PASSWORD:EA5}#${DATABASE_HOSTNAME:local4host}:${DATABASE_PORT:49161}:${DATABASE_SERVICE:EE}
driver-class-name: oracle.jdbc.OracleDriver
ds2:
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
url: jdbc:sqlserver://${DATABASE_EF_HOSTNAME:localhost};databaseName=${DATABASE_EF_NAME:test}
username: ${DATABASE_EF_USERNAME:test}
password: ${DATABASE_EF_PASSWORD:test}
And to run query when connection/session is created in ds1 I did this in my class annotated with #Configuration:
#Primary
#Bean
#ConfigurationProperties("spring.datasource.ds1")
public DataSourceProperties ds1DataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.ds1.configuration")
public DataSource ds1DataSource() {
HikariDataSource hikariDataSource = ds1DataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
hikariDataSource.setConnectionInitSql("ALTER SESSION SET NLS_DATE_FORMAT='DD/MM/YYYY' " +
"NLS_SORT='Binary' " +
"NLS_NUMERIC_CHARACTERS='.,' " +
"NLS_LANGUAGE='FRENCH'");
return hikariDataSource;
}
I'm not able to open more than 10 connections with spring-webflux and r2dbc (with r2dbc-pool driver 0.8.0.M8). My config looks like:
#Configuration
public class PostgresConfig extends AbstractR2dbcConfiguration {
#Override
#Bean
public ConnectionFactory connectionFactory() {
ConnectionFactory connectionFactory = ConnectionFactories.get(ConnectionFactoryOptions.builder()
.option(DRIVER, "pool")
.option(PROTOCOL, "postgresql")
.option(HOST, host)
.option(USER, user)
.option(PASSWORD, password)
.option(DATABASE, database)
.build());
ConnectionPoolConfiguration configuration = ConnectionPoolConfiguration.builder(connectionFactory)
.maxIdleTime(Duration.ofMinutes(30))
.initialSize(initialSize)
.maxSize(maxSize)
.maxCreateConnectionTime(Duration.ofSeconds(1))
.build();
return new ConnectionPool(configuration);
}
}
When I'm specifying more than 10 connections I get errors like:
org.springframework.dao.DataAccessResourceFailureException:
Failed to obtain R2DBC Connection; nested exception is
java.util.concurrent.TimeoutException:
Did not observe any item or terminal signal within 1000ms in 'lift'
(and no fallback has been configured)
at org.springframework.data.r2dbc.connectionfactory.ConnectionFactoryUtils
.lambda$getConnection$0(ConnectionFactoryUtils.java:71)
Moreover, number of connections remain the same as initial size. New connections are not created.
Spring boot (at least 2.3.4) have a tricky "gotcha" regarding the pool size when set by properties/yaml. If you include "pool" in your database url, then the size set (initial size or max size) won't have any effect and the defaults for the r2dbc pool will be used, 10 and 10.
This is due to PooledConnectionFactoryCondition in ConnectionFactoryConfigurations.java failing when both spring.r2dbc.pool.enabled=true, which it is if the r2dbc-pool dependency is on the classpath, and "pool" being part of the spring.r2dbc.url property.
From the PooledConnectionFactoryCondition docs:
Condition that checks that a ConnectionPool is requested. The
condition matches if pooling was opt-in via configuration and the r2dbc url does not contain pooling-related options.
This in turn does lead to the ConnectionPool bean not being created.
Skip the "pool" keyword in the r2dbc url property and have the r2dbc-pool dependency, then you will get a correctly configured pool.
Ok, the MAX_SIZE should be also specified for ConnectionFactoryOptions. Otherwise connection pool size still remains 10.
import static io.r2dbc.pool.PoolingConnectionFactoryProvider.MAX_SIZE;
ConnectionFactory connectionFactory = ConnectionFactories.get(ConnectionFactoryOptions.builder()
.option(DRIVER, "pool")
.option(PROTOCOL, "postgresql")
.option(HOST, host)
.option(USER, user)
.option(PASSWORD, password)
.option(DATABASE, database)
.option(MAX_SIZE, maxSize)
.build());
Note that, you can use the release version 0.8.4.RELEASE (which is the latest) https://mvnrepository.com/artifact/io.r2dbc/r2dbc-postgresql/0.8.4.RELEASE which does not require you to instantiate a ConnectionFactory
If you are using spring-boot-starter-data-r2dbc, then the min and max is configurable in application.properties
spring.r2dbc.pool.initialSize=2
spring.r2dbc.pool.maxSize=2
See org.springframework.boot.autoconfigure.r2dbc.R2dbcProperties class
Below is my configuration for spring-boot-starter-data-r2dbc Check if that helps you:
spring:
r2dbc:
url: r2dbc:postgresql://127.0.0.1:5432/test?schema=public
username: postgres
password: postgres
pool:
name: TEST-POOL
initial-size: 1
max-size: 10
max-idle-time: 30m
This question already has answers here:
Does Spring embedded database support different SQL dialects?
(3 answers)
Closed 5 years ago.
I want to create an in-memory database populated with test data for fast testing, so I declare this bean in my configuration file, but I also want so set this properties:
MODE=MySQL
DB_CLOSE_ON_EXIT=FALSE
but I don't know where to do it
#Bean
public DataSource dataSource(){
return
(new EmbeddedDatabaseBuilder())
.setType(EmbeddedDatabaseType.H2) //.H2
.addScript("classpath:db/H2.schema.sql")
.addScript("classpath:db/H2.data.sql")
.build();
}
try this
#Bean
public DataSource dataSource(){
return
new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.setName("testDB;DB_CLOSE_ON_EXIT=FALSE;MODE=MySQL")
.addScript("classpath:db/H2.schema.sql")
.addScript("classpath:db/H2.data.sql")
.build();
}
You can try using EmbeddedDatabaseBuilder.setName()
#Bean
public DataSource dataSource() {
return
new EmbeddedDatabaseBuilder()
.setName("testdb;MODE=MySQL;DB_CLOSE_ON_EXIT=false")
.setType(EmbeddedDatabaseType.H2) //.H2
.addScript("classpath:db/H2.schema.sql")
.addScript("classpath:db/H2.data.sql")
.build();
}
Note: I have not tried this myself, but found a clue on this answer
An alternative approach is to wire DataSources using properties. This way, you can set the JDBC URL which can contain additional properties.
The mode can be done via a SQL statement on H2, but I am pretty sure the DB_CLOSE_DELAY must be set as part of the URL which there is no easy hook into. You would be best just setting this in the application.properties/yml and let spring autoconfigure it
jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;MODE=MySQL would be the spring.datasource.url additional properties are available for the schema and data
I am using db4free's mysql server for my spring-webmvc project.But the problem is , I can't get a connection to the server and the Exception is
org.springframework.web.util.NestedServletException: Request processing failed;
nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException:
Must specify port after ':' in connection string
But I have specified port correctly just after ':' , Here is my configuration java class:
#Configuration
#ComponentScan(basePackages="org.ratajo.amaderbari")
#EnableWebMvc
public class MvcConfiguration extends WebMvcConfigurerAdapter{
#Bean
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://http://www.db4free.net:3306/myDB");
dataSource.setUsername("user");
dataSource.setPassword("pass");
return dataSource;
}
Here is the sample program I am trying to execute
MvcConfiguration mv = new MvcConfiguration();
JdbcTemplate jdbcTemplate = new JdbcTemplate(mv.getDataSource());
String sql="CREATE TABLE 'contact' ('contact_id' int(11) NOT NULL AUTO_INCREMENT,) ENGINE=InnoDB AUTO_INCREMENT=25 DEFAULT CHARSET=utf8";
jdbcTemplate.execute(sql);
The url looks weird :
dataSource.setUrl("jdbc:mysql://http://www.db4free.net:3306/myDB");
should be something like
dataSource.setUrl("jdbc:mysql://www.db4free.net:3306/myDB");
otherwise it is trying to use http as hostname and //www.db4free.net as port. (which explains the error). But I would also double check the hostname as it looks weird to go to a host 'www.something'.
OTOH jdbc url's are weird.
I need to create a connection pool from a spring application running in a tomcat server.
This application has many catalogs, the main catalog (its is static) called 'db' has just one table with all existing catalog names and a boolean flag for the "active" one.
When the application starts I need to choose from the main catalogs the active one, then I have to select it as default catalog.
How can I accomplish this?
Until now I used a custom class DataSourceSelector extends DriverManagerDataSource but now I need to improve the db connection using a pool, then I thought about a tomcat dbcp pool.
I would suggest the following steps:
Extend BasicDataSourceFactory to produce customized BasicDataSources.
Those customized BasicDataSources would already know which catalog is active and have the defaultCatalog property set accordingly.
Use your extended BasicDataSourceFactory in the Tomcat configuration.
#Configuration
public class DataAccessConfiguration {
#Bean(destroyMethod = "close")
public javax.sql.DataSource dataSource() {
org.apache.tomcat.jdbc.pool.DataSource ds = new org.apache.tomcat.jdbc.pool.DataSource();
ds.setDriverClassName("com.mysql.jdbc.Driver");
ds.setUrl("jdbc:mysql://localhost/db");
ds.setUsername("javauser");
ds.setPassword("");
ds.setInitialSize(5);
ds.setMaxActive(10);
ds.setMaxIdle(5);
ds.setMinIdle(2);
ds.get
return ds;
}
}