We Keep Coding

How to refresh app instances using Spring cloud bus with data which isn't controlled by config server?

I'm trying to use Spring cloud bus with Kafka in my microservices application, and indeed I could use it, but only data which is controlled by Spring cloud config server got refreshed!
I'm using jdbc back-end with my config server, and in order to simulate my need, I'm changing some value in properties file in one of my services, beside the properties table, and call the /monintor end point again (mentioned here section 4.3 https://www.baeldung.com/spring-cloud-bus); as a result, only data coming from properties table is changed.
This is the yml file for my Config server
spring:
cloud:
config:
server:
jdbc:
sql: SELECT KEY,VALUE from PROPERTIES where APPLICATION=? and PROFILE=? and LABEL=?
order: 1
stream:
kafka:
binder:
brokers: localhost:9092
datasource:
url: jdbc:mysql://localhost:3306/sweprofile?zeroDateTimeBehavior=convertToNull
username: 123
password: 123ertbnm
hikari:
maximum-pool-size: 10
connection-timeout: 5000
profiles:
active:
- jdbc
application:
name: configServer
These are yml files for One of my Miscroservices and its propeties file respectively
spring:
datasource:
username: 123
password: 123ertbnm
url: jdbc:mysql://localhost:3306/sweprofile?zeroDateTimeBehavior=convertToNull
jpa:
properties:
hibernate:
format_sql: true
ddl-auto: none
application:
name: auth-service
cloud:
config:
discovery:
enabled: true
service-id: configServer
bus:
refresh:
enabled: true
profiles:
active: jdbc
management:
endpoints:
web:
exposure:
include: ["health","info","refresh", "bus-refresh"]
# This line is dummy data for testing purpose
ali.man = " Ola 12333"
This is snapshot from rest controller
#RestController
#RequestMapping("/user")
#RefreshScope
public class AuthController {
private UserAuthService userAuthService;
#Value("${name}")
private String name; // changed normally
// Calling the key value mentioned in properties file after changing
#Value("${ali.man}")
private String k; // -> not changed
public AuthController(UserAuthService userAuthService) {
this.userAuthService = userAuthService;
}
#GetMapping("authTest")
public String getAuth() {
return name + k;
}
}
What did I miss? Why value from Properties file is not changed? hopefully I can use Spring cloud bus with Kafka to refresh these external data.

After some hours of investigation, I found that there is some recommended way. Cloud bus can send Refresh Event and Spring boot has RefreshEvent Listener to that event; this what I build my solution on.
So when event is sent by the bus; all instances will do the same logic ( Refreshing data ) on the loaded in memory configurations.
I used this snippet to apply this
#Configuration
public class ReloadLookupEvent implements ApplicationListener<RefreshScopeRefreshedEvent> {
#Autowired
private CacheService cacheService;
#Override
public void onApplicationEvent(RefreshScopeRefreshedEvent event) {
cacheService.refreshLookUp();
}
}
I could refresh all other configurations on demand, maybe it is a workaround, but acceptable.

Import spring properties from an external file located in the server filesystem

I need to import Spring properties (in Spring Boot) as spring.datasource, server.port... from a file that is located in the file system (out of the java application).
This is for a Spring Boot application that needs to connect to a database.
spring:
datasource:
url: jdbc:oracle:thin:#X.X.X.X:XXXX:XXXX
username: XX
password: XX
driver-class-name: oracle.jdbc.driver.OracleDriver
hikari:
connection-timeout: 60000
maximum-pool-size: 5
application:
name: XX
server:
port: 9000
contextPath: /
servlet:
session:
cookie:
http-only: true
secure: true
By the moment I am not able to import properties from file using #PropertySource(value = "C:/test.properties") in class.

There are multiple ways to achieve this. My preferred one is to annotate your applications main class with #PropertySource and configure it to read your property file.
Example:
#SpringBootApplication
#PropertySource({
"file:C:\test.properties"
})
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

Spring boot 2 connection to rabbitmq via Apache Camel

I have problem with connection to rabbitmq via Apache Camel on Spring Boot 2.
I did following steps:
My dependencies:
implementation "org.apache.camel:camel-spring-boot-starter:${camelVersion}"
implementation "org.apache.camel:camel-jackson-starter:${camelVersion}"
implementation "org.apache.camel:camel-core:${camelVersion}"
implementation "org.apache.camel:camel-rabbitmq-starter:${camelVersion}"
implementation "org.springframework.boot:spring-boot-starter-amqp"
Application.yaml
spring:
rabbitmq:
dynamic: true
host: 192.168.1.1
port: 5672
username: X
password: Y
And I have following route:
#Component
public class BasicRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("direct:loggerQueue")
.id("loggerQueue")
.to("rabbitmq://TEST-QUEUE.exchange?queue=TEST-QUEUE.queue&autoDelete=false&connectionFactory=#rabbitConnectionFactory")
.end();
}
}
Finnaly I have still following issue:
2019-03-06 12:46:05.766 WARN 19464 --- [ restartedMain] o.a.c.c.rabbitmq.RabbitMQProducer : Failed to create connection. It will attempt to connect again when publishing a message.
java.net.ConnectException: Connection refused: connect
Connection seems ok, I tested it. Something is bad with rabbitConnectionFactory.
I don't know what I have bad.

The problem appears to be that RabbitMQComponent is expecting to find a connection factory of type com.rabbitmq.client.ConnectionFactory.
However, the springboot auto-configure is creating a connection factory of type org.springframework.amqp.rabbit.connection.CachingConnectionFactory.
So, whenever the RabbitMQComponent attempts to find the appropriate connection factory, because it is looking for the specific type, and because it does not subclass the rabbitmq ConnectionFactory, it returns a null value, and fails to use the appropriate host name and configuration parameters specified in your application.yml.
You should also see the following in your log if you have debug level set:
2019-12-15 17:58:53.631 DEBUG 48710 --- [ main] o.a.c.c.rabbitmq.RabbitMQComponent : Creating RabbitMQEndpoint with host null:0 and exchangeName: asterix
2019-12-15 17:58:55.927 DEBUG 48710 --- [ main] o.a.c.c.rabbitmq.RabbitMQComponent : Creating RabbitMQEndpoint with host null:0 and exchangeName: asterix-sink
EDIT:
The CachingConnectionFactory is configured with the required Rabbit connection factory as part of the autoconfiguration. However, you need to provide a link to the correct factory.
Therefore, you need to add a #Bean to disambiguate.
#Configuration
#RequiredArgsConstructor
public class CamelConfig {
private final CachingConnectionFactory rabbitConnectionFactory;
#Bean
com.rabbitmq.client.ConnectionFactory rabbitSourceConnectionFactory() {
return rabbitConnectionFactory.getRabbitConnectionFactory();
}
}
and in your endpoint configuration:
rabbitmq:asterix?connectionFactory=#rabbitSourceConnectionFactory
Note that the # is optional, as it gets stripped out within the code when it is trying to find the rabbit connection factory bean.
In your application.yml, configure the connection parameters (the url is no longer included in the endpoint URI).
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest

Jhipster - Configure multiple datasources

I have springBatch App where I was using multiple datasources which I configured as below which is pretty much straight forward. It was working fine.
How to use 2 or more databases with spring?
Now I had to integrate it to Jhipster project. I did similar kind of configuration in application.yml. I removed auto generated Datasource from application.yml and added configurations similar to above post and injected the primary Datasource in DataBaseConfiguration.java which is Jhipster generated class. With this configuration I am unable to do CRUD operation on Database entities form JHipster UI. I am not seeing any errors in logs.
I am not sure what's the right/simple way of configuring multiple datasources in Jhipster project. A sample example utilizing multiple databases would give a good start for me. I didn't find much resources on this.
Below are code sample's for changes I performed to have multiple datasources in JHipster
# ===================================================================
# Spring Boot configuration for the "dev" profile.
#
# This configuration overrides the application.yml file.
#
# More information on profiles: http://www.jhipster.tech/profiles/
# More information on configuration properties: http://www.jhipster.tech/common-application-properties/
# ===================================================================
# ===================================================================
# Standard Spring Boot properties.
# Full reference is available at:
# http://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
# ===================================================================
spring:
profiles:
active: dev
include: swagger
devtools:
restart:
enabled: true
livereload:
enabled: false # we use gulp + BrowserSync for livereload
jackson:
serialization.indent_output: true
jpa:
database-platform: org.hibernate.dialect.Oracle12cDialect
database: ORACLE
show-sql: true
properties:
hibernate.id.new_generator_mappings: true
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: true
hibernate.default_schema: ********x
mail:
host: localhost
port: 25
username:
password:
messages:
cache-seconds: 1
thymeleaf:
cache: false
batch:
job:
enabled: false
liquibase:
contexts: dev
# ===================================================================
# To enable SSL, generate a certificate using:
# keytool -genkey -alias ********x-storetype PKCS12 -keyalg RSA -keysize 2048 -keystore keystore.p12 -validity 3650
#
# You can also use Let's Encrypt:
# https://maximilian-boehm.com/hp2121/Create-a-Java-Keystore-JKS-from-Let-s-Encrypt-Certificates.htm
#
# Then, modify the server.ssl properties so your "server" configuration looks like:
#
# server:
# port: 8443
# ssl:
# key-store: keystore.p12
# key-store-password: <your-password>
# keyStoreType: PKCS12
# keyAlias: ********x
# ===================================================================
server:
port: 8080
# ===================================================================
# JHipster specific properties
#
# Full reference is available at: http://www.jhipster.tech/common-application-properties/
# ===================================================================
jhipster:
http:
version: V_1_1 # To use HTTP/2 you will need SSL support (see above the "server.ssl" configuration)
# CORS is only enabled by default with the "dev" profile, so BrowserSync can access the API
cors:
allowed-origins: "*"
allowed-methods: "*"
allowed-headers: "*"
exposed-headers: "Authorization"
allow-credentials: true
max-age: 1800
security:
authentication:
jwt:
secret: my-secret-token-to-change-in-production
# Token is valid 24 hours
token-validity-in-seconds: 86400
token-validity-in-seconds-for-remember-me: 2592000
mail: # specific JHipster mail property, for standard properties see MailProperties
from: ********x#localhost
base-url: http://127.0.0.1:8080
metrics: # DropWizard Metrics configuration, used by MetricsConfiguration
jmx.enabled: true
graphite: # Use the "graphite" Maven profile to have the Graphite dependencies
enabled: false
host: localhost
port: 2003
prefix: ********x
prometheus: # Use the "prometheus" Maven profile to have the Prometheus dependencies
enabled: false
endpoint: /prometheusMetrics
logs: # Reports Dropwizard metrics in the logs
enabled: false
report-frequency: 60 # in seconds
logging:
logstash: # Forward logs to logstash over a socket, used by LoggingConfiguration
enabled: false
host: localhost
port: 5000
queue-size: 512
# ===================================================================
# Application specific properties
# Add your own application properties here, see the ApplicationProperties class
# to have type-safe configuration, like in the JHipsterProperties above
#
# More documentation is available at:
# http://www.jhipster.tech/common-application-properties/
# ===================================================================
datasource.********x.type: com.zaxxer.hikari.HikariDataSource
datasource.********x.url: "jdbc:oracle:thin:#********x"
datasource.********x.username: ********x
datasource.********x.password: ********x
datasource.********xy.type: com.zaxxer.hikari.HikariDataSource
datasource.********xy.url: "jdbc:oracle:thin:#yyyyyy"
datasource.********xy.username:********x
datasource.********xy.password: "********x"
package com.********x
import io.github.jhipster.config.JHipsterConstants;
import io.github.jhipster.config.liquibase.AsyncSpringLiquibase;
import liquibase.integration.spring.SpringLiquibase;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.liquibase.LiquibaseProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.core.task.TaskExecutor;
import org.springframework.data.jpa.repository.config.EnableJpaAuditing;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.sql.DataSource;
#Configuration
#EnableJpaRepositories("com.********x")
#EnableJpaAuditing(auditorAwareRef = "springSecurityAuditorAware")
#EnableTransactionManagement
public class DatabaseConfiguration {
private final Logger log = LoggerFactory.getLogger(DatabaseConfiguration.class);
private final Environment env;
public DatabaseConfiguration(Environment env) {
this.env = env;
}
#Autowired
#Qualifier("********x")
private DataSource dataSource;
#Bean
public SpringLiquibase liquibase(#Qualifier("taskExecutor") TaskExecutor taskExecutor
, LiquibaseProperties liquibaseProperties) {
// Use liquibase.integration.spring.SpringLiquibase if you don't want Liquibase to start asynchronously
SpringLiquibase liquibase = new AsyncSpringLiquibase(taskExecutor, env);
liquibase.setDataSource(dataSource);
liquibase.setChangeLog("classpath:config/liquibase/master.xml");
liquibase.setContexts(liquibaseProperties.getContexts());
liquibase.setDefaultSchema(liquibaseProperties.getDefaultSchema());
liquibase.setDropFirst(liquibaseProperties.isDropFirst());
if (env.acceptsProfiles(JHipsterConstants.SPRING_PROFILE_NO_LIQUIBASE)) {
liquibase.setShouldRun(false);
} else {
liquibase.setShouldRun(liquibaseProperties.isEnabled());
log.debug("Configuring Liquibase");
}
return liquibase;
}
}
So I injected primary Datasource in DatabaseConfiguration.java and using other datasource where required. Both these datasources are created similar to the above post like how spring suggests.

Here is how I make two datasource work in JHipster project :
The default datasource properties generate by JHipster:
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:h2:file:./target/h2db/db/mockpartenaire;DB_CLOSE_DELAY=-1
username: mockpartenaire
password:
Configs for the JHispter default datasource :
#Configuration
#EnableTransactionManagement
public class PrimaryDbConfig {
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties defaultDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSource defaultDataSource() {
return defaultDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean(name = "entityManagerFactory")
#Primary
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(defaultDataSource())
.packages(Input.class)
.persistenceUnit("default")
.build();
}
#Bean(name = "transactionManager")
#Primary
public JpaTransactionManager db2TransactionManager(#Qualifier("entityManagerFactory") final EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
}
The second datasource properties:
partner:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:postgresql://localhost:5432/partner
username: partner
password: partner
driver-class-name: org.postgresql.Driver
The second datasource configs :
#Configuration
#EnableTransactionManagement
#EntityScan(basePackages = "com.my.test.custom.domain")
#EnableJpaRepositories(transactionManagerRef = "partnerTransactionManager", entityManagerFactoryRef = "partnerEntityManagerFactory", basePackages = "com.my.test.custom.repository")
public class PartnerDbConfig {
#Bean
#ConfigurationProperties("partner.datasource")
public DataSourceProperties partnerDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("partner.datasource")
public DataSource partnerDataSource() {
return partnerDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean(name = "partnerEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
Properties properties = new Properties();
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
LocalContainerEntityManagerFactoryBean emf = builder
.dataSource(partnerDataSource())
.packages(TestPost.class)
.persistenceUnit("partner")
.build();
emf.setJpaProperties(properties);
return emf;
}
#Bean(name = "partnerTransactionManager")
public JpaTransactionManager db2TransactionManager(#Qualifier("partnerEntityManagerFactory") final EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
}
The default datasource configured by JHipster will continue to work it's entities. For your custom entities using the second datasource, here is the repository configuration :
#SuppressWarnings("unused")
#Repository
#Transactional("partnerTransactionManager")
#PersistenceContext(name = "partnerEntityManagerFactory")
public interface TestPostRepository extends JpaRepository<TestPost,Long>{
}

I found #freemanpolys answer great but it needs an extra property if you want it to work with postGres.
Add the following property to both the default and secondary datasource:
autocommit: false

How to use Hibernate with multiple databases on a single server

Our application allows our customer to have multiple databases, all running on one instance of the database server.
For example, databases might be dbcommon, dbLive, dbStaging, dbUAT, dbDev, dbSandbox. The common database and the Production database always exists, but the others are optional (and there is no limit). In dbcommon there is a table that tells us all the databases....so that's where I would need to start. The tables in common are different from the others, and the others all have the same schema (subscriber data)
Using Hibernate, how can I dynamically create/use a connection to either Live or Staging (or any of the others)? I am using Spring if that helps.
I have come across answers that suggest creating different connections in configuration, but because the number of subscriber databases can vary (per install, not while the app is running), this isn't an option for me.

As I discovered after posting this question, and as the user manish suggested, Hibernate's Multi Tenancy support (using the Database MultiTenancyStrategy) works for me. I had to piece together a solution using various resources (listed below).
http://www.ticnfae.co.uk/blog/2014/07/16/hibernate-multi-tenancy-with-spring/
Setting up a MultiTenantConnectionProvider using Hibernate 4.2 and Spring 3.1.1
Multi-Tenancy with Spring + Hibernate: "SessionFactory configured for multi-tenancy, but no tenant identifier specified"
I'm still looking for a way to be able to reference the common (shared) database at the same time as tenant databases...and will try to add that to this answer when complete.

The simplest way I can see to do this is to manage everything via profiles in Spring.
I accomplished this by using application.yml. I'm also using a Hikari connection pool, but that doesn't effect the configuration too much.
Here is an example of a application.yml with 3 profiles listed, and I've defined two of them as an example.
spring:
profiles:
include: dev,test,production
active: dev
---
spring:
profiles: dev
oms:
omsDataSource:
driverClassName: com.informix.jdbc.IfxDriver
jdbcUrl: jdbc:informix-sqli://devdb:9000/hol:INFORMIXSERVER=m_tcp_1;client_deve=en_US.8859-1;db_deve=en_US.8859-1;LOBCACHE=-1
password: oms
username: oms
connectionTestQuery: select count(*) from systables
maximumPoolSize: 5
---
spring:
profiles: test
oms:
omsDataSource:
driverClassName: com.informix.jdbc.IfxDriver
jdbcUrl: jdbc:informix-sqli://testdb:9000/hol:INFORMIXSERVER=m_tcp_1;client_deve=en_US.8859-1;db_deve=en_US.8859-1;LOBCACHE=-1
password: oms
username: oms
connectionTestQuery: select count(*) from systables
maximumPoolSize: 5
In my DB config class, I set the JPA repositories, and tell it what entityManager to use. I also setup the configuration properties to pull frmo the application.yml . This means it will swap out the details based on the profile the app is using on launch.
#EnableJpaRepositories(entityManagerFactoryRef = "entityManagerFactoryOms",
transactionManagerRef = "transactionManagerOms",
basePackages= "persistence.oms")
#Configuration
#ConfigurationProperties(prefix = "oms.omsDataSource")
public class omsDbConfig extends HikariConfig {
//This will automatically fill in the required fields from the application.yml.
#Bean
public HikariDataSource orcaDataSource() throws SQLException {
return new HikariDataSource(this);
}
//I use that datasource to define my entityMangerFactory
#Bean(name = "entityManagerFactoryOms")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryOrca() throws SQLException {
JpaVendorAdapter adapter = new HibernateJpaVendorAdapter();
Properties props = new Properties();
props.setProperty("hibernate.dialect","org.hibernate.dialect.InformixDialect");
LocalContainerEntityManagerFactoryBean emfb =
new LocalContainerEntityManagerFactoryBean();
emfb.setDataSource(orcaDataSource());
emfb.setPackagesToScan("persistence.oms");
emfb.setJpaProperties(props);
emfb.setJpaVendorAdapter(adapter);
return emfb;
}
}
The entities and repositories are defined normally, there's nothing special there. The DB will switch connection based on whatever profile I tell it to run.
I just switch out the active profile in the application.yml to whichever one I need.
Safety note: Define a production profile, don't have production as a default profile.