We Keep Coding

Spring - yaml and .properties configurations

i've been working with Spring for some time and have a question regarding the very common configuration properties files (like the common application.properties that comes with every spring boot app you initialize). Recently, i also found that configurations like this can be done in yaml files. I have two questions:
When in a application.properties file, we write something like:
# application.properties
spring.debug = true
some-random-value = 15
does this mean that these values will be injected in the application context?
When we write something like:
# application.properties
spring.debug = true
does this mean, that somewhere, there is some class, that has an attribute that looks something like this? -->
#Component
class SomeClass{
#Value("spring.debug")
boolean shouldIRunInDebugMode;
...
}
2.a. If the answer to question 2 is yes, then how can I, looking at something like this:
# application.properties
spring.debug = true
find that class that is expecting that value. The same would apply to if i was looking at something like:
# application.yaml
someThidPartyLibraryName:
shouldWeLog: true
If i see a yaml configuration file, just looking at all the configuration there usually is not enough for me to know what is happening. How can i find the class that is being affected by this configuration, so that i can better understand what this config is doing?
Thank you!

The response is generally yes. If you declare a property in the application.properties or application.yaml is mainly, because you would use it later in the code, for example injecting in some bean with the support of #Value annotation. However, there are also many built-in properties (let's say for example server.port), which you usually don't have to declare and therefore you won't see explicitly in the code. Use an IDE to search the configuration properties and the manual to check the preconfigured ones in case of need.

Your understanding regarding spring value injections from application.properties is correct. #2 - is Yes. Any property from application.properties can be injected to any java class as #Value.
Regarding #2.a - Yaml is just another format on how you organize your variable hierarchy by indentations. That's a superset to the JSON structure.
For example,
in application.properties file you can add something like this
myapp.db.url=<dburl>
myapp.db.username=<dbuser>
myapp.db.password=<dbpassword>
the same can be represented in Yaml in a much efficient manner as below
myapp:
db:
url:<dburl>
username:<dbuser>
password:<dbpassword>
And in either case, for your Jave file you can inject as
#Value("myapp.db.url"
private String dbUrl;

Properties defined in yaml or a properties file may be accessed using the #Value annotation to inject, or using a #ConfigurationProperties class - see https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-external-config-typesafe-configuration-properties for complete details.
Finding the property usage is supported by some IDEs - IntelliJ allows you to click through. Otherwise it's a search through the source. For #ConfigurationProperties, once you find the class then just look for code that calls its accessor methods.

Properties files and yaml files are used in Spring Boot for configurations. The main difference between the two is yaml provides structuring/grouping of configurations where as Properties are usually flat and may be repeating the same information:
For example;
Properties file
server.port = 8080
server.host = localhost
yaml file
server:
port: 8080
host: localhost
But in a Spring Boot AutoConfiguration class regardless of yaml or Properties used, a following looking ConfigurationProperties class will be used which will map server.port and server.host
#ConfigurationProperties(prefix = "server")
public class ServerConfiguration {
private int port;
private String host;
}
#Configuration
#EnableConfigurationProperties(ServerConfiguration.class)
public class ServerAutoConfiguration {
}
Hope this answers your questions.

Setting the Transaction Isolation Level through Spring Boot properties

I have a spring boot application which I have configured most of the properties through the properties file. However, I was looking if there is a way to set the TRANSACTION_ISOLATION_LEVEL through the Spring boot properties. Could someone help me on this.
I'm initializing the data source bean in the following way:
#Bean
#ConfigurationProperties(prefix="spring.datasource")
public DataSource dataSource() {
return buildDataSource("spring.datasource");
}
private DataSource buildDataSource(String propPrefix) {
Stirng driverClassName = env.getProperty(propPrefix + ".driver-class-name");
return DataSourceBuilder.create()
.driverClassName(driverClassName)
.build();
}
Could someone please help me on how to specify the TRANSACTION_ISOLATION_LEVEL either through properties or during the data source initialization.

So, the javax.sql.DataSource does not provide a way to set default Transaction Isolation Level. Still, you can do it, but you must strict to particular implementation. Let me give you c couple of examples:
In case you use Apache BasicDataSource implementation of DataSource, then you can use this. This is for DBCP.
If you are using Apache BasicDataSource, but for DBCP2, you can do something like this.
But in most cases, if we are talking about Spring, we use Hikari Connection Pool. So, in case of HikariCP, you can use this method.
The same is applicable to Spring Boot. Let me explain - using Spring Boot properties file, you can set default transaction isolation level, but for specific DataSource, I mean this property:
spring.datasource.hikari.transaction-isolation
as you probably noticed, let you set default transaction isolation level for HikariCP (if you are using one) And this property:
spring.datasource.dbcp2.default-transaction-isolation
allow you to configure default isolation level for DBCP2.
Hope it helped, have a nice day! :)

Configure Multiple Database on Spring-Boot with JPA and Hibernate Web Application

I have ONE spring boot (1.5.4.RELEASE) project using java 8 deployed on AWS HPC. This project architect scope works for Spring Web Application(Website), Rest API Services(Mobile Developer) & Account Administration for Company.
So there is 3 different respective Database like (2-SQL Server & 1-MySQL).
Here on stack-overflow, I'm posting my question for find a best way to implementation this Spring-Boot Project by help of talented stack-overflow users.
Here is my configure properties files.
application.properties
#For Public Website
spring.datasource.clone.web.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.clone.web.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName=PROD_WEB;
# Username and password
spring.datasource.web.username = web
spring.datasource.web.password = ED5FLW64ZU976Q36
#For Rest API
spring.datasource.clone.url = jdbc:mysql://localhost:3306/PROD_REST;
# Username and password
spring.datasource.clone.username = rest
spring.datasource.clone.password = Firewall77#
#For Account Administration for Company Users
spring.datasource.admin.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName=PROD_ADMIN;
# Username and password
spring.datasource.admin.username = admin
spring.datasource.admin.password = Firewall77#
# Backup & Cron Policy
...
I would greatly appreciate for some very good suggestion to implement it. your knowledge on this subject would help me, Thanks.

You need to implement two different beans, one for each datasource and make them take the corresponding configuration properties respectively:
First bean will be responsible for the first datasource configuration, and should be daclared as primary datasource with #Primary, so it can be setup as the main datasource for the project.
The second bean will configure the second datasource.
This is how they should be implemented in Spring:
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource.web")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.datasource.rest")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Here's how should be your application.properties configured, to take these two beans configuration into account:
#For Public Website
spring.datasource.web.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.web.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName= PROD_WEB;
# Username and password
spring.datasource.web.username = web
spring.datasource.web.password = ED5FLW64ZU976Q36
#For Rest API
spring.datasource.rest.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.rest.url = jdbc:mysql://localhost:3306/PROD_REST;
# Username and password
spring.datasource.rest.username = rest
spring.datasource.rest.password = Firewall77#
Note:
I changed the configuration properties here so they can be differentiated between web and rest datasources:
Properties starting with spring.datasource.web will be dedicated to configure the first datasource.
Properties starting with spring.datasource.rest will be dedicated to configure the second datasource.

try to use DataSource configuration specified at Configuration for each data sources for more help check this Using multiple datasources with Spring Boot and Spring Data

Cannot undeploy from Tomcat due to specific Spring JMS configuration

I have used ActiveMQ as JMS implementation (activemq-spring 5.12.1) and Spring JMS integration (spring-jms 4.2.3.RELEASE), all wrapped in Spring Boot web application, being deployed on Tomcat.
I have following Spring configuration (code reduced for the verbosity of code sample):
#Configuration
#EnableJms
public class AppConfiguration {
#Bean
public XAConnectionFactory jmsXaConnection(String activeMqUsername, String activeMqPassword) {
ActiveMQXAConnectionFactory activeMQXAConnectionFactory = new ActiveMQXAConnectionFactory(activeMqUsername, activeMqPassword, activeMqUrl);
ActiveMQPrefetchPolicy prefetchPolicy = new ActiveMQPrefetchPolicy();
prefetchPolicy.setAll(0);
activeMQXAConnectionFactory.setPrefetchPolicy(prefetchPolicy);
return activeMQXAConnectionFactory;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory, JtaTransactionManager jtaTransactionManager) {
DefaultJmsListenerContainerFactory containerFactory = new DefaultJmsListenerContainerFactory();
containerFactory.setConnectionFactory(connectionFactory);
containerFactory.setTransactionManager(jtaTransactionManager);
containerFactory.setSessionTransacted(true);
containerFactory.setTaskExecutor(Executors.newFixedThreadPool(2));
containerFactory.setConcurrency("2-2");
containerFactory.setCacheLevel(DefaultMessageListenerContainer.CACHE_CONSUMER);
return containerFactory;
}
}
My target was to configure two consumers (hence concurrecny set to 2-2) and to prevent any messages caching (hence prefetch policy set to 0).
It works, but causes very unpleasent side effect:
When I try to undeploy the application via Tomcat Manager, it hangs for a while and then indefinitely, every second produces following DEBUG message:
"DefaultMessageListenerContainer:563 - Still waiting for shutdown of 2 Message listener invokers".
Therefore, I am forced to kill Tomcat process every time. What have I done wrong?
One of my lucky shots (documentation both ActiveMQ and Spring JMS was not that helpful), was to set prefetch policy to 1 instead of 0. Then it undeploys gracefully, but I cannot see how it can relate.
Also I am curious, why having cache level set to CACHE_CONSUMER is required for the ActiveMQ to create two consumers. When default setting was left (CACHE_NONE while using external transaction manager), only one consumer was created (while concurrency was still set two 2-2, and so was TaskExecutor).
If it matters, for connection factory and transaction manager, Atomikos is used. I can paste its configuration also, but it seems irrelevant.

Most likely this means the consumer threads are "stuck" in user code; take a thread dump with jstack to see what the container threads are doing.

Spring Data JPA with ssh tunnel to a remote MySQL server

I'm using Spring Data JPA with Hibernate as persistence provider in conjunction with a remote MySQL5 Server for a job that periodically replicates a subset of internal data. The job (i.e. a quartz-scheduled java application) runs once per dai and needs approx. 30seconds to complete the synchronization). For safety reasons we don't want to open the remote server for direct connections from outside (i.e. other than localhost).
I've seen examples with Jsch to programmatically set up an ssh tunnel, but could not finde any resources on how to integrate Jsch with spring data. One problem I'm seeing is that certain of my spring beans (i.e. org.apache.commons.configuration.DatabaseConfiguration) are created at application startup and already needs access to the datasource.
I could open the ssh tunnel outside of the application, but then it would be opened all the time, but I wanted to avoid that as I only need it opened 30seconds per day.
EDIT:
After some research I found several ways to get a ssh tunnel
A) Implementing my own DataSource (I extended org.springframework.jdbc.datasource.DriverManagerDataSource) and then used PostContruct and Predestroy to setup / close the ssh tunnel with Jsch
--> Problem: The ssh tunnel remains open for the lifetime of the application, what is not what I want
B) Implementing my own Driver (I extended com.mysql.jdbc.Driver) and overwrite "connect" to create the ssh tunnel before the connection
--> Problem: I'm not able to close the ssh tunnel connection
Any more suggestions are welcome

If you have a DataSource bean in your Spring configuration, you can create your own DataSource implementation that opens an SSH tunnel before attempting to make a connection using the provided JDBC URL. As an example, consider the following configuration that uses a HikariDataSource:
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource">
<bean class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">...</bean>
</property>
</bean>
You can extend the class HikariDataSource to provide your own implementation. An example below:
class TunneledHikariDataSource extends HikariDataSource implements InitializingBean {
private boolean createTunnel = true;
private int tunnelPort = 3306;
public void afterPropertiesSet() {
if(createTunnel) {
// 1. Extract remote host name from the JDBC URL.
// 2. Extract/infer remote tunnel port (e.g. 3306)
// from the JDBC URL.
// 3. Create a tunnel using Jsch and sample code
// at http://www.jcraft.com/jsch/examples/PortForwardingL.java.html
...
}
}
}
Then, instantiate a bean instance for the custom class instead of HikariDataSource.

Expanding on #manish's answer here is my solution that works with Spring Boot in 2022:
#Configuration
class DataSourceInitializer {
#Bean
fun dataSource(properties: DataSourceProperties): DataSource {
return properties.initializeDataSourceBuilder().type(SshTunnelingHikariDataSource::class.java).build()
}
}
class SshTunnelingHikariDataSource : HikariDataSource(), InitializingBean {
override fun afterPropertiesSet() {
val jsch = JSch()
val filePath = javaClass.classLoader.getResource("id_rsa")?.toURI()?.path
jsch.addIdentity(filePath, "optional_key_file_passphrase")
val session = jsch.getSession("root", "remote-host.com")
val config = Properties()
config["StrictHostKeyChecking"] = "no";
session.setConfig(config)
session.connect()
session.setPortForwardingL(3307, "db-host.com", 3306)
}
}
Depending on your connection you may not need the identity file (i. e. a private key) but instead you may need username + password which you can provide at getSession.
remote-host.com is the host you want to have your SSH session to terminate in, having access to the database
db-host.com is the host that your remote-host can resolve. It may as well be localhost if the database is running locally on your remote-host