I have a Spring Boot application which copies external JAR files to a folder, depending on certain conditions. These JARs can contain many Spring components (i.e. classes annotated or meta-annotated with #Component) and the Spring application should be able scan and instantiate for these beans. Is it possible, based on certain conditions, to dynamically load the contents of the JAR files and make them available to the Spring application context? I am fully aware of the security implications this has.
I have read about the different types of Launchers which Spring provides for its executable JAR format, such as JarLauncher and PropertiesLauncher, but it looks like that these launchers do not detect changes to the classpath, but instead only scan the directories once for JAR files.
The following simple application demonstrates the problem:
// .../Application.java
#SpringBootApplication
public class Application {
public static void main(String[] args) {
System.out.println("Please copy JAR files and press Enter ...");
System.in.read();
SpringApplication.run(Application.class, args);
}
}
Replace the default JarLauncher with PropertiesLauncher:
// build.gradle
tasks.named('bootJar') {
manifest {
attributes 'Main-Class': 'org.springframework.boot.loader.PropertiesLauncher',
'Start-Class': 'com.example.customlauncher.Application'
}
}
Specify the location to the external JARs in the properties file of the PropertiesLauncher:
# .../resources/loader.properties
loader.path=file:/path/to/dir
The application is a Spring Initializer Gradle application and packaged by running the bootJar task: ./gradlew bootJar.
It is then started with the following command:
java -jar build/libs/customlauncher-0.0.1-SNAPSHOT.jar
This works if the JAR file is already present at the specified location (/path/to/dir), but it does not work if the java command is executed while the directory is empty and the JAR file is then copied while the app waits for the user to copy the files and press Enter ↲.
There are a couple of related questions, but it looks like they all assume that the JAR files already exist at the time of starting the JVM:
How to put a directory first on the classpath with Spring Boot?
Spring Boot Executable Jar with Classpath
SpringBoot external jar not load
Is there a way to achieve this without too many awkard hacks? Or is recommended to utilize something like OSGi? Am I looking at this completely wrong and there is a better way to have JARs on the classpath that do not need always need loading (if the JAR is "disabled", it should not be loaded/compiled by the JVM, should not be picked up by Spring, etc.)?
It looks like this is possible if the JAR files are copied before starting the Spring application. It feels hackish, but it works. Use at your own risk!
You need two classes, one for bootstrapping the external JARs, which will then start the second via a manually created PropertiesLauncher. The bootstrapping class can be a plain old regular Java class (but it can be a Spring Boot Application too) and only the second class needs to be a SpringBootApplication.
// BootstrapApplication.java
public class BootstrapApplication {
public static void main(String[] args) {
System.out.println("Please copy JAR files and press Enter ...");
System.in.read();
PropertiesLauncher.main(args);
}
}
// Application.java
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
In the gradle file, we can switch back to the default JarLauncher, by removing the bootJar task manifest configuration and applying settings via the springBoot configuration block. mainClass will end up as Start-Class in the MANIFEST.MF file.
// build.gradle
springBoot {
mainClass = 'com.example.customlauncher.BootstrapApplication'
}
In the properties file for the loader, a new property needs to be set, which points to the real application class. The settings in this file are only picked up by PropertiesLauncher and ignored by JarLauncher. In other words: JarLauncher delegates to Start-Class from the manifest file and PropertiesLauncher delegates to loader.main from its properties file.
# .../resources/loader.properties
loader.path=file:/path/to/dir
loader.main=com.example.customlauncher.Application
Spring (Boot) will first call the main method of BootstrapApplication, as specified in the MANIFEST.MF file (controlled via springBoot configuration block in the build.gradle file). In the implementation of this main method, a new PropertiesLauncher is created with the main class set to the "real" application (i.e. Application).
Executing the application is still done via the same invocation:
java -jar build/libs/customlauncher-0.0.1-SNAPSHOT.jar
Any JAR files added to /path/to/dir after the JVM has started, but before calling PropertiesLauncher#main in BootstrapApplication are then available in the classpath and application context as seen from Application.
Related
I have some library jar lib.jar (made using spring boot but packaged as normal jar without spring boot plugin) which is made of spring boot and contains spring.components file generated by spring-context-indexer.
Now, I'm using this jar in my application which also has spring-context-indexer and its own spring.components file and uses some of the bean defined in lib.jar.
When I start my application, spring should register all beans defined in spring.components of lib.jar and spring.components of application. But spring isn't registering any of bean of lib.jar.
I tried using basePackages property of #SpringBootApplication but no results.
I even copied all entries of spring.components of lib.jar into spring.components of my application but no result.
Can anyone please help me?
TL;DR
If you're using Spring Data, #SpringBootApplication.scanBasePackages is not enough, you also need #EnableJdbcRepositories (or *Jpa* or whatsoever).
package application;
// without this annotation all Repository classes
// from library will be missing
#EnableJdbcRepositories({
"application",
"library"
})
#SpringBootApplication(
scanBasePackages = {
"application",
"library"
}
)
public class Application {
public static void main(final String[] args) {
SpringApplication.run(Application.class, args);
}
}
Some more info
Okay, maybe I'm a bit late, but I decided to investigate this case a bit.
That's what I've found as of 2 Feb 2022:
All META-INF/spring.components files are loaded in CandidateComponentsIndexLoader.doLoadIndex. You can use debug to check whether it sees file from lib or not
CandidateComponentsIndexLoader then creates CandidateComponentsIndex, which is then stored in the component scanner, for me it is AnnotationConfigServletWebServerApplicationContext.scanner.componentsIndex
Then in ClassPathScanningCandidateComponentProvider findCandidateComponents is called, which, if componentsIndex is not null, just gets components from that index by provided basePackage.
That's why missing basePackage is crucial.
I haven't dug into the Spring Data algorithms, but in my case Spring hadn't been generating library Repositories until I added the #EnableJdbcRepositories with packages.
P.S. All links represent files at the 5.3.15 tag, latest atm.
I have a spring boot application that I can package in a war that I want to deploy to different environments. To automate this deployment it'd be easier to have the configuration file externalized.
Currently everything works fine with a application.properties file in src/main/resources. Then I use ´mvn install´ to build a war deployable to tomcat.
But I would like to use a .yml file that does not need to be present on mvn install but that would be read from during deployment of the war and is in the same or a directory relative to my war.
24. externalized configuration shows where spring boot will look for files and 72.3 Change the location of external properties of an application gives more detail on how to configure this but I just do not understand how to translate this to my code.
My application class looks like this:
package be.ugent.lca;
Updated below
Do I need to add a #PropertySource to this file? How would I refer to a certain relative path?
I feel like it's probably documented in there as most spring boot documentation but I just don't understand how they mean me to do this.
EDIT
Not sure if this should be a separate issue but I think it's still related.
Upon setting the os variable the error of yaml file not found went away. Yet I still get the same error again as when I had no application .properties or .yml file.
Application now looks like this:
#Configuration
**#PropertySource("file:${application_home}/application.yml")**
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
The application_home OS variable
$ echo $application_home
C:\Masterproef\clones\la15-lca-web\rest-service\target
My application.yml file(part it complains about):
sherpa:
package:
base: be.ugent.lca
Error upon java -jar *.war
All variations upon:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'sherpa.package.base' in string value "${sherpa.package.base}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:204)
at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:178)
at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$2.resolveStringValue(PropertySourcesPlaceholderConfigurer.java:172)
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:808)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1027)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1014)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:545)
... 142 more
Using external properties files
The answer lies in the Spring Boot Docs, I'll try to break it down for you.
First of all, no you should not use #PropertySource when working with Yaml configuration, as mentioned here under the Yaml shortcomings :
YAML files can’t be loaded via the #PropertySource annotation. So in the case that you need to load values that way, you need to use a properties file.
So, how to load propery files? That is explained here Application Property Files
One is loaded for you: application.yml , place it in one of the directories as mentioned in the link above. This is great for your general configuration.
Now for your environment specific configuration (and stuff like passwords) you want to use external property files, how to do that is also explained in that section :
If you don’t like application.properties as the configuration file name you can switch to another by specifying a spring.config.name environment property. You can also refer to an explicit location using the spring.config.location environment property (comma-separated list of directory locations, or file paths).
So you use the spring.config.location environment property.
Imagine you have an external config file: application-external.yml in the conf/ dir under your home directory, just add it like this:
-Dspring.config.location=file:${home}/conf/application-external.yml as a startup parameter of your JVM.
If you have multiple files, just seperate them with a comma. Note that you can easily use external properties like this to overwrite properties, not just add them.
I would advice to test this by getting your application to work with just your internal application.yml file , and then overwrite a (test) property in your external properties file and log the value of it somewhere.
Bind Yaml properties to objects
When working with Yaml properties I usually load them with #ConfigurationProperties, which is great when working with for example lists or a more complex property structure. (Which is why you should use Yaml properties, for straightforward properties you are maybe better of using regular property files). Read this for more information: Type-Safe Configuration properties
Extra: loading these properties in IntelliJ, Maven and JUnit tests
Sometimes you want to load these properties in your maven builds or when performing tests. Or just for local development with your IDE
If you use IntelliJ for development you can easily add this by adding it to your Tomcat Run Configuration : "Run" -> "Edit Configurations" , select your run configuration under "Tomcat Server" , check the Server tab and add it under "VM Options".
To use external configuration files in your Maven build : configure the maven surefire plugin like this in your pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dspring.config.location=file:${home}/conf/application-external.yml</argLine>
</configuration>
</plugin>
When running JUnit tests in IntelliJ:
Run → Edit Configurations
Defaults → JUnit
add VM Options -> -ea -Dspring.config.location=file:${home}/conf/application-external.yml
Yes, you need to use #PropertySource as shown below.
The important point here is that you need to provide the application_home property (or choose any other name) as OS environment variable or System property or you can pass as a command line argument while launching Spring boot. This property tells where the configuration file (.properties or .yaml) is exactly located (example: /usr/local/my_project/ etc..)
#Configuration
#PropertySource("file:${application_home}config.properties")//or specify yaml file
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
There is a very simple way to achieve this.
Inside your original application.properties file you can just specify the following line:
spring.config.import=file:Directory_To_The_File/Property_Name.properties
It will automatically sync all the properties from the external property file.
Now lets say that you have a situation where you need to get properties from multiple property files. In that case, you can mention the same line in the external property file which in turn will take the remaining properties from the second property file and so on.
Consider the following example.
application.properties:
spring.config.import=file:Resources/Custom1.properties
Custom1.properties:
server.port=8090
.
.
.
spring.config.import=file:Resources/Custom2.properties
One of the easiest way to use externalized property file using system environment variable is, in application.properties file you can use following syntax:
spring.datasource.url = ${OPENSHIFT_MYSQL_DB_HOST}:${OPENSHIFT_MYSQL_DB_PORT}/"nameofDB"
spring.datasource.username = ${OPENSHIFT_MYSQL_DB_USERNAME}
spring.datasource.password = ${OPENSHIFT_MYSQL_DB_PORT}
Now, declare above used environment variables,
export OPENSHIFT_MYSQL_DB_HOST="jdbc:mysql://localhost"
export OPENSHIFT_MYSQL_DB_PORT="3306"
export OPENSHIFT_MYSQL_DB_USERNAME="root"
export OPENSHIFT_MYSQL_DB_PASSWORD="123asd"
This way you can use different value for same variable in different environments.
Use below code in your boot class:
#PropertySource({"classpath:omnicell-health.properties"})
use below code in your controller:
#Autowired
private Environment env;
i want to use Spring Boot with Spring Security and Apache Wicket 8.0 as web view.
It`s easy to do it with SpringBoot and just simply run jar file, but i want to do something like this:
Minecraft Server (Spigot, Bukkit, etc..) Scanning /plugins/ folder, and looking for .jar files.
All files must contain plugin.yml in root of .jar archive. This plugin.yml file contains a path to class file, for example - BukkitMain and this class MUST extend JavaPlugin.
Then MinecraftServer core is executing onEnable() method of BukkitMain.class (so, if we override this method - server code will execute all code inside this method)
For Example:
public class BukkitMain extends JavaPlugin {
#Override
public void onEnable(){
//SpringApplicationBuilder.run(....);
}
}
So, i can use Spring-boot-plugin in build part of maven .pom file, but it's packing all class files and resources to BOOT-INF inside jar and it's requiring main(String args[]) method to run SpringApplication. It`s bad variant, because i can't access my BukkitMain to run it.
When i use Maven Compiler Plugin and packing it to jar - it's all ok, but there is new problem. My BukkitMain trying to run SpringApplicationBuilder, but i'm getting ClassNotFoundException, cause there is no SpringApplicationBuilder inside jar.
How can I set the logging path relative to tomcat dir /logs/mylog.log?
What I tried: changing the logging.file property in application.properties
leaving the filename out: #logging.file= -> everything is logged to console, thus written into tomcat/logs/localhost.yyyy-mm-dd.log
logging.file=mylog.log -> written to console, thus same as #logging.file
logging.file=d:/mylog.log -> written to the location d:/mylog.log
logging.file=../logs/mylog.log -> written to console, thus still to localhost*.log
None was successful.
I'm not interested in externalising the configuration eg by providing system or environment variables.
I just created a simple Spring-bootapp from spring starter build as war file. I have just this modification in #SpringBootApplication class:
#SpringBootApplication
public class LogApplication {
private static final Logger logger = Logger.getLogger(LogApplication.class);
public static void main(String[] args) {
SpringApplication.run(LogApplication.class, args);
}
#Controller
#ResponseBody
public static class IndexController{
#RequestMapping("/")
public String getindex(){
logger.error("Error Logging");
return "Hello";
}
}
}
And this property in application.properties:
logging.file=../logs/mylog.log
Build the application using maven mvn clean install and put the war file inside webapps folder of tomcat. Started tomcat using startup.bat and hit successful the endpoint http://localhost:8080/demo-0.0.1-SNAPSHOT.
And the log was written in logs/mylog.log:
2017-01-04 14:57:10.755 ERROR 8236 --- [http-apr-8080-exec-4] com.example.LogApplication : Error Logging
You can make use of the environment variable for configuring the log path.
Tomcat sets a catalina.home system property which you can use
log4j.rootCategory=DEBUG,errorfile
log4j.appender.errorfile.File=${catalina.home}/logs/LogFilename.log
Note:-
This may not work On Debian (including Ubuntu), ${catalina.home} will not work because that points at /usr/share/tomcat6 which has no link to /var/log/tomcat6. Here just use ${catalina.base}. Check this link
I'm going to second Tomz's response and point you to the docs because they show you how to switch over from logback to log4j which is probably easier for you.
I would strongly recommend not deploying Spring Boot in war files, but as executable fat jars. It makes things a lot easier when you can just type this to test a configuration and deploy it:
java -jar my-service.jar /opt/my-service/conf/application.yml
I have web application running with a default impl of a backend service. One should be able to implement the interface and drop the jar into the plugins folder (which is not in the apps classpath). Once the server is restarted, the idea is to load the new jar into the classloader, and have it take part in dependency injection. I am using Spring DI using #Autowired. The new plugin service impl will have #Primary annotation. So given two impls of the interface, the primary should be loaded.
I got the jar loaded into the classloader and can invoke the impl manually. But I haven't been able to get to to participate in the Dependency Injection, and have it replace the default impl.
Here's a simplified example:
#Controller
public class MyController {
#Autowired
Service service;
}
//default.jar
#Service
DefaultService implements Service {
public void print() {
System.out.println("printing DefaultService.print()");
}
}
//plugin.jar not in classpath yet
#Service
#Primary
MyNewService implements Service {
public void print() {
System.out.println("printing MyNewService.print()");
}
}
//For lack of better place, I loaded the plugin jar from the ContextListener
public class PluginContextLoaderListener extends org.springframework.web.context.ContextLoaderListener {
#Override
protected void customizeContext(ServletContext servletContext,
ConfigurableWebApplicationContext wac) {
System.out.println("Init Plugin");
PluginManager pluginManager = PluginManagerFactory.createPluginManager("plugins");
pluginManager.init();
//Prints the MyNewService.print() method
Service service = (Service) pluginManager.getService("service");
service.print();
}
}
<listener>
<listener-class>com.plugin.PluginContextLoaderListener</listener-class>
</listener>
Even after I have loaded the jar into the classloader, DefaultService is still being injected as service. Any idea how I get the plugin jar to participate into the spring's DI lifecycle?
Edited:
To put it simply, I have a war file that has a few plugin jars in a plugins directory inside the war. Based on a value from a configuration file that the app looks at, when the app is started, I want to load that particular plugin jar and run the application with it. That way, I can distribute the war to anyone, and they can choose which plugin to run based on a config value without having to to repackage everything. This is the problem I am trying to solve.
It seems like all You need is to create the Spring ApplicationContext properly. I think it's possible without classpath mingling. What matters most are the locations of the Spring configuration files within the classpath. So put all Your plugin jar's into WEB-INF/lib and read on.
Let's start with the core module. We'll make it to create it's ApplicationContext from files located at classpath*:META-INF/spring/*-corecontext.xml.
Now we'll make all plugins to have their config files elsewhere. I.e. 'myplugin1' will have its config location like this: classpath*:META-INF/spring/*-myplugin1context.xml. And anotherplugin will have the configs at classpath*:META-INF/spring/*-anotherplugincontext.xml.
What You see is a convension. You can also use subdirectiries if You like:
core: classpath*:META-INF/spring/core/*.xml
myplugin1: classpath*:META-INF/spring/myplugin1/*.xml
anotherplugin: classpath*:META-INF/spring/anotherplugin/*.xml
What matters is that the locations have to be disjoint.
All that remains is to pass the right locations to the ApplicationContext creator. For web applications the right place for this would be to extend the ContextLoaderListener and override the method customizeContext(ServletContext, ConfigurableWebApplicationContext).
All that remains is to read Your config file (its location can be passed as servlet init parameter). Than You need to construct the list of config locations:
String locationPrefix = "classpath*:META-INF/spring/";
String locationSiffix = "/*.xml";
List<String> configLocations = new ArrayList<String>();
configLocations.add(locationPrefix + "core" + locationSiffix);
List<String> pluginsTurnedOn = getPluginsTurnedOnFromConfiguration();
for (String pluginName : pluginsTurnedOn) {
configLocations.add(locationPrefix + pluginName + locationSiffix);
}
applicationContext.setConfigLocations(configLocations.toArray(new String[configLocations.size()]));
This way You can easily manage what is and what is not loaded into Spring ApplicationContext.
Update:
To make it work there's one more hidden assumption I made that I'm about to explain now. The base package of the core module and each plugin should also be disjoint. That is i.e.:
com.mycompany.myapp.core
com.mycompany.myapp.myplugin1
com.mycompany.myapp.anotherplugin
This way each module can use <context:componet-scan /> (on equivalent in JavaConfig) easily to add classpath scanning for it's own classes only. The core module should not contain any package scanning of any plugin packages. The plugins should extend configuration of ApplicationContext to add their own packages to classpath scanning.
If you restart the server, I see no reason why you can't just add the JAR to the WEB-INF/lib and have it in the CLASSPATH. All the complication of a custom class loader and context listener goes away, because you treat it just like any other class under Spring's control.
If you do it this way because you don't want to open or modify a WAR, why not put it in the server /lib directory? Let the server class loader pick it up. This makes all plugin classes available to all deployed apps.
The answer depends on how important the separate /plugin directory is. If it's key to the solution, and you can't add the JAR to the server's /lib directory, then that's that. I've got nothing. But I think it'd be worthwhile to at least revisit the solution you have to make sure that it's the only way to accomplish what you want.