We Keep Coding

How to add external JARs to Spring application without restarting JVM?

I have a Spring Boot application which copies external JAR files to a folder, depending on certain conditions. These JARs can contain many Spring components (i.e. classes annotated or meta-annotated with #Component) and the Spring application should be able scan and instantiate for these beans. Is it possible, based on certain conditions, to dynamically load the contents of the JAR files and make them available to the Spring application context? I am fully aware of the security implications this has.
I have read about the different types of Launchers which Spring provides for its executable JAR format, such as JarLauncher and PropertiesLauncher, but it looks like that these launchers do not detect changes to the classpath, but instead only scan the directories once for JAR files.
The following simple application demonstrates the problem:
// .../Application.java
#SpringBootApplication
public class Application {
public static void main(String[] args) {
System.out.println("Please copy JAR files and press Enter ...");
System.in.read();
SpringApplication.run(Application.class, args);
}
}
Replace the default JarLauncher with PropertiesLauncher:
// build.gradle
tasks.named('bootJar') {
manifest {
attributes 'Main-Class': 'org.springframework.boot.loader.PropertiesLauncher',
'Start-Class': 'com.example.customlauncher.Application'
}
}
Specify the location to the external JARs in the properties file of the PropertiesLauncher:
# .../resources/loader.properties
loader.path=file:/path/to/dir
The application is a Spring Initializer Gradle application and packaged by running the bootJar task: ./gradlew bootJar.
It is then started with the following command:
java -jar build/libs/customlauncher-0.0.1-SNAPSHOT.jar
This works if the JAR file is already present at the specified location (/path/to/dir), but it does not work if the java command is executed while the directory is empty and the JAR file is then copied while the app waits for the user to copy the files and press Enter ↲.
There are a couple of related questions, but it looks like they all assume that the JAR files already exist at the time of starting the JVM:
How to put a directory first on the classpath with Spring Boot?
Spring Boot Executable Jar with Classpath
SpringBoot external jar not load
Is there a way to achieve this without too many awkard hacks? Or is recommended to utilize something like OSGi? Am I looking at this completely wrong and there is a better way to have JARs on the classpath that do not need always need loading (if the JAR is "disabled", it should not be loaded/compiled by the JVM, should not be picked up by Spring, etc.)?

It looks like this is possible if the JAR files are copied before starting the Spring application. It feels hackish, but it works. Use at your own risk!
You need two classes, one for bootstrapping the external JARs, which will then start the second via a manually created PropertiesLauncher. The bootstrapping class can be a plain old regular Java class (but it can be a Spring Boot Application too) and only the second class needs to be a SpringBootApplication.
// BootstrapApplication.java
public class BootstrapApplication {
public static void main(String[] args) {
System.out.println("Please copy JAR files and press Enter ...");
System.in.read();
PropertiesLauncher.main(args);
}
}
// Application.java
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
In the gradle file, we can switch back to the default JarLauncher, by removing the bootJar task manifest configuration and applying settings via the springBoot configuration block. mainClass will end up as Start-Class in the MANIFEST.MF file.
// build.gradle
springBoot {
mainClass = 'com.example.customlauncher.BootstrapApplication'
}
In the properties file for the loader, a new property needs to be set, which points to the real application class. The settings in this file are only picked up by PropertiesLauncher and ignored by JarLauncher. In other words: JarLauncher delegates to Start-Class from the manifest file and PropertiesLauncher delegates to loader.main from its properties file.
# .../resources/loader.properties
loader.path=file:/path/to/dir
loader.main=com.example.customlauncher.Application
Spring (Boot) will first call the main method of BootstrapApplication, as specified in the MANIFEST.MF file (controlled via springBoot configuration block in the build.gradle file). In the implementation of this main method, a new PropertiesLauncher is created with the main class set to the "real" application (i.e. Application).
Executing the application is still done via the same invocation:
java -jar build/libs/customlauncher-0.0.1-SNAPSHOT.jar
Any JAR files added to /path/to/dir after the JVM has started, but before calling PropertiesLauncher#main in BootstrapApplication are then available in the classpath and application context as seen from Application.

Replace java platform system logger with slf4j in spring boot application

I've got a spring boot application build as multi-modular gradle project (old-style, not fancy jigsaw)
What I want to achieve - is replace java platform loggers (e.g., SSlLogger/ System.getLogger) with sl4j & logback that are used in my app and are managed by spring-boot-admin server at runtime. I need all my loggers to write to file instead of console - or else I won't see logs in Logz.io.
I do not control how my app is deployed, but I control the way fat jar is built (so, manuals with terminal commands 'java - ...' are not very helpful :( )
I started to follow https://www.baeldung.com/java-9-logging-api guide, but got stuck.
for simplicity, my structure is ->
build.gradle
/application-module
build.gradle (combines 3 other modules)
/src /...
/rest-module
build.gradle
/src /...
/service-module
build.gradle
/src /...
/persistency-module
build.gradle
/src /...
So, I want to add one more module
/log-module
/src -> with actual classes
module-info.java
Slf4jLogger implements System.Logger
Slf4jLoggerFinder extends System.LoggerFinder
and include it into my application-module
but when trying to build it all, I get 'error: module not found: org.slf4j', and app is not build.
So, what am I doing wrong? What additional plugins/config do I need? And will it even allow me to achieve my goal?

Okay, I managed to find the solution. It's a combination of
https://www.baeldung.com/java-9-logging-api
https://www.baeldung.com/java-spi
So, I don't even needed jigsaw modules - only the JDK's service provider mechanism
So, in fact you need 3 files [these are id's of pastebin's samples; but they are almost the same as in the java-9-logging-api article]
AkXY3zgu -> adapter class
YFUkZwat -> logger provider
CD6NNibj -> meta-inf file - and that's the trickiest part (with file name :) )
file name -> META_INF/services/java.lang.System$LoggerFinder
com.my-projects.commons.logs.Slf4jLoggerFinder
An now on regular app startup system logger will be replaced with slf4j-adapter.
But still, check how system logger is created -> for example, I mostly need SSLLogger, and there is some system-prop-based logic there...

Overriden application.yml in multi-module Spring Boot app

There is a Spring Boot 2 app with such a structure:
parent-module
module-1
src
main
java
resources
- application.yml
module-2
src
main
java
resources
- application.yml
Also, module-1 depends on module-2, specified in pom.xml dependencies section.
The problem is that when I specify some properties in module-2's application.yml - they are not visible in main module-1's components (via #Value annotation).
As was answered here seems like module-1's application.yml overrides module-2's application.yml. There is a workaround - if I use name application.yaml in module-2 everything works fine, but I'm going to add more modules and, finally, it's dirty hack.
What I'm doing wrong? Should such an hierarchy of property files specified somehow?
I will be happy to provide more details if it's needed.
Thank you!

Spring Boot is a runtime framework. I understand that your modules are not spring-boot applications by themselves (you can't make a dependency on a spring boot application packaged with spring boot maven plugin, because it produces an artifact that is not really a JAR from the Java's standpoint although it does have *.jar extension).
If so, they're probably regular jars. So you should have a "special" module that assembles the application. This special module lists both 'module1' and 'module2' in <dependency> section and should contain a definition of spring-boot-maven-plugin in its build section (assuming you're using maven). But if so you shouldn't really have more than one application.yml - it will be misleading. Instead, put the application.yml to the src/main/resources of that "special" module.
If you really have to for whatever reason work with multiple application.yaml files, make sure you've read this thread

I know, this is already a well-aged post.
I just came accross the same issue and the best solution I found was to import the module-specific configurations with the spring.config.import directive as described here.
In this case you still have your module specific configuration in property or yaml files within that specific module and do not have too much unwanted dependencies in your project setup.

application.yml is, as the name indicates, an application-level file, not a module-level file.
It is the build script that assembles the final application, e.g. the .war file, that needs to include a application.yml file, if any.
If modules need properties, and cannot rely on the defaults, e.g. using the : syntax in #Value("${prop.name:default}"), they need to provide a module-level property file using #PropertySource("classpath:/path/to/module-2.properties").
Note: By default, #PropertySource doesn't load YAML files (see official documentation), but Spring Boot can be enhanced to support it. See #PropertySource with YAML Files in Spring Boot | Bealdung.
Alternative: Have the application-level build script (the one building the .war file) merge multiple module-level build scripts into a unified application.yml file.

Log4j2 custom plugins not working in EAR

I am migrating an EAR application from Log4J 1.2.17 to Log4J2 2.4. Please find below the EAR structure.
EAR
-- APPLICATION JAR 1 (contains custom plugin)
-- APPLICATION JAR 2
-- APPLICATION JAR 3 (contains custom plugin)
-- APPLICATION JAR 4
-- APPLICATION WAR 1
-- APPLICATION WAR 2
-- APPLICATION WAR 3
-- OTHER THIRD PARTY APIs
-- lib/log4j-api-2.4.jar
-- lib/log4j-core-2.4.jar
-- lib/log4j-jcl-2.4.jar
-- lib/log4j-web-2.4.1.jar
-- META-INF/log4j2.xml
-- META-INF/MANIFEST.MF (contains all jars in class-path entry)
Custom plugin classes in all the jars are in the same package - com.test.it.logging.
PFB the initialization code.
Adding the custom plugins package.
PluginManager.addPackage("com.test,it.logging");
Initializing the logging configuration using log4j2.xml.
String path = "path/log4j2.xml";
System.setProperty("log4j.configurationFile", path);
None of the defined custom plugins are getting detected and I tried all the combinations available to initialize log4j2.xml and plugins initialization but nothing worked.
It gives me a feel that custom plugins is not at all working in EAR as I tried all the permutations and combinations. is this a BUG in log4j2 (version: 2.4) ? If no, then please guide me about how to define logging configuration containing custom plugins in an EAR containing custom plugins that are scattered across many jars within an EAR ?
Can anyone please let me know about how to configure
Also, PFB my question posted in stackoverflow on the same.
Custom plugin not getting detected in EAR with log4j2 API
I am using Wildfly 8.2.0-Final AS and maven for building EAR.
Just adding a note that I am always finding Log4JPlugins.dat file inside Jars containing custom plugins irrespective of the options I try regarding detecting plugins.
Your response is highly important to me and thanks.

I don't believe the log4j classes have visibility into the classloaers for the war and application jars.

When compiling a custom Plugin, the Log4J pom.xml defines a plugin that automatically generates cache data in the file META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
You can see this under your target/classes in a Maven project.
The log4j-core-2.x.x.jar also contains a Log4j2Plugins.dat defining its cache data.
The problem is a single JAR is created when testing an EAR using ShrinkWrap and normally the log4j-core-2.x.x.jar Log4j2Plugins.dat is added to the test JAR as it would most likely be first in the class path.
This means your custom plugin cache is missing.
The solution using ShrinkWrap is to create a new Log4j2Plugins.dat merging any required custom plugin cache files with the cores and then adding that to the JAR.
The following function achieves that...
private static void mergeLog4J2Log4j2PluginsFile(JavaArchive ja, Class... uniqueJARClasses) {
// #Author: Johnathan Ingram <jingram#rogueware.org>
// Log4J2 uses /META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat within a JAR to define custom plugins
// This is automatically generated by the plugin defined in the log4j-core-2.x.x pom.xml when compiling your custom plugin
// The problem with shrinkwrap is that the JAR is not preserved and only a single
// /META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
// file can exist as JAR files cannot be added to a JAR file as a library.
// This is normally the default contained in log4j-core-2.x.x.jar which does not expose any custom plugins
// To rectify, both the core and the custom plugin JAR file Log4j2Plugins.dat need to be merged into a single Log4j2Plugins.dat
try {
// List of a unique class in each JAR containing a Log4j2Plugins.dat requiring merging
Vector<URL> datUrls = new Vector<URL>();
for (Class klass : uniqueJARClasses) {
// Find the JAR the class belongs to
URL classLoc = klass.getProtectionDomain().getCodeSource().getLocation();
URL resourceURL = classLoc.toString().endsWith(".jar")
? new URL("jar:" + URLDecoder.decode(classLoc.toString(), "UTF-8") + "!/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat")
: new URL(URLDecoder.decode(classLoc.toString(), "UTF-8") + "/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");
datUrls.add(resourceURL);
}
// Use the Log4J2 PluginCache to build a merged Log4j2Plugins.dat
File mergedDatFile = new File("target/Log4j2Plugins.dat");
try (FileOutputStream fo = new FileOutputStream(mergedDatFile)) {
org.apache.logging.log4j.core.config.plugins.processor.PluginCache pc = new org.apache.logging.log4j.core.config.plugins.processor.PluginCache();
pc.loadCacheFiles(datUrls.elements());
pc.writeCache(fo);
}
// Replace the default Log4j2Plugins.dat if present
ja.delete("/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");
ja.addAsManifestResource(mergedDatFile, "org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat");
} catch (Exception ex) {
ex.printStackTrace(System.err);
}
}
To run:
JavaArchive ja = ShrinkWrap.create(JavaArchive.class, "my-test.jar");
...
mergeLog4J2Log4j2PluginsFile(ja, org.apache.logging.log4j.core.config.plugins.processor.PluginCache.class, MyCustomPlugin.class);

How to provide a log4j.properties to an OSGI Bundle (Eclipse-Plugin)?

Project setup:
Logging-1.0.jar
contains a Logger.class which uses slf4j/log4j
depends on slf4j-api.jar, slf4j-log4j.jar, log4j.jar
LoggingOSGI-1.0.jar
wraps the logging project
contains an Activator and MANIFEST.MF
lib/ contains logging-1.0.jar, slf4j-api.jar, slf4j-log4j.jar, log4j.jar
jars from lib/ are added to classpath and packages from logging-1.0.jar are exported
SomeBundle-1.2.jar
contains an Activator and MANIFEST.MF
has a dependency on LoggingOSGI-1.0.jar
Accessing the Logger class from SomeBundle works, but the logging project can't find the log4j.properties (log4j:WARN No appenders could be found for logger).
Questions:
Where do i have to place the log4j.properties?
Any ideas what i could try? (already tried: different directories, Eclipse-Buddies, -Dlog4j.configuration as VM argument)
Would be an extension point, which tells the logging project the location of the log4j.properties, a good solution?

When I last tried this around six years ago, the solution turned to be to create a fragment bundle with the log4j.properties file, and then to attach that fragment (via the Fragment-Host manifest header) to the bundle that loads the logging library ("Logging-1.0.jar," in your case). It felt like a lot of project structure, build time, and deployment overhead for what seems like such a simple goal.
See section 3.14 of the OSGi Service Platform Core Specification for more detail on fragment bundles.
An alternate idea is to consider using the Configuration Admin Service to designate the path to a logging configuration file on disk, outside of your bundles. That would require augmenting your logging library to look up a configuration (or, better, listen for one) and then pass that configuration through to the logging implementation.
I would also be remiss to not point out the OSGi Log Service, specified in section 101 of the OSGi Service Platform Service Compendium.

To solve my problem i added this code to the Activator of the LoggingOSGI-1.0 which configures log4j. The file path is taken from a System property: -Dlog4j.configuration=path/to/log4j.properties.
Still interested in other approaches or opinions to this solution.
private static final String LOG4J_CONFIG_KEY = "log4j.configuration";
public void start(BundleContext bundleContext) throws Exception {
Activator.context = bundleContext;
if (System.getProperties().containsKey(LOG4J_CONFIG_KEY)) {
String file = System.getProperties().getProperty(LOG4J_CONFIG_KEY);
PropertyConfigurator.configure(file);
}
}