We Keep Coding

How do I replace the values of a YAML file with definitions in the same file? [duplicate]

We have a spring boot application with configuration being driven from application.yml file. In this configuration file we use the feature of defining a property by referring to another property inside the same application.yml file:
my-games-app:
base-property: foo
games:
- game-one:
game-name: ${my-games-app.base-property}one
game-location: ${my-games-app.base-property}/one
- game-two:
game-name: ${my-games-app.base-property}two
game-location: ${my-games-app.base-property}/two
And we have a #ConfigurationProperties bean loading games configuration:
#Configuration
#ConfigurationProperties(prefix = "my-games-app.games")
public class GamesConfig {
private Map<String, Game> games;
...
}
Useless to say the above is just an example, in reality it is a very complex setup with GamesConfig bean being used as a constructor argument for many other beans inside our application:
#Component
public class GamesRunner {
private final GamesConfig gamesConfig;
...
}
Everything works as expected. The problem we have is related to testing the beans where GamesConfig is injected; in the above example GamesRunner. At the moment we use #SpringBootTest to get hold of the beans we want to test. This again, works OK but the main inconvenient is that the whole application needs to be started in order to access the GamesConfig bean. This means setting up a lot of infrastructure such as a Database a JMS message broker and a Kafka broker. This takes time and makes our CI builds longer to run which started to become a bit of an inconvenient. Because the beans we want to test don't need any other setup than having the GamesConfig constructor argument provided we would prefer to have unit tests in place rather than integration tests as they are much faster to run.
In other words, we want to be able to recreate GamesConfig by hand by parsing our application.yml with a test helper method. To do this we use snakeyaml library:
public final class TestHelper {
public static GamesConfig getGamesConfig() {
var yaml = new Yaml();
var applicationYaml = (Map<String, Object>) yaml.load(readResourceAsString("application.yml");
return createGamesConfig(applicationYaml.get("games");
}
private static GamesConfig createGamesConfig(Object config) {
// The config Object passed here is a `Map<String, Map<String, String>>`
// as defeined in our `games` entry in our `application.yml`.
// The issue is that game name and game locations are loaded exactly like
// configured without property place holders being resolved
return gamesConfig;
}
}
We resolved the issue by manually parsing the property placeholders and looking up their values in the application.yml file. Even if our own property placeholder implementation is quite generic, my feeling is that this extra work is not needed as it should be a basic expectation the library would have some specific set up to do this out of the box. Being very new to snakeyaml I hope someone else hit the same problem and knows how to do it.
We use snakeyaml because it just happened to be in the class path as a transitive dependency, we are open to any suggestions that would achieve the same thing.
Thank you in advance.

To my knowledge, SnakeYAML only supports substitution of environment variables, which is why what you want is not possible as far as I know. What you can do instead, of course, is simply use Spring's classes without setting up a full ApplicationContext.
For example, assuming your game config from above, you could use:
final var loader = new YamlPropertySourceLoader();
final var sources = loader.load(
"games-config.yml",
new ClassPathResource("games-config.yml")
);
final var mutablePropertySources = new MutablePropertySources();
sources.forEach(mutablePropertySources::addFirst);
final var resolver = new PropertySourcesPropertyResolver(mutablePropertySources);
resolver.setIgnoreUnresolvableNestedPlaceholders(true);
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-location"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-location"));
which outputs:
fooone
foo/one
footwo
foo/two
If you are actually interested in how Spring does it, a good starting point is the source code of the PropertySourcesPlaceholderConfigurer class.

Accessing CDI from simple objects?

Assume I have a configuration class accessible via the stock CDI that defines some application-wide parameters:
#ApplicationScoped
class AppConfig {
public double getMaxAllowedBrightness() { ... }
};
And I have a simple class for my data objects:
class LightSource {
double brightness;
...
boolean isValid() {
double maxAllowedBrightness = ...; // Somehow use AppConfig#getMaxAllowedBrightness() here
return brightness <= maxAllowedBrightness;
}
}
How can my data object access the single AppConfig instance?
Somehow I hate the idea of autowiring AppConfig into every single data object (there are lots of them). Is there any other way to get access to AppConfig in the above example from my data object?
What's the best pattern to use here?

The simplest example is a runtime lookup akin to:
import jakarta.enterprise.inject.spi.CDI;
CDI.current().select(cls).get();
With cls being the class that you're looking up. (Note the package name, this is the latest version of CDI 2.x in the new jakarta namespace, the original is in javax.)
It gets more detailed from there, but that's the gist of it.
Note, that semantically there's little difference between autowiring something and doing a runtime lookup, especially for something mostly static at the instance level. It's still a dependency. You still have to touch the code of the classes to pull it off.
A nice thing of relying on the autowiring is that you can disable it situationally, and the class reverts to a simple bean, that you can do with what you will. Coding in the lookup, it's a little bit more than that.
Dynamic lookup is more for special circumstances.

On my current project, our team has been doing this using the #Value annotation. In our case, we have all the properties in a properties bean, which I'll call mainAppConfiguration. The bean is populated from a properties file like main-app-config.properties (which was read into the bean with a Properties prop = new Properties().load(mainAppConfigFilePath) method.
Assuming you have something like that set up, then we inject the properties into the classes that need them using a little SpEL magic something like:
private Integer refreshRateSeconds;
#Value("#{ mainAppConfiguration.getProperties()['funny-property-base-name.refreshRateSeconds'] }")
public void setRefreshRateSeconds(Integer refreshRateSeconds) {
if (refreshRateSeconds == null) {
throw new IllegalArgumentException("Required config property 'funny-property-base-name.refreshRateSeconds' was not found"));
}
this.refreshRateSeconds = refreshRateSeconds;
}
Baeldung has examples (without defaults) and more with defaults.

How to check if a Java interface has an implementation? [duplicate]

Can I do it with reflection or something like that?

I have been searching for a while and there seems to be different approaches, here is a summary:
reflections library is pretty popular if u don't mind adding the dependency. It would look like this:
Reflections reflections = new Reflections("firstdeveloper.examples.reflections");
Set<Class<? extends Pet>> classes = reflections.getSubTypesOf(Pet.class);
ServiceLoader (as per erickson answer) and it would look like this:
ServiceLoader<Pet> loader = ServiceLoader.load(Pet.class);
for (Pet implClass : loader) {
System.out.println(implClass.getClass().getSimpleName()); // prints Dog, Cat
}
Note that for this to work you need to define Petas a ServiceProviderInterface (SPI) and declare its implementations. you do that by creating a file in resources/META-INF/services with the name examples.reflections.Pet and declare all implementations of Pet in it
examples.reflections.Dog
examples.reflections.Cat
package-level annotation. here is an example:
Package[] packages = Package.getPackages();
for (Package p : packages) {
MyPackageAnnotation annotation = p.getAnnotation(MyPackageAnnotation.class);
if (annotation != null) {
Class<?>[] implementations = annotation.implementationsOfPet();
for (Class<?> impl : implementations) {
System.out.println(impl.getSimpleName());
}
}
}
and the annotation definition:
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.PACKAGE)
public #interface MyPackageAnnotation {
Class<?>[] implementationsOfPet() default {};
}
and you must declare the package-level annotation in a file named package-info.java inside that package. here are sample contents:
#MyPackageAnnotation(implementationsOfPet = {Dog.class, Cat.class})
package examples.reflections;
Note that only packages that are known to the ClassLoader at that time will be loaded by a call to Package.getPackages().
In addition, there are other approaches based on URLClassLoader that will always be limited to classes that have been already loaded, Unless you do a directory-based search.

What erickson said, but if you still want to do it then take a look at Reflections. From their page:
Using Reflections you can query your metadata for:
get all subtypes of some type
get all types annotated with some annotation
get all types annotated with some annotation, including annotation parameters matching
get all methods annotated with some

In general, it's expensive to do this. To use reflection, the class has to be loaded. If you want to load every class available on the classpath, that will take time and memory, and isn't recommended.
If you want to avoid this, you'd need to implement your own class file parser that operated more efficiently, instead of reflection. A byte code engineering library may help with this approach.
The Service Provider mechanism is the conventional means to enumerate implementations of a pluggable service, and has become more established with the introduction of Project Jigsaw (modules) in Java 9. Use the ServiceLoader in Java 6, or implement your own in earlier versions. I provided an example in another answer.

Spring has a pretty simple way to acheive this:
public interface ITask {
void doStuff();
}
#Component
public class MyTask implements ITask {
public void doStuff(){}
}
Then you can autowire a list of type ITask and Spring will populate it with all implementations:
#Service
public class TaskService {
#Autowired
private List<ITask> tasks;
}

The most robust mechanism for listing all classes that implement a given interface is currently ClassGraph, because it handles the widest possible array of classpath specification mechanisms, including the new JPMS module system. (I am the author.)
try (ScanResult scanResult = new ClassGraph().whitelistPackages("x.y.z")
.enableClassInfo().scan()) {
for (ClassInfo ci : scanResult.getClassesImplementing("x.y.z.SomeInterface")) {
foundImplementingClass(ci); // Do something with the ClassInfo object
}
}

With ClassGraph it's pretty simple:
Groovy code to find implementations of my.package.MyInterface:
#Grab('io.github.classgraph:classgraph:4.6.18')
import io.github.classgraph.*
new ClassGraph().enableClassInfo().scan().withCloseable { scanResult ->
scanResult.getClassesImplementing('my.package.MyInterface').findAll{!it.abstract}*.name
}

What erikson said is best. Here's a related question and answer thread - http://www.velocityreviews.com/forums/t137693-find-all-implementing-classes-in-classpath.html
The Apache BCEL library allows you to read classes without loading them. I believe it will be faster because you should be able to skip the verification step. The other problem with loading all classes using the classloader is that you will suffer a huge memory impact as well as inadvertently run any static code blocks which you probably do not want to do.
The Apache BCEL library link - http://jakarta.apache.org/bcel/

Yes, the first step is to identify "all" the classes that you cared about. If you already have this information, you can enumerate through each of them and use instanceof to validate the relationship. A related article is here: https://web.archive.org/web/20100226233915/www.javaworld.com/javaworld/javatips/jw-javatip113.html

Also, if you are writing an IDE plugin (where what you are trying to do is relatively common), then the IDE typically offers you more efficient ways to access the class hierarchy of the current state of the user code.

I ran into the same issue. My solution was to use reflection to examine all of the methods in an ObjectFactory class, eliminating those that were not createXXX() methods returning an instance of one of my bound POJOs. Each class so discovered is added to a Class[] array, which was then passed to the JAXBContext instantiation call. This performs well, needing only to load the ObjectFactory class, which was about to be needed anyway. I only need to maintain the ObjectFactory class, a task either performed by hand (in my case, because I started with POJOs and used schemagen), or can be generated as needed by xjc. Either way, it is performant, simple, and effective.

A new version of #kaybee99's answer, but now returning what the user asks: the implementations...
Spring has a pretty simple way to acheive this:
public interface ITask {
void doStuff();
default ITask getImplementation() {
return this;
}
}
#Component
public class MyTask implements ITask {
public void doStuff(){}
}
Then you can autowire a list of type ITask and Spring will populate it with all implementations:
#Service
public class TaskService {
#Autowired(required = false)
private List<ITask> tasks;
if ( tasks != null)
for (ITask<?> taskImpl: tasks) {
taskImpl.doStuff();
}
}

Alternative To Singleton Util Class

So I have a class like so:
public class HBaseUtil {
private final String fileName = "hbase.properties";
private Configuration config;
private HBaseUtil() {
try {
config = new PropertiesConfiguration(fileName);
} catch (ConfigurationException e) {
// some exception handling logging
}
}
// now some getters pulling data out of the config object
public static String getProperty(String fieldKeyName) {...}
public static String getColumnFamily(String fieldName) {...}
// ... some more getters
// NO setters (thus making this a read-only class)
}
Thus, basically I have for myself a Singleton class, that the very first time that it is put to use, sets up a configuration object, and then simply keeps listening for get calls. There are a number of problems with this class:
Unit testing the static methods within class HBaseUtil becomes difficult because of a tight-knit coupling between the Singleton and the configurations file.
What I really want is me being able to supply the filename/filename+path to the class so that it can go in there, read the configuration properties from that file and offer them to incoming read requests. One important note here though: I need this flexibility in specifying the properties file ONLY ONCE per JVM launch. So I certainly don't need to maintain state.
Here is what I was able to come up with:
Instead of a Singleton, I have a normal class with all static methods and no explicit constructor defined.
public class HBaseUtil {
// directly start with getters
public static String getProperty(Configuration config, String fieldKeyName) {...}
public static String getColumnFamily(Configuration config, String fieldKeyName) {...}
// ...and so on
}
And then, instead of using the class in my other code like such:
HBaseUtil.getProperty(String fieldKeyName)
I'd use it like so:
Configuration externalConfig = new PropertiesConfiguration("my-custom-hbase.properties");
HbaseUtil.getProperty(externalConfig, fieldKeyName)
My questions:
Am I even thinking in the right direction? My requirement is to have the flexibility in the class only ONCE per JVM. All that needs to be configurable in my project for this, is the location/contents of the HBase .properties file. I was thinking having a Singleton is overkill for this requirement.
What other better approaches are there for my requirement (stated in above point)?
Thanks!
Note: I've read this StackOverflow discussion, but now it's gotten me even more confused.

You should avoid all static methods and instead design a class which does not mandate its lifecycle: it can be a typical immutable POJO with a public constructor.
Then, when you need it as a singleton, use it as a singleton. For testing, use it in some other way.
Usually, dependency injection is the preferred avenue to solve these problems: instead of hard-coding a pulling mechanism for your configuration object, you have the object delivered to any class which needs it. Then you can decide late what bean you will deliver.
Since you are probably not using Spring (otherwise dependency injection would be your default), consider using Guice, which is a very lightweight and non-intrusive approach to dependency injection.

Spring dependency injection with static constructors

I have been using Guice for a few years now and just switched to a company that uses Spring. I am a fan of Dependency Injection but having a few issues figuring out how to get Spring to do what I want.
Here is what I have in the code right now (its not scala code, just shorter so using that syntax):
class A(b: B)
class B(exe: ExecutorService)
...
#Value("${search.threads}") var searchThreads: int
exe = Executors.newFixedThreadPool(searchThreads)
In guava I could use Named annotations to have different executors, or just use one executor for anyone that needed it and just define
final int searchThreads = readSearchThreadsFromConfigs()
bind(Executor.class).toInstance(Executors.newFixedThreadPool(searchThreads));
I am not sure how to set this up within Spring. It seems every example I see doesn't really cover generics, nor does it really go over static constructors or being able to "provide" the value.
What is the best way to get similar results to what I had above from Guice? Is there a notion of a "module" like guice and dagger use (other than the xml file, something statically checked)?
EDIT:
Here is a bit of the code currently used. It creates the executor within the constructor:
#Autowired
public LogsModule(#Value("${search.threads}") final int searchThreads) {
searchPool = Executors.newFixedThreadPool(searchThreads);
}

In Spring it's basically the same.
Your example can be rewritten as follows using #Configuration:
#Bean(value = "searchExecutor", destroyMethod = "shutdownNow")
public ExecutorService executorService(Environment env) {
final int searchThreads = env.getProperty("searchThreads", Integer.class, 3);
return Executors.newFixedThreadPool(searchThreads));
}
This example uses Environment - you can either add properties from your config to it, or use your config directly instead.
With XML configuration it would be more complex, but you can mix #Configuration with XML.
If you need multiple executors, you can use #Qualifier (or perhaps #Named) to distinguish between candidates by their bean names:
#Autowired
public LogsModule(#Qualifier("searchExecutor") ExecutorService e) { ... }