We Keep Coding

Unable to look at cache metrics(hit/miss/size) in spring boot?

I have implemented two cacheManagers.
One using caffiene and one using redis.
I have exposed them as beans and they are working as expected of them.
There isn't any cache endpoint in the list available at /actuator/metrics path either. I was able to only load /actuator/caches and /actuator/caches/{cacheName} endpoint. These endpoint only show the name and class of the cache being used. I am unable to see any metrics related to them.
I am using springboot 2.1.3 and spring-boot-actuator.
#Bean
public CacheManager caffeine() {
CaffeineCacheManager caffeineCacheManager = new CaffeineCacheManager();
caffeineCacheManager.setCaffeine(caffeineCacheBuilder());
return caffeineCacheManager;
}
private Caffeine<Object, Object> caffeineCacheBuilder() {
return Caffeine.newBuilder().
initialCapacity(initialCapacity).
maximumSize(maxCapacity).
expireAfterAccess(Duration.parse(ttl)).
recordStats();
}

Turned out #Cacheable makes the cache dynamically at runtime, thus if we want the annotated caches to have metrics, we have to set the cache names explicitly in the cacheManagerBean which inturn makes the cacheManager static.
Or create a custom registryBinder as stated in after upgrade to Spring Boot 2, how to expose cache metrics to prometheus?
and call the register method only after the cache is created.

#Cacheable doesn't intercept the method, cache is always empty

I have a method as following:
#Cacheable(value = "SAMPLE")
public List<SomeObj> find() {
// Method that initiates and returns the List<SomeObj> and takes around 2-3 seconds, does some logging too
}
And I am enabling caching in one of my configuration classes:
#EnableCaching
#Configuration
public SomeConf extends CachingConfigurerSupport {
// Here I also initialize my classes with #Cacheable annotation
#Bean
#Override
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList((new ConcurrentMapCache("SAMPLE"))));
return cacheManager;
}
#Bean
#Override
public CacheResolver cacheResolver() {
return new SimpleCacheResolver(cacheManager());
}
#Bean
#Override
public KeyGenerator keyGenerator() {
return new SimpleKeyGenerator();
}
}
I have the following in my pom.xml:
<!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
<version>1.5.14.RELEASE</version>
</dependency>
I am declaring a CacheManager as follows:
#Bean
public CacheManager cacheManager(){
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList((new ConcurrentMapCache("SAMPLE"))));
return cacheManager;
}
When I get an #Autowired CacheManager instance into one of my #Services I can see that there exists a cache with name "SAMPLE", but its entries are always empty. I call again and again the method find(), but it doesn't seem to populate the cache.
I have tried to put an argument (say int a) to the find() method and put it as key = "#a" to #Cacheable, but nothing changed.
When I try to recreate the issue in an isolated environment, I can see that it works properly. But when I add my dependencies (non open-source company libraries, which include an EhCache configuration as well) it doesn't work. How can I debug this, what am I doing wrong?
Update:
I have also tried to use cacheManager = myCacheManager in #Cacheable as well. No luck.
Update 2:
I am using AspectJ and Spring AOP. I think that it may have something to do with it. I have tried #EnableCaching(mode = AdviceMode.ASPECTJ) with #EnableLoadTimeWeaving but same thing.
Update 3:
I was finally able to reproduce the issue, here it is: client-api-cache
Basically when you run the application and telnet localhost 9000 after you send any line to it, it should print NOT CACHED once even though the method is called twice in CachedController (Second coming from the cache). But it prints twice.

The root cause is that you are misusing "afterPropertiesSet". So, what you are doing is endless loop and never pass control back to the Spring pipeline, so Spring isn't able to init caching facility properly.
Check out code which fixes our problem: https://dumpz.org/cbx8h28KeAss

As Andrews S noted, this does sound like colliding caches in the auto configuration.
#EnableCaching's javadoc has some useful notes regarding how the cacheManager is selected and, specifically, an idea on how to proceed. What you'd do in this case is establish a CachingConfigurer to select your cache - perhaps you can just extend CachingConfigurerSupport (as in example below) and be done.
A bean of type CacheManager must be registered, as there is no reasonable default that the framework can use as a convention. And whereas the <cache:annotation-driven> element assumes a bean named "cacheManager", #EnableCaching searches for a cache manager bean by type. Therefore, naming of the cache manager bean method is not significant.
For those that wish to establish a more direct relationship between #EnableCaching and the exact cache manager bean to be used, the CachingConfigurer callback interface may be implemented. Notice the #Override-annotated methods below:
#Configuration
#EnableCaching
public class AppConfig extends CachingConfigurerSupport {
#Bean
public MyService myService() {
// configure and return a class having #Cacheable methods
return new MyService();
}
#Bean
#Override
public CacheManager cacheManager() {
// configure and return an implementation of Spring's CacheManager SPI
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Arrays.asList(new ConcurrentMapCache("default")));
return cacheManager;
}
#Bean
#Override
public KeyGenerator keyGenerator() {
// configure and return an implementation of Spring's KeyGenerator SPI
return new MyKeyGenerator();
}
}
This approach may be desirable simply because it is more explicit, or it may be necessary in order to distinguish between two CacheManager beans present in the same container.
Notice also the keyGenerator method in the example above. This allows for customizing the strategy for cache key generation, per Spring's KeyGenerator SPI. Normally, #EnableCaching will configure Spring's SimpleKeyGenerator for this purpose, but when implementing CachingConfigurer, a key generator must be provided explicitly. Return null or new SimpleKeyGenerator() from this method if no customization is necessary.
CachingConfigurer offers additional customization options: it is recommended to extend from CachingConfigurerSupport that provides a default implementation for all methods which can be useful if you do not need to customize everything. See CachingConfigurer Javadoc for further details.
You may also find this thread useful: How to have multiple cache manager configuration in spring cache java
Edit:
So thankfully I have a project using caching and wrote this little bit:
#Bean
#Override
public CacheResolver cacheResolver() {
return new SimpleCacheResolver(cacheManager()) {
#Override
protected Collection<String> getCacheNames(CacheOperationInvocationContext<?> context) {
Collection<String> toReturn = super.getCacheNames(context);
toReturn.forEach(System.out::println);
return toReturn;
}
#Override
public Collection<? extends Cache> resolveCaches(CacheOperationInvocationContext<?> context) {
System.out.println(Arrays.toString(context.getArgs()));
System.out.println(context.getClass());
System.out.println(context.getMethod());
System.out.println(context.getOperation());
return super.resolveCaches(context);
}
};
}
Aside from seeing my established cache names pop out, I do note that context is outputting:
[]
class org.springframework.cache.interceptor.CacheAspectSupport$CacheOperationContext
public abstract ...transferobjects.CacheContainer ...service.LookupService.getCacheSelections()
Builder[public ...transferobjects.CacheContainer ...dao.LookupFacade.getCacheSelections()] caches=[cacheselections] | key='' | keyGenerator='' | cacheManager='' | cacheResolver='' | condition='' | unless='' | sync='true'
The context.getClass() output is of interest, given your Aspect question. That said I have a very similar logging/timing aspect in my own code that isn't causing any confusion for the rest of the caching. Try my resolver out and see if the output is instructive on what calls are being made against the cache code.
Edit #2:
The TLDR issue appears to be that we're expecting #Cacheable to work in places where it can't - mainly due to the fact that the framework hasn't fully brought itself up yet. You are using InitializingBean and I tried replacing that functionality with #PostConstruct which both fail.
Spring cache using #Cacheable during #PostConstruct does not work
I got your github code. I initially ran into the blocking issue you reported in the other thread but, in the interest of pursuing one thing at a time, I just commented out that blocking code and called service.cachedMethod() directly.
I ran your jar file with --trace and noted that the cacheManager was scanned and created, but it wasn't initially obvious to me when it was being invoked. So, no big deal, I just had the bean method throw a RuntimeException. Upon doing that I noted that both your NOT CACHED lines print prior to that invocation - another hint that things aren't set up as expected.
Finally, I added System.out.println(service.toString()); in your Controller's afterPropertiesSet() method and saw com.company.client.api.domain.CachedServiceImpl#2bbfb8b - your object, not a proxied object. Thus no caching.
So, in general, you'll need to rework how you're trying to enable this functionality.

Make sure you are setting your cache into cachemanager as below.
#EnableCaching
#Configuration
public SomeConf {
#Bean
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Arrays.asList(new ConcurrentMapCache("find():List<SomeObj>")));
return cacheManager;
}
}

How to enable ehCache (3.x) in SpringMVC (5.x) project

In our project we use the ehcache 2.x and now we want to migrate to 3.x but somehow I fail tremendously. In 2.x we have the configuration in ehcache.xml. To what I read is that I need to change the content as well when using 3.x.
But first of all I have no clue how to wire the Cache by itself. I have
#EnableCaching
#Configuration
public class CacheConf {
#Bean
public CacheManager cacheManager() {
final CacheManagerBuilder<org.ehcache.CacheManager> ret = CacheManagerBuilder.newCacheManagerBuilder();
ret.withCache("properties", getPropCache());
ret.withCache("propertyTypes", getPropCache());
return (CacheManager) ret.build(true);
}
....
But this configuration I get a java.lang.ClassCastException: org.ehcache.core.EhcacheManager cannot be cast to org.springframework.cache.CacheManager which means that I cannot set the ehcache properly.
Note: In 2.x we configured different types of caches, more or less due to the expiration period - and since other object types are stored therein.
Hints to get it running, cause I didn't find a working sample with Spring 5 and ehcache 3.x? Best would be a config without xml!

The Ehcache CacheManager isn't a Spring CacheManager. This is why you get the ClassCastException.
Ehcache 3 is JSR107 (JCache) compliant. So Spring will connect to it using that.
You will find an example here and a simpler one in Spring Boot petclinic.
If I take your example, you will do something like
#EnableCaching
#Configuration
public class CacheConf {
#Bean
public CacheManager cacheManager() {
EhcacheCachingProvider provider = (EhcacheCachingProvider) Caching.getCachingProvider();
Map<String, CacheConfiguration<?, ?>> caches = new HashMap<>();
caches.put("properties", getPropCache());
caches.put("propertyTypes", getPropCache());
DefaultConfiguration configuration = new DefaultConfiguration(caches, provider.getDefaultClassLoader());
return new JCacheCacheManager(provider.getCacheManager(provider.getDefaultURI(), configuration));
}
private CacheConfiguration<?, ?> getPropCache() {
// access to the heap() could be done directly cause this returns what is required!
final ResourcePoolsBuilder res = ResourcePoolsBuilder.heap(1000);
// Spring does not allow anything else than Objects...
final CacheConfigurationBuilder<Object, Object> newCacheConfigurationBuilder = CacheConfigurationBuilder
.newCacheConfigurationBuilder(Object.class, Object.class, res);
return newCacheConfigurationBuilder.build();
}
}

Spring two different application context - property placeholder collision

I have been created an SDK using Spring framework that will be used
to integrate with a REST backend, to make use of the dependency injection.
In this SDK, I have MapPropertySources to handle PropertyPlaceHolders.
Basically I register programmatically some properties there which I want
to be resolved within the SDK with #Value annotation.
It works fine within the SDK, but when I build the SDK (using the builder)
inside a Spring-boot app, the properties from MapPropertiesPlaceHolder
are not resolved any longer.
I have this piece of code from the builder class:
public MyRepository build() {
AnnotationConfigApplicationContext context =
new AnnotationConfigApplicationContext();
StandardEnvironment env = new StandardEnvironment();
context.setEnvironment(env);
Map<String, Object> propertiesMap = new HashMap<>();
propertiesMap.put("baseUrl", baseUrl);
MutablePropertySources mutablePropertySources = context.getEnvironment().getPropertySources();
mutablePropertySources.addFirst(new MapPropertySource("customPropertiesMap", propertiesMap));
if(jerseyClient == null){
jerseyClient = JerseyClientBuilder.createClient();
}
context.getBeanFactory().registerSingleton("jerseyClient", jerseyClient);
context.setParent(null);
context.register(MySdk.class);
context.refresh();
MySdk mySdk = new MySSdk(context);
return mySdk;
}
This is the way I instantiate the SDK and I create a new
Spring context inside it.
The problem is that the properties within the MapPropertySource
are not resolved when I use the SDK as a maven dependency in another
spring-boot application. It might have anything to do with the parent
context? The properties are just not resolved... Where should I investigate?
Long problem short, I have the #Value('${baseUrl}) being resolved within the SDK's tests, but when I include this SDK in another spring-boot application, it will not be resolved anylonger. Why?
Edit:
MySdk class looks like this:
#ComponentScan
#Service
#PropertySource("classpath:application.properties")
public class DeviceRepository {
private ApplicationContext context;
public MySdk(){
}
public MySdk(ApplicationContext context) {
this.context = context;
}
// other methods that calls beans from context like
// this.context.getBean(MyBean.class).doSomething()
Everything works fine in the tests from within the same SDK. The
baseUrl property is resolved fine, but when I wire this SDK in another
spring application, the properties passed in the builder, they are not
recognised by the #Value annotation.

Have you defined a PropertySourcesPlaceholderConfigurer bean in your Spring configuration? Whenever property resolution fails inside of an #Value annotation, this is one of the first things that comes to mind.
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
Link to Javadoc:
http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/context/support/PropertySourcesPlaceholderConfigurer.html

hmm just read something interesting about leveraging the java configuration with property placeholders. Check out this link #3. It looks like you have to register another bean.
http://www.baeldung.com/2012/02/06/properties-with-spring/
Also worth noting there is a whole scope discussion on child vs parent contexts. I don't believe you can have two contexts without creating this relationship.

Spring data multiple datasources from multiple files

I have 2 (or more) different configuration properties file located in the project and I want to load them for different datasources.
I tried to do as following:
#Bean
#ConfigurationProperties(locations = {"#{myconfigroot.getRootFolder()}/datasource1.properties"}
public static DataSource getFirstDatasource() {
return DataSourceBuilder.create().build();
}
But obviously this won't work as the ConfigurationProperties annotation locations property doesn't go through the spEL. (Or may be I write it wrong?) myconfigroot.getRootFolder() is a static method which returns the path to the datasource1.properties.
Please advice. Thanks.
===== Edited =======
I believe this is a common problem when somebody want their application want to load different configuration files. Due to some reasons the file location and name can't be put in the startup script or command line, or, the path can only be determined in runtime, that would require spring to load them during the bean creation.
I tried once using PropertySourcePlaceHolderConfigurer but seems not work either.
Anybody can share some lights?

Latest Spring boot (version 1.3.5) doesn’t support SpEL in this case.
See JavaDoc of annotation #ConfigurationProperties
Note that contrary to {#code #Value}, SpEL expressions are not
evaluated since property values are externalized.
I found a way to customize Spring boot default behavior as follows:
For example, I have database.properties file in somewhere, for some reason I cannot get the location before runtime.
username=mike
password=password
Accordingly, define POJO mapping to properties:
#Component
#ConfigurationProperties(locations = "myConfiguration")// myConfiguration is customized placeholder
public class MyProperties{
String username;
String password;
//Getters, Setters…
}
Then, to extend default StandardEnvironment:
public class MyEnvironment extends StandardEnvironment {
#Override
public String resolvePlaceholders(String location) {
if (location.equals("myConfiguration")) {
//Whatever you can do, SpEL, method call...
//Return database.properties path at runtime in this case
return getRootFolder() + "datasource.properties";
} else {
return super.resolvePlaceholders(text);
}
}
}
Last, apply it in Spring boot main method entry:
#SpringBootApplication
public class MyApplication extends SpringBootServletInitializer {
public static void main(String[] args) {
new SpeedRestApplication()
.configure(new SpringApplicationBuilder(SpeedRestApplication.class).environment(new MyEnvironment()))//Replace default StandardEnvironment
.run(args);
}
}
Once Spring boot starts up, the MyProperties bean name and password fields are injected from database.properties. Then you could wire the MyProperties bean to other beans as configuration.
Hope it helps!

I finally got it work by using the following mechanism:
public class DatasourcePostProcessor implements EnvironmentPostProcessor {
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Properties p = new Properties();
p.load(new FileInputStream(new File(getRootFolder() + "/datasource1.properties")));
Map<String, Object> propMap = new HashMap<>();
for (Map.Entry<Object, Object> entry : p.entrySet()) {
propMap.put(entry.getKey().toString(), entry.getValue());
}
MapPropertySource source = new MapPropertySource("datasource1", propMap);
environment.getPropertySources().addLast(source);
}
}
and register the environment post processor into the spring.factories:
org.springframework.boot.env.EnvironmentPostProcessor=com.myorg.test.DatasourcePostProcessor
Anyway, hope this helps people and accept the first anwer as it enlight me. Also post the following references from the google search that found during research:
Where I found how to wire the property source with the environment: https://github.com/spring-projects/spring-boot/issues/4595
Where I found how to load the customized properties file: How to configure a custom source to feed Spring Boot's #ConfigurationProperties