This question is to validate an observed behavior to ensure Guava Cache is used in correct way.
I have setup two Guava Caches (see code below): with and without a builder - as Guava documentation states:
Caches built with CacheBuilder do not perform cleanup and evict values
"automatically," or instantly after a value expires, or anything of
the sort.
It appears that expiration is only observed if getIfPresent() method is used, i.e. when a key is queried then value of null is returned after a period of time > expiry interval passes upon key/value is written to the cache. In case of Cache built with CacheLoader using get() or getUnchecked() results in CacheLoader.load() method to be executed thus expiry is not observed i.e. null value is never returned.
Is this the correct expectation?
Thank you for your patience and help.
// excerpt from test code
private static final FakeTicker fakeTicker = new FakeTicker();
private static LoadingCache<Integer, String> usingCacheLoader = CacheBuilder.newBuilder()
.expireAfterWrite(2, TimeUnit.MINUTES)
.ticker(fakeTicker)
.build(new CacheLoader<Integer, String>() {
public String load(Integer keyName) throws Exception {
logger.info("Getting value for key: {}", keyName);
return getValue(keyName, "with_cache_loader");
}
});
private static Cache<Integer, String> withoutCacheLoader = CacheBuilder.newBuilder()
.expireAfterWrite(2, TimeUnit.MINUTES)
.ticker(fakeTicker)
.build();
It is true that if you call get or getUnchecked you will never get null.
The expiration can be "observed" both in terms of performance - how long it takes to get for a specific key and whether it has to be freshly computed - and in whether the actual value you get reflects perhaps out of date information.
Related
I'm having an issue with the unless condition of the Cacheable annotation.
From the documentation, I understand that the unless condition is verified after the method being annotated is called and the value the method returns is cached (and actually returned) only if the unless condition is not met. Otherwise, the cached value should be returned.
Firstly, is this assumption true?
EDIT:
[From the Spring documentation] As the name implies, #Cacheable is used
to demarcate methods that are cacheable - that is, methods for whom
the result is stored into the cache so on subsequent invocations (with
the same arguments), the value in the cache is returned without having
to actually execute the method.
[My understanding] So for a given key, the method will be always executed until the unless condition is not met once. Then the cached value will be
returned for all subsequent calls to the method.
To illustrate my issue, I tried to break down my code into four simples classes:
1) DummyObject that represents instances to be cached and retrieved. It's a timestamp wrapper to show what is last value that has been cached. The toBeCached boolean is a flag that should be checked in the unless condition to know if the instance returned should be cached or not.
2) DummyDAO that returns DummyObject instances based on provided keys. Upon retrieval of an instance, the DAO checks when was the last value retrieved and verified if it should be cached or not (independently of what key is provided. Doesn't matter if this logic is "broken" as I'm always using the same key for my example). The DAO then marks the instance returned with the flag toBeCached. If the value is marked to be cached, the DAO actually updates its lastRetrieved timestamp as the instance should be eventually cached by the CachedDAO (because the unless condition won't be met).
3) DummyCachedDao that calls the DummyDAO to retrieve instances of DummyObject. If instances are marked toBeCached, it should cache the newly returned value. It should return the previously cached value otherwise.
4) The Application that retrieves a value (that will be cached), sleeps for a short time (not long enough for the cache duration to pass), retrieves a value (that should be the cached one), sleeps again (long enough for cache duration to pass), retrieves again a value (that should be a new value to be cached).
Unfortunately, this code does not work as expected as the retrieved value is always the original value that has been cached.
To ensure that the logic worked as expected, I checked if the unless conditions are met or not by replacing the retrieveTimestamp by retrieveTimestampBypass in the Application class. Since internal calls bypass the Spring proxy, the retrieveTimestamp method and whatever breakpoints or logs I put in are actually caught/shown.
What would cause the value to never be cached again? Does the cache need to be clean from previous values first?
public class DummyObject
{
private long timestamp;
private boolean toBeCached;
public DummyObject(long timestamp, boolean toBeCached)
{
this.timestamp = timestamp;
this.toBeCached = toBeCached;
}
public long getTimestamp()
{
return timestamp;
}
public boolean isToBeCached()
{
return toBeCached;
}
}
#Service
public class DummyDAO
{
private long cacheDuration = 3000;
private long lastRetrieved;
public DummyObject retrieveTimestamp(String key)
{
long renewalTime = lastRetrieved + cacheDuration;
long time = System.currentTimeMillis();
boolean markedToBeCached = renewalTime < time;
System.out.println(renewalTime + " < " + time + " = " + markedToBeCached);
if(markedToBeCached)
{
lastRetrieved = time;
}
return new DummyObject(time, markedToBeCached);
}
}
#Service
public class DummyCachedDAO
{
#Autowired
private DummyDAO dao;
// to check the flow.
public DummyObject retrieveTimestampBypass(String key)
{
return retrieveTimestamp(key);
}
#Cacheable(cacheNames = "timestamps", unless = "#result.isToBeCached() != true")
public DummyObject retrieveTimestamp(String key)
{
return dao.retrieveTimestamp(key);
}
}
#SpringBootApplication
#EnableCaching
public class Application
{
public final static String KEY = "cache";
public final static String MESSAGE = "Cached timestamp is: %s [%s]";
public static void main(String[] args) throws InterruptedException
{
SpringApplication app = new SpringApplication(Application.class);
ApplicationContext context = app.run(args);
DummyCachedDAO cache = (DummyCachedDAO) context.getBean(DummyCachedDAO.class);
// new value
long value = cache.retrieveTimestamp(KEY).getTimestamp();
System.out.println(String.format(MESSAGE, value, new Date(value)));
Thread.sleep(1000);
// expecting same value
value = cache.retrieveTimestamp(KEY).getTimestamp();
System.out.println(String.format(MESSAGE, value, new Date(value));
Thread.sleep(5000);
// expecting new value
value = cache.retrieveTimestamp(KEY).getTimestamp();
System.out.println(String.format(MESSAGE, value, new Date(value));
SpringApplication.exit(context, () -> 0);
}
}
There are so many details and maybe issues here but first of all you should remove
private long lastRetrieved;
from DummyDao class.
DummyDao is a singleton instance lastRetrieved field is not thread safe.
As you can also see from the logs after you cache the item first time it will always be retrieved from there as it has cached in the first call.
Otherwise you should have seen following log
3000 < 1590064933733 = true
The problem is actually quite simple.
There is no solution to my problem and rightfully so.
The original assumption I had was that "the unless condition is verified every time after the method being annotated is called and the value the method returns is cached (and actually returned) only if the unless condition is not met. Otherwise, the cached value should be returned."
However, this was not the actual behavior because as the documentation states, "#Cacheable is used to demarcate methods that are cacheable - that is, methods for whom the result is stored into the cache so on subsequent invocations (with the same arguments), the value in the cache is returned without having to actually execute the method."
So for a given key, the method will be always executed until the unless condition is not met once. Then the cached value will be returned for all subsequent calls to the method.
So I tried to approach the problem in a different way for my experiment, by trying to use a combination of annotations (#Caching with #Cacheable and #CachePut, although the documentation advises against it).
The value that I was retrieving was always the new one while the one in the cache was always the expected one. (*)
That's when I tilted THAT I couldn't upload the value in the cache based on an internal timestamp that would have been generated in the method that is being cached AND retrieving at the same the cached value if the unless condition was met or the new one otherwise.
What would be the point of executing the method every single time to compute the latest value but returning the cached one (because of the unless condition I was setting)? There is no point...
What I wanted to achieve (update the cache if a period expired) would have been possible if the condition of cacheable was to specify when to retrieve the cached version or retrieve/generate a new one. As far as I am aware, the cacheable is only to specify when a method needs to be cached in the first place.
That is the end of my experiment. The need to test this arose when I came across an issue with an actual production project that used an internal timestamp with this unless condition.
FYI, the most obvious solution to this problem is to use a cache provider that actually provides TTL capabilities.
(*) PS: I also tried few #caching combinations of #CacheEvict (with condition="#root.target.myNewExpiringCheckMethod()==true") and #Cacheable but it failed as well as the CacheEvict enforce the execution of the annotated method.
New to Caffeine and I'm clearly missing something super fundamental. All the basic usage examples I see look like this:
LoadingCache<String,Fizzbuzz> fizzbuzzes = Caffeine.newBuilder()
.maximumSize(100)
.expireAfterWrite(10, TimeUnit.DAYS)
.build(k -> fetchFizzbuzzes());
What I'm struggling with is the role that the fetchFizzbuzzes() function plays:
Is it used to populate the initial cache?
Is it used as some sort of fallback in case a key doesn't exist in the cache?
Something else?
Actually this is the most important part of the builder. The function passed to the build(CacheLoader) method takes a key and computes the value for this key. This function is called if there is currently no value for this key in the cache. The computed value will be added to the cache afterwards. There is a build() method without arguments as well, which can be used if you want to manually check for elements to be present in the cache and add them manually as well.
Your example however doesn't make too much sense as the fetchFizzbuzzes() method has no arguments. That is - without side effects - it will probably return the same value for all keys k.
Take the examples below:
LoadingCache<String,Fizzbuzz> fizzbuzzes = Caffeine.newBuilder()
.maximumSize(100)
.expireAfterWrite(10, TimeUnit.DAYS)
.build(key -> fetchFizzbuzzes(key));
fizzbuzzes.get("myKey"); // will automatically invoke fetchFizzbuzzes with 'myKey' as argument
fizzbuzzes.get("myKey"); // fetchFizzbuzzes won't be called as return value from previous call is added to the cache automatically
Cache<String, Fizzbuzz> fizzbuzzesManual = Caffeine.newBuilder()
.maximumSize(100)
.expireAfterWrite(10, TimeUnit.DAYS)
.build();
fizzbuzzesManual.getIfPresent("myKey"); // will return null as no element for 'myKey' was added to the cache before
See the Caffeine wiki for additional details.
I'm trying to write a AsyncLoadingCache that accepts a CacheWriter and I'm getting an IllegalStateException.
Here's my code:
CacheWriter<String, UUID> cacheWriter = new CacheWriter<String, UUID>() {
#Override
public void write(String key, UUID value) {
}
#Override
public void delete(String key, UUID value, RemovalCause cause) {
}
};
AsyncLoadingCache<String, UUID> asyncCache = Caffeine.newBuilder()
.expireAfterWrite(60, TimeUnit.SECONDS)
.writer(cacheWriter)
.maximumSize(100L)
.buildAsync((String s) -> { /* <== line 41, exception occurs here */
return UUID.randomUUID();
});
And I'm getting this trace
Exception in thread "main" java.lang.IllegalStateException
at com.github.benmanes.caffeine.cache.Caffeine.requireState(Caffeine.java:174)
at com.github.benmanes.caffeine.cache.Caffeine.buildAsync(Caffeine.java:854)
at com.mycompany.caffeinetest.Main.main(Main.java:41)
If I'll change the cache to a LoadingCache or remove .writer(cacheWriter) the code will run properly. What am I doing wrong? it seems I'm providing the right types to both objects.
Unfortunately these two features are incompatible. While the documentation states this, I have updated the exception to communicate this better. In Caffeine.writer it states,
This feature cannot be used in conjunction with {#link #weakKeys()} or {#link #buildAsync}.
A CacheWriter is a synchronous interceptor for a mutation of an entry. For example, it might be used to evict into a disk cache as a secondary layer, whereas a RemovalListener is asynchronous and using it would leave a race where the entry is not present in either caches. The mechanism is to use ConcurrentHashMap's compute methods to perform the write or removal, and call into the CacheWriter within that block.
In AsyncLoadingCache, the value materializes later when the CompletableFuture is successful, or is automatically removed if null or an error. When the entry is modified within the hash table, this future may be in-flight. This would mean that the CacheWriter would often be called without the materialized value and likely cannot do very intelligent things.
From an API perspective, unfortunately telescoping builders (which use the type system to disallow incompatible chains) become more confusing than using runtime exceptions. Sorry for not making the error clear, which should now be fixed.
Can any one point to a good implementation, if one exists, of what I call Ticking collection/Map in Java. Where the elements in the collection has some expiry time. When a particular element of the collection is expired then collection raises a certain kind of alarm or call a handler.
I saw a Guava implementation of an expiring map which automatically removes the key which has been expired.
Expiring map
guava supports a callback on eviction:
Cache<String, String> cache = CacheBuilder.newBuilder()
.expireAfterAccess(100, TimeUnit.SECONDS)
.removalListener(new RemovalListener<Object, Object>() {
public void onRemoval(RemovalNotification<Object, Object> objectObjectRemovalNotification) {
//do something
}
})
.build();
Two things I really like about Guava 11's CacheLoader (thanks, Google!) are loadAll(), which allows me to load multiple keys at once, and reload(), which allows me to reload a key asynchronously when it's "stale" but an old value exists. I'm curious as to how they play together, since reload() operates on but a single key.
Concretely, extending the example from CachesExplained:
LoadingCache<Key, Graph> graphs = CacheBuilder.newBuilder()
.maximumSize(1000)
.refreshAfterWrite(1, TimeUnit.MINUTES)
.build(
new CacheLoader<Key, Graph>() {
public Graph load(Key key) { // no checked exception
return getGraphFromDatabase(key);
}
public Map<Key, Graph> loadAll(Iterable<? extends K> keys) {
return getAllGraphsFromDatabase(keys);
}
public ListenableFuture<Graph> reload(final Key key, Graph prevGraph) {
if (neverNeedsRefresh(key)) {
return Futures.immediateFuture(prevGraph);
} else {
// asynchronous!
return ListenableFutureTask.create(new Callable<Graph>() {
public Graph call() {
return getGraphFromDatabase(key);
}
});
}
}
});
...where "getAllGraphsFromDatabase()" does an aggregate database query rather than length(keys) individual queries.
How do these two components of a LoadingCache play together? If some keys in my request to getAll() aren't present in the cache, they are loaded as a group with loadAll(), but if some need refreshing, do they get reloaded individually with load()? If so, are there plans to support a reloadAll()?
Here's how refreshing works.
Refreshing on a cache entry can be triggered in two ways:
Explicitly, with cache.refresh(key).
Implicitly, if the cache is configured with refreshAfterWrite and the entry is queried after the specified amount of time after it was written.
If an entry that is eligible for reload is queried, then the old value is returned, and a (possibly asynchronous) refresh is triggered. The cache will continue to return the old value for the key while the refresh is in progress. (So if some keys in a getAll request are eligible for refresh, their old values will be returned, but the values for those keys will be (possibly asynchronously) reloaded.)
The default implementation of CacheLoader.reload(key, oldValue) just returns Futures.immediateFuture(load(key)), which (synchronously) recomputes the value. More sophisticated, asynchronous implementations are recommended if you expect to be doing cache refreshes.
I don't think we're inclined to provide reloadAll at the moment. I suspect it's possible, but things are complicated enough as it is, and I think we're inclined to wait until we see specific demand for such a thing.