I use the CacheBuilder with expireAfterWrite(2000, TimeUnit.Milliseconds). I send 10000 requests to my program and I expect the CacheBuilder to call RemovalListener 10000 times after 2 seconds for each request. I do not observe this behaviour and instead I get RemovalListener called 1 or 2 times.
Can someone please explain to me what CacheBuilder is doing because as I explained above it is doing something totally different from the documentation that Guava is providing.
In the same spirit as above, I use maximumSize(1000) and after sending my program 10000 requests, I expect the RemovalListener to be called 9000 times. But it's called only 1 or 2 times.
How does this module works in reality?
EDIT
I explicitly call clean cleanup each time I receive a request
The removal behavior is documented and works as expected (emphasis mine):
When Does Cleanup Happen?
Caches built with CacheBuilder do not perform cleanup and evict values "automatically," or instantly after a value expires, or anything of the sort. Instead, it performs small amounts of maintenance during write operations, or during occasional read operations if writes are rare.
The reason for this is as follows: if we wanted to perform Cache maintenance continuously, we would need to create a thread, and its operations would be competing with user operations for shared locks. Additionally, some environments restrict the creation of threads, which would make CacheBuilder unusable in that environment.
Instead, we put the choice in your hands. If your cache is high-throughput, then you don't have to worry about performing cache maintenance to clean up expired entries and the like. If your cache does writes only rarely and you don't want cleanup to block cache reads, you may wish to create your own maintenance thread that calls Cache.cleanUp() at regular intervals.
If you want to have more control over the cache and have dedicated executor to take care for calling RemovalListeners, use Caffeine -- a high performance, near optimal caching library based on Java 8 -- which has an API similar to Guava's Cache (same author). Caffeine has more advanced removal handling:
You may specify a removal listener for your cache to perform some operation when an entry is removed, via Caffeine.removalListener(RemovalListener). The RemovalListener gets passed the key, value, and RemovalCause.
Removal listener operations are executed asynchronously using an Executor. The default executor is ForkJoinPool.commonPool() and can be overridden via Caffeine.executor(Executor). When the operation must be performed synchronously with the removal, use CacheWriter instead.
Related
From my application logs, I feel like removeListener is not called immediately after the cache key is expired. This is creating a problem in the below scenario
Cache configuration:
SimpleCacheManager simpleCacheManager = new SimpleCacheManager();
GuavaCache cache = new GuavaCache("cacheData", CacheBuilder.newBuilder().expireAfterAccess(10, TimeUnit.MINUTES).removalListener(expiredCacheListener()).build());
In the application logic, I see that when the cache.get(key) is called then if there is no value(because the cache is expired because of expireAfterAccess() method time limit) then it puts a new value in the cache for the same key since the old key is expired/removed.
After this write operation then immediately, I think that the removalListener is calling the expiredCacheListener() method which has the logic of changing the value for the expired key.... //But this is actually changing the new value!!!
Now I have a valid key with an incorrect value in the cache
If a thread is able to make a key as expired then shouldn't the same thread call the removalListener immediately? How can I solve this?
That's just how Guava Cache works, see CachesExplained:
When Does Cleanup Happen?
Caches built with CacheBuilder do not perform cleanup and evict values
"automatically," or instantly after a value expires, or anything of
the sort. Instead, it performs small amounts of maintenance during
write operations, or during occasional read operations if writes are
rare.
Read more to know that Guava creators "put the choice in your hands"; you're free to maintain cleanup threads by yourself.
For more advanced Cache use cases use Caffeine, which "provides an in-memory cache using a Google Guava inspired API." Removal wiki page mentions that for synchronous removal listeners you could use CacheWriter.
We want to use guava cache for caching third party data to have better response times. The cache needs to be pre-loaded by making a sequence of api calls (~ 4000 api calls are to be made). The api response contains the cache key and its value. These api calls need to be made in parallel from multiple threads (i.e. thread pool) to speed up the cache loading. Each cache entry would have an expiry time. This can be set using expireAfterAccess() call.
After a cache entry expires, it needs to be refreshed automatically in the background. Also there should be a way (api) through which we can stop this background cache refresh so that we do not keep making api calls endlessly. We will call this api once we stop getting user requests after a configured time interval.
Is it possible to delegate the thread management for cache loading and refresh to guava? i.e. Given the api call, the code to map the json response to java object and cache key-value design, can guava perform the pre-loading and refresh on its own?
Thanks.
Automatic refreshing in Guava can be enabled via CacheBuilder.refreshAfterWrite(). The relevant semantics are described as:
Specifies that active entries are eligible for automatic refresh once
a fixed duration has elapsed after the entry's creation, or the most
recent replacement of its value. [ ... ] Currently automatic refreshes
are performed when the first stale request for an entry occurs.
When overriding the method CacheLoader.reload() you can use a thread pool to load values asynchronously.
The problem with this behavior is that you always have a few reads of stale values, before the new value has been loaded, if it succeeds. An alternative cache implementation, like cache2k starts the refreshing immediately after the duration. The latter approach leads to more recent data, but possibly more needless reads. See some recent discussion about that here: https://github.com/ben-manes/caffeine/issues/261
Say I have a Cache that is defined like this:
private static Cache<String, Long> alertsUIDCache = CacheBuilder.newBuilder().
expireAfterAccess(60).build();
From what I read (please correct me if I am wrong):
If value is written to Cache at 0:00, it should be moved to "ready to be evicted" status after 60 seconds. The actual removing of the value from the Cache will happen at the next cache modification (what is exactly is cache modification ?). is that right?
Also, I am not sure what the difference between the invalidateAll() and the cleanUp() methods, can someone provide an explanation?
first part from this link : How does Guava expire entries in its CacheBuilder?
I'm going to focus on expireAfterAccess, but the procedure for expireAfterWrite is almost identical. In terms of the mechanics, when you specify expireAfterAccess in the CacheBuilder, then each segment of the cache maintains a linked list access queue for entries in order from least-recent-access to most-recent-access. The cache entries are actually themselves nodes in the linked list, so when an entry is accessed, it removes itself from its old position in the access queue, and moves itself to the end of the queue.
second part :
from this link : Guava CacheLoader - invalidate does not immediately invalidate entry if both expireAfterWrite and expireAfterAccess are set
invalidate should remove the entry immediately -- not waiting for another query -- and should force the value to get reloaded on the very next query to that key.
cleanUp: Performs any pending maintenance operations needed by the cache. Exactly which activities are performed -- if any -- is implementation-dependent.
from guava documents: https://github.com/google/guava/wiki/CachesExplained
Explicit Removals
At any time, you may explicitly invalidate cache entries rather than waiting for entries to be evicted. This can be done:
individually, using Cache.invalidate(key)
in bulk, using Cache.invalidateAll(keys)
to all entries, using Cache.invalidateAll()
When Does Cleanup Happen?
Caches built with CacheBuilder do not perform cleanup and evict values "automatically," or instantly after a value expires, or anything of the sort. Instead, it performs small amounts of maintenance during write operations, or during occasional read operations if writes are rare.
The reason for this is as follows: if we wanted to perform Cache maintenance continuously, we would need to create a thread, and its operations would be competing with user operations for shared locks. Additionally, some environments restrict the creation of threads, which would make CacheBuilder unusable in that environment.
Instead, we put the choice in your hands. If your cache is high-throughput, then you don't have to worry about performing cache maintenance to clean up expired entries and the like. If your cache does writes only rarely and you don't want cleanup to block cache reads, you may wish to create your own maintenance thread that calls Cache.cleanUp() at regular intervals.
If you want to schedule regular cache maintenance for a cache which only rarely has writes, just schedule the maintenance using ScheduledExecutorService.
I want to use Caffeine for caching and I need to have a write-behind. I want to limit the amount of times I write to the database. The documentation speaks of write-back cache so it should be possible, but there is no example there on how to configure it. I have implemented a CacheWriter, but I don't understand how to configure it to for example only call the writer once every 10 seconds (If something changed to the cache ofcourse).
CacheWriter is an extension point and the documentation describes the use-cases where it may make sense. Those cases are beyond the scope of the library and if implemented instead could have been too rigid.
The writer is called atomically during a normal write operation (but not a computation). This ensures that a sequential order of changes is observed for a given key. For write-behind the writer would add the entry into a queue that is processed asynchronously, e.g. to batch the operations.
When implementing this capability you might want to consider,
Coalescing the updates (e.g. collect into a LinkedHashMap)
Performing a batch prior to a periodic write-behind if it exceeds a threshold size
Loading from the write-behind buffer if the operations have not yet been flushed (This avoids an inconsistent view, e.g. due to eviction)
Handling retrials, rate limiting, and striping depending on the the characteristics of the external resource
Update:
Wim Deblauwe provided a nice example using RxJava.
I have used google guava in our application based on weakValues like below.
CacheBuilder
.newBuilder()
.weakValues()
.concurrencyLevel(Runtime.getRuntime().availableProcessors())
.removalListener(removalListener)
.build(cacheLoader);
The removal listener is not called immediately after the cache weak value is removed. I have to clear native resource based on this and thus my program ends without releasing memory at native end. Is it a known issue?
In the Google Guava docs, it says the following:
Caches built with CacheBuilder do not perform cleanup and evict values "automatically," or instantly after a value expires, or anything of the sort. Instead, it performs small amounts of maintenance during write operations, or during occasional read operations if writes are rare.
It is also explained why they took this decision and the alternatives available.
I don't think the listener will be called if JVM GC claimed the weakRef. that listener should only work when eviction policy triggers the eviction. e.g. expireAfterWriter() or expireAfterAccess()