When using ehcache, is there a way to expire cache on certain time of the day?
Thanks,
Lawardy
There is no such functionality out-of-the-box. You need an external solution like Quartz, also from Terracotta umbrella.
In fact, even normal timeToLive parameter does not remove element in question after this time elapses, because this would required additional thread. Instead the item is removed when new one is to be added which takes its place.
Related
I'm adding a caching functionality to one of the DoFns inside a Dataflow Pipeline in Java. The DoFn is currently using a REST client to send a request to an endpoint (that charges based on number of request, and the response will change roughly every hour) for every input element, and what I want to achieve is to cache the response from the endpoint, and have it expiries every 15 mins.
I found two ways to do this: one from a similar stackoverflow question that suggested to use static variable to host a cache service (I used guava for caching). However I wasn't sure how to update the expiry from outside of the DoFn.
Another approach that I found is to use stateful processing to store a hash that keep track of the requests and responses, and use a TimerSpec to clear the "cache" every 15 mins. Although it appears that there is no way to set a timer for each element in the cache.
I haven't tried the second approach yet. While I'm going to implement it, I wonder if someone had running into similar situations, and has any suggestions, or has better approaches.
However I wasn't sure how to update the expiry from outside of the DoFn.
Do you need to? DoFn does a lookup first and if the entry already expired it issues a request and updates cache. Cache reads & write need to be thread-safe.
Although it appears that there is no way to set a timer for each element in the cache.
You can probably sets a timer that scan the entire cache every X minutes and refresh expired entries. However, if your state is not keyed using a global state like this will limit the parallelism of your pipeline.
I don't know if it's a real question or not... But i'd like to know how some of you will approach this...
I have a Spring Boot application.
Then I have a Interruttore.class, which has, among others this field timeoutDatewhich is a Date.
In the app, various instances of this class are used. The timeoutDate field can be updated, for every single object, by various factors. I need to know when the actual date reaches the timeutDate.
In a very simple (and not optimized) way i would have created a #Scheduled task, but the delay will be too short and i don't like it, how can i do?
In a very simple (and not optimized) way i would have created a
#Scheduled task, but the delay will be too short and i don't like it,
how can i do?
Why too short ?
You can use the delay you wish.
#Scheduled(fixedDelay=50000) // 50 secs
#Scheduled(fixedDelay=1000) // 1 secs
Look at the documentation for Spring's various task scheduling APIs: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html
You have plenty of choices. I think the "not optimised" idea you might have is to schedule a repeating task which searches your beans to find the expired ones. That would indeed be inefficient for large numbers of beans.
You could simply create a scheduled task for each bean with a timeoutDate, created at the same time as that bean, and when its timeoutdate is updated (Spring AOP could help with this).
Alternatively you could keep a list of beans, sorted by timeout date. Schedule a task for the time of the earliest expiry. It reaps that bean and any others who's time is past, then schedules a new task for the time of the next expiry.
If you do this, you need to make sure that:
- it handles new objects added to the list (perhaps with an expiry date earlier than the currently scheduled cull)
- it handles the case where an object is removed for a reason other than a timeout
(Unless neither of those things can happen -- in which case don't worry about it!)
You can use Quartz or Jesque(redis). Whatever task needs to be executed, you can schedule that task at that time.
If this time value can be updated anytime, you can cancel(unschedule) the previously scheduled task(using task identifiers or keys) and reschedule it with the updated time.
My use-case is that I need to implement a cache on top of a service should expire entries after a certain amount of time (from their time of creation).
And if the entry is getting expired, then service look up should be done to get the latest entry. lets call is service refresh.
But, lets say if service refresh fails, then I should be able to use the stale data in the cache.
But since the cache is already expired, I don't have that entry.
So, I am thinking of controlling the expiration of the cache and cache entry would only be expired only if service is available to get the latest data, otherwise don't remove that entry.
I was looking into Google Guava cache, but it only provides a removalListener which would just notify me with the event but I am not able to control the expiration event with this.
Is there any third party cache implementation which can serve my purpose?
This kind of resilience and robustness semantics are implemented in cache2k. We use this in production for quite some time. The setup looks like
CacheBuilder.newCache(Key.class, Value.class)
.name("myCache")
.source(new YourSourceImplementation())
.backgroundRefresh(true)
.suppressExceptions(true)
.expiryDuration(60, TimeUnit.SECONDS)
.build();
With exceptionExpiryDuration you can actually set a shorter interval for the retry. There was a similar question on SO, which I also answered, see: Is it possible to configure Guava Cache (or other library) behaviour to be: If time to reload, return previous entry, reload in background (see specs) There you find some more details about it.
Regardless what cache you use, you will run into a lot of issues, since exception handling in general and building robust and resilient applications needs some thoughts in the details.
That said, I am not totally happy with the solution yet, since I think we need more control, e.g. how long stale data should be served. Furthermore, the cache needs to have some alerting if there is stale data in it. I put some thoughts on how to improve this here: https://github.com/cache2k/cache2k/issues/26
Feedback is very welcome.
Intro
This is regarding infinispan cache but I think this is a generic enough question.
In my infinispan cache Im inputting items into cache using putIfAbsent method and removing them with remove method. (jboss doc here)
Basic Behavior
I can put items into my cache and remove items from cache using a id. But if I do not remove the cache item explicitly, infinispan will remove it automatically after the specified life span is passed.
In my remove method I run some custom code before removal.
But when I am not invoking the remove method, infinispan remove the cache entry due to the expiry of provided life span.
In that automatic removal scenario I cannot run my custom code because it happens under the hood. (by infinispan). So what is the possible way? Following code will explain this a bit more hopefully.
I belive I am clear enough. Please give me some insight.
Thank you.
void putToCache(String id){
myCache.putIfAbsent(id,value,LIFESPAN_DURATION,timeUnit);
}
void removeFromCache(String id){
//MY CUSTOM CODE - WANT TO RUN THIS IN LIFE SPAN DURATION EXPIRY ALSO
myCache.remove(id);
}
Before going into the answer I want to clarify between eviction and expiration.
Eviction
Eviction is when an entry is removed from the in memory container due to it growing outside of the configured maximum size (max-entries). Eviction can only be enabled at configuration time and is not controlled at runtime which Expiration can be, please see here. Note that eviction never removes an entry from a store if you have one configured, thus you don't just lose an entry due to having too many if you don't want to.
Infinispan 7 and older versions have a listener for Eviction as you already found, and the event raised would be this one.
Expiration
Expiration defines an entry being removed after a period of inactivity or a set duration. This can be configured through configuration or you can override the configured value by using the API as you did your in example.
Now to answer your question, an Infinispan Expiration event was added with Infinispan 8. It is detailed more here. Note an important piece is that the expiration event will be fired holding the lock for the key that is expiring (unless your listener is defined as async). This is important to guarantee proper ordering.
Also you should keep in mind that expiration events are not guaranteed to fire exactly when the entry expires. Rather the event is only fired upon accessing said expired entry or if the expiration reaper thread finds the expired entry. The expiration reaper defaults to run every minute, you can disable this or change the time by changing the expiration interval configuration setting.
In my scenario, getting the fresh (non-cached) values is a very expensive operation so it is imperative pre-calculated cached values exist at all times instead of refreshing them on read, like EhCache seems to do.
For this it sounds reasonable to have a thread firing on TTL expiration repopulating the cache with fresh values, so no reads are ever waiting.
Is there a way to achieve this using Ehcache? Listening for OnElementExpired/Evicted events to repopulate the cache seems like a no-go (by the time I receive the event, a read would already be waiting).
I guess I could make the cache itself eternal and have my own scheduled task that repopulates, but then I get nothing from EhCache over dumb maps that I have now. Is this really how it is? Is there no way to have EhCache help me in this situation?
Ehcache provides a way of doing what you want with scheduled refresh.
You will need two things in order to make this work with Ehcache:
Use a cache loader - that is move to a cache read-through pattern. This is required as otherwise Ehcache has no idea how to get to the data mapped to a key.
Configure scheduled refresh - this works by launching a quartz scheduler instance.
Take a look at RefreshAheadCache, provided by EHCache.
However, I cannot find any examples of its use and indicators that this is mature.
The comment of the class says:
A cache decorator which implements read ahead refreshing. Read ahead occurs when a cache entry is accessed prior to its expiration, and triggers a reload of the value in the background.
This does not directly solve the problem as you mention below:
My problem is how to repopulate the cache without waiting for a read to trigger it
As far as I know there is no standard way to do it. The reason for it, is that the expiry is not timer based.
(Shameless) hint: Since I think this is quite useful, I implemented this in cache2k. The feature is called background refresh, enabled by CacheBuilder.backgroundRefresh(true).
may be this code could help :
https://github.com/jsr107/jsr107spec/issues/328