example from : SpringSource
#Cacheable(value = "vets")
public Collection<Vet> findVets() throws DataAccessException {
return vetRepository.findAll();
}
How does findVets() work exactly ?
For the first time, it takes the data from vetRepository and saves the result in cache. But what happens if a new vet is inserted in the database - does the cache update (out of the box behavior) ? If not, can we configure it to update ?
EDIT:
But what happens if the DB is updated from an external source (e.g. an application which uses the same DB) ?
#CachePut("vets")
public void save(Vet vet) {..}
You have to tell the cache that an object is stale. If data change without using your service methods then, of course, you would have a problem. You can, however, clear the whole cache with
#CacheEvict(value = "vets", allEntries = true)
public void clearCache() {..}
It depends on the Caching Provider though. If another app updates the database without notifying your app, but it uses the same cache it, then the other app would probably update the cache too.
It would not do it automatically and there is not way for the cache to know if the data has been externally introduced.
Check #CacheEvict which will help you invalidate the cache entry in case of any change to the underlying collections.
#CacheEvict(value = "vet", allEntries = true)
public void saveVet() {
// Intentionally blank
}
allEntries
Whether or not all the entries inside the cache(s) are removed or not.
By default, only the value under the associated key is removed. Note that specifying setting this parameter to true and specifying a key is not allowed.
you also can use #CachePut on the method which creates the new entry. The return type has to be the same as in your #Cachable method.
#CachePut(value = "vets")
public Collection<Vet> updateVets() throws DataAccessException {
return vetRepository.findAll();
}
In my opinion an exernal service has to call the same methods.
Related
Is there a way to populate a Map once from the DB (through Mongo repository) data and reuse it when required from multiple classes instead of hitting the Database through the repository.
As per your comment, what you are looking for is a Caching mechanism. Caches are components which allow data to live in memory, as opposed to files, databases or other mediums so as to allow for the fast retrieval of information (against a higher memory footprint).
There are probably various tutorials online, but usually caches all have the following behaviour:
1. They are key-value pair structures.
2. Each entity living in the cache also has a Time To Live, that is, how long will it considered to be valid.
You can implement this in the repository layer, so the cache mechanism will be transparent to the rest of your application (but you might want to consider exposing functionality that allows to clear/invalidate part or all the cache).
So basically, when a query comes to your repository layer, check in the cache. If it exists in there, check the time to live. If it is still valid, return that.
If the key does not exist or the TTL has expired, you add/overwrite the data in the cache. Keep in mind that when updating the data model yourself, you also invalidate the cache accordingly so that new/fresh data will be pulled from the DB on the next call.
You can declare the map field as public static and this would allow application wide access to hit via ClassLoadingData.mapField
I think a better solution, if I understood the problem would be a memoized function, that is a function storing the value of its call. Here is a sketch of how this could be done (note this does not handle possible synchronization problem in a multi threaded environment):
class ClassLoadingData {
private static Map<KeyType,ValueType> memoizedValues = new HashMap<>();
public Map<KeyType,ValueType> getMyData() {
if (memoizedData.isEmpty()) { // you can use more complex if to handle data refresh
populateData(memoizedData);
} else {
return memoizedData;
}
}
private void populateData() {
// do your query, and assign result to memoizedData
}
}
Premise: I suggest you to use an object-relational mapping tool like Hibernate on your java project to map the object-oriented
domain model to a relational database and let the tool handle the
cache mechanism implicitally. Hibernate specifically implements a multi-level
caching scheme ( take a look at the following link to get more
informations:
https://www.tutorialspoint.com/hibernate/hibernate_caching.htm )
Regardless my suggestion on premise you can also manually create a singleton class that will be used from every class in the project that goes to interact with the DB:
public class MongoDBConnector {
private static final Logger LOGGER = LoggerFactory.getLogger(MongoDBConnector.class);
private static MongoDBConnector instance;
//Cache period in seconds
public static int DB_ELEMENTS_CACHE_PERIOD = 30;
//Latest cache update time
private DateTime latestUpdateTime;
//The cache data layer from DB
private Map<KType,VType> elements;
private MongoDBConnector() {
}
public static synchronized MongoDBConnector getInstance() {
if (instance == null) {
instance = new MongoDBConnector();
}
return instance;
}
}
Here you can define then a load method that goes to update the map with values stored on the DB and also a write method that instead goes to write values on the DB with the following characteristics:
1- These methods should be synchronized in order to avoid issues if multiple calls are performed.
2- The load method should apply a cache period logic ( maybe with period configurable ) to avoid to load for each method call the data from the DB.
Example: Suppose your cache period is 30s. This means that if 10 read are performed from different points of the code within 30s you
will load data from DB only on the first call while others will read
from cached map improving the performance.
Note: The greater is the cache period the more is the performance of your code but if the DB is managed you'll create inconsistency
with cache if an insertion is performed externally ( from another tool
or manually ). So choose the best value for you.
public synchronized Map<KType, VType> getElements() throws ConnectorException {
final DateTime currentTime = new DateTime();
if (latestUpdateTime == null || (Seconds.secondsBetween(latestUpdateTime, currentTime).getSeconds() > DB_ELEMENTS_CACHE_PERIOD)) {
LOGGER.debug("Cache is expired. Reading values from DB");
//Read from DB and update cache
//....
sampleTime = currentTime;
}
return elements;
}
3- The store method should automatically update the cache if insert is performed correctly regardless the cache period is expired:
public synchronized void storeElement(final VType object) throws ConnectorException {
//Insert object on DB ( throws a ConnectorException if insert fails )
//...
//Update cache regardless the cache period
loadElementsIgnoreCachePeriod();
}
Then you can get elements from every point in your code as follow:
Map<KType,VType> liveElements = MongoDBConnector.getElements();
I'm attempting to use both the #Cacheable and #PostFilter annotations in Spring. The desired behavior is that the application will cache the full, unfiltered listed of Segments (it's a very small and very frequently referenced list so performance is the desire), but that a User will only have access to certain Segments based on their roles.
I started out with both #Cacheable and #PostFilter on a single method, but when that wasn't working I broke them out into two separate classes so I could have one annotation on each method. However, it seems to behave the same either way I do it, which is to say when User A hits the service for the first time they get their correct filtered list, then when User B hits the service next they get NO results because the cache is only storing User A's filtered results, and User B does not have access to any of them. (So the PostFilter still runs, but the Cache seems to be storing the filtered list, not the full list.)
So here's the relevant code:
configuration:
#Configuration
#EnableCaching
#EnableGlobalMethodSecurity(prePostEnabled = true)
public class BcmsSecurityAutoConfiguration {
#Bean
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Arrays.asList(
new ConcurrentMapCache("bcmsSegRoles"),
new ConcurrentMapCache("bcmsSegments")
));
return cacheManager;
}
}
Service:
#Service
public class ScopeService {
private final ScopeRepository scopeRepository;
public ScopeService(final ScopeRepository scopeRepository) {
this.scopeRepository = scopeRepository;
}
// Filters the list of segments based on User Roles. User will have 1 role for each segment they have access to, and then it's just a simple equality check between the role and the Segment model.
#PostFilter(value = "#bcmsSecurityService.canAccessSegment( principal, filterObject )")
public List<BusinessSegment> getSegments() {
List<BusinessSegment> segments = scopeRepository.getSegments();
return segments; // Debugging shows 4 results for User A (post-filtered to 1), and 1 result for User B (post-filtered to 0)
}
}
Repository:
#Repository
public class ScopeRepository {
private final ScopeDao scopeDao; // This is a MyBatis interface.
public ScopeRepository(final ScopeDao scopeDao) {
this.scopeDao = scopeDao;
}
#Cacheable(value = "bcmsSegments")
public List<BusinessSegment> getSegments() {
List<BusinessSegment> segments = scopeDao.getSegments(); // Simple SELECT * FROM TABLE; Works as expected.
return segments; // Shows 4 results for User A, breakpoint not hit for User B cache takes over.
}
}
Does anyone know why the Cache seems to be storing the result of the Service method after the filter runs, rather than storing the full result set at the Repository level as I'm expecting it should? Or have another way to achieve my desired behavior?
Bonus points if you know how I could gracefully achieve both caching and filtering on the same method in the Service. I only built the superfluous Repository because I thought splitting the methods would resolve the caching problem.
Turns out that the contents of Spring caches are mutable, and the #PostFilter annotation modifies the returned list, it does not filter into a new one.
So when #PostFilter ran after my Service method call above it was actually removing items from the list stored in the Cache, so the second request only had 1 result to start with, and the third would have zero.
My solution was to modify the Service to return new ArrayList<>(scopeRepo.getSegments()); so that PostFilter wasn't changing the cached list.
(NOTE, that's not a deep clone of course, so if someone modified a Segment model upstream from the Service it would likely change in the model in the cache as well. So this may not be the best solution, but it works for my personal use case.)
I can't believe Spring Caches are mutable...
I am using EhCache core 3.0. It internally uses BaseExpiry and Eh107Expiry class to check whether cache is expired or not. These classes implement Expiry interface. My query is, can we extend methods which are used to check whether cache is expired or not. I don't want to expire contents of the cache even if time is elapsed if my method is using some data from that cache.
Have a look at the dedicated section on Expiry in the documentation. It will help you understand what you can do and how to do it.
If that does not help you, please expand your question as suggested in comments.
If you add time-to-idle in xml or override getExpiryForAccess from Expiry interface,then your entries will not delete when you are accessing them.Below is the code to build Eh cache with custom Expire.This blog will help you for other properties with explanation.
CacheConfigurationBuilder<Integer,String> cacheConfigurationBuilder = CacheConfigurationBuilder.newCacheConfigurationBuilder();
cacheConfigurationBuilder.withExpiry(new Expiry() {
#Override
public Duration getExpiryForCreation(Object key, Object value) {
return new Duration(120, TimeUnit.SECONDS);
}
#Override
public Duration getExpiryForAccess(Object key, Object value) {
return new Duration(120, TimeUnit.SECONDS);
}
#Override
public Duration getExpiryForUpdate(Object key, Object oldValue, Object newValue) {
return null;
}
})
.usingEvictionPrioritizer(Eviction.Prioritizer.LFU)
.withResourcePools(ResourcePoolsBuilder.newResourcePoolsBuilder().heap(200, EntryUnit.ENTRIES))
// adding defaultSerializer config service to configuration
.add(new DefaultSerializerConfiguration(CompactJavaSerializer.class, SerializerConfiguration.Type.KEY))
.buildConfig(Integer.class, String.class);
I guess you can use an ehcache decorator and reimplement isExpiry to add your own conditions. Please refer to https://www.ehcache.org/documentation/2.8/apis/cache-decorators.html.
I had the problem, that every time i retrieved a collection from the gwt request factory, there was the "findEntity()"-method called for every entity in that collection. And this "findEntity()"-method calls the SQL-Database.
I found out that this happens because request factory checks the "liveness" of every entity in the "ServiceLayerDecorator.isLive()"-method (also described here: requestfactory and findEntity method in GWT)
So i provided my own RequestFactoryServlet:
public class MyCustomRequestFactoryServlet extends RequestFactoryServlet {
public MyCustomRequestFactoryServlet() {
super(new DefaultExceptionHandler(), new MyCustomServiceLayerDecorator());
}
}
And my own ServiceLayerDecorator:
public class MyCustomServiceLayerDecorator extends ServiceLayerDecorator {
/**
* This check does normally a lookup against the db for every element in a collection
* -> Therefore overridden
*/
#Override
public boolean isLive(Object domainObject) {
return true;
}
}
This works so far and I don't get this massive amount of queries against the database.
Now I am wondering if I will get some other issues with that? Or is there a better way to solve this?
RequestFactory expects a session-per-request pattern with the session guaranteeing a single instance per entity (i.e. using a cache).
The proper fix is to have isLive hit that cache, not the database. If you use JPA or JDO, they should do that for you for free. What matters is what "the request" thinks about it (if you issued a delete request, isLive should return false), not really what's exactly stored in the DB, taking into account what other users could have done concurrently.
That being said, isLive is only used for driving EntityProxyChange events on the client side, so if you don't use them, it shouldn't cause any problem unconditionally returning true like you do.
Two things I really like about Guava 11's CacheLoader (thanks, Google!) are loadAll(), which allows me to load multiple keys at once, and reload(), which allows me to reload a key asynchronously when it's "stale" but an old value exists. I'm curious as to how they play together, since reload() operates on but a single key.
Concretely, extending the example from CachesExplained:
LoadingCache<Key, Graph> graphs = CacheBuilder.newBuilder()
.maximumSize(1000)
.refreshAfterWrite(1, TimeUnit.MINUTES)
.build(
new CacheLoader<Key, Graph>() {
public Graph load(Key key) { // no checked exception
return getGraphFromDatabase(key);
}
public Map<Key, Graph> loadAll(Iterable<? extends K> keys) {
return getAllGraphsFromDatabase(keys);
}
public ListenableFuture<Graph> reload(final Key key, Graph prevGraph) {
if (neverNeedsRefresh(key)) {
return Futures.immediateFuture(prevGraph);
} else {
// asynchronous!
return ListenableFutureTask.create(new Callable<Graph>() {
public Graph call() {
return getGraphFromDatabase(key);
}
});
}
}
});
...where "getAllGraphsFromDatabase()" does an aggregate database query rather than length(keys) individual queries.
How do these two components of a LoadingCache play together? If some keys in my request to getAll() aren't present in the cache, they are loaded as a group with loadAll(), but if some need refreshing, do they get reloaded individually with load()? If so, are there plans to support a reloadAll()?
Here's how refreshing works.
Refreshing on a cache entry can be triggered in two ways:
Explicitly, with cache.refresh(key).
Implicitly, if the cache is configured with refreshAfterWrite and the entry is queried after the specified amount of time after it was written.
If an entry that is eligible for reload is queried, then the old value is returned, and a (possibly asynchronous) refresh is triggered. The cache will continue to return the old value for the key while the refresh is in progress. (So if some keys in a getAll request are eligible for refresh, their old values will be returned, but the values for those keys will be (possibly asynchronously) reloaded.)
The default implementation of CacheLoader.reload(key, oldValue) just returns Futures.immediateFuture(load(key)), which (synchronously) recomputes the value. More sophisticated, asynchronous implementations are recommended if you expect to be doing cache refreshes.
I don't think we're inclined to provide reloadAll at the moment. I suspect it's possible, but things are complicated enough as it is, and I think we're inclined to wait until we see specific demand for such a thing.