I have dozens of data access objects like PersonDao with methods like:
Person findById(String id) {}
List<Person> search(String firstName, LastName, Page) {}
int searchCount(String firstName, LastName) {}
I've experimented by adding guava cache with one of these classes and it's really nice, but there's a lot of boilerplate.
Here's an example of making findById look in the cache first:
private final LoadingCache<String, Person> cacheById = CacheBuilder.newBuilder()
.maximumSize(maxItemsInCache)
.expireAfterWrite(cacheExpireAfterMinutes, TimeUnit.MINUTES)
.build(new CacheLoader<String, Person>() {
public Person load(String key) {
return findByIdNoCache(key);
});
//.... and update findById to call the cache ...
#Override
public Person findById(String id) {
return cacheById.getUnchecked(id);
}
So, because each method has different params and return types, I end up created a separate cacheLoader for every method!
I tried consolidating everything into a single CacheLoader that returns Object type and accepts a Map of objects, but then I end up with big ugly if/else to figure out which method to call to load the cache.
I'm struggling to find an elegant way to add caching to these data access objects, any suggestions? Maybe guava cache isn't meant for this use case?
Try this. Unfortunately, there are compiler warnings due to generics... But we may supress them because we know nothing bad will happen.
public class CacheContainer {
private static final long maxItemsInCache = 42;
private static final long cacheExpireAfterMinutes = 42;
private final Map<String, LoadingCache> caches = Maps.newHashMap();
public <K, V> V getFromCache(String cacheId, K key, CacheLoader<K, V> loader) throws ExecutionException {
LoadingCache<K, V> cache = caches.get(cacheId);
if (cache == null) {
cache = CacheBuilder.newBuilder().
maximumSize(maxItemsInCache).
expireAfterWrite(cacheExpireAfterMinutes, TimeUnit.MINUTES).
build(loader);
caches.put(cacheId, cache);
}
return cache.get(key);
}
}
And then in your dao:
private final CacheContainer cacheContainer = new CacheContainer();
public Person findById(String id) {
cacheContainer.getFromCache("personById", id, new CacheLoader<String, Person>() {
#Override
public Person load(String key) {
return findByIdNoCache(key);
});
}
Other methods in the same way. I don't think you can reduce boilerplate any more than that.
Creating a CacheLoader (and separate cache) for each method you want to cache the results of is necessary. You could simplify things a little by creating a single CacheBuilder with the cache configuration you want and then creating each of the caches like that:
private final CacheBuilder<Object, Object> builder = CacheBuilder.newBuilder()
.maximumSize(maxItemsInCache)
.expireAfterWrite(cacheExpireAfterMinutes, TimeUnit.MINUTES);
private final LoadingCache<String, Person> cacheById = builder.build(
new CacheLoader<String, Person>() {
// ...
});
private final LoadingCache<Search, List<Person>> searchCache = builder.build(
new CacheLoader<Search, List<Person>>() {
// ...
});
// etc.
Related
I have a simple in-memory cache that looks like (oversimplified) below:
class Cache {
private Map<String, Integer> m = new HashMap<>();
public String get(String key) {
if(!m.containsKey(key))
return null;
return m.get(key);
}
public void set(String key, String value) {
m.put(key, value);
}
public Set<String> getKeys() {
return m.keySet();
}
}
Now I want to write Unit Tests for this class. I was thinking I can do something along the lines of:
[TestMethod]
public void Get_Success() {
Cache c = new Cache();
c.set("Apple", 1);
c.set("Grape", 2);
c.set("Banana", 3);
Assert.Equals(1, c.get("Apple"));
Assert.Equals(2, c.get("Grape"));
Assert.Equals(3, c.get("Banana"));
}
But the problem is, it tests both Set and Get methods. So if the test breaks, I will not know whether the bug is in Set or Get.
What is the recommended way to test the Get and Set methods individually?
One way I thought was, to modify the Cache class to accept a Map<String, Integer> instance like:
class Cache {
private Map<String, Integer> m = new HashMap<>();
public Cache(Map<String, Integer> m) {
this.m = m;
}
...
...
}
Then from the test, I can create an instance of Map and pass it to Cache constructor. Then all the Set operations will happen in the map instance that I passed and I can validate whether Set worked as expected.
But I am not very sure whether it's the right approach.
Similarly, I want the ability to test Get as well without depending on Set.
Right now my cache looks like the following:
public class TestCache {
private LoadingCache<String, List<ObjectABC>> cache;
TestCache() {
cache = CacheBuilder.newBuilder().expireAfterAccess(10, TimeUnit.MINUTES).maximumSize(25)
.build(new CacheLoader<String, List<ObjectABC>>(
) {
#Override
public List<ObjectABC> load(String key) throws Exception {
// TODO Auto-generated method stub
return addCache(key);
}
});
}
private List<ObjectABC> addCache(String key) {
final JoiObjectMapper mapper = new JoiObjectMapper();
final Collection<File> allConfigFiles = FileUtils.listFiles(new File(key), null, true);
final List<ObjectABC> configsList = new ArrayList<>();
allConfigFiles.forEach(configFile -> {
try {
configsList.add(mapper.readValue(configFile, new TypeReference<ObjectABC>() {
}));
} catch (Exception e) {
throw new RuntimeException(e);
}
});
return configsList;
}
public List<ObjectABC> getEntry(String key) {
try {
return cache.get(key);
} catch (ExecutionException e) {
throw new NonRetriableException(String.format(
"Exception occured while trying to get data from cache for the key : {} Exception: {}",
key.toString(), e));
}
}
}
In the above code, when I pass a String key (which is path to a local folder) it takes all the files present in that location and maps them to ObjectABC using ObjectMapper.
Now my problem is that I want to instead have a generic loading cache like
LoadingCache<String, List<Object>>.
And I want to map files in different folders to different Objects, e.g. map files in /root/Desktop/folder1 to List<ObjectABC> and map files in /root/Desktop/folder2 to List<ObjectDEF> and be able to store and retrieve that information from the cache.
How can I pass to the cache the information of which object to use for mapping?
You can create a custom class wrapping a LoadingCache<Key<?>, Object> like that:
class HeterogeneousCache {
private final LoadingCache<Key<?>, Object> cache;
public <T> T get(Key<T> key) throws ExecutionException {
return key.getType().cast(cache.get(key));
}
}
#Value // provides constructor, getters, equals, hashCode
class Key<T> {
private final String identifier;
private final Class<T> type;
}
(I used Lombok's #Value annotation for simplicity)
Of course, this is just a stub and you might need to adapt this to your needs. The main problem might be that you can't get a Class<List<ObjectABC>> - you can only get a Class<List>. The easiest way out of this is to wrap the List<ObjectABC> in some custom type. The harder way (not recommended) is to use Guava's TypeToken.
Attribution: This answer is based on the post by Frank Appel entitled How to Map Distinct Value Types Using Java Generics, which itself is based on Joshua Bloch's typesafe hetereogeneous containers from Effective Java.
Edit: A Complete Solution
Since the OP wants List<T> as result, and since he needs instances of TypeReference<T>, I replaced Class<T> with TypeReference<T> in Key<T>:
#Value // provides constructor, getters, equals, hashCode
class Key<T> {
private final String identifier;
private final TypeReference<T> typeReference;
}
Here's how CustomHeterogeneousCache looks now:
class CustomHeterogeneousCache {
private final LoadingCache<Key<?>, List<?>> cache = CacheBuilder.newBuilder()
.expireAfterAccess(10, TimeUnit.MINUTES)
.maximumSize(25)
.build(CacheLoader.from(this::computeEntry));
#SuppressWarnings("unchecked")
public <T> List<T> getEntry(Key<T> key) {
return (List<T>) cache.getUnchecked(key);
}
private <T> List<T> computeEntry(Key<T> key) {
final JoiObjectMapper mapper = new JoiObjectMapper();
final Collection<File> allConfigFiles = FileUtils.listFiles(new File(key.getIdentifier()), null, true);
return allConfigFiles.stream()
.map(configFile -> {
try {
return mapper.readValue(configFile, key.getTypeReference());
} catch (Exception e) {
throw new RuntimeException(e);
}
})
.collect(Collectors.toList());
}
}
Since implementations of TypeReference do not have value semantics, the user must make sure that every Key is created once, and then only referenced, e.g.:
class Keys {
public static final Key<ObjectABC> ABC = new Key<>("/root/Desktop/folder1", new TypeReference<ObjectABC>() {
});
public static final Key<ObjectDEF> DEF = new Key<>("/root/Desktop/folder2", new TypeReference<ObjectDEF>() {
});
}
I am facing an issue with Guava Caches. When I have only one element in cache, things are fine. But when I load a second element, Its trying to pick with key of earlier entry
private static LoadingCache<String, MyClass> cache = null;
....
public MyClass method(final String id1, final long id2) {
log.error("inside with "+id1);
final String cacheKey = id1+"-"+id2;
if(cache == null){
cache = CacheBuilder.newBuilder()
.maximumSize(1000)
.build(
new CacheLoader<String, MyClass>() {
#Override
public MyClass load(String key) {
return getValue(cacheKey);
}
}
);
}
try {
return cache.get(cacheKey);
} catch (ExecutionException ex) {
log.error("EEE missing entry",ex);
}
}
private MyClass getValue(String cacheKey){
log.error("not from cache "+cacheKey);
...
}
The log says:
inside with 129890038707408035563943963861595603358
not from cache 1663659699-315839912047403113610285801857400882820 // This is key for the earlier entry
For eg, When I call method("1", 2), it loads the value in cache and I am able to get it from cache subsequently. Now I call method ("3", 4), this is not in cache, so getValue() is called and the log prints out the key for method("1", 2)
Where am I going wrong?
Your problem is related to how you create your CacheLoader, if you check well you will see that you initialize it with a given cache key (the value of the local variable cacheKey at the time the cache is lazily initialized) while it should be more generic and rely on the key provided as parameter to the method load of your CacheLoader otherwise it will load the cache by calling getValue(key) with the same key.
It should be this:
new CacheLoader<String, MyClass>() {
#Override
public MyClass load(String key) {
return getValue(key); // instead of return getValue(cacheKey);
}
}
NB: The way you initialize your cache is not thread safe, indeed if it has not been initialized and your method method is called by several threads concurrently it will be created several times instead of one.
One way could be to use the double-checked locking idiom as next:
private static volatile LoadingCache<String, MyClass> cache = null;
public MyClass method(final String id1, final long id2) {
...
if(cache == null){
synchronized (MyClass.class) {
if(cache == null){
cache = ...
}
}
}
NB: Do not initialize a static cache with a CacheLoader based on a non static method, it is much too error prone. Make them both non static or static but don't mix them.
Assuming that you can make both static, your cache initialization will be very simply, it would simply be:
private static final LoadingCache<String, MyClass> cache = CacheBuilder.newBuilder()...
No need to initialize it lazily which will also simplify a lot the code of your method as it will simply be reduce to:
public MyClass method(final String id1, final long id2) {
log.error("inside with "+id1);
final String cacheKey = id1+"-"+id2;
try {
return cache.get(cacheKey);
} catch (ExecutionException ex) {
log.error("EEE missing entry",ex);
}
}
I'm trying to use a snippet of code for a Stash plugin, but the compiler keeps giving me an error that I can't seem to solve. It's using com.google.common.cache.Cache (Guava)
static final RepositorySettings DEFAULT_SETTINGS = new RepositorySettings(0);
private final PluginSettings pluginSettings;
private final Cache<Integer, RepositorySettings> cache = CacheBuilder.newBuilder().build(
new CacheLoader<Integer, RepositorySettings>()
{
#Override
public RepositorySettings load(#Nonnull Integer repositoryId)
{
#SuppressWarnings("unchecked")
Map<String, String> data = (Map) pluginSettings.get(repositoryId.toString());
return data == null ? DEFAULT_SETTINGS : deserialize(data);
}
});
The .build is giving me the following error
The method build(CacheLoader<? super Integer,RepositorySettings>) is ambiguous for the type CacheBuilder<Object,Object>
Cache has a build() method that takes no parameters, LoadingCache on the other hand has a build() method that takes CacheLoader as a parameter.
private final LoadingCache<Integer, RepositorySettings> cache = CacheBuilder.newBuilder().build(
new CacheLoader<Integer, RepositorySettings>() {
#Override
public RepositorySettings load(#Nonnull Integer repositoryId) {
#SuppressWarnings("unchecked")
Map<String, String> data = (Map) pluginSettings.get(repositoryId.toString());
return data == null ? DEFAULT_SETTINGS : deserialize(data);
}
});
This should work.
As reference:
http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/cache/CacheBuilder.html
I have several Java enums that looks something like below (edited for confidentiality, etc).
In each case, I have a lookup method that I'm really not satisfied with; in the example below, it is findByChannelCode.
public enum PresentationChannel {
ChannelA("A"),
ChannelB("B"),
ChannelC("C"),
ChannelD("D"),
ChannelE("E");
private String channelCode;
PresentationChannel(String channelCode) {
this.channelCode = channelCode;
}
public String getChannelCode() {
return this.channelCode;
}
public PresentationChannel findByChannelCode(String channelCode) {
if (channelCode != null) {
for (PresentationChannel presentationChannel : PresentationChannel.values()) {
if (channelCode.equals(presentationChannel.getChannelCode())) {
return presentationChannel;
}
}
}
return null;
}
}
The problem is, I feel silly doing these linear lookups when I could just be using a HashMap<String, PresentationChannel>. So I thought of the solution below, but it's a little messier that I would hope and, more to the point, I didn't care to re-invent the wheel when surely someone else has come across this. I wanted to get some of the sage wisdom of this group: what is the proper way to index an enum by value?
My solution:
ImmutableMap<String, PresentationChannel> enumMap = Maps.uniqueIndex(ImmutableList.copyOf(PresentationChannel.values()), new Function<PresentationChannel, String>() {
public String apply(PresentationChannel input) {
return input.getChannelCode();
}});
and, in the enum:
public static PresentationChannel findByChannelCode(String channelCode) {
return enumMap.get(channelCode);
}
I think you're using non-JDK classes here right?
A similar solution with JDK API:
private static final Map<String, PresentationChannel> channels = new HashMap<String, PresentationChannel>();
static{
for (PresentationChannel channel : values()){
channels.put(channel.getChannelCode(), channel);
}
}
I wanted to get some of the sage wisdom of this group: what is the proper way to index an enum by value?
Quite possibly not doing it at all.
While hash tables provide O(1) lookup, they also have quite a large constant overhead (for hash calculations etc), so for small collections a linear search may well be faster (if "the efficient way" is your definition of "the proper way").
If you just want a DRY way to do it, I suppose Guava's Iterables.find is an alternative:
return channelCode == null ? null : Iterables.find(Arrays.asList(values()),
new Predicate<PresentationChannel>() {
public boolean apply(PresentationChannel input) {
return input.getChannelCode().equals(channelCode);
}
}, null);
Why don't you name your members A, B, C, D, E and use valueOf?
I was looking for something similar and found on this site a simple, clean and straight to the point way. Create and initialize a static final map inside your enum and add a static method for the lookup, so it would be something like:
public enum PresentationChannel {
ChannelA("A"),
ChannelB("B"),
ChannelC("C"),
ChannelD("D"),
ChannelE("E");
private String channelCode;
PresentationChannel(String channelCode) {
this.channelCode = channelCode;
}
public String getChannelCode() {
return this.channelCode;
}
private static final Map<String, PresentationChannel> lookup
= new HashMap<String, PresentationChannel>();
static {
for(PresentationChannel pc : EnumSet.allOf(PresentationChannel.class)) {
lookup.put(pc.getChannelCode(), pc);
}
}
public static PresentationChannel get(String channelCode) {
return lookup.get(channelCode);
}
}
for few values that's ok, iteration through the values array(). One note only: use smth like that. values() clones the array on each invocation.
static final PresentationChannel[] values=values();
static PresentationChannel getByCode(String code){
if (code==null)
return null;
for(PresentationChannel channel: values) if (code.equals(channel.channelCode)) return channel;
return null;
}
if you have more Channels.
private static final Map<String code, PresentationChannel> map = new HashMap<String code, PresentationChannel>();
static{//hashmap sucks a bit, esp if you have some collisions so you might need to initialize the hashmap depending on the values count and w/ some arbitrary load factor
for(PresentationChannel channel: values()) map.put(channel.channelCode, channel);
}
static PresentationChannel getByCode(String code){
return map.get(code);
}
Edit:
So implement an helper interface, like shown below, another example why java syntax generics blows and sometimes - better not used.
Usage PresentationChannel channel = EnumRepository.get(PresentationChannel.class, "A");
There will be overhead but well, it's quite fool proof.
public interface Identifiable<T> {
T getId();
public static class EnumRepository{
private static final ConcurrentMap<Class<? extends Identifiable<?>>, Map<?, ? extends Identifiable<?>>> classMap = new ConcurrentHashMap<Class<? extends Identifiable<?>>, Map<?,? extends Identifiable<?>>>(16, 0.75f, 1);
#SuppressWarnings("unchecked")
public static <ID, E extends Identifiable<ID>> E get(Class<E> clazz, ID value){
Map<ID, E> map = (Map<ID, E>) classMap.get(clazz);
if (map==null){
map=buildMap(clazz);
classMap.putIfAbsent(clazz, map);
}
return map.get(value);
}
private static <ID, E extends Identifiable<ID>> Map<ID, E> buildMap( Class<E> clazz){
E[] enumConsts = clazz.getEnumConstants();
if (enumConsts==null)
throw new IllegalArgumentException(clazz+ " is not enum");
HashMap<ID, E> map = new HashMap<ID, E>(enumConsts.length*2);
for (E e : enumConsts){
map.put(e.getId(), e);
}
return map;
}
}
}
enum X implements Identifiable<String>{
...
public String getId(){...}
}
Minor warning: if you put Identifiable somewhere out there, and many projects/wepapp depend on it (and share it) and so on, it's possible to leak classes/classloaders.
Here is another way to implement an unmodifiable map:
protected static final Map<String, ChannelCode> EnumMap;
static {
Map<String, ChannelCode> tempMap = new HashMap<String, ChannelCode>();
tempMap.put("A", ChannelA);
tempMap.put("B", ChannelB);
tempMap.put("C", ChannelC);
tempMap.put("D", ChannelD);
tempMap.put("E", ChannelE);
EnumMap = Collections.unmodifiableMap(tempMap);
}
You can use EnumMap.get(someCodeAthroughE) to quickly retrieve the ChannelCode. If the expression is null then your someCodeAthroughE was not found.
If you are expecting the provided channelCode to always be valid then you can just try and get the correct instance of the enum using the valueOf() method. If the provided value is invalid you can return null or propagate the exception.
try {
return PresentationChannel.valueOf(channelCode);
catch (IllegalArgumentException e) {
//do something.
}