I am developing an application where I would like to use the observer pattern in the following way:
I have 2 classes:
public abstract class Storage<V>{
private Set<V> values;
private String filename;
protected Storage(String filename) throws ClassNotFoundException, IOException {
values = new HashSet<>();
this.filename = filename;
load();
}
...
public boolean add(V v) throws IllegalArgumentException {
if (values.contains(v))
throw new IllegalArgumentException("L'elemento è già presente");
return values.add(v);
}
...
}
Repository which is a class for saving a collection of Objects. below is a subclass that implements the singleton pattern (the others are practically the same, only the specified generic type changes)
public class AccountStorage extends Storage<Account>{
private static AccountStorage instance = null;
private AccountStorage(String filename) throws ClassNotFoundException, IOException {
super(filename);
}
public static synchronized AccountStorage getInstance() throws ClassNotFoundException, IOException {
if (instance == null) {
String savefile = "accounts.ob";
instance = new AccountStorage(savefile);
}
return instance;
}
after which I have a controller class (Controller for Spring MVC) which through a post request receives an Account in JSON format, deserializes it and adds it to the collection (Tremite the AccountStorage class) like this:
#PostMapping(value = "new/user", consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
#ResponseBody
public ResponseEntity<String> newAccount(#RequestBody Account a) {
synchronized (accounts) {
try {
accounts.add(a);
// accounts.save()
} catch (IllegalArgumentException e) {
return new ResponseEntity<String>(e.getMessage(), HttpStatus.BAD_REQUEST);
} catch (IOException e) {
return new ResponseEntity<String>(e.getMessage(), HttpStatus.INTERNAL_SERVER_ERROR);
}
}
}
where accounts is: AccountStorage accounts = AccountStorage.getInstance();
I would like to make sure that, after each addition (or other methods that modify the collection) it is saved to file without calling the function affixed each time after the modification.
My idea is to use the Observer pattern. But I don't know which class must be an Observer and which Observable (assuming this approach is the correct solution).
The common practice for implementing the Observer pattern is to define an Observer interface (Listener) which will declare a general contact and each observer-implementation should provide an action which would be triggered whenever an event occurs.
A subject maintains a collection of observers (listeners), and exposes methods which allow to add and remove (subscribe/unsubscribe) an observer. Event-related behavior resides in the subject, and when a new event happens, every subscribed observer (i.e. each observer that is currently present in the collection) will be notified.
An event to which we are going to listen to is a case when a new Account gets added into an AccountStorage. And AccountStorage would be a subject. That implies that AccountStorage should hold a reference to a collection of observers, provide a functionality to subscribe/unsubscribe and override method add() of the Storage class in order to trigger all the observers when a new account will be added.
Why can't we add a collection of observers and all related functionality into the Storage class so that every implementation will inherit it? It's a valid question, the answer is that in such a scenario we can't be specific in regard to the nature of the event because we even don't know its type - method add(V) expects a mysterious V. Hence, the observer interface and its method would be faceless. It was the downside of the standard interfaces Observer and Observable that are deprecated since JDK version 9. Their names as well as the method-name update() tell nothing about an event that would be observed. It's only slightly better than define an interface MyInterface with a method myMethod() - no clue where you can use it and what actions should follow when myMethod() is fired.
It's a good practice when names of observers are descriptive, so that it's clear without looking at the code what they are meant to do. And it's not only related to the Observer pattern, it is a general practice which is called a self-documenting code.
Let's start by defining an observer interface, I'll call it listener just because AccountAddedListener sounds a bit smoothly, and it's quite common to use the terms listener and observer interchangeably.
public interface AccountAddedListener {
void onAccountAdded(Account account);
}
Now let's proceed with an implementation of the observer, let's say we need a notification manager:
public class NotificationManager implements AccountAddedListener {
#Override
public void onAccountAdded(Account account) {
// send a notification message
}
}
Now it's time to turn the AccountStorage into a subject. It should maintain a reference collection of observers, Set is a good choice because it'll not allow to add the same observer twice (which would be pointless) and is able to add and remove elements in a constant time.
Whenever a new account gets added, subject iterates over the collection of observers and invokes onAccountAdded() method on each of them.
We need to define a method to add a new observer, and it's also good practice to add another one to be able to unregister the observer when it's no longer needed.
public class AccountStorage extends Storage<Account> {
private Set<AccountAddedListener> listeners = new HashSet<>(); // collection of observers
#Override
public boolean add(Account account) throws IllegalArgumentException {
listeners.forEach(listener -> listener.onAccountAdded(account)); // notifying observers
return super.add(account);
}
public boolean registerAccountAddedListener(AccountAddedListener listener) {
return listeners.add(listener);
}
public boolean unregisterAccountAddedListener(AccountAddedListener listener) {
return listeners.remove(listener);
}
// all other functionality of the AccountStorage
}
I'm trying to implement (just starting work with Java and Flink) a non-keyed state in KafkaConsumer object, since in this stage no keyBy() in called. This object is the front end and the first module to handle messages from Kafka.
SourceOutput is a proto file representing the message.
I have the KafkaConsumer object :
public class KafkaSourceFunction extends ProcessFunction<byte[], SourceOutput> implements Serializable
{
#Override
public void processElement(byte[] bytes, ProcessFunction<byte[], SourceOutput>.Context
context, Collector<SourceOutput> collector) throws Exception
{
// Here, I want to call to sorting method
collector.collect(output);
}
}
I have an object (KafkaSourceSort) that do all the sorting and should keep the unordered message in priorityQ in the state and also responsible to deliver the message if it comes in the right order thru the collector.
class SessionInfo
{
public PriorityQueue<SourceOutput> orderedMessages = null;
public void putMessage(SourceOutput Msg)
{
if(orderedMessages == null)
orderedMessages = new PriorityQueue<SourceOutput>(new SequenceComparator());
orderedMessages.add(Msg);
}
}
public class KafkaSourceState implements Serializable
{
public TreeMap<String, SessionInfo> Sessions = new TreeMap<>();
}
I read that I need to use a non-keyed state (ListState) which should contain a map of sessions while each session contains a priorityQ holding all messages related to this session.
I found an example so I implement this:
public class KafkaSourceSort implements SinkFunction<KafkaSourceSort>,
CheckpointedFunction
{
private transient ListState<KafkaSourceState> checkpointedState;
private KafkaSourceState state;
#Override
public void snapshotState(FunctionSnapshotContext functionSnapshotContext) throws Exception
{
checkpointedState.clear();
checkpointedState.add(state);
}
#Override
public void initializeState(FunctionInitializationContext context) throws Exception
{
ListStateDescriptor<KafkaSourceState> descriptor =
new ListStateDescriptor<KafkaSourceState>(
"KafkaSourceState",
TypeInformation.of(new TypeHint<KafkaSourceState>() {}));
checkpointedState = context.getOperatorStateStore().getListState(descriptor);
if (context.isRestored())
{
state = (KafkaSourceState) checkpointedState.get();
}
}
#Override
public void invoke(KafkaSourceState value, SinkFunction.Context contex) throws Exception
{
state = value;
// ...
}
}
I see that I need to implement an invoke message which probably will be called from processElement() but the signature of invoke() doesn't contain the collector and I don't understand how to do so or even if I did OK till now.
Please, a help will be appreciated.
Thanks.
A SinkFunction is a terminal node in the DAG that is your job graph. It doesn't have a Collector in its interface because it cannot emit anything downstream. It is expected to connect to an external service or data store and send data there.
If you share more about what you are trying to accomplish perhaps we can offer more assistance. There may be an easier way to go about this.
I have to test a method which uses a mutable object
private final List<LogMessage> buffer;
...
flushBuffer() {
sender.send(buffer);
buffer.clear();
}
I need to test that it sends buffers with exact size.
ArgumentCaptor is not applicable because the captured collection is clear by the time of assertion.
Is there a kind of matcher which can reuse Hamcrest's hasSize() and does check right in time of method call?
I would prefer something like this hypothetical collectionWhich matcher:
bufferedSender.flushBuffer();
verify(sender).send(collectionWhich(hasSize(5)));
A lightweight alternative to David's idea: Use an Answer to make a copy at the time of the call. Untested code, but this should be pretty close:
final List<LogMessage> capturedList = new ArrayList<>();
// This uses a lambda, but you could also do it with an anonymous inner class:
// new Answer<Void>() {
// #Override public Void answer(InvocationOnMock invocation) { /* ... */ }
// }
when(sender.send(any())).thenAnswer(invocation -> {
List<LogMessage> argument = (List<LogMessage>) invocation.getArguments()[0];
capturedList.addAll(argument);
});
bufferedSender.flushBuffer();
assertThat(capturedList).hasSize(5);
The Jeff Bowman answer is fine but I think that we can improve it by inlining the assertion in the Answer object itself. It avoids creating unnecessary copy objects and additional local variable(s).
Besides in cases of we need to copy the state of custom objects (by performing a deep copy of it), this way is much simpler. Indeed, it doesn't require any custom code or library to perform the copies as the assertion is done on the fly.
In Java 8, it would give :
import static org.mockito.Mockito.*;
when(sender.send(any())).thenAnswer(invocation -> {
List<LogMessage> listAtMockTime = invocation.getArguments()[0];
Assert.assertEquals(5, listAtMockTime.getSize());
});
bufferedSender.flushBuffer();
Note that InvocationOnMock.getArgument(int index) returns an unbounded wildcard (?). So no cast is required from the caller as the returned type is defined by the target : here the declared variable for which one we assign the result.
You would have the same issue than with ArgumenCaptor as the verify() method checks the invocation with the state of the object after the execution. No capture is performed to keep only the state at the invocation time.
So with a mutable object I think that a better way would be to not use Mockito and instead create a stub of the Sender class where you capture the actual size of the collection as send() is invoked.
Here is a sample stub class (minimal example that you could of course enrich/adapt) :
class SenderStub extends Sender {
private int bufferSize;
private boolean isSendInvoked;
public int getBufferSize() {
return bufferSize;
}
public boolean isSendInvoked(){
return isSendInvoked;
}
#Override
public void send(List<LogMessage> buffer ) {
this.isSendInvoked = true;
this.bufferSize = buffer.size();
}
}
Now you have a way to check whether the Sender was invoked and the size (or even more) of that.
And so put aside Mockito to create this mock and verify its behavior :
SenderStub sender = new SenderStub();
MyClassToTest myClass = new MyClassToTest(sender);
// action
myClass.flushBuffer();
// assertion
Assert.assertTrue(sender.isInvoked());
Assert.assertEquals(5, sender.getBufferSize());
Since it is still not 100% clear when a LDM should be used I tried a simple memory test.
I created a DataView with a DataProvider that simply creates a list of few 100 entities with some big data inside (long String):
private class HeavyDataProvider implements IDataProvider<HeavyBean> {
#Override
public void detach() {
}
#Override
public Iterator<? extends HeavyBean> iterator(int first, int count) {
List<HeavyBean> l = newArrayList();
for (int i = 0; i < this.size(); i++) {
l.add(new HeavyBean());
}
return l.iterator();
}
#Override
public IModel<HeavyBean> model(HeavyBean heavyBean) {
return new CompoundPropertyModel<HeavyBean>(heavyBean);
}
#Override
public int size() {
return 500;
}
}
Using wicket's DebugBar is see this creates a Page with a size of 5MB. In the javadoc of DataProvider it is stated that the model return in above model method is usually a detachable one so I changed this method to:
#Override
public IModel<HeavyBean> model(final HeavyBean heavyBean) {
return new LoadableDetachableModel<HeavyBean>() {
#Override
protected HeavyBean load() {
return heavyBean;
}
};
}
Naively I was expecting the pagesize to be reduced in a big way now since the heavybeans will no longer be part of the model. Actual result: 5MB. Since the model will detach the heavyBean this must mean that something else still has a hold of it (DataView? Item?).
I saw other examples where DataView and DataProvider are combined in a similar fashion but for me it is unclear what the point is since it does not seem to make any difference regarding the pageSize/memory usage.
So, am I understanding/doing something wrong (likely) or are LDM's useless in DataProviders?
Side question (sorry), in which scenario would you use an LDM?
Your implementation of LDM is just plain wrong. It is holding a direct reference to the bean itself, and just returning it. This way, the bean will be serialized along the model, making it completely pointless.
You should do something like this:
#Override
public IModel<HeavyBean> model(final HeavyBean heavyBean) {
final Integer id = heavyBean.getId();
return new LoadableDetachableModel<HeavyBean>() {
#Override
protected HeavyBean load() {
return ServiceLocator.get(HeavyDao.class).get(id);
}
};
}
If you use the wicket-ioc module, the HeavyDao reference could be injected into the enclosing page/component.
I think Wicket is really easy to use, but you must understand the basics of Java serialization, or else you may end up with a very bloated http session.
For the LDM to work, you will have to actually detach the data in the detach() method. LDMs are meant to be used with databases, where you can restore / load the data on the next request with only the knowledge of an ID. So, in detach() you would trow away all data but the ID (or watever you need to relaod the data when needed) and in the load() (is this right? can't lock up the api right now) you will restore the data.
Hope that helps.
let's say we have a CountryList object in our application that should return the list of countries. The loading of countries is a heavy operation, so the list should be cached.
Additional requirements:
CountryList should be thread-safe
CountryList should load lazy (only on demand)
CountryList should support the invalidation of the cache
CountryList should be optimized considering that the cache will be invalidated very rarely
I came up with the following solution:
public class CountryList {
private static final Object ONE = new Integer(1);
// MapMaker is from Google Collections Library
private Map<Object, List<String>> cache = new MapMaker()
.initialCapacity(1)
.makeComputingMap(
new Function<Object, List<String>>() {
#Override
public List<String> apply(Object from) {
return loadCountryList();
}
});
private List<String> loadCountryList() {
// HEAVY OPERATION TO LOAD DATA
}
public List<String> list() {
return cache.get(ONE);
}
public void invalidateCache() {
cache.remove(ONE);
}
}
What do you think about it? Do you see something bad about it? Is there other way to do it? How can i make it better? Should i look for totally another solution in this cases?
Thanks.
google collections actually supplies just the thing for just this sort of thing: Supplier
Your code would be something like:
private Supplier<List<String>> supplier = new Supplier<List<String>>(){
public List<String> get(){
return loadCountryList();
}
};
// volatile reference so that changes are published correctly see invalidate()
private volatile Supplier<List<String>> memorized = Suppliers.memoize(supplier);
public List<String> list(){
return memorized.get();
}
public void invalidate(){
memorized = Suppliers.memoize(supplier);
}
Thanks you all guys, especially to user "gid" who gave the idea.
My target was to optimize the performance for the get() operation considering the invalidate() operation will be called very rare.
I wrote a testing class that starts 16 threads, each calling get()-Operation one million times. With this class I profiled some implementation on my 2-core maschine.
Testing results
Implementation Time
no synchronisation 0,6 sec
normal synchronisation 7,5 sec
with MapMaker 26,3 sec
with Suppliers.memoize 8,2 sec
with optimized memoize 1,5 sec
1) "No synchronisation" is not thread-safe, but gives us the best performance that we can compare to.
#Override
public List<String> list() {
if (cache == null) {
cache = loadCountryList();
}
return cache;
}
#Override
public void invalidateCache() {
cache = null;
}
2) "Normal synchronisation" - pretty good performace, standard no-brainer implementation
#Override
public synchronized List<String> list() {
if (cache == null) {
cache = loadCountryList();
}
return cache;
}
#Override
public synchronized void invalidateCache() {
cache = null;
}
3) "with MapMaker" - very poor performance.
See my question at the top for the code.
4) "with Suppliers.memoize" - good performance. But as the performance the same "Normal synchronisation" we need to optimize it or just use the "Normal synchronisation".
See the answer of the user "gid" for code.
5) "with optimized memoize" - the performnce comparable to "no sync"-implementation, but thread-safe one. This is the one we need.
The cache-class itself:
(The Supplier interfaces used here is from Google Collections Library and it has just one method get(). see http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/base/Supplier.html)
public class LazyCache<T> implements Supplier<T> {
private final Supplier<T> supplier;
private volatile Supplier<T> cache;
public LazyCache(Supplier<T> supplier) {
this.supplier = supplier;
reset();
}
private void reset() {
cache = new MemoizingSupplier<T>(supplier);
}
#Override
public T get() {
return cache.get();
}
public void invalidate() {
reset();
}
private static class MemoizingSupplier<T> implements Supplier<T> {
final Supplier<T> delegate;
volatile T value;
MemoizingSupplier(Supplier<T> delegate) {
this.delegate = delegate;
}
#Override
public T get() {
if (value == null) {
synchronized (this) {
if (value == null) {
value = delegate.get();
}
}
}
return value;
}
}
}
Example use:
public class BetterMemoizeCountryList implements ICountryList {
LazyCache<List<String>> cache = new LazyCache<List<String>>(new Supplier<List<String>>(){
#Override
public List<String> get() {
return loadCountryList();
}
});
#Override
public List<String> list(){
return cache.get();
}
#Override
public void invalidateCache(){
cache.invalidate();
}
private List<String> loadCountryList() {
// this should normally load a full list from the database,
// but just for this instance we mock it with:
return Arrays.asList("Germany", "Russia", "China");
}
}
Whenever I need to cache something, I like to use the Proxy pattern.
Doing it with this pattern offers separation of concerns. Your original
object can be concerned with lazy loading. Your proxy (or guardian) object
can be responsible for validation of the cache.
In detail:
Define an object CountryList class which is thread-safe, preferably using synchronization blocks or other semaphore locks.
Extract this class's interface into a CountryQueryable interface.
Define another object, CountryListProxy, that implements the CountryQueryable.
Only allow the CountryListProxy to be instantiated, and only allow it to be referenced
through its interface.
From here, you can insert your cache invalidation strategy into the proxy object. Save the time of the last load, and upon the next request to see the data, compare the current time to the cache time. Define a tolerance level, where, if too much time has passed, the data is reloaded.
As far as Lazy Load, refer here.
Now for some good down-home sample code:
public interface CountryQueryable {
public void operationA();
public String operationB();
}
public class CountryList implements CountryQueryable {
private boolean loaded;
public CountryList() {
loaded = false;
}
//This particular operation might be able to function without
//the extra loading.
#Override
public void operationA() {
//Do whatever.
}
//This operation may need to load the extra stuff.
#Override
public String operationB() {
if (!loaded) {
load();
loaded = true;
}
//Do whatever.
return whatever;
}
private void load() {
//Do the loading of the Lazy load here.
}
}
public class CountryListProxy implements CountryQueryable {
//In accordance with the Proxy pattern, we hide the target
//instance inside of our Proxy instance.
private CountryQueryable actualList;
//Keep track of the lazy time we cached.
private long lastCached;
//Define a tolerance time, 2000 milliseconds, before refreshing
//the cache.
private static final long TOLERANCE = 2000L;
public CountryListProxy() {
//You might even retrieve this object from a Registry.
actualList = new CountryList();
//Initialize it to something stupid.
lastCached = Long.MIN_VALUE;
}
#Override
public synchronized void operationA() {
if ((System.getCurrentTimeMillis() - lastCached) > TOLERANCE) {
//Refresh the cache.
lastCached = System.getCurrentTimeMillis();
} else {
//Cache is okay.
}
}
#Override
public synchronized String operationB() {
if ((System.getCurrentTimeMillis() - lastCached) > TOLERANCE) {
//Refresh the cache.
lastCached = System.getCurrentTimeMillis();
} else {
//Cache is okay.
}
return whatever;
}
}
public class Client {
public static void main(String[] args) {
CountryQueryable queryable = new CountryListProxy();
//Do your thing.
}
}
Your needs seem pretty simple here. The use of MapMaker makes the implementation more complicated than it has to be. The whole double-checked locking idiom is tricky to get right, and only works on 1.5+. And to be honest, it's breaking one of the most important rules of programming:
Premature optimization is the root of
all evil.
The double-checked locking idiom tries to avoid the cost of synchronization in the case where the cache is already loaded. But is that overhead really causing problems? Is it worth the cost of more complex code? I say assume it is not until profiling tells you otherwise.
Here's a very simple solution that requires no 3rd party code (ignoring the JCIP annotation). It does make the assumption that an empty list means the cache hasn't been loaded yet. It also prevents the contents of the country list from escaping to client code that could potentially modify the returned list. If this is not a concern for you, you could remove the call to Collections.unmodifiedList().
public class CountryList {
#GuardedBy("cache")
private final List<String> cache = new ArrayList<String>();
private List<String> loadCountryList() {
// HEAVY OPERATION TO LOAD DATA
}
public List<String> list() {
synchronized (cache) {
if( cache.isEmpty() ) {
cache.addAll(loadCountryList());
}
return Collections.unmodifiableList(cache);
}
}
public void invalidateCache() {
synchronized (cache) {
cache.clear();
}
}
}
I'm not sure what the map is for. When I need a lazy, cached object, I usually do it like this:
public class CountryList
{
private static List<Country> countryList;
public static synchronized List<Country> get()
{
if (countryList==null)
countryList=load();
return countryList;
}
private static List<Country> load()
{
... whatever ...
}
public static synchronized void forget()
{
countryList=null;
}
}
I think this is similar to what you're doing but a little simpler. If you have a need for the map and the ONE that you've simplified away for the question, okay.
If you want it thread-safe, you should synchronize the get and the forget.
What do you think about it? Do you see something bad about it?
Bleah - you are using a complex data structure, MapMaker, with several features (map access, concurrency-friendly access, deferred construction of values, etc) because of a single feature you are after (deferred creation of a single construction-expensive object).
While reusing code is a good goal, this approach adds additional overhead and complexity. In addition, it misleads future maintainers when they see a map data structure there into thinking that there's a map of keys/values in there when there is really only 1 thing (list of countries). Simplicity, readability, and clarity are key to future maintainability.
Is there other way to do it? How can i make it better? Should i look for totally another solution in this cases?
Seems like you are after lazy-loading. Look at solutions to other SO lazy-loading questions. For example, this one covers the classic double-check approach (make sure you are using Java 1.5 or later):
How to solve the "Double-Checked Locking is Broken" Declaration in Java?
Rather than just simply repeat the solution code here, I think it is useful to read the discussion about lazy loading via double-check there to grow your knowledge base. (sorry if that comes off as pompous - just trying teach to fish rather than feed blah blah blah ...)
There is a library out there (from atlassian) - one of the util classes called LazyReference. LazyReference is a reference to an object that can be lazily created (on first get). it is guarenteed thread safe, and the init is also guarenteed to only occur once - if two threads calls get() at the same time, one thread will compute, the other thread will block wait.
see a sample code:
final LazyReference<MyObject> ref = new LazyReference() {
protected MyObject create() throws Exception {
// Do some useful object construction here
return new MyObject();
}
};
//thread1
MyObject myObject = ref.get();
//thread2
MyObject myObject = ref.get();
This looks ok to me (I assume MapMaker is from google collections?) Ideally you wouldn't need to use a Map because you don't really have keys but as the implementation is hidden from any callers I don't see this as a big deal.
This is way to simple to use the ComputingMap stuff. You only need a dead simple implementation where all methods are synchronized, and you should be fine. This will obviously block the first thread hitting it (getting it), and any other thread hitting it while the first thread loads the cache (and the same again if anyone calls the invalidateCache thing - where you also should decide whether the invalidateCache should load the cache anew, or just null it out, letting the first attempt at getting it again block), but then all threads should go through nicely.
Use the Initialization on demand holder idiom
public class CountryList {
private CountryList() {}
private static class CountryListHolder {
static final List<Country> INSTANCE = new List<Country>();
}
public static List<Country> getInstance() {
return CountryListHolder.INSTANCE;
}
...
}
Follow up to Mike's solution above. My comment didn't format as expected... :(
Watch out for synchronization issues in operationB, especially since load() is slow:
public String operationB() {
if (!loaded) {
load();
loaded = true;
}
//Do whatever.
return whatever;
}
You could fix it this way:
public String operationB() {
synchronized(loaded) {
if (!loaded) {
load();
loaded = true;
}
}
//Do whatever.
return whatever;
}
Make sure you ALWAYS synchronize on every access to the loaded variable.