We Keep Coding

Take elements until a certain character and group them with RxJava

I have a simple setup to a problem but the solution seems to be more complicated.
Setup: I have a hot observable which originates from a scanner that will emit every number as a different element and an R when a code is complete.
Problem: From this I want a hot observable that emits every full code as 1 element.
I tried playing around with different flatMap, takeUntil and groupByoperators but haven't been able to come to a solution.

You can use the buffer operator.
PublishSubject<Token<Integer>> s = PublishSubject.create();
Observable<Token<Integer>> markers = s.filter(x->x.isMarker());
s.buffer(markers).subscribe(
v->{
Optional<Integer> reduce = v.stream()
.filter(t->!t.isMarker())
.map(t->(ValueToken<Integer>)t)
.map(ValueToken::get)
.reduce((a,b)->a+b);
reduce.ifPresent(System.out::println);
}
);
s.onNext(value(12));
s.onNext(value(13));
s.onNext(marker()); // will emit 25
s.onNext(value(10));
s.onNext(value(7));
s.onNext(marker()); // will emit 17
s.onNext(value(10));
s.onNext(value(7)); // Not emitting yet
I made a class to wrap both values and markers in the flow.
public abstract class Token<T> {
private static final MarkerToken MARKER = new MarkerToken<>();
public boolean isMarker() {
return false;
}
public static <T> MarkerToken<T> marker() {
return MARKER;
}
public static <T> ValueToken<T> value(T o) {
return new ValueToken<>(o);
}
public static class ValueToken<T> extends Token<T> {
T value;
public ValueToken(T value) {
this.value = value;
}
public T get() {
return value;
}
}
public static class MarkerToken<T> extends Token<T> {
public boolean isMarker() {
return true;
}
}
}
update (using scan)
The previous method would emit also on the closing of the stream, with this solution you can emit only complete buffers.
The message class function as an accumulator, it will accumulate tokens until closing marker is accumulated.
When this happens the next message will start from scratch.
The presence of the closing mark as last element marks the message as complete.
public static class Message<T> {
List<Token<T>> tokens = new ArrayList<>();
public Message<T> append(Token<T> t) {
Message<T> mx = new Message<T>();
if(!isComplete()) {
mx.tokens.addAll(tokens);
}
mx.tokens.add(t);
return mx;
}
public boolean isComplete() {
int n = tokens.size();
return n>0 && tokens.get(n-1).isMarker();
}
public Optional<List<Token<T>>> fullMessage(){
return isComplete() ? Optional.of(tokens):Optional.empty();
}
}
Scanning the source you emit a message for each token emitted, then you filter out incomplete message and emit just the one marked as complete.
s.scan(new Message<Integer>(), (a, b) -> a.append(b))
.filter(Message::isComplete)
.map(Message::fullMessage)
.map(Optional::get).subscribe(v -> {
System.out.println(v);
});
s.onNext(value(12));
s.onNext(value(13));
s.onNext(marker());// [V(12), V(13), MARKER]
s.onNext(value(10));
s.onNext(value(7));
s.onNext(marker()); // [V(10), V(7), MARKER]
s.onNext(value(10));
s.onNext(value(127));
s.onComplete(); // Not emitting incomplete messages on the closing of the subject.

Java synchronization depending on method parameter

how can I provide synchronization upon method parameter values?
All method calls using the 'same' parameter value A should be synchronized. A method call with a different parameter value e.g. B can access, even when calls with A are already waiting. The next concurrent call for B must wait also for the first B to be released.
My use case: I want to synchronize the access to JPA entities on ID level but want to avoid pessimistic locking because I need kind of a queue. The 'key' for locking is intended to be the entity ID - which is in fact of the type Java Long.
protected void entityLockedAccess(SomeEntity myEntity) {
//getId() returns different Long objects so the lock does not work
synchronized (myEntity.getId()) {
//the critical section ...
}
}
I read about lock objects but I am not sure how they would suit in my case.
On the top level I want to manage a specific REST call to my application which executes critical code.
Thanks,
Chris

As far as I understood you basically want a different, unique lock for each of your SomeEntity IDs.
You could realize this with a Map<Integer, Object>.
You simply map each ID to an object. Should there already be an object, you reuse it. This could look something like this:
static Map<Integer, Object> locks = new ConcurrentHashMap<>();
public static void main(String[] args)
{
int i1 = 1;
int i2 = 2;
foo(i1);
foo(i1);
foo(i2);
}
public static void foo(int o)
{
synchronized (locks.computeIfAbsent(o, k -> new Object()))
{
// computation
}
}
This will create 2 lock objects in the map as the object for i1 is reused in the second foo(i1) call.

Objects which are pooled and potentially reused should not be used for synchronization. If they are, it can cause unrelated threads to deadlock with unhelpful stacktraces.
Specifically, String literals, and boxed primitives such as Integers should NOT be used as lock objects because they are pooled and reused.
The story is even worse for Boolean objects because there are only two instances of Boolean, Boolean.TRUE and Boolean.FALSE and every class that uses a Boolean will be referring to one of the two.
I read about lock objects but I am not sure how they would suit in my
case. On the top level I want to manage a specific REST call to my
application which executes critical code.
You DB will take care for concurrent writes and other transactional issues.
All you need to do is use Transactions.
I would also recommend you to go through the classical problems (DIRTY READs NON Repeatable reads). You can also use Optimistic Locking for

The problem is that you simply should not synchronize on values (for example strings, or Integer objects).
Meaning: you would need to define some special EntityId class here, and of course, all "data" that uses the same ID would somehow need to be using the same EntityId object then.

private static final Set<Integer> lockedIds = new HashSet<>();
private void lock(Integer id) throws InterruptedException {
synchronized (lockedIds) {
while (!lockedIds.add(id)) {
lockedIds.wait();
}
}
}
private void unlock(Integer id) {
synchronized (lockedIds) {
lockedIds.remove(id);
lockedIds.notifyAll();
}
}
public void entityLockedAccess(SomeEntity myEntity) throws InterruptedException {
try {
lock(myEntity.getId());
//Put your code here.
//For different ids it is executed in parallel.
//For equal ids it is executed synchronously.
} finally {
unlock(myEntity.getId());
}
}
id can be not only an 'Integer' but any class with correctly overridden 'equals' and 'hashCode' methods.
try-finally - is very important - you must guarantee to unlock waiting threads after your operation even if your operation threw exception.
It will not work if your back-end is distributed across multiple servers/JVMs.

Just use this class:
(and the map will NOT increase in size over time)
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Consumer;
public class SameKeySynchronizer<T> {
private final ConcurrentHashMap<T, Object> sameKeyTasks = new ConcurrentHashMap<>();
public void serializeSameKeys(T key, Consumer<T> keyConsumer) {
// This map will never be filled (because function returns null), it is only used for synchronization purposes for the same key
sameKeyTasks.computeIfAbsent(key, inputArgumentKey -> acceptReturningNull(inputArgumentKey, keyConsumer));
}
private Object acceptReturningNull(T inputArgumentKey, Consumer<T> keyConsumer) {
keyConsumer.accept(inputArgumentKey);
return null;
}
}
Like in this test:
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
class SameKeySynchronizerTest {
private static final boolean SHOW_FAILING_TEST = false;
#Test
void sameKeysAreNotExecutedParallel() throws InterruptedException {
TestService testService = new TestService();
TestServiceThread testServiceThread1 = new TestServiceThread(testService, "a");
TestServiceThread testServiceThread2 = new TestServiceThread(testService, "a");
testServiceThread1.start();
testServiceThread2.start();
testServiceThread1.join();
testServiceThread2.join();
Assertions.assertFalse(testService.sameKeyInProgressSimultaneously);
}
#Test
void differentKeysAreExecutedParallel() throws InterruptedException {
TestService testService = new TestService();
TestServiceThread testServiceThread1 = new TestServiceThread(testService, "a");
TestServiceThread testServiceThread2 = new TestServiceThread(testService, "b");
testServiceThread1.start();
testServiceThread2.start();
testServiceThread1.join();
testServiceThread2.join();
Assertions.assertFalse(testService.sameKeyInProgressSimultaneously);
Assertions.assertTrue(testService.differentKeysInProgressSimultaneously);
}
private class TestServiceThread extends Thread {
TestService testService;
String key;
TestServiceThread(TestService testService, String key) {
this.testService = testService;
this.key = key;
}
#Override
public void run() {
testService.process(key);
}
}
private class TestService {
private final SameKeySynchronizer<String> sameKeySynchronizer = new SameKeySynchronizer<>();
private Set<String> keysInProgress = ConcurrentHashMap.newKeySet();
private boolean sameKeyInProgressSimultaneously = false;
private boolean differentKeysInProgressSimultaneously = false;
void process(String key) {
if (SHOW_FAILING_TEST) {
processInternal(key);
} else {
sameKeySynchronizer.serializeSameKeys(key, inputArgumentKey -> processInternal(inputArgumentKey));
}
}
#SuppressWarnings("MagicNumber")
private void processInternal(String key) {
try {
boolean keyInProgress = !keysInProgress.add(key);
if (keyInProgress) {
sameKeyInProgressSimultaneously = true;
}
try {
int sleepTimeInMillis = 100;
for (long elapsedTimeInMillis = 0; elapsedTimeInMillis < 1000; elapsedTimeInMillis += sleepTimeInMillis) {
Thread.sleep(sleepTimeInMillis);
if (keysInProgress.size() > 1) {
differentKeysInProgressSimultaneously = true;
}
}
} catch (InterruptedException e) {
throw new IllegalStateException(e);
}
} finally {
keysInProgress.remove(key);
}
}
}
}

Check number of invocations within class in case of JUnit testing

I have a code which calculates something, caches is, and if already calculated, then reads from the cache; similar to this:
public class LengthWithCache {
private java.util.Map<String, Integer> lengthPlusOneCache = new java.util.HashMap<String, Integer>();
public int getLenghtPlusOne(String string) {
Integer cachedStringLenghtPlusOne = lengthPlusOneCache.get(string);
if (cachedStringLenghtPlusOne != null) {
return cachedStringLenghtPlusOne;
}
int stringLenghtPlusOne = determineLengthPlusOne(string);
lengthPlusOneCache.put(string, new Integer(stringLenghtPlusOne));
return stringLenghtPlusOne;
}
protected int determineLengthPlusOne(String string) {
return string.length() + 1;
}
}
I want to test if function determineLengthPlusOne has been called adequate number of times, like this:
public class LengthWithCacheTest {
#Test
public void testGetLenghtPlusOne() {
LengthWithCache lengthWithCache = new LengthWithCache();
assertEquals(6, lengthWithCache.getLenghtPlusOne("apple"));
// here check that determineLengthPlusOne has been called once
assertEquals(6, lengthWithCache.getLenghtPlusOne("apple"));
// here check that determineLengthPlusOne has not been called
}
}
Mocking class LengthWithCache does not seem a good option, as I want to test their functions. (According to my understanding we mock the classes used by the tested class, and not the tested class itself.) Which is the most elegant solution for this?
My first idea was to create another class LengthPlusOneDeterminer containing function determineLengthPlusOne, add pass it to function getLenghtPlusOne as parameter, and mock LengthPlusOneDeterminer in case of unit testing, but that seems a bit strange, as it has unnecessary impact on the working code (the real clients of class LengthWithCache).
Basically I am using Mockito, but whatever mock framework (or other solution) is welcome! Thank you!

Most elegant way would be to create a separate class that does the caching and decorate with it the current class (after removal of the caching), this way you can safely unit test the caching itself without interfering with functionalities of the base class.
public class Length {
public int getLenghtPlusOne(String string) {
int stringLenghtPlusOne = determineLengthPlusOne(string);
lengthPlusOneCache.put(string, new Integer(stringLenghtPlusOne));
return stringLenghtPlusOne;
}
protected int determineLengthPlusOne(String string) {
return string.length() + 1;
}
}
public class CachedLength extends Length {
private java.util.Map<String, Integer> lengthPlusOneCache = new java.util.HashMap<String, Integer>();
public CachedLength(Length length) {
this.length = length;
}
public int getLenghtPlusOne(String string) {
Integer cachedStringLenghtPlusOne = lengthPlusOneCache.get(string);
if (cachedStringLenghtPlusOne != null) {
return cachedStringLenghtPlusOne;
}
return length.getLenghtPlusOne(string);
}
}
Then you can easily test the caching my injecting a mocked Length:
Length length = Mockito.mock(Length.class);
CachedLength cached = new CachedLength(length);
....
Mockito.verify(length, Mockito.times(5)).getLenghtPlusOne(Mockito.anyInt());

You don't need mock to address your need.
To test the internal behavior (is getLenghtPlusOne() was called or not called), you would need to have a method to access to the cache in LengthWithCache.
But at level of your design, we imagine that you don't want to open the cache in a public method. Which is normal.
Multiple solutions exist to do a test on the cache behavior despite this constraint.
I will present my way of doing. Maybe, there is better.
But I think that in most of cases, you will be forced to use some tricks or to complexify your design to do your unit test.
It relies on augmenting your class to test by extending it in order to add the needed information and behavior for your test.
And it's this subclass you will use in your unit test.
The most important point in this class extension is not to break or modify the behavior of the object to test.
It must add new information and add new behavior and not modify information and behavior of the original class otherwise the test loses its value since it doesn't test any longer the behavior in the original class.
The key points :
- having a private field lengthPlusOneWasCalledForCurrentCallwhich registers for the current call if the method lengthPlusOneWasCalledwas called
- having a public method to know the value of lengthPlusOneWasCalledForCurrentCall for the string used as parameter. It enable the assertion.
- having a public method to clean the state of lengthPlusOneWasCalledForCurrentCall. It enable to keep a clean state after the assertion.
package cache;
import java.util.HashSet;
import java.util.Set;
import org.junit.Assert;
import org.junit.Test;
public class LengthWithCacheTest {
private class LengthWithCacheAugmentedForTest extends LengthWithCache {
private Set<String> lengthPlusOneWasCalledForCurrentCall = new HashSet<>();
#Override
protected int determineLengthPlusOne(String string) {
// start : info for testing
this.lengthPlusOneWasCalledForCurrentCall.add(string);
// end : info for testing
return super.determineLengthPlusOne(string);
}
// method for assertion
public boolean isLengthPlusOneCalled(String string) {
return lengthPlusOneWasCalledForCurrentCall.contains(string);
}
// method added for clean the state of current calls
public void cleanCurrentCalls() {
lengthPlusOneWasCalledForCurrentCall.clear();
}
}
#Test
public void testGetLenghtPlusOne() {
LengthWithCacheAugmentedForTest lengthWithCache = new LengthWithCacheAugmentedForTest();
final String string = "apple";
// here check that determineLengthPlusOne has been called once
Assert.assertEquals(6, lengthWithCache.getLenghtPlusOne(string));
Assert.assertTrue(lengthWithCache.isLengthPlusOneCalled(string));
// clean call registered
lengthWithCache.cleanCurrentCalls();
// here check that determineLengthPlusOne has not been called
Assert.assertEquals(6, lengthWithCache.getLenghtPlusOne(string));
Assert.assertFalse(lengthWithCache.isLengthPlusOneCalled(string));
}
}
Edit 28-07-16 to show why more code is needed to handle more scenarios
Suppose, i will improve the test by asserting that there is no side effects : adding an element in the cache for a key has not effect on how the cache is handled for other keys.
This test fails because it doesn't rely on the string key. So, it always increments.
#Test
public void verifyInvocationCountsWithDifferentElementsAdded() {
final AtomicInteger plusOneInvkCounter = new AtomicInteger();
LengthWithCache lengthWithCache = new LengthWithCache() {
#Override
protected int determineLengthPlusOne(String string) {
plusOneInvkCounter.incrementAndGet();
return super.determineLengthPlusOne(string);
}
};
Assert.assertEquals(0, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne("apple");
Assert.assertEquals(1, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne("pie");
Assert.assertEquals(1, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne("eggs");
Assert.assertEquals(1, plusOneInvkCounter.get());
}
My version is longer because it provides more features and so, it can handle a broader range of unit testing scenarios .
Edit 28-07-16 to point the Integer caching
No direct relation with the original question but little wink :)
Your getLenghtPlusOne(String string) should use Integer.valueOf(int) instead of new Integer(int)
Integer.valueOf(int) uses in internal a cache

It feels like using mocks is overthinking it. The LengthWithCache can be overridden as an anonymous-inner class within the context of a test to get the invocation count. This requires no restructuring of the existing class being tested.
public class LengthWithCacheTest {
#Test
public void verifyLengthEval() {
LengthWithCache lengthWithCache = new LengthWithCache();
assertEquals(6, lengthWithCache.getLenghtPlusOne("apple"));
}
#Test
public void verifyInvocationCounts() {
final AtomicInteger plusOneInvkCounter = new AtomicInteger();
LengthWithCache lengthWithCache = new LengthWithCache() {
#Override
protected int determineLengthPlusOne(String string) {
plusOneInvkCounter.incrementAndGet();
return super.determineLengthPlusOne(string);
}
};
lengthWithCache.getLenghtPlusOne("apple");
assertEquals(1, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne("apple");
lengthWithCache.getLenghtPlusOne("apple");
lengthWithCache.getLenghtPlusOne("apple");
lengthWithCache.getLenghtPlusOne("apple");
lengthWithCache.getLenghtPlusOne("apple");
lengthWithCache.getLenghtPlusOne("apple");
assertEquals(1, plusOneInvkCounter.get());
}
}
It's worth noting the separation between the two tests. One verifies
that the length eval is right, the other verifies the invocation
count.
If a wider data set for validation is required, then you can turn the Test above into a Parameterized test and provide multiple data sets and expectations. In the sample below I've added a data set of 50 strings (lengths 1-50), an empty string, and a null value.
Null fails
#RunWith(Parameterized.class)
public class LengthWithCacheTest {
#Parameters(name="{0}")
public static Collection<Object[]> buildTests() {
Collection<Object[]> paramRefs = new ArrayList<Object[]>();
paramRefs.add(new Object[]{null, 0});
paramRefs.add(new Object[]{"", 1});
for (int counter = 1 ; counter < 50; counter++) {
String data = "";
for (int index = 0 ; index < counter ; index++){
data += "a";
}
paramRefs.add(new Object[]{data, counter+1});
}
return paramRefs;
}
private String stringToTest;
private int expectedLength;
public LengthWithCacheTest(String string, int length) {
this.stringToTest = string;
this.expectedLength = length;
}
#Test
public void verifyLengthEval() {
LengthWithCache lengthWithCache = new LengthWithCache();
assertEquals(expectedLength, lengthWithCache.getLenghtPlusOne(stringToTest));
}
#Test
public void verifyInvocationCounts() {
final AtomicInteger plusOneInvkCounter = new AtomicInteger();
LengthWithCache lengthWithCache = new LengthWithCache() {
#Override
protected int determineLengthPlusOne(String string) {
plusOneInvkCounter.incrementAndGet();
return super.determineLengthPlusOne(string);
}
};
assertEquals(0, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne(stringToTest);
assertEquals(1, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne(stringToTest);
assertEquals(1, plusOneInvkCounter.get());
lengthWithCache.getLenghtPlusOne(stringToTest);
assertEquals(1, plusOneInvkCounter.get());
}
}
Parameterized testing is one of the best ways to vary your data set through a test, but it adds complexity to the test and can be difficult to maintain. It's useful to know about, but not always the right tool for the job.

As this was an interesting question, I decided to write the tests. In two different ways, one with mocking and the other without. (Personally, I prefer the version without mocking.) In either case, the original class is tested, with no modifications:
package example;
import mockit.*;
import org.junit.*;
import static org.junit.Assert.*;
public class LengthWithCacheMockedTest {
#Tested(availableDuringSetup = true) #Mocked LengthWithCache lengthWithCache;
#Before
public void recordComputedLengthPlusOneWhileFixingTheNumberOfAllowedInvocations() {
new Expectations() {{
lengthWithCache.determineLengthPlusOne(anyString); result = 6; times = 1;
}};
}
#Test
public void getLenghtPlusOneNotFromCacheWhenCalledTheFirstTime() {
int length = lengthWithCache.getLenghtPlusOne("apple");
assertEquals(6, length);
}
#Test
public void getLenghtPlusOneFromCacheWhenCalledAfterFirstTime() {
int length1 = lengthWithCache.getLenghtPlusOne("apple");
int length2 = lengthWithCache.getLenghtPlusOne("apple");
assertEquals(6, length1);
assertEquals(length1, length2);
}
}
package example;
import mockit.*;
import org.junit.*;
import static org.junit.Assert.*;
public class LengthWithCacheNotMockedTest {
#Tested LengthWithCache lengthWithCache;
#Test
public void getLenghtPlusOneNotFromCacheWhenCalledTheFirstTime() {
long t0 = System.currentTimeMillis(); // millisecond precision is enough here
int length = lengthWithCache.getLenghtPlusOne("apple");
long dt = System.currentTimeMillis() - t0;
assertEquals(6, length);
assertTrue(dt >= 100); // assume at least 100 millis to compute the expensive value
}
#Test
public void getLenghtPlusOneFromCacheWhenCalledAfterFirstTime() {
// First time: takes some time to compute.
int length1 = lengthWithCache.getLenghtPlusOne("apple");
// Second time: gets from cache, takes no time.
long t0 = System.nanoTime(); // max precision here
int length2 = lengthWithCache.getLenghtPlusOne("apple");
long dt = System.nanoTime() - t0;
assertEquals(6, length1);
assertEquals(length1, length2);
assertTrue(dt < 1000000); // 1000000 nanos = 1 millis
}
}
Just one detail: for the tests above to work, I added the following line inside the LengthWithCache#determineLengthPlusOne(String) method, in order to simulate the real-world scenario where the computation takes some time:
try { Thread.sleep(100); } catch (InterruptedException ignore) {}

Based on the proposal by krzyk here is my fully working solution:
The calculator itself:
public class LengthPlusOneCalculator {
public int calculateLengthPlusOne(String string) {
return string.length() + 1;
}
}
The separate caching mechanism:
public class LengthPlusOneCache {
private LengthPlusOneCalculator lengthPlusOneCalculator;
private java.util.Map<String, Integer> lengthPlusOneCache = new java.util.HashMap<String, Integer>();
public LengthPlusOneCache(LengthPlusOneCalculator lengthPlusOneCalculator) {
this.lengthPlusOneCalculator = lengthPlusOneCalculator;
}
public int calculateLenghtPlusOne(String string) {
Integer cachedStringLenghtPlusOne = lengthPlusOneCache.get(string);
if (cachedStringLenghtPlusOne != null) {
return cachedStringLenghtPlusOne;
}
int stringLenghtPlusOne = lengthPlusOneCalculator.calculateLengthPlusOne(string);
lengthPlusOneCache.put(string, new Integer(stringLenghtPlusOne));
return stringLenghtPlusOne;
}
}
The unit test for checking the LengthPlusOneCalculator:
import static org.junit.Assert.assertEquals;
import org.junit.Test;
public class LengthPlusOneCalculatorTest {
#Test
public void testCalculateLengthPlusOne() {
LengthPlusOneCalculator lengthPlusOneCalculator = new LengthPlusOneCalculator();
assertEquals(6, lengthPlusOneCalculator.calculateLengthPlusOne("apple"));
}
}
And finally, the unit test for LengthPlusOneCache, checking the number of invocations:
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.*;
import org.junit.Test;
public class LengthPlusOneCacheTest {
#Test
public void testNumberOfInvocations() {
LengthPlusOneCalculator lengthPlusOneCalculatorMock = mock(LengthPlusOneCalculator.class);
when(lengthPlusOneCalculatorMock.calculateLengthPlusOne("apple")).thenReturn(6);
LengthPlusOneCache lengthPlusOneCache = new LengthPlusOneCache(lengthPlusOneCalculatorMock);
verify(lengthPlusOneCalculatorMock, times(0)).calculateLengthPlusOne("apple"); // verify that not called yet
assertEquals(6, lengthPlusOneCache.calculateLenghtPlusOne("apple"));
verify(lengthPlusOneCalculatorMock, times(1)).calculateLengthPlusOne("apple"); // verify that already called once
assertEquals(6, lengthPlusOneCache.calculateLenghtPlusOne("apple"));
verify(lengthPlusOneCalculatorMock, times(1)).calculateLengthPlusOne("apple"); // verify that not called again
}
}
We can safely do the mocking mechanism, as we are already convinced that the mocked class works properly, using its own unit tests.
Normally this is built into a build system; this example can be complied and run from command line as follows (files junit-4.10.jar and mockito-all-1.9.5.jar have to be copied to the working directory):
javac -cp .;junit-4.10.jar;mockito-all-1.9.5.jar *.java
java -cp .;junit-4.10.jar org.junit.runner.JUnitCore LengthPlusOneCalculatorTest
java -cp .;junit-4.10.jar;mockito-all-1.9.5.jar org.junit.runner.JUnitCore LengthPlusOneCacheTest
However, I'm still not fully satisfied with this approach. My issues are the following:
Function calculateLengthPlusOne is mocked. I would prefer such a solution where a mocking or whatever framework just calculates the number of invocations, but the original code runs. (Somehow mentioned by davidhxxx, however I do not find also that a perfect one.)
The code became a bit over-complicated. This is not the way one would create normally. Therefore this approach is not adequate if the original code is not of our fully control. This could be a constraint in reality.
Normally I would make function calculateLengthPlusOne static. This approach does not work in such a case. (But maybe my Mockito knowledge is weak.)
If some could address any of these issues, I would really appreciate it!

How do I make async cache-on-top-of-database query-with-update using vertx3

Below is an illustration on how an in-memory cache can be used on top of a database, using async vertx-mongo-client. It's made as minimalistic as possible. It is fully executable, with the important stuff at the top.
Unfortunately there's nothing there to prevent multiple database accesses and cache insertions for the same key, while the cache entry is established. Using standard locking mechanisms is out of the question because it would block the event loop.
I need an async cache-and-database call, so that the cache.get() calls, subsequent to the first call that will actually hit the db, will 'return to the event-loop' (can I say that?), 'waiting' for the cache entry to become available. (What a horrible sentence, my apologies..)
How do I achieve that? I was thinking about studying some vertx module source code, eg. vertx-mongo-client, to learn how it is done. But if somebody is able to provide an answer here, it would be great.
package q;
import io.vertx.core.AbstractVerticle;
import io.vertx.core.Vertx;
import io.vertx.core.json.JsonObject;
import io.vertx.ext.mongo.MongoClient;
import java.util.HashMap;
import java.util.Map;
public class ExampleVerticle extends AbstractVerticle
{
#Override
public void start() throws Exception {
MongoClient mongoClient = MongoClient.createShared(vertx, new JsonObject().put("db_name", "example_db"));
SomeCache cache = new SomeCache();
vertx.eventBus().consumer("sesame_street", messageHandler -> {
Integer lookUpKey = Integer.valueOf(messageHandler.body().toString());
JsonObject result = cache.get(lookUpKey);
if(result != null) {
messageHandler.reply(result);
System.out.println("Was served from cache");
} else {
mongoClient.findOne("example_collection", new JsonObject().put("_id", lookUpKey),
new JsonObject(), resultHandler -> {
if(resultHandler.succeeded()) {
messageHandler.reply(resultHandler.result());
cache.put(lookUpKey, resultHandler.result());
System.out.println("Value stored in cache");
} else {
messageHandler.fail(0xBADC0DE, resultHandler.cause().toString());
}
});
}
});
}
static class SomeCache
{
Map<Integer, JsonObject> elMapa = new HashMap<>();
public void put(Integer key, JsonObject value) {
elMapa.put(key, value);
}
public JsonObject get(Integer key) {
return elMapa.get(key);
}
}
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
vertx.deployVerticle(new ExampleVerticle(), completionHandler -> {
if(completionHandler.succeeded()) {
vertx.eventBus().send("sesame_street", 1, replyHandler -> {
if(replyHandler.succeeded()) {
//Comment out this println and you'll maybe see 'Value stored in cache' twice in the console output.
System.out.println("Yes! " + replyHandler.result().body());
vertx.eventBus().send("sesame_street", 1);
}
vertx.close();
});
}
});
}
}

Vertx has the concept of an Asynchronous Lock. This doesn't block, but will call a handler with the lock when's it's been obtained.
Have a look at:
http://vertx.io/docs/vertx-core/java/#_cluster_wide_locks
http://vertx.io/docs/apidocs/io/vertx/core/shareddata/Lock.html

Bounded, auto-discarding, non-blocking, concurrent collection

I'm looking for a collection that:
is a Deque/List - i.e. supports inserting elements at "the top" (newest items go to the top) - deque.addFirst(..) / list.add(0, ..). It could be a Queue, but the iteration order should be reverse - i.e. the most recently added items should come first.
is bounded - i.e. has a limit of 20 items
auto-discards the oldest items (those "at the bottom", added first) when the capacity is reached
non-blocking - if the deque is empty, retrievals should not block. It should also not block / return false / null / throw exception is the deque is full.
concurrent - multiple threads should be able to operate on it
I can take LinkedBlockingDeque and wrap it into my custom collection that, on add operations checks the size and discards the last item(s). Is there a better option?

I made this simple imeplementation:
public class AutoDiscardingDeque<E> extends LinkedBlockingDeque<E> {
public AutoDiscardingDeque() {
super();
}
public AutoDiscardingDeque(int capacity) {
super(capacity);
}
#Override
public synchronized boolean offerFirst(E e) {
if (remainingCapacity() == 0) {
removeLast();
}
super.offerFirst(e);
return true;
}
}
For my needs this suffices, but it should be well-documented methods different than addFirst / offerFirst are still following the semantics of a blocking deque.

I believe what you're looking for is a bounded stack. There isn't a core library class that does this, so I think the best way of doing this is to take a non-synchronized stack (LinkedList) and wrap it in a synchronized collection that does the auto-discard and returning null on empty pop. Something like this:
import java.util.Iterator;
import java.util.LinkedList;
public class BoundedStack<T> implements Iterable<T> {
private final LinkedList<T> ll = new LinkedList<T>();
private final int bound;
public BoundedStack(int bound) {
this.bound = bound;
}
public synchronized void push(T item) {
ll.push(item);
if (ll.size() > bound) {
ll.removeLast();
}
}
public synchronized T pop() {
return ll.poll();
}
public synchronized Iterator<T> iterator() {
return ll.iterator();
}
}
...adding methods like isEmpty as required, if you want it to implement eg List.

The simplest and classic solution is a bounded ring buffer that overrides the oldest elements.
The implementation is rather easy. You need one AtomicInteger/Long for index + AtomicReferenceArray and you have a lock free general purpose stack with 2 methods only offer/poll, no size(). Most concurrent/lock-free structures have hardships w/ size(). Non-overriding stack can have O(1) but w/ an allocation on put.
Something along the lines of:
package bestsss.util;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReferenceArray;
public class ConcurrentArrayStack<E> extends AtomicReferenceArray<E>{
//easy to extend and avoid indirections,
//feel free to contain the ConcurrentArrayStack if feel purist
final AtomicLong index = new AtomicLong(-1);
public ConcurrentArrayStack(int length) {
super(length); //returns
}
/**
* #param e the element to offer into the stack
* #return the previously evicted element
*/
public E offer(E e){
for (;;){
long i = index.get();
//get the result, CAS expect before the claim
int idx = idx(i+1);
E result = get(idx);
if (!index.compareAndSet(i, i+1))//claim index spot
continue;
if (compareAndSet(idx, result, e)){
return result;
}
}
}
private int idx(long idx){//can/should use golden ratio to spread the index around and reduce false sharing
return (int)(idx%length());
}
public E poll(){
for (;;){
long i = index.get();
if (i==-1)
return null;
int idx = idx(i);
E result = get(idx);//get before the claim
if (!index.compareAndSet(i, i-1))//claim index spot
continue;
if (compareAndSet(idx, result, null)){
return result;
}
}
}
}
Last note:
having mod operation is an expensive one and power-of-2 capacity is to preferred, via &length()-1 (also guards vs long overflow).

Here is an implementation that handles concurrency and never returns Null.
import com.google.common.base.Optional;
import java.util.Deque;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.locks.ReentrantLock;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
public class BoundedStack<T> {
private final Deque<T> list = new ConcurrentLinkedDeque<>();
private final int maxEntries;
private final ReentrantLock lock = new ReentrantLock();
public BoundedStack(final int maxEntries) {
checkArgument(maxEntries > 0, "maxEntries must be greater than zero");
this.maxEntries = maxEntries;
}
public void push(final T item) {
checkNotNull(item, "item must not be null");
lock.lock();
try {
list.push(item);
if (list.size() > maxEntries) {
list.removeLast();
}
} finally {
lock.unlock();
}
}
public Optional<T> pop() {
lock.lock();
try {
return Optional.ofNullable(list.poll());
} finally {
lock.unlock();
}
}
public Optional<T> peek() {
return Optional.fromNullable(list.peekFirst());
}
public boolean empty() {
return list.isEmpty();
}
}

For the solution #remery gave, could you not run into a race condition where after if (list.size() > maxEntries) you could erroneously remove the last element if another thread runs pop() in that time period and the list is now within capacity. Given there is no thread synchronization across pop() and public void push(final T item).
For the solution #Bozho gave I would think a similar scenario could be possible? The synchronization is happening on the AutoDiscardingDeque and not with the ReentrantLock inside LinkedBlockingDeque so after running remainingCapacity() another thread could remove some objects from the list and the removeLast() would remove an extra object?