Using asynchronous methods in enterprise system

Using asynchronous methods in enterprise system - java

I have been working on very large enterprise system for financial institution for quite some time. I have only noticed few usages of asynchronous methods (frankly speaking maybe 2 or 3). Lets say i have 3 methods: doSomething1(), doSomething2(), doSomething3();
// X = {1,2,3}
SomeResult doSomethingX() {
// execution of this method takes 5-15 secs
}
xxx foo() {
SomeResult result1 = doSomething1();
SomeResult result2 = doSomething2();
SomeResult result3 = doSomething3();
// some code
}
So the execution of foo takes about 3x(5-15)sec = ~30sec
There is a lot of methods similar to foo in our system and I am wondering why there are not any async methods? Wouldn't just adding #Async to doSomethings() methods make it much faster? Or is it just 'we dont use threads explicitly in enterprise systems'

It is always worth remembering that code written before you joined a project may have been written by someone who had more experience, or who had to solve a unique issue you have not seen, and after trying smarter ways had to do something that seems strange to you. Maybe there is some state you're missing that would not be in place if it was done asynchronously.
But of course, it could just be the case that either:
a) the developers didn't know about it/use it
or
b) it wasn't available at the time for whatever reason.
Enterprises certainly aren't allergic to asynchronous code, multi-threading, or anything else you may thing of.

If you are using spring, you can use the #Async annotation to doSomething(), but it's not all you have to do:
You have to return an AsyncResult from the method and you have to use Future to manage your return values. The following "code" is taken more or less whole cloth from the spring example: https://spring.io/guides/gs/async-method/:
Future res1 = doSomething("one");
Future res2 = doSomething("two");
Future res3 = doSomething("three");
// Wait until they are all done
while (!(res1.isDone() && res2.isDone() && res3.isDone())) {
Thread.sleep(10); //10-millisecond pause between each check
}
System.out.println(res1.get());
Thats already a fair amount of orchestration (perhaps there are better ways), but it gives you an idea of the amount of labor that will go in to handling concurrency at a low level. With complexity comes risk.
It seems to me that most folks have come to the conclusion that it's better to let the container handle such scaling issues rather than to handle them by hand. You're supposed to let the container scale your EJBs and your queue workers. There are plenty of java implementations that let you scale in this way.
Nonetheless, if you made something that took 60 seconds take 5 using a low level method like the above, go for it. You'll be a hero.

Related

How Do You Test Method For Thread Safety With JUnit [duplicate]

I have thus far avoided the nightmare that is testing multi-threaded code since it just seems like too much of a minefield. I'd like to ask how people have gone about testing code that relies on threads for successful execution, or just how people have gone about testing those kinds of issues that only show up when two threads interact in a given manner?
This seems like a really key problem for programmers today, it would be useful to pool our knowledge on this one imho.

Look, there's no easy way to do this. I'm working on a project that is inherently multithreaded. Events come in from the operating system and I have to process them concurrently.
The simplest way to deal with testing complex, multithreaded application code is this: If it's too complex to test, you're doing it wrong. If you have a single instance that has multiple threads acting upon it, and you can't test situations where these threads step all over each other, then your design needs to be redone. It's both as simple and as complex as this.
There are many ways to program for multithreading that avoids threads running through instances at the same time. The simplest is to make all your objects immutable. Of course, that's not usually possible. So you have to identify those places in your design where threads interact with the same instance and reduce the number of those places. By doing this, you isolate a few classes where multithreading actually occurs, reducing the overall complexity of testing your system.
But you have to realize that even by doing this, you still can't test every situation where two threads step on each other. To do that, you'd have to run two threads concurrently in the same test, then control exactly what lines they are executing at any given moment. The best you can do is simulate this situation. But this might require you to code specifically for testing, and that's at best a half step towards a true solution.
Probably the best way to test code for threading issues is through static analysis of the code. If your threaded code doesn't follow a finite set of thread safe patterns, then you might have a problem. I believe Code Analysis in VS does contain some knowledge of threading, but probably not much.
Look, as things stand currently (and probably will stand for a good time to come), the best way to test multithreaded apps is to reduce the complexity of threaded code as much as possible. Minimize areas where threads interact, test as best as possible, and use code analysis to identify danger areas.

It's been a while when this question was posted, but it's still not answered ...
kleolb02's answer is a good one. I'll try going into more details.
There is a way, which I practice for C# code. For unit tests you should be able to program reproducible tests, which is the biggest challenge in multithreaded code. So my answer aims toward forcing asynchronous code into a test harness, which works synchronously.
It's an idea from Gerard Meszaros's book "xUnit Test Patterns" and is called "Humble Object" (p. 695): You have to separate core logic code and anything which smells like asynchronous code from each other. This would result to a class for the core logic, which works synchronously.
This puts you into the position to test the core logic code in a synchronous way. You have absolute control over the timing of the calls you are doing on the core logic and thus can make reproducible tests. And this is your gain from separating core logic and asynchronous logic.
This core logic needs be wrapped around by another class, which is responsible for receiving calls to the core logic asynchronously and delegates these calls to the core logic. Production code will only access the core logic via that class. Because this class should only delegate calls, it's a very "dumb" class without much logic. So you can keep your unit tests for this asychronous working class at a minimum.
Anything above that (testing interaction between classes) are component tests. Also in this case, you should be able to have absolute control over timing, if you stick to the "Humble Object" pattern.

Tough one indeed! In my (C++) unit tests, I've broken this down into several categories along the lines of the concurrency pattern used:
Unit tests for classes that operate in a single thread and aren't thread aware -- easy, test as usual.
Unit tests for Monitor objects (those that execute synchronized methods in the callers' thread of control) that expose a synchronized public API -- instantiate multiple mock threads that exercise the API. Construct scenarios that exercise internal conditions of the passive object. Include one longer running test that basically beats the heck out of it from multiple threads for a long period of time. This is unscientific I know but it does build confidence.
Unit tests for Active objects (those that encapsulate their own thread or threads of control) -- similar to #2 above with variations depending on the class design. Public API may be blocking or non-blocking, callers may obtain futures, data may arrive at queues or need to be dequeued. There are many combinations possible here; white box away. Still requires multiple mock threads to make calls to the object under test.
As an aside:
In internal developer training that I do, I teach the Pillars of Concurrency and these two patterns as the primary framework for thinking about and decomposing concurrency problems. There's obviously more advanced concepts out there but I've found that this set of basics helps keep engineers out of the soup. It also leads to code that is more unit testable, as described above.

I have faced this issue several times in recent years when writing thread handling code for several projects. I'm providing a late answer because most of the other answers, while providing alternatives, do not actually answer the question about testing. My answer is addressed to the cases where there is no alternative to multithreaded code; I do cover code design issues for completeness, but also discuss unit testing.
Writing testable multithreaded code
The first thing to do is to separate your production thread handling code from all the code that does actual data processing. That way, the data processing can be tested as singly threaded code, and the only thing the multithreaded code does is to coordinate threads.
The second thing to remember is that bugs in multithreaded code are probabilistic; the bugs that manifest themselves least frequently are the bugs that will sneak through into production, will be difficult to reproduce even in production, and will thus cause the biggest problems. For this reason, the standard coding approach of writing the code quickly and then debugging it until it works is a bad idea for multithreaded code; it will result in code where the easy bugs are fixed and the dangerous bugs are still there.
Instead, when writing multithreaded code, you must write the code with the attitude that you are going to avoid writing the bugs in the first place. If you have properly removed the data processing code, the thread handling code should be small enough - preferably a few lines, at worst a few dozen lines - that you have a chance of writing it without writing a bug, and certainly without writing many bugs, if you understand threading, take your time, and are careful.
Writing unit tests for multithreaded code
Once the multithreaded code is written as carefully as possible, it is still worthwhile writing tests for that code. The primary purpose of the tests is not so much to test for highly timing dependent race condition bugs - it's impossible to test for such race conditions repeatably - but rather to test that your locking strategy for preventing such bugs allows for multiple threads to interact as intended.
To properly test correct locking behavior, a test must start multiple threads. To make the test repeatable, we want the interactions between the threads to happen in a predictable order. We don't want to externally synchronize the threads in the test, because that will mask bugs that could happen in production where the threads are not externally synchronized. That leaves the use of timing delays for thread synchronization, which is the technique that I have used successfully whenever I've had to write tests of multithreaded code.
If the delays are too short, then the test becomes fragile, because minor timing differences - say between different machines on which the tests may be run - may cause the timing to be off and the test to fail. What I've typically done is start with delays that cause test failures, increase the delays so that the test passes reliably on my development machine, and then double the delays beyond that so the test has a good chance of passing on other machines. This does mean that the test will take a macroscopic amount of time, though in my experience, careful test design can limit that time to no more than a dozen seconds. Since you shouldn't have very many places requiring thread coordination code in your application, that should be acceptable for your test suite.
Finally, keep track of the number of bugs caught by your test. If your test has 80% code coverage, it can be expected to catch about 80% of your bugs. If your test is well designed but finds no bugs, there's a reasonable chance that you don't have additional bugs that will only show up in production. If the test catches one or two bugs, you might still get lucky. Beyond that, and you may want to consider a careful review of or even a complete rewrite of your thread handling code, since it is likely that code still contains hidden bugs that will be very difficult to find until the code is in production, and very difficult to fix then.

I also had serious problems testing multi- threaded code. Then I found a really cool solution in "xUnit Test Patterns" by Gerard Meszaros. The pattern he describes is called Humble object.
Basically it describes how you can extract the logic into a separate, easy-to-test component that is decoupled from its environment. After you tested this logic, you can test the complicated behaviour (multi- threading, asynchronous execution, etc...)

There are a few tools around that are quite good. Here is a summary of some of the Java ones.
Some good static analysis tools include FindBugs (gives some useful hints), JLint, Java Pathfinder (JPF & JPF2), and Bogor.
MultithreadedTC is quite a good dynamic analysis tool (integrated into JUnit) where you have to set up your own test cases.
ConTest from IBM Research is interesting. It instruments your code by inserting all kinds of thread modifying behaviours (e.g. sleep & yield) to try to uncover bugs randomly.
SPIN is a really cool tool for modelling your Java (and other) components, but you need to have some useful framework. It is hard to use as is, but extremely powerful if you know how to use it. Quite a few tools use SPIN underneath the hood.
MultithreadedTC is probably the most mainstream, but some of the static analysis tools listed above are definitely worth looking at.

Awaitility can also be useful to help you write deterministic unit tests. It allows you to wait until some state somewhere in your system is updated. For example:
await().untilCall( to(myService).myMethod(), greaterThan(3) );
or
await().atMost(5,SECONDS).until(fieldIn(myObject).ofType(int.class), equalTo(1));
It also has Scala and Groovy support.
await until { something() > 4 } // Scala example

Another way to (kinda) test threaded code, and very complex systems in general is through Fuzz Testing.
It's not great, and it won't find everything, but its likely to be useful and its simple to do.
Quote:
Fuzz testing or fuzzing is a software testing technique that provides random data("fuzz") to the inputs of a program. If the program fails (for example, by crashing, or by failing built-in code assertions), the defects can be noted. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior.
...
Fuzz testing is often used in large software development projects that employ black box testing. These projects usually have a budget to develop test tools, and fuzz testing is one of the techniques which offers a high benefit to cost ratio.
...
However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a bug-finding tool rather than an assurance of quality.

Testing MT code for correctness is, as already stated, quite a hard problem. In the end it boils down to ensuring that there are no incorrectly synchronised data races in your code. The problem with this is that there are infinitely many possibilities of thread execution (interleavings) over which you do not have much control (be sure to read this article, though). In simple scenarios it might be possible to actually prove correctness by reasoning but this is usually not the case. Especially if you want to avoid/minimize synchronization and not go for the most obvious/easiest synchronization option.
An approach that I follow is to write highly concurrent test code in order to make potentially undetected data races likely to occur. And then I run those tests for some time :) I once stumbled upon a talk where some computer scientist where showing off a tool that kind of does this (randomly devising test from specs and then running them wildly, concurrently, checking for the defined invariants to be broken).
By the way, I think this aspect of testing MT code has not been mentioned here: identify invariants of the code that you can check for randomly. Unfortunately, finding those invariants is quite a hard problem, too. Also they might not hold all the time during execution, so you have to find/enforce executions points where you can expect them to be true. Bringing the code execution to such a state is also a hard problem (and might itself incur concurrency issues. Whew, it's damn hard!
Some interesting links to read:
Deterministic interleaving: A framework that allows to force certain thread interleavings and then check for invariants
jMock Blitzer : Stress test synchronization
assertConcurrent : JUnit version of stress testing synronization
Testing concurrent code : Short overview of the two primary methods of brute force (stress test) or deterministic (going for the invariants)

I've done a lot of this, and yes it sucks.
Some tips:
GroboUtils for running multiple test threads
alphaWorks ConTest to instrument classes to cause interleavings to vary between iterations
Create a throwable field and check it in tearDown (see Listing 1). If you catch a bad exception in another thread, just assign it to throwable.
I created the utils class in Listing 2 and have found it invaluable, especially waitForVerify and waitForCondition, which will greatly increase the performance of your tests.
Make good use of AtomicBoolean in your tests. It is thread safe, and you'll often need a final reference type to store values from callback classes and suchlike. See example in Listing 3.
Make sure to always give your test a timeout (e.g., #Test(timeout=60*1000)), as concurrency tests can sometimes hang forever when they're broken.
Listing 1:
#After
public void tearDown() {
if ( throwable != null )
throw throwable;
}
Listing 2:
import static org.junit.Assert.fail;
import java.io.File;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Proxy;
import java.util.Random;
import org.apache.commons.collections.Closure;
import org.apache.commons.collections.Predicate;
import org.apache.commons.lang.time.StopWatch;
import org.easymock.EasyMock;
import org.easymock.classextension.internal.ClassExtensionHelper;
import static org.easymock.classextension.EasyMock.*;
import ca.digitalrapids.io.DRFileUtils;
/**
* Various utilities for testing
*/
public abstract class DRTestUtils
{
static private Random random = new Random();
/** Calls {#link #waitForCondition(Integer, Integer, Predicate, String)} with
* default max wait and check period values.
*/
static public void waitForCondition(Predicate predicate, String errorMessage)
throws Throwable
{
waitForCondition(null, null, predicate, errorMessage);
}
/** Blocks until a condition is true, throwing an {#link AssertionError} if
* it does not become true during a given max time.
* #param maxWait_ms max time to wait for true condition. Optional; defaults
* to 30 * 1000 ms (30 seconds).
* #param checkPeriod_ms period at which to try the condition. Optional; defaults
* to 100 ms.
* #param predicate the condition
* #param errorMessage message use in the {#link AssertionError}
* #throws Throwable on {#link AssertionError} or any other exception/error
*/
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms,
Predicate predicate, String errorMessage) throws Throwable
{
waitForCondition(maxWait_ms, checkPeriod_ms, predicate, new Closure() {
public void execute(Object errorMessage)
{
fail((String)errorMessage);
}
}, errorMessage);
}
/** Blocks until a condition is true, running a closure if
* it does not become true during a given max time.
* #param maxWait_ms max time to wait for true condition. Optional; defaults
* to 30 * 1000 ms (30 seconds).
* #param checkPeriod_ms period at which to try the condition. Optional; defaults
* to 100 ms.
* #param predicate the condition
* #param closure closure to run
* #param argument argument for closure
* #throws Throwable on {#link AssertionError} or any other exception/error
*/
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms,
Predicate predicate, Closure closure, Object argument) throws Throwable
{
if ( maxWait_ms == null )
maxWait_ms = 30 * 1000;
if ( checkPeriod_ms == null )
checkPeriod_ms = 100;
StopWatch stopWatch = new StopWatch();
stopWatch.start();
while ( !predicate.evaluate(null) ) {
Thread.sleep(checkPeriod_ms);
if ( stopWatch.getTime() > maxWait_ms ) {
closure.execute(argument);
}
}
}
/** Calls {#link #waitForVerify(Integer, Object)} with <code>null</code>
* for {#code maxWait_ms}
*/
static public void waitForVerify(Object easyMockProxy)
throws Throwable
{
waitForVerify(null, easyMockProxy);
}
/** Repeatedly calls {#link EasyMock#verify(Object[])} until it succeeds, or a
* max wait time has elapsed.
* #param maxWait_ms Max wait time. <code>null</code> defaults to 30s.
* #param easyMockProxy Proxy to call verify on
* #throws Throwable
*/
static public void waitForVerify(Integer maxWait_ms, Object easyMockProxy)
throws Throwable
{
if ( maxWait_ms == null )
maxWait_ms = 30 * 1000;
StopWatch stopWatch = new StopWatch();
stopWatch.start();
for(;;) {
try
{
verify(easyMockProxy);
break;
}
catch (AssertionError e)
{
if ( stopWatch.getTime() > maxWait_ms )
throw e;
Thread.sleep(100);
}
}
}
/** Returns a path to a directory in the temp dir with the name of the given
* class. This is useful for temporary test files.
* #param aClass test class for which to create dir
* #return the path
*/
static public String getTestDirPathForTestClass(Object object)
{
String filename = object instanceof Class ?
((Class)object).getName() :
object.getClass().getName();
return DRFileUtils.getTempDir() + File.separator +
filename;
}
static public byte[] createRandomByteArray(int bytesLength)
{
byte[] sourceBytes = new byte[bytesLength];
random.nextBytes(sourceBytes);
return sourceBytes;
}
/** Returns <code>true</code> if the given object is an EasyMock mock object
*/
static public boolean isEasyMockMock(Object object) {
try {
InvocationHandler invocationHandler = Proxy
.getInvocationHandler(object);
return invocationHandler.getClass().getName().contains("easymock");
} catch (IllegalArgumentException e) {
return false;
}
}
}
Listing 3:
#Test
public void testSomething() {
final AtomicBoolean called = new AtomicBoolean(false);
subject.setCallback(new SomeCallback() {
public void callback(Object arg) {
// check arg here
called.set(true);
}
});
subject.run();
assertTrue(called.get());
}

I handle unit tests of threaded components the same way I handle any unit test, that is, with inversion of control and isolation frameworks. I develop in the .Net-arena and, out of the box, the threading (among other things) is very hard (I'd say nearly impossible) to fully isolate.
Therefore, I've written wrappers that looks something like this (simplified):
public interface IThread
{
void Start();
...
}
public class ThreadWrapper : IThread
{
private readonly Thread _thread;
public ThreadWrapper(ThreadStart threadStart)
{
_thread = new Thread(threadStart);
}
public Start()
{
_thread.Start();
}
}
public interface IThreadingManager
{
IThread CreateThread(ThreadStart threadStart);
}
public class ThreadingManager : IThreadingManager
{
public IThread CreateThread(ThreadStart threadStart)
{
return new ThreadWrapper(threadStart)
}
}
From there, I can easily inject the IThreadingManager into my components and use my isolation framework of choice to make the thread behave as I expect during the test.
That has so far worked great for me, and I use the same approach for the thread pool, things in System.Environment, Sleep etc. etc.

Pete Goodliffe has a series on the unit testing of threaded code.
It's hard. I take the easier way out and try to keep the threading code abstracted from the actual test. Pete does mention that the way I do it is wrong but I've either got the separation right or I've just been lucky.

For Java, check out chapter 12 of JCIP. There are some concrete examples of writing deterministic, multi-threaded unit tests to at least test the correctness and invariants of concurrent code.
"Proving" thread-safety with unit tests is much dicier. My belief is that this is better served by automated integration testing on a variety of platforms/configurations.

Have a look at my related answer at
Designing a Test class for a custom Barrier
It's biased towards Java but has a reasonable summary of the options.
In summary though (IMO) its not the use of some fancy framework that will ensure correctness but how you go about designing you multithreaded code. Splitting the concerns (concurrency and functionality) goes a huge way towards raising confidence. Growing Object Orientated Software Guided By Tests explains some options better than I can.
Static analysis and formal methods (see, Concurrency: State Models and Java Programs) is an option but I've found them to be of limited use in commercial development.
Don't forget that any load/soak style tests are rarely guaranteed to highlight problems.
Good luck!

I like to write two or more test methods to execute on parallel threads, and each of them make calls into the object under test. I've been using Sleep() calls to coordinate the order of the calls from the different threads, but that's not really reliable. It's also a lot slower because you have to sleep long enough that the timing usually works.
I found the Multithreaded TC Java library from the same group that wrote FindBugs. It lets you specify the order of events without using Sleep(), and it's reliable. I haven't tried it yet.
The biggest limitation to this approach is that it only lets you test the scenarios you suspect will cause trouble. As others have said, you really need to isolate your multithreaded code into a small number of simple classes to have any hope of thoroughly testing them.
Once you've carefully tested the scenarios you expect to cause trouble, an unscientific test that throws a bunch of simultaneous requests at the class for a while is a good way to look for unexpected trouble.
Update: I've played a bit with the Multithreaded TC Java library, and it works well. I've also ported some of its features to a .NET version I call TickingTest.

I just recently discovered (for Java) a tool called Threadsafe. It is a static analysis tool much like findbugs but specifically to spot multi-threading issues. It is not a replacement for testing but I can recommend it as part of writing reliable multi-threaded Java.
It even catches some very subtle potential issues around things like class subsumption, accessing unsafe objects through concurrent classes and spotting missing volatile modifiers when using the double checked locking paradigm.
If you write multithreaded Java give it a shot.

The following article suggests 2 solutions. Wrapping a semaphore (CountDownLatch) and adds functionality like externalize data from internal thread. Another way of achieving this purpose is to use Thread Pool (see Points of Interest).
Sprinkler - Advanced synchronization object

I spent most of last week at a university library studying debugging of concurrent code. The central problem is concurrent code is non-deterministic. Typically, academic debugging has fallen into one of three camps here:
Event-trace/replay. This requires an event monitor and then reviewing the events that were sent. In a UT framework, this would involve manually sending the events as part of a test, and then doing post-mortem reviews.
Scriptable. This is where you interact with the running code with a set of triggers. "On x > foo, baz()". This could be interpreted into a UT framework where you have a run-time system triggering a given test on a certain condition.
Interactive. This obviously won't work in an automatic testing situation. ;)
Now, as above commentators have noticed, you can design your concurrent system into a more deterministic state. However, if you don't do that properly, you're just back to designing a sequential system again.
My suggestion would be to focus on having a very strict design protocol about what gets threaded and what doesn't get threaded. If you constrain your interface so that there is minimal dependancies between elements, it is much easier.
Good luck, and keep working on the problem.

I have had the unfortunate task of testing threaded code and they are definitely the hardest tests I have ever written.
When writing my tests, I used a combination of delegates and events. Basically it is all about using PropertyNotifyChanged events with a WaitCallback or some kind of ConditionalWaiter that polls.
I am not sure if this was the best approach, but it has worked out for me.

Assuming under "multi-threaded" code was meant something that is
stateful and mutable
AND accessed/modified by multiple threads
concurrently
In other words we are talking about testing custom stateful thread-safe class/method/unit - which should be a very rare beast nowadays.
Because this beast is rare, first of all we need to make sure that there are all valid excuses to write it.
Step 1. Consider modifying state in same synchronization context.
Today it is easy to write compose-able concurrent and asynchronous code where IO or other slow operations offloaded to background but shared state is updated and queried in one synchronization context. e.g. async/await tasks and Rx in .NET etc. - they are all testable by design, "real" Tasks and schedulers can be substituted to make testing deterministic (however this is out of scope of the question).
It may sound very constrained but this approach works surprisingly well. It is possible to write whole apps in this style without need to make any state thread-safe (I do).
Step 2. If manipulating of shared state on single synchronization context is absolutely not possible.
Make sure the wheel is not being reinvented / there's definitely no standard alternative that can be adapted for the job. It should be likely that code is very cohesive and contained within one unit e.g. with a good chance it is a special case of some standard thread-safe data structure like hash map or collection or whatever.
Note: if code is large / spans across multiple classes AND needs multi-thread state manipulation then there's a very high chance that design is not good, reconsider Step 1
Step 3. If this step is reached then we need to test our own custom stateful thread-safe class/method/unit.
I'll be dead honest : I never had to write proper tests for such code. Most of the time I get away at Step 1, sometimes at Step 2. Last time I had to write custom thread-safe code was so many years ago that it was before I adopted unit testing / probably I wouldn't have to write it with the current knowledge anyway.
If I really had to test such code (finally, actual answer) then I would try couple of things below
Non-deterministic stress testing. e.g. run 100 threads simultaneously and check that end result is consistent.
This is more typical for higher level / integration testing of multiple users scenarios but also can be used at the unit level.
Expose some test 'hooks' where test can inject some code to help make deterministic scenarios where one thread must perform operation before the other.
As ugly as it is, I can't think of anything better.
Delay-driven testing to make threads run and perform operations in particular order. Strictly speaking such tests are non-deterministic too (there's a chance of system freeze / stop-the-world GC collection which can distort otherwise orchestrated delays), also it is ugly but allows to avoid hooks.

Running multiple threads is not difficult; it is piece of cake. Unfortunately, threads usually need to communicate with each other; that's what's difficult.
The mechanism that was originally invented to allow communication between modules was function calls; when module A wants to communicate with module B, it just invokes a function in module B. Unfortunately, this does not work with threads, because when you call a function, that function still runs in the current thread.
To overcome this problem, people decided to fall back to an even more primitive mechanism of communication: just declare a certain variable, and let both threads have access to that variable. In other words, allow the threads to share data. Sharing data is literally the first thing that naturally comes to mind, and it appears like a good choice because it seems very simple. I mean, how hard can it be, right? What could possibly go wrong?
Race conditions. That's what can, and will, go wrong.
When people realized their software was suffering from random, non-reproducible catastrophic failures due to race conditions, they started inventing elaborate mechanisms such as locks and compare-and-swap, aiming to protect against such things happening. These mechanisms fall under the broad category of "synchronization". Unfortunately, synchronization has two problems:
It is very difficult to get it right, so it is very prone to bugs.
It is completely untestable, because you cannot test for a race condition.
The astute reader might notice that "Very prone to bugs" and "Completely untestable" is a deadly combination.
Now, the mechanisms I mentioned above were being invented and adopted by large parts of the industry before the concept of automated software testing became prevalent; So, nobody could see how deadly the problem was; they just regarded it as a difficult topic which requires guru programmers, and everyone was okay with that.
Nowadays, whatever we do, we put testing first. So, if some mechanism is untestable, then the use of that mechanism is just out of the question, period. Thus, synchronization has fallen out of grace; very few people still practice it, and they are becoming fewer and fewer every day.
Without synchronization threads cannot share data; however, the original requirement was not to share data; it was to allow threads to communicate with each other. Besides sharing data, there exist other, more elegant mechanisms for inter-thread communication.
One such mechanism is message-passing, otherwise known as events.
With message passing, there is only one place in the entire software system which utilizes synchronization, and that is the concurrent blocking queue collection class that we use for storing messages. (The idea is that we should be able to get at least that little part right.)
The great thing about message passing is that it does not suffer from race conditions and is fully testable.

For J2E code, I've used SilkPerformer, LoadRunner and JMeter for concurrency testing of threads. They all do the same thing. Basically, they give you a relatively simple interface for administrating their version of the proxy server, required, in order to analyze the TCP/IP data stream, and simulate multiple users making simultaneous requests to your app server. The proxy server can give you the ability to do things like analyze the requests made, by presenting the whole page and URL sent to the server, as well as the response from the server, after processing the request.
You can find some bugs in insecure http mode, where you can at least analyze the form data that is being sent, and systematically alter that for each user. But the true tests are when you run in https (Secured Socket Layers). Then, you also have to contend with systematically altering the session and cookie data, which can be a little more convoluted.
The best bug I ever found, while testing concurrency, was when I discovered that the developer had relied upon Java garbage collection to close the connection request that was established at login, to the LDAP server, when logging in. This resulted in users being exposed to other users' sessions and very confusing results, when trying to analyze what happened when the server was brought to it's knees, barely able to complete one transaction, every few seconds.
In the end, you or someone will probably have to buckle down and analyze the code for blunders like the one I just mentioned. And an open discussion across departments, like the one that occurred, when we unfolded the problem described above, are most useful. But these tools are the best solution to testing multi-threaded code. JMeter is open source. SilkPerformer and LoadRunner are proprietary. If you really want to know whether your app is thread safe, that's how the big boys do it. I've done this for very large companies professionally, so I'm not guessing. I'm speaking from personal experience.
A word of caution: it does take some time to understand these tools. It will not be a matter of simply installing the software and firing up the GUI, unless you've already had some exposure to multi-threaded programming. I've tried to identify the 3 critical categories of areas to understand (forms, session and cookie data), with the hope that at least starting with understanding these topics will help you focus on quick results, as opposed to having to read through the entire documentation.

Concurrency is a complex interplay between the memory model, hardware, caches and our code. In the case of Java at least such tests have been partly addressed mainly by jcstress. The creators of that library are known to be authors of many JVM, GC and Java concurrency features.
But even this library needs good knowledge of the Java Memory Model specification so that we know exactly what we are testing. But I think the focus of this effort is mircobenchmarks. Not huge business applications.

There is an article on the topic, using Rust as the language in the example code:
https://medium.com/#polyglot_factotum/rust-concurrency-five-easy-pieces-871f1c62906a
In summary, the trick is to write your concurrent logic so that it is robust to the non-determinism involved with multiple threads of execution, using tools like channels and condvars.
Then, if that is how you've structured your "components", the easiest way to test them is by using channels to send messages to them, and then block on other channels to assert that the component sends certain expected messages.
The linked-to article is fully written using unit-tests.

It's not perfect, but I wrote this helper for my tests in C#:
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace Proto.Promises.Tests.Threading
{
public class ThreadHelper
{
public static readonly int multiThreadCount = Environment.ProcessorCount * 100;
private static readonly int[] offsets = new int[] { 0, 10, 100, 1000 };
private readonly Stack<Task> _executingTasks = new Stack<Task>(multiThreadCount);
private readonly Barrier _barrier = new Barrier(1);
private int _currentParticipants = 0;
private readonly TimeSpan _timeout;
public ThreadHelper() : this(TimeSpan.FromSeconds(10)) { } // 10 second timeout should be enough for most cases.
public ThreadHelper(TimeSpan timeout)
{
_timeout = timeout;
}
/// <summary>
/// Execute the action multiple times in parallel threads.
/// </summary>
public void ExecuteMultiActionParallel(Action action)
{
for (int i = 0; i < multiThreadCount; ++i)
{
AddParallelAction(action);
}
ExecutePendingParallelActions();
}
/// <summary>
/// Execute the action once in a separate thread.
/// </summary>
public void ExecuteSingleAction(Action action)
{
AddParallelAction(action);
ExecutePendingParallelActions();
}
/// <summary>
/// Add an action to be run in parallel.
/// </summary>
public void AddParallelAction(Action action)
{
var taskSource = new TaskCompletionSource<bool>();
lock (_executingTasks)
{
++_currentParticipants;
_barrier.AddParticipant();
_executingTasks.Push(taskSource.Task);
}
new Thread(() =>
{
try
{
_barrier.SignalAndWait(); // Try to make actions run in lock-step to increase likelihood of breaking race conditions.
action.Invoke();
taskSource.SetResult(true);
}
catch (Exception e)
{
taskSource.SetException(e);
}
}).Start();
}
/// <summary>
/// Runs the pending actions in parallel, attempting to run them in lock-step.
/// </summary>
public void ExecutePendingParallelActions()
{
Task[] tasks;
lock (_executingTasks)
{
_barrier.SignalAndWait();
_barrier.RemoveParticipants(_currentParticipants);
_currentParticipants = 0;
tasks = _executingTasks.ToArray();
_executingTasks.Clear();
}
try
{
if (!Task.WaitAll(tasks, _timeout))
{
throw new TimeoutException($"Action(s) timed out after {_timeout}, there may be a deadlock.");
}
}
catch (AggregateException e)
{
// Only throw one exception instead of aggregate to try to avoid overloading the test error output.
throw e.Flatten().InnerException;
}
}
/// <summary>
/// Run each action in parallel multiple times with differing offsets for each run.
/// <para/>The number of runs is 4^actions.Length, so be careful if you don't want the test to run too long.
/// </summary>
/// <param name="expandToProcessorCount">If true, copies each action on additional threads up to the processor count. This can help test more without increasing the time it takes to complete.
/// <para/>Example: 2 actions with 6 processors, runs each action 3 times in parallel.</param>
/// <param name="setup">The action to run before each parallel run.</param>
/// <param name="teardown">The action to run after each parallel run.</param>
/// <param name="actions">The actions to run in parallel.</param>
public void ExecuteParallelActionsWithOffsets(bool expandToProcessorCount, Action setup, Action teardown, params Action[] actions)
{
setup += () => { };
teardown += () => { };
int actionCount = actions.Length;
int expandCount = expandToProcessorCount ? Math.Max(Environment.ProcessorCount / actionCount, 1) : 1;
foreach (var combo in GenerateCombinations(offsets, actionCount))
{
setup.Invoke();
for (int k = 0; k < expandCount; ++k)
{
for (int i = 0; i < actionCount; ++i)
{
int offset = combo[i];
Action action = actions[i];
AddParallelAction(() =>
{
for (int j = offset; j > 0; --j) { } // Just spin in a loop for the offset.
action.Invoke();
});
}
}
ExecutePendingParallelActions();
teardown.Invoke();
}
}
// Input: [1, 2, 3], 3
// Ouput: [
// [1, 1, 1],
// [2, 1, 1],
// [3, 1, 1],
// [1, 2, 1],
// [2, 2, 1],
// [3, 2, 1],
// [1, 3, 1],
// [2, 3, 1],
// [3, 3, 1],
// [1, 1, 2],
// [2, 1, 2],
// [3, 1, 2],
// [1, 2, 2],
// [2, 2, 2],
// [3, 2, 2],
// [1, 3, 2],
// [2, 3, 2],
// [3, 3, 2],
// [1, 1, 3],
// [2, 1, 3],
// [3, 1, 3],
// [1, 2, 3],
// [2, 2, 3],
// [3, 2, 3],
// [1, 3, 3],
// [2, 3, 3],
// [3, 3, 3]
// ]
private static IEnumerable<int[]> GenerateCombinations(int[] options, int count)
{
int[] indexTracker = new int[count];
int[] combo = new int[count];
for (int i = 0; i < count; ++i)
{
combo[i] = options[0];
}
// Same algorithm as picking a combination lock.
int rollovers = 0;
while (rollovers < count)
{
yield return combo; // No need to duplicate the array since we're just reading it.
for (int i = 0; i < count; ++i)
{
int index = ++indexTracker[i];
if (index == options.Length)
{
indexTracker[i] = 0;
combo[i] = options[0];
if (i == rollovers)
{
++rollovers;
}
}
else
{
combo[i] = options[index];
break;
}
}
}
}
}
}
Example usage:
[Test]
public void DeferredMayBeBeResolvedAndPromiseAwaitedConcurrently_void0()
{
Promise.Deferred deferred = default(Promise.Deferred);
Promise promise = default(Promise);
int invokedCount = 0;
var threadHelper = new ThreadHelper();
threadHelper.ExecuteParallelActionsWithOffsets(false,
// Setup
() =>
{
invokedCount = 0;
deferred = Promise.NewDeferred();
promise = deferred.Promise;
},
// Teardown
() => Assert.AreEqual(1, invokedCount),
// Parallel Actions
() => deferred.Resolve(),
() => promise.Then(() => { Interlocked.Increment(ref invokedCount); }).Forget()
);
}

One simple test pattern that can work for some (not all!) cases is to repeat the same test many times. For example, suppose you have a method:
def process(input):
# Spawns several threads to do the job
# ...
return output
Create a bunch of tests:
process(input1) -> expect to return output1
process(input2) -> expect to return output2
...
Now run each of those tests many times.
If the implementation of process contains a subtle bug (e.g. deadlock, race condition, etc.) that has 0.1% chance to emerge, running the test 1000 times gives 64% probability for the bug to emerge at least once. Running the test 10000 times gives >99% probability.

If you are testing simple new Thread(runnable).run()
You can mock Thread to run the runnable sequentially
For instance, if the code of the tested object invokes a new thread like this
Class TestedClass {
public void doAsychOp() {
new Thread(new myRunnable()).start();
}
}
Then mocking new Threads and run the runnable argument sequentially can help
#Mock
private Thread threadMock;
#Test
public void myTest() throws Exception {
PowerMockito.mockStatic(Thread.class);
//when new thread is created execute runnable immediately
PowerMockito.whenNew(Thread.class).withAnyArguments().then(new Answer<Thread>() {
#Override
public Thread answer(InvocationOnMock invocation) throws Throwable {
// immediately run the runnable
Runnable runnable = invocation.getArgumentAt(0, Runnable.class);
if(runnable != null) {
runnable.run();
}
return threadMock;//return a mock so Thread.start() will do nothing
}
});
TestedClass testcls = new TestedClass()
testcls.doAsychOp(); //will invoke myRunnable.run in current thread
//.... check expected
}

(if possible) don't use threads, use actors / active objects. Easy to test.

You may use EasyMock.makeThreadSafe to make testing instance threadsafe

How to test multi-thread logic in java [duplicate]

I have thus far avoided the nightmare that is testing multi-threaded code since it just seems like too much of a minefield. I'd like to ask how people have gone about testing code that relies on threads for successful execution, or just how people have gone about testing those kinds of issues that only show up when two threads interact in a given manner?
This seems like a really key problem for programmers today, it would be useful to pool our knowledge on this one imho.

It's been a while when this question was posted, but it's still not answered ...
kleolb02's answer is a good one. I'll try going into more details.
There is a way, which I practice for C# code. For unit tests you should be able to program reproducible tests, which is the biggest challenge in multithreaded code. So my answer aims toward forcing asynchronous code into a test harness, which works synchronously.
It's an idea from Gerard Meszaros's book "xUnit Test Patterns" and is called "Humble Object" (p. 695): You have to separate core logic code and anything which smells like asynchronous code from each other. This would result to a class for the core logic, which works synchronously.
This puts you into the position to test the core logic code in a synchronous way. You have absolute control over the timing of the calls you are doing on the core logic and thus can make reproducible tests. And this is your gain from separating core logic and asynchronous logic.
This core logic needs be wrapped around by another class, which is responsible for receiving calls to the core logic asynchronously and delegates these calls to the core logic. Production code will only access the core logic via that class. Because this class should only delegate calls, it's a very "dumb" class without much logic. So you can keep your unit tests for this asychronous working class at a minimum.
Anything above that (testing interaction between classes) are component tests. Also in this case, you should be able to have absolute control over timing, if you stick to the "Humble Object" pattern.

Tough one indeed! In my (C++) unit tests, I've broken this down into several categories along the lines of the concurrency pattern used:
Unit tests for classes that operate in a single thread and aren't thread aware -- easy, test as usual.
Unit tests for Monitor objects (those that execute synchronized methods in the callers' thread of control) that expose a synchronized public API -- instantiate multiple mock threads that exercise the API. Construct scenarios that exercise internal conditions of the passive object. Include one longer running test that basically beats the heck out of it from multiple threads for a long period of time. This is unscientific I know but it does build confidence.
Unit tests for Active objects (those that encapsulate their own thread or threads of control) -- similar to #2 above with variations depending on the class design. Public API may be blocking or non-blocking, callers may obtain futures, data may arrive at queues or need to be dequeued. There are many combinations possible here; white box away. Still requires multiple mock threads to make calls to the object under test.
As an aside:
In internal developer training that I do, I teach the Pillars of Concurrency and these two patterns as the primary framework for thinking about and decomposing concurrency problems. There's obviously more advanced concepts out there but I've found that this set of basics helps keep engineers out of the soup. It also leads to code that is more unit testable, as described above.

I also had serious problems testing multi- threaded code. Then I found a really cool solution in "xUnit Test Patterns" by Gerard Meszaros. The pattern he describes is called Humble object.
Basically it describes how you can extract the logic into a separate, easy-to-test component that is decoupled from its environment. After you tested this logic, you can test the complicated behaviour (multi- threading, asynchronous execution, etc...)

There are a few tools around that are quite good. Here is a summary of some of the Java ones.
Some good static analysis tools include FindBugs (gives some useful hints), JLint, Java Pathfinder (JPF & JPF2), and Bogor.
MultithreadedTC is quite a good dynamic analysis tool (integrated into JUnit) where you have to set up your own test cases.
ConTest from IBM Research is interesting. It instruments your code by inserting all kinds of thread modifying behaviours (e.g. sleep & yield) to try to uncover bugs randomly.
SPIN is a really cool tool for modelling your Java (and other) components, but you need to have some useful framework. It is hard to use as is, but extremely powerful if you know how to use it. Quite a few tools use SPIN underneath the hood.
MultithreadedTC is probably the most mainstream, but some of the static analysis tools listed above are definitely worth looking at.

Awaitility can also be useful to help you write deterministic unit tests. It allows you to wait until some state somewhere in your system is updated. For example:
await().untilCall( to(myService).myMethod(), greaterThan(3) );
or
await().atMost(5,SECONDS).until(fieldIn(myObject).ofType(int.class), equalTo(1));
It also has Scala and Groovy support.
await until { something() > 4 } // Scala example

Another way to (kinda) test threaded code, and very complex systems in general is through Fuzz Testing.
It's not great, and it won't find everything, but its likely to be useful and its simple to do.
Quote:
Fuzz testing or fuzzing is a software testing technique that provides random data("fuzz") to the inputs of a program. If the program fails (for example, by crashing, or by failing built-in code assertions), the defects can be noted. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior.
...
Fuzz testing is often used in large software development projects that employ black box testing. These projects usually have a budget to develop test tools, and fuzz testing is one of the techniques which offers a high benefit to cost ratio.
...
However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a bug-finding tool rather than an assurance of quality.

I've done a lot of this, and yes it sucks.
Some tips:
GroboUtils for running multiple test threads
alphaWorks ConTest to instrument classes to cause interleavings to vary between iterations
Create a throwable field and check it in tearDown (see Listing 1). If you catch a bad exception in another thread, just assign it to throwable.
I created the utils class in Listing 2 and have found it invaluable, especially waitForVerify and waitForCondition, which will greatly increase the performance of your tests.
Make good use of AtomicBoolean in your tests. It is thread safe, and you'll often need a final reference type to store values from callback classes and suchlike. See example in Listing 3.
Make sure to always give your test a timeout (e.g., #Test(timeout=60*1000)), as concurrency tests can sometimes hang forever when they're broken.
Listing 1:
#After
public void tearDown() {
if ( throwable != null )
throw throwable;
}
Listing 2:
import static org.junit.Assert.fail;
import java.io.File;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Proxy;
import java.util.Random;
import org.apache.commons.collections.Closure;
import org.apache.commons.collections.Predicate;
import org.apache.commons.lang.time.StopWatch;
import org.easymock.EasyMock;
import org.easymock.classextension.internal.ClassExtensionHelper;
import static org.easymock.classextension.EasyMock.*;
import ca.digitalrapids.io.DRFileUtils;
/**
* Various utilities for testing
*/
public abstract class DRTestUtils
{
static private Random random = new Random();
/** Calls {#link #waitForCondition(Integer, Integer, Predicate, String)} with
* default max wait and check period values.
*/
static public void waitForCondition(Predicate predicate, String errorMessage)
throws Throwable
{
waitForCondition(null, null, predicate, errorMessage);
}
/** Blocks until a condition is true, throwing an {#link AssertionError} if
* it does not become true during a given max time.
* #param maxWait_ms max time to wait for true condition. Optional; defaults
* to 30 * 1000 ms (30 seconds).
* #param checkPeriod_ms period at which to try the condition. Optional; defaults
* to 100 ms.
* #param predicate the condition
* #param errorMessage message use in the {#link AssertionError}
* #throws Throwable on {#link AssertionError} or any other exception/error
*/
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms,
Predicate predicate, String errorMessage) throws Throwable
{
waitForCondition(maxWait_ms, checkPeriod_ms, predicate, new Closure() {
public void execute(Object errorMessage)
{
fail((String)errorMessage);
}
}, errorMessage);
}
/** Blocks until a condition is true, running a closure if
* it does not become true during a given max time.
* #param maxWait_ms max time to wait for true condition. Optional; defaults
* to 30 * 1000 ms (30 seconds).
* #param checkPeriod_ms period at which to try the condition. Optional; defaults
* to 100 ms.
* #param predicate the condition
* #param closure closure to run
* #param argument argument for closure
* #throws Throwable on {#link AssertionError} or any other exception/error
*/
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms,
Predicate predicate, Closure closure, Object argument) throws Throwable
{
if ( maxWait_ms == null )
maxWait_ms = 30 * 1000;
if ( checkPeriod_ms == null )
checkPeriod_ms = 100;
StopWatch stopWatch = new StopWatch();
stopWatch.start();
while ( !predicate.evaluate(null) ) {
Thread.sleep(checkPeriod_ms);
if ( stopWatch.getTime() > maxWait_ms ) {
closure.execute(argument);
}
}
}
/** Calls {#link #waitForVerify(Integer, Object)} with <code>null</code>
* for {#code maxWait_ms}
*/
static public void waitForVerify(Object easyMockProxy)
throws Throwable
{
waitForVerify(null, easyMockProxy);
}
/** Repeatedly calls {#link EasyMock#verify(Object[])} until it succeeds, or a
* max wait time has elapsed.
* #param maxWait_ms Max wait time. <code>null</code> defaults to 30s.
* #param easyMockProxy Proxy to call verify on
* #throws Throwable
*/
static public void waitForVerify(Integer maxWait_ms, Object easyMockProxy)
throws Throwable
{
if ( maxWait_ms == null )
maxWait_ms = 30 * 1000;
StopWatch stopWatch = new StopWatch();
stopWatch.start();
for(;;) {
try
{
verify(easyMockProxy);
break;
}
catch (AssertionError e)
{
if ( stopWatch.getTime() > maxWait_ms )
throw e;
Thread.sleep(100);
}
}
}
/** Returns a path to a directory in the temp dir with the name of the given
* class. This is useful for temporary test files.
* #param aClass test class for which to create dir
* #return the path
*/
static public String getTestDirPathForTestClass(Object object)
{
String filename = object instanceof Class ?
((Class)object).getName() :
object.getClass().getName();
return DRFileUtils.getTempDir() + File.separator +
filename;
}
static public byte[] createRandomByteArray(int bytesLength)
{
byte[] sourceBytes = new byte[bytesLength];
random.nextBytes(sourceBytes);
return sourceBytes;
}
/** Returns <code>true</code> if the given object is an EasyMock mock object
*/
static public boolean isEasyMockMock(Object object) {
try {
InvocationHandler invocationHandler = Proxy
.getInvocationHandler(object);
return invocationHandler.getClass().getName().contains("easymock");
} catch (IllegalArgumentException e) {
return false;
}
}
}
Listing 3:
#Test
public void testSomething() {
final AtomicBoolean called = new AtomicBoolean(false);
subject.setCallback(new SomeCallback() {
public void callback(Object arg) {
// check arg here
called.set(true);
}
});
subject.run();
assertTrue(called.get());
}

I handle unit tests of threaded components the same way I handle any unit test, that is, with inversion of control and isolation frameworks. I develop in the .Net-arena and, out of the box, the threading (among other things) is very hard (I'd say nearly impossible) to fully isolate.
Therefore, I've written wrappers that looks something like this (simplified):
public interface IThread
{
void Start();
...
}
public class ThreadWrapper : IThread
{
private readonly Thread _thread;
public ThreadWrapper(ThreadStart threadStart)
{
_thread = new Thread(threadStart);
}
public Start()
{
_thread.Start();
}
}
public interface IThreadingManager
{
IThread CreateThread(ThreadStart threadStart);
}
public class ThreadingManager : IThreadingManager
{
public IThread CreateThread(ThreadStart threadStart)
{
return new ThreadWrapper(threadStart)
}
}
From there, I can easily inject the IThreadingManager into my components and use my isolation framework of choice to make the thread behave as I expect during the test.
That has so far worked great for me, and I use the same approach for the thread pool, things in System.Environment, Sleep etc. etc.

Pete Goodliffe has a series on the unit testing of threaded code.
It's hard. I take the easier way out and try to keep the threading code abstracted from the actual test. Pete does mention that the way I do it is wrong but I've either got the separation right or I've just been lucky.

For Java, check out chapter 12 of JCIP. There are some concrete examples of writing deterministic, multi-threaded unit tests to at least test the correctness and invariants of concurrent code.
"Proving" thread-safety with unit tests is much dicier. My belief is that this is better served by automated integration testing on a variety of platforms/configurations.

Have a look at my related answer at
Designing a Test class for a custom Barrier
It's biased towards Java but has a reasonable summary of the options.
In summary though (IMO) its not the use of some fancy framework that will ensure correctness but how you go about designing you multithreaded code. Splitting the concerns (concurrency and functionality) goes a huge way towards raising confidence. Growing Object Orientated Software Guided By Tests explains some options better than I can.
Static analysis and formal methods (see, Concurrency: State Models and Java Programs) is an option but I've found them to be of limited use in commercial development.
Don't forget that any load/soak style tests are rarely guaranteed to highlight problems.
Good luck!

I like to write two or more test methods to execute on parallel threads, and each of them make calls into the object under test. I've been using Sleep() calls to coordinate the order of the calls from the different threads, but that's not really reliable. It's also a lot slower because you have to sleep long enough that the timing usually works.
I found the Multithreaded TC Java library from the same group that wrote FindBugs. It lets you specify the order of events without using Sleep(), and it's reliable. I haven't tried it yet.
The biggest limitation to this approach is that it only lets you test the scenarios you suspect will cause trouble. As others have said, you really need to isolate your multithreaded code into a small number of simple classes to have any hope of thoroughly testing them.
Once you've carefully tested the scenarios you expect to cause trouble, an unscientific test that throws a bunch of simultaneous requests at the class for a while is a good way to look for unexpected trouble.
Update: I've played a bit with the Multithreaded TC Java library, and it works well. I've also ported some of its features to a .NET version I call TickingTest.

I just recently discovered (for Java) a tool called Threadsafe. It is a static analysis tool much like findbugs but specifically to spot multi-threading issues. It is not a replacement for testing but I can recommend it as part of writing reliable multi-threaded Java.
It even catches some very subtle potential issues around things like class subsumption, accessing unsafe objects through concurrent classes and spotting missing volatile modifiers when using the double checked locking paradigm.
If you write multithreaded Java give it a shot.

The following article suggests 2 solutions. Wrapping a semaphore (CountDownLatch) and adds functionality like externalize data from internal thread. Another way of achieving this purpose is to use Thread Pool (see Points of Interest).
Sprinkler - Advanced synchronization object

I spent most of last week at a university library studying debugging of concurrent code. The central problem is concurrent code is non-deterministic. Typically, academic debugging has fallen into one of three camps here:
Event-trace/replay. This requires an event monitor and then reviewing the events that were sent. In a UT framework, this would involve manually sending the events as part of a test, and then doing post-mortem reviews.
Scriptable. This is where you interact with the running code with a set of triggers. "On x > foo, baz()". This could be interpreted into a UT framework where you have a run-time system triggering a given test on a certain condition.
Interactive. This obviously won't work in an automatic testing situation. ;)
Now, as above commentators have noticed, you can design your concurrent system into a more deterministic state. However, if you don't do that properly, you're just back to designing a sequential system again.
My suggestion would be to focus on having a very strict design protocol about what gets threaded and what doesn't get threaded. If you constrain your interface so that there is minimal dependancies between elements, it is much easier.
Good luck, and keep working on the problem.

I have had the unfortunate task of testing threaded code and they are definitely the hardest tests I have ever written.
When writing my tests, I used a combination of delegates and events. Basically it is all about using PropertyNotifyChanged events with a WaitCallback or some kind of ConditionalWaiter that polls.
I am not sure if this was the best approach, but it has worked out for me.

Running multiple threads is not difficult; it is piece of cake. Unfortunately, threads usually need to communicate with each other; that's what's difficult.
The mechanism that was originally invented to allow communication between modules was function calls; when module A wants to communicate with module B, it just invokes a function in module B. Unfortunately, this does not work with threads, because when you call a function, that function still runs in the current thread.
To overcome this problem, people decided to fall back to an even more primitive mechanism of communication: just declare a certain variable, and let both threads have access to that variable. In other words, allow the threads to share data. Sharing data is literally the first thing that naturally comes to mind, and it appears like a good choice because it seems very simple. I mean, how hard can it be, right? What could possibly go wrong?
Race conditions. That's what can, and will, go wrong.
When people realized their software was suffering from random, non-reproducible catastrophic failures due to race conditions, they started inventing elaborate mechanisms such as locks and compare-and-swap, aiming to protect against such things happening. These mechanisms fall under the broad category of "synchronization". Unfortunately, synchronization has two problems:
It is very difficult to get it right, so it is very prone to bugs.
It is completely untestable, because you cannot test for a race condition.
The astute reader might notice that "Very prone to bugs" and "Completely untestable" is a deadly combination.
Now, the mechanisms I mentioned above were being invented and adopted by large parts of the industry before the concept of automated software testing became prevalent; So, nobody could see how deadly the problem was; they just regarded it as a difficult topic which requires guru programmers, and everyone was okay with that.
Nowadays, whatever we do, we put testing first. So, if some mechanism is untestable, then the use of that mechanism is just out of the question, period. Thus, synchronization has fallen out of grace; very few people still practice it, and they are becoming fewer and fewer every day.
Without synchronization threads cannot share data; however, the original requirement was not to share data; it was to allow threads to communicate with each other. Besides sharing data, there exist other, more elegant mechanisms for inter-thread communication.
One such mechanism is message-passing, otherwise known as events.
With message passing, there is only one place in the entire software system which utilizes synchronization, and that is the concurrent blocking queue collection class that we use for storing messages. (The idea is that we should be able to get at least that little part right.)
The great thing about message passing is that it does not suffer from race conditions and is fully testable.

Concurrency is a complex interplay between the memory model, hardware, caches and our code. In the case of Java at least such tests have been partly addressed mainly by jcstress. The creators of that library are known to be authors of many JVM, GC and Java concurrency features.
But even this library needs good knowledge of the Java Memory Model specification so that we know exactly what we are testing. But I think the focus of this effort is mircobenchmarks. Not huge business applications.

There is an article on the topic, using Rust as the language in the example code:
https://medium.com/#polyglot_factotum/rust-concurrency-five-easy-pieces-871f1c62906a
In summary, the trick is to write your concurrent logic so that it is robust to the non-determinism involved with multiple threads of execution, using tools like channels and condvars.
Then, if that is how you've structured your "components", the easiest way to test them is by using channels to send messages to them, and then block on other channels to assert that the component sends certain expected messages.
The linked-to article is fully written using unit-tests.

It's not perfect, but I wrote this helper for my tests in C#:
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace Proto.Promises.Tests.Threading
{
public class ThreadHelper
{
public static readonly int multiThreadCount = Environment.ProcessorCount * 100;
private static readonly int[] offsets = new int[] { 0, 10, 100, 1000 };
private readonly Stack<Task> _executingTasks = new Stack<Task>(multiThreadCount);
private readonly Barrier _barrier = new Barrier(1);
private int _currentParticipants = 0;
private readonly TimeSpan _timeout;
public ThreadHelper() : this(TimeSpan.FromSeconds(10)) { } // 10 second timeout should be enough for most cases.
public ThreadHelper(TimeSpan timeout)
{
_timeout = timeout;
}
/// <summary>
/// Execute the action multiple times in parallel threads.
/// </summary>
public void ExecuteMultiActionParallel(Action action)
{
for (int i = 0; i < multiThreadCount; ++i)
{
AddParallelAction(action);
}
ExecutePendingParallelActions();
}
/// <summary>
/// Execute the action once in a separate thread.
/// </summary>
public void ExecuteSingleAction(Action action)
{
AddParallelAction(action);
ExecutePendingParallelActions();
}
/// <summary>
/// Add an action to be run in parallel.
/// </summary>
public void AddParallelAction(Action action)
{
var taskSource = new TaskCompletionSource<bool>();
lock (_executingTasks)
{
++_currentParticipants;
_barrier.AddParticipant();
_executingTasks.Push(taskSource.Task);
}
new Thread(() =>
{
try
{
_barrier.SignalAndWait(); // Try to make actions run in lock-step to increase likelihood of breaking race conditions.
action.Invoke();
taskSource.SetResult(true);
}
catch (Exception e)
{
taskSource.SetException(e);
}
}).Start();
}
/// <summary>
/// Runs the pending actions in parallel, attempting to run them in lock-step.
/// </summary>
public void ExecutePendingParallelActions()
{
Task[] tasks;
lock (_executingTasks)
{
_barrier.SignalAndWait();
_barrier.RemoveParticipants(_currentParticipants);
_currentParticipants = 0;
tasks = _executingTasks.ToArray();
_executingTasks.Clear();
}
try
{
if (!Task.WaitAll(tasks, _timeout))
{
throw new TimeoutException($"Action(s) timed out after {_timeout}, there may be a deadlock.");
}
}
catch (AggregateException e)
{
// Only throw one exception instead of aggregate to try to avoid overloading the test error output.
throw e.Flatten().InnerException;
}
}
/// <summary>
/// Run each action in parallel multiple times with differing offsets for each run.
/// <para/>The number of runs is 4^actions.Length, so be careful if you don't want the test to run too long.
/// </summary>
/// <param name="expandToProcessorCount">If true, copies each action on additional threads up to the processor count. This can help test more without increasing the time it takes to complete.
/// <para/>Example: 2 actions with 6 processors, runs each action 3 times in parallel.</param>
/// <param name="setup">The action to run before each parallel run.</param>
/// <param name="teardown">The action to run after each parallel run.</param>
/// <param name="actions">The actions to run in parallel.</param>
public void ExecuteParallelActionsWithOffsets(bool expandToProcessorCount, Action setup, Action teardown, params Action[] actions)
{
setup += () => { };
teardown += () => { };
int actionCount = actions.Length;
int expandCount = expandToProcessorCount ? Math.Max(Environment.ProcessorCount / actionCount, 1) : 1;
foreach (var combo in GenerateCombinations(offsets, actionCount))
{
setup.Invoke();
for (int k = 0; k < expandCount; ++k)
{
for (int i = 0; i < actionCount; ++i)
{
int offset = combo[i];
Action action = actions[i];
AddParallelAction(() =>
{
for (int j = offset; j > 0; --j) { } // Just spin in a loop for the offset.
action.Invoke();
});
}
}
ExecutePendingParallelActions();
teardown.Invoke();
}
}
// Input: [1, 2, 3], 3
// Ouput: [
// [1, 1, 1],
// [2, 1, 1],
// [3, 1, 1],
// [1, 2, 1],
// [2, 2, 1],
// [3, 2, 1],
// [1, 3, 1],
// [2, 3, 1],
// [3, 3, 1],
// [1, 1, 2],
// [2, 1, 2],
// [3, 1, 2],
// [1, 2, 2],
// [2, 2, 2],
// [3, 2, 2],
// [1, 3, 2],
// [2, 3, 2],
// [3, 3, 2],
// [1, 1, 3],
// [2, 1, 3],
// [3, 1, 3],
// [1, 2, 3],
// [2, 2, 3],
// [3, 2, 3],
// [1, 3, 3],
// [2, 3, 3],
// [3, 3, 3]
// ]
private static IEnumerable<int[]> GenerateCombinations(int[] options, int count)
{
int[] indexTracker = new int[count];
int[] combo = new int[count];
for (int i = 0; i < count; ++i)
{
combo[i] = options[0];
}
// Same algorithm as picking a combination lock.
int rollovers = 0;
while (rollovers < count)
{
yield return combo; // No need to duplicate the array since we're just reading it.
for (int i = 0; i < count; ++i)
{
int index = ++indexTracker[i];
if (index == options.Length)
{
indexTracker[i] = 0;
combo[i] = options[0];
if (i == rollovers)
{
++rollovers;
}
}
else
{
combo[i] = options[index];
break;
}
}
}
}
}
}
Example usage:
[Test]
public void DeferredMayBeBeResolvedAndPromiseAwaitedConcurrently_void0()
{
Promise.Deferred deferred = default(Promise.Deferred);
Promise promise = default(Promise);
int invokedCount = 0;
var threadHelper = new ThreadHelper();
threadHelper.ExecuteParallelActionsWithOffsets(false,
// Setup
() =>
{
invokedCount = 0;
deferred = Promise.NewDeferred();
promise = deferred.Promise;
},
// Teardown
() => Assert.AreEqual(1, invokedCount),
// Parallel Actions
() => deferred.Resolve(),
() => promise.Then(() => { Interlocked.Increment(ref invokedCount); }).Forget()
);
}

One simple test pattern that can work for some (not all!) cases is to repeat the same test many times. For example, suppose you have a method:
def process(input):
# Spawns several threads to do the job
# ...
return output
Create a bunch of tests:
process(input1) -> expect to return output1
process(input2) -> expect to return output2
...
Now run each of those tests many times.
If the implementation of process contains a subtle bug (e.g. deadlock, race condition, etc.) that has 0.1% chance to emerge, running the test 1000 times gives 64% probability for the bug to emerge at least once. Running the test 10000 times gives >99% probability.

If you are testing simple new Thread(runnable).run()
You can mock Thread to run the runnable sequentially
For instance, if the code of the tested object invokes a new thread like this
Class TestedClass {
public void doAsychOp() {
new Thread(new myRunnable()).start();
}
}
Then mocking new Threads and run the runnable argument sequentially can help
#Mock
private Thread threadMock;
#Test
public void myTest() throws Exception {
PowerMockito.mockStatic(Thread.class);
//when new thread is created execute runnable immediately
PowerMockito.whenNew(Thread.class).withAnyArguments().then(new Answer<Thread>() {
#Override
public Thread answer(InvocationOnMock invocation) throws Throwable {
// immediately run the runnable
Runnable runnable = invocation.getArgumentAt(0, Runnable.class);
if(runnable != null) {
runnable.run();
}
return threadMock;//return a mock so Thread.start() will do nothing
}
});
TestedClass testcls = new TestedClass()
testcls.doAsychOp(); //will invoke myRunnable.run in current thread
//.... check expected
}

(if possible) don't use threads, use actors / active objects. Easy to test.

You may use EasyMock.makeThreadSafe to make testing instance threadsafe

Time taken to execute all methods in a method stack

A lot of times while writing applications, I wish to profile and measure the time taken for all methods in a stacktrace. What I mean is say:
Method A --> Method B --> Method C ...
A method internally calls B and it might call another. I wish to know the time taken to execute inside each method. This way in a web application, I can precisely know the percentage of time being consumed by what part of the code.
To explain further, most of the times in spring application, I write an aspect which collects information for every method call of a class. Which finally gives me summary. But I hate doing this, its repetitive and verbose and need to keep changing regex to accommodate different classes. Instead I would like this:
#Monitor
public void generateReport(int id){
...
}
Adding some annotation on method will trigger instrumentation api to collect all statistics of time taken by this method and any method later called. And when this method is exited, it stops collection information. I think this should be relatively easy to implement.
The questions is: Are there any reasonable alternatives that lets me do that for general java code? Or any quick way of collection this information. Or even a spring plugin for spring applications?
PS: Exactly like XRebel, it generates beautiful summaries of time take by the security, dao, service etc part of code. But it costs a bomb. If you can afford, you should definitely buy it.

You want to write a Java agent. Such an agent allows you to redefine a class when it is loaded. This way, you can implement an aspect without polluting your source code. I have written a library, Byte Buddy, which makes this fairly easy.
For your monitor example, you could use Byte Buddy as follows:
new AgentBuilder.Default()
.rebase(declaresMethod(isAnnotatedWith(Monitor.class))
.transform( (builder, type) ->
builder
.method(isAnnotatedWith(Monitor.class))
.intercept(MethodDelegation.to(MonitorInterceptor.class);
);
class MonitorInterceptor {
#RuntimeType
Object intercept(#Origin String method,
#SuperCall Callable<?> zuper)
throws Exception {
long start = System.currentTimeMillis();
try {
return zuper.call();
} finally {
System.out.println(method + " took " + (System.currentTimeMillis() - start);
}
}
}
The above built agent can than be installed on an instance of the instrumentation interface which is provided to any Java agent.
As an advantage over using Spring, the above agent will work for any Java instance, not only for Spring beans.

I don't know if theres already a library doing it nor can I give you a ready to use code. But I can give you a description how you can implement it on your own.
First of all i assume its no problem to include AspectJ into your project. Than create an annotation f.e. #Monitor which acts as marker for the time measurment of whatever you like.
Than create a simple data strucutre holding the information you wana track.
An example for this could be the following :
public class OperationMonitoring {
boolean active=false;
List<MethodExecution> methodExecutions = new ArrayList<>();
}
public class MethodExecution {
MethodExcecution invoker;
List<MethodExeuction> invocations = new ArrayList<>();
long startTime;
long endTime;
}
Than create an Around advice for all methods. On execution check if the called Method is annotated with your Monitoring annotation. If yes started monitoring each method execution in this thread. A simple example code could look like:
#Aspect
public class MonitoringAspect {
private ThreadLocal<OperationMonitoring> operationMonitorings = new ThreadLocal<>();
#Around("execution(* *.*(..))")
public void monitoring(ProceedingJoinPoint pjp) {
Method method = extractMethod(pjp);
if (method != null) {
OperationMonitoring monitoring = null;
if(method.isAnnotationPresent(Monitoring.class){
monitoring = operationMonitorings.get();
if(monitoring!=null){
if(!monitoring.active) {
monitoring.active=true;
}
} else {
// Create new OperationMonitoring object and set it
}
}
if(monitoring == null){
// this method is not annotated but is the tracking already active?
monitoring = operationMonitoring.get();
}
if(monitoring!=null && monitoring.active){
// do monitoring stuff and invoke the called method
} else {
// invoke the called method without monitoring
}
// Stop the monitoring by setting monitoring.active=false if this method was annotated with Monitoring (and it started the monitoring).
}
}
private Method extractMethod(JoinPoint joinPoint) {
if (joinPoint.getKind().equals(JoinPoint.METHOD_EXECUTION) && joinPoint.getSignature() instanceof MethodSignature) {
return ((MethodSignature) joinPoint.getSignature()).getMethod();
}
return null;
}
}
The code above is just a how to. I would also restructure the code but I've written it in a textfield, so please be aware of architectural flaws. As mentioned with a comment at the end. This solution does not supporte multiple annotated methods along the way. But it would be easy to add this.
A limitation of this approach is that it fails when you start additional threads during a tracked path. Adding support for starting new threads in a monitored Thread is not that easy. Thats also the reason why IoC frameworks have own features for handling threads to be able to track this.
I hope you understand the general concept of this, if not feel free to ask further questions.

This is the exact reason why I built the open source tool stagemonitor, which uses Byte Buddy to insert profiling code. If you want to monitor a web application you don't have to alter or annotate your code. If you have a standalone application, there is a #MonitorRequests annotation you can use.

You say you want to know the percentage of time taken within each routine on the stack.
I assume you mean inclusive time.
I also assume you mean wall-clock time, on the theory that if one of those lower-level callees happens to do some I/O, locking, etc., you don't want to be blind to that.
So a stack-sampling profiler that samples on wall-clock time will be getting the right kind of information.
The percentage time that A takes is the percentage of samples containing A, same for B, etc.
To get the percentage of A's time used by B, it is the percentage of samples containing A that happen to have B at the next level below.
The information is all in the stack samples, but it may be hard to get the profiler to extract just the information you want.
You also say you want precise percentage.
That means you also need a large number of stack samples.
For example, if you want to shrink the uncertainty of your measurements by a factor of 10, you need 100 times as many samples.
In my experience finding performance problems, I am willing to tolerate an uncertainty of 10% or more, because my goal is to find big wastage, not to know with precision how bad it is.
So I take samples manually, and look at them manually.
In fact, if you look at the statistics, you only have to see something wasteful on as few as two samples to know it's bad, and the fewer samples you take before seeing it twice, the worse it is.
(Example: If the problem wastes 30% of time, it takes on average 2/30% = 6.67 samples to see it twice. If it wastes 90% of time, it only takes 2.2 samples, on average.)

What's the difference between a Future and a Promise?

What's the difference between Future and Promise?
They both act like a placeholder for future results, but where is the main difference?

(I'm not completely happy with the answers so far, so here is my attempt...)
I think that Kevin Wright's comment
You can make a Promise and it's up to you to keep it. When someone else makes you a promise you must wait to see if they honour it in the Future
summarizes it pretty well, but some explanation can be useful.
Futures and promises are pretty similar concepts, the difference is that a future is a read-only container for a result that does not yet exist, while a promise can be written (normally only once). The Java 8 CompletableFuture and the Guava SettableFuture can be thought of as promises, because their value can be set ("completed"), but they also implement the Future interface, therefore there is no difference for the client.
The result of the future will be set by "someone else" - by the result of an asynchronous computation. Note how FutureTask - a classic future - must be initialized with a Callable or Runnable, there is no no-argument constructor, and both Future and FutureTask are read-only from the outside (the set methods of FutureTask are protected). The value will be set to the result of the computation from the inside.
On the other hand, the result of a promise can be set by "you" (or in fact by anybody) anytime because it has a public setter method. Both CompletableFuture and SettableFuture can be created without any task, and their value can be set at any time. You send a promise to the client code, and fulfill it later as you wish.
Note that CompletableFuture is not a "pure" promise, it can be initialized with a task just like FutureTask, and its most useful feature is the unrelated chaining of processing steps.
Also note that a promise does not have to be a subtype of future and it does not have to be the same object. In Scala a Future object is created by an asynchronous computation or by a different Promise object. In C++ the situation is similar: the promise object is used by the producer and the future object by the consumer. The advantage of this separation is that the client cannot set the value of the future.
Both Spring and EJB 3.1 have an AsyncResult class, which is similar to the Scala/C++ promises. AsyncResult does implement Future but this is not the real future: asynchronous methods in Spring/EJB return a different, read-only Future object through some background magic, and this second "real" future can be used by the client to access the result.

According to this discussion, Promise has finally been called CompletableFuture for inclusion in Java 8, and its javadoc explains:
A Future that may be explicitly completed (setting its value and status), and may be used as a CompletionStage, supporting dependent functions and actions that trigger upon its completion.
An example is also given on the list:
f.then((s -> aStringFunction(s)).thenAsync(s -> ...);
Note that the final API is slightly different but allows similar asynchronous execution:
CompletableFuture<String> f = ...;
f.thenApply(this::modifyString).thenAccept(System.out::println);

I am aware that there's already an accepted answer but would like to add my two cents nevertheless:
TLDR: Future and Promise are the two sides of an asynchronous operation: consumer/caller vs. producer/implementor.
As a caller of an asynchronous API method, you will get a Future as a handle to the computation's result. You can e.g. call get() on it to wait for the computation to complete and retrieve the result.
Now think of how this API method is actually implemented: The implementor must return a Future immediately. They are responsible for completing that future as soon as the computation is done (which they will know because it is implementing the dispatch logic ;-)). They will use a Promise/CompletableFuture to do just that: Construct and return the CompletableFuture immediately, and call complete(T result) once the computation is done.

I will give an example of what is Promise and how its value could be set at any time, in opposite to Future, which value is only readable.
Suppose you have a mom and you ask her for money.
// Now , you trick your mom into creating you a promise of eventual
// donation, she gives you that promise object, but she is not really
// in rush to fulfill it yet:
Supplier<Integer> momsPurse = ()-> {
try {
Thread.sleep(1000);//mom is busy
} catch (InterruptedException e) {
;
}
return 100;
};
ExecutorService ex = Executors.newFixedThreadPool(10);
CompletableFuture<Integer> promise =
CompletableFuture.supplyAsync(momsPurse, ex);
// You are happy, you run to thank you your mom:
promise.thenAccept(u->System.out.println("Thank you mom for $" + u ));
// But your father interferes and generally aborts mom's plans and
// completes the promise (sets its value!) with far lesser contribution,
// as fathers do, very resolutely, while mom is slowly opening her purse
// (remember the Thread.sleep(...)) :
promise.complete(10);
Output of that is:
Thank you mom for $10
Mom's promise was created , but waited for some "completion" event.
CompletableFuture<Integer> promise...
You created such event, accepting her promise and announcing your plans to thank your mom:
promise.thenAccept...
At this moment mom started open her purse...but very slow...
and father interfered much faster and completed the promise instead of your mom:
promise.complete(10);
Have you noticed an executor that I wrote explicitly?
Interestingly, if you use a default implicit executor instead (commonPool) and father is not at home, but only mom with her "slow purse", then her promise will only complete, if the program lives longer than mom needs to get money from the purse.
The default executor acts kind of like a "daemon" and does not wait for all promises to be fulfilled. I have not found a good description of this fact...

Not sure if this can be an answer but as I see what others have said for someone it may look like you need two separate abstractions for both of these concepts so that one of them (Future) is just a read-only view of the other (Promise) ... but actually this is not needed.
For example take a look at how promises are defined in javascript:
https://promisesaplus.com/
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
The focus is on the composability using the then method like:
asyncOp1()
.then(function(op1Result){
// do something
return asyncOp2();
})
.then(function(op2Result){
// do something more
return asyncOp3();
})
.then(function(op3Result){
// do something even more
return syncOp4(op3Result);
})
...
.then(function(result){
console.log(result);
})
.catch(function(error){
console.log(error);
})
which makes asynchronous computation to look like synchronous:
try {
op1Result = syncOp1();
// do something
op1Result = syncOp2();
// do something more
op3Result = syncOp3();
// do something even more
syncOp4(op3Result);
...
console.log(result);
} catch(error) {
console.log(error);
}
which is pretty cool.
(Not as cool as async-await but async-await just removes the boilerplate ....then(function(result) {.... from it).
And actually their abstraction is pretty good as the promise constructor
new Promise( function(resolve, reject) { /* do it */ } );
allows you to provide two callbacks which can be used to either complete the Promise successfully or with an error. So that only the code that constructs the Promise can complete it and the code that receives an already constructed Promise object has the read-only view.
With inheritance the above can be achieved if resolve and reject are protected methods.

For client code, Promise is for observing or attaching callback when a result is available, whereas Future is to wait for result and then continue. Theoretically anything which is possible to do with futures what can done with promises, but due to the style difference, the resultant API for promises in different languages make chaining easier.

Future vs Promise
Future and Promise are proxy object for unknown result
Promise completes a Future
Promise - write/producer of unknown result.
Future - read/consumer of unknown result. It has next states: pending, fulfilled, canceled
//Future has a reference to Promise
Future -> Promise
As a producer I promise something and responsible for it
As a consumer who retrieved a promise I expect to have a result in future. In future I can use the promise or reject it
As for Java CompletableFutures it is a Promise because you can set the result and also it implements Future

No set method in Future interface, only get method, so it is read-only.
About CompletableFuture, this article maybe helpful.
completablefuture

In this example you can take a look at how Promises can be used in Java
for creating asynchronous sequences of calls:
doSomeProcess()
.whenResult(result -> System.out.println(String.format("Result of some process is '%s'", result)))
.whenException(e -> System.out.println(String.format("Exception after some process is '%s'", e.getMessage())))
.map(String::toLowerCase)
.mapEx((result, e) -> e == null ? String.format("The mapped result is '%s'", result) : e.getMessage())
.whenResult(s -> System.out.println(s));

Computation with time limit

I'm trying to write a construct which allows me to run computations in a given time window. Something like:
def expensiveComputation(): Double = //... some intensive math
val result: Option[Double] = timeLimited( 45 ) { expensiveComputation() }
Here the timeLimited will run expensiveComputation with a timeout of 45 minutes. If it reaches the timeout it returns None, else it wrapped the result into Some.
I am looking for a solution which:
Is pretty cheap in performance and memory;
Will run the time-limited task in the current thread.
Any suggestion ?
EDIT
I understand my original problem has no solution. Say I can create a thread for the calculation (but I prefer not using a threadpool/executor/dispatcher). What's the fastest, safest and cleanest way to do it ?

Runs the given code block or throws an exception on timeout:
#throws(classOf[java.util.concurrent.TimeoutException])
def timedRun[F](timeout: Long)(f: => F): F = {
import java.util.concurrent.{Callable, FutureTask, TimeUnit}
val task = new FutureTask(new Callable[F]() {
def call() = f
})
new Thread(task).start()
task.get(timeout, TimeUnit.MILLISECONDS)
}

Only an idea: I am not so familiar with akka futures. But perhaps its possible to stick the future executing thread to the current thread and use akka futures with timeouts?

To the best of my knowledge, either you yield (the computation calls to some scheduler) or you use a thread, which gets manipulated from the "outside".

If you want to run the task in the current thread and if there should be no other threads involved, you would have to check whether the time limit is over inside of expensiveComputation. For example, if expensiveComputation is a loop, you could check for the time after each iteration.

If you are ok for the code of expensiveComputation to check Thread.interrupted() frequently, pretty easy. But I suppose you are not.
I don't think there is any solution that will work for arbitrary expensiveComputation code.
The question is what are you prepared to have as constraint on expensiveComputation.
You have the deprecated and quite unsafe Thead.stop(Throwable) too. If your code does not modify any object but those it created by itself, it might work.

I saw a pattern like this work well for time-limited tasks (Java code):
try {
setTimeout(45*60*1000); // 45 min in ms
while (not done) {
checkTimeout();
// do some stuff
// if the stuff can take long, again:
checkTimeout();
// do some more stuff
}
return Some(result);
}
catch (TimeoutException ex) {
return None;
}
The checkTimeout() function is cheap to call; you add it to code so that it is called reasonably often, but not too often. All it does is check current time against timer value set by setTimeout() plus the timeout value. If current time exceeds that value, checkTimeout() raises a TimeoutException.
I hope this logic can be reproduced in Scala, too.

For a generic solution (without having to go litter each of your expensiveComputations with checkTimeout() code) perhaps use Javassist.
http://www.csg.is.titech.ac.jp/~chiba/javassist/
You can then insert various checkTimeout() methods dynamically.
Here is the intro text on their website:
Javassist (Java Programming Assistant) makes Java bytecode manipulation simple. It is a class library for editing bytecodes in Java; it enables Java programs to define a new class at runtime and to modify a class file when the JVM loads it. Unlike other similar bytecode editors, Javassist provides two levels of API: source level and bytecode level. If the users use the source-level API, they can edit a class file without knowledge of the specifications of the Java bytecode. The whole API is designed with only the vocabulary of the Java language. You can even specify inserted bytecode in the form of source text; Javassist compiles it on the fly. On the other hand, the bytecode-level API allows the users to directly edit a class file as other editors.
Aspect Oriented Programming: Javassist can be a good tool for adding new methods into a class and for inserting before/after/around advice at the both caller and callee sides.
Reflection: One of applications of Javassist is runtime reflection; Javassist enables Java programs to use a metaobject that controls method calls on base-level objects. No specialized compiler or virtual machine are needed.

In the currentThread?? Phhhew...
Check after each step in computation
Well if your "expensive computation" can be broken up into multiple steps or has iterative logic you could capture the time when you start and then check periodically between your steps. This is by no means a generic solution but will work.
For a more generic solution you might make use of aspects or annotation processing, that automatically litters your code with these checks. If the "check" tells you that your time is up return None.
Ill ponder a solution in java quickly below using annotations and an annotation processor...
public abstract Answer{}
public class Some extends Answer {public Answer(double answer){answer=answer}Double answer = null;}
public class None extends Answer {}
//This is the method before annotation processing
#TimeLimit(45)
public Answer CalculateQuestionToAnswerOf42() {
double fairydust = Math.Pi * 1.618;
double moonshadowdrops = (222.21) ^5;
double thedevil == 222*3;
return new Answer(fairydust + moonshadowdrops + thedevil);
}
//After annotation processing
public Answer calculateQuestionToAnswerOf42() {
Date start = new Date() // added via annotation processing;
double fairydust = Math.Pi * 1.618;
if(checkTimeout(start, 45)) return None; // added via annotation processing;
double moonshadowdrops = (222.21) ^5;
if(checkTimeout(start, 45)) return None; // added via annotation processing;
double thedevil == 222*3;
if(checkTimeout(start, 45)) return None; // added via annotation processing;
return new Answer(fairydust + moonshadowdrops + thedevil);
}

If you're very seriously in need of this you could create a compiler plugin that inserts check blocks in loops and conditions. These check blocks can then check Thread.isInterrupted() and throw an Exception to escape.
You could possibly use an annotation, i.e. #interruptible, to mark the methods to enhance.

We Keep Coding

Java is a programming language and computing platform first released by Sun Microsystems in 1995.