We Keep Coding

Real world example of Memory Consistency Errors in multi-threading?

In the tutorial of java multi-threading, it gives an exmaple of Memory Consistency Errors. But I can not reproduce it. Is there any other method to simulate Memory Consistency Errors?
The example provided in the tutorial:
Suppose a simple int field is defined and initialized:
int counter = 0;
The counter field is shared between two threads, A and B. Suppose thread A increments counter:
counter++;
Then, shortly afterwards, thread B prints out counter:
System.out.println(counter);
If the two statements had been executed in the same thread, it would be safe to assume that the value printed out would be "1". But if the two statements are executed in separate threads, the value printed out might well be "0", because there's no guarantee that thread A's change to counter will be visible to thread B — unless the programmer has established a happens-before relationship between these two statements.

I answered a question a while ago about a bug in Java 5. Why doesn't volatile in java 5+ ensure visibility from another thread?
Given this piece of code:
public class Test {
volatile static private int a;
static private int b;
public static void main(String [] args) throws Exception {
for (int i = 0; i < 100; i++) {
new Thread() {
#Override
public void run() {
int tt = b; // makes the jvm cache the value of b
while (a==0) {
}
if (b == 0) {
System.out.println("error");
}
}
}.start();
}
b = 1;
a = 1;
}
}
The volatile store of a happens after the normal store of b. So when the thread runs and sees a != 0, because of the rules defined in the JMM, we must see b == 1.
The bug in the JRE allowed the thread to make it to the error line and was subsequently resolved. This definitely would fail if you don't have a defined as volatile.

This might reproduce the problem, at least on my computer, I can reproduce it after some loops.
Suppose you have a Counter class:
class Holder {
boolean flag = false;
long modifyTime = Long.MAX_VALUE;
}
Let thread_A set flag as true, and save the time into
modifyTime.
Let another thread, let's say thread_B, read the Counter's flag. If thread_B still get false even when it is later than modifyTime, then we can say we have reproduced the problem.
Example code
class Holder {
boolean flag = false;
long modifyTime = Long.MAX_VALUE;
}
public class App {
public static void main(String[] args) {
while (!test());
}
private static boolean test() {
final Holder holder = new Holder();
new Thread(new Runnable() {
#Override
public void run() {
try {
Thread.sleep(10);
holder.flag = true;
holder.modifyTime = System.currentTimeMillis();
} catch (Exception e) {
e.printStackTrace();
}
}
}).start();
long lastCheckStartTime = 0L;
long lastCheckFailTime = 0L;
while (true) {
lastCheckStartTime = System.currentTimeMillis();
if (holder.flag) {
break;
} else {
lastCheckFailTime = System.currentTimeMillis();
System.out.println(lastCheckFailTime);
}
}
if (lastCheckFailTime > holder.modifyTime
&& lastCheckStartTime > holder.modifyTime) {
System.out.println("last check fail time " + lastCheckFailTime);
System.out.println("modify time " + holder.modifyTime);
return true;
} else {
return false;
}
}
}
Result
last check time 1565285999497
modify time 1565285999494
This means thread_B get false from Counter's flag filed at time 1565285999497, even thread_A has set it as true at time 1565285999494(3 milli seconds ealier).

The example used is too bad to demonstrate the memory consistency issue. Making it work will require brittle reasoning and complicated coding. Yet you may not be able to see the results. Multi-threading issues occur due to unlucky timing. If someone wants to increase the chances of observing issue, we need to increase chances of unlucky timing.
Following program achieves it.
public class ConsistencyIssue {
static int counter = 0;
public static void main(String[] args) throws InterruptedException {
Thread thread1 = new Thread(new Increment(), "Thread-1");
Thread thread2 = new Thread(new Increment(), "Thread-2");
thread1.start();
thread2.start();
thread1.join();
thread2.join();
System.out.println(counter);
}
private static class Increment implements Runnable{
#Override
public void run() {
for(int i = 1; i <= 10000; i++)
counter++;
}
}
}
Execution 1 output: 10963,
Execution 2 output: 14552
Final count should have been 20000, but it is less than that. Reason is count++ is multi step operation,
1. read count
2. increment count
3. store it
two threads may read say count 1 at once, increment it to 2. and write out 2. But if it was a serial execution it should have been 1++ -> 2++ -> 3.
We need a way to make all 3 steps atomic. i.e to be executed by only one thread at a time.
Solution 1: Synchronized
Surround the increment with Synchronized. Since counter is static variable you need to use class level synchronization
#Override
public void run() {
for (int i = 1; i <= 10000; i++)
synchronized (ConsistencyIssue.class) {
counter++;
}
}
Now it outputs: 20000
Solution 2: AtomicInteger
public class ConsistencyIssue {
static AtomicInteger counter = new AtomicInteger(0);
public static void main(String[] args) throws InterruptedException {
Thread thread1 = new Thread(new Increment(), "Thread-1");
Thread thread2 = new Thread(new Increment(), "Thread-2");
thread1.start();
thread2.start();
thread1.join();
thread2.join();
System.out.println(counter.get());
}
private static class Increment implements Runnable {
#Override
public void run() {
for (int i = 1; i <= 10000; i++)
counter.incrementAndGet();
}
}
}
We can do with semaphores, explicit locking too. but for this simple code AtomicInteger is enough

Sometimes when I try to reproduce some real concurrency problems, I use the debugger.
Make a breakpoint on the print and a breakpoint on the increment and run the whole thing.
Releasing the breakpoints in different sequences gives different results.
Maybe to simple but it worked for me.

Please have another look at how the example is introduced in your source.
The key to avoiding memory consistency errors is understanding the happens-before relationship. This relationship is simply a guarantee that memory writes by one specific statement are visible to another specific statement. To see this, consider the following example.
This example illustrates the fact that multi-threading is not deterministic, in the sense that you get no guarantee about the order in which operations of different threads will be executed, which might result in different observations across several runs. But it does not illustrate a memory consistency error!
To understand what a memory consistency error is, you need to first get an insight about memory consistency. The simplest model of memory consistency has been introduced by Lamport in 1979. Here is the original definition.
The result of any execution is the same as if the operations of all the processes were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program
Now, consider this example multi-threaded program, please have a look at this image from a more recent research paper about sequential consistency. It illustrates what a real memory consistency error might look like.
To finally answer your question, please note the following points:
A memory consistency error always depends on the underlying memory model (A particular programming languages may allow more behaviours for optimization purposes). What's the best memory model is still an open research question.
The example given above gives an example of sequential consistency violation, but there is no guarantee that you can observe it with your favorite programming language, for two reasons: it depends on the programming language exact memory model, and due to undeterminism, you have no way to force a particular incorrect execution.
Memory models are a wide topic. To get more information, you can for example have a look at Torsten Hoefler and Markus Püschel course at ETH Zürich, from which I understood most of these concepts.
Sources
Leslie Lamport. How to Make a Multiprocessor Computer That Correctly Executes Multiprocessor Programs, 1979
Wei-Yu Chen, Arvind Krishnamurthy, Katherine Yelick, Polynomial-Time Algorithms for Enforcing Sequential Consistency in SPMD Programs with Arrays, 2003
Design of Parallel and High-Performance Computing course, ETH Zürich

Java - Implement threading in large input set

I need to do some computations/processing on a large set of ids (about 100k to 1 Million). Since the number of ids is quite large and each processing does take some time, i was thinking about implementing threads in my Java code.
Assuming we cant have 100K threads running at once, how do i implement threading in this case ?
Note - The only solution i can think of is have about 100 or more threads running where each thread would process about a 1000 or more IDs.

Use Java's built in thread pooling and executors.
ExecutorService foo = Executors.newFixedThreadPool(100);
foo.submit(new MyRunnable());
There are various thread pools you can create to tailor how many you want, if it's dynamic, etc.

Using ThreadPool:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class ThreadIDS implements Runnable
{
public static final int totalIDS = 1000000;
int start;
int range;
public ThreadIDS(int start, int range)
{
this.start=start;
this.range=range;
}
public static void main(String[] args)
{
int availableProcessors = Runtime.getRuntime().availableProcessors();
int eachThread = totalIDS/availableProcessors + 1;
ExecutorService threads = Executors.newFixedThreadPool(availableProcessors);
for(int i = 0 ; i < availableProcessors ; i++)
{
threads.submit(new ThreadIDS(i*eachThread, eachThread));
}
while(!threads.awaitTermination(1000, TimeUnit.MILLISECONDS))System.out.println("Waiting for threads to finish");
}
public void processID(int id)
{
}
public void run()
{
for(int i = start ; i < Math.min(start+range, totalIDS) ; i++)
{
processID(i);
}
}
}
Edited the run method. Since we add 1 when dividing to avoid integer division making us miss ids, we could potentially run over the totalIDS limit. The Math.min avoids that.
If you don't want to use ThreadPools, then change the main to:
public static void main(String[] args)
{
int availableProcessors = Runtime.getRuntime().availableProcessors();
int eachThread = totalIDS/availableProcessors + 1;
for(int i = 0 ; i < availableProcessors ; i++)
{
new Thread(new ThreadIDS(i * eachThread, eachThread)).start();
}
}

Run as many threads as you have CPU cores (Runtime.getRuntime().availableProcessors()). Let each thread runs loop like this:
public void run() {
while (!ids.isEmpty()) {
Id id = ids.poll(); // exact access method depends on how your set of ids is organized
processId(id);
}
}
Comparing to using thread pool, this is simpler and requires less memory (no need to create Runnable for each id).

Splitting your work into 4 Runnables (1 per core) is probably not the best idea if there is any variation in processing time for a given ID. A better solution would be to split your work up into small chunks so that one core doesn't get stuck with all the "hard" work while the other 3 cores plow through theirs and then do nothing.
You could split your tasks into small chunks in advance and submit them to a ThreadPoolExecutor, but it might be better to use the Fork/Join framework. It's designed to handle this type of thing very efficiently.
Something like this would make sure all 4 cores stayed busy until all the work was done:
public class Test
{
public void workTest()
{
ForkJoinPool pool = new ForkJoinPool(); //Defaults to # of cores
List<ObjectThatWeProcess> work = getWork(); //Get IDs or whatever
FJAction action = new FJAction(work);
pool.invoke(action);
}
public static class FJAction extends RecursiveAction
{
private static final workSize = 1000; //Only do work if 1000 objects or less
List<ObjectThatWeProcess> work;
FJAction(List<ObjectThatWeProcess> work)
{
this.work = work;
}
public void compute()
{
if(work.size() > workSize)
{
invokeAll(new FJAction(work.subList(0,work.size()/2)),
new FJAction(work.subList(work.size()/2,work.size())));
}
else
processWork();
}
private void processWork()
{
//do something
}
}
}
You could also extend RecursiveTask<T> if the "work" returned a value that was relevant to you.

Metrics from multiple threads

So this seems like a pretty common use case, and maybe I'm over thinking it, but I'm having an issue with keeping centralized metrics from multiple threads. Say I have multiple worker threads all processing records and I every 1000 records I want to spit out some metric. Now I could have each thread log individual metrics, but then to get throughput numbers, but I'd have to add them up manually (and of course time boundaries won't be exact). Here's a simple examples:
public class Worker implements Runnable {
private static int count = 0;
private static long processingTime = 0;
public void run() {
while (true) {
...get record
count++;
long start = System.currentTimeMillis();
...do work
long end = System.currentTimeMillis();
processingTime += (end-start);
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
}
}
}
Hope that makes some sense. Also I know the two static variables will probably be AtomicInteger and AtomicLong . . . but maybe not. Interested in what kinds of ideas people have. I had thought about using Atomic variables and using a ReeantrantReadWriteLock - but I really don't want the metrics to stop the processing flow (i.e. the metrics should have very very minimal impact on the processing). Thanks.

Offloading the actual processing to another thread can be a good idea. The idea is to encapsulate your data and hand it off to a processing thread quickly so you minimize impact on the threads that are doing meaningful work.
There is a small handoff contention, but that cost is usually a lot smaller than any other type of synchronization that it should be a good candidate in many situations. I think M. Jessup's solution is pretty close to mine, but hopefully the following code illustrates the point clearly.
public class Worker implements Runnable {
private static final Metrics metrics = new Metrics();
public void run() {
while (true) {
...get record
long start = System.currentTimeMillis();
...do work
long end = System.currentTimeMillis();
// process the metric asynchronously
metrics.addMetric(end - start);
}
}
private static final class Metrics {
// a single "background" thread that actually handles
// processing
private final ExecutorService metricThread =
Executors.newSingleThreadExecutor();
// data (no synchronization needed)
private int count = 0;
private long processingTime = 0;
public void addMetric(final long time) {
metricThread.execute(new Runnable() {
public void run() {
count++;
processingTime += time;
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
}
});
}
}
}

I would suggest if you don't want the logging to interfere with the processing, you should have a separate log worker thread and have your processing threads simply provide some type of value object that can be handed off. In the example I choose a LinkedBlockingQueue since it has the ability to block for an insignificant amount of time using offer() and you can defer the blocking to another thread that pulls the values from a queue. You might need to have increased logic in the MetricProcessor to order data, etc depending on your requirements, but even if it is a long running operation it wont keep the VM thread scheduler from restarting the real processing threads in the mean time.
public class Worker implements Runnable {
public void run() {
while (true) {
... do some stuff
if (count % 1000 == 0) {
... log some metrics
if(MetricProcessor.getInstance().addMetrics(
new Metrics(processingTime, count, ...)) {
processingTime = 0;
count = 0;
} else {
//the call would have blocked for a more significant
//amount of time, here the results
//could be abandoned or just held and attempted again
//as a larger data set later
}
}
}
}
}
public class WorkerMetrics {
...some interesting data
public WorkerMetrics(... data){
...
}
...getter setters etc
}
public class MetricProcessor implements Runnable {
LinkedBlockingQueue metrics = new LinkedBlockingQueue();
public boolean addMetrics(WorkerMetrics m) {
return metrics.offer(m); //This may block, but not for a significant amount of time.
}
public void run() {
while(true) {
WorkMetrics m = metrics.take(); //wait here for something to come in
//the above call does all the significant blocking without
//interrupting the real processing
...do some actual logging, aggregation, etc of the metrics
}
}
}

If you depend on the state of count and the state of processingTime to be in synch then you would have to be using a Lock. For example if when ++count % 1000 == 0 is true, you want to evaluate the metrics of processingTime at THAT time.
For that case, it would make sense to use a ReentrantLock. I wouldn't use a RRWL because there isn't really an instance where a pure read is occuring. It is always a read/write set. But you would need to Lock around all of
count++
processingTime += (end-start);
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
Whether or not count++ is going to be at that location, you will need to lock around that also.
Finally if you are using a Lock, you do not need an AtomicLong and AtomicInteger. It just adds to the overhead and isn't more thread-safe.

Counting the number of games played in 1 second using Java Threads

I want to know how many games my computer can play in 1000 ms. I did the tests before without using Threads (it plays 13k). Now that I think I'm using threads, I still get the same. Since I don't have much experience with Java threads, I assume I'm doing something wrong but I just can't get it.
Thanks in advance
public class SpeedTest<T extends BoardGame> implements Runnable
{
public static int gamesPlayed = 0;
private ElapsedTimer timer;
private double maxTime;
private BoardAgent<T> agent;
private BoardGame<T> game;
public SpeedTest(BoardGame<T> game, ElapsedTimer timer, double maxTime, Random rng)
{
this.game = game;
this.timer = timer;
this.maxTime = maxTime;
this.agent = new RandomAgent<T>(rng);
}
#Override
public void run()
{
while (true)
{
BoardGame<T> newBoard = game.copy();
while (!newBoard.isGameOver())
newBoard.makeMove(agent.move(newBoard));
gamesPlayed++;
if (timer.elapsedMilliseconds() > maxTime) {
break;
}
}
}
public static void main(String[] args)
{
Random rng = new Random();
BoardGame<Connect4> game = new Connect4(6, 7);
double maxTime = 1000;
ElapsedTimer timer = new ElapsedTimer();
SpeedTest<Connect4> speedTest1 = new SpeedTest<Connect4>(game, timer, maxTime, rng);
SpeedTest<Connect4> speedTest2 = new SpeedTest<Connect4>(game, timer, maxTime, rng);
Thread t1 = new Thread(speedTest1);
Thread t2 = new Thread(speedTest2);
t1.start();
t2.start();
try {
Thread.sleep((long) maxTime);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Games: " + SpeedTest.gamesPlayed);
}
}

I suspect that the reason that you are not seeing any speedup is that your application is only using 1 physical processor. If it is only using one processor, then the two threads won't be running in parallel. Instead, the processor will be "time-slicing" between the two threads.
What can you do about this?
Run on a dual-core etc processor. Or if you have a single processor machine with HT support, enable HT.
Run the test over a longer time; e.g. a number of minutes.
The reason I suggest the latter is that this could be a JVM warmup effect. When a JVM starts a new application, it needs to do a lot of class loading and JIT compilation behind the scenes. These tasks will be largely (if not totally) single-threaded. Running the tests over a longer period of time reduces the contribution of the "warm up" overheads to the average time per "game".
There is a fix that you ought to make to make the program thread-safe. Change
public static int gamesPlayed = 0;
to
private static final AtomicInteger gamesPlayed = new AtomicInteger();
and then use getAndIncrement() to increment the counter and intValue() to fetch its value. (This is simpler than having each thread maintain its own counter and summing them at the end.)
However, I strongly suspect that this change (or #Erik's alternative) will make little difference to the results you are seeing. I'm now sure it is either:
JVM warmup issue as described above,
a consequence of high object creation rates and/or heap starvation, or
some hidden synchronization issue between the instances of your game.

Don't use a static int, use a normal member int.
Instead of the sleep, call .join on both threads.
Then finally add the member ints.

Non-blocking I/O versus using threads (How bad is context switching?)

We use sockets a lot in a program that I work on and we handle connections from up to about 100 machines simultaneously at times. We have a combination of non-blocking I/O in use with a state table to manage it and traditional Java sockets which use threads.
We have quite a few problems with non-blocking sockets and I personally like using threads to handle sockets much better. So my question is:
How much saving is made by using non-blocking sockets on a single thread? How bad is the context switching involved in using threads and how many concurrent connections can you scale to using the threaded model in Java?

I/O and non-blocking I/O selection depends from your server activity profile. E.g. if you use long-living connections and thousands of clients I/O may become too expensive because of system resources exhaustion. However, direct I/O that doesn't crowd out CPU cache is faster than non-blocking I/O. There is a good article about that - Writing Java Multithreaded Servers - whats old is new.
About context switch cost - it's rather chip operation. Consider the simple test below:
package com;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.Set;
import java.util.concurrent.ConcurrentSkipListSet;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
public class AAA {
private static final long DURATION = TimeUnit.NANOSECONDS.convert(30, TimeUnit.SECONDS);
private static final int THREADS_NUMBER = 2;
private static final ThreadLocal<AtomicLong> COUNTER = new ThreadLocal<AtomicLong>() {
#Override
protected AtomicLong initialValue() {
return new AtomicLong();
}
};
private static final ThreadLocal<AtomicLong> DUMMY_DATA = new ThreadLocal<AtomicLong>() {
#Override
protected AtomicLong initialValue() {
return new AtomicLong();
}
};
private static final AtomicLong DUMMY_COUNTER = new AtomicLong();
private static final AtomicLong END_TIME = new AtomicLong(System.nanoTime() + DURATION);
private static final List<ThreadLocal<CharSequence>> DUMMY_SOURCE = new ArrayList<ThreadLocal<CharSequence>>();
static {
for (int i = 0; i < 40; ++i) {
DUMMY_SOURCE.add(new ThreadLocal<CharSequence>());
}
}
private static final Set<Long> COUNTERS = new ConcurrentSkipListSet<Long>();
public static void main(String[] args) throws Exception {
final CountDownLatch startLatch = new CountDownLatch(THREADS_NUMBER);
final CountDownLatch endLatch = new CountDownLatch(THREADS_NUMBER);
for (int i = 0; i < THREADS_NUMBER; i++) {
new Thread() {
#Override
public void run() {
initDummyData();
startLatch.countDown();
try {
startLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
while (System.nanoTime() < END_TIME.get()) {
doJob();
}
COUNTERS.add(COUNTER.get().get());
DUMMY_COUNTER.addAndGet(DUMMY_DATA.get().get());
endLatch.countDown();
}
}.start();
}
startLatch.await();
END_TIME.set(System.nanoTime() + DURATION);
endLatch.await();
printStatistics();
}
private static void initDummyData() {
for (ThreadLocal<CharSequence> threadLocal : DUMMY_SOURCE) {
threadLocal.set(getRandomString());
}
}
private static CharSequence getRandomString() {
StringBuilder result = new StringBuilder();
Random random = new Random();
for (int i = 0; i < 127; ++i) {
result.append((char)random.nextInt(0xFF));
}
return result;
}
private static void doJob() {
Random random = new Random();
for (ThreadLocal<CharSequence> threadLocal : DUMMY_SOURCE) {
for (int i = 0; i < threadLocal.get().length(); ++i) {
DUMMY_DATA.get().addAndGet(threadLocal.get().charAt(i) << random.nextInt(31));
}
}
COUNTER.get().incrementAndGet();
}
private static void printStatistics() {
long total = 0L;
for (Long counter : COUNTERS) {
total += counter;
}
System.out.printf("Total iterations number: %d, dummy data: %d, distribution:%n", total, DUMMY_COUNTER.get());
for (Long counter : COUNTERS) {
System.out.printf("%f%%%n", counter * 100d / total);
}
}
}
I made four tests for two and ten thread scenarios and it shows performance loss is about 2.5% (78626 iterations for two threads and 76754 for ten threads), System resources are used by the threads approximately equally.
Also 'java.util.concurrent' authors suppose context switch time to be about 2000-4000 CPU cycles:
public class Exchanger<V> {
...
private static final int NCPU = Runtime.getRuntime().availableProcessors();
....
/**
* The number of times to spin (doing nothing except polling a
* memory location) before blocking or giving up while waiting to
* be fulfilled. Should be zero on uniprocessors. On
* multiprocessors, this value should be large enough so that two
* threads exchanging items as fast as possible block only when
* one of them is stalled (due to GC or preemption), but not much
* longer, to avoid wasting CPU resources. Seen differently, this
* value is a little over half the number of cycles of an average
* context switch time on most systems. The value here is
* approximately the average of those across a range of tested
* systems.
*/
private static final int SPINS = (NCPU == 1) ? 0 : 2000;

For your questions the best method might be to build a test program, get some hard measurement data and make the best decision based on the data. I usually do this when trying to make such decisions, and it helps to have hard numbers to bring with you to back up your argument.
Before starting though, how many threads are you talking about? And with what type of hardware are you running your software?

For 100 connections are are unlikely to have a problem with blocking IO and using two threads per connection (one for read and write) That's the simplest model IMHO.
However you may find using JMS is a better way to manage your connections. If you use something like ActiveMQ you can consolidate all your connections.