The below code increments a static variable within a Thread and checks to see if its value is incremented by one. This is what Assert.assertEquals(currentAVal+1, accessCounter); checks.
The test consistently passes for 10'000 runs. But why is no race condition causing the test to fail ?
I would expect two or more threads to increment accessCounter at line accessCounter = accessCounter + 1; before the assert takes place but this does not seem to be occurring ?
public class RunnableTest {
private static int accessCounter = 0;
private class Post implements Runnable {
public void run() {
int currentAVal = accessCounter;
accessCounter = accessCounter + 1;
Assert.assertEquals(currentAVal+1, accessCounter);
System.out.println("Access counter : "+accessCounter);
}
}
#Test
public void runTest(){
Runnable r = new Post();
ScheduledExecutorService executor = Executors.newScheduledThreadPool(4);
for(int executorCount = 0; executorCount < 10000; ++executorCount) {
executor.execute(r);
}
}
}
Update : from Gray's answer I've updated the code and I am now receiving a race condition (test failure) when I remove the println statement :
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import junit.framework.Assert;
public class RunnableTest {
private static int accessCounter = 0;
private static List<String> li = new ArrayList<String>();
private class Post implements Runnable {
public synchronized void run() {
int currentAVal = accessCounter;
accessCounter = accessCounter + 1;
li.add(String.valueOf(currentAVal+1+","+accessCounter));
}
}
#Test
public void runTest(){
Runnable r = new Post();
ScheduledExecutorService executor = Executors.newScheduledThreadPool(4);
for(int executorCount = 0; executorCount < 10000; ++executorCount) {
executor.execute(r);
}
//Wait for threads to finish
// we shut it down once we've submitted all jobs to it
executor.shutdown();
// now we wait for all of those jobs to finish
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
for(String s : li){
Assert.assertEquals(s.split(",")[0], s.split(",")[1]);
}
}
}
Adding synchronized to the run method causes the test to pass
The test consistently passes for 10'000 runs. But why is no race condition causing the test to fail ?
The definition of a race condition is that you might get timing problems -- it is not guaranteed. If you ran this on another architecture you might get wildly different results.
However, I don't think that asserts in other threads are seen by junit. For example, if I change you test the following. I do see times that the value differs but the fail is not seen by the test method -- the test still passes.
if (currentAVal+1 != accessCounter) {
System.out.println("Access counter not equal: "+accessCounter);
Assert.fail();
}
One reason why you may be seeing proper values in accessCounter is that System.out.println(...) is a synchronized method which is (as a byproduct) synchronizing the value of accessCounter.
Also, you are not shutting down your executor nor are you waiting for the executor service to actually complete. You should do something like:
// we shut it down once we've submitted all jobs to it
executor.shutdown();
// now we wait for all of those jobs to finish
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
But this doesn't solve the other thread issue. To actually see the results of the threads you could do something like:
List<Future<?>> futures = new ArrayList<Future<?>>();
for (int executorCount = 0; executorCount < 10000; ++executorCount) {
futures.add(executor.submit(r));
}
executor.shutdown();
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
for (Future<?> future : futures) {
// this will throw an exception if an assert happened
future.get();
}
Firstly as other answers have pointed out that Race is not guaranteed to occur. Remove the sysout statement as that can cause the code to synchronize.
The answer is in the question itself. It's a race condition :
You can't guarantee that it will ever occur no matter HOW many times threads you try to throw at it or times you try to run it. That's why it's a race condition. It's non-deterministic
Assuming uniform probability distribution for this isn't even remotely correct unless you can show why. You aren't flipping a coin here. The code could run for months before the race is exposed, I've seen this very thing happen many times. That's why race conditions are hard to solve and important to prevent.
Secondly you aren't seeding any amount of random noise into the scenario. If you say had each thread's run function sleep a random amount of time first so that they actually were likely to coincide with one another this would be more intersting... but you're threads a so short they are likely finished and never even running in parallel compared to the time it takes to spawn dispatch the jobs.
Related
I'm new to using Java and I'm trying to learn threading, so it'd be helpful to get some pointers on this. I don't understand much about using threads, so a for-dummies type of explanation would also help a lot.
I'm working on a project where I have a function foo, and a List of strings which I would like to pass through foo. If the processing time for foo(S) goes over T milliseconds, I will stop processing foo(S), and then I would move on to foo(S+1). I have a large dataset of strings and they can take too long so I tried using threading and termination to speed things up, but I'm not sure how to terminate if processing takes too long. This is more or less what foo does:
public static foo(String s){
// use a while loop to manipulate string
// return some integer
}
After some research, I found out that ExecutorService would be a good approach, and this is the structure of what I have so far:
public static void process(List<String> l, int n) {
ExecutorService executor = Executors.newFixedThreadPool(n);
for (final String s: l) {
executor.submit(new Runnable() {
#Override
public void run() {
System.out.println(foo(s));
//interrupt this process if foo(s) takes >T
}
});
}
executor.shutdown();
}
However, if I run my code without on a small set of strings, it does not terminate nor print anything at all and I am unsure why. Is it stuck inside the for loop in foo? How do I fix that? Can I add a sleep(T) between threads to interrupt a thread that is taking too long? thanks!
It's not really possible to cancel a task exactly after a specific time. Canceling a task at a specific time (canceling in a sense that the task terminates immediately) would be dangerous, because it could leave data in an inconsistent state, without having the chance to "clear things up".
Therefore, we have interrupts to do a similar thing. You interrupt a task (or a thread), which is basically a notification for the task e.g. to stop its execution by itself. In this way, the task is able to terminate in a controlled manner and leave data in a consistent state.
In your example, it would work like so: Every processing of a string in your list would be a task. You have two ways to implement the timeout-mechanism: The task is canceled externally (using interrupts) or it cancels itself. When the task cancels itself, it periodically checks if it is running longer than a timeout, while processing the string. If the timeout is exceeded, the initial value is the result, otherwise the processed string.
In Java it would be something like this:
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
class Main {
static class ProcessTask implements Callable<String> {
final String initialValue;
final long timeout;
ProcessTask(String initialValue, long timeout) {
this.initialValue = initialValue;
this.timeout = timeout;
}
#Override
public String call() throws Exception {
long complexity = (long) (Math.random() * 1_000_000L);
long starttime = System.currentTimeMillis();
for (long i = 0; i < complexity; ++i) {
Math.sqrt(Math.log(i));
if (System.currentTimeMillis() - starttime > timeout) {
return initialValue;
}
}
return "new";
}
}
static void process(List<String> initialList, int parallelism, int timeout) {
ExecutorService executor = Executors.newFixedThreadPool(parallelism);
List<Future<String>> futures = new ArrayList<>();
initialList.forEach(s -> {
futures.add(executor.submit(new ProcessTask(s, timeout)));
});
for (int i = 0; i < initialList.size(); ++i) {
try {
initialList.set(i, futures.get(i).get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdown();
}
public static void main(String[] args) {
List<String> myList = new ArrayList<>();
for (int i = 0; i < 1_000; ++i) {
myList.add("old");
}
process(myList, Runtime.getRuntime().availableProcessors(), 10);
myList.forEach(System.out::println);
}
}
This is a very explicit way of doing it, but you've asked for it. There are definitely ways of doing that less wordy. If you are unsure about any class here, just do a little bit of research. There is nothing crazy I've used.
What you should see is a mix of olds and news in the output, where new just indicates that the processing was finished and old that the timeout was exceeded and the corresponding task was stopped. If you see something else, play around with the timeout parameter of process.
The most important part for you to look at is the call method. There is the mechanism that stops a task after a timeout by periodically checking the runtime. You need place those checks at correct places in your processing algorithm to stop the task.
In one of the interview, a coding question was asked to me and I had to find the problem in that code and suggest proper solution.
Please find below the entire code:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
public class Atomic {
static AtomicInteger count = new AtomicInteger(0);
static int counter = 0;
public static class Runnable extends Thread {
public void run() {
while (count.getAndSet(1) != 0) {
try {
Thread.sleep(3000);
} catch (Exception e) {
}
}
counter = counter + 1;
count.set(0);
}
}
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable runnable = new Runnable();
executor.execute(runnable);
}
executor.shutdown();
}
}
This code is running properly. But question is , there is some problem in this code if number of threads get increased or if I run For loop for almost 10000 times.
I tried to find the problem, but couldn't find one.
There are several things wrong with this code. You've not stated with "there is some problem" means, but here are the things that jump out.
Firstly, the counter variable is not updated safely. Multiple threads don't have guaranteed visibility of the last-written value; nor do you have the guarantee that no other thread has updated its value in between the read and the write.
The simple solution to this: change counter to an AtomicInteger, and use getAndIncrement or incrementAndGet to increment it.
Secondly, public static class Runnable extends Thread { is extremely dubious.
Don't hide the names of commonly-known Java classes (this is hiding java.lang.Runnable)
Don't extend Thread directly, especially when all you need is a java.lang.Runnable to add execute with an ExecutorService.
A more suitable class declaration would be:
public static class MyRunnable implements Runnable {
(or whatever you want to call it)
Or you can just declare an anonymous class:
executor.execute(new Runnable() { /* body */ });
Or you can just declare a lambda:
executor.execute(() -> { /* body */ });
Thirdly, count doesn't really seem to be serving an obvious purpose here. The logic of the runnable seems to be:
If "flag" is false:
Set "flag" to true
Increment a variable
Set "flag" to false
Otherwise:
Wait 3 seconds
Try again
count is playing the role of "flag" here. It's effectively just an AtomicBoolean.
But you don't need a separate count variable at all, if you make the counter an AtomicInteger:
while (true) {
int current = counter.get();
if (counter.compareAndSet(current, current + 1)) {
// Nothing else is trying to update "current" at the same time:
// we updated it. Stop.
break;
}
// Something else is trying to update at the same time.
// Sleep for 3 seconds.
Thread.sleep(3000);
}
I'm learning multithreaded counter and I'm wondering why no matter how many times I ran the code it produces the right result.
public class MainClass {
public static void main(String[] args) {
Counter counter = new Counter();
for (int i = 0; i < 3; i++) {
CounterThread thread = new CounterThread(counter);
thread.start();
}
}
}
public class CounterThread extends Thread {
private Counter counter;
public CounterThread(Counter counter) {
this.counter = counter;
}
public void run() {
for (int i = 0; i < 10; i++) {
this.counter.add();
}
this.counter.print();
}
}
public class Counter {
private int count = 0;
public void add() {
this.count = this.count + 1;
}
public void print() {
System.out.println(this.count);
}
}
And this is the result
10
20
30
Not sure if this is just a fluke or is this expected? I thought the result is going to be
10
10
10
Try increasing the loop count from 10 to 10000 and you'll likely see some differences in the output.
The most logical explanation is that with only 10 additions, a thread is too fast to finish before the next thread gets started and adds on top of the previous result.
I'm learning multithreaded counter and I'm wondering why no matter how many times I ran the code it produces the right result.
<ttdr> Check out #manouti's answer. </ttdr>
Even though you are sharing the same Counter object, which is unsynchronized, there are a couple of things that are causing your 3 threads to run (or look like they are running) serially with data synchronization. I had to work hard on my 8 proc Intel Linux box to get it to show any interleaving.
When threads start and when they finish, there are memory barriers that are crossed. According to the Java Memory Model, the guarantee is that the thread that does the thread.join() will see the results of the thread published to it but I suspect a central memory flush happens when the thread finishes. This means that if the threads run serially (and with such a small loop it's hard for them not to) they will act as if there is no concurrency because they will see each other's changes to the Counter.
Putting a Thread.sleep(100); at the front of the thread run() method causes it to not run serially. It also hopefully causes the threads to cache the Counter and not see the results published by other threads that have already finished. Still needed help though.
Starting the threads in a loop after they all have been instantiated helps concurrency.
Another thing that causes synchronization is:
System.out.println(this.count);
System.out is a Printstream which is a synchronized class. Every time a thread calls println(...) it is publishing its results to central memory. If you instead recorded the value and then displayed it later, it might show better interleaving.
I really wonder if some Java compiler inlining of the Counter class at some point is causing part of the artificial synchronization. For example, I'm really surprised that a Thread.sleep(1000) at the front and end of the thread.run() method doesn't show 10,10,10.
It should be noted that on a non-intel architecture, with different memory and/or thread models, this might be easier to reproduce.
Oh, as commentary and apropos of nothing, typically it is recommended to implement Runnable instead of extending Thread.
So the following is my tweaks to your test program.
public class CounterThread extends Thread {
private Counter counter;
int result;
...
public void run() {
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
Thread.currentThread().interrupt(); // good pattern
return;
}
for (int i = 0; i < 10; i++) {
counter.add();
}
result = counter.count;
// no print here
}
}
Then your main could do something like:
Counter counter = new Counter();
List<CounterThread> counterThreads = new ArrayList<>();
for (int i = 0; i < 3; i++) {
counterThread.add(new CounterThread(counter));
}
// start in a loop after constructing them all which improves the overlap chances
for (CounterThread counterThread : counterThreads) {
counterThread.start();
}
// wait for them to finish
for (CounterThread counterThread : counterThreads) {
counterThread.join();
}
// print the results
for (CounterThread counterThread : counterThreads) {
System.out.println(counterThread.result);
}
Even with this, I never see 10,10,10 output on my box and I often see 10,20,30. Closest I get is 12,12,12.
Shows you how hard it is to properly test a threaded program. Believe me, if this code was in production and you were expecting the "free" synchronization is when it would fail you. ;-)
Does using a synchronized block inside the run method makes any sense? I thought it does, as long as I'm using a relevant lock, not the instance of Runnable containing this run method. Reading the answers to similar questions on stackoverflow seemed to confirm this. I tried to write some simple code to test it and the synchronized block inside the run method doesn't prevent from data corruption:
public class Test {
public Test() {
ExecutorService es = Executors.newCachedThreadPool();
for (int i = 0; i < 1000; i++) {
es.execute(new Runnable() {
#Override
public void run() {
synchronized (lock) {
sum += 1;
}
}
});
}
es.shutdown();
while(!es.isTerminated()) {
}
}
private int sum = 0;
private final Object lock = new Object();
public static void main(String[] args) {
Test t = new Test();
System.out.println(t.sum);
}
}
Why this code generates incorrect results? Is this because the synchronized block or some other mistake? I feel like I'm missing something basic here.
It's possible your executor encounters some sort of unexpected error. If that happens you won't know it because you are not getting any return value to check.
Try switching to submit() instead of execute() and store a list of Future instances the Executor gives you. If the final sum is less than 1000, iterate the futures and get() each one. If an exception is raised you'll see what happened with that particular runnable task.
Apart from your simple example, which looks OK, you should be careful with synchronization in Runnables to prevent them from blocking each other when one Runnable waits for some resource to be released only by another Runnable later in the queue that has not started yet and never will since the current waiting Runnable must finish first.
With enough worker Threads executing the jobs this is less likely to occur, though.
I'm trying to get started with learning threading in Java and here's a simple example that I tried from here
Here's my code :
A simple main class:
package com.vogella.Thread;
import java.util.ArrayList;
import java.util.List;
public class Main {
public static void main(String[] args) {
// We will store the threads so that we can check if they are done
List<Thread> threads = new ArrayList<Thread>();
// We will create 500 threads
for (int i = 0; i < 500; i++) {
Runnable task = new MyRunnable(10000000L + i);
Thread worker = new Thread(task);
// We can set the name of the thread
worker.setName(String.valueOf(i));
// Start the thread, never call method run() direct
worker.start();
// Remember the thread for later usage
threads.add(worker);
}
int running = 0;
do {
running = 0;
for (Thread thread : threads) {
if (thread.isAlive()) {
running++;
}
}
System.out.println("We have " + running + " running threads. ");//-A
} while (running > 0);
}
}
and the MyRunnable class is as follows :
package com.vogella.Thread;
public class MyRunnable implements Runnable {
private final long countUntil;
MyRunnable(long countUntil) {
this.countUntil = countUntil;
}
#Override
public void run() {
long sum = 0;
for (long i = 1; i < countUntil; i++) {
sum += i;
}
System.out.println(sum);
System.out.println("Test123");
}
}
And this is my output
49999995000000
Test123
50000005000000
Test123
50000015000001
Test123...
However I dont understand why the line marked with comment A in Main.java never prints. Any insight on this would be helpful.
Thanks!
It should print. Check you're not missing it in the program output.
Try commenting out the println()'s in your Runnable
There are a few things wrong with that code in terms of best practices. Unless that site mentions them (too lazy to look), I might consider finding a different tutorial. Also, have you checked the entire output? It's probably printing. It is not guaranteed to print as the last thing as I am going to guess you're assuming.
It should be able to print something. A few ideas for testing:
comment the println statements in your runnable - maybe it gets printed but you don't see it because there's too much output on the console
add a System.println(threads.size()) before the do statement to see how many threads have been added (should print 500).
add a Thread.sleep(100) before the do statement. Maybe the first thread gets alive after the do-while block has finished...