some elements processed more than once, some not at all - java

I have a fairly straightforward task: I have a list of strings each of which is processed and a score is produced. The string and its score then get added to a map:
public class My1Thread
{
final private static List<String> ids = Arrays.asList("id1","id2","id3","id4","id5");
private static HashMap<String,Double> result = null;
private Double computeResult(String id)
{
Double res = 0.0;
// do stuff to compute result
return res;
}
public static void main(String[] args)
{
result = new HashMap<String,Double>();
for (String id: ids)
{
result.put(id,computeResult(id));
}
}
}
Since scores of any two strings are independent of each other, this seems to be a perfect case to use multithreading. However, I am getting unexpected results, which is probably a typical result for a multithreading newbie.
Here's a m/t version of the above:
public class MyMultiThread
{
final private static int nWorkers = 3; // number of threads
final private static List<String> ids = Arrays.asList("id1","id2","id3","id4","id5");
private static int curIndex = 0; // indexing pointing to position in ids currently being processed
private static HashMap<String,Double> result = null;
public static class Worker implements Runnable {
private int id;
public Worker(int id) {
this.id = id;
}
synchronized void setCounter(final int counter)
{
curIndex = counter;
}
synchronized int getCounter()
{
return curIndex;
}
synchronized void addToResult(final String id, final Double score)
{
result.put(id,score);
}
#Override
public void run()
{
try {
while (true)
{
int index = getCounter();
if (index >= ids.size())
{
// exit thread
return;
}
String id = ids.get(index);
setCounter(index+1);
System.out.print(String.format("Thread %d: processing %s from pos %d\n", id, id, curIndex-1));
Double score = ... // compute score here
addToResult(id,score);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
public static void main(String[] args)
{
result = new HashMap<String,ArrayList<Pair<Document,Double>>>();
for (int i = 0; i < nWorkers; i++) {
Thread worker = new Thread(new MyMultiThread.Worker(i));
worker.start();
}
}
}
According to the output produced by System.out.print, this code appears to be processing some elements of ids more than once while not processing others at all. What am I doing wrong here?

Your while(true) loop inside the thread starts at the index specified in the constructor, and then increment it by one, and then the loop starts again. So thread 0 does index 0, then index 1, etc.. Thread 1 does index 1, then index 2, etc... So index 2 will be done 3 times.
I would use a synchronized linked list for ids, and have each thread take and remove the first element of the list, until the list is empty. Use LinkedList.removeFirst().
Also the result hash map also needs to be synchronized, since multiple threads may write to it at the same time.

The problem is that the map is being modified concurrently in multiple threads, so some updates are getting lost.
You declared the methods that modify the map as synchronized, but note that they are synchronized on multiple worker objects: not on a single object, which would provide the locking you are after.
I'd recommend using ConcurrentHashMap and getting rid of all the synchronized declarations.

Some of your synchronization is too narrow - for example, this bit here:
int index = getCounter();
if (index >= ids.size())
{
// exit thread
return;
}
String id = ids.get(index);
setCounter(index+1);
What happens if thread A reads the counter, thread B reads the counter, then thread A updates the counter?
A: int index = getCounter(); // returns 3
B: int index = getCounter(); // returns 3
...
A: setCounter(index + 1); // sets it to 4
B: setCounter(index + 1); // Uh-oh, sets it to 4 as well, we lost an update!
In this case, when you read a variable, then write to it based on the value you read, both the read and the write need to be within the same synchronization block. Declaring getCounter and setCounter as synchronized is not enough.

Simply use Java 8 Stream API :
Map<String, Double> map = ids.parallelStream().collect(Collectors.toConcurrentMap(id -> id, id -> computeScore(id)));
...
Double computeScore(String id) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return ThreadLocalRandom.current().nextDouble(100);
}
Here's a nice tutorial.

Related

Adding numbers using multiple threads in java

I am having trouble figuring out what my code is doing as this is my first time coding using multiple threads. To start off, in attempt to learn this type of programming I decided to write a miniature program that uses 8 threads to sum a number. However, no matter what I do it seems as if my program never stops when count = 10, it continues onward. I am using 8 threads as I planned on expanding my program to do large calculations. However, these threads are not correlating at all. They are going way past 10. I have used a synchronized method. I have tried a lock. I have tried implementing both at the same time. No matter what, it appears as if the threads still calculate past 10. See below for my current code.
public class calculator implements Runnable {
static int counter = 0;
static int sum = 0;
private synchronized static int getAndIncrement()
{
// System.out.println("counter is : " + counter);
int temp = counter;
counter = counter + 1;
System.out.println("counter is now : " + counter);
return temp;
}
private synchronized void addToSum(int value)
{
// System.out.println("sum : " + sum + " value: " + value);
sum += value;
}
#Override
public void run()
{
// TODO Auto-generated method stub
while(counter < 10)
{
int tempVal = getAndIncrement();
System.out.println("temp val : " + tempVal);
addToSum(tempVal);
// System.out.println("sum is now : " + sum);
}
}
}
This is my main method:
public static void main(String[] args)
{
calculator[] calc = new calculator[8];
Thread[] thread = new Thread[8];
final long startTime = System.currentTimeMillis();
for(int i = 0; i < 8; i++)
{
calc[i] = new calculator();
thread[i] = new Thread(calc[i]);
thread[i].start();
}
while(thread[0].isAlive() ||thread[1].isAlive() || thread[2].isAlive() || thread[3].isAlive() || thread[4].isAlive() || thread[5].isAlive() || thread[6].isAlive() || thread[7].isAlive())
{}
final long endTime = System.currentTimeMillis();
System.out.println(calculator.sum);
System.out.println("Execution time : " + (startTime - endTime));
}
I appreciate the help!
The synchronized keyword takes the object
lock. This means that two methods that are synchronized cannot execute on the same object. They will, however, execute concurrently on invocation on 2 different objects.
In your example, your code had 8 objects of calculator. The synchronized methods do not help you. Each thread uses it's separate object. You can completely remove the synchronized keyword, and your code will be semantically equivalent.
To avoid this, use the atomic version of the objects (AtomicInt) or lock on the objects themselves: synchronized(counter){...} but for this to work you will have to change the type to Integer.
I've just tested your sample and found the addToSum method doesn't work as expected here with heavy multi-thread, even if synchronized keyword is present.
Here, as sum variable is static, the method can be made static too.
After adding the static keyword, the behavior is as expected:
private static synchronized void addToSum(int value)
{
sum += value;
}
Here a simple test (addToSum replaced by incSum for simplicity) :
class IncrementorThread implements Runnable {
private static int sum = 0;
private static synchronized void incSum()
{
sum ++;
}
public void run() {
incSum();
Thread.yield();
}
}
void testIncrementorThread1() {
ExecutorService executorService = Executors.newCachedThreadPool();
//ExecutorService executorService = Executors.newSingleThreadExecutor() // result always ok without needing concurrency precaution
for(int i = 0; i < 5000; i++)
executorService.execute(new IncrementorThread());
executorService.shutdown();
executorService.awaitTermination(4000, TimeUnit.MILLISECONDS);
System.out.println("res = "+IncrementorThread.sum); // must be 5000
}
Result must be 5000, which is not the case if we remove the static keyword from the method incSum()

Erastotenes Sieve in paraller programm Java

I would like to make a programm which count prime numbers using Erastotenes Sieve. In this issue I want to use semaphore to communicate between thread to make calculations on table with numbers.
So far I have written code like that.
public static void main( String[] args ) throws InterruptedException {
System.out.println("Podaj gorny zakres\n");
Scanner scanner = new Scanner(System.in);
Erastotenes erastotenes = new Erastotenes(Integer.parseInt(scanner.nextLine()));
erastotenes.initializeTable();
long start = System.nanoTime();
List<SingleProcess.MyThread> list = new ArrayList<>();
List<Integer> numbers = Dollar.$(2,erastotenes.getMaximumNumber()+1).toList();
for(int i=0;i<2;i++)
{
list.add(new SingleProcess.MyThread(erastotenes,numbers.subList((numbers.size()/2)*i,(numbers.size()/2)*i+numbers.size()/2)));
list.get(list.size()-1).start();
list.get(list.size()-1).join();
}
System.out.println(System.nanoTime() - start);
//System.out.println("Liczba elementów: "+erastotenes.countPrimeElements());
}
Erastotenes class.
public class Erastotenes {
private int upperRange;
private int maximumNumber;
private int table[];
public Erastotenes(int upperRange) {
this.upperRange = upperRange;
this.maximumNumber = (int)(Math.floor(Math.sqrt(upperRange)));
this.table = new int[upperRange+1];
}
public int getMaximumNumber() {
return maximumNumber;
}
public int getUpperRange() {
return upperRange;
}
public void initializeTable()
{
for(int i=1;i<=upperRange;i++) {
table[i] = i;
}
}
public void makeSelectionOfGivenNumber(int number)
{
if (table[number] != 0) {
int multiple;
multiple = number+number;
while (multiple<=upperRange) {
table[multiple] = 0;
multiple += number;
}
}
}
public List<Integer> getList()
{
List<Integer> list = Ints.asList(table);
return list.stream().filter(item->item.intValue()!=0 && item.intValue()!=1).collect(Collectors.toList());
}
}
The class describing single Thread to make calculations with static Semaphore looks like this.
public class SingleProcess {
static Semaphore semaphore = new Semaphore(1);
static class MyThread extends Thread {
Erastotenes erastotenes;
List<Integer> numbers;
MyThread(Erastotenes erastotenes,List<Integer> numbers) {
this.erastotenes = erastotenes;
this.numbers=numbers;
}
public void run() {
for(int number:numbers) {
try {
semaphore.acquire();
//1System.out.println(number + " : got the permit!");
erastotenes.makeSelectionOfGivenNumber(number);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
semaphore.release();
}
}
}
}
}
I thought that splitting on half table with numbers from 2 to maximum numbers as square root like in erastotrenes algorithm for these two Threads will boost calculations, but with upperRange to 100000000 the difference between paraller and sequence is not so big. How can I in another realize this problem of paraller programming Erastotenes Sieve?
I think your main problem is this:
for(int i=0;i<2;i++)
{
list.add(new SingleProcess.MyThread(erastotenes,numbers.subList((numbers.size()/2)*i,(numbers.size()/2)*i+numbers.size()/2)));
list.get(list.size()-1).start();
list.get(list.size()-1).join();
}
You start a thread and then immediately wait for it to finish; that kills the parallelism entirely. You can start and wait in the end:
for(int i=0;i<2;i++)
{
list.add(new SingleProcess.MyThread(erastotenes,numbers.subList((numbers.size()/2)*i,(numbers.size()/2)*i+numbers.size()/2)));
list.get(list.size()-1).start();
}
for (Thread t : list) {
t.join();
}
But, there's also a problem with your semaphore tbh. Each thread blocks all other threads from doing anything as long as it's working on a number; that means that again, all parallelism is gone.
You can do away with the semaphore altogether IMO; there's not really a lot of danger in setting the same index to 0 several times, which is all that happens in this "critical section" - but it's not critical at all because no one ever reads the array value in question before all threads are finished.

Synchronized across instances of object

I have an object Rotor which has a goalSpeed and a currentSpeed. Each one tries to change its currentSpeed to match the goalSpeed set. I have 4 of these rotors running ing 4 separate threads. Each one gets assigned a new goalSpeed periodically by a controller.
When I attempt in each Rotor to change its currentSpeed, I cannot ever exceed the sum of all rotor's currentSpeed to exceed X value. sum(currentSpeed(Rotor1) + ... + currentSpeed(Rotor2)) !> X.
Here is my issue: when I check wether I can increase the current speed of a Rotor, I make an if statement on the sum of speeds condition. However, it is possible that right after this check, since each rotor is a separate thread that another one changes its value. Therefore my check in the other thread is not valid anymore. How can I make sure that while I'm in the setNewSpeed() method of one rotor, no other rotor will change its current speed?
class Rotor implements Runnable {
private int id;
private int goalSpeed;
private int currentSpeed;
private Controller controller;
private int Y;
private int failedAttempts;
private int successAttempts;
private int maxSpeed;
public int getSuccessAttempts() {
return successAttempts;
}
public void setSuccessAttempts(int successAttempts) {
this.successAttempts = successAttempts;
}
public int getMaxSpeed() {
return maxSpeed;
}
public void setMaxSpeed(int maxSpeed) {
this.maxSpeed = maxSpeed;
}
public int getFailedAttempts() {
return failedAttempts;
}
public Rotor(Controller c, int Y, int id){
this.controller = c;
this.Y = Y;
this.id = id;
this.currentSpeed = 0;
this.failedAttempts = 0;
this.goalSpeed = 0;
this.maxSpeed = 0;
this.successAttempts = 0;
}
synchronized public void setGoalSpeed(int s){
this.goalSpeed = s;
}
public int getCurrentSpeed(){
return currentSpeed;
}
synchronized private void setNewSpeed(){
int currentDrain = 0;
for(Rotor r : controller.getRotors()){
currentDrain = currentDrain + r.getCurrentSpeed();
}
if((currentDrain + (goalSpeed - currentSpeed)) > 20){
//we cannot increase by total amount because drain too high
System.out.println("failed");
this.failedAttempts++;
currentSpeed = currentSpeed + (20 - currentDrain);
System.out.println("currentSpeed:" + currentSpeed);
} else {
System.out.println("success");
successAttempts++;
currentSpeed = goalSpeed;
}
// System.out.println("goalSpeed:" + goalSpeed);
// System.out.println("currentDrain:" + currentDrain);
}
public void run() {
try {
while(true){
setNewSpeed();
if(currentSpeed > maxSpeed){
maxSpeed = currentSpeed;
}
Thread.sleep(Y);
}
} catch (InterruptedException e) {
System.out.println("Rotor " + id + ": checks=" + (int)(successAttempts + failedAttempts) + ", success rate=" + successAttempts + ", failedAttempts=" + failedAttempts + ", max=" + maxSpeed);
}
}
}
Synchronize on a lock that's shared between all the rotors. Now each of them is synchronizing on their own lock (i.e. this), so even though the method is synchronized, it can be called on different objects at the same time.
The most simple way to synchronize them all is to use a static synchronized method.
That said, using an explicit lock object shared among the instances is probably a better approach.
1) You should not write
synchronized private void setNewSpeed() and synchronized public void setGoalSpeed(int s)
but private synchronized void setNewSpeed() and public synchronized void setGoalSpeed(int s) if you want to respect conventions and standard.
2) You declare two synchronized methods in your Rotor Runnable class but it makes no sense because in the synchronized methods you don't manipulate data shared between the threads.
3) You have multiple ways to address your problem.
A flexible solution consists of using a artificial object shared between the threads and performing the lock on this object when you call the setNewSpeed() method. It allows each tread to wait for the lock to be removed before entering in setNewSpeed().
Here is the idea to implement the solution :
Before instantiating the Rotor, create the shared object in this way :
Object lockObject = new Object();
change public Rotor(Controller c, int Y, int id) to public Rotor(Controller c, int Y, int id, Object lockObject)
invoke the constructor of Rotor by adding the same lockObject instance for all Rotors which you want synchronize between them the speed change.
Store the lockObject as an instance field of the Rotor in the constructor body.
In Rotor use the lockObject to make the synchronization in this way :
sample code :
private void setNewSpeed(){
synchronized(lockObject){
... your actual processing
}
}

java multi threads access Hashtable

I tried to use multi threads to access the Hashtable, since Hashtable is thread safe on get. But I cannot get it work.
I thought the sum of local counter should be equal to the size of the Hashtable or the global_counter. But it is not.
Serveral threads get java.util.NoSuchElementException: Hashtable Enumerator error. I think the error is due to the enumeration of Hashtable. Is that so?
TestMain:
public class TestMain {
// MAIN
public static void main(String argv[]) throws InterruptedException
{
Hashtable<Integer, Integer> id2 = new Hashtable<Integer, Integer>();
for (int i = 0; i < 100000; ++i)
id2.put(i, i+1);
int num_threads = Runtime.getRuntime().availableProcessors() - 1;
ExecutorService ExeSvc = Executors.newFixedThreadPool(num_threads);
for (int i = 0; i < num_threads; ++i)
{
ExeSvc.execute(new CalcLink(id2, i));
}
ExeSvc.shutdown();
ExeSvc.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
}
}
CalcLink:
public class CalcLink implements Runnable {
private Hashtable<Integer, Integer> linktable;
private static Enumeration keys;
private static int global_counter;
private int thread_id;
private int total_size;
public CalcLink(Hashtable<Integer, Integer> lt, int id)
{
linktable = lt;
keys = lt.keys();
thread_id = id;
total_size = lt.size();
global_counter = 0;
}
private synchronized void increment()
{
++global_counter;
}
#Override
public void run()
{
int counter = 0;
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
System.out.println("local counter = " + Integer.toString(counter));
if (thread_id == 1)
System.out.println("global counter = " + Integer.toString(global_counter));
}
}
while (keys.hasMoreElements()) // here you check whether there's an element
{
++counter; // other stuff...
increment(); // other stuff...
Integer key = (Integer)keys.nextElement(); // only here you step
During you are in the other stuff in "this thread" you can enter the other stuff in another thread, thus IMHO you might see higher number in the global counter than what you expect.
This is also the reason you see NoSuchElementException in some of the threads, that entered to the "other stuff" together, but are trying to catch the last element. The later threads won't have the element there when they get to nextElement();
The problem is that this block isn't synchronized :
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
keys.hasMoreElements() can be evaluated to true in multiple threads when there is still only one element in the Enumeration. In those threads : the first one reaching keys.nextElement() will be fine, but all the others will raise a NoSuchElementException
Try this :
#Override
public void run()
{
int counter = 0;
synchronized (keys){
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
}
System.out.println("local counter = " + Integer.toString(counter));
if (thread_id == 1)
System.out.println("global counter = " + Integer.toString(global_counter));
}
An naive solution: I just let each thread to process Length / num_threads of records. Only the last thread will process length/num_threads + length%num_threads records.

Circular Buffer with Threads Consumer and Producer: it get stucks some executions

I'm developing a circular buffer with two Threads: Consumer and Producer.
I'm using active waiting with Thread.yield.
I know that it is possible to do that with semaphores, but I wanted the buffer without semaphores.
Both have a shared variable: bufferCircular.
While the buffer is not full of useful information, producer write data in the position pof array, and while there are some useful information consumer read data in the position c of array. The variable nElem from BufferCircular is the number of value datas that haven't been read yet.
The program works quite good 9/10 times that runs. Then, sometimes, it get stucks in a infinite loop before show the last element on screen (number 500 of loop for), or just dont' show any element.
I think is probably a liveLock, but I can't find the mistake.
Shared Variable:
public class BufferCircular {
volatile int[] array;
volatile int p;
volatile int c;
volatile int nElem;
public BufferCircular(int[] array) {
this.array = array;
this.p = 0;
this.c = 0;
this.nElem = 0;
}
public void writeData (int data) {
this.array[p] = data;
this.p = (p + 1) % array.length;
this.nElem++;
}
public int readData() {
int data = array[c];
this.c = (c + 1) % array.length;
this.nElem--;
return data;
}
}
Producer Thread:
public class Producer extends Thread {
BufferCircular buffer;
int bufferTam;
int contData;
public Productor(BufferCircular buff) {
this.buffer = buff;
this.bufferTam = buffer.array.length;
this.contData = 0;
}
public void produceData() {
this.contData++;
this.buffer.writeData(contData);
}
public void run() {
for (int i = 0; i < 500; i++) {
while (this.buffer.nElem == this.bufferTam) {
Thread.yield();
}
this.produceData();
}
}
}
Consumer Thread:
public class Consumer extends Thread {
BufferCircular buffer;
int cont;
public Consumer(BufferCircular buff) {
this.buffer = buff;
this.cont = 0;
}
public void consumeData() {
int data = buffer.readData();
cont++;
System.out.println("data " + cont + ": " + data);
}
public void run() {
for (int i = 0; i < 500; i++) {
while (this.buffer.nElem == 0) {
Thread.yield();
}
this.consumeData();
}
}
}
Main:
public class Main {
public static void main(String[] args) {
Random ran = new Random();
int tamArray = ran.nextInt(21) + 1;
int[] array = new int[tamArray];
BufferCircular buffer = new BufferCircular(array);
Producer producer = new Producer (buffer);
Consumer consumer = new Consumer (buffer);
producer.start();
consumer.start();
try {
producer.join();
consumer.join();
} catch (InterruptedException e) {
System.err.println("Error with Threads");
e.printStackTrace();
}
}
}
Any help will be welcome.
Your problem here is that your BufferCircular methods are sensitive to race conditions. Take for example writeData(). It executes in 3 steps, some of which are also not atomic:
this.array[p] = data; // 1
this.p = (p + 1) % array.length; // 2 not atomic
this.nElem++; // 3 not atomic
Suppose that 2 threads entered writeData() at the same time. At step 1, they both have the same p value, and both rewrite array[p] value. Now, array[p] is rewritten twice and data that first thread had to write, is lost, because second thread wrote to the same index after. Then they execute step 2--and result is unpredictable since p can be incremented by 1 or 2 (p = (p + 1) % array.length consists of 3 operations, where threads can interact). Then, step 3. ++ operator is also not atomic: it uses 2 operations behind the scenes. So nElem becomes also incremented by 1 or 2.
So we have fully unpredictable result. Which leads to poor execution of your program.
The simplest solution is to make readData() and writeData() methods serialized. For this, declare them synchronized:
public synchronized void writeData (int data) { //...
public synchronized void readData () { //...
If you have only one producer and one consumer threads, race conditions may occur on operations involving nElem. Solution is to use AtomicInteger instead of int:
final AtomicInteger nElem = new AtomicInteger();
and use its incrementAndGet() and decrementAndGet() methods.

Categories