We Keep Coding

parellel search with communications between threads

Suppose I have an int array, an element num and 4 threads.
I'm giving each thread 1/4 of the array to search for num.
(The search method is given below)
public static boolean contains(int[] array, int minIdx, int maxIdx, int num) { ...}
At my "top level", I can schedule 4 threads to search 4 quarters of the array, but how do I ensure ALL the threads stop searching as soon as one of them finds the element (assuming there is NO duplicate in the array, hence the element can appear at most once).
P.S: You see, suppose my 4th thread found the element at the first iteration, I want the top-level method to return immediately as opposed to wait for other 3 guys to complete.

You need explicit signaling. You might consider the built-in interruption mechanism, or you may roll your own because it's very simple.
One idea: share an AtomicBoolean among all the threads and let each one periodically check it. When a thread finds the answer, it flips the boolean. The best option to achieve periodic checking is a nested loop:
for (int i = start; i < end && !done.get();) {
for (int batchLimit = Math.min(i + BATCH_SIZE, end); i < batchLimit; i++) {
// your logic
}
}
This is the easiest for the JIT compiler to optimize.
The price of checking the value is very low whenever the value didn't change. It will be in the L3 cache. The case when the value did change is irrelevant because at that point you're done.

Use a flag to signal when you found the answer and share it between threads. AtomicBoolean is a good option.
Add the boolean to your loop end conditions for example
for (int i = minIdxs ; i < maxIdxs && found.get() == false; ++i){...}
Also share a CountDownLatch of size 4 and countDown() when you are returning from each thread.
Have your main thread await() and it'll mean all threads gracefully finish before you move on in your main thread.

You can write a class who will act like a controller. this class will know each thread and every thread knows the controller. (its like an observer pattern)
If one thread finds the answer, the thread can tell it to the controller which can inform the other threads to stop.
class ControllerOfAllTheThreads{
ArrayList<TheClassesWhichDoTheSearch> list = new ArrayList<TheClassesWhichDoTheSearch>();
public void tellThemWeFoundHim(){
for (TheClassesWhichDoTheSearch theThreads : list) {
if(theThreads.isAlive() && !theThreads.isInterrupted())
theThreads.interrupt();
}
}
}

threads did not proceed after barrier.await() using CyclicBarrier

Please help me with my code. I'm making a race using
There are 2 points in the Race starting point and the finish line. All horses started at the gate and proceed to the gate. The race will only starts once all horses reached the gate. The one who finished first wins the race.

The mistake you have is the number you provide here
gate= new CyclicBarrier(numHorses);
What is numHorses ?
It's the number of all horses but here :
list.removeIf(...);
You are removing the horses under 18, pretend we had 5 horses total, numHorses==5 is true, two of these are under 18 so we're left with 3.
You're creating three threads but the barrier is set on 5, the barrier will never get passed because it's waiting for 5 threads when there are only three.
The solution is to make addHorsesToList add horses without the barrier(so you'll need a Horse constructor without barrier too), then you move gate = new CyclicBarrier to after the list.removeIf and change numHorses to list.size().
After that you assign the barrier to the gate field of every horse object you have in list.
Do like this :
int canJoin=0;
while(canJoin<2){
list= new ArrayList<Horse>();
numHorses=checkNumHorses();
addHorsesToList();
printHorses();
canJoin=countJoinAge();
}
list.removeIf(p -> p.Age().equals(18));
int numHealthyHorses=list.size();
gate=new CyclicBarrier(numHealthyHorses);
System.out.println("Horses who can join");
Thread[] threads = new Thread[numHealthyHorses];
for(Horse horse: list){ //create thread for each Horse
horse.setGateBarrier(gate); //setter method, needs to be added to Horse class
threads[numThread]= new Thread(horse);
}
for(int i=0;i<numHealthyHorses;i++){ //start the thread for each horse
threads[i].start();
}}

Java - Semaphore release without acquire

I have threads which are given random number (1 to n) and are instructed to print them in sorted order. I used semaphore such that I acquire the number of permits = random number and release one permit more than what was acquired.
acquired = random number; released = 1+random number
Initial permit count for semaphore is 1. So thread with random number 1 should get permit and then 2 and so on.
This is supported as per the documentation given below
There is no requirement that a thread that releases a permit must have acquired that permit by calling acquire().
The problem is my program gets stuck after 1 for n>2.
My program is given below:
import java.util.concurrent.Semaphore;
public class MultiThreading {
public static void main(String[] args) {
Semaphore sem = new Semaphore(1,false);
for(int i=5;i>=1;i--)
new MyThread(i, sem);
}
}
class MyThread implements Runnable {
int var;Semaphore sem;
public MyThread(int a, Semaphore s) {
var =a;sem=s;
new Thread(this).start();
}
#Override
public void run() {
System.out.println("Acquiring lock -- "+var);
try {
sem.acquire(var);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(var);
System.out.println("Releasing lock -- "+var);
sem.release(var+1);
}
}
Output is :
Acquiring lock -- 4
Acquiring lock -- 5
Acquiring lock -- 3
Acquiring lock -- 2
Acquiring lock -- 1
1
Releasing lock -- 1
While If I modify my code with tryAcquire, it runs perfectly well.
Below is new run implementation
#Override
public void run() {
boolean acquired = false;
while(!acquired) {
acquired = sem.tryAcquire(var);
}
System.out.println(var);
sem.release(var+1);
}
Can someone please explain the semaphore's permit acquire mechanism when mulitple threads are waiting with different permit request??

It's a clever strategy, but you're misunderstanding how Sempahore hands out permits. If you run your code enough times you'll actually see it reach step two:
Acquiring lock -- 5
Acquiring lock -- 1
1
Releasing lock -- 1
Acquiring lock -- 3
Acquiring lock -- 2
2
Acquiring lock -- 4
Releasing lock -- 2
If you keep on re-running it enough times you'd actually see it successfully finish. This happens because of how Semaphore hands out permits. You're assuming Semaphore will try to accommodate an acquire() call as soon as it has enough permits to do so. If we look carefully at the documentation for Semaphore.aquire(int) we'll see that is not the case (emphasis mine):
If insufficient permits are available then the current thread becomes disabled for thread scheduling purposes and lies dormant until ... some other thread invokes one of the release methods for this semaphore, the current thread is next to be assigned permits and the number of available permits satisfies this request.
In other words Semaphore keeps a queue of pending acquire request and, upon each call to .release(), only checks the head of the queue. In particular if you enable fair queuing (set the second constructor argument to true) you'll see even step one doesn't occur, because step 5 is (usually) the first in the queue and even new acquire() calls that could be fulfilled will be queued up behind the other pending calls.
In short this means you cannot rely on .acquire() to return as soon as possible, as your code assumes.
By using .tryAcquire() in a loop instead you avoid making any blocking calls (and therefore put a lot more load on your Semaphore) and as soon as the necessary number of permits becomes available a tryAcquire() call will successfully obtain them. This works but is wasteful.
Picture a wait-list at a restaurant. Using .aquire() is like putting your name on the list and waiting to be called. It may not be perfectly efficient, but they'll get to you in a (reasonably) fair amount of time. Imagine instead if everyone just shouted at the host "Do you have a table for n yet?" as often as they could - that's your tryAquire() loop. It may still work out (as it does in your example) but it's certainly not the right way to go about it.
So what should you do instead? There's a number of possibly useful tools in java.util.concurrent, and which is best somewhat depends on what exactly you're trying to do. Seeing as you're effectively having each thread start the next one I might use a BlockingQueue as the synchronization aid, pushing the next step into the queue each time. Each thread would then poll the queue, and if it's not the activated thread's turn replace the value and wait again.
Here's an example:
public class MultiThreading {
public static void main(String[] args) throws Exception{
// Use fair queuing to prevent an out-of-order task
// from jumping to the head of the line again
// try setting this to false - you'll see far more re-queuing calls
BlockingQueue<Integer> queue = new ArrayBlockingQueue<>(1, true);
for (int i = 5; i >= 1; i--) {
Thread.sleep(100); // not necessary, just helps demonstrate the queuing behavior
new MyThread(i, queue).start();
}
queue.add(1); // work starts now
}
static class MyThread extends Thread {
int var;
BlockingQueue<Integer> queue;
public MyThread(int var, BlockingQueue<Integer> queue) {
this.var = var;
this.queue = queue;
}
#Override
public void run() {
System.out.println("Task " + var + " is now pending...");
try {
while (true) {
int task = queue.take();
if (task != var) {
System.out.println(
"Task " + var + " got task " + task + " instead - re-queuing");
queue.add(task);
} else {
break;
}
}
} catch (InterruptedException e) {
// If a thread is interrupted, re-mark the thread interrupted and terminate
Thread.currentThread().interrupt();
return;
}
System.out.println("Finished task " + var);
System.out.println("Registering task " + (var + 1) + " to run next");
queue.add(var + 1);
}
}
}
This prints the following and terminates successfully:
Task 5 is now pending...
Task 4 is now pending...
Task 3 is now pending...
Task 2 is now pending...
Task 1 is now pending...
Task 5 got task 1 instead - re-queuing
Task 4 got task 1 instead - re-queuing
Task 3 got task 1 instead - re-queuing
Task 2 got task 1 instead - re-queuing
Finished task 1
Registering task 2 to run next
Task 5 got task 2 instead - re-queuing
Task 4 got task 2 instead - re-queuing
Task 3 got task 2 instead - re-queuing
Finished task 2
Registering task 3 to run next
Task 5 got task 3 instead - re-queuing
Task 4 got task 3 instead - re-queuing
Finished task 3
Registering task 4 to run next
Task 5 got task 4 instead - re-queuing
Finished task 4
Registering task 5 to run next
Finished task 5
Registering task 6 to run next

The Javadoc for Semaphore.acquire(int) says:
If insufficient permits are available then the current thread becomes
disabled for thread scheduling purposes and lies dormant until one of
two things happens:
Some other thread invokes one of the release methods for this semaphore,
the current thread is next to be assigned permits and the number of
available permits satisfies this request [or the thread is interrupted].
The thread that is "next to be assigned" is probably thread 4 in your example. It is waiting until there are 4 permits available. However, thread 1, which gets a permit upon calling acquire(), only releases 2 permits, which is not enough to unblock thread 4. Meanwhile, thread 2, which is the only thread for which there are sufficient permits, is not the next to be assigned, so it doesn't get the permits.
Your modified code runs fine because the threads don't block when they try to get a semaphore; they just try again, going to the back of the line. Eventually thread 2 reaches the front of the line and is thus next to be assigned, and so gets its permits.

Java Multithreading priority: Why in this example, sometimes t1 occurs before t2 is completed, even if t2 has higher priority?

Example:
class MyThread extends Thread{
public MyThread(String name) {
super(name);
}
public void run(){
for (int i=0; i<5; i++) {
System.out.println(Thread.currentThread().getName()
+"("+Thread.currentThread().getPriority()+ ")"
+", loop "+i);
}
}
};
public class Demo {
public static void main(String[] args) {
System.out.println(Thread.currentThread().getName()
+"("+Thread.currentThread().getPriority()+ ")");
Thread t1=new MyThread("t1"); // t1
Thread t2=new MyThread("t2"); // t2
t1.setPriority(1); // t1 priority 1
t2.setPriority(10); //t2 priority 10
t1.start(); // start t1
t2.start(); // start t2
}
}
When I execute the program, some times I have the output like below:
//Output1
main(5)
t2(10), loop 0
t2(10), loop 1
t2(10), loop 2
t2(10), loop 3
t2(10), loop 4
t1(1), loop 0
t1(1), loop 1
t1(1), loop 2
t1(1), loop 3
t1(1), loop 4
Sometimes I have output like below:
//Output2
main(5)
t1(1), loop 0
t1(1), loop 1
t1(1), loop 2
t1(1), loop 3
t1(1), loop 4
t2(10), loop 0
t2(10), loop 1
t2(10), loop 2
t2(10), loop 3
t2(10), loop 4
In some other occasions I have output where t1 starts first, and t2 starts before t1 completes all output.
I thought output1 makes more sense as “Threads with higher priority are executed in preference to threads with lower priority.”
How can we understand the reasoning behind this example?

Your lower-prioritized thread is started first, therefore it may in some cases complete even before the higher-prioritized even starts. 5 iterations is not that much.
On my (Windows) machine, if I replace the number of iterations with 100, the higher-prioritized thread is consistently selected first.

As you already mentioned in your post:
“Threads with higher priority are executed in preference to threads with lower priority.”
This does mean, that a thread with higher priority has a higher likeliness of being executed than a low-priority-thread. It doesn't mean that a thread with higher priority will always be executed first/finish first. The actual thread-handling depends upon the OS (java simply uses the thread-libraries provided by the OS it runs on).

My notes about threads priorities:
You can use it for improving performance
Priorities depends on OS (Windows has 7, Linux ignores them)
Don't design your app in way where correct working depends on priorities
If there are some threads with high priorities which are never deactivated, threads with lower priorities can never be run

Many independent threads dependent on single thread

I have a file which has n columns and many rows
Col 1 col2 col3 .......col n
I want to read it once and write multiple (say m) outputs grouping the rows by few key columns. Say 3 outputs have to be produced:
For output 1:
groupingKeys[0]={1,2) //group the records on col 1 and 2
For output 2:
groupingKeys[1]={1,4,5} //group the records on col 1 4 5
For output 3
groupingKeys[2]={2,3} //group on col 2,3
In Main-thread I read the input file line by line. For each read line I want to process the read line in m different threads. So basically I want that the calls
map[0].process(data,groupingKeys[0]);
map[1].process(data,groupingKeys[1]);
map[2].process(data,groupingKeys[2]);
should run in 3 different threads and each of the 3 threads should proceed only after main thread has read the line.
I can create m different threads with run method of i-th thread having
map[i].process(data,groupingKeys[i]);
But these 3 threads should proceed only when main-thread which reads the line so that they see correct value of data[]. How can i achieve this?
Main thread thread-0 thread-1 thread-2
running waiting waiting waiting
waiting running running running
running waiting waiting waiting
At each step a line is read and processed
By processed i mean something similar to sql groupby is done for each of the grouping keys
Below is the sample code referred above.
public void writeMultipleGroupedOutputs(String inputfile,int groupingKeys[][])
{
Mymap<key,value>[] mapArr= new Mymap<key,value>[k]; //k maps to group records in k ways as per k grouping keys
String line;
while((line = br.readLine()) != null) {
String[] data=line.split(regex); **//one line is read in main thread**
for(int i=0;i<m;i++)
map[i].process(data,groupingKeys[i]); **//process in m different ways.How to make this happen in m independent threads?**
}
class Mymap extends HashMap<key,value> {
void process(String[] data,int[] keyIndexes)
{
//extract key from key indexes
//extract value from value indexes
put(key,value);
}
#Override
public Value put(Key k, Value v) {
if (containsKey(k)) {
oldval=get(k);
put(k,oldval.aggregate(v)); //put sum of old and new
return oldval;
}else{
put(k,v);
return null;
}
}
}
}
Sorry if i haven't made my point clear.In simple words the i want map[i].process(data,groupingKeys[i]); to happen in separate(i-th thread)
a b 5
a b 10
a c 15
so if i want to group by {1} and {1,2}
read line map1 map2
a b 5 [a--> b,5] [a,b ->5]
a b 10 [a-> b 15] [a,b->15]
a c 15 [a->b 30] [a,b->15 a,c->15]
Edit:
The question is not related to how i process or the logic of grouping but it is that: After each line is read i want to do something with the read line in different threads.

If I understand correctly, you wish to wait with processing until all the file is read. If so, depending on the details, you may want to check out CyclicBarrier or CountDownLatch
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CyclicBarrier.html
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html