We Keep Coding

Why does ExecutorService waits for all threads to complete but Completable future not? [duplicate]

This question already has an answer here:
CompletableFuture is not getting executed. If I use the ExecutorService pool it works as expected but not with the common ForkJoinPool
(1 answer)
Closed 1 year ago.
In this following code,
class MainX {
static void run(int i) {
try {
System.out.println(i + " called");
Thread.sleep(1000);
String s = "";
for (int j = 0; j < 20000; j++) {
s = s + j;
}
System.out.println(i + " completed" + " " + Thread.currentThread().getName());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
int p = i;
executorService.submit(() -> MainX.run(p));
}
System.out.println("all called");
executorService.shutdown();
System.out.println("all called" + " Thr:" + Thread.currentThread().getName());
}
}
(Vs)
class MainX {
static void run(int i) {
try {
System.out.println(i + " called");
Thread.sleep(1000);
String s = "";
for (int j = 0; j < 20000; j++) {
s = s + j;
}
System.out.println(i + " completed" + " " + Thread.currentThread().getName());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
for(int i = 0; i < 10; i++) {
int p = i;
CompletableFuture.runAsync(() -> MainX.run(p));
}
}
}
In the first case, jvm keeps on running until all the threads are completed. But in the second case, jvm and other threads are killed as soon as main thread dies.
Any reason for this?

From my viewpoint, 'CompletableFuture' does not itself execute anything, so it has no threads to wait for. It relies on other mechanisms for running stages.
'runAsync', without an Executor, runs tasks in its common ForkJoin pool, which is documented as having the behaviour you observe.
That doesn't answer your question of 'why', except to say that it's intentionally designed that way. I can only hand-wave and say that its designers likely considered it to be the best default choice.
(I concur: In code I've written, if I get to the point of program termination, what I want is for everything to just go away. In the rare case I need it to complete, I'd wait for it before exiting.)

Inline Thread objects runs much faster than class that inherits from Thread

Here is a main that runs a simple counting loop three ways:
Single-threaded
2 threads using inline code that creates two distinct Thread objects
2 threads using instances of the CountingThread class that inherits from Thread
package main;
import java.util.ArrayList;
public class Main {
public static void main(String[] args) {
demo();
}
public static void demo() {
final long limit = 100_000_000_000L;
long startTime = System.currenatTimeMillis();
for (long i = 0; i < limit; i++) {
// Nothing to see here, just counting
}
long endTime = System.currentTimeMillis();
System.out.println("Single threaded: Total execution time: " + (endTime - startTime) + " milliseconds.");
// Now try it in two threads. Each thread will perform 1/2 of the counting
Thread t1 = new Thread(new Runnable() {
#Override
public void run() {
for (long i = 0; i < limit/2; i++) {
// Nothing to see here, just counting
}
}
});
Thread t2 = new Thread(new Runnable() {
#Override
public void run() {
for (long i = limit/2; i < limit; i++) {
// Nothing to see here, just counting
}
}
});
startTime = System.currentTimeMillis();
t1.start();
t2.start();
// Join t1 until it ends, then join t2 until it ends. Note that t1 and t2 are running in parallel with this thread.
try {t1.join();} catch (InterruptedException e) {}
try {t2.join();} catch (InterruptedException e) {}
endTime = System.currentTimeMillis();
System.out.println("2 threaded using inline code: Total execution time: " + (endTime - startTime) + " milliseconds.");
// Now try it with 2 instances of the CountingThread class.
ArrayList<CountingThread> countingThreads = new ArrayList<CountingThread>();
int numberOfThreads = 2;
long increment = limit / numberOfThreads;
for (int i = 0; i < numberOfThreads; i++) {
long start, end;
start = i * increment;
end = start + increment;
countingThreads.add(new CountingThread(start, end));
}
// Launch all the threads to run in parallel
startTime = System.currentTimeMillis();
for (int i = 0; i < numberOfThreads; i++) {
countingThreads.get(i).run();
}
// Wait for all the threads to finish
for (int i = 0; i < numberOfThreads; i++) {
try {countingThreads.get(i).join();} catch(InterruptedException ex) {}
}
endTime = System.currentTimeMillis();
System.out.println(numberOfThreads + " threaded using classes: Total execution time: " + (endTime - startTime) + " milliseconds.");
}
}
Here is the class that inherits from Thread:
package main;
/**
* Count from one long int up to another long int. Really simple
*
*/
public class CountingThread extends Thread {
private long start, end;
public CountingThread(long start, long end) {
this.start = start;
this.end = end;
}
#Override
public void run() {
for(long i = start; i <= end; i++) {
}
// System.out.println("Thread counted from " + start + " to " + end);
}
}
Here is the output:
Single threaded: Total execution time: 40379 milliseconds.
2 threaded using inline code: Total execution time: 23312 milliseconds.
2 threaded using classes: Total execution time: 40358 milliseconds.
It seems like methods 2 and 3 should take about the same amount of time. What's up with that?
The machine has 4 cores.

You made a mistake and call #run instead of #start. Run method is executed in the same thread.
countingThreads.get(i).run();

Java [threads] - summarizing table with multiple threads doesn't seem to work

I am learning java, and trying to summarize elements in table with multiple threads, but I am always getting wrong result.
I tried 4 different methods of threads synchronization and all of them failed. Everything is explained in the comments.
My result (bad):
Without threads: 4949937, 15ms
With threads: 4944805, 78ms
Maybe am I executing the System.out.println on summarizeT() too early? I mean before all the threads finish work. With .join() the summarizeT() method works good but. Is the .join() blocking the "main" thread until all other threads are finished?
Main class:
public class Main
{
static int size = 100000; //size of tab
static int length = 100; //each thread gets 100 elements of tab, thread 1 calculates sum from 0 to 99, thread 2 from 100 to 199 etc.
static int[] tab = new int[size];
static Random generator = new Random();
static void initialize()
{
for (int i = 0; i < size; i++)
tab[i] = generator.nextInt(100);
}
static int summarize() //summarize with only one thread
{
int sum = 0;
for (int i = 0; i < size; i++)
sum += tab[i];
return sum;
}
static int summarizeT() //summarize with more threads (size / length)
{
int threadsCounter = size/length;
int start = 0; //pointer to table from where each thread should start counting
int[] sum = new int[1]; //I am sharing the sum value between threads with table, not sure if it is best method to pass the value between them
sum[0] = 0;
Thread[] threads = new Thread[threadsCounter]; //nedeed for .join() test
for (int i = 0; i < threadsCounter; i++)
{
threads[i] = new Thread(new MyThread(tab, start, sum));
threads[i].start();
start += length; //moving the start pointer, next thread should start counting from next 100 indexes
}
/*for (int i = 0; i < threadsCounter; i++) // adding .join() solves the problem, but is it a good solution?
{
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}*/
return sum[0];
}
public static void main(String[] args)
{
initialize();
long start = Calendar.getInstance().getTimeInMillis();
System.out.println("Without threads: " + summarize() + ", " + (Calendar.getInstance().getTimeInMillis() - start) + "ms");
start = Calendar.getInstance().getTimeInMillis();
System.out.println("With threads: " + summarizeT() + ", " + (Calendar.getInstance().getTimeInMillis() - start) + "ms"); //giving wrong answer
}
}
MyThread class:
import java.util.concurrent.Semaphore;
public class MyThread extends Thread
{
int[] tab;
int[] sum;
int start;
MyThread(int tab[], int start, int sum[]) //in args: main table, starting index, value that is being shared between threads
{
this.tab = tab;
this.start = start;
this.sum = sum;
}
#Override
public void run()
{
int end = start + Main.length; //place where thread should stop counting
int temp = 0; //nedeed to sumarize the "subtable"
while (start < end)
{
temp += tab[start];
start++;
}
// Method 1
Semaphore semaphore = new Semaphore(1);
try {
semaphore.acquire();
} catch (InterruptedException e1) {
e1.printStackTrace();
}
try
{
sum[0] += temp;
} catch (Exception e) {
} finally {
semaphore.release();
}
// Method 2
/*Object lock = new Object();
synchronized(lock)
{
sum[0] += temp;
}*/
// Method 3
/*synchronized(this)
{
sum[0] += temp;
}*/
// Method 4
//summarize(temp);
// Method 5 - no threads synchronization, works only when .join() is used, the same as other methods
//sum[0] += temp;
}
private synchronized void summarize(int value)
{
sum[0] += value;
}
}

Isn't the problem here that you create a lock object , or semaphore object, in each thread, rather than having one object that all threads synchronise on?
Each Thread creates is own Semaphore object ( for example ) so no other thread will ever contend with it. You need to create an object that ALL threads have access to and synchronise on that. You might consider synchronising on the array that you are writing the results in to itself.

There are some problems with your solution.
You should use AtomicInteger to hold results. that way you don't need to synchronize sum update.
BTW the way you synchronize is invalid. For semaphores to work you need to share same instance between all threads. And your try/catch/finally blocks are invalid. You should do acquire() and sum update in one try block, and release() in it's finally. That way you did it. It is possible that you'll do sum update even though acquire() failed.
Also you return from summarizeT() without waiting for threads to finish. You have to implement thread.join() logic or some other way to wait.

Producer Consumer-Average Wait times not outputting/buffer query

I am currently making a hypothetical producer consumer problem using java. The object is to have an operating system which is 1000 bytes, but only 500 bytes available to use for threads as 500 bytes have already been consumed by drivers and other operations. The threads are as follows:
A thread to start a BubbleWitch2 session of 10 seconds, which requires 100 bytes of RAM per
second
A thread to start a Spotify stream of 20 seconds, which requires 250 bytes of RAM per second
You should also take into account the fact that the operating system is simultaneously supporting system
activity and managing the processor, memory and disk space of the device on which it is installed.
Therefore, additionally create:
System and management threads, which, together, require 50 bytes of RAM per second, and
execute for a random length of time, once invoked.
A thread to install a new security update of 2 KB, which will be stored to disk, and requires 150
bytes of RAM per second while installing. Assume sufficient disk capacity in the system to support
this thread.
The operating system has only capacity for 200 bytes per second, therefore a larger thread such as spotify will experience delays or be forced to wait. I have used code which as far as I can tell, implements this. I am also required to generate exit times which I have done with timestamps and to calculate average waiting times for threads.
I have included code in my solution for the average waiting times with system.out.print but no matter what I do, it is not actually outputting the times at all-as if they did not exist.
I am also not sure if the buffer size limitations are working as it is a matter of milliseconds-is there any way to tell if this is working from the code below?
My main method.
public class ProducerConsumerTest {
public static void main(String[] args) throws InterruptedException {
Buffer c = new Buffer();
BubbleWitch2 p1 = new BubbleWitch2(c,1);
Processor c1 = new Processor(c, 1);
Spotify p2 = new Spotify(c, 2);
SystemManagement p3 = new SystemManagement(c, 3);
SecurityUpdate p4 = new SecurityUpdate(c, 4, p1, p2, p3);
p1.setName("BubbleWitch2 ");
p2.setName("Spotify ");
p3.setName("System Management ");
p4.setName("Security Update ");
p1.setPriority(10);
p2.setPriority(10);
p3.setPriority(10);
p4.setPriority(5);
c1.start();
p1.start();
p2.start();
p3.start();
p4.start();
p2.join();
p3.join();
p4.join();
System.exit(0);
}
}
My buffer class
import java.text.DateFormat;
import java.text.SimpleDateFormat;
/**
* Created by Rory on 10/08/2014.
*/
class Buffer {
private int contents, count = 0, process = 0;
private boolean available = false;
private long start, end, wait, request= 0;
private DateFormat time = new SimpleDateFormat("mm:ss:SSS");
public synchronized int get() {
while (process <= 500) {
try {
wait();
} catch (InterruptedException e) {
}
}
process -= 200;
System.out.println("CPU After Process " + process);
notifyAll();
return contents;
}
public synchronized void put(int value) {
while (process >= 1000) {
start = System.currentTimeMillis();
try {
wait();
} catch (InterruptedException e) {
}
end = System.currentTimeMillis();
wait = end - start;
count++;
request += wait;
System.out.println("Application Request Wait Time: " + time.format(wait));
process += value;
contents = value;
notifyAll();
}
}
}
My security update class
import java.lang.*;
import java.lang.System;
/**
* Created by Rory on 11/08/2014.
*/
class SecurityUpdate extends Thread {
private Buffer buffer;
private int number;
private int bytes = 150;
private int process = 0;
public SecurityUpdate(Buffer c, int number, BubbleWitch2 bubbleWitch2, Spotify spotify, SystemManagement systemManagement) throws InterruptedException {
buffer = c;
this.number = number;
bubbleWitch2.join();
spotify.join();
systemManagement.join();
}
public void run() {
for (int i = 0; i < 15; i++) {
buffer.put(i);
System.out.println(getName() + this.number
+ " put: " + i);
try {
sleep(1500);
} catch (InterruptedException e) {
}
}
System.out.println("-----------------------------");
System.out.println("Security Update has finished executing.");
System.out.println("------------------------------");
}
}
My processor class
class Processor extends Thread {
private Buffer processor;
private int number;
public Processor(Buffer c, int number) {
processor = c;
this.number = number;
}
public void run() {
int value = 0;
for (int i = 0; i < 60; i++) {
value = processor.get();
System.out.println("Processor #"
+ this.number
+ " got: " + value);
}
}
}
My bubblewitch class
import java.lang.*;
import java.lang.System;
import java.sql.Timestamp;
/**
* Created by Rory on 10/08/2014.
*/
class BubbleWitch2 extends Thread {
private Buffer buffer;
private int number;
private int bytes = 100;
private int duration;
public BubbleWitch2(Buffer c, int pduration) {
buffer = c;
duration = pduration;
}
long startTime = System.currentTimeMillis();
public void run() {
for (int i = 0; i < 10; i++) {
buffer.put(bytes);
System.out.println(getName() + this.number
+ " put: " + i);
try {
sleep(1000);
} catch (InterruptedException e) {
}
}
long endTime = System.currentTimeMillis();
long timeTaken = endTime - startTime;
java.util.Date date = new java.util.Date();
System.out.println("-----------------------------");
System.out.println("BubbleWitch2 has finished executing.");
System.out.println("Time taken to execute was " +timeTaken+ " milliseconds");
System.out.println("Time Bubblewitch2 thread exited Processor was " + new Timestamp(date.getTime()));
System.out.println("-----------------------------");
}
}
My system management
class SystemManagement extends Thread {
private Buffer buffer;
private int number, min = 1, max = 15;
private int loopCount = (int) (Math.random() * (max - min));
private int bytes = 50;
private int process = 0;
public SystemManagement(Buffer c, int number) {
buffer = c;
this.number = number;
}
public void run() {
for (int i = 0; i < loopCount; i++) {
buffer.put(50);
System.out.println(getName() + this.number
+ " put: " + i);
try {
sleep(1000);
} catch (InterruptedException e) {
}
}
System.out.println("-----------------------------");
System.out.println("System Management has finished executing.");
System.out.println("-----------------------------");
}
}
My spotify class
import java.sql.Timestamp;
/**
* Created by Rory on 11/08/2014.
*/
class Spotify extends Thread {
private Buffer buffer;
private int number;
private int bytes = 250;
public Spotify(Buffer c, int number) {
buffer = c;
this.number = number;
}
long startTime = System.currentTimeMillis();
public void run() {
for (int i = 0; i < 20; i++) {
buffer.put(bytes);
System.out.println(getName() + this.number
+ " put: " + i);
try {
sleep(1000);
} catch (InterruptedException e) {
}
}
long endTime = System.currentTimeMillis();
long timeTaken = endTime - startTime;
java.util.Date date = new java.util.Date();
System.out.println(new Timestamp(date.getTime()));
System.out.println("-----------------------------");
System.out.println("Spotify has finished executing.");
System.out.println("Time taken to execute was " + timeTaken + " milliseconds");
System.out.println("Time that Spotify thread exited Processor was " + date);
System.out.println("-----------------------------");
}
}
I may need to add timestamps to one or two classes yet but does anyone have any idea how to get my average times to actually print out? Or what is preventing it and if the buffer limitation is effectively being shown here(given that we are talking about milliseconds?)
Thanks.

The reason why sys out's are not printing is because of the below condition in your buffer class:-
public synchronized void put(int value) {
while (process >= 1000) {
.....
notifyAll();
}
}
this condition never gets satisified as the process never is greater than 1000
This is the reason why your Processor thread also gets stuck because when it calls get() it finds that the process is less than 500 and hence it indefinitely waits when it reaches the wait() line of code.
Rectifying the process condition appropriately in your put should let your missing sys out get printed
public synchronized void put(int value) {
if(process <= 500) {
process+=value;
} else {
//while (process >= 1000) {
start = System.currentTimeMillis();
try {
wait();
} catch (InterruptedException e) {
}
end = System.currentTimeMillis();
wait = end - start;
count++;
request += wait;
System.out.println("Application Request Wait Time: " + time.format(wait));
process += value;
contents = value;
//}
}
notifyAll();
}

If you want securityupdate thread to always run at the last then the correct way of using join within that thread is as below:-
class SecurityUpdate extends Thread {
private Buffer buffer;
private int number;
private int bytes = 150;
private int process = 0;
private BubbleWitch2 bubbleWitch2;
private Spotify spotify;
private SystemManagement systemManagement;
public SecurityUpdate(Buffer c, int number, BubbleWitch2 bubbleWitch2, Spotify spotify, SystemManagement systemManagement) throws InterruptedException {
buffer = c;
this.number = number;
this.bubbleWitch2 = bubbleWitch2;
this.spotify = spotify;
this.systemManagement = systemManagement;
}
public void run() {
try {
bubbleWitch2.join();
spotify.join();
systemManagement.join();
} catch (InterruptedException e) {
}
System.out.println("Finally starting the security update");
for (int i = 0; i < 15; i++) {
buffer.put(bytes); // Paul check if it should be i or bytes
System.out.println(getName() + this.number
+ " put: " + i);
try {
sleep(1500); // Paul why is this made to sleep 1500 seconds?
} catch (InterruptedException e) {
}
}
System.out.println("-----------------------------");
System.out.println("Security Update has finished executing.");
System.out.println("------------------------------");
}
}

ExecutorService.submit(<callable>) taking more time?

I am trying to understand the utilities in java.util.concurrent package and learnt that we can submit callable objects to the ExecutorService, which returns Future, which is filled with the value returned by the callable, after successful completion of task within call() method.
I am understanding that all the callables are executed concurrently using multiple threads.
When I wanted to see how much improvement ExecutorService gives over the batch task execution, i thought of capturing time.
Following is the code which i tried to execute -
package concurrency;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class ExecutorExample {
private static Callable<String> callable = new Callable<String>() {
#Override
public String call() throws Exception {
StringBuilder builder = new StringBuilder();
for(int i=0; i<5; i++) {
builder.append(i);
}
return builder.toString();
}
};
public static void main(String [] args) {
long start = System.currentTimeMillis();
ExecutorService service = Executors.newFixedThreadPool(5);
List<Future<String>> futures = new ArrayList<Future<String>>();
for(int i=0; i<5; i++) {
Future<String> value = service.submit(callable);
futures.add(value);
}
for(Future<String> f : futures) {
try {
System.out.println(f.isDone() + " " + f.get());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
long end = System.currentTimeMillis();
System.out.println("Executer callable time - " + (end - start));
service.shutdown();
start = System.currentTimeMillis();
for(int i=0; i<5; i++) {
StringBuilder builder = new StringBuilder();
for(int j=0; j<5; j++) {
builder.append(j);
}
System.out.println(builder.toString());
}
end = System.currentTimeMillis();
System.out.println("Normal time - " + (end - start));
}
}
and here is the output of this -
true 01234
true 01234
true 01234
true 01234
true 01234
Executer callable time - 5
01234
01234
01234
01234
01234
Normal time - 0
Please let me know if I am missing something OR understanding something in a wrong way.
Thanks in advance for your time and help for this thread.

If you task in Callable is to small, you won't get benefits from concurrency due task switching and overhead for initialisation. Try to add more heavier loop in callable, say 1000000 iterations, and you can see difference

When you run any code esp for the first time, it takes time. If you pass a task to another thread it can take 1-10 micro-seconds and if your task take less time than this, the overhead can be greater than the benefit. i.e. using multiple threads can be much slower than using a single thread if your overhead is high enough.
I suggest you
increase the cost of the task to 1000 iterations.
make sure the result is not discarded in the single threaded example
run both tests for at least a couple of seconds to ensure the code has warmed up.

Not an answer (but I am not sure the code will fit a comment). To expand a bit on what Peter said, there is usually a sweet spot for the size of your jobs (measured in execution time), to balance pool/queue overhead with fair work distribution among workers. The code example helps find an estimate for that sweet spot. Run on your target hardware.
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
public class FibonacciFork extends RecursiveTask<Long> {
private static final long serialVersionUID = 1L;
public FibonacciFork( long n) {
super();
this.n = n;
}
static ForkJoinPool fjp = new ForkJoinPool( Runtime.getRuntime().availableProcessors());
static long fibonacci0( long n) {
if ( n < 2) {
return n;
}
return fibonacci0( n - 1) + fibonacci0( n - 2);
}
static int rekLimit = 8;
private static long stealCount;
long n;
private long forkCount;
private static AtomicLong forks = new AtomicLong( 0);
public static void main( String[] args) {
int n = 45;
long times[] = getSingleThreadNanos( n);
System.out.println( "Single Thread Times complete");
for ( int r = 2; r <= n; r++) {
runWithRecursionLimit( r, n, times[ r]);
}
}
private static long[] getSingleThreadNanos( int n) {
final long times[] = new long[ n + 1];
ExecutorService es = Executors.newFixedThreadPool( Math.max( 1, Runtime.getRuntime().availableProcessors() / 2));
for ( int i = 2; i <= n; i++) {
final int arg = i;
Runnable runner = new Runnable() {
#Override
public void run() {
long start = System.nanoTime();
final int minRuntime = 1000000000;
long runUntil = start + minRuntime;
long result = fibonacci0( arg);
long end = System.nanoTime();
int ntimes = Math.max( 1, ( int) ( minRuntime / ( end - start)));
if ( ntimes > 1) {
start = System.nanoTime();
for ( int i = 0; i < ntimes; i++) {
result = fibonacci0( arg);
}
end = System.nanoTime();
}
times[ arg] = ( end - start) / ntimes;
}
};
es.execute( runner);
}
es.shutdown();
try {
es.awaitTermination( 1, TimeUnit.HOURS);
} catch ( InterruptedException e) {
System.out.println( "Single Timeout");
}
return times;
}
private static void runWithRecursionLimit( int r, int arg, long singleThreadNanos) {
rekLimit = r;
long start = System.currentTimeMillis();
long result = fibonacci( arg);
long end = System.currentTimeMillis();
// Steals zählen
long currentSteals = fjp.getStealCount();
long newSteals = currentSteals - stealCount;
stealCount = currentSteals;
long forksCount = forks.getAndSet( 0);
System.out.println( "Fib(" + arg + ")=" + result + " in " + ( end-start) + "ms, recursion limit: " + r +
" at " + ( singleThreadNanos / 1e6) + "ms, steals: " + newSteals + " forks " + forksCount);
}
static long fibonacci( final long arg) {
FibonacciFork task = new FibonacciFork( arg);
long result = fjp.invoke( task);
forks.set( task.forkCount);
return result;
}
#Override
protected Long compute() {
if ( n <= rekLimit) {
return fibonacci0( n);
}
FibonacciFork ff1 = new FibonacciFork( n-1);
FibonacciFork ff2 = new FibonacciFork( n-2);
ff1.fork();
long r2 = ff2.compute();
long r1 = ff1.join();
forkCount = ff2.forkCount + ff1.forkCount + 1;
return r1 + r2;
}
}