We Keep Coding

Java scheduleAtFixedRate + Thread.sleep

I'm just exploring method scheduleAtFixedRate of class ScheduledExecutorService in Java.
Here is my suspicious code:
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(5);
Runnable command = () -> {
System.out.println("Yo");
try {
Thread.sleep(4000);
} catch (InterruptedException e) {
e.printStackTrace();
}
};
scheduledExecutorService.scheduleAtFixedRate(command, 0, 1, TimeUnit.SECONDS);
I expected that every 1 second scheduledExecutorService will try to take new thread from the pool and start it.
API says: "scheduledExecutorService creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period. /(unimportant deleted)/ If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute."
Result - every new thread starts every 4 seconds.
So, the questions:
What's the catch - Does Thread.sleep() stop all threads or nuance in this behavior - "If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute"?
If "will not concurrently execute" is true in this situation - why do we need this pool of several threads if every thread will start after execution of previous thread?
Is there any simple valid example of usage of scheduleAtFixedRate, where one thread starts while previous still executes?

The answer is in the quote you provided. Executor waits until the task finishes before launching this task again. It prevents concurrent execution of many instances of one task - in most cases this behaviour is needed. In your case Executor starts a task, then waits 1 second of delay, then waits 3 more seconds until current task is done and only then starts this task again (It does not necessarily start new thread, it may start the task in the same thread).
Your code does not use thread pool at all - you can get exactly same result using single thread executor.
If you want to get this behaviour:
I expected that every 1 second scheduledExecutorService will try to take new thread
from the pool and start it.
Then you may write is like this:
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(5);
Runnable command = () -> {
System.out.println("Yo");
try {
Thread.sleep(4000);
} catch (InterruptedException e) {
e.printStackTrace();
}
};
Runnable commandRunner = () -> {
scheduledExecutorService.schedule(command, 0, TimeUnit.SECONDS);
}
scheduledExecutorService.scheduleAtFixedRate(commandRunner, 0, 1, TimeUnit.SECONDS);
(It's better to create a single-threaded ScheduledExecutorService that runs commandRunner and create a thread pool based ExecutorService that is used by commandRunner to execute command)

What's the catch - Does Thread.sleep() stop all threads or nuance in
this behavior - "If any execution of this task takes longer than its
period, then subsequent executions may start late, but will not
concurrently execute"?
I didn't quite understand what you mean here. But, essentially speaking, in the code that you have shared, Thread.sleep() is just making the thread execution take 4 seconds, which is longer than the set period of 1 second. Thus, subsequent threads will not execute after 1 second, but only after ~4 seconds of execution of the previous thread.
If "will not concurrently execute" is true in this situation - why do
we need this pool of several threads if every thread will start after
execution of previous thread?
You may want to schedule some other type of threads (which do a different job) in the same executor, which may run in parallel to the code which you have shared. Your current code only needs 1 thread in the pool though, since you are scheduling only one job (Runnable).
Is there any simple valid example of usage of scheduleAtFixedRate,
where one thread starts while previous still executes?
As stated in the documentation, concurrent execution will not happen for the job that you scheduled at fixed rate (with the current code)

public class Poll {
ScheduledFuture<?> future;
static int INIT_DELAY = 1;
static int REPEAT_PERIOD = 2;
static int MAX_TRIES = 3;
int tries = 1;
Runnable task = () -> {
System.out.print( tries + ": " + Thread.currentThread().getName() + " " );
if ( ++tries > MAX_TRIES ) {
future.cancel( false );
}
};
void poll() {
ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
future = executor.scheduleAtFixedRate( task, INIT_DELAY, REPEAT_PERIOD, TimeUnit.SECONDS );
System.out.println( "Start: " + tries + ": " + Thread.currentThread().getName() + " " );
try {
future.get();
} catch ( InterruptedException | ExecutionException e ) {
System.out.println( e.getMessage() );
} catch ( CancellationException e ) {
System.out.println( "Regular End Of Scheduled Task as Designed.");
} finally {
executor.shutdown();
executor.shutdownNow();
}
System.out.println( "Return The Result." );
}
// The Driver
public static void main( String[] args ) {
new Poll().poll();
}
}

ScheduledExecutorService execute tasks late, with low CPU & RAM usage

I need to create multiple tasks, each of that executes every n seconds. I've decided to use ScheduledExecutorService to schedule task execution. The problem is tasks not executed in time. I thought the reason is not enough processor time, but actual CPU usage is about 4-5 percents.
My schedulers creator:
class SchedulersCreator {
private final ScheduledExecutorService scheduler
= Executors.newScheduledThreadPool(1);
public SchedulersCreator(int tasksAmount, int repeatCount) {
for (int taskId = 0; taskId <= tasksAmount; taskId++) {
// create new task, that executes every 2 seconds
MyTask task = new MyTask(scheduler, repeatCount, 2, taskId);
// execute new task
task.run();
}
}
public static void main(String[] args) {
System.out.println("Program started");
// create & start 10 tasks, each of the executes 10 times with period 2 seconds
SchedulersCreator scheduler = new SchedulersCreator(10, 10);
System.out.println("All tasks created & started");
}
}
My task:
class MyTask implements Runnable {
// number of executions
private int executesTimesLeft;
// execution period
private final int periodSeconds;
// task id
private final int id;
// scheduler
private ScheduledExecutorService scheduler;
// field to measure time between executions
private long lastExecution = 0;
public MyTask(ScheduledExecutorService scheduler, int executes, int periodSeconds, int id) {
this.executesTimesLeft = executes;
this.id = id;
this.periodSeconds = periodSeconds;
this.scheduler = scheduler;
}
private void performAction() {
long before = System.currentTimeMillis();
long time = (before - lastExecution) % 1_000_000;
lastExecution = before;
// Simulates useful calculations
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
}
long after = System.currentTimeMillis();
if (id % 100_000 == 0) {
long duration = after - before;
System.out.println("Time since prev execution:\t" + time + "\t"
+ "Task " + id + ": "
+ executesTimesLeft + " executions lefts; "
+ "current duration\t" + duration);
}
}
#Override
public void run() {
// perform useful calculation in another thread
new Thread(() -> performAction()).run();
executesTimesLeft--;
if (executesTimesLeft > 0) { // schedule next task execution
scheduler.schedule(this, periodSeconds, SECONDS);
}
}
}
The code at the ideone: https://ideone.com/s3iDif.
I've expected time between executions about 2 seconds, but the actual result is 3-4 seconds.
Program output:
...
Time since prev execution: 3028 Task 0: 2 executions lefts; current duration 1000
Time since prev execution: 4001 Task 0: 1 executions lefts; current duration 1001

Your code doesn't use the scheduler properly.
// perform useful calculation in another thread
new Thread(() -> performAction()).run();
This doesn't actually run the code in a new thread. To do that you need to call start(), not run(). Calling run() makes the code execute in the current thread, no different than if you had just written performAction();.
However, you shouldn't be explicitly creating a new thread at all. You can and should do the work right in MyTask.run().
Tasks don't need to know about the scheduler or their frequency. Change this code:
MyTask task = new MyTask(scheduler, repeatCount, 2, taskId);
// execute new task
task.run();
to:
MyTask task = new MyTask(repeatCount, taskId);
Future<?> future = scheduler.scheduleAtFixedRate(task, 0, 2, SECONDS);
You want the task to repeat, so use the scheduler method that does so. That'll allow the scheduler to adjust the time in between tasks based on how long they take to run.
Move all of performAction() into MyTask.run(). When you want the task to stop repeating, use the future to cancel it.

How to check if the threads have completed its task or not?

OK, I created couples of threads to do some complex task. Now How may I check each threads whether it has completed successfully or not??
class BrokenTasks extends Thread {
public BrokenTasks(){
super();
}
public void run(){
//Some complex tasks related to Networking..
//Example would be fetching some data from the internet and it is not known when can it be finished
}
}
//In another class
BrokenTasks task1 = new BrokenTasks();
BrokenTasks task2 = new BrokenTasks();
BrokenTasks task3 = new BrokenTasks();
BrokenTasks task4 = new BrokenTasks();
task1.start();
.....
task4.start();
So how can I check if these all tasks completed successfully from
i) Main Program (Main Thread)
ii)From each consecutive threads.For example: checking if task1 had ended or not from within task2..

A good way to use threads is not to use them, directly. Instead make a thread pool. Then in your POJO task encapsulation have a field that is only set at the end of computation.
There might be 3-4 milliseconds delay when another thread can see the status - but finally the JVM makes it so. As long as other threads do not over write it. That you can protect by making sure each task has a unique instance of work to do and status, and other threads only poll that every 1-5 seconds or have a listener that the worker calls after completion.
A library I have used is my own
https://github.com/tgkprog/ddt/tree/master/DdtUtils/src/main/java/org/s2n/ddt/util/threads
To use : in server start or static block :
package org.s2n.ddt.util;
import org.apache.log4j.Logger;
import org.junit.Test;
import org.s2n.ddt.util.threads.PoolOptions;
import org.s2n.ddt.util.threads.DdtPools;
public class PoolTest {
private static final Logger logger = Logger.getLogger(PoolTest.class);
#Test
public void test() {
PoolOptions options = new PoolOptions();
options.setCoreThreads(2);
options.setMaxThreads(33);
DdtPools.initPool("a", options);
Do1 p = null;
for (int i = 0; i < 10; i++) {
p = new Do1();
DdtPools.offer("a", p);
}
LangUtils.sleep(3 + (int) (Math.random() * 3));
org.junit.Assert.assertNotNull(p);
org.junit.Assert.assertEquals(Do1.getLs(), 10);
}
}
class Do1 implements Runnable {
volatile static long l = 0;
public Do1() {
l++;
}
public void run() {
// LangUtils.sleep(1 + (int) (Math.random() * 3));
System.out.println("hi " + l);
}
public static long getLs() {
return l;
}
}
Things you should not do:
* Don't do things every 10-15 milliseconds
* Unless academic do not make your own thread
* don't make it more complex then it needs for 97% of cases

You can use Callable and ForkJoinPool for this task.
class BrokenTasks implements Callable {
public BrokenTasks(){
super();
}
public Object call() thrown Exception {
//Some complex tasks related to Networking..
//Example would be fetching some data from the internet and it is not known when can it be finished
}
}
//In another class
BrokenTasks task1 = new BrokenTasks();
BrokenTasks task2 = new BrokenTasks();
BrokenTasks task3 = new BrokenTasks();
BrokenTasks task4 = new BrokenTasks();
ForkJoinPool pool = new ForkJoinPool(4);
Future result1 = pool.submit(task1);
Future result2 = pool.submit(task2);
Future result3 = pool.submit(task3);
Future result4 = pool.submit(task4);
value4 = result4.get();//blocking call
value3 = result3.get();//blocking call
value2 = result2.get();//blocking call
value1 = result1.get();//blocking call
And don't forget to shutdown pool after that.

Classically you simply join on the threads you want to finish. Your thread does not proceed until join completes. For example:
// await all threads
task1.join();
task2.join();
task3.join();
task4.join();
// continue with main thread logic
(I probably would have put the tasks in a list for cleaner handling)

If a thread has not been completed its task then it is still alive. So for testing whether the thread has completed its task you can use isAlive() method.

There are two different questions here
One is if the thread still working.
The other one is if the task still not finished.
Thread is a very expensive method to solve problem, when we start a thread in java, the VM has to store context informations and solve synchronize problems(such as lock). So we usually use thread pool instead of directly thread. The benefit of thread pool is that we can use few thread to handle many different tasks. That means few threads keeps alive, while many tasks are finished.
Don’t find task status from a thread.
Thread is a worker, and tasks are jobs.
A thread may work on many different jobs one by one.
I don’t think we should ask a worker if he has finished a job. I’d rather ask the job if it is finished.
When I want to check if a job is finished, I use signals.
Use signals (synchronization aid)
There are many synchronization aid tools since JDK 1.5 works like a signal.
CountDownLatch
This object provides a counter(can be set only once and count down many times). This counter allows one or more threads to wait until a set of operations being performed in other threads completes.
CyclicBarrier
This is another useful signal that allows a set of threads to all wait for each other to reach a common barrier point.
more tools
More tools could be found in JDK java.util.concurrent package.

You can use Thread.isAlive method, see API: "A thread is alive if it has been started and has not yet died". That is in task2 run() you test task1.isAlive()
To see task1 from task2 you need to pass it as an argument to task2's construtor, or make tasks fields instead of local vars

You can use the following..
task1.join();
task2.join();
task3.join();
task4.join();
// and then check every thread by using isAlive() method
e.g : task1.isAlive();
if it return false means that thread had completed it's task
otherwise it will true

I'm not sure of your exact needs, but some Java application frameworks have handy abstractions for dealing with individual units of work or "jobs". The Eclipse Rich Client Platform comes to mind with its Jobs API. Although it may be overkill.
For plain old Java, look at Future, Callable and Executor.

My timer's thread may be cheating on him with Thread.sleep?

So essentially I am concerned that my timertask's run method is not being called after 10 minutes because I am putting the main thread to sleep for 10 seconds to avoid crazy CPU usage to just run through an empty while loop all day. Here is the code for the java main method.
private static boolean testDone = false;
public static void main(String[] args)
{
final int minutes = 10;
final StressTest test = new StressTest(someParams);
test.start();
Timer timer = new Timer();
timer.schedule(new TimerTask(){
#Override
public void run() {
testDone = true;
int completedSynths = test.stop();
System.out.println("Completed Synths: " + completedSynths);
System.out.println("Elapsed Time in Minutes: " + minutes);
System.out.println("Throughput (synths/min): " + completedSynths/minutes);
}}, minutes*60*1000);
while(!testDone)
{
System.out.println("Still not done... sleeping for 10 secs....");
Thread.sleep(10000);
}
System.exit(0);
Even crazier, the System.out in the while loop is never printing. What have I done??
EDIT: TO add pseudocode for StressTest object
public class StressTest
{
private SecureRandom random = new SecureRandom();
private volatile int completedSynths = 0;
private volatile boolean shouldStop = false;
private Thread[] threads;
/**
* Instantiate a new StressTest object.
*
* #param threadCount number of concurrent threads to be polling server
*/
public StressTest(int threadCount)
{
threads = new Thread[threadCount];
}
public void start()
{
System.out.println("Starting Stress Test....");
for(int i = 0; i < threads.length; i++)
{
Runnable synthCaller = new SynthApiCaller();
threads[i] = new Thread(null, synthCaller, "SynthThread" + i);
threads[i].run();
}
}
public int stop()
{
System.out.println("Stopping Stress Test...");
shouldStop = true;
return completedSynths;
}
private String randId()
{
return new BigInteger(130, random).toString(32);
}
private class SynthApiCaller implements Runnable
{
#Override
public void run()
{
while(!shouldStop)
{
try
{
//this class makes an HTTP request to a server and then writes result to a file
MyClass.writeFile( "someFileLoc/file.data");
completedSynths++;
Runtime.getRuntime().exec("rm -r " + fileLoc);
System.out.println("Synth # " + completedSynths);
}
catch (IOException e)
{
System.out.println(e.getMessage());
}
}
System.out.println("Thread terminated...");
}
}
}

I am concerned that my timertask's run method is not being called after 10 minutes because I am putting the main thread to sleep for 10 seconds
The Thread.sleep(...) in your main thread will not affect the running of the Timer. If the timer is not running after 10 minutes then is it possible that test.stop() is blocking?
It is important to realize that if the test is started in the main-thread and then is being stopped in the Timer thread then some synchronization will be necessary. I assume the test is running in another thread. You probably will need it to be synchronized then inside of the Timer thread you would call something like:
synchronized (test) {
test.start();
}
If you are new to Java synchronization, there are some good tutorials online.
If you are wondering whether or not the timer is being called at all, I'd set a break point inside your timer task where is sets testDone = true and see if it gets there.
Here's a good tutorial of using a debugger in eclipse.
Even crazier, the System.out in the while loop is never printing. What have I done??
As #assylias mentioned, the System.out(...) in your while loop not showing up must mean that testDone is set to true. Since testDone is being updated and accessed in different threads, you need to make sure it is also volatile.
I just ran your code sample without the test.start() and stop() and it seems to work fine. The problem may be in your test code.
Still not done... sleeping for 10 secs....
Still not done... sleeping for 10 secs....
...
Completed Synths: 1
Elapsed Time in Minutes: 10
Throughput (synths/min): 0
Now that you've added more code, here are some comments:
completedSynths++; should be changed to be an AtomicInteger. ++ is not an atomic operation so even tho the field is volatile, multiple threads can overwrite each other's increment.
If you are trying to wait for the threads to complete, instead of sleeping for 10 minutes, I'd recommend calling thread[i].join() with the threads. Even better would be use an ExecutorService and use the awaitTermination(...) method.
You call shouldStop = true; and then return the completedSynths;. You may want to wait for the threads to finish or something.
I'd not pass in a null ThreadGroup to the Thread constructor. Just use the constructor without the ThreadGroup.

I suggest making testDone volatile. As it is, I don't see anything forcing changes to testDone to be visible to reads in threads other than the one making the change.

Make a Scheduled Call to run One More time

We have a scheduled task that runs every 10 seconds and a thread pool with 3 threads that actually update a static common map. Every 10 seconds the scheduled action prints this map.
The problem is that I want the scheduler to stop printing after the 3 threads finish with the map. But here is the key. I don't want to stop scheduler instantly, I want to print first ( the final version of the map) and then finishes.
public class myClass implements ThreadListener {
public static ArrayList<Pair<String, Integer>> wordOccurenceSet = new ArrayList<Pair<String, Integer>>();
int numberOfThreads = 0;
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
public void getAnswer(Collection<CharacterReader> characterReaders, Outputter outputter) {
ExecutorService executor = Executors.newFixedThreadPool(characterReaders.size());
OutputterWriteBatch scheduledThread = new OutputterWriteBatch(outputter,wordOccurenceSet);
scheduler.scheduleAtFixedRate(scheduledThread, 10, 10, TimeUnit.SECONDS);
for (CharacterReader characterReader : characterReaders) {
NotifyingRunnable runnable = new CharacterReaderTask(characterReader, wordOccurenceSet);
runnable.addListener(this);
executor.execute(runnable);
}
}
#Override
public void notifyRunnableComplete(Runnable runnable) {
numberOfThreads += 1;
if(numberOfThreads == 3 ){
//All threads finished... What can I do to terminate after one more run?
}
}
}
The Listener actually just get notified when a thread finishes.

First of all, make your numberOfThreads synchronized. You don't want it to become corrupted when two Reader threads finish concurrently. It's a primitive int so it may not be corruptable (i am not that proficient with JVM), but the general rules of thread safety should be followed anyway.
// 1. let finish OutputterWriteBatch if currently running
scheduler.shutdown();
// 2. will block and wait if OutputterWriteBatch was currently running
scheduler.awaitTermination(someReasonableTimeout);
// 3. one more shot.
scheduler.schedule(scheduledThread,0);
// You could also run it directly if your outputting logic in run()
// is published via separate method, but i don't know the API so i suppose
// only Runnable is published
But this shouldn't be called directly from notifyRunnableComplete, of course. The listener method is called from your Reader threads, so it would block the last one of 3 threads from finishing timely. Rather make a notification object which some other thread will wait() on (preferably the one which executed getAnswer()), notify() it when numberOfThreads reaches 3 and put the above code after the wait().
Oh, and when wait() unblocks, you should double check that numberOfThreads is really 3, if not, cycle back to wait(). Google "spurious wakeup" to explanation why this is needed.