We Keep Coding

Java, Thread and Priorities

I'm working on a problem, that was supposed to be VERY simple to solve, however I am not getting it done so easily.
The problem is quite simple: I have a Java Program running on Linux/x86 that can perform two basic functionalities F1 and F2. I would like to set F1 to have a higher priority, even though it is a MUST that F2 executes from times to times, i.e, the fact of having F1 requests on queue cannot put F2 requests waiting forever.
My first though was just having separate queues with a thread pool for each functionality, I set the F1 pool to have 8 threads while the F2 pool got only 2 threads.
On my expectaion linux would give fairly time share for each thread, so F1 would have 8 quantums while F2 would get just 2. If there was no F1 requests, F2 pool could get every quantum to itself, the same should be true for F1 just in case F2 has no requests.
However, the program is not behavingthat way, if I get a burst of F2 requests and just couple of F1 requets, the latter is taking a long time to get its turn.
Doest that make sense talking about Oracle HotSpot/linux scheduling? Or it should not be happening, what would point to an implementation error from my part?
PS: I've read about linux scheduling, and it seems that SCHED_OTHER (TS) gives time share for each task, however every time a task ready is not executed it gets a bigger quantum, and if that is happening to F2 pool, that might explain the above mentioned behavior.
Thanks and Regards.
Below there is a sample source code.
package test;
import java.util.Properties;
import java.util.concurrent.ArrayBlockingQueue;
/**
* Created by giscardff on 08/07/18.
*/
public class TestThread {
// Test Program
public static void main(String args[]) throws Exception {
// queues containing jobs to be done
ArrayBlockingQueue<MyDTO> queueA = new ArrayBlockingQueue<>(100);
ArrayBlockingQueue<MyDTO> queueB = new ArrayBlockingQueue<>(100);
// create pool for functionality A
for(int i = 1; i <= 8; i++){
MyThread thread = new MyThread("ThreadA" + i, queueA);
thread.start();
}
// create pool for functionality B
for(int i = 1; i <= 2; i++){
MyThread thread = new MyThread("ThreadB" + i, queueB);
thread.start();
}
// create producer for A
// it will take 100ms between requests
Producer producerA = new Producer(queueA, 0);
producerA.start();
// create producer for B
// it will take 0ms between requests
Producer producerB = new Producer(queueB, 0);
producerB.start();
}
}
/**
* Just put a request into a queue
*/
class Producer extends Thread {
private ArrayBlockingQueue<MyDTO> queue;
private long sleep;
public Producer(ArrayBlockingQueue<MyDTO> queue, long sleep){
this.queue = queue;
this.sleep = sleep;
}
#Override
public void run() {
try {
while (true) {
if(sleep > 0)Thread.sleep(sleep);
queue.put(new MyDTO());
}
}catch(Exception ex){}
}
}
/**
* Retrieve a request from a queue, calculate how long request took to
* be received for each 1M requests
*/
class MyThread extends Thread {
private ArrayBlockingQueue<MyDTO> queue;
private long delay = 0;
private int count = 0;
public MyThread(String name, ArrayBlockingQueue<MyDTO> queue){
super(name);
this.queue = queue;
}
#Override
public void run() {
try {
while (true) {
MyDTO input = queue.take();
delay += System.currentTimeMillis() - Long.parseLong(input.getTime());
if(++count % 1000 == 0){
System.out.printf("%s: %d\n", getName(), delay / 10);
count = 0;
}
}
}catch(Exception ex){ex.printStackTrace();}
}
}
/**
* Just a DTO representing a request
* NOTE: The time was set as String to force CPU to do something more than just math operations
*/
class MyDTO {
private String time;
public MyDTO(){
this.time = "" + System.currentTimeMillis();
}
public String getTime() {
return time;
}
}

It looks like you've got a few issues. I'll try to summarize them and provide starting point for a path forward:
Thread Contention
Using the BlockingQueue comes with a cost - every write operation (put & take) is lock contended between the producers or consumers. Your "A pool" has 9 threads contending over the write lock for queueA (1 producer, 8 consumers), while your "B pool" has 3 threads contending over the lock for queueB (1 producer, 2 consumers).
This related answer provides a bit more detail about contention. The simplest ways around this are to "use less threads" or use "lock-free" mechanisms to eliminate the contention.
Thread Scheduling
As mentioned in the comments, you're at the mercy of how the JVM is scheduling your threads.
If java thread scheduling used perfectly fair time shares on the CPU, you'd probably see consumption counts on each thread in the same pool extremely close to each other. You've probably noticed they're not - my runs of your (slightly modified) code occasionally give me a count spread of 300K or more across the threads.
You can often get this better when there are enough CPU cores for each CPU-bound thread (you've got 12 in your sample code), but it's far from ideal in many cases, especially in the face of thread contention.
What can you do?
Build your own fairness logic - don't rely on the JVM thread scheduler to be fair, because it won't be.
A simple idea for your case would be to keep both queues, but use a single pool to process both - use either round-robin or Math.random() (ie: if (rand < 0.8) { queueA.poll();}) to determine which queue to poll from. Note - Use poll so you can easily handle the case when a queue is empty without blocking.
Experiment with number of CPU-bound threads running on your hardware. With my suggestion for (1) above, you could even have just one worker thread fairly processing both queues. Remember, too many threads contending over the same resources will slow down your processing.
Isn't threading fun? :)

Java - Implement threading in large input set

I need to do some computations/processing on a large set of ids (about 100k to 1 Million). Since the number of ids is quite large and each processing does take some time, i was thinking about implementing threads in my Java code.
Assuming we cant have 100K threads running at once, how do i implement threading in this case ?
Note - The only solution i can think of is have about 100 or more threads running where each thread would process about a 1000 or more IDs.

Use Java's built in thread pooling and executors.
ExecutorService foo = Executors.newFixedThreadPool(100);
foo.submit(new MyRunnable());
There are various thread pools you can create to tailor how many you want, if it's dynamic, etc.

Using ThreadPool:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class ThreadIDS implements Runnable
{
public static final int totalIDS = 1000000;
int start;
int range;
public ThreadIDS(int start, int range)
{
this.start=start;
this.range=range;
}
public static void main(String[] args)
{
int availableProcessors = Runtime.getRuntime().availableProcessors();
int eachThread = totalIDS/availableProcessors + 1;
ExecutorService threads = Executors.newFixedThreadPool(availableProcessors);
for(int i = 0 ; i < availableProcessors ; i++)
{
threads.submit(new ThreadIDS(i*eachThread, eachThread));
}
while(!threads.awaitTermination(1000, TimeUnit.MILLISECONDS))System.out.println("Waiting for threads to finish");
}
public void processID(int id)
{
}
public void run()
{
for(int i = start ; i < Math.min(start+range, totalIDS) ; i++)
{
processID(i);
}
}
}
Edited the run method. Since we add 1 when dividing to avoid integer division making us miss ids, we could potentially run over the totalIDS limit. The Math.min avoids that.
If you don't want to use ThreadPools, then change the main to:
public static void main(String[] args)
{
int availableProcessors = Runtime.getRuntime().availableProcessors();
int eachThread = totalIDS/availableProcessors + 1;
for(int i = 0 ; i < availableProcessors ; i++)
{
new Thread(new ThreadIDS(i * eachThread, eachThread)).start();
}
}

Run as many threads as you have CPU cores (Runtime.getRuntime().availableProcessors()). Let each thread runs loop like this:
public void run() {
while (!ids.isEmpty()) {
Id id = ids.poll(); // exact access method depends on how your set of ids is organized
processId(id);
}
}
Comparing to using thread pool, this is simpler and requires less memory (no need to create Runnable for each id).

Splitting your work into 4 Runnables (1 per core) is probably not the best idea if there is any variation in processing time for a given ID. A better solution would be to split your work up into small chunks so that one core doesn't get stuck with all the "hard" work while the other 3 cores plow through theirs and then do nothing.
You could split your tasks into small chunks in advance and submit them to a ThreadPoolExecutor, but it might be better to use the Fork/Join framework. It's designed to handle this type of thing very efficiently.
Something like this would make sure all 4 cores stayed busy until all the work was done:
public class Test
{
public void workTest()
{
ForkJoinPool pool = new ForkJoinPool(); //Defaults to # of cores
List<ObjectThatWeProcess> work = getWork(); //Get IDs or whatever
FJAction action = new FJAction(work);
pool.invoke(action);
}
public static class FJAction extends RecursiveAction
{
private static final workSize = 1000; //Only do work if 1000 objects or less
List<ObjectThatWeProcess> work;
FJAction(List<ObjectThatWeProcess> work)
{
this.work = work;
}
public void compute()
{
if(work.size() > workSize)
{
invokeAll(new FJAction(work.subList(0,work.size()/2)),
new FJAction(work.subList(work.size()/2,work.size())));
}
else
processWork();
}
private void processWork()
{
//do something
}
}
}
You could also extend RecursiveTask<T> if the "work" returned a value that was relevant to you.

Java: How Executors relate to Queues

So in Java concurrency, there is the concept of a task which is really any implementing Runnable or Callable (and, more specifically, the overridden run() or call() method of that interface).
I'm having a tough time understanding the relationship between:
A task (Runnable/Callable); and
An ExecutorService the task is submitted to; and
An underlying, concurrent work queue or list structure used by the ExecutorService
I believe the relationship is something of the following:
You, the developer, must select which ExecutorService and work structure best suits the task at hand
You initialize the ExecutorService (say, as a ScheduledThreadPool) with the underlying structure to use (say, an ArrayBlockingQueue) (if so, how?!?!)
You submit your task to the ExecutorService which then uses its threading/pooling strategy to populate the given structure (ABQ or otherwise) with copies of the task
Each spawned/pooled thread now pulls copies of the task off of the work structure and executes it
First off, please correct/clarify any of the above assumptions if I am off-base on any of them!
Second, if the task is simply copied/replicated over and over again inside the underlying work structure (e.g., identical copies in each index of a list), then how do you ever decompose a big problem down into smaller (concurrent) ones? In other words, if the task simply does steps A - Z, and you have an ABQ with 1,000 of those tasks, then won't each thread just do A - Z as well? How do you say "some threads should work on A - G, while other threads should work on H, and yet other threads should work on I - Z", etc.?
For this second one I might need a code example to visualize how it all comes together. Thanks in advance.

Your last assumption is not quite right. The ExecutorService does not pull copies of the task. The program must supply all tasks individually to be performed by the ExecutorService. When a task has finished, the next task in the queue is executed.
An ExecutorService is an interface for working with a thread pool. You generally have multiple tasks to be executed on the pool, and each operates on a different part of the problem. As the developer, you must specify which parts of the problem each task should work on when creating it, before sending it to the ExecutorService. The results of each task (assuming they are working on a common problem) should be added to a BlockingQueue or other concurrent collection, where another thread may use the results or wait for all tasks to finish.
Here is an article you may want to read about how to use an ExecutorService: http://www.vogella.com/articles/JavaConcurrency/article.html#threadpools
Update: A common use of the ExecutorService is to implement the producer/consumer pattern. Here is an example I quickly threw together to get you started--it is intended for demonstration purposes only, as some details and concerns have been omitted for simplicity. The thread pool contains multiple producer threads and one consumer thread. The job being performed is to sum the numbers from 0...N. Each producer thread sums a smaller interval of numbers, and publishes the result to the BlockingQueue. The consumer thread processes each result added to the BlockingQueue.
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class NumberCounter {
private final ExecutorService pool = Executors.newFixedThreadPool(2);
private final BlockingQueue<Integer> queue = new ArrayBlockingQueue(100);
public void startCounter(int max, int workers) {
// Create multiple tasks to add numbers. Each task submits the result
// to the queue.
int increment = max / workers;
for (int worker = 0; worker < workers; worker++) {
Runnable task = createProducer(worker * increment, (worker + 1) * increment);
pool.execute(task);
}
// Create one more task that will consume the numbers, adding them up
// and printing the results.
pool.execute(new Runnable() {
#Override
public void run() {
int sum = 0;
while (true) {
try {
Integer result = queue.take();
sum += result;
System.out.println("New sum is " + sum);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
});
}
private Runnable createProducer(final int start, final int stop) {
return new Runnable() {
#Override
public void run() {
System.out.println("Worker started counting from " + start + " to " + stop);
int count = 0;
for (int i = start; i < stop; i++) {
count += i;
}
queue.add(count);
}
};
}
public static void main(String[] args) throws InterruptedException {
NumberCounter counter = new NumberCounter();
counter.startCounter(10000, 5);
}
}

Metrics from multiple threads

So this seems like a pretty common use case, and maybe I'm over thinking it, but I'm having an issue with keeping centralized metrics from multiple threads. Say I have multiple worker threads all processing records and I every 1000 records I want to spit out some metric. Now I could have each thread log individual metrics, but then to get throughput numbers, but I'd have to add them up manually (and of course time boundaries won't be exact). Here's a simple examples:
public class Worker implements Runnable {
private static int count = 0;
private static long processingTime = 0;
public void run() {
while (true) {
...get record
count++;
long start = System.currentTimeMillis();
...do work
long end = System.currentTimeMillis();
processingTime += (end-start);
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
}
}
}
Hope that makes some sense. Also I know the two static variables will probably be AtomicInteger and AtomicLong . . . but maybe not. Interested in what kinds of ideas people have. I had thought about using Atomic variables and using a ReeantrantReadWriteLock - but I really don't want the metrics to stop the processing flow (i.e. the metrics should have very very minimal impact on the processing). Thanks.

Offloading the actual processing to another thread can be a good idea. The idea is to encapsulate your data and hand it off to a processing thread quickly so you minimize impact on the threads that are doing meaningful work.
There is a small handoff contention, but that cost is usually a lot smaller than any other type of synchronization that it should be a good candidate in many situations. I think M. Jessup's solution is pretty close to mine, but hopefully the following code illustrates the point clearly.
public class Worker implements Runnable {
private static final Metrics metrics = new Metrics();
public void run() {
while (true) {
...get record
long start = System.currentTimeMillis();
...do work
long end = System.currentTimeMillis();
// process the metric asynchronously
metrics.addMetric(end - start);
}
}
private static final class Metrics {
// a single "background" thread that actually handles
// processing
private final ExecutorService metricThread =
Executors.newSingleThreadExecutor();
// data (no synchronization needed)
private int count = 0;
private long processingTime = 0;
public void addMetric(final long time) {
metricThread.execute(new Runnable() {
public void run() {
count++;
processingTime += time;
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
}
});
}
}
}

I would suggest if you don't want the logging to interfere with the processing, you should have a separate log worker thread and have your processing threads simply provide some type of value object that can be handed off. In the example I choose a LinkedBlockingQueue since it has the ability to block for an insignificant amount of time using offer() and you can defer the blocking to another thread that pulls the values from a queue. You might need to have increased logic in the MetricProcessor to order data, etc depending on your requirements, but even if it is a long running operation it wont keep the VM thread scheduler from restarting the real processing threads in the mean time.
public class Worker implements Runnable {
public void run() {
while (true) {
... do some stuff
if (count % 1000 == 0) {
... log some metrics
if(MetricProcessor.getInstance().addMetrics(
new Metrics(processingTime, count, ...)) {
processingTime = 0;
count = 0;
} else {
//the call would have blocked for a more significant
//amount of time, here the results
//could be abandoned or just held and attempted again
//as a larger data set later
}
}
}
}
}
public class WorkerMetrics {
...some interesting data
public WorkerMetrics(... data){
...
}
...getter setters etc
}
public class MetricProcessor implements Runnable {
LinkedBlockingQueue metrics = new LinkedBlockingQueue();
public boolean addMetrics(WorkerMetrics m) {
return metrics.offer(m); //This may block, but not for a significant amount of time.
}
public void run() {
while(true) {
WorkMetrics m = metrics.take(); //wait here for something to come in
//the above call does all the significant blocking without
//interrupting the real processing
...do some actual logging, aggregation, etc of the metrics
}
}
}

If you depend on the state of count and the state of processingTime to be in synch then you would have to be using a Lock. For example if when ++count % 1000 == 0 is true, you want to evaluate the metrics of processingTime at THAT time.
For that case, it would make sense to use a ReentrantLock. I wouldn't use a RRWL because there isn't really an instance where a pure read is occuring. It is always a read/write set. But you would need to Lock around all of
count++
processingTime += (end-start);
if (count % 1000 == 0) {
... log some metrics
processingTime = 0;
count = 0;
}
Whether or not count++ is going to be at that location, you will need to lock around that also.
Finally if you are using a Lock, you do not need an AtomicLong and AtomicInteger. It just adds to the overhead and isn't more thread-safe.

Counting the number of games played in 1 second using Java Threads

I want to know how many games my computer can play in 1000 ms. I did the tests before without using Threads (it plays 13k). Now that I think I'm using threads, I still get the same. Since I don't have much experience with Java threads, I assume I'm doing something wrong but I just can't get it.
Thanks in advance
public class SpeedTest<T extends BoardGame> implements Runnable
{
public static int gamesPlayed = 0;
private ElapsedTimer timer;
private double maxTime;
private BoardAgent<T> agent;
private BoardGame<T> game;
public SpeedTest(BoardGame<T> game, ElapsedTimer timer, double maxTime, Random rng)
{
this.game = game;
this.timer = timer;
this.maxTime = maxTime;
this.agent = new RandomAgent<T>(rng);
}
#Override
public void run()
{
while (true)
{
BoardGame<T> newBoard = game.copy();
while (!newBoard.isGameOver())
newBoard.makeMove(agent.move(newBoard));
gamesPlayed++;
if (timer.elapsedMilliseconds() > maxTime) {
break;
}
}
}
public static void main(String[] args)
{
Random rng = new Random();
BoardGame<Connect4> game = new Connect4(6, 7);
double maxTime = 1000;
ElapsedTimer timer = new ElapsedTimer();
SpeedTest<Connect4> speedTest1 = new SpeedTest<Connect4>(game, timer, maxTime, rng);
SpeedTest<Connect4> speedTest2 = new SpeedTest<Connect4>(game, timer, maxTime, rng);
Thread t1 = new Thread(speedTest1);
Thread t2 = new Thread(speedTest2);
t1.start();
t2.start();
try {
Thread.sleep((long) maxTime);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Games: " + SpeedTest.gamesPlayed);
}
}

I suspect that the reason that you are not seeing any speedup is that your application is only using 1 physical processor. If it is only using one processor, then the two threads won't be running in parallel. Instead, the processor will be "time-slicing" between the two threads.
What can you do about this?
Run on a dual-core etc processor. Or if you have a single processor machine with HT support, enable HT.
Run the test over a longer time; e.g. a number of minutes.
The reason I suggest the latter is that this could be a JVM warmup effect. When a JVM starts a new application, it needs to do a lot of class loading and JIT compilation behind the scenes. These tasks will be largely (if not totally) single-threaded. Running the tests over a longer period of time reduces the contribution of the "warm up" overheads to the average time per "game".
There is a fix that you ought to make to make the program thread-safe. Change
public static int gamesPlayed = 0;
to
private static final AtomicInteger gamesPlayed = new AtomicInteger();
and then use getAndIncrement() to increment the counter and intValue() to fetch its value. (This is simpler than having each thread maintain its own counter and summing them at the end.)
However, I strongly suspect that this change (or #Erik's alternative) will make little difference to the results you are seeing. I'm now sure it is either:
JVM warmup issue as described above,
a consequence of high object creation rates and/or heap starvation, or
some hidden synchronization issue between the instances of your game.

Don't use a static int, use a normal member int.
Instead of the sleep, call .join on both threads.
Then finally add the member ints.