We Keep Coding

Send data to serial port when data is available

I'm building an interactive LED table with a 14x14 matrix consisting of addressable LED strips for an university assignment. Those are being controlled by 2 arduinos that get the data about which LED should have which RGB value from a Pi running a server that runs several games which should be playable on the LED table. To control the games I send respective int codes from an android app to the server running on the Raspi.
The serial communication is realized by using jSerialComm. The problem I'm facing is, that I don't want to permanently send data over the serial port but only at the moment, when a new array that specifies the matrix is available.
Therefore I don't want to be busy waiting and permanently checking if the matrix got updated not do I want to check for a update with
while(!matrixUpdated) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {}
}
So what I've been trying was running a while(true) in which I call wait(), so the thread stops until I wake the thread up by calling notify when an updated matrix is available.
My run() method in the serial thread looks like this at the moment:
#Override
public void run() {
arduino1.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 0);
arduino2.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 0);
try {
Thread.sleep(100);
} catch (Exception e) {}
PrintWriter outToArduino1 = new PrintWriter(arduino1.getOutputStream());
PrintWriter outToArduino2 = new PrintWriter(arduino2.getOutputStream());
while(true) {
try {
wait();
} catch (InterruptedException e) {}
System.out.println("Matrix received");
outToArduino1.print(matrix);
outToArduino2.print(matrix);
}
}
I wake the thread up by this method which is nested in the same class:
public void setMatrix(int[][][] pixelIdentifier) {
matrix = pixelIdentifier;
notify();
}
I also tried notifyAll() which didn't change the outcome.
In one of the games (Tic Tac Toe) I call this method after every game turn to update and send the matrix to the arduinos:
private void promptToMatrix() {
synchronized (GameCenter.serialConnection) {
GameCenter.serialConnection.setMatrix(matrix);
}
}
I previously called it without using the synchronized block but as I've been reading through many articles on that topic on StackOverflow I have read that one should use synchronized for this. Further I have also read that using wait() and notify() is not recommended, however as the assignment needs to get done quite quickly I don't know if any other approach makes sense as I don't want to restructure my whole application as I run up to 5 threads when a game is being played (due to threads for communication and so on).
If there is a possibility to solve this using wait() and notify() I would be really grateful to hear how that would be done, as I have not been able to really comprehend how working properly with the synchronized block is being done and so on.
However if such a solution is not possible or would also end in restructuring the whole application I'm also open to different suggestions. Pointing out that using wait() and notify() is not recommended however doesn't help me, as I've already read that often enough, I'm aware of that but prefer to use it in that case if possible!!!
EDIT:
The application executes like this:
Main Thread
|--> SerialCommunication Thread --> waiting for updated data
|--> NetworkController Thread
|--> Client Thread --> interacting with the game thread
|--> Game Thread --> sending updated data to the waiting SerialCommunication Thread
Really appreciate any help and thanks in advance for your time!

You are dealing with asynchronous update possibly running on different threads, the best match in my opinion is using RxJava.
You could use a Subject to receive matrix event and then subscribe to it to update the leds.
You can write something like this (don't take it too literally).
public static void main(String[] args) {
int[][] initialValue = new int[32][32];
BehaviorSubject<int[][]> matrixSubject = BehaviorSubject.createDefault(initialValue);
SerialPort arduino1 = initSerial("COM1");
SerialPort arduino2 = initSerial("COM2");;
PrintWriter outToArduino1 = new PrintWriter(arduino1.getOutputStream());
PrintWriter outToArduino2 = new PrintWriter(arduino2.getOutputStream());
Observable<String> serializedMatrix = matrixSubject.map(Sample::toChars);
serializedMatrix.observeOn(Schedulers.io()).subscribe(mat -> {
// Will run on a newly created thread
outToArduino1.println(mat);
});
serializedMatrix.observeOn(Schedulers.io()).subscribe(mat -> {
// Will run on a newly created thread
outToArduino2.println(mat);
});
// Wait forever
while(true) {
try {
// get your matrix somehow ...
// then publish it on your subject
// your subscribers will receive the data and use it.
matrixSubject.onNext(matrix);
Thread.sleep(100);
} catch (InterruptedException e) {
// SWALLOW error
}
}
}
public static String toChars(int[][] data) {
// Serialize data
return null;
}
There are may operators that you could use to make it do what you need, also you can use different schedulers to choose from different thread policies.
You can also transform your input in the subject you publish, an observable or a subject can be created directly from your input.

Java start multiple threads in a class

I am consuming from a certain source (say Kafka) and periodically dumping the collected messages (to, say, S3). My class definition is as follows:
public class ConsumeAndDump {
private List<String> messages;
public ConsumeAndDump(){
messages = new ArrayList<>();
// initialize required resources
}
public void consume(){
// this runs continuously and keeps consuming from the source.
while(true){
final String message = ...// consume from Kafka
messages.add(message);
}
}
public void dump(){
while(true){
final String allMessages = String.join("\n", messages);
messages.clear(); // shown here simply, but i am synchronising this to avoid race conditions
// dump to destination (file, or S3, or whatever)
TimeUnit.SECONDS.sleep(60); // sleep for a minute
}
}
public void run() {
// This is where I don't know how to proceed.
// How do I start consume() and dump() as separate threads?
// Is it even possible in Java?
// start consume() as thread
// start dump() as thread
// wait for those to finish
}
}
I want to have two threads - consume and dump. consume should run continuously whereas dump wakes up periodically, dumps the messages, clears the buffer and then goes back to sleep again.
I am having trouble starting consume() and dump() as threads. Honestly, I don't know how to do that. Can we even run member methods as threads? Or do I have to make separate Runnable classes for consume and dump? If so, how would I share messages between those?

First of all, you can't really use ArrayList for this. ArrayList is not thread-safe. Check out BlockingQueue for example. You will have to deal with things like back pressure. Don't use an unbounded queue.
Starting a thread is pretty simple, you can use lambdas for it.
public void run() {
new Thread(this::consume).start();
new Thread(this::produce).start();
}
Should work, but gives you little to no control over when those processes should end.

is it safe to store threads in a ConcurrentMap?

I am building a backend service whereby a REST call to my service creates a new thread. The thread waits for another REST call if it does not receive anything by say 5 minutes the thread will die.
To keep track of all the threads I have a collection that keeps track of all the currently running threads so that when the REST call finally comes in such as a user accepting or declining an action, I can then identify that thread using the userID. If its declined we will just remove that thread from the collection if its accepted the thread can carry on doing the next action. i have implemented this using a ConcurrentMap to avoid concurrency issues.
Since this is my first time working with threads I want to make sure that I am not overlooking any issues that may arise. Please have a look at my code and tell me if I could do it better or if there's any flaws.
public class UserAction extends Thread {
int userID;
boolean isAccepted = false;
boolean isDeclined = false;
long timeNow = System.currentTimeMillis();
long timeElapsed = timeNow + 50000;
public UserAction(int userID) {
this.userID = userID;
}
public void declineJob() {
this.isDeclined = true;
}
public void acceptJob() {
this.isAccepted = true;
}
public boolean waitForApproval(){
while (System.currentTimeMillis() < timeElapsed){
System.out.println("waiting for approval");
if (isAccepted) {
return true;
} else if (declined) {
return false;
}
}
return isAccepted;
}
#Override
public void run() {
if (!waitForApproval) {
// mustve timed out or user declined so remove from list and return thread immediately
tCollection.remove(userID);
// end the thread here
return;
}
// mustve been accepted so continue working
}
}
public class Controller {
public static ConcurrentHashMap<Integer, Thread> tCollection = new ConcurrentHashMap<>();
public static void main(String[] args) {
int barberID1 = 1;
int barberID2 = 2;
tCollection.put(barberID1, new UserAction(barberID1));
tCollection.put(barberID2, new UserAction(barberID2));
tCollection.get(barberID1).start();
tCollection.get(barberID2).start();
Thread.sleep(1000);
// simulate REST call accepting/declining job after 1 second. Usually this would be in a spring mvc RESTcontroller in a different class.
tCollection.get(barberID1).acceptJob();
tCollection.get(barberID2).declineJob();
}
}

You don't need (explicit) threads for this. Just a shared pool of task objects that are created on the first rest call.
When the second rest call comes, you already have a thread to use (the one that's handling the rest call). You just need to retrieve the task object according to the user id. You also need to get rid of expired tasks, which can be done with for example a DelayQueue.
Pseudocode:
public void rest1(User u) {
UserTask ut = new UserTask(u);
pool.put(u.getId(), ut);
delayPool.put(ut); // Assuming UserTask implements Delayed with a 5 minute delay
}
public void rest2(User u, Action a) {
UserTask ut = pool.get(u.getId());
if(!a.isAccepted() || ut == null)
pool.remove(u.getId());
else
process(ut);
// Clean up the pool from any expired tasks, can also be done in the beginning
// of the method, if you want to make sure that expired actions aren't performed
while((UserTask u = delayPool.poll()) != null)
pool.remove(u.getId());
}

There's a synchronization issue that you should make your flags isAccepted and isDeclined of class AtomicBoolean.
A critical concept is that you need to take steps to make sure changes to memory in one thread are communicated to other threads that need that data. They're called memory fences and they often occur implicitly between synchronization calls.
The idea of a (simple) Von Neumann architecture with a 'central memory' is false for most modern machines and you need to know data is being shared between caches/threads correctly.
Also as others suggest, creating a thread for each task is a poor model. It scales badly and leaves your application vulnerable to keeling over if too many tasks are submitted. There is some limit to memory so you can only have so many pending tasks at a time but the ceiling for threads will be much lower.
That will be made all the worse because you're spin waiting. Spin waiting puts a thread into a loop waiting for a condition. A better model would wait on a ConditionVariable so threads not doing anything (other than waiting) could be suspended by the operating system until notified that the thing they're waiting for is (or may be) ready.
There are often significant overheads in time and resources to creating and destroying threads. Given that most platforms can be simultaneously only executing a relatively small number of threads creating lots of 'expensive' threads to have them spend most of their time swapped out (suspended) doing nothing is very inefficient.
The right model launches a pool of a fixed number of threads (or relatively fixed number) and places tasks in a shared queue that the threads 'take' work from and process.
That model is known generically as a "Thread Pool".
The entry level implementation you should look at is ThreadPoolExecutor:
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html

Rejection handler in Executors.newScheduledThreadPool

I have a ArrayBlocking queue, , upon which a single thread fixed rate Scheduled works.
I may have failed task. I want re-run that or re-insert in queue at high priority or top level

Some thoughts here -
Why are you using ArrayBlockingQueue and not PriorityBlockingQueue ? Sounds like what you need to me . At first set all your elements to be with equal priority.
In case you receive an exception - re-insert to the queue with a higher priority

Simplest thing might be a priority queue. Attach a retry number to the task. It starts as zero. After an unsuccessful run, throw away all the ones and increment the zeroes and put them back in the queue at a high priority. With this method, you can easily decide to run everything three times, or more, if you want to later. The down side is you have to modify the task class.
The other idea would be to set up another, non-blocking, thread-safe, high-priority queue. When looking for a new task, you check the non-blocking queue first and run what's there. Otherwise, go to the blocking queue. This might work for you as is, and so far it's the simplest solution. The problem is the high priority queue might fill up while the scheduler is blocked on the blocking queue.
To get around this, you'd have to do your own blocking. Both queues should be non-blocking. (Suggestion: java.util.concurrent.ConcurrentLinkedQueue.) After polling both queues with no results, wait() on a monitor. When anything puts something in a queue, it should call notifyAll() and the scheduler can start up again. Great care is needed lest the notification occur after the scheduler has checked both queues but before it calls wait().
Addition:
Prototype code for third solution with manual blocking. Some threading is suggested, but the reader will know his/her own situation best. Which bits of code are apt to block waiting for a lock, which are apt to tie up their thread (and core) for minutes while doing extensive work, and which cannot afford to sit around waiting for the other code to finish all needs to be considered. For instance, if a failed run can immediately be rerun on the same thread with no time-consuming cleanup, most of this code can be junked.
private final ConcurrentLinkedQueue mainQueue = new ConcurrentLinkedQueue();
private final ConcurrentLinkedQueue prioQueue = new ConcurrentLinkedQueue();
private final Object entryWatch = new Object();
/** Adds a new job to the queue. */
public void addjob( Runnable runjob ) {
synchronized (entryWatch) { entryWatch.notifyAll(); }
}
/** The endless loop that does the work. */
public void schedule() {
for (;;) {
Runnable run = getOne(); // Avoids lock if successful.
if (run == null) {
// Both queues are empty.
synchronized (entryWatch) {
// Need to check again. Someone might have added and notifiedAll
// since last check. From this point until, wait, we can be sure
// entryWatch is not notified.
run = getOne();
if (run == null) {
// Both queues are REALLY empty.
try { entryWatch.wait(); }
catch (InterruptedException ie) {}
}
}
}
runit( run );
}
}
/** Helper method for the endless loop. */
private Runnable getOne() {
Runnable run = (Runnable) prioQueue.poll();
if (run != null) return run;
return (Runnable) mainQueue.poll();
}
/** Runs a new job. */
public void runit( final Runnable runjob ) {
// Do everthing in another thread. (Optional)
new Thread() {
#Override public void run() {
// Run run. (Possibly in own thread?)
// (Perhaps best in thread from a thread pool.)
runjob.run();
// Handle failure (runit only, NOT in runitLast).
// Defining "failure" left as exercise for reader.
if (failure) {
// Put code here to handle failure.
// Put back in queue.
prioQueue.add( runjob );
synchronized (entryWatch) { entryWatch.notifyAll(); }
}
}
}.start();
}
/** Reruns a job. */
public void runitLast( final Runnable runjob ) {
// Same code as "runit", but don't put "runjob" in "prioQueue" on failure.
}

Have threads run indefinitely in a java application

I am trying to program a game in which I have a Table class and each person sitting at the table is a separate thread. The game involves the people passing tokens around and then stopping when the party chime sounds.
how do i program the run() method so that once I start the person threads, they do not die and are alive until the end of the game
One solution that I tried was having a while (true) {} loop in the run() method but that increases my CPU utilization to around 60-70 percent. Is there a better method?

While yes, you need a loop (while is only one way, but it is simplest) you also need to put something inside the loop that waits for things to happen and responds to them. You're aiming to have something like this pseudocode:
loop {
event = WaitForEvent();
RespondToEvent(event);
} until done;
OK, that's the view from 40,000 feet (where everything looks like ants!) but it's still the core of what you want. Oh, and you also need something to fire off the first event that starts the game, obviously.
So, the key then becomes the definition of WaitForEvent(). The classic there is to use a queue to hold the events, and to make blocking reads from the queue so that things wait until something else puts an event in the queue. This is really a Concurrency-101 data-structure, but an ArrayBlockingQueue is already defined correctly and so is what I'd use in my first implementation. You'll probably want to hide its use inside a subclass of Thread, perhaps like this:
public abstract class EventHandlingThread<Event> extends Thread {
private ArrayBlockingQueue<Event> queue = new ArrayBlockingQueue<Event>();
private boolean done;
protected abstract void respondToEvent(Event event);
public final void postEvent(Event event) throws InterruptedException {
queue.put(event);
}
protected final void done() {
done = true;
}
public final void run() {
try {
while (!done) {
respondToEvent(queue.take());
}
} catch (InterruptedException e) {
// Maybe log this, maybe not...
} catch (RuntimeException e) {
// Probably should log this!
}
}
}
Subclass that for each of your tasks and you should be able to get going nicely. The postEvent() method is called by other threads to send messages in, and you call done() on yourself when you've decided enough is enough. You should also make sure that you've always got some event that can be sent in which terminates things so that you can quit the game…

I would look into Locks and Conditions. This way you can write code that waits for a certain condition to happen. This won't take a lot of CPU power and is even much more efficient and better performing than sleeping .

To make a thread run for an infinite time:
final Object obj = new Object();
try {
Thread th = new Thread(new Runnable() {
public void run() {
synchronized(obj) {
try {
System.out.println("Waiting");
obj.wait();
System.out.println("Done waiting");
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
});
th.start();
System.out.println("Waiting to join.");
// Dont notify; but wait for joining. This will ensure that main thread is running always.
th.join();
System.out.println("End of the Program");
} catch(Exception ex) {
ex.printStackTrace();
}

You may add Thread.sleep() with appropriate time to minimize useless loop iterations.
Another solution is using synchronization. While threads are not required to do anything, they enter into a sleeping state on a monitor using the wait() method, and then when the turn comes, required thread is woken up by the notify() method.

Actor model seems suitable for this scenario. Each person sitting on the table and the table itself can be modelled as actors and the event of passing the tokens and starting and stopping of the game can be modelled as messages to be passed between the actors.
As a bonus, by modelling the scenario as actors you get rid of explicit manipulation of threads, synchronization and locking.
On JVM I will prefer using Scala for modelling actors. For Java you can use libraries like Kilim. See this post for a comparison of Actor model related libraries in Java.

One Way is to use while loop but keep a check i.e
while(true){
if(condition!=true){
Thread.sleep(time);
}else{
break;
}
}
This way if your condition says game is not over it will keep person thread at sleep and memory consumption will be very low.

You should test for a condition in the while loop:
while (!gameOver)
{
do_intersting_stuff();
}
Heavy CPU load is typical for busy wait. Is your loop actually just checking a flag over and over, like
while (!gameOver)
{
if (actionNeeded)
{
do_something();
}
}
you might change to another notification system to sleep and wake up, as this just burns CPU time for nothing.