We Keep Coding

Returning a value from thread

First of all, yes I looked up this question on google and I did not find any answer to it. There are only answers, where the thread is FINISHED and than the value is returned. What I want, is to return an "infinite" amount of values.
Just to make it more clear for you: My thread is reading messages from a socket and never really finishes. So whenever a new message comes in, I want another class to get this message. How would I do that?
public void run(){
while(ircMessage != null){
ircMessage = in.readLine();
System.out.println(ircMessage);
if (ircMessage.contains("PRIVMSG")){
String[] ViewerNameRawRaw;
ViewerNameRawRaw = ircMessage.split("#");
String ViewerNameRaw = ViewerNameRawRaw[2];
String[] ViewerNameR = ViewerNameRaw.split(".tmi.twitch.tv");
viewerName = ViewerNameR[0];
String[] ViewerMessageRawRawRaw = ircMessage.split("PRIVMSG");
String ViewerMessageRawRaw = ViewerMessageRawRawRaw[1];
String ViewerMessageRaw[] = ViewerMessageRawRaw.split(":", 2);
viewerMessage = ViewerMessageRaw[1];
}
}
}

What you are describing is a typical scenario of asynchronous communication. Usually solution could be implemented with Queue. Your Thread is a producer. Each time your thread reads a message from socket it builds its result and sends it into a queue. Any Entity that is interested to receive the result should be listening to the Queue (i.e. be a consumer). Read more about queues as you can send your message so that only one consumer will get it or (publishing) means that all registered consumers may get it. Queue implementation could be a comercialy available products such as Rabbit MQ for example or as simple as Java provided classes that can work as in memory queues. (See Queue interface and its various implementations). Another way to go about it is communication over web (HTTP). Your thread reads a message from a socket, builds a result and sends it over http using let's say a REST protocol to a consumer that exposes a rest API that your thread can call to.

Why not have a status variable in your thread class? You can then update this during execution and before exiting. Once the thread has completed, you can still query the status.
public static void main(String[] args) throws InterruptedException {
threading th = new threading();
System.out.println("before run Status:" + th.getStatus());
th.start();
Thread.sleep(500);
System.out.println("running Status:" + th.getStatus());
while(th.isAlive()) {}
System.out.println("after run Status:" + th.getStatus());
}
Extend thread to be:
public class threading extends Thread {
private int status = -1; //not started
private void setStatus(int status){
this.status = status;
}
public void run(){
setStatus(1);//running
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
setStatus(0); //exit clean
}
public int getStatus(){
return this.status;
}
}
And get an output of:
before run Status:-1
running Status:1
after run Status:0

Synchronize on DataOutputStream

I have gone through so many tutorials on Synchronization now that my head is spinning. I have never truly understood it :(.
I have a Java server(MainServer), that when a client connects creates a new thread(ServerThread) with a DataOutputStream.
The client talks to the ServerThread and the ServerThread responds. Every now and then the MainServer will distribute a message to all clients utilizing each ServerThread's DataOutputStream object.
I am quite certain that every now and then my issue is because both the MainServer and ServerThread are trying to send something to the client at the same time. Therefore I need to lock on the DataOutputStream object. For the life of me I cannot understand this concept any further. Every example I read is confusing.
What is the correct way to handle this?
ServerThread's send to client method:
public void replyToOne(String reply){
try {
commandOut.writeUTF(reply);
commandOut.flush();
} catch (IOException e) {
logger.fatal("replyToOne", e);
}
logger.info(reply);
}
MainServer's distribute to all clients method:
public static void distribute(String broadcastMessage){
for (Map.Entry<String, Object[]> entry : AccountInfoList.entrySet()) {
Object[] tmpObjArray = entry.getValue();
DataOutputStream temporaryCOut = (DataOutputStream) tmpObjArray[INT_COMMAND_OUT]; //can be grabbed while thread is using it
try {
temporaryCOut.writeUTF(broadcastMessage);
temporaryCOut.flush();
} catch (IOException e) {
logger.error("distribute: writeUTF", e);
}
logger.info(broadcastMessage);
}
}
I am thinking I should have something like this in my ServerThread class.
public synchronized DataOutputStream getCommandOut(){
return commandOut;
}
Is it really that simple? I know this has likely been asked and answered, but I don't seem to be getting it still, without individual help.

If this were me.....
I would have a LinkedBlockingQueue on each client-side thread. Then, each time the client thread has a moment of idleness on the socket, it checks the queue. If there's a message to send from the queue, it sends it.
Then, the server, if it needs to, can just add items to that queue, and, when the connection has some space, it will be sent.
Add the queue, have a method on the ServerThread something like:
addBroadcastMessage(MyData data) {
broadcastQueue.add(data);
}
and then, on the socket side, have a loop that has a timeout-block on it, so that it breaks out of the socket if it is idle, and then just:
while (!broadcastQueue.isEmpty()) {
MyData data = broadcastQueue.poll();
.... send the data....
}
and you're done.
The LinkedBlockingQueue will manage the locking and synchronization for you.

You are on the right track.
Every statement modifying the DataOutputStream should be synchronized on this DataOutputStream so that it is not concurrently accessed (and thus do not have any concurrent modification):
public void replyToOne(String reply){
try {
synchronized(commandOut) { // writing block
commandOut.writeUTF(reply);
commandOut.flush();
}
} catch (IOException e) {
logger.fatal("replyToOne", e);
}
logger.info(reply);
}
And:
public static void distribute(String broadcastMessage){
for (Map.Entry<String, Object[]> entry : AccountInfoList.entrySet()) {
Object[] tmpObjArray = entry.getValue();
DataOutputStream temporaryCOut = (DataOutputStream) tmpObjArray[INT_COMMAND_OUT]; //can be grabbed while thread is using it
try {
synchronized(temporaryCOut) { // writing block
temporaryCOut.writeUTF(broadcastMessage);
temporaryCOut.flush();
}
} catch (IOException e) {
logger.error("distribute: writeUTF", e);
}
logger.info(broadcastMessage);
}
}

Just putting my 2 cents:
The way I implement servers is this:
Each server is a thread with one task only: listening for connections. Once it recognizes a connection it generates a new thread to handle the connection's input/output (I call this sub-class ClientHandler).
The server also keeps a list of all connected clients.
ClientHandlers are responsible for user-server interactions. From here, things are pretty simple:
Disclaimer: there are no try-catches blocks here! add them yourself. Of course you can use thread executers to limit the number of concurrent connections.
Server's run() method:
#Override
public void run(){
isRunning = true;
while(isRunning){
ClientHandler ch = new ClientHandler(serversocket.accept());
clients.add(ch);
ch.start();
}
}
ClientHandler's ctor:
public ClientHandler(Socket client){
out = new ObjectOutputStream(client.getOutputStream());
in = new ObjectInputStream(client.getInputStream());
}
ClientHandler's run() method:
#Override
public void run(){
isConnected = true;
while(isConnected){
handle(in.readObject());
}
}
and handle() method:
private void handle(Object o){
//Your implementation
}
If you want a unified channel say for output then you'll have to synchronize it as instructed to avoid unexpected results.
There are 2 simple ways to do this:
Wrap every call to output in synchronized(this) block
Use a getter for output (like you did) with synchronized keyword.

How to make sure the main thread continues only after child thread has started?

I am trying to write a program that first ssh'es to two different machines, and then executes some http requests against them. This means that in order to be able to execute my http requests the ssh tunnel should be running.
What I have done is that I have two threads, each running the ssh command to one of the boxes:
Thread thread1 = new Thread(new Runnable(){
public void run(){
try{
Process p1 = Runtime.getRuntime().exec("ssh -A -L12345:localhost:54321 firsthost.com");
p1.waitFor();
}catch (Exception e){}
}
}) ;
thread1.start();
Thread thread2 = new Thread(new Runnable(){
public void run(){
try{
Process p2 = Runtime.getRuntime().exec("ssh -A -L12345:localhost:54321 secondhost.com");
p2.waitFor();
}catch (Exception e){}
}
}) ;
thread2.start();
Now the problem is that after starting the threads, they do not always immediately start running, which means that I will send my requests before the connection is made. Is there a simple way (without using locks or mutex) that I can make sure I only return to my main program after the threads are started? (I don't want of course to wait for them to end because they never end. Just run the first command once :) Also if there is a better way to run the processes in the background instead of having these two seperate threads, that would be great too!)
Thanks!

Try adding the -f and -N flags to tell ssh to background itself once the port forwarding is setup. Then you can execute the ssh commands in the main thread and wait until they exit (i.e. background themselves) before continuing.
-f
Requests ssh to go to background just before command execution. This is useful if ssh is going to ask for passwords or passphrases, but the user wants it in the background. This implies -n. The recommended way to start X11 programs at a remote site is with something like ssh -f host xterm.
If the ExitOnForwardFailure configuration option is set to “yes”, then a client started with -f will wait for all remote port forwards to be successfully established before placing itself in the background.
-N
Do not execute a remote command. This is useful for just forwarding ports (protocol version 2 only).
(Unfortunately I'm unable to test this at the moment. Apologies if it doesn't work.)

First of all using threads directly is not encouraged in JDK5 onwards. The well packed Executors will do the job for us. If you have a lower version of java or you want to do this still in this way then, here is one quick solution,
public class Main {
public static void main(String... objs) throws Exception {
final Process1Thread p1Thread = new Process1Thread();
final Process2Thread p2Thread = new Process2Thread();
Thread p1 = new Thread(p1Thread);
Thread p2 = new Thread(p2Thread);
p1.start(); p2.start();
while(true) {
if(p1Thread.isFinishedCommand() && p2Thread.isFinishedCommand()) {
//send commands
System.out.println("sending commands");
break;
}
}
}
interface FinishedCommand {
boolean isFinishedCommand();
}
static class Process1Thread implements Runnable, FinishedCommand {
private boolean finished = false;
#Override
public boolean isFinishedCommand() {
synchronized(this) {
return finished;
}
}
#Override
public void run() {
try {
final Process p2 = Runtime.getRuntime().exec("dir");
synchronized(this) {
finished = true;
}
p2.waitFor();
} catch (Exception e) {
e.printStackTrace();
}
}
}
static class Process2Thread implements Runnable, FinishedCommand {
private boolean finished = false;
#Override
public boolean isFinishedCommand() {
synchronized(this) {
return finished;
}
}
#Override
public void run() {
try {
final Process p2 = Runtime.getRuntime().exec("dir");
synchronized(this) {
finished = true;
}
p2.waitFor();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
Regards
Lyju

You can use thread1.getState() method to get the state of the thread. After you can thread1.start() it should be moved to RUNNABLE state. When thread1 is waiting for the process to finish it will be in BLOCKED state.

Thread Blocks in server mode

This method notifes an event loop to start processing a message. However, if the event loop is already processing a message then, this method blocks until it receives a notification of completed event processing (which is triggered at the end of the event loop).
public void processEvent(EventMessage request) throws Exception {
System.out.println("processEvent");
if (processingEvent) {
synchronized (eventCompleted) {
System.out.println("processEvent: Wait for Event to completed");
eventCompleted.wait();
System.out.println("processEvent: Event completed");
}
}
myRequest = request;
processingEvent = true;
synchronized (eventReady) {
eventReady.notifyAll();
}
}
This works in client mode. If I switch to server mode and the time spent in the event loop processing the message is too quick, then the method above blocks forever waiting for the event to completed. For some reason the event complete notification is sent after the processingEvent check and before the eventCompleted.wait(). It makes no difference if I remove the output statements. I can not repeat the same problem in client mode.
Why does this only happen in server mode and what can I do to prevent this happening?
Here is the eventReady wait and eventCompleted notification:
public void run() {
try {
while (true) {
try {
synchronized (eventReady) {
eventReady.wait();
}
nx.processEvent(myRequest, myResultSet);
if (processingEvent > 0) {
notifyInterface.notifyEventComplete(myRequest);
}
} catch (InterruptedException e) {
throw e;
} catch (Exception e) {
notifyInterface.notifyException(e, myRequest);
} finally {
processingEvent--;
synchronized (eventCompleted) {
eventCompleted.notifyAll();
}
}
} // End of while loop
} catch (InterruptedException Ignore) {
} finally {
me = null;
}
Here is revised code which seems to work without the deadlock problem - which BTW happened in client mode randomely after about 300 events.
private BlockingQueue<EventMessage> queue = new SynchronousQueue<EventMessage>();
public void processEvent(EventMessage request) throws Exception {
System.out.println("processEvent");
queue.put(request);
}
public void run() {
try {
while (true) {
EventMessage request = null;
try {
request = queue.take();
processingEvent = true;
nx.processEvent(request, myResultSet);
notifyInterface.notifyEventComplete(request);
} catch (InterruptedException e) {
throw e;
} catch (Exception e) {
notifyInterface.notifyException(e, request);
} finally {
if (processingEvent) {
synchronized (eventCompleted) {
processingEvent = false;
eventCompleted.notifyAll();
}
}
}
} // End of while loop
} catch (InterruptedException Ignore) {
} finally {
me = null;
}
}

If you call notifyAll and no thread is wait()ing, the notify is lost.
The correct approach is to always change a state, inside the synchronized block, when calling notify() and always check that state, inside the synchronized block, before calling wait().
Also your use of processingEvent doesn't appear to be thread safe.
Can you provide the code which waits on eventReady and notifies eventCompleted?
Your program can happen to work if your speed up or slow down your application just right e.g. if you use -client, but if you use a different machine, JVM or JVM options it can fail.

There are a number of race conditions in your code. Even declaring processingEvent volatile or using an AtomicBoolean won't help. I would recommend using a SynchronousQueue which will block the event until the processer is ready for it. Something like:
private final BlockingQueue<Request> queue = new SynchronousQueue<Request>();
...
// this will block until the processor dequeues it
queue.put(request);
Then the event processor does:
while (!done) {
// this will block until an event is put-ed to the queue
Request request = queue.take();
process the event ...
}
Only one request will be processed at once and all of the synchronization, etc. will be handled by the SynchronousQueue.

If processingEvent isn't declared volatile or accessed from within a synchronized block then updates made by one thread may not become visible to other threads immediately. It's not clear from your code whether this is the case, though.
The "server" VM is optimised for speed (at the expense of startup time and memory usage) which could be the reason why you didn't encounter this problem when using the "client" VM.

There is a race condition in your code that may be exasperated by using the server VM, and if processingEvent is not volatile then perhaps certain optimizations made by the server VM or its environment are further influencing the problem.
The problem with your code (assuming this method is accessed by multiple threads concurrently) is that between your check of processingEvent and eventCompleted.wait(), another thread can already notify and (I assume) set processingEvent to false.
The simplest solution to your blocking problem is to not try to manage it yourself, and just let the JVM do it by using a shared lock (if you only want to process one event at a time). So you could just synchronize the entire method, for instance, and not worry about it.
A second simple solution is to use a SynchronousQueue (this is the type of situation it is designed for) for your event passing; or if you have more executing threads and want more than 1 element in the queue at a time then you can use an ArrayBlockingQueue instead. Eg:
private SynchronousQueue<EventMessage> queue = new SynchronousQueue<EventMessage>();
public void addEvent(EventMessage request) throws Exception
{
System.out.println("Adding event");
queue.put(request);
}
public void processNextEvent()
{
EventMessage request = queue.take();
processMyEvent(request);
}
// Your queue executing thread
public void run()
{
while(!terminated)
{
processNextEvent();
}
}

Where to stop/destroy threads in Android Service class?

I have created a threaded service the following way:
public class TCPClientService extends Service{
...
#Override
public void onCreate() {
...
Measurements = new LinkedList<String>();
enableDataSending();
}
#Override
public IBinder onBind(Intent intent) {
//TODO: Replace with service binding implementation
return null;
}
#Override
public void onLowMemory() {
Measurements.clear();
super.onLowMemory();
}
#Override
public void onDestroy() {
Measurements.clear();
super.onDestroy();
try {
SendDataThread.stop();
} catch(Exception e){
...
}
}
private Runnable backgrounSendData = new Runnable() {
public void run() {
doSendData();
}
};
private void enableDataSending() {
SendDataThread = new Thread(null, backgrounSendData, "send_data");
SendDataThread.start();
}
private void addMeasurementToQueue() {
if(Measurements.size() <= 100) {
String measurement = packData();
Measurements.add(measurement);
}
}
private void doSendData() {
while(true) {
try {
if(Measurements.isEmpty()) {
Thread.sleep(1000);
continue;
}
//Log.d("TCP", "C: Connecting...");
Socket socket = new Socket();
socket.setTcpNoDelay(true);
socket.connect(new InetSocketAddress(serverAddress, portNumber), 3000);
//socket.connect(new InetSocketAddress(serverAddress, portNumber));
if(!socket.isConnected()) {
throw new Exception("Server Unavailable!");
}
try {
//Log.d("TCP", "C: Sending: '" + message + "'");
PrintWriter out = new PrintWriter( new BufferedWriter( new OutputStreamWriter(socket.getOutputStream())),true);
String message = Measurements.remove();
out.println(message);
Thread.sleep(200);
Log.d("TCP", "C: Sent.");
Log.d("TCP", "C: Done.");
connectionAvailable = true;
} catch(Exception e) {
Log.e("TCP", "S: Error", e);
connectionAvailable = false;
} finally {
socket.close();
announceNetworkAvailability(connectionAvailable);
}
} catch (Exception e) {
Log.e("TCP", "C: Error", e);
connectionAvailable = false;
announceNetworkAvailability(connectionAvailable);
}
}
}
...
}
After I close the application the phone works really slow and I guess it is due to thread termination failure.
Does anyone know what is the best way to terminate all threads before terminating the application?

Addendum: The Android framework provides many helpers for one-off work, background work, etc, which may be preferable over trying to roll your own thread in many instances. As mentioned in a below post, AsyncTask is a good starting point to look into. I encourage readers to look into the framework provisions first before even beginning to think about doing their own threading.
There are several problems in the code sample you posted I will address in order:
1) Thread.stop() has been deprecated for quite some time now, as it can leave dependent variables in inconsistent states in some circumstances. See this Sun answer page for more details (Edit: that link is now dead, see this page for why not to use Thread.stop()). A preferred method of stopping and starting a thread is as follows (assuming your thread will run somewhat indefinitely):
private volatile Thread runner;
public synchronized void startThread(){
if(runner == null){
runner = new Thread(this);
runner.start();
}
}
public synchronized void stopThread(){
if(runner != null){
Thread moribund = runner;
runner = null;
moribund.interrupt();
}
}
public void run(){
while(Thread.currentThread() == runner){
//do stuff which can be interrupted if necessary
}
}
This is just one example of how to stop a thread, but the takeaway is that you are responsible for exiting a thread just as you would any other method. Maintain a method of cross thread communcation (in this case a volatile variable, could also be through a mutex, etc) and within your thread logic, use that method of communication to check if you should early exit, cleanup, etc.
2) Your measurements list is accessed by multiple threads (the event thread and your user thread) at the same time without any synchronization. It looks like you don't have to roll your own synchronization, you can use a BlockingQueue.
3) You are creating a new Socket every iteration of your sending Thread. This is a rather heavyweight operation, and only really make sense if you expect measurements to be extremely infrequent (say one an hour or less). Either you want a persistent socket that is not recreated every loop of the thread, or you want a one shot runnable you can 'fire and forget' which creates a socket, sends all relevant data, and finishes. (A quick note about using a persistent Socket, socket methods which block, such as reading, cannot be interrupted by Thread.interrupt(), and so when you want to stop the thread, you must close the socket as well as calling interrupt)
4) There is little point in throwing your own exceptions from within a Thread unless you expect to catch it somewhere else. A better solution is to log the error and if it is irrecoverable, stop the thread. A thread can stop itself with code like (in the same context as above):
public void run(){
while(Thread.currentThread() == runner){
//do stuff which can be interrupted if necessary
if(/*fatal error*/){
stopThread();
return; //optional in this case since the loop will exit anyways
}
}
}
Finally, if you want to be sure a thread exits with the rest of your application, no matter what, a good technique is to call Thread.setDaemon(true) after creation and before you start the thread. This flags the thread as a daemon thread, meaning the VM will ensure that it is automatically destroyed if there are no non-daemon threads running (such as if your app quits).
Obeying best practices with regards to Threads should ensure that your app doesn't hang or slow down the phone, though they can be quite complex :)

Actually, you don't need the "runner" variable as described above, something like:
while (!interrupted()) {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
break;
}
}
But generally, sitting in a Thread.sleep() loop is a really bad idea.
Look at the AsyncTask API in the new 1.5 API. It will probably solve your problem more elegantly than using a service. Your phone is getting slow because the service never shuts down - there's nothing that will cause the service to kill itself.