We Keep Coding

Non deterministic behaviour of socket based application

I'm working on Client-Server application and I'v hit the wall with one issue. I got ServerWorker that is responsible for one connected client, it creates 2 threads, 1 to listen for incoming data from this client and 1 to send data to him.
class ServerWorker {
private DataProcessor dataProcessor;
private ObjectInputStream inputStream;
private ObjectOutputStream outputStream;
private Thread receiverThread;
private Thread senderThread;
private Optional<DataPacket> dataToSend;
private ServerWorker(Socket socket) {
try {
dataToSend = Optional.empty();
dataProcessor = new DataProcessor();
receiverThread = new Thread(this::readAndProcessData);
senderThread = new Thread(this::sendData);
inputStream = new ObjectInputStream(socket.getInputStream());
outputStream = new ObjectOutputStream(socket.getOutputStream());
} catch (IOException e) {
//TODO
}
}
static ServerWorker create(Socket socket) {
return new ServerWorker(socket);
}
void start() {
receiverThread.start();
senderThread.start();
}
void stop() {
receiverThread.interrupt();
senderThread.interrupt();
}
private void readAndProcessData() {
DataPacket dataPacket;
try {
while((dataPacket = (DataPacket)inputStream.readObject()) != null) {
System.out.println("incoming message: " + dataPacket.getContent());
dataToSend = Optional.of(dataProcessor.process(dataPacket));
}
} catch (ClassNotFoundException | IOException e) {
//TODO
}
}
private void sendData() {
while(true) { //TODO
dataToSend.ifPresent(data -> {
try {
outputStream.writeObject(data);
outputStream.flush();
dataToSend = Optional.empty();
} catch (IOException e) {
//TODO
}
});
}
}
}
And DataProcessor is just a small class for now
public class DataProcessor {
public DataPacket process(DataPacket packet){
packet.setContent(packet.getContent().toUpperCase());
return packet;
}
}
and ofcourse, DataPacket which is the same for both client and server
public class DataPacket implements Serializable {
private String content;
public String getContent() {
return content;
}
public void setContent(String content) {
this.content = content;
}
}
DataProcessor and DataPackets are just kind of POCs now, this will grow into much larger and more complicated classes, long story short, ServerWorker will recieve data and pass it to process, then after some logic is done, returning data will be stored inside dataToSend variable and removed after sending. Problem is, code I'v posted above works only sometimes. 90% of the time when I run my server app and client one (code below) nothing happens, uppercased "hello world" isnt going back to client. What's funny, when I run my server in debug mode (even without any breakpoints!), it works... Any ideas what the heck went wrong?
public static void main(String... args) throws Exception {
Socket socket = new Socket("localhost", 9999);
DataPacket dataPacket = new DataPacket();
dataPacket.setContent("hello world");
ObjectOutputStream os = new ObjectOutputStream(socket.getOutputStream());
os.writeObject(dataPacket);
os.flush();
ObjectInputStream inputStream = new ObjectInputStream(socket.getInputStream());
while((dataPacket = (DataPacket)inputStream.readObject()) != null) {
System.out.println(dataPacket.getContent());
}
}
edit#
adding one more class, ConnectionDispatcher that is responsible for creating ServerWorker objects
class ConnectionDispatcher implements Runnable {
private ServerSocket serverSocket;
private List<ServerWorker> serverWorkers;
private volatile boolean isReceiving;
private ConnectionDispatcher(int port) throws IOException {
serverSocket = new ServerSocket(port);
serverWorkers = new ArrayList<>();
isReceiving = false;
}
static ConnectionDispatcher create(int port) throws IOException {
return new ConnectionDispatcher(port);
}
#Override
public void run() {
isReceiving = true;
while(isReceiving) {
acceptIncomingConnections();
}
}
private void acceptIncomingConnections() {
try {
ServerWorker worker = ServerWorker.create(serverSocket.accept());
serverWorkers.add(worker);
worker.start();
} catch (IOException e) {
e.printStackTrace();
}
}
}

It seems that you're modifying dataToSend from one thread while simultaneously reading its value in another. This isn't thread-safe, and the thread that's reading its value may never see its updated value set by the other thread. For this reason, I'd declare dataToSend as volatile.
private volatile Optional<DataPacket> dataToSend;
I have not yet had the chance to test this out myself, but I can in about an hour (assuming this change doesn't fix your problem).

You could uses a Array Blocking Queue, to simulate a producer and consumer pattern.
Let the receiver thread, put new DataPacket into the queue, and let the sender take from the queue and process it and send it.
This will eliminate threading issues and acts as a buffer.
With your current code, you might loose packets,when they arrive at higher rate.
And i agree with user930, private Optional<DataPacket> dataToSend; should be volatile.
Also you can make your code much scalable with JavaNIO, you could look into Apache Mina project.

Multithreading Thread restart

i'am writing a small app to remote control a (Behringer x32) Mixing console. And i got a problem with the communication.
I'am sending data from the pc (app) to the console (port 10023 UDP Protocol), then the console answers to the port from the pc has send data, (random port).
So i have 2 Threads one for sending data, and one for listening for data from the console..... so every time i send data to the console, i need to change the listening port... so i have to kill the listening thread and start it new.
But after some time a have and the app has about x1000 threads open.
How can i restart the Thread or update the listening port without create a new thread?
here's the code for this section, the whole files are # gihub
the listening thread class:
public class Receiver implements Runnable {
private List<IReceiverListener> listeners;
private final static int PACKETSIZE = 48;
private int port;
public Receiver() {
listeners = new ArrayList();
}
public void addReceiverListener(IReceiverListener listener) {
listeners.add(listener);
}
private void update(String data, String adress) {
for (IReceiverListener listener : listeners) {
listener.receiveConsoleData(data, adress);
if (data.indexOf("active") > -1) {
listener.incrementWatchDog();
}
}
}
#Override
public void run() {
try {
// Convert the argument to ensure that is it valid
// Construct the socket
while (true) {
//System.out.println("Listen on Port:" + this.port);
DatagramSocket socket = new DatagramSocket(this.port);
// Create a packet
DatagramPacket packet = new DatagramPacket(new byte[PACKETSIZE], PACKETSIZE);
// Receive a packet (blocking)
socket.receive(packet);
// Print the packet
update(new String(packet.getData()), packet.getAddress().toString());
//logger.addLogData(new String(packet.getData())+" "+packet.getAddress().toString());
// Return the packet to the sender
socket.close();
}
} catch (IOException e) {
}
}
public void setPort(int port) {
this.port = port;
}
public int getPort() {
return port;
}
}
and here my port updateFunction
#Override
public void updatePort(int port) {
receiverThread.interrupt();
receiverThread = null;
receiver.setPort(port);
receiverThread = new Thread(receiver);
receiverThread.start();
}
and the sending thread does this, when it sends data:
listener.updatePort(dsocket.getLocalPort());

This is actually not a threading problem. The problem is, that the receiver thread is stuck in the receive method, so it cannot react to the changed port. However, calling the method DatagramSocket#close from another thread releases the blocking receiver thread with a SocketException.
Thus, you can solve this by closing the currently receiving socket when the receiving port was changed. The receiving thread can now catch the SocketException and create a new DatagramSocket that listens on the new port.
There is no need to kill and recreate threads.
First you put the socket into a field. This allows you to access it from another thread, so you can call the socket.close() method. Second, you put another try-catch block into the while(true) loop, which only catches SocketException.
Something like this might work fine:
public class Receiver implements Runnable {
private static final int PACKETSIZE = 48;
private final ConcurrentLinkedQueue<IReceiverListener> listeners = new ConcurrentLinkedQueue<>();
private volatile DatagramSocket socket;
private volatile int port;
public Receiver(int port) {
this.port = port;
}
public void addReceiverListener(IReceiverListener listener) {
listeners.add(listener);
}
public void updatePort(int port) {
this.port = port;
DatagramSocket socket = this.socket;
if (socket != null) {
socket.close();
}
}
#Override
public void run() {
try {
while (true) {
receiveLoop(new DatagramSocket(port));
}
} catch (IOException e) {
// handle error
}
}
private void receiveLoop(DatagramSocket newSocket) throws IOException {
try (DatagramSocket socket = newSocket) {
this.socket = newSocket;
while (true) {
DatagramPacket packet = new DatagramPacket(new byte[PACKETSIZE], PACKETSIZE);
socket.receive(packet);
process(packet);
}
} catch (SocketException e) {
// port was changed -> return and restart with a new socket
} finally {
this.socket = null;
}
}
private void process(DatagramPacket packet) {
update(new String(packet.getData()), packet.getAddress().toString());
}
private void update(String data, String adress) {
for (IReceiverListener listener : listeners) {
listener.receiveConsoleData(data, adress);
if (data.indexOf("active") > -1) {
listener.incrementWatchDog();
}
}
}
}
Please note, that this might still contains some bugs. It is only supposed to give you a rough idea of how to solve this.

As you are using DatagramSocket, you can change the used port by Binding the socket to a new port rather than the used one:
socket.bind(new InetSocketAddress(new_port));
But remember that bind() method won't work unless the socket is already opened and a port assigned to it, so at the first time you have to create the socket regularly, then when you try to change the port, just bind it.
And the following is a complete visualization of the process:
public void video_udp_server(int port) throws Exception
{
byte[] receiveData = new byte[Integer.MAX_VALUE/100];
for(int i = 0; i < receiveData.length; i++){
receiveData[i] = ' ';
}
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
DatagramSocket socket = null;
try{
socket = new DatagramSocket(port);
}catch(Exception ex){
socket.bind(new InetSocketAddress(port));
}
socket.setReuseAddress(true);
socket.receive(receivePacket);
System.out.println(new String(receivePacket.getData()));
}

Java Socket I/O Streams Null Pointer Exception

I've got a simple client and server that I've written to teach myself a bit of networking. The way it's set up is I've got a main server class which will deal with creating/destroying sockets, and the ConnectionThread class that represents each connection (each of which is given its own thread). The client is super simple.
The problem lies in creating the input/output streams in the ConnectionThread class. I'm not sure exactly what the problem is, but it crashes when the simple test client tries to connect, giving me this:
~~MMO Server Alpha .1~~
Constructed Server
Server Initialized, preparing to start...
Server preparing to check if it should be listening...
Server should be listening, continuing as planned.
ServerSocket passed to ConnectionThread: ServerSocket[addr=0.0.0.0/0.0.0.0,localport=6969]
Constructing ConnectionThread.
Socket[addr=/10.0.1.10,port=55332,localport=6969]
ConnectionThread constructed.
Exception in thread "main" java.lang.NullPointerException
at ConnectionThread.init(ConnectionThread.java:65)
at Server.listen(Server.java:98)
at Server.start(Server.java:62)
at Server.main(Server.java:122)
ConnectionThread added to queue.
Establishing in and out streams:
null
Here are the classes (amended for brevity):
public class Server {
int PORT;
boolean shouldListen;
ArrayList<ConnectionThread> connections = new ArrayList<ConnectionThread>();
ServerSocket serverSocket;
public Server() {
try {
PORT = 6969;
shouldListen = true;
serverSocket = new ServerSocket(PORT);
}
catch (IOException e) {
System.out.println("Error in server constructor.");
System.exit(1);
}
}
public void start() {
System.out.println("Server preparing to check if it should be listening...");
listen();
System.out.println("Server finished listening.");
}
public void listen() {
while (shouldListen) {
ConnectionThread conn = null;
System.out.println("Server should be listening, continuing as planned.");
try {
conn = new ConnectionThread(serverSocket);
}
catch (Exception e) {
System.out.println("____Error constructing ConnectionThread. Could there be another instance of the server running?");
System.exit(1);
}
System.out.println("ConnectionThread constructed.");
connections.add(conn);
System.out.println("ConnectionThread added to queue.");
conn.init();
System.out.println("Finished ConnectionThread initialization, verifying...");
if (conn.isInitialized) {
System.out.println("ConnectionThread Initialized, preparing to start new thread.");
(new Thread(conn)).start();
}
}
}
public static void main(String[] args) {
System.out.println("~~MMO Server Alpha .1~~");
Server server = new Server();
System.out.println("Constructed Server");
server.init();
System.out.println("Server Initialized, preparing to start...");
server.start();
}
}
Here's the ConnectionThread class:
public class ConnectionThread implements Runnable {
boolean shouldBeListening = true;
boolean isThereAnUnsentOutgoingMessage = false;
String outgoingMessage = "OUTGOING UNINITIALIZED";
boolean IsThereAnUnsentIncomingMessage = false;
String incomingMessage = "INCOMING UNITIALIZED";
boolean isInitialized = false;
PrintWriter out;
BufferedReader in;
String currentInputMessage = "Test Input Message from the Server ConnectionThread";
String previousInputMessage = null;
Socket socket;
public ConnectionThread(ServerSocket s) {
System.out.println("ServerSocket passed to ConnectionThread: " + s);
/*
* The purpose of the constructor is to establish a socket
* as soon as possible. All transmissions/logic/anything else
* should happen in init() and/or run().
*/
System.out.println("Constructing ConnectionThread.");
try {
Socket socket = s.accept();
System.out.println(socket);
}
catch (IOException e) {
System.out.println("Error in ConnectionThread constructor");
System.exit(1);
}
}
public void init() {
/*
* Everything should be set up here before run is called.
* Once init is finished, run() should be set to begin work.
* This is to ensure each packet is efficiently processed.
*/
try {
System.out.println("Establishing in and out streams:");
System.out.println(socket);
out = new PrintWriter(socket.getOutputStream(), true);
System.out.println("ConnectionThread: Output Stream (PrintWriter) Established");
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
System.out.println("ConnectionThread: InputStream (BufferedReader) Established");
}
catch (IOException e) {
System.out.println("Error in ConnectionThread method Init.");
System.exit(1);
}
isInitialized = true;
}
And optionally, here's the test client:
public class TestClient {
static PrintWriter out;
BufferedReader in;
public final int PORT = 6969;
Socket socket = null;
InetAddress host = null;
public TestClient() {
out = null;
in = null;
socket = null;
host = null;
}
public void connectToServer() {
System.out.println("Connecting to server...");
try {
host = InetAddress.getLocalHost();
socket = new Socket(host.getHostName(), PORT);
}
catch (Exception e) {
System.out.println("Error establishing host/socket");
System.exit(1);
}
try {
System.out.println("Establishing I/O Streams");
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
}
catch (Exception e) {
System.out.println("Error establishing in/out streams");
System.exit(1);
}
}
public static void main(String[] args) {
System.out.println("~~TestClient Alpha .1~~");
TestClient c = new TestClient();
c.connectToServer();
System.out.println("Should be connected to server. Sending test message...");
while (true) {
System.out.println("here");
out.println("Hello there");
}
}
}

The 'socket' variable in the constructor of ConnectionThread shouldn't be local. It is shadowing the member variable.
It is customary to call accept() in the listen() loop, and pass the accepted socket to the ConnectionThread.

As EJP said, in your ConnectionThread constructor you think that you are assigning the value to the socket field, however you are actually assigning the value to the socket method variable, thus the socket field remains null, and in init() you see socket as null.

In addition to EJP answer: you did not provide ConnectionThread.run() method, but I assume you are going to use fields in, out and socket in your run() method. Since these fields are not marked as volatile or final, depending on your luck and number of core on your computer, you may also get NullPointerException at run() method.
This is because new variable value may be not propagated between caches and new thread will not see value of changed.
Explanation of this possible problem is here - The code example which can prove "volatile" declare should be used

Handling sockets in java

I am creating a socket to a server in java and after the socket is connected it creates a new thread which can access the socket input and output stream and this thread then blocks and processes the input lines when they come in.
I understand that the readln method on the BufferedReader will return null when the input stream ends. This doesn't necessarily mean that the socket is closed though does it? What does this mean? So I would then want to run the close method on the socket to close it nicely.
I also understand that the readln method can throw an IOException and that this is thrown after the close method is called on a socket if it is currently blocking. When else can this be thrown? Could the socket still be open after this is thrown or would it always be closed and ready for garbage collection etc.
This is the code I have at the moment and I don't really know how to handle disconnects properly. At the moment I think this could end up in a deadlock if the disconnect method is called whilst the socket is waiting for a line because disconnect will call close on the socket. This will then throw the IOException on readLine and this will then result in that catch block calling disconnect again.
public class SocketManager {
private Socket socket = null;
private PrintWriter out = null;
private BufferedReader in = null;
private String ip;
private int port;
private Object socketLock = new Object();
public SocketManager(String ip, int port) {
this.ip = ip;
this.port = port;
}
public void connect() throws UnableToConnectException, AlreadyConnectedException {
synchronized(socketLock) {
if (socket == null || socket.isClosed()) {
throw (new AlreadyConnectedException());
}
try {
socket = new Socket(ip, port);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (IOException e) {
throw (new UnableToConnectException());
}
new Thread(new SocketThread()).start();
}
}
public void disconnect() throws NotConnectedException {
synchronized(socketLock) {
if (isConnected()) {
throw (new NotConnectedException());
}
try {
socket.close();
} catch (IOException e) {}
}
}
public boolean isConnected() {
synchronized(socketLock) {
return (socket != null && !socket.isClosed());
}
}
private class SocketThread implements Runnable {
#Override
public void run() {
String inputLine = null;
try {
while((inputLine = in.readLine()) != null) {
// do stuff
}
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e) {}
}
} catch (IOException e) {
// try and disconnect (if not already disconnected) and end thread
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e1) {}
}
}
}
}
}
I basically want to know the best way of achieving the following:
Writing a connect method that connects to a socket and starts a separate thread listening for input.
Writing a disconnect method that disconnects from the socket and terminates the thread that's listening for input.
Handling the scenario of the connection to the remote socket being broken.
I have read through the java tutorial on sockets but in my opinion it doesn't really cover these in much detail.
Thanks!

When I said that it could end up as a deadlock I think I was wrong.
What would happen is:
disconnect() called whilst in.readLine() blocking
socket.close() executed.
in.readline() throws IOException.
I was then thinking that the exception handler in the SocketThread would call disconnect whilst disconnect is waiting for that exception to finish. It wouldn't matter through because they are both different threads so the code in disconnect() would continue whilst the exception is being caught in the SocketThread. The SocketThread would then call disconnect() but would then have to wait until the first instance of disconnect() finished. Then disconnect() would execute again but would get the NotConnectedException thrown which would be caught in the SocketThread and nothing would happen. The SocketThread would exit and that's the wanted result.
However I have looked into the socket class and it also contains these methods:
shutdownInput()
shutdownOutput()
shutdownInput() sends the end EOF symbol into the input stream meaning in.readline() returns null and the loop exits cleanly. shutdownOutput() sends the TCP termination sequence informing the server that it's disconnecting.
Calling both of these before socket.close() makes more sense because it means the thread will exit nicely instead of exiting as a result of an exception being thrown which has more overhead.
So this is the modified code:
public class SocketManager {
private Socket socket = null;
private PrintWriter out = null;
private BufferedReader in = null;
private String ip;
private int port;
private Object socketLock = new Object();
public SocketManager(String ip, int port) {
this.ip = ip;
this.port = port;
}
public void connect() throws UnableToConnectException, AlreadyConnectedException {
synchronized(socketLock) {
if (socket == null || socket.isClosed()) {
throw (new AlreadyConnectedException());
}
try {
socket = new Socket(ip, port);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (IOException e) {
throw (new UnableToConnectException());
}
new Thread(new SocketThread()).start();
}
}
public void disconnect() throws NotConnectedException {
synchronized(socketLock) {
if (isConnected()) {
throw (new NotConnectedException());
}
try {
socket.shutdownInput();
} catch (IOException e) {}
try {
socket.shutdownOutput();
} catch (IOException e) {}
try {
socket.close();
} catch (IOException e) {}
}
}
public boolean isConnected() {
synchronized(socketLock) {
return (socket != null && !socket.isClosed());
}
}
private class SocketThread implements Runnable {
#Override
public void run() {
String inputLine = null;
try {
while((inputLine = in.readLine()) != null) {
// do stuff (probably in another thread)
}
// it will get here if socket.shutdownInput() has been called (in disconnect)
// or possibly when the server disconnects the clients
// if it is here as a result of socket.shutdownInput() in disconnect()
// then isConnected() will block until disconnect() finishes.
// then isConnected() will return false and the thread will terminate.
// if it ended up here because the server disconnected the client then
// isConnected() won't block and return true meaning that disconnect()
// will be called and the socket will be completely closed
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e) {}
}
} catch (IOException e) {
// try and disconnect (if not already disconnected) and end thread
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e1) {}
}
}
}
}
}

In order to be sure that all resources associated with the socket are relased you have to call close() method when you finish work with that socket.
Typical IO exception handling pattern is that you catch it and then perform best efforts to clean everything calling close() method.
So the only thing you have to do is to ensure that you call close() on every socket during it's lifetime.

You're on the right track. I wouldn't use "readline", only raw read, and "do stuff"
should be limited to constructing a queue of received data. Likewise writing replies
ought to be a separate thread that empties a queue of data to be sent.
Despite socket's guarantees of integrity, stuff will go wrong and you'll sometimes receive data that doesn't make sense. There's a crapload of stuff below "read" and "write" and no system is perfect or bug free. Add your own wrapper with checksums at the level of YOUR read and write so you can be sure you're receiving what was intended to be sent.

Sockets, Threads and Services in android, how to make them work together?

I am facing a probleme with threads and sockets I cant figure it out, if someone can help me please i would really appreciate.
There are the facts :
I have a service class NetworkService, inside this class I have a Socket attribute.
I would like it be at the state of connected for the whole lifecycle of the service.
To connect the socket I do it in a thread, so if the server has to timeout, it would not block my UI thread.
Problem is, into the thread where I connect my socket everything is fine, it is connected and I can talk to my server, once this thread is over and I try to reuse the socket, in another thread, I have the error message Socket is not connected.
Questions are :
- Is the socket automatically disconnected at the end of the thread?
- Is their anyway we can pass back a value from a called thread to the caller ?
Thanks a lot,
Here is my code
public class NetworkService extends Service {
private Socket mSocket = new Socket();
private void _connectSocket(String addr, int port) {
Runnable connect = new connectSocket(this.mSocket, addr, port);
new Thread(connect).start();
}
private void _authentification() {
Runnable auth = new authentification();
new Thread(auth).start();
}
private INetwork.Stub mBinder = new INetwork.Stub() {
#Override
public int doConnect(String addr, int port) throws RemoteException {
_connectSocket(addr, port);
_authentification();
return 0;
}
};
class connectSocket implements Runnable {
String addrSocket;
int portSocket;
int TIMEOUT=5000;
public connectSocket(String addr, int port) {
addrSocket = addr;
portSocket = port;
}
#Override
public void run() {
SocketAddress socketAddress = new InetSocketAddress(addrSocket, portSocket);
try {
mSocket.connect(socketAddress, TIMEOUT);
PrintWriter out = new PrintWriter(mSocket.getOutputStream(), true);
out.println("test42");
Log.i("connectSocket()", "Connection Succesful");
} catch (IOException e) {
Log.e("connectSocket()", e.getMessage());
e.printStackTrace();
}
}
}
class authentification implements Runnable {
private String constructFirstConnectQuery() {
String query = "toto";
return query;
}
#Override
public void run() {
BufferedReader in;
PrintWriter out;
String line = "";
try {
in = new BufferedReader(new InputStreamReader(mSocket.getInputStream()));
out = new PrintWriter(mSocket.getOutputStream(), true);
out.println(constructFirstConnectQuery());
while (mSocket.isConnected()) {
line = in.readLine();
Log.e("LINE", "[Current]- " + line);
}
}
catch (IOException e) {e.printStackTrace();}
}
}

Define the output stream as a member variable, attach it in your thread, and only close that stream when you're done...
Currently you're opening (and implicitly closing) the output stream within the thread. As the thread dies, it will close that output stream, which in turn may be killing the socket/connection.
If you define the stream outside of the thread, you can attach it within the thread, and close it at a later time such as when the service is asked to terminate.