We Keep Coding

java client server objects serialization: read objects in the same order they were sent

I'm developing a client server application in java. Communication takes place by serializing/deserializing immuable objects containing the information required, with ObjectOutputStream and ObjectInputStream.
In the server, every time I accept a new tcp connection, I instantiate a new Thread to handle the connection with that particular client.
This thread has a reference to the socket and it keeps reading objects:
while (true){
Object receivedObject = inputStream.readObject();
if (receivedObject instanceof MessageA){
//do sth
} else if (receivedObject instanceof MessageB){
//do sth else
}
this Runnable also has a method to send objects to the client:
public void sendMessage(Message message) {
try {
output.writeObject(message);
output.reset();
} catch (IOException e) {
e.printStackTrace();
}
}
client-side, when the user enters the server's address, I instantiate a SocketClient object to handle the communication with the server:
public class SocketClient extends Observable {
private final Socket socket;
private final ObjectOutputStream outputStream;
private final ObjectInputStream inputStream;
private static int TIMEOUT = 5000;
public SocketClient(String serverAddress, int serverPort) throws IOException {
this.socket = new Socket();
this.socket.connect(new InetSocketAddress(serverAddress, serverPort), TIMEOUT);
this.outputStream = new ObjectOutputStream(socket.getOutputStream());
this.inputStream = new ObjectInputStream(socket.getInputStream());
}
public void readMessage() {
Thread readerThread = new Thread(() -> {
boolean read = true;
while (read) {
Message message;
try {
message = (Message) inputStream.readObject();
notifyObservers(message);
} catch (IOException | ClassNotFoundException e) {
message = new ErrorMessage(null, "Connection lost with the server.");
disconnect();
read = false;
}
}
});
readerThread.start();
}
public void sendMessage(Message message) {
try {
outputStream.writeObject(message);
outputStream.reset();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public void disconnect() {
try {
socket.close();
} catch (IOException e) {
// disconnection error
e.printStackTrace();
}
}
}
messages are sent and received in both applications, the only problem is that when I send multiple messages sequentially in the server, they are sometimes received in a different order in the clients.
How may I fix that?

Non deterministic behaviour of socket based application

I'm working on Client-Server application and I'v hit the wall with one issue. I got ServerWorker that is responsible for one connected client, it creates 2 threads, 1 to listen for incoming data from this client and 1 to send data to him.
class ServerWorker {
private DataProcessor dataProcessor;
private ObjectInputStream inputStream;
private ObjectOutputStream outputStream;
private Thread receiverThread;
private Thread senderThread;
private Optional<DataPacket> dataToSend;
private ServerWorker(Socket socket) {
try {
dataToSend = Optional.empty();
dataProcessor = new DataProcessor();
receiverThread = new Thread(this::readAndProcessData);
senderThread = new Thread(this::sendData);
inputStream = new ObjectInputStream(socket.getInputStream());
outputStream = new ObjectOutputStream(socket.getOutputStream());
} catch (IOException e) {
//TODO
}
}
static ServerWorker create(Socket socket) {
return new ServerWorker(socket);
}
void start() {
receiverThread.start();
senderThread.start();
}
void stop() {
receiverThread.interrupt();
senderThread.interrupt();
}
private void readAndProcessData() {
DataPacket dataPacket;
try {
while((dataPacket = (DataPacket)inputStream.readObject()) != null) {
System.out.println("incoming message: " + dataPacket.getContent());
dataToSend = Optional.of(dataProcessor.process(dataPacket));
}
} catch (ClassNotFoundException | IOException e) {
//TODO
}
}
private void sendData() {
while(true) { //TODO
dataToSend.ifPresent(data -> {
try {
outputStream.writeObject(data);
outputStream.flush();
dataToSend = Optional.empty();
} catch (IOException e) {
//TODO
}
});
}
}
}
And DataProcessor is just a small class for now
public class DataProcessor {
public DataPacket process(DataPacket packet){
packet.setContent(packet.getContent().toUpperCase());
return packet;
}
}
and ofcourse, DataPacket which is the same for both client and server
public class DataPacket implements Serializable {
private String content;
public String getContent() {
return content;
}
public void setContent(String content) {
this.content = content;
}
}
DataProcessor and DataPackets are just kind of POCs now, this will grow into much larger and more complicated classes, long story short, ServerWorker will recieve data and pass it to process, then after some logic is done, returning data will be stored inside dataToSend variable and removed after sending. Problem is, code I'v posted above works only sometimes. 90% of the time when I run my server app and client one (code below) nothing happens, uppercased "hello world" isnt going back to client. What's funny, when I run my server in debug mode (even without any breakpoints!), it works... Any ideas what the heck went wrong?
public static void main(String... args) throws Exception {
Socket socket = new Socket("localhost", 9999);
DataPacket dataPacket = new DataPacket();
dataPacket.setContent("hello world");
ObjectOutputStream os = new ObjectOutputStream(socket.getOutputStream());
os.writeObject(dataPacket);
os.flush();
ObjectInputStream inputStream = new ObjectInputStream(socket.getInputStream());
while((dataPacket = (DataPacket)inputStream.readObject()) != null) {
System.out.println(dataPacket.getContent());
}
}
edit#
adding one more class, ConnectionDispatcher that is responsible for creating ServerWorker objects
class ConnectionDispatcher implements Runnable {
private ServerSocket serverSocket;
private List<ServerWorker> serverWorkers;
private volatile boolean isReceiving;
private ConnectionDispatcher(int port) throws IOException {
serverSocket = new ServerSocket(port);
serverWorkers = new ArrayList<>();
isReceiving = false;
}
static ConnectionDispatcher create(int port) throws IOException {
return new ConnectionDispatcher(port);
}
#Override
public void run() {
isReceiving = true;
while(isReceiving) {
acceptIncomingConnections();
}
}
private void acceptIncomingConnections() {
try {
ServerWorker worker = ServerWorker.create(serverSocket.accept());
serverWorkers.add(worker);
worker.start();
} catch (IOException e) {
e.printStackTrace();
}
}
}

It seems that you're modifying dataToSend from one thread while simultaneously reading its value in another. This isn't thread-safe, and the thread that's reading its value may never see its updated value set by the other thread. For this reason, I'd declare dataToSend as volatile.
private volatile Optional<DataPacket> dataToSend;
I have not yet had the chance to test this out myself, but I can in about an hour (assuming this change doesn't fix your problem).

You could uses a Array Blocking Queue, to simulate a producer and consumer pattern.
Let the receiver thread, put new DataPacket into the queue, and let the sender take from the queue and process it and send it.
This will eliminate threading issues and acts as a buffer.
With your current code, you might loose packets,when they arrive at higher rate.
And i agree with user930, private Optional<DataPacket> dataToSend; should be volatile.
Also you can make your code much scalable with JavaNIO, you could look into Apache Mina project.

Handling sockets in java

I am creating a socket to a server in java and after the socket is connected it creates a new thread which can access the socket input and output stream and this thread then blocks and processes the input lines when they come in.
I understand that the readln method on the BufferedReader will return null when the input stream ends. This doesn't necessarily mean that the socket is closed though does it? What does this mean? So I would then want to run the close method on the socket to close it nicely.
I also understand that the readln method can throw an IOException and that this is thrown after the close method is called on a socket if it is currently blocking. When else can this be thrown? Could the socket still be open after this is thrown or would it always be closed and ready for garbage collection etc.
This is the code I have at the moment and I don't really know how to handle disconnects properly. At the moment I think this could end up in a deadlock if the disconnect method is called whilst the socket is waiting for a line because disconnect will call close on the socket. This will then throw the IOException on readLine and this will then result in that catch block calling disconnect again.
public class SocketManager {
private Socket socket = null;
private PrintWriter out = null;
private BufferedReader in = null;
private String ip;
private int port;
private Object socketLock = new Object();
public SocketManager(String ip, int port) {
this.ip = ip;
this.port = port;
}
public void connect() throws UnableToConnectException, AlreadyConnectedException {
synchronized(socketLock) {
if (socket == null || socket.isClosed()) {
throw (new AlreadyConnectedException());
}
try {
socket = new Socket(ip, port);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (IOException e) {
throw (new UnableToConnectException());
}
new Thread(new SocketThread()).start();
}
}
public void disconnect() throws NotConnectedException {
synchronized(socketLock) {
if (isConnected()) {
throw (new NotConnectedException());
}
try {
socket.close();
} catch (IOException e) {}
}
}
public boolean isConnected() {
synchronized(socketLock) {
return (socket != null && !socket.isClosed());
}
}
private class SocketThread implements Runnable {
#Override
public void run() {
String inputLine = null;
try {
while((inputLine = in.readLine()) != null) {
// do stuff
}
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e) {}
}
} catch (IOException e) {
// try and disconnect (if not already disconnected) and end thread
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e1) {}
}
}
}
}
}
I basically want to know the best way of achieving the following:
Writing a connect method that connects to a socket and starts a separate thread listening for input.
Writing a disconnect method that disconnects from the socket and terminates the thread that's listening for input.
Handling the scenario of the connection to the remote socket being broken.
I have read through the java tutorial on sockets but in my opinion it doesn't really cover these in much detail.
Thanks!

When I said that it could end up as a deadlock I think I was wrong.
What would happen is:
disconnect() called whilst in.readLine() blocking
socket.close() executed.
in.readline() throws IOException.
I was then thinking that the exception handler in the SocketThread would call disconnect whilst disconnect is waiting for that exception to finish. It wouldn't matter through because they are both different threads so the code in disconnect() would continue whilst the exception is being caught in the SocketThread. The SocketThread would then call disconnect() but would then have to wait until the first instance of disconnect() finished. Then disconnect() would execute again but would get the NotConnectedException thrown which would be caught in the SocketThread and nothing would happen. The SocketThread would exit and that's the wanted result.
However I have looked into the socket class and it also contains these methods:
shutdownInput()
shutdownOutput()
shutdownInput() sends the end EOF symbol into the input stream meaning in.readline() returns null and the loop exits cleanly. shutdownOutput() sends the TCP termination sequence informing the server that it's disconnecting.
Calling both of these before socket.close() makes more sense because it means the thread will exit nicely instead of exiting as a result of an exception being thrown which has more overhead.
So this is the modified code:
public class SocketManager {
private Socket socket = null;
private PrintWriter out = null;
private BufferedReader in = null;
private String ip;
private int port;
private Object socketLock = new Object();
public SocketManager(String ip, int port) {
this.ip = ip;
this.port = port;
}
public void connect() throws UnableToConnectException, AlreadyConnectedException {
synchronized(socketLock) {
if (socket == null || socket.isClosed()) {
throw (new AlreadyConnectedException());
}
try {
socket = new Socket(ip, port);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (IOException e) {
throw (new UnableToConnectException());
}
new Thread(new SocketThread()).start();
}
}
public void disconnect() throws NotConnectedException {
synchronized(socketLock) {
if (isConnected()) {
throw (new NotConnectedException());
}
try {
socket.shutdownInput();
} catch (IOException e) {}
try {
socket.shutdownOutput();
} catch (IOException e) {}
try {
socket.close();
} catch (IOException e) {}
}
}
public boolean isConnected() {
synchronized(socketLock) {
return (socket != null && !socket.isClosed());
}
}
private class SocketThread implements Runnable {
#Override
public void run() {
String inputLine = null;
try {
while((inputLine = in.readLine()) != null) {
// do stuff (probably in another thread)
}
// it will get here if socket.shutdownInput() has been called (in disconnect)
// or possibly when the server disconnects the clients
// if it is here as a result of socket.shutdownInput() in disconnect()
// then isConnected() will block until disconnect() finishes.
// then isConnected() will return false and the thread will terminate.
// if it ended up here because the server disconnected the client then
// isConnected() won't block and return true meaning that disconnect()
// will be called and the socket will be completely closed
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e) {}
}
} catch (IOException e) {
// try and disconnect (if not already disconnected) and end thread
if (isConnected()) {
try {
disconnect();
} catch (NotConnectedException e1) {}
}
}
}
}
}

In order to be sure that all resources associated with the socket are relased you have to call close() method when you finish work with that socket.
Typical IO exception handling pattern is that you catch it and then perform best efforts to clean everything calling close() method.
So the only thing you have to do is to ensure that you call close() on every socket during it's lifetime.

You're on the right track. I wouldn't use "readline", only raw read, and "do stuff"
should be limited to constructing a queue of received data. Likewise writing replies
ought to be a separate thread that empties a queue of data to be sent.
Despite socket's guarantees of integrity, stuff will go wrong and you'll sometimes receive data that doesn't make sense. There's a crapload of stuff below "read" and "write" and no system is perfect or bug free. Add your own wrapper with checksums at the level of YOUR read and write so you can be sure you're receiving what was intended to be sent.

Object stream I/O issue, client object not updated

i'm working a client-server based game. The game is mainly caracterized by a Grid object which must be synchronized between clients. To do that I use the Object I/O Streams over Sockets.
However i encounter an issue during the synchronization process. The Grid is sent and recieved by all clients but its state is not modified after the first upload to each client.
I mean by that that clients do recieve the object in its present state when they connect but subsequent receiptions (initiated by either another client connection or previous client moves) don't present any modification over the initial sent state...
here are (stripped-down) snippets of the server side code:
while(true) //continuously accept new connections
{
//wait incoming connection, and accept it
Socket newSocket = serverListener.accept();
//create player details and save it in hashtable
Player newPlayer = new Player(newSocket); //streams saved here
Players.put(newPlayer);
//update all clients
sendGridToAll();
}
The Player class constructor:
public Player(Socket s) throws IOException
{
this.Tx = new ObjectOutputStream(socket.getOutputStream());
this.Rx = new DataInputStream(socket.getInputStream());
}
The SendToAll method:
public void sendGridToAll()
{
synchronized(Players) //do nothing while players HT is being modified
{
for(Enumeration e = Players.elements(); e.hasMoreElements(); )
{
Player tmpPlayer = (Player)e.nextElement();
ObjectOutputStream tmpTx = tmpPlayer.getTx();
try {
tmpTx.writeObject(grid);
}
catch(IOException ex)
{
ex.printStackTrace();
}
}
print.log("Grid update sent");
}
}
and here is the client side snippet handling the object reception (ran in a thread):
public void run()
{
ObjectInputStream RX = new ObjectInputStream(socket.getInputStream());
while(true)
{
try
{
RX_grid = (Grid)RX.readObject();
}
catch (IOException ex)
{
print.log("IO Error");
ex.printStackTrace();
}
catch (ClassNotFoundException ex)
{
print.log("Bad grid class UID");
}
finally
{
print.log("grid recieved");
c.updateGui(RX_grid);
}
}
}
Thank you for your help

Java Server socket stuck on accept call (Android Client, Java Server)

Below I have put a fragment of code to help understand my problem. I have a server code, works fine for the first time the client loads and sends a packet. After the first packet is received, the server is stuck on "accept".
I have wireshark configured for this port, and the server is getting those packets. I just wonder why accept wont return more than once. Its driving me nuts.
Server Code
public class DAPool implements Runnable {
private ServerSocket serverSocket;
private ArrayList<DA> pool;
private LinkedList<Socket> clientConnQ;
public DAPool(int newPoolSize, int serverPort) {
try {
serverSocket = new ServerSocket(serverPort, 500, InetAddress.getByName("127.0.0.1"));
} catch (IOException e) {
e.printStackTrace();
return;
}
poolSize = newPoolSize;
clientConnQ = new LinkedList<Socket>();
pool = new ArrayList<DA>(poolSize);
DA deviceThread;
for (int threads = 0; threads < poolSize; threads++) {
deviceThread = new DA();
connPool.add(deviceThread);
deviceThread.start();
}
}
public void run() {
while (true) {
Socket incomingSocket;
try {
incomingSocket = serverSocket.accept();
} catch (IOException e) {
e.printStackTrace();
return;
}
insertNewConnToQ(incomingSocket);
}
}
private class DA extends Thread {
private Socket clientSocket;
private ObjectInputStream inputObjectStream;
public DA() {
}
public void run() {
while (true) {
while (clientConnQ.isEmpty()) {
synchronized (clientConnQ) {
try {
clientConnQ.wait();
} catch (InterruptedException ignored) {
ignored.printStackTrace();
}
}
}
synchronized (clientConnQ) {
clientSocket = (Socket) clientConnQ.removeFirst();
try {
inputObjectStream = new ObjectInputStream(clientSocket.getInputStream());
} catch (IOException e) {
e.printStackTrace();
return;
}
// Do something useful here
}
}
}
}
}
Client Code
public class SendQueue extends Thread {
LinkedList<Message> requestQ;
Message sendRequest, requestMessage;
Socket clientSocket;
OutputStream outputStream;
ObjectOutputStream objectOutputStream;
public SendQueue(Socket newClientSocket) {
requestQ = new LinkedList<Message>();
clientSocket = newClientSocket;
}
public void run() {
while (true) {
synchronized (requestQ) {
while (requestQ.isEmpty()) {
try {
requestQ.wait();
} catch (InterruptedException ignored) {
ignored.printStackTrace();
}
}
sendRequest = requestQ.removeFirst();
}
try {
outputStream = clientSocket.getOutputStream();
objectOutputStream = new ObjectOutputStream(outputStream);
objectOutputStream.writeObject(sendRequest);
objectOutputStream.flush();
outputStream.flush();
} catch (IOException e) {
e.printStackTrace();
} catch (RuntimeException e) {
e.printStackTrace();
}
}
}
public int sendRequest(Message message) {
synchronized (requestQ) {
requestQ.addLast(message);
requestQ.notify();
}
return 0;
}
}

I don't see a timeout being set on the serverSocket.
ServerSocket.accept() is a blocking operation so it will block until either an error occurs, a timeout occurs, or a connection is accepted.
Try
SererSocket.setSOTimeout(10000)
You also don't seem to be closing your streams when your finished.

Are you sure that it is sticking on the accept call? Did you get a stacktrace that shows it waiting on accept?
Assuming it is getting stuck elsewhere I'm wondering if it isn't because clientConnQ is being held in one of your DA instances. The synchronized block covers the // Do something useful here section.
I wonder if it might work if you changed the code to be
synchronized (clientConnQ) {
clientSocket = (Socket) clientConnQ.removeFirst();
}
try {
...
Once you have your clientSocket from clientConnQ then no other instance can process that socket.

Ok, if I got a $ for everytime I asked a silly question :)
Here goes. A client socket connects and thats when a server receives a accept call. For some silly reason I was waiting on accept for receiving further data from the client. Infact, I should just wait for something on the "stream" and then process the stream. I should not wait on the accept for that connection.
Accept is to be called to "connect" to the socket, not to receive data continuously.
Thanks for your all your help. You forced me to think about thread synchronization, the design, sockets in general and finally arrive that the solution.
Fantastic responses people. Thanks.
Siddharth