We Keep Coding

Receiving data from multiple sockets at once (Multithreading)

I'm new at network programming and i have been searching for a solution to my problem here but couldn't find one. What I want is to have a server that can receive files from multiple sockets at the same time. When a server accepts new connection socket it wraps that socket with a ClientThread class. Here is the code:
public class Server extends Thread {
private ServerSocket server;
private Vector<ClientThread> clients;
#Override
public void run() {
listen();
}
private void listen() {
new Thread("Listening Thread") {
#Override
public void run() {
while (true) {
try {
Socket socket = server.accept();
ClientThread newClient = new ClientThread(socket);
newClient.start();
clients.addElement(newClient);
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
}
}
}.start();
}
ClientThread is a private class inside the Server class. It's always listening for an Object from ObjectInputStream, but also I want to be able to receive one big file after the object. And that is why I think i should use multithreading. Here is the code:
private class ClientThread extends Thread {
public Socket socket;
private boolean loggedIn;
private ObjectInputStream ois;
private BufferedInputStream bis;
public ClientThread(Socket socket) {
this.socket = socket;
loggedIn = true;
InputStream is = socket.getInputStream();
ois = new ObjectInputStream(is);
bis = new BufferedInputStream(is);
}
#Override
public void run() {
receive();
}
private void receive() {
while (loggedIn) {
try {
// this method blocks i guess
Object object = ois.readObject();
// after the object comes the large file
byte[] bytes = new byte[SOME_SIZE];
int bytesRead;
int totalRead = 0;
// reading the large file into memory
while ((bytesRead = bis.read(bytes, totalRead, bytes.length - totalRead)) > -1) {
totalRead += bytesRead;
}
// rest of the code for handling received bytes.......
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace();
}
}
}
}
I'm not sure if receiving data like this is even possible since all these client sockets are sending data to the same port on this server (i guess?). And if clients are sending data at the same time, Server needs to know which data is for which client. Is this already taken care of, or i need entirely different approach here?
I don't know if this is a stupid question, but like I said I'm just starting learning this stuff. Also i couldn't test my program because i don't even have code for the Client yet. Just want to make sure I don't go wrong at the very start. If this is wrong, feel free to post some ideas. :) Thanks!

For a start it's not bad :)
You can improve later on by using a Selector but that's another topic.
Some clarifications though: the ServerSocket listens on a specific port. When a remote client connects to it, a communication channel (i.e. socket) is created. If another client connects, another socket is created. Both sockets are different channels and won't interfere with each other because they are connected to a different remote IP and port.
It all has to do with how TCP headers and IP headers are formed: a TCP data packet is sent with its header containing the source and destination port, on top of IP header containing the source and destination IP. Those are used to discriminate between the different sockets.
Regarding the "broadcast" you want to do (as per your comment in #Rajesh's answer), you have options:
Do it yourself in pure TCP with ServerSocket and Socket like you started
Switch to UDP and use MulticastSocket, which has the advantage of issueing a single send, but you'll have to deal with missing/unordered datagrams in your client code (UDP does not guarantee delivery or ordering, like TCP does)
Check NIO with Selector and SocketChannel
Investigate frameworks like jGroups or Netty which do the I/O stuff for you
As you're learning, I suggest you do that in the above order. Using a framework is nice, but going through coding yourself will teach you a lot more.

This will work functionally. Each thread is reading from a separate socket connected to different client (address + port). They are separate streams, so no issues in reading from that like this.
However it would be much better to use asynchronous sockets.
Few things that can be taken care in the current implementation:
1) As a good practice, close the streams/sockets when transfer is complete.
2) For every new connection, a new thread is created. That will not scale. Even some one can send many requests and bring down your app. Would be better to use a thread pool. "ClientThread" can just implement "Runnable" and when a new connection is received, just submit the new "ClientThread" to thread pool. (In this case, would be better to name it as ClientTask instead of ClientThread)
As mentioned, it would be much more efficient and scalable to use asynchronous socket, but it will take some time master it. With this, you can use just one thread to read all sockets in parallel and depending on load, can use the same thread or a pool of threads to process the data received from all the sockets. Note that, even if use a pool, you will not need separate thread for processing each socket...Just to make best use of multiple CPU Cores, can use multiple threads to process the data.
You may try either java nio (Selector + SocketChannels) or netty library. Netty is much easier to use compared to nio.

Share common data between two threads serving a Socket connection

I saw plenty of similar questions on SO but hardly any of them have Socket in the picture. So please take time to read the question.
I have server app (using ServerSocket) which listens for requests, and when a client attempts to connect, new thread is created to serve the client (and server is back to listening mode for new requests). Now, I need to respond one client based on what other client sent to server.
Example:
ServerSocket listening for incoming connections.
Client A connects, new thread is created to serve A.
Client B connects, new thread is created to serve B.
A sends message "Hello from A" to the Server.
Send this message as a response to Client B.
I'm new to this whole "inter-thread communication" thing. Obviously, above mentioned situation sounds dead simple, but I'm describing this to get a hint, as I'll be exchanging huge amount data among clients keeping server as intermediate.
Also, what if I want to keep a shared object limited to, say 10, particular Clients? such that, when 11th client connects to the server, I create new shared object, which will be used to exchange data between 11th, 12th, 13th..... upto 20th client. And so on for every single set of 10 clients.
What I tried: (foolish I guess)
I have a public class with that object supposed to be shared as public static, so that I can use it as global without instantiating it, like MyGlobalClass.SharedMsg.
That doesn't work, I was unable to send data received in one thread to the other.
I'm aware that there is an obvious locking problem since if one thread is writing to an object, other can't be accessing it until the first thread is done writing.
So what would be an ideal approach to this problem?
Update
Since the way in which I create threads for serving incoming connection requests, I can't understand how I can share same object among the threads, since using Global object as mentioned above doesn't work.
Following is how I listen for incoming connections and create serving threads dynamically.
// Method of server class
public void startServer()
{
if (!isRunning)
{
try
{
isRunning = true;
while (isRunning)
{
try
{
new ClientHandler(mysocketserver.accept()).start();
}
catch (SocketTimeoutException ex)
{
//nothing to perform here, go back again to listening.
}
catch (SocketException ex)
{
//Not to handle, since I'll stop the server using SocketServer's close() method, and its going to throw SocketException anyway.
}
}
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
else
System.out.println("Server Already Started!");
}
And the ClientHandler class.
public class ClientHandler extends Thread
{
private Socket client = null;
private ObjectInputStream in = null;
private ObjectOutputStream out = null;
public ClientHandler(Socket client)
{
super("ClientHandler");
this.client = client;
}
//This run() is common for every Client that connects, and that's where the problem is.
public void run()
{
try
{
in = new ObjectInputStream(client.getInputStream());
out = new ObjectOutputStream(client.getOutputStream());
//Message received from this thread.
String msg = in.readObject().toString();
System.out.println("Client # "+ client.getInetAddress().getHostAddress() +" Says : "+msg);
//Response to this client.
out.writeObject("Message Received");
out.close();
in.close();
client.close();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
I believe that the way I'm creating dynamic threads to serve each client that connects, sharing the same data source is not possible using Global object, since the body of run() above is exactly the same for every client that connects, hence this same method is both consumer and producer. What fixes should I make such that I could create dynamic threads for each connection and still share the same object.

You probably want a queue for communication between each client. Each Queue will be the 'pipeline' for data pushed from one client to the other.
You would use it like so (pseudo code):
Thread 1:
Receive request from Client A, with message for Client B
Put message on back of concurrent Queue A2B
Respond to Client A.
Thread 2:
Receive request from Client B.
Pop message from front of Queue A2B
Respond to Client B with message.
You might also want it generic, so you have a AllToB Queue that many clients (and thus many threads) can write to.
Classes of note: ConcurrentLinkedQueue, ArrayBlockingQueue.
If you want to limit the number of messages, then ArrayBlockingQueue with its capacity constructor allows you to do this. If you don't need the blocking functionality, you can use the methods offer and poll rather than put and take.
I wouldn't worry about sharing the queues, it makes the problem significantly more complicated. Only do this if you know there is a memory usage problem you need to address.
EDIT: Based on your update:
If you need to share a single instance between all dynamically created instances you can either:
Make a static instance.
Pass it into the constructor.
Example of 1:
public class ClientHandler extends Thread
{
public static final Map<ClientHandler, BlockingQueue<String>> messageQueues
= new ConcurrentHashMap<>();
<snip>
public ClientHandler(Socket client)
{
super("ClientHandler");
this.client = client;
// Note: Bad practice to reference 'this' in a constructor.
// This can throw an error based on what the put method does.
// As such, if you are to do this, put it at the end of the method.
messageQueues.put(this, new ArrayBlockingQueue<>());
}
// You can now access this in the run() method like so:
// Get messages for the current client.
// messageQueues.get(this).poll();
// Send messages to the thread for another client.
// messageQueues.get(someClient).offer(message);
A couple of notes:
The messageQueues object should really contain some sort of identifier for the client rather than an object reference that is short lived.
A more testable design would pass the messageQueues object into the constructor to allow mocking.
I would probably recommend using a wrapper class for the map, so you can just call offer with 2 parameters rather than having to worry about the map semantics.

Can I use Java's BufferedReader with an event listener rather than while(true)?

I'm in the process of writing a Java client and server, and they're both talking to each other correctly. At the moment, however, the client must always send a message to the server before it can receive one - but I want the server to be able to send a message without the client asking for it.
In AS3, my primary language, I'd have added an event listener to the socket and handled the data that way - but I can't seem to find out how I would do that in Java.
Currently my client has this code in it:
public String send(String message) {
out.println(message);
try {
return in.readLine();
} catch (IOException ex) {
return "";
}
}
Basically I call it, passing my message, and it returns giving me the server's response.
Instead of doing this I want to send a message there, like below, and have the response picked up somewhere different (by an event listener).
public void send(String message) {
out.println(message);
}
As I couldn't figure out how an event listener would work, I did contemplate using a separate thread to run while(true){in.readLine;} but I want to avoid this if at all possible; I'm not too comfy with threads, and I want to keep it as simple as possible.

The simplest way to do this is to use threads. I would have at least one thread reading the socket which allows the server to send a message to the client any time.
The problem with mixed responses and events is that your first block won't work because the "response" could be an event.
If you don't want to use threads, you could have something like
// simple but probably not a good idea.
public String send(String message) {
out.println(message);
try {
while(true) {
String line = in.readLine();
if (isEvent(line))
processEvent(line);
else
return line;
} catch (IOException ex) {
return "";
}
}
However, it will only read events when a request is sent and if this is not done regularly it can cause the stream to bank up all the way to the server causing it to block as well.

java.io.StreamCorruptedException: invalid stream header: 7371007E

I have a client Server application which communicate using objects.
when I send only one object from the client to server all works well.
when I attempt to send several objects one after another on the same stream I get
StreamCorruptedException.
Can some one direct me to the cause of this error?
client write method
private SecMessage[] send(SecMessage[] msgs)
{
SecMessage result[]=new SecMessage[msgs.length];
Socket s=null;
ObjectOutputStream objOut =null;
ObjectInputStream objIn=null;
try
{
s=new Socket("localhost",12345);
objOut=new ObjectOutputStream( s.getOutputStream());
for (SecMessage msg : msgs)
{
objOut.writeObject(msg);
}
objOut.flush();
objIn=new ObjectInputStream(s.getInputStream());
for (int i=0;i<result.length;i++)
result[i]=(SecMessage)objIn.readObject();
}
catch(java.io.IOException e)
{
alert(IO_ERROR_MSG+"\n"+e.getMessage());
}
catch (ClassNotFoundException e)
{
alert(INTERNAL_ERROR+"\n"+e.getMessage());
}
finally
{
try {objIn.close();} catch (IOException e) {}
try {objOut.close();} catch (IOException e) {}
}
return result;
}
server read method
//in is an inputStream Defined in the server
SecMessage rcvdMsgObj;
rcvdMsgObj=(SecMessage)new ObjectInputStream(in).readObject();
return rcvdMsgObj;
and the SecMessage Class is
public class SecMessage implements java.io.Serializable
{
private static final long serialVersionUID = 3940341617988134707L;
private String cmd;
//... nothing interesting here , just a bunch of fields , getter and setters
}

If you are sending multiple objects, it's often simplest to put them some kind of holder/collection like an Object[] or List. It saves you having to explicitly check for end of stream and takes care of transmitting explicitly how many objects are in the stream.
EDIT: Now that I formatted the code, I see you already have the messages in an array. Simply write the array to the object stream, and read the array on the server side.
Your "server read method" is only reading one object. If it is called multiple times, you will get an error since it is trying to open several object streams from the same input stream. This will not work, since all objects were written to the same object stream on the client side, so you have to mirror this arrangement on the server side. That is, use one object input stream and read multiple objects from that.
(The error you get is because the objectOutputStream writes a header, which is expected by objectIutputStream. As you are not writing multiple streams, but simply multiple objects, then the next objectInputStream created on the socket input fails to find a second header, and throws an exception.)
To fix it, create the objectInputStream when you accept the socket connection. Pass this objectInputStream to your server read method and read Object from that.

when I send only one object from the client to server all works well.
when I attempt to send several objects one after another on the same stream I get StreamCorruptedException.
Actually, your client code is writing one object to the server and reading multiple objects from the server. And there is nothing on the server side that is writing the objects that the client is trying to read.

This exception may also occur if you are using Sockets on one side and SSLSockets on the other. Consistency is important.

Forwarding incoming TCP "Commands" to a function?

In Java, how would you set up a socket listener that listened to a socket for a series of bytes that represented a command and on recieving called a method which parsed the incoming data and invoked the appropriate command?
Clarification:
My issue is not with handling the commands (Which might also be error codes or responses to commands from the server) but with creating the socket and listening to it.
More Clarification:
What I want to do is mimic the following line of .Net (C#) code:
_stream.BeginRead(_data,0, _data.Length, new
AsyncCallback(this.StreamEventHandler), _stream);
Where:
_stream is a network stream created from a socket
_data is an array of Byte of length 9
this.StreamHandler is a delegate (function pointer) which get executed when data is read.
I am rewriting a library from C# into Java and the component I am currently writing passes commands to a server over TCPIP but also has to be able to bubble up events/responses to the layer above it.
In C# this seems to be trivial and it's looking less and less so in Java.

Starting from my other answer: The specific part you request is the one that goes into the section: "Magic goes here". It can be done in ohh so many ways, but one is:
final InputStream in = socket.getInputStream();
// This creates a new thread to service the request.
new Thread(new Runnable(){
public void run(){
byte[] retrievedData= new byte[ITEM_LENGTH];
in.read(retrievedData, 0, ITEM_LENGTH);
in.close();
// Here call your delegate or something to process the data
callSomethingWithTheData(retrievedData);
}
}).start();

Have a small main method which sets up the socket and listens for incoming connections. Pass each connection to a worker object (possibly in its own thread).
The worker object should have two APIs: The server and the client. The client API gets a connection and reads data from it, the server API takes a connection and writes data to it.
I like to keep these two in a single class because that makes it much more simple to keep the two in sync. Use a helper class to encode/decode the data for transmission, so you have single point to decide how to transmit integers, commands, options, etc.
If you want to go further, define a command class and write code to serialize that to a socket connection and read it from it. This way, you worker objects just need to declare which command class they handle and the server/client API gets even more simple (at the expense of the command class).

I would
put each command into a class of its own, where each class implements a specific interface (e.g. Command)
create a Map<String,Command> which contains a lookup table from each command string to an instance of the class that implements that command

This should help.
Lesson 1: Socket Communications

The TCP connection provides you with one InputStream and one OutputStream. You could just poll the InputStream continuously for the next command (and its inputs) on a dedicated thread. ByteBuffer.wrap(byte[] array) may be useful in interpreting the bytes as chars, ints, longs, etc. You could also pass objects around using serialization.

Any naive approach most likely will not scale well.
Consider using a REST-approach with a suitable small web-server. Jetty is usually a good choice.

To create an listen to a socket, in a very naive way:
mServerSocket = new ServerSocket(port);
listening = true;
while (listening) {
// This call blocks until a connection is made
Socket socket = serverSocket.accept();
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
// Here you do your magic, reading and writing what you need from the streams
// You would set listening to true if you have some command to close the server
// remotely
out.close();
in.close();
socket.close();
}
Normally it is a good idea to delegate the processing of the input stream to some other thread, so you can answer the next request. Otherwise, you will answer all requests serially.
You also need to define some kind of protocol of what bytes you expect on the input and output streams, but from your question it looks like you already have one.

You could create an enum with one member per command
interface Comamnd {
// whatever you expect all command to know to perform their function
void perform(Context context);
}
enum Commands implements Command{
ACTIONONE() {
void perform(Context context) {
System.out.println("Action One");
}
},
ACTIONTWO() {
void perform(Context context) {
System.out.println("Action Two");
}
}
}
// initialise
DataInputStream in = new DataInputStream(socket.getInputStream());
// in a loop
byte[] retrievedData= new byte[ITEM_LENGTH];
in.readFully(retrievedData);
String command = new String(retrievedData, 0);
Commands.valueOf(command).perform(context);