I am writing two types of packets to a PLC which responds by sending different packets for each type of input packets. I am using socket class with two different threads to achieve this i.e. one thread to handle one type of message. Since received messages/packets at sockets can come at any thread it is getting difficult to synchronize and achieve the necessary task.
There are two types of byte data that needs to be send to the PLC socket i.e. ByteFrame1 and ByteFrame2. After sending the packets different responses are received.
Following code snippet shows Thread 1.
Thread workerThread1 = new Thread() {
#Override
public void run() {
try {
dos = new DataOutputStream(socket.getOutputStream());
while (socket.isConnected()) {
dos.write(ByteFrame1);
try {
DataInputStream dis = new DataInputStream(socket.getInputStream());
readLength = dis.read(OutputBytes1, 0, receiveBufferSize1);
//Interpret OutputBytes1 code follows
Similarly code snippet for second thread is as follows:
Thread workerThread2 = new Thread() {
#Override
public void run() {
try {
dos = new DataOutputStream(socket.getOutputStream());
while (socket.isConnected()) {
dos.write(ByteFrame2);
try {
DataInputStream dis = new DataInputStream(socket.getInputStream());
readLength = dis.read(OutputBytes2, 0, receiveBufferSize2);
//Interpret OutputBytes2 code follows
I found that both threads are receiving some garbled data that is difficult to interpret. Probable reason can be socket receive getting out of synch due to threads out of order execution. How to achieve synchronization at each thread so that the response received at each thread don't gets mixed up?
Okay a couple of things here.
You should only ever have 1 thread sending messages through a socket. If you have more, you risk gabling one message by adding another message inside it. You can use a queue to send messages sequentially.
Do NOT use Data streams! This is Java-only and now many years beyond its reason d'etre (it was created as a solution to the big/small endian issue - which is no longer an issue)
You should put together a message standard for your communications ala how an HTTP header has information about what is being sent. I recommend using XML or JSON to structure your messages (it's how I do it over Sockets and WebSockets).
Related
I'm new at network programming and i have been searching for a solution to my problem here but couldn't find one. What I want is to have a server that can receive files from multiple sockets at the same time. When a server accepts new connection socket it wraps that socket with a ClientThread class. Here is the code:
public class Server extends Thread {
private ServerSocket server;
private Vector<ClientThread> clients;
#Override
public void run() {
listen();
}
private void listen() {
new Thread("Listening Thread") {
#Override
public void run() {
while (true) {
try {
Socket socket = server.accept();
ClientThread newClient = new ClientThread(socket);
newClient.start();
clients.addElement(newClient);
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
}
}
}.start();
}
ClientThread is a private class inside the Server class. It's always listening for an Object from ObjectInputStream, but also I want to be able to receive one big file after the object. And that is why I think i should use multithreading. Here is the code:
private class ClientThread extends Thread {
public Socket socket;
private boolean loggedIn;
private ObjectInputStream ois;
private BufferedInputStream bis;
public ClientThread(Socket socket) {
this.socket = socket;
loggedIn = true;
InputStream is = socket.getInputStream();
ois = new ObjectInputStream(is);
bis = new BufferedInputStream(is);
}
#Override
public void run() {
receive();
}
private void receive() {
while (loggedIn) {
try {
// this method blocks i guess
Object object = ois.readObject();
// after the object comes the large file
byte[] bytes = new byte[SOME_SIZE];
int bytesRead;
int totalRead = 0;
// reading the large file into memory
while ((bytesRead = bis.read(bytes, totalRead, bytes.length - totalRead)) > -1) {
totalRead += bytesRead;
}
// rest of the code for handling received bytes.......
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace();
}
}
}
}
I'm not sure if receiving data like this is even possible since all these client sockets are sending data to the same port on this server (i guess?). And if clients are sending data at the same time, Server needs to know which data is for which client. Is this already taken care of, or i need entirely different approach here?
I don't know if this is a stupid question, but like I said I'm just starting learning this stuff. Also i couldn't test my program because i don't even have code for the Client yet. Just want to make sure I don't go wrong at the very start. If this is wrong, feel free to post some ideas. :) Thanks!
For a start it's not bad :)
You can improve later on by using a Selector but that's another topic.
Some clarifications though: the ServerSocket listens on a specific port. When a remote client connects to it, a communication channel (i.e. socket) is created. If another client connects, another socket is created. Both sockets are different channels and won't interfere with each other because they are connected to a different remote IP and port.
It all has to do with how TCP headers and IP headers are formed: a TCP data packet is sent with its header containing the source and destination port, on top of IP header containing the source and destination IP. Those are used to discriminate between the different sockets.
Regarding the "broadcast" you want to do (as per your comment in #Rajesh's answer), you have options:
Do it yourself in pure TCP with ServerSocket and Socket like you started
Switch to UDP and use MulticastSocket, which has the advantage of issueing a single send, but you'll have to deal with missing/unordered datagrams in your client code (UDP does not guarantee delivery or ordering, like TCP does)
Check NIO with Selector and SocketChannel
Investigate frameworks like jGroups or Netty which do the I/O stuff for you
As you're learning, I suggest you do that in the above order. Using a framework is nice, but going through coding yourself will teach you a lot more.
This will work functionally. Each thread is reading from a separate socket connected to different client (address + port). They are separate streams, so no issues in reading from that like this.
However it would be much better to use asynchronous sockets.
Few things that can be taken care in the current implementation:
1) As a good practice, close the streams/sockets when transfer is complete.
2) For every new connection, a new thread is created. That will not scale. Even some one can send many requests and bring down your app. Would be better to use a thread pool. "ClientThread" can just implement "Runnable" and when a new connection is received, just submit the new "ClientThread" to thread pool. (In this case, would be better to name it as ClientTask instead of ClientThread)
As mentioned, it would be much more efficient and scalable to use asynchronous socket, but it will take some time master it. With this, you can use just one thread to read all sockets in parallel and depending on load, can use the same thread or a pool of threads to process the data received from all the sockets. Note that, even if use a pool, you will not need separate thread for processing each socket...Just to make best use of multiple CPU Cores, can use multiple threads to process the data.
You may try either java nio (Selector + SocketChannels) or netty library. Netty is much easier to use compared to nio.
I'm developing a simple Client-Server application over socket, but I can't get why client freezes when he is reading an object.
Server must be capable of dealing with multiple client.
Keeping it simple, my Server looks like:
...
server_thread = new Thread(new Runnable() {
#Override
public void run() {
int p = 0;
ObjectInputStream in;
ObjectOutputStream out;
NetworkOffer message;
try (ServerSocket serverSocket = new ServerSocket(port)) {
// get connections
LinkedList<Socket> client_sockets = new LinkedList<>();
while (p++ < partecipants) client_sockets.add(serverSocket.accept());
// sending welcome object
for (Socket socket : client_sockets) {
out = new ObjectOutputStream(socket.getOutputStream());
message = new NetworkOffer();
out.writeObject(buyer_offer);
}
...
My Client:
...
client_thread = new Thread(new Runnable() {
#Override
public void run() {
ObjectInputStream in;
NetworkOffer smessage;
try {
Socket ssocket = new Socket("localhost", port);
in = new ObjectInputStream(ssocket.getInputStream());
// waiting server message
------------->Object o = in.readObject();
smessage = (NetworkOffer)o;
System.out.println(smessage.toString());
...
EDIT:
To make things clearer, this is the protocol I want to implement:
N clients connect to Server
Server send welcome to Clients
Every client makes an offer
Server chooses best offer, and sends a message to each Client with Accept/Reject
If there isn't an acceptable offer goto 3.
Client sticks on Object o = in.readObject(); even if server has already sent his message.
No error, nothing. Thread is simply freezed there waiting for something.
What's going on?
The problem is ServerSocket.accept() is a blocking call meaning the server will hang until somebody connects to it. When somebody connects, the server will add that new socket to the client_sockets. If the number of sockets added is less than participants, it will then call accept() again and wait for another connection. It will only enter your for loop when the total number of sockets is equal to participants. You need to spawn a new thread to handle each incoming client socket and allow the server to return immediately to ServerSocket.accept(). Have a look at the Reactor pattern for a good example of how to implement this.
What your code should look like is this:
Server waits for connections.
When client connects, spawn a new thread to handle the connection.
Server returns to waiting for connections.
New thread sends welcome message on socket, adds the socket to the list of client_sockets and waits for the clients offer.
Store the clients offer.
When all offers have been received, compare to find the best.
Send Accept/Reject messages.
As I said before: are you sure that the server have sent the data to the client - there is no buffer flush so it can still be cached.
out.flush() will make sure that buffer is flushed.
It will make sense to handle clients separately and send them periodic messages to update them about the status.
It is useful for your server code to handle the client disconnect/connection drop too.
On the side note:
message = new NetworkOffer();
out.writeObject(buyer_offer);
Your code seems to be sending something else that is not present in your example. Is that correct?
Is the code below sufficient to accept concurrent UDP transmissions? More specifically, if 2 clients transmit concurrently, will DatagramSocket queue up the transmissions and deliver them one by one as I call receive(), or will only one make it through?
DatagramSocket socket = new DatagramSocket(port, address);
byte[] buffer = new byte[8192];
while(!disconnect){
DatagramPacket p = new DatagramPacket(buffer, buffer.length);
socket.receive(p);
}
There is no queuing by default. The client may retry till timeout or similiar are reach.
UDP is quiet fast but on heavy load you may have clients that cannot connect.
If the packets make it to your networking interface (imagine lost packets on a congested wireless channel) they will passed up and the blocking method socket.receive(p) will be called. If there is a collision of packets on the channel because of two clients transmitting at the same time you will not get any of the two packets. But this is most likely not possible because the access technology of networking interfaces will take care of this, check
CSMA/CA or CSMA/CD
After calling socket.receive(p) you should create a new thread to process the packet itself. That will make sure that the next packet can be received on the socket.
EDIT:
Description of INTEL's TX and RX descriptors
A basic solution would have on thread responsible for handling a number of incoming requests (with your desired limit) and then handing them off to other worker/request handler threads. This basic structure is very much the same with most servers: a main thread responsible for handing off requests to worker threads. When each of these worker threads is finished, the you can update a shared/global counter to let the main thread know that it can establish a new connection. This will require synchronization, but it's a neat and simple abstraction.
Here's the idea:
Server Thread:
// Receive Packet
while (true) {
serverLock.acquire();
try {
if (numberOfRequests < MAX_REQUESTS) {
packet = socket.receive();
numberOfRequests++;
requestThread(packet).run();
} else {
serverMonitor.wait(serverLock);
}
} finally {
serverLock.release();
}
}
Request Thread:
// Handle Packet
serverLock.acquire();
try {
if (numberOfRequests == MAX_REQUESTS){
numberOfRequests--;
serverMonitor.pulse();
}
} finally {
serverLock.release();
}
This is just to give you an idea of what you can start out with. But when you get the hang of it, you'll be able to make optimizations and enhancements to make sure the synchronization is all correct.
One particular enhancement, which also lends itself to limited number of requests, is something called a ThreadPool.
I've written a java intake program that send an PDF-formatted intake to a shared folder so that other people in the network can read it. However, there is not a way for the other people to know that an intake was sent unless someone tells them, so I want the program to send an alert message to the other computers telling them that an intake has been sent.
Now I've done some research into this and figured that TCP is the way to go since it's reliable. I also know that this is a one-to-many sending going on, so I assume that my Intake program will act as the server an the other computers will be the client, or should it be the other way around?
Now I assume that I have to create a client program that listens to the server and waits for it to send a message.
With that in mind, how do I:
Create a client program that listens for the message continuously until the program is closed. I assume that I'll be using "while (true)" and sleep. If so, how long do I put the program to sleep?
Make it as part of Windows service so that can load up when Windows start.
On the server end, how do I:
Send messages to more than one computer, since TCP is not capable of multicasting or broadcasting. I assume an array/vector will play a part here.
Oh, this is a one-way communication. The client doesn't have to respond back to the server.
First of all, UDP is quite reliable (in fact, as reliable as the IP protocol itself). TCP simply ensures that the data was received which involved quite a lot of magic in the back end. Unless you absolutely need to be sure that other machines got the message, you could do it with UDP. Mind that I'm not saying “Don't use TCP”, I just want to make it straight that you should take UDP into consideration as well.
Anyway, yes, you can create a simple listening program. Here is an example of a client in Java that reads messages from the server. It overrides the run method of a Thread class:
public void run() {
try {
String messageFromServer = reader.readLine();
while (messageFromServer != null) {
// Do things with messageFromServer here
// processor.processFromServer(messageFromServer);
messageFromServer = reader.readLine(); // Blocks the loop, waits for message
}
}
catch (IOException e) {
// Handle your exception
}
}
Amongst other things, my thread was set up as such:
public CommunicationThread(String hostname, int port, int timeout) throws IOException, SocketTimeoutException {
InetSocketAddress address = new InetSocketAddress(hostname, port);
socket = new Socket();
socket.connect(address, 2000); // 2000ms time out
// You can use the writer to write messages back out to the server
writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
}
Now, regards to server-side you can do something as follows:
Write a program to allow clients to contact, given that they know your address.
Accept the connections, and store the sockets in a list.
When you need to send out a message, traverse the list and send the data to everyone on it.
You can start listening on your server with
this.socket = new ServerSocket(port);
You could (or even should(?)) make it threaded so that you can accept clients while serving others. You can accept new clients with:
socket.accept(); // Blocks, waiting for someone to connect, returns open socket
Feel free to pass that to a whole new class which can deal with BufferedWriter (and maybe even BufferedReader if you want to read from clients as well). That class is where you would implement things such as writeToClient(message)
Consider the situation where you have a ClientConnection class that has writeToClient(String s) method and (Server server, Socket socket) and initialized ArrayList conList.
Here is how you would follow:
In a separate thread in Server, accept connections with
ClientConnection con = new ClientConnection(this, socket.accept());
conList.add(con);
Then, when you want to write to clients:
for (ClientConnection c : conList) {
c.writeToClient("I'm sending you a message!");
}
I hope you get a vague idea of what you need to do. Read the Socket documentation, it's very useful. Also, as always with threaded applications, make sure you aren't doing things such as modifying a list while traversing it and avoid race conditions.
Good luck!
I'm trying to write a server in Java. I know very little Java. I've found an example using Selector.
It looks good, but it behaves strangely. When I do my_socket_output_stream.writeBytes("hello world") in client code, the server reads this message one byte at a time. Shouldn't I be notified only when the complete message is sent? Now I'd have to check my buffer after getting every byte to know if I can already work with it. Seems terribly inefficient.
I wonder if that's due to Selector or is that just how sockets work (it's been a long time since I used them). Could I make them wait for the full message somehow? Also, can I associate some objects with a channel? Right now all sockets use the same buffer. I'm sure you see how that is a problem..
The reason I want to use a Selector is that my server is only going to do io with a HashTable. Multiple threads would just be constantly waiting. And I only have one core anyway. Though maybe a combination of ThreadPoolExecutor and ConcurrentHashMap would be a good choice? It would surely enable me to have a buffer per socket..
I'd appreciate suggestions.
I faced the same problem a long time ago. I solved by first sending the number of bytes of the message, then sending the message itself byte by byte. Then I expanded it to line by line.
At the sender's side:
// code at sender side
StreamConnectionNotifier service = (StreamConnectionNotifier) Connector.open( url );
//System.out.println("opened");
StreamConnection con = (StreamConnection) service.acceptAndOpen();
OutputStream outputStream = con.openOutputStream();
// file to send
Scanner in = new Scanner(inFile);
//just count lines
String s=null;
int countLines=0;
while(in.hasNext()) {
s=in.nextLine();
countLines++;
}
//send num of lines
outputStream.write(Integer.toHexString(countLines).getBytes());
try{Thread.sleep(100);} catch(InterruptedException e){}
//send lines
in = new Scanner(inFile);
for(int i=0; i<countLines; i++) {
s=in.nextLine()+"\n";
outputStream.write(s.getBytes());
Thread.sleep(100);
}
At the receiver's side:
// code at receiver side
byte buffer[] = new byte[80];
int bytes_read = inputStream.read( buffer );
String received = new String(buffer, 0, bytes_read);
try{Thread.sleep(100);} catch(InterruptedException e){}
int receiveLines = Integer.parseInt(received);
PrintWriter out = new PrintWriter(new FileOutputStream("received.txt"));
for(int i=0; i<receiveLines; i++) {
bytes_read = inputStream.read( buffer );
received = new String(buffer, 0, bytes_read);
out.println(received);
Thread.sleep(100);
}
I hope this helps :)
Unless you have gained some skill in understanding of the issues of multi-threading and synchronization, avoid NIO. It is good stuff, but you are (currently) not properly equipped to debug it, much less fully appreciate and understand its synchronization needs.
Write a Runnable class that wraps a ServerSocket in a while loop, allowing the loop to block on the accept method. Then grab that return socket and construct a "client handler" thread which will handle whatever data came in the NIC.
This resource will give you some pointers on writing this older, slightly slower, and much more understandable server listening loop. I linked to a "middle" page in the article as that's the code listing, but you might want to read the entire article.
This uses the older "one Thread to handle the request" model of network processing. It's not terribly bad, but it can encounter scalability issues.
The alternative is to take the deep dive and do it with non-blocking NIO. It's not terribly hard, but it does require you to completely structure your server code in a manner that's not straightforward. Effectively you get "pools" of worker Threads than can perform various tasks, and then you synchronize on passing the data from worker to worker.
Shouldn't I be notified only when the complete message is sent?
No. Without specifying how messages should be separated from each other, the API can only give you one byte at a time (or all available bytes). The easiest way to separate strings would be to use a java.io.PrintStream on the side that is sending the message and a java.io.BufferedReader on the side that is receiving, like so:
// code that sends strings
OutputStream out = ...; // get the output stream from the socket
PrintStream sender = new PrintStream(out);
sender.println("Hello, world.");
// code that receives strings
InputStream in = ...; // get the input stream from the socket
BufferedReader receiver = new BufferedReader(new InputStreamReader(in));
String message = receiver.readLine(); // reads "Hello, world."