We Keep Coding

Broken pipe error communicating with Java server & C# client in multi thread env

I hope to find any help on my old annoying problem.
I have a TCP sever program with java and client program with c#
packet protocol between those two is simply consist of 4byte length & body ASCII data.
The Problem is that C# client faces FormatException which is from parsing fail on length byte. If I look into an error from client side, then client is trying to parse somewhere in the body which is not length header.
But apparently, Server does not send broken packet.
meanwhile, at the server, I could find an Broken pipe error whenever this kind of problem happens.
Unfortunately this error does not always happen and was not able to recreate the problem situation. it makes me difficult to find exact cause of this problem
Please see below codes for server side
public class SimplifiedServer {
private Map<InetAddress, DataOutputStream> outMap;
private Map<InetAddress,DataInputStream> inMap;
protected void onAcceptNewClient(Socket client) {
DataOutputStream out = null;
DataInputStream in = null;
try {
out = new DataOutputStream(client.getOutputStream());
in = new DataInputStream(client.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
outMap.put(client.getInetAddress(), out);
inMap.put(client.getInetAddress(), in);
}
public void writeToAll(String packet) {
outMap.forEach((key, out) -> {
try {
byte[] body = packet.getBytes("UTF-8");
int len = body.length;
if (len > 9999) {
throw new IllegalArgumentException("packet length is longer than 10000, this try will be neglected");
}
String lenStr = String.format("%04d%s", len, packet);
byte[] obuf = lenStr.getBytes();
synchronized (out) {
out.write(obuf);
out.flush();
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
public void listenClient(Socket client) {
try {
DataOutputStream out = outMap.get(client.getInetAddress());
DataInputStream in = inMap.get(client.getInetAddress());
while (true) {
byte[] received = SimplePacketHandler.receiveLpControlerData(in);
byte[] lenBytes = new byte[4];
for( int i = 0 ; i < 4 ; i ++){
lenBytes[i] = in.readByte();
}
String lenString = new String(lenBytes);
int length = Integer.parseInt(lenString);
byte[] data = new byte[length];
for ( int i = 0 ; i < length ; i ++){
data[i] = in.readByte();
}
if ( data == null ){
System.out.println("NetWork error, closing socket :" + client.getInetAddress());
in.close();
out.close();
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
return;
}
doSomethingWithData(out, data);
}
} catch (NumberFormatException e) {
e.printStackTrace();
} catch ( Exception e ) {
e.printStackTrace();
} finally {
try {
System.out.println(client.getRemoteSocketAddress().toString() + " closing !!! ");
// remove stream handler from map
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
//close socket.
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
And here is client side code
public class ClientSide
{
public TcpClient client;
public String ip;
public int port;
public NetworkStream ns;
public BinaryWriter writer;
public BinaryReader reader;
public Boolean isConnected = false;
public System.Timers.Timer t;
public String lastPacketSucceeded = String.Empty;
public ClientSide(String ip, int port)
{
this.ip = ip;
this.port = port;
client = new TcpClient();
}
public bool connect()
{
try
{
client.Connect(ip, port);
}
catch (SocketException e)
{
Console.WriteLine(e.ToString());
return false;
}
Console.WriteLine("Connection Established");
reader = new BinaryReader(client.GetStream());
writer = new BinaryWriter(client.GetStream());
isConnected = true;
return true;
}
public void startListen()
{
Thread t = new Thread(new ThreadStart(listen));
t.Start();
}
public void listen()
{
byte[] buffer = new byte[4];
while (true)
{
try
{
reader.Read(buffer, 0, 4);
String len = Encoding.UTF8.GetString(buffer);
int length = Int32.Parse(len);
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
String body = Encoding.UTF8.GetString(bodyBuf);
doSomethingWithBody(body);
}
catch (FormatException e)
{
Console.WriteLine(e.Message);
}
}
}
public void writeToServer(String bodyStr)
{
byte[] body = Encoding.UTF8.GetBytes(bodyStr);
int len = body.Length;
if (len > 10000)
{
Console.WriteLine("Send Abort:" + bodyStr);
}
len = len + 10000;
String lenStr = Convert.ToString(len);
lenStr = lenStr.Substring(1);
byte[] lengthHeader = Encoding.UTF8.GetBytes(lenStr);
String fullPacket = lenStr + bodyStr;
byte[] full = Encoding.UTF8.GetBytes(fullPacket);
try
{
writer.Write(full);
}
catch (Exception)
{
reader.Close();
writer.Close();
client.Close();
reader = null;
writer = null;
client = null;
Console.WriteLine("Send Fail" + fullPacket);
}
Console.WriteLine("Send complete " + fullPacket);
}
}
Considering it is impossible to recreate problem, I would guess this problem is from multithread issue. but I could not find any further clue to fix this problem.
Please let me know if you guys need any more information to solve this out.
Any help will be great appreciated, thanks in advance.

A broken pipe exception is caused by closing the connection on the other side. Most likely the C# client has a bug, causing the format exception which causes it to close the connection and therefore the broken pipe on the server side. See what is the meaning of Broken pipe Exception?.
Check the return value of this read:
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
According to Microsoft documentation for BinaryReader.Read https://msdn.microsoft.com/en-us/library/ms143295%28v=vs.110%29.aspx
[The return value is ] The number of bytes read into buffer. This might be less than the number of bytes requested if that many bytes are not available, or it might be zero if the end of the stream is reached.
If it reads less than the length bytes then next time it will be parsing the length using data somewhere in the middle of the last message.

These broke pipe exceptions happen when the client (browser) has closed the connection, but the server (your tag) continues to try to write to the stream.
This usually happens when someone clicks Back, Stop, etc. in the browser and it disconnects from the server before the request is finished. Sometimes, it can happen because, for example, the Content-Length header is incorrect (and the browser takes its value as true).
Usually, this is a non-event, and nothing to worry about. But if you are seeing them in your dev environment when you know you have not interrupted your browser, you might dig a bit more to find out why.
WLS server will try to filter these exceptions from the web container out of the log, since it is due to client (browser) action and we can't do anything about it. But the server doesn't catch all of them.
refer from :: https://community.oracle.com/thread/806884

Server Client communication fails at Outputstreams

My program is basically:
Client sends a String to Server,
Based on this String, Server is creating an ArrayList,
ArrayList is sent back to the Client.
What is failing here is:
After Client sends a String, the Server receives it and doesn't do anything else. In this time Client keeps on working and gets a NullPointer.
Client side:
public static ArrayList<String> sendStringToServer(String report) {
Socket socket;
ArrayList<String> fieldsList = new ArrayList<String>();
try {
socket = new Socket("localhost", 2345);
OutputStream os = socket.getOutputStream();
PrintStream ps = new PrintStream(os, true);
ps.println(report);
ps.flush();
//Here the debugger should stop and wait for server to create a List
//at this point there is no answer, code breaks
ObjectInputStream objectInput = new ObjectInputStream(socket.getInputStream());
Object object = objectInput.readObject();
fieldsList = (ArrayList<String>) object;
socket.close();
return fieldsList;
} catch (IOException e1) {
e1.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}
Server side:
public class Server {
private ServerSocket serverSocket;
private Socket clientSocket;
private String telegram;
private StringBuilder telegramSB;
public static void main(String[] args) throws IOException, JRException {
new Server();
}
public Server() {
try {
serverSocket = new ServerSocket(2345);
while (true) {
clientSocket = serverSocket.accept();
InputStream is = clientSocket.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
try {
//debugger goes to here and then stops
telegram = br.readLine();
int counter = 0;
boolean startSeq = false;
for (char ch : telegram.toCharArray()) {
if (counter == 0 && ch == '/') {
startSeq = true;
}
if (startSeq == true) {
telegramSB = new StringBuilder();
telegramSB.append(ch);
}
if (ch == '\n') {
if (telegram.length() < 255) {
sendListWithFields();
} else {
new Launcher(telegram).run();
}
}
counter++;
}
} catch (JRException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
System.out.println(e);
}
}

My guess here would be that the BufferedReader is waiting to fill its buffer and you haven't sent enough data for it to do that and return so its waiting for more data to come through that never does (because your clients stops writing and starts to read). You could test this theory temporarily by dumping a load more data into the OutputStream on the client and flushing it.
If the above is the case then you probably want to not use BufferedReader but you have other issues here which also mean you probably want to avoid using PrintStream and BufferedReader for communication and serialisation anyway. For example the default character encoding on two different machines and JVMs could be different. When you create your PrintStream and InputStreamReader you don't specify a character encoding so they could end up being mismatched and the string that you write (including the newline character) could end up being understood completely differently by the remote side, this could also be a reason why its blocking (the client side encodes the newline character in one way but the server is expecting it to be encoded a completely different way), though less likely I think .
If you don't have to use PrintStream then I would suggest instead using DataOutputStream / DataInputStream:
//Client
BufferedOutputStream bufout = new BufferedOutputStream(socket.getOutputStream());
DataOutputStream dout = new DataOutputStream(bufout);
dout.writeUTF(report);
dout.flush();
//Server
BufferedInputStream bufin = new BufferedInputStream(socket.getInputStream());
DataInputStream din = new DataInputStream(bufin);
String report = din.readUTF();
You still get buffering from the BufferedIn/OutputStreams so it will be performant but the DataIn/OutputStreams will manage termination of variable length objects for you - they will send a length prefixing the string to tell the other side exactly how many bytes to read, so you don't need to use a special character to terminate the string you wrote, and this also means it doesn't matter what the content of your String is. In your example above even if it was working if your String had a newline character in it the server would read up until that first newline character, not to the end of the string you sent and that would put them out of sync for the next send/receive along that stream.
Using write/readUTF also specifies an encoding (UTF-8) so there is no mismatch there either.

Why is the code not reachable?

the following is giving me a "code unreachable" message on 'out.close();'
I can't find the issue as it is more or less identical to other code I have run which worked!
import java.io.*;
import java.net.*;
public class MyClient {
private static String SERVER = "127.0.0.1";
private static Integer PORT = 8765;
public static void main(String[] args) throws IOException {
// Connect to the server and create the writer and reader
Socket socket = new Socket(SERVER,PORT);
PrintWriter out = new PrintWriter(socket.getOutputStream(),true);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// Loop forever
while(true) {
out.println("Question:");
String sum = System.console().readLine();
out.println(sum);
String line = in.readLine().trim();
if(line==null || line.startsWith("Finished")) {
socket.close();
return;
}
else if (line.startsWith("My answer is: ")){
System.out.println(line);
String message = System.console().readLine();//correct or wrong!!
out.println(message);
}
}
// Close the in and out and socket
out.close();
in.close();
socket.close();
}
}

You are doing a return from within the while loop. You should do break instead.

Because the code never gets to :
// Close the in and out and socket
out.close();
in.close();
socket.close();
Change return to break:
if(line==null || line.startsWith("Finished")) {
socket.close();
break; //<------------------CHANGE
}

Here's the problem
// Loop forever
while(true) {
It'll loop forever, you never stop it so the next line after the loop will never be executed. That's it :P

Because you have an infinite loop (while(true)) with no break or other means of exit.

It is not a good style to do a return inside a loop, but if want to be sure to free resources you can wrap your loop with a try ... finally:
try {
while(true) {
// ...
if(condition) {
return;
}
// ...
}
} finally {
out.close(); // this is called just before leaving the surrounding function
// ...
}
This works even if there is an exception thrown inside the loop.

Java BufferedReader, how to only call if will not block?

I have 2 sockets and I am using BufferedReader around it's InputStreams. What I am trying to do is take all input from the first socket and send it to the other socket (and visa versa).
The problem is that if the first one does not send a message, it will still block on the first readLine() even though the 2nd socket has already sent some data and is ready. I would like to continue with this simple approach of using no additional threads.
Here's some code that I wrote up, as you can see I have 2 BufferedReaders (in0 and in1) , the program gets stuck at in0.readLine() (blocking).
private void network()
{
PrintWriter out0 = null, out1 = null;
BufferedReader in0 = null,in1 = null;
try{
//clients[] is an array of Socket[2]
in0 = new BufferedReader(new InputStreamReader(clients[0].getInputStream()));
out0 = new PrintWriter(clients[0].getOutputStream(), true);
in1 = new BufferedReader(new InputStreamReader(clients[1].getInputStream()));
out1 = new PrintWriter(clients[1].getOutputStream(), true);
} catch (IOException e) {
System.out.println("Accept failed: 4445");
System.exit(-1);
}
int count = 1;
while(true)
{
System.out.println("network check loop # " + count);
++count;
String nextMessage = null;
try {
if( (nextMessage = in0.readLine()) != null)
{
this.relayMessage(nextMessage,out1);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Middle of network check loop");
nextMessage = null;
try {
if((nextMessage = in1.readLine()) != null)
{
this.relayMessage(nextMessage,out0);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
How can I just skip that statement if in0 is not ready to give me some data? I have seen BufferedReader's ready() method and have attempted to use in0.ready() && readLine() but this causes an infinite loop as neither of the bufferedreaders appear to ever be 'ready'. As well, I am certain that the messages being sent over the socket end in newline characters so readLine() should process correctly!
Any ideas?

Try to use setSoTimeout to put a timeout on your read(), then you just need to catch the SocketTimeoutException if the timer has expired.

Here break and continue keywords are your friends.

The simplest approach is to use two threads. This way you don't have to write your own scheduling code to determine which thread should be running. BTW: The code to copy from one socket to another is the same in each thread, reducing duplication.
To manage your threads I would use an ExecutorService which will make shutting downt eh threads easier.

Client to close the connection

I got a socket listener which keep listening for data. The problem now is that the client which send data will finally close the connection by itself. Based on my codes below I am wondering do I still need to perform this part of the codes where it does writeBuffer.close();?
Should I remove the final part and just put the socket closing the catch?
public void run()
{
BufferedWriter writeBuffer = null;
BufferedReader readBuffer = null;
String message="";
try {
writeBuffer = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
readBuffer = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
int m = 0, count=0;
int nextChar=0;
while ((nextChar=readBuffer.read()) != -1)
{
message += (char) nextChar;
if (nextChar == '#')
{
System.out.println("\n\nSending PA : "+message);
writeBuffer.write("$PA\r\n");
writeBuffer.flush();
message="";
}
}
}
catch (Exception ex)
{
System.out.println("MyError:Exception has been caught in in the main first try");
ex.printStackTrace(System.out);
}
/*finally
{
try
{
if ( writeBuffer != null )
{
writeBuffer.close();
}
else
{
System.out.println("MyError:writeBuffer is null in finally close");
}
}
catch(IOException ex)
{
ex.printStackTrace(System.out);
}
}*/
}

It's always a good idea to explicitly close the connections you're using. Think about it, it might be possible that the client never closes the connection (of course, then you'd have to implement some kind of timeout mechanism that closes the connection on the server side after a certain amount of time, but that's a different matter).
My point is - it never hurts to be careful, and manage your resources in a conservative fashion.