We Keep Coding

Java Packets mess

I am programming a server in Java. Here's my main thread code :
public class EntryThread extends Thread {
volatile byte[] toSend;
public EntryThread() {
}
public void TreatRequest(byte[] data, InetAddress IPAddress)
{
try {
switch (data[0])
{
case 0: // ping
toSend = new byte[] { (byte) 255, data[1], (byte) 255};
Server.serverSocket.send(new DatagramPacket(toSend, 3, IPAddress, 17550));
break;
case 2:
break;
}
} catch (Exception e)
{
System.out.println("Exception because of a packet malformation issue. You can ignore it.");
e.printStackTrace();
}
}
public void run() {
Runtime.getRuntime().addShutdownHook(new Thread(){public void run(){
try {
Server.serverSocket.close();
System.out.println("The server is shut down!");
} catch (Exception e) { /* failed */ }
}});
try {
Server.serverSocket = new DatagramSocket(Configuration.port);
byte[] receiveData = new byte[512];
DatagramPacket receivePacket = new DatagramPacket(receiveData,
receiveData.length);
while(true) {
Server.serverSocket.receive(receivePacket);
byte[] data = receivePacket.getData();
System.out.println("RECEIVED: " + new String(data));
InetAddress IPAddress = receivePacket.getAddress();
/*
* data[0] : command/255 if
* data[1] : C-ID.
* data[2] : arguments/content
*/
TreatRequest(data, IPAddress);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
All seems good, but when I send packets with some software (PacketSender), it's starting to become weird. Here's my console :
RECEIVED: [][][][][][]... // I sent 00 in hex, no problem there)
RECEIVED: ÿ[]ÿ[][][][]... // here's the response of the server)
RECEIVED: [][]ÿ[][][][]... // this ÿ is still there even if I just sent 00 in hex.
So it only overwrites on a array I'm forced to make it big.
So I would like to know :
How can I make the packet date to be reset to lots of 0s each time ?
Can I just get an array of the size of the packet, instead of this weird "pass the array as an argument" method ?

The buffer used by DatagramPacket is not cleared each time a packet is received. However, the method getLength() tells you how much data was received in the current packet, and you should be using that to limit what you extract from the buffer. For your example:
System.out.println("RECEIVED: " + new String(data,0,receivePacket.getLength()));

As documented in the receive javadoc:
The length field of the datagram packet object contains the length of
the received message.
System.out.println("RECEIVED: " + new String(data, 0, receivePacket.getLength()));

Broken pipe error communicating with Java server & C# client in multi thread env

I hope to find any help on my old annoying problem.
I have a TCP sever program with java and client program with c#
packet protocol between those two is simply consist of 4byte length & body ASCII data.
The Problem is that C# client faces FormatException which is from parsing fail on length byte. If I look into an error from client side, then client is trying to parse somewhere in the body which is not length header.
But apparently, Server does not send broken packet.
meanwhile, at the server, I could find an Broken pipe error whenever this kind of problem happens.
Unfortunately this error does not always happen and was not able to recreate the problem situation. it makes me difficult to find exact cause of this problem
Please see below codes for server side
public class SimplifiedServer {
private Map<InetAddress, DataOutputStream> outMap;
private Map<InetAddress,DataInputStream> inMap;
protected void onAcceptNewClient(Socket client) {
DataOutputStream out = null;
DataInputStream in = null;
try {
out = new DataOutputStream(client.getOutputStream());
in = new DataInputStream(client.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
outMap.put(client.getInetAddress(), out);
inMap.put(client.getInetAddress(), in);
}
public void writeToAll(String packet) {
outMap.forEach((key, out) -> {
try {
byte[] body = packet.getBytes("UTF-8");
int len = body.length;
if (len > 9999) {
throw new IllegalArgumentException("packet length is longer than 10000, this try will be neglected");
}
String lenStr = String.format("%04d%s", len, packet);
byte[] obuf = lenStr.getBytes();
synchronized (out) {
out.write(obuf);
out.flush();
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
public void listenClient(Socket client) {
try {
DataOutputStream out = outMap.get(client.getInetAddress());
DataInputStream in = inMap.get(client.getInetAddress());
while (true) {
byte[] received = SimplePacketHandler.receiveLpControlerData(in);
byte[] lenBytes = new byte[4];
for( int i = 0 ; i < 4 ; i ++){
lenBytes[i] = in.readByte();
}
String lenString = new String(lenBytes);
int length = Integer.parseInt(lenString);
byte[] data = new byte[length];
for ( int i = 0 ; i < length ; i ++){
data[i] = in.readByte();
}
if ( data == null ){
System.out.println("NetWork error, closing socket :" + client.getInetAddress());
in.close();
out.close();
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
return;
}
doSomethingWithData(out, data);
}
} catch (NumberFormatException e) {
e.printStackTrace();
} catch ( Exception e ) {
e.printStackTrace();
} finally {
try {
System.out.println(client.getRemoteSocketAddress().toString() + " closing !!! ");
// remove stream handler from map
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
//close socket.
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
And here is client side code
public class ClientSide
{
public TcpClient client;
public String ip;
public int port;
public NetworkStream ns;
public BinaryWriter writer;
public BinaryReader reader;
public Boolean isConnected = false;
public System.Timers.Timer t;
public String lastPacketSucceeded = String.Empty;
public ClientSide(String ip, int port)
{
this.ip = ip;
this.port = port;
client = new TcpClient();
}
public bool connect()
{
try
{
client.Connect(ip, port);
}
catch (SocketException e)
{
Console.WriteLine(e.ToString());
return false;
}
Console.WriteLine("Connection Established");
reader = new BinaryReader(client.GetStream());
writer = new BinaryWriter(client.GetStream());
isConnected = true;
return true;
}
public void startListen()
{
Thread t = new Thread(new ThreadStart(listen));
t.Start();
}
public void listen()
{
byte[] buffer = new byte[4];
while (true)
{
try
{
reader.Read(buffer, 0, 4);
String len = Encoding.UTF8.GetString(buffer);
int length = Int32.Parse(len);
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
String body = Encoding.UTF8.GetString(bodyBuf);
doSomethingWithBody(body);
}
catch (FormatException e)
{
Console.WriteLine(e.Message);
}
}
}
public void writeToServer(String bodyStr)
{
byte[] body = Encoding.UTF8.GetBytes(bodyStr);
int len = body.Length;
if (len > 10000)
{
Console.WriteLine("Send Abort:" + bodyStr);
}
len = len + 10000;
String lenStr = Convert.ToString(len);
lenStr = lenStr.Substring(1);
byte[] lengthHeader = Encoding.UTF8.GetBytes(lenStr);
String fullPacket = lenStr + bodyStr;
byte[] full = Encoding.UTF8.GetBytes(fullPacket);
try
{
writer.Write(full);
}
catch (Exception)
{
reader.Close();
writer.Close();
client.Close();
reader = null;
writer = null;
client = null;
Console.WriteLine("Send Fail" + fullPacket);
}
Console.WriteLine("Send complete " + fullPacket);
}
}
Considering it is impossible to recreate problem, I would guess this problem is from multithread issue. but I could not find any further clue to fix this problem.
Please let me know if you guys need any more information to solve this out.
Any help will be great appreciated, thanks in advance.

A broken pipe exception is caused by closing the connection on the other side. Most likely the C# client has a bug, causing the format exception which causes it to close the connection and therefore the broken pipe on the server side. See what is the meaning of Broken pipe Exception?.
Check the return value of this read:
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
According to Microsoft documentation for BinaryReader.Read https://msdn.microsoft.com/en-us/library/ms143295%28v=vs.110%29.aspx
[The return value is ] The number of bytes read into buffer. This might be less than the number of bytes requested if that many bytes are not available, or it might be zero if the end of the stream is reached.
If it reads less than the length bytes then next time it will be parsing the length using data somewhere in the middle of the last message.

These broke pipe exceptions happen when the client (browser) has closed the connection, but the server (your tag) continues to try to write to the stream.
This usually happens when someone clicks Back, Stop, etc. in the browser and it disconnects from the server before the request is finished. Sometimes, it can happen because, for example, the Content-Length header is incorrect (and the browser takes its value as true).
Usually, this is a non-event, and nothing to worry about. But if you are seeing them in your dev environment when you know you have not interrupted your browser, you might dig a bit more to find out why.
WLS server will try to filter these exceptions from the web container out of the log, since it is due to client (browser) action and we can't do anything about it. But the server doesn't catch all of them.
refer from :: https://community.oracle.com/thread/806884

Android UDP needs to recover from back packets

So I have a listener thread for UDP in order for it to bring in packets for h264 and KLV. The problem I have is that it runs great, but if it happens to miss a few packets (I repro by playing a video online), it never recovers and instead is just jumbled frame after frame. Here is the code handling the input data:
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port) throws Exception
{
if (m_socket == null || m_socket.isClosed())
{
m_socket = new DatagramSocket(port);
m_socket.setBroadcast(true);
}
m_socket.setSoTimeout(1000);
DatagramPacket packet = new DatagramPacket(m_recvPacket, ARSTools.m_packetSize);
//Log.e("UDP", "Waiting for UDP broadcast");
try
{
m_socket.receive(packet);
}
catch (Exception e)
{
Log.i("UDP", "Socket Read Error: " + e);
return;
}
//Log.e("UDP", "UDP packet obtained: " + packet.getLength());
int count = packet.getLength();
byte [] newPacket = new byte[count];
for(int i = 0; i < count; ++i)
newPacket[i] = packet.getData()[i];
ARSDemuxer.GetInstance().m_inPackets.addLast(newPacket);
}
Obviously my ignorance to this is the cause, but I am not sure what the solution is. Once 3 or 4 packets are lost the count starts being almost every frame and what you see after is just junk.

Blackberry Http Connection times out on image download. Why?

My application loops through about 200 urls that are all jpg images.
In the simulator it reads ok, then stores the byte array in persistentStore with no problems.
On the device, it gives java.io.IOException: TCP read timed out on basically every image.
Every now and then, one gets through. Not even sure how. The image sizes don't give insight either. Some are 6k, some are 11k. Size doesn't seem to matter for timing out.
I'll try to post what I believe to be the relevant code, but I am not really an expert here, so if I left something out, please say so.
Call http connection through loop and join thread:
for(int i = 0; i < images.size(); i ++)
{
try {
String url = images.elementAt(i).toString();
HttpRequest data3 = new HttpRequest(url, "GET", false);
data3.start();
data3.join();
} catch (IOException e) {
Dialog.inform("wtf " + e);
}
}
Make the actual connection in HttpConnection class with the proper suffix:
try
{
HttpConnection connection = (HttpConnection)Connector.open(url + updateConnectionSuffix());
int responseCode = connection.getResponseCode();
if(responseCode != HttpConnection.HTTP_OK)
{
connection.close();
return;
}
String contentType = connection.getHeaderField("Content-type");
long length = connection.getLength();
InputStream responseData = connection.openInputStream();
connection.close();
outputFinal(responseData, contentType, length);
}
catch(IOException ex)
{
} catch (SAXException ex) {
} catch (ParserConfigurationException ex) {
}
Finally, read the stream and write the bytes to a byte array:
else if(contentType.equals("image/png") || contentType.equals("image/jpeg") || contentType.equals("image/gif"))
{
try
{
if((int) length < 1)
length = 15000;
byte[] responseData = new byte[(int) length];
int offset = 0;
int numRead = 0;
StringBuffer rawResponse = new StringBuffer();
int chunk = responseData.length-offset;
if(chunk < 1)
chunk = 1024;
while (offset < length && (numRead=result.read(responseData, offset, chunk)) >= 0){
rawResponse.append(new String(responseData, offset, numRead));
offset += numRead;
}
String resultString = rawResponse.toString();
byte[] dataArray = resultString.getBytes();
result.close();
database db = new database();
db.storeImage(venue_id, dataArray);
}
catch( Exception e )
{
System.out.println(">>>>>>>----------------> total image fail: " + e);
}
}
Things to consider:
Length is always byte length in simulator. In device it is always -1.
The chunk var is a test to see if I force a 15k byte array, will it try to read as expected since byte[-1] gave an out of bounds exception. The results are the same. Sometimes it writes. Mostly it times out.
Any help is appreciated.

You can adjust the length of TCP timeouts on Blackberry using the parameter 'ConnectionTimeout'.
In your code here:
HttpConnection connection = (HttpConnection)Connector.open(url + updateConnectionSuffix());
You'll want to append ConnectionTimeout. You might write it into updateConnectionSuffix() or just append it.
HttpConnection connection = (HttpConnection)Connector.open(url + updateConnectionSuffix() + ";ConnectionTimeout=54321");
This sets the timeout to 54321 milliseconds.
Timeouts occur when the client is waiting for the server to send an ack and it doesn't get one in a specified amount of time.
edit: also, are you able to use the browser and stuff? You may also want to play with the deviceside parameter.

I think the problem may be that you're closing the connection before reading the bytes from the input stream. Try moving the connection.close() after the bytes have been read in.

File transfer from C++ client to Java server

I have a c++ client which needs to send a file to a c++ server. I'm splitting the file to chunks of PACKET_SIZE (=1024) bytes and send them over a TCP socket. At the server side I read at most PACKET_SIZE bytes to a buffer. When the client sends files which are less than PACKET_SIZE, the server receives more bytes than sent. Even when I limit the number of bytes to be exactly the size of the file, the files differ. I know the problem does not have to do with the client because I've tested it with a c++ server and it works flawlessly.
Thanks.
Server:
public void run() {
DataInputStream input = null;
PrintWriter output = null;
try {
input = new DataInputStream (_client.getInputStream());
}
catch (Exception e) {/* Error handling code */}
FileHeader fh = recvHeader(input);
size = fh._size;
filename = fh._name;
try {
output = new PrintWriter(_client.getOutputStream(), true);
}
catch (Exception e) {/* Error handling code */}
output.write(HEADER_ACK);
output.flush();
FileOutputStream file = null;
try {
file = new FileOutputStream(filename);
}
catch (FileNotFoundException fnfe) {/* Error handling code */}
int total_bytes_rcvd = 0, bytes_rcvd = 0, packets_rcvd = 0;
byte [] buf = new byte [PACKET_DATA_SIZE];
try {
int max = (size > PACKET_DATA_SIZE)? PACKET_DATA_SIZE: size;
bytes_rcvd = input.read(buf,0, max);
while (total_bytes_rcvd < size) {
if (-1 == bytes_rcvd) {...}
++packets_rcvd;
total_bytes_rcvd += bytes_rcvd;
file.write (buf,0, bytes_rcvd);
if (total_bytes_rcvd < size)
bytes_rcvd = input.read(buf);
}
file.close();
}
catch (Exception e) {/* Error handling code */}
}
Client:
char packet [PACKET_SIZE] ;
file.open (filename, ios::in | ios::binary);//fopen (file_path , "rb");
int max = 0;
if (file.is_open()) {
if (size > PACKET_SIZE)
max = PACKET_SIZE;
else
max = size;
file.read (packet , max);
}
else {...}
int sent_packets = 0;
while (sent_packets < (int) ceil (((float)size)/PACKET_SIZE) ) {
_write=send(_sd , packet, max,0);
if (_write <0) {...}
else {
++sent_packets;
if (size > PACKET_SIZE* sent_packets) {
if (size - PACKET_SIZE* sent_packets >= PACKET_SIZE)
max = PACKET_SIZE;
else
max = size - PACKET_SIZE* sent_packets;
file.read (packet , max);
}
}
}

Is the sending socket closed at the end of the file, or is the next file streamed over the same socket? If more than one file is streamed, you could pick up data from the next file if you have the endedness wrong for the file size in recvHeader(), i.e. you send a file of length 0x0102 and try to read one of length 0x0201.
Other question, why do you provide a max for the first read, but not for the following reads on the same file?

One issue I see is that it appears that you assume that if the send returns a non-error, that it sent the entire chunk you requested it to send. This is not necessarily true, especially with stream sockets. How large are the packets you are sending, and how many? The most likely reason this could occur would be if the sndbuf for the socket filled, and your socket _sd is set to non-blocking. I'm not positive (depends on stack implementation), but I believe it could also likely occur if the TCP transmit window was full for your connection, and tcp couldn't enqueue your entire packet.
You should probably loop on the send until max is sent.
Thusly:
int send_ct=0;
while( (_write = send(_sd, packet + send_ct, max-send_ct, 0)) > 0) {
send_ct += _write;
if(send_ct >= max) {
break;
} else {
// Had to do another send
}
}

the code is not complete. E.g. you have omitted the sending of the filename and the filesize, as well as the parsing of those values. Are those values correct? If not first ensure that these values are the right ones before investigating further.