We Keep Coding

TCP sending/receiving missing bytes

I am using C# to create a server software for Windows and Java to create the client software.
It works fine most of the time, except for those few exceptions that I don't understand.
I am generally using .ReadLine() and .WriteLine() on both ends to communicate, unless I try to send binary data. That's when I write and read the bytes directly.
This is how the software is supposed work:
Client requests the binary data
Server responds with the length of the binary data as a string
Client receives the length and converts it into an integer and starts reading (length) bytes
Server starts writing (length) bytes
It works in most cases, but sometimes the client app doesn't receive the full data and blocks. The server always immediately flushes after writing data, so flushing is not the problem.
Furthermore I've noticed this usually happens with larger files, small files (up to ~1 MB) usually are not a problem.
NOTE It seems like the C# server does send the data completely, so the problem is most likely somewhere in the Java code.
EDIT - Here are some logs from the client side
Working download: pastebin.com/hFd5TvrF
Failing download: pastebin.com/Q3zFWRLB
It seems like the client is waiting for 2048 bytes at the end (as it should be, as length - processed = 2048 in this case), but for some reason the client blocks.
Any ideas what I'm doing wrong? Below are the source codes of both server and client:
C# Server:
public void Write(BinaryWriter str, byte[] data)
{
int BUFFER = 2048;
int PROCESSED = 0;
// WriteString sends the String using a StreamWriter (+ flushing)
WriteString(data.Length.ToString());
while (PROCESSED < data.Length)
{
if (PROCESSED + BUFFER > data.Length)
BUFFER = data.Length - PROCESSED;
str.Write(data, PROCESSED, BUFFER);
str.Flush();
PROCESSED += BUFFER;
}
}
Java Client:
public byte[] ReadBytes(int length){
byte[] buffer = new byte[length];
int PROCESSED = 0;
int READBUF = 2048;
TOTAL = length;
progress.setMax(TOTAL);
InputStream m;
try {
m = clientSocket.getInputStream();
while(PROCESSED < length){
if(PROCESSED + READBUF > length)
READBUF = length - PROCESSED;
try {
PROCESSED += m.read(buffer, PROCESSED, READBUF);
} catch (IOException e) {
}
XPROCESSED = PROCESSED;
}
} catch (IOException e1) {
// Removed because of sensitive data
}
return decryptData(buffer);
}

I've found a fix. As of now, the server sends the length and right after sends the byte array. For some reason this does not work.
So what I've changed is:
Send length and wait for the client to respond with "OK"
Start writing bytes
Not sure why, but it works. Ran it in a while(true) loop and it's been sending data 1000 times in 4 minutes straight and no problems, so I guess it's fixed.

Cant read with a boost C++ client from a java server

I am trying to create a simple messaging application, using a C++ client and a Java server.
It seems I can't get around because the C++ client fails to get information from the Java server, yet I can't track down the problem.
I have tried connecting with a Java client to the Java server and it works well.
I have tried connecting with the C++ Client to a simple C++ echo server I made for this purpose, and everything is going well (it reads the information).
I have tried connecting with the java client to the c++ echo server and it works well.
Keep in mind that the Java server get all the information from the client, and is responding (eg: when I try to Log in, the server gets it, logs me in, and "sends" a http response setting the cookie and displaying a welcome message, but the client never gets it).
Here is the java code which send the reply:
while ((msg = tokenizer.nextMessage()) != null)
{
System.out.println("Received \"" + msg + "\" from client");
String response = (String)protocol.processMessage(msg);
System.out.println(response); // used for testing
if (response != null)
{
clientSocket.getOutputStream().write(response.getBytes("UTF-8"));
//the Out below this line is being initialized on connection, just put it here for you to read
//Also the out.println(response) doesn't work as well, those are 2 attempts i have made
//out = new PrintWriter(new OutputStreamWriter(clientSocket.getOutputStream(), "UTF-8"), true);
//out.println(response);
}
if (protocol.isEnd(msg))
{
break;
}
}
Here is the client side code (C++):
//while (_socket.available() == 0){
// boost::this_thread::sleep(boost::posix_time::milliseconds(10));
// std::cout << "test";
//}
char reply[max_length];
size_t reply_length = boost::asio::read(_socket,boost::asio::buffer(reply,10));
//size_t reply_length = boost::asio::read(_socket, boost::asio::buffer(reply, _socket.available()));
std::cout << "Reply is:\n";
std::cout.write(reply, reply_length);
std::cout << "\n";
Note that the while above in the start of the code is used in order to wait for the response after each sent message, I have tried replacing it with a longer sleep time so I wont have to check the size of the incoming buffer, as you can see just after it I am trying to read a buffer the size of 10, just from testing, and i put the "real" read line in a comment just after it.
Any help would be appreciated.
Thank you.
EDIT - Forgot to mention that if I close the socket after sending the information passes, but doing so fails the purpose, as I am trying to keep the socket open until the client performs a log out.
EDIT #2- I have tried a diffrent method of reading, by using a delimiter char and reaing the buffer 1 char at the time, it just get stuck blocking with the empty buffer.
Here is the code for the second type of reading which i have tried:
std::string respone = "";
char ch='0';
boost::system::error_code error;
try {
while(!error && ch!='$'){
size_t tmp=0;
try {
std::cout << "going to read:";
tmp = _socket.read_some(boost::asio::buffer(&ch + tmp, 1 - tmp), error);
std::cout << "finish reading 1 char";
if (error)
throw boost::system::system_error(error);
}
catch (std::exception& e) {
std::cerr << "recv failed (Error: " << e.what() << ')' << std::endl;
}
respone.append(1, ch);
}
}

A bit silly, but apparntly my anti virus was blocking all the packets from the java server (and not the c++ for some reason), turning it off solved every thing.

How to manage lots of incoming packets

I have a socketserver set up with a remote client, and it is functional. Upon opening the client and logging in, I noticed that sometimes, there is an error that seems to be due to the client reading an int when it shouldn't be.
Upon logging on, the server sends a series of messages/packets to the client, and these are anything from string messages to information used to load variables on the client's side.
Occasionally, while logging in, an error gets thrown showing that the client has read a packet of size 0 or a very large size. Upon converting the large-sized number into ascii I once found that it was a bit of a string "sk." (I located this string in my code so it's not entirely random).
Looking at my code, I'm not sure why this is happening. Is it possible that the client is reading an int at the wrong time? If so, how can I fix this?
InetAddress address = InetAddress.getByName(host);
connection = new Socket(address, port);
in = new DataInputStream(connection.getInputStream());
out = new DataOutputStream(connection.getOutputStream());
String process;
System.out.println("Connecting to server on "+ host + " port " + port +" at " + timestamp);
process = "Connection: "+host + ","+port+","+timestamp + ". Version: "+version;
write(0, process);
out.flush();
while (true) {
int len = in.readInt();
if (len < 2 || len > 2000) {
throw new Exception("Invalid Packet, length: "+len+".");
}
byte[] data = new byte[len];
in.readFully(data);
for (Byte b : data) {
System.out.printf("0x%02X ",b);
}
try {
reader.handlePackets(data);
} catch (Exception e) {
e.printStackTrace();
//connection.close();
//System.exit(0);
//System.out.println("Exiting");
}
}
//Here is code for my write function (Server sided):
public static void write(Client c, Packet pkt) {
for (Client client : clients) {
if (c.equals(client)) {
try {
out.writeInt(pkt.size());
out.write(pkt.getBytes());
out.flush();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
So looking at the write function, I don't really see how it could be confusing the client and making it read for the size of the packet twice for one packet (at least that's what I think is happening).
If you need more information please ask me.

The client side code looks fine, and the server side code looks fine too.
The most likely issue is that this is some kind of issue with multi-threading and (improper) synchronization. For example, maybe two server-side threads are trying to write a packet to the same client at the same time.
It is also possible that your Packet class has inconsistent implementations of size() and getBytes() ... or that one thread is modifying a Packet objects while a second one is sending it.

Unable to send large files using tcp socket from Java client to C++ server

I'm writing a C++ that has a feature of receiving file via socket from a Java client.
The file's size I'm trying to send is 510KB, but the server only receives 46KB and then gets stuck "waiting" for more bytes.
This is the server code: (I tried to receive 510KB at once, but the server got stuck, so I tried also to receive 64KB each iteration and again, the server got stuck).
int bytes_to_receive;
size_t length;
while(file_length > 0)
{
if(file_length >= 65536)
bytes_to_receive = 65536;
else
bytes_to_receive = file_length;
char buf5[bytes_to_receive];
length = 0;
while( length < bytes_to_receive )
length += socket.read_some(boost::asio::buffer(&buf5[length], bytes_to_receive - length), error);
string temp(buf5);
file_parts.push_back(temp);
file_length -= bytes_to_receive;
}
string file("");
vector<string>::const_iterator it;
for(it = file_parts.begin(); it != file_parts.end(); ++it)
{
file += *it;
}
The Java client load the file to a string, and then sends the string using
writeBytes
Note: When sending a 1KB file everything works.
Why does this happens and how can I fix it? Any help would be highly appreciated.
Edit: any other way of receiving large data using boost would be appreciated.

Java - UDP sending data over socket.. not rec. all data

It would seem that the Client - Server application i wrote does work however it seems that not all data is processed every time.
I am testing it on a local machine in Eclipse env.
Server:
private void sendData() throws Exception
{
DatagramPacket data = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);
InputStream fis = new FileInputStream(responseData);
int a;
while((a = fis.read(outgoingData,0,512)) != -1)
{
serverSocket.send(data);
}
}
Client:
private void receiveData() throws Exception
{
DatagramPacket receiveData = new DatagramPacket(incomingData, incomingData.length);
OutputStream fos = new FileOutputStream(new File("1"+data));
while(true)
{
clientSocket.receive(receiveData);
fos.write(incomingData);
}
}
I used to have if else in the while(true) loop to check if packet length is less than 512 bytes so it knew when to break;
I was thinking there was a problem whit that but seems that was oke for now i just wait few minutes and then stop the Client.java app
The file does transfer but the original file is 852kb and so far i got 777, 800, 850,.. but never all of it.

The fundamental problem with your approach is that UDP does not guarantee delivery. If you have to use UDP (rather than, say, TCP), you have to implement a scheme that would detect and deal with packets that got lost, arrive out of order, or are delivered multiple times.
See When is it appropriate to use UDP instead of TCP?

You should probably use TCP to transfer files. You are probably losing packets because you are sending them so fast in that while loop.
int a;
while((a = fis.read(outgoingData,0,512)) != -1)
{
serverSocket.send(data);
}
since you're sending so fast I highly doubt it will have a chance to be received in the right order. some packets will probably be lost because of it too.
Also since your sending a fixed size of 512 bytes the last packet you send will probably not be exactly that size, so you will see the end of the file "look wierd."