I'm programming a server (Java) - client (Android/Java) application. The server is a W7. All the communication goes well until one read in the client that freezes and stops reading data until I send it 2 times more.
The data not read is a byte array. I repeat that all the communication goes well until this point.
Here's the code that I use to send the data:
Long lLength = new Long(length);
byte [] bLength = this.longToBytes(lLength.longValue());
dos.write(bLength);
dos.flush();
dos.write(bLength);
dos.flush();
dos.write(bLength);
dos.flush();
This code transforms a long value into an 8 bytes array. As I said, when the first write is executed (and the client is waiting for data), the read is not done. It is not done until I execute the last write().
And here's the code to read it:
byte length[] = {0,0,0,0,0,0,0,0};
dis.read(length);
I've used Wireshark to sniff the traffic, and I can see that the byte array is send, and the client answers with an ACK, but the read is not done.
In the client and the server, the sockets are setup like this:
socket = new Socket(sIP, oiPort.intValue());
dos = new DataOutputStream(socket.getOutputStream());
dis = new DataInputStream(socket.getInputStream());
This is driving me mad... I don't know why, at one moment, the application stops reading the data, when I send it the same way as always.
I suppose that the problem may be in the input buffers of the client socket... But I don't know what to do or what to try...
Say that I've also test the server in a WXPSP3 and it still doesn't work.
First thing I'd look at is the code for your longToBytes method. Is it really creating a byte array of 8 bytes? If it is generating an array of less than 8 bytes, then that explains the problem. (Your client is expecting 8 bytes, and will block until they all arrive.)
Next thing I'd ask myself is why I'm not just using writeLong and readLong. It would simplify your code, and quite possibly cure the problem.
Related
In my system, I am using tcp to transfer messages between an android application, and a desktop application (that is developed in Qt).
I am using msgpack library to serialize/deserialize structures in both applications. In order to make sure that the structure is always received as a whole, I always send the number of bytes (encoded as a 32-bit big endian unsigned integer not using msgpack so that it always takes the first 4 bytes) before the message itself (that is encoded in msgpack). If there is a better way please tell me.
In the android app, I read the first 4 bytes into a byte[] and decode it into a long msgSize, next I need a way for the thread to keep blocking until the next msgSize bytes are received. After reading other questions and answers, I think I can write something like this:
InputStream is= sock.getInputStream();
byte[] msgSizeBuff= new byte[4];
is.read(msgSizeBuff, 0, 4);
long msgSize= MyDecodeFunction(msgSizeBuff);
DataInputStream dis = new DataInputStream(is);
byte[] msg = new byte[msgSize];
dis.readFully(msg);
After that I can use the msg array with msgpack, and I am sure that the whole message is received.
So:
is my usage of DataInputStream class alright, and am I guranteed that I will wait until I receive the specified number of bytes? because in the reference they say:
If insufficient bytes are available, EOFException is thrown.
I want a way to tell this thread to cancel the operation, so what would happen if another thread calls socket.close() or socket.shutdownInput(), will I get an exception?
I have several questions-
1. I have two computers connected by socket connection. When the program executes
outputStream.writeInt(value);
outputStream.flush();
what actually happens? Does the program wait until the other computer reads the integer value?
2. How can I empty the outputStream or inputStream? Meaning, when emptying
the outputStream or inputStream, whatever is written to that stream gets removed.
(please don't suggest to do it by closing the connection!)
I tried to empty the inputStream this way-
byte[] eatup=new byte[20*1024];
int available=0;
while(true)
{
available=serverInputStream.available();
if(available==0)
break;
serverInputStream.read(eatup,0,available);
}
eatup=null;
String fileName=(String)serverInputStream.readObject();
Program should not process the line as nothing else is being written on the outputStream .
But my program executes it anyway and throws a java.io.OptionalDataException error.
Note: I am working on a client-server file transfer project. The client sends files to
the server. The second code is for server terminal. If 'cancel button' is pressed on server
end then it stops reading bytes from the serverInputStream and sends a signal(I used int -1)
to the client. When client receieves this signal it stops sending data to the server, but I've
noticed that serverInputStream is not empty. So I need to empty this serverInputStream so that
the client computer is able to send the server computer files again(That's why I can't manage a lock
from read method)
1 - No. On the flush() the data will be written to the OS kernel which will likely immediately hand it to the network card driver, which in turn will send it to the receiving end. In a nutshell the send is fire and forget.
2 - As Jeffrey commented available() is not reliable for this sort of operation. If doing blocking IO then as he suggests you should just use read() speculatively. However it should be said that you really need to define a protocol on top of the raw streams, even if it's just using DataInput/DataOutputStream. When using raw write/read the golden rule is one write != one read. For example if you were to write 10 bytes on one side and had a reading loop on the other there is no guarantee that one read will read all 10 bytes. It may be "read" as any combination of chunks. Similarly two writes of 10 bytes might appear as one read of 20 bytes on the receiving side. Put another way there is no concept of a "packet" unless you create a higher level protocol on top of the raw bytes to do packets. An example would be each send is prefixed by a byte length so the receiving side knows how much data to expect in the current packet.
If you do need to do anything more complicated than a basic apps I strongly encourage you to investigate some higher level libraries that have solved many of the gnarly issues of network IO. I would recommend Netty which I use for production apps. However it is quite a big leap in understanding from a simple IO stream to Netty's more event based system. There may be other libraries somewhere in the middle.
I am trying to read some data from a network socket using the following code -
Socket s = new Socket(address, 502);
response = new byte[1024];
InputStream is = s.getInputStream();
int count = is.read(response, 0, 100);
The amount of data isn't large. It is 16 bytes in total. However the read() statement does not read all the data in one go. It reads only 8 bytes of data into my buffer.
I have to make multiple calls to read() like this in order to read the data -
Socket s = new Socket(address, 502);
response = new byte[1024];
InputStream is = s.getInputStream();
int count = is.read(response, 0, 100);
count += is.read(response, count, 100-count);
Why is this happening? Why does read() not read the entire stream in one go?
Please note that the data is not arriving gradually. If I wait for 2 seconds before reading the data by making a call to Thread.sleep(2000) the behavior remains the same.
Why does read() not read the entire stream in one go?
Because it isn't specified to do so. See the Javadoc. It blocks until at least one byte is available, then returns some number between 1 and the supplied length, inclusive.
That in turn is because the data doesn't necessarily arrive all in one go. You have no control over how TCP sends and receives data. You are obliged to just treat it as a byte stream.
I understand that it blocks until data arrives. "That in turn is because the data doesn't necessarily arrive all in one go." Why not is my question.
The data doesn't necessarily all arrive in one go because the network typically breaks it up into packets. IP is a packet switching protocol.
Does TCP transmit it blocks of 8 bytes?
Possibly, but probably not. The packet size depends on the network / networks that the data has traversed, but a typical internet packet size is around 1500 bytes.
If you are getting 8 bytes at a time, either your data is either coming through a network with an unusually small packet size, or (more likely) the sender is sending the data 8 bytes at a time. The second explanation more or less jives with what your other comments say.
And since I explicitly specify 100, a number much larger than the data in buffer shouldn't it attempt to read up till atleast 100 bytes?
Well no. It is not specified to work that way, and it doesn't work that way. You need to write your code according to what the spec says.
It is possible that this has something to do with the way the device is being "polled". But without looking at the specs for the device (or even knowing what it is exactly) this is only a guess.
Maybe the data is arriving gradually not because of your reading but because of the sender.
The sender should use a BufferedOutputStream (in the middle) to make big chunks before sending (and use flush only when it's needed).
I have written a small program which send/receives files from one client to another. I've set up progressbars for both the receiver and the client, but the problem is that the sender seems to have his progressbar finish much quicker than the actual transfer. The problem lies with the how it calculates how many bytes that have been written. I'm assuming it's counting how many bytes I've read into buffer, rather than bytes that were sent through the network, so how can I find a solution to this problem? The receiver is calculating his received bytes at a correct rate, but the sender is not doing his part correctly.
Setting a lower buffersize offsets the difference a bit, but it's still not correct. I've tried wrapping the outputstream with a CountingOutputStream, but it returns the same result as the code snippet below. The transfer eventually completes correctly, but I need the proper "sent" values to update my progressbar, as in what was actually received and written to disc at the receiver side. I've included a very stripped down code snippet which represents my way of calculating transferred bytes. Any examples of a solution would be very helpful.
try
{
int sent = 0;
Socket sk = new Socket(ip, port);
OutputStream output = sk.getOutputStream();
FileInputStream file = new FileInputStream(filepath);
byte[] buffer = new byte[8092];
while ((bytesRead = file.read(buffer)) > 0)
{
output.write(buffer, 0, bytesRead);
sent += bytesRead;
System.out.println(sent); // Shows incorrect values for the actual speed.
}
}
In short, I don't think you can get the sort of accurate visibility you're looking for solely from the "sender" side, given the number of buffers between you and the "wire" itself. But also, I don't think that matters. Here's why:
Bytes count as "sent" when they are handed to the network stack. When you are sending a small number of bytes (such as your 8K example) those bytes are going to be buffered & the write() calls will return quickly.
Once you're reached network saturation, your write() calls will start to block as the various network buffers become full - and thus then you'll get a real sense of the timings.
If you really must have some sort of "how many bytes have you received?" you'll have to have the receiving end send that data back periodically via an out-of-band mechanism (such as suggested by glowcoder)
Get the input stream from the socket, and on the other side, when you've written a selection of bytes to disk, write the result to the output stream. Spawn a second thread to handle the reading of this information, and link it to your counter.
Your variable is sent - it is accurate. What you need is a received or processed variable, and for that you will need two-way communication.
I have a Java socket server and the the connection socket working just fine. What I need help with is streaming a response back to the client.
I get the output stream with socket.getOutputStream(). How can I make it so that when I write to the output stream it is immediately sent, but in the future on the same connection I can send another chunk of data.
I tried simply using write and write in conjunction with flush, but I don't really know what I am doing...
Depending on native implementation, the socket may have a buffer, and not send the bytes the second you call write(). flush() however, will force the bytes to be sent. Typically it is good practice to send larger chunks rather than byte by byte (for streaming you generally start by building up a buffer on the receiver's side). Optimal network usage is likely to be to send as large packets as possible (limited by MTU). To have a local buffer in java, wrap the socket outputstream in a BufferedOutputStream.
flush() will force the data to be sent to the OS. The OS can buffer the data and so can the OS on the client. If you want the OS to send data earlier, I suggest you try turning Nagle's Algorithm off. socket.setTcpNoDelay(true); However, you will find that OS/driver parameters can still introduce some buffering/packet coelesing.
If you look at Sun's JDK 6 java.net.SocketOutputStream you will see the flush() method does nothing. This is not guarenteed to be the case on all platforms and a flush() may be required.
Another solution could be DataOutputStream
DataOutputStream dataOut = new DataOutputStream(socket.getOutputStream());
dataOut.writeInt(1)