The problem I am having is that when I use an InputStream to read bytes, it blocks until the connection is finished. EG:
InputStream is = socket.getInputStream();
byte[] buffer = new byte[20000];
while (is.read(buffer) != -1) {
System.out.println("reading");
}
System.out.println("socket read");
"socket read" doesn't print out until the FYN packet is actually recieved, thus closing the connection. What is the proper way to receive all the bytes in without blocking and waiting for the connection to drop?
Take a look at java.nio which has non-blocking IO support.
Reading till you get -1 means that you want to read until EOS. If you don't want to read until EOS, don't loop till the -1: stop sooner. The question is 'when?'
If you want to read a complete 'message' and no more, you must send the message in such a way that the reader can find its end: for example, a type-length-value protocol, or more simply a size word before each message, or a self-describing protocol such as XML.
With traditional sockets the point is that usually you do want them to block: what you do when logically you don't want your program to block is you put your reading/writing code in another thread, so that the separate read/write thread blocks, but not your whole program.
Failing that, you can use the available() method to see if there is actually any input available before reading. But then you need to be careful not to sit in a loop burning CPU by constantly calling available().
Edit: if the problem is that you're happy to block until the bytes have arrived, but not until the connection has dropped (and that is what is happeningh), then you need to make the client at the other end call flush() on its output stream after it has sent the bytes.
Try this:
InputStream is = socket.getInputStream();
byte[] buffer = new byte[20000];
int bytesRead;
do {
System.out.println("reading");
bytesRead = is.read(buffer);
}
while (is.available() > 0 && bytesRead != -1);
System.out.println("socket read");
More info: https://docs.oracle.com/javase/1.5.0/docs/api/java/io/InputStream.html#available()
Example taken from exampledepot on java.nio
// Create a direct buffer to get bytes from socket.
// Direct buffers should be long-lived and be reused as much as possible.
ByteBuffer buf = ByteBuffer.allocateDirect(1024);
try {
// Clear the buffer and read bytes from socket
buf.clear();
int numBytesRead = socketChannel.read(buf);
if (numBytesRead == -1) {
// No more bytes can be read from the channel
socketChannel.close();
} else {
// To read the bytes, flip the buffer
buf.flip();
// Read the bytes from the buffer ...;
// see Getting Bytes from a ByteBuffer
}
} catch (IOException e) {
// Connection may have been closed
}
Be sure to understand buffer flipping because it causes a lot of headache. Basically, you have to reverse your buffer to read from it. If you are to reuse that buffer to have the socket to write in it, you have to flip it again. However clear() resets the buffer direction.
the code is probably not doing what you think it does.
read(buffer) returns the number of bytes it read, in other words: it is not guaranties to fill up your buffer anyway.
See DataInputStream.readFully() for code that fill up the entire array:
or you can use this functions (which are based on DataInputStream.readFully()) :
public final void readFully(InputStream in, byte b[]) throws IOException
{
readFully(in, b, 0, b.length);
}
public final void readFully(InputStream in, byte b[], int off, int len) throws IOException
{
if (len < 0) throw new IndexOutOfBoundsException();
int n = 0;
while (n < len)
{
int count = in.read(b, off + n, len - n);
if (count < 0) throw new EOFException();
n += count;
}
}
Your code would look like:
InputStream is = socket.getInputStream();
byte[] buffer = new byte[20000];
readFully(is, buffer);
System.out.println("socket read");
Related
I'm trying to serialize Object between NIO SocketChannel and blocking IO Socket. Since I can't use Serializable/writeObject on NIO, I thought to write code to serialize object into an ByteArrayOutputStream then send array length followed by array.
Sender function is
public void writeObject(Object obj) throws IOException{
ByteArrayOutputStream serializedObj = new ByteArrayOutputStream();
ObjectOutputStream writer = new ObjectOutputStream(serializedObj);
writer.writeUnshared(obj);
ByteBuffer size = ByteBuffer.allocate(4).putInt(serializedObj.toByteArray().length);
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
}
and receiver is:
public Object readObject(){
try {
//Leggi dimensione totale pacchetto
byte[] dimension = new byte[4];
int byteRead = 0;
while(byteRead < 4) {
byteRead += this.getInputStream().read(dimension, byteRead, 4 - byteRead);
}
int size = ByteBuffer.wrap(dimension).getInt(); /* (*) */
System.out.println(size);
byte[] object = new byte[size];
while(size > 0){
size -= this.getInputStream().read(object);
}
InputStream in = new ByteArrayInputStream(object, 0, object.length);
ObjectInputStream ois = new ObjectInputStream(in);
Object res = ois.readUnshared();
ois.close();
return res;
} catch (IOException | ClassNotFoundException e) {
return null;
}
}
The problem is that size (*) is always equals to -1393754107 when serializedObj.toByteArray().length in my test is 316.
I don't understand why casting not works properly.
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
If the result of getSocket() is a SocketChannel in non-blocking mode, the problem is here. You aren't checking the result of write(). In non-blocking mode it can write less than the number of bytes remaining in the ByteBuffer; indeed it can write zero bytes.
So youu aren't writing all the data you think you're writing, so the other end overruns and reads the next length word as part of the data being written, and reads part of the next data as the next length word, and gets a wrong answer. I'm surprised it didn't barf earlier. In fact it probably did, but your deplorable practice of ignoring IOExceptions masked it. Don't do that. Log them.
So you need to loop until all requested data has been written, and if any write() returns zero you need to select on OP_WRITE until it fires, which adds a considerable complication into your code as you have to return to the select loop while remembering that there is an outstanding ByteBuffer with data remaining to be written. And when you get the OP_WRITE and the writes complete you have to deregister interest in OP_WRITE, as it's only of interest after a write() has returned zero.
NB There is no casting in your code.
The problem was write() returned 0 always. This happens because the buffer wasn't flipped before write().
I am using java comm library to try accomplish a simple read/write to a serial port. I am able to successfully write to the port, and catch the return input from the input stream, but when I read from the input stream I am only able to read 1 byte (when I know there should be 11 returned)
I can write to the port successfully using Putty and am receiving the correct return String there. I am pretty new to Java, buffers and serial i/o and think maybe there is some obvious syntax or understanding of how data is returned to the InputStream. Could someone help me? Thanks!
case SerialPortEvent.DATA_AVAILABLE:
System.out.println("Data available..");
byte[] readBuffer = new byte[11];
try {
System.out.println("We trying here.");
while (inputStream.available() > 0) {
int numBytes = inputStream.read(readBuffer, 1, 11);
System.out.println("Number of bytes read:" + numBytes);
}
System.out.println(new String(readBuffer));
} catch (IOException e) {System.out.println(e);}
break;
}
This code returns the following output:
Data available..
We trying here.
Number of bytes read:1
U
As the documentation states
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read.
This behavior is perfectly legal. I would also expect that a SerialPortEvent.DATA_AVAILABLE does not guarantee that all data is available. It's potentially just 1 byte and you get that event 11 times.
Things you can try:
1) Keep reading until you have all your bytes. E.g. wrap your InputStream into a DataInputStream and use readFully, that's the simplest way around the behavior of the regular read method. This might fail if the InputStream does not provide any more bytes and signals end of stream.
DataInputStream din = new DataInputStream(in);
byte[] buffer = new byte[11];
din.readFully(buffer);
// either results in an exception or 11 bytes read
2) read them as they come and append them to some buffer. Once you have all of them take the context of the buffer as result.
private StringBuilder readBuffer = new StringBuilder();
public void handleDataAvailable(InputStream in) throws IOException {
int value;
// reading just one at a time
while ((value = in.read()) != -1) {
readBuffer.append((char) value);
}
}
Some notes:
inputStream.read(readBuffer, 1, 11)
Indices start at 0 and if you want to read 11 bytes into that buffer you have to specify
inputStream.read(readBuffer, 0, 11)
It would otherwise try to put the 11th byte at the 12th index which will not work.
This question already has answers here:
Can't get out the receive file loop in send file using socket java
(2 answers)
Closed 9 years ago.
I am developing a simple file tranfer app between client and server.
Here is my code:
My Client(Sender):
try{
File file_sender=XMLParser.register(peername, address, port, listfile);
int count;
byte[] buffer = new byte[1024];
int len=(int) file_sender.length();
byte[] mybytearray = new byte[len];
DataOutputStream output=new DataOutputStream(Info.connection.getOutputStream());
output.writeInt(len);
System.out.println(len);
BufferedInputStream bis=new BufferedInputStream(new FileInputStream(file_sender));
bis.read(mybytearray, 0, len);
output.write(mybytearray, 0, len);
bis.close();
output.close();
// Info.connection.close();
}catch(Exception ex){
System.out.println(ex.toString());
}
My server(Receiver):
public class IOThread extends Thread {
private Socket connection;
private DataInputStream input;
public IOThread(Socket connection) throws IOException{
this.connection=connection;
input=new DataInputStream(connection.getInputStream());
}
#Override
public void run(){
while(true){
try{
int filesize=12022386;
int bytesRead;
int currentTot = 0;
byte[] bytearray = new byte [filesize];
int len=input.readInt();
FileOutputStream fos = new FileOutputStream("data.xml");
BufferedOutputStream bos = new BufferedOutputStream(fos);
bytesRead=input.read(bytearray, 0,bytearray.length);
System.out.println(len);
currentTot=bytesRead;
do{
bytesRead=input.read(bytearray, currentTot, (bytearray.length-currentTot));
if(bytesRead>=0) currentTot+=bytesRead;
System.out.println("pos: "+currentTot);
} while(len!=currentTot);
bos.write(bytearray, 0, currentTot);
bos.close();
//connection.close();
System.out.println("Done");
}catch(EOFException ex){
System.out.println(ex.toString());
break;
}catch( Exception ex){}
}
}
I want to tranfer more than one file, so I don't want to close socket. So, I use variable "len" to check the file is tranfered completely.
If I close "output" after send file then server thows EOFException and the file is sent success.
When I don't close output, server can't receive file success. But Server doesn't throws EOFException.
Can you help me solve this problem?
Update: Here is my output on the console screen if I don't close variable "output" at client:
246
pos: 496
Thanks.
Sorry for my poor English
Two things:
First of all, you seem to be ignoring the file length in your receiving code? You have:
int filesize=12022386;
int bytesRead;
int currentTot = 0;
byte[] bytearray = new byte [filesize];
int len=input.readInt();
You are sizing bytearray to 12022386 bytes regardless of the value of len, and you are asking for that many bytes from the input stream.
Secondly, when you close the output stream, any cached/buffered data that hasn't been written yet is automatically flushed (i.e., in your case, sent to the server).
Flush the output stream explicitly when you're done sending the first file to force it to finish sending all of the data to the server. You can then continue using that output stream for other things.
Do output.flush() in your sending code when the transfer is complete.
The problem is that you read beyond the size of your file, i.e. instead of reading len bytes, you read bytearray.length bytes, which is larger.
Consequently, you read more than len bytes, so len!=currentTot is never satisfied, because, as you can see from your sample output, currentTot == 496 and len == 246.
Make the following changes in your while loop:
do{
bytesRead=input.read(bytearray, currentTot, (len - currentTot));
if(bytesRead>=0) currentTot += bytesRead;
System.out.println("pos: " + currentTot);
} while(len != currentTot);
Just to make sure you don't end up in an endless loop due to a similar error, you could use currentTot < len instead of len != currentTot as your condition.
Also, since you're already using DataInputStream, consider reading the socket contents using readFully. This blocks until a given number of bytes is read from the socket, essentially eliminating your need for the while loop. Read more here.
Throw all the loops away and use DataInputStream.readFully(); close the socket after sending the file; don't s end the file length; and use a normal size buffer, say 8192 bytes.
Your BufferedOutputStream filling seems incorrect. Below code must be inside the while loop.
bos.write(bytearray, 0, currentTot);
Try something like this instead:
BufferedOutputStream buffOut=new BufferedOutputStream(fos);
byte []arr = new byte [1024 * 1024];
int available = -1;
while((available = buffIn.read(arr)) != -1) {
buffOut.write(arr, 0, available);
}
buffOut.flush();
buffOut.close();
And test again.
Edit: Updated my answer with #Jason's correct comment.
This is my code, I'm using rxtx.
public void Send(byte[] bytDatos) throws IOException {
this.out.write(bytDatos);
}
public byte[] Read() throws IOException {
byte[] buffer = new byte[1024];
int len = 20;
while(in.available()!=0){
in.read(buffer);
}
System.out.print(new String(buffer, 0, len) + "\n");
return buffer;
}
the rest of code is just the same as this, i just changed 2 things.
InputStream in = serialPort.getInputStream();
OutputStream out = serialPort.getOutputStream();
They are global variables now and...
(new Thread(new SerialReader(in))).start();
(new Thread(new SerialWriter(out))).start();
not exist now...
I'm sending this (each second)
Send(("123456789").getBytes());
And this is what i got:
123456789123
456789
123456789
1234567891
23456789
can anybody help me?
EDIT
Later, i got the better way to solve it. Thanks, this was the Read Code
public byte[] Read(int intEspera) throws IOException {
try {
Thread.sleep(intEspera);
} catch (InterruptedException ex) {
Logger.getLogger(COM_ClComunica.class.getName()).log(Level.SEVERE, null, ex);
}//*/
byte[] buffer = new byte[528];
int len = 0;
while (in.available() > 0) {
len = in.available();
in.read(buffer,0,528);
}
return buffer;
}
It was imposible for me to erase that sleep but it is not a problem so, thanks veer
You should indeed note that InputStream.available is defined as follows...
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream. The next invocation might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.
As you can see, this is not what you expected. Instead, you want to check for end-of-stream, which is indicated by InputStream.read() returning -1.
In addition, since you don't remember how much data you have already read in prior iterations of your read loop, you are potentially overwriting prior data in your buffer, which is again not something you likely intended.
What you appear to want is something as follows:
private static final int MESSAGE_SIZE = 20;
public byte[] read() throws IOException {
final byte[] buffer = new byte[MESSAGE_SIZE];
int total = 0;
int read = 0;
while (total < MESSAGE_SIZE
&& (read = in.read(buffer, total, MESSAGE_SIZE - total)) >= 0) {
total += read;
}
return buffer;
}
This should force it to read up to 20 bytes, less in the case of reaching the end of the stream.
Special thanks to EJP for reminding me to maintain the quality of my posts and make sure they're correct.
Get rid of the available() test. All it is doing is telling you whether there is data ready to be read without blocking. That isn't the same thing as telling you where an entire message ends. There are few correct uses for available(), and this isn't one of them.
And advance the buffer pointer when you read. You need to keep track of how many bytes you have read so far, and use that as the 2nd parameter to read(), with buffer.length as the third parameter.
I'm developing Server-Client application and I have a problem with waiting for input data on input stream.
I have thread dedicated to reading input data. Currently it uses while loop to hold until data is available. (N.B. protocol is as follow: send size of packet, say N, as int then send N bytes).
public void run(){
//some initialization
InputStream inStream = sock.getInputStream();
byte[] packetData;
//some more stuff
while(!interrupted){
while(inStream.available()==0);
packetData = new byte[inStream.read()];
while(inStream.available()<packetData.length);
inStream.read(packetData,0,packetData.length);
//send packet for procession in other thread
}
}
It works but blocking the thread by while loop is IMO a bad idea. I could use Thread.sleep(X) to prevent resources being continously consumed by the loop, but there surely must be a better way.
Also I can not rely on InputStream.read to block the thread as part of the data may be sent by the server with delays. I have tried but it always resulted in unexpected behaviour.
I'd appreciate any ideas :)
You can use DataInputStream.readFully()
DataInputStream in = new DataInputStream(sock.getInputStream());
//some more stuff
while(!interrupted) {
// readInt allows lengths of up to 2 GB instead of limited to 127 bytes.
byte[] packetData = new byte[in.readInt()];
in.readFully(packetData);
//send packet for procession in other thread
}
I prefer to use blocking NIO which supports re-usable buffers.
SocketChannel sc =
ByteBuffer bb = ByteBuffer.allocateDirect(1024 *1024); // off heap memory.
while(!Thread.currentThread.isInterrupted()) {
readLength(bb, 4);
int length = bb.getInt(0);
if (length > bb.capacity())
bb = ByteBuffer.allocateDirect(length);
readLength(bb, length);
bb.flip();
// process buffer.
}
static void readLength(ByteBuffer bb, int length) throws EOFException {
bb.clear();
bb.limit(length);
while(bb.remaining() > 0 && sc.read(bb) > 0);
if (bb.remaining() > 0) throw new EOFException();
}
As UmNyobe said, available() is meant to be used if you dont want to block as the default behaviour is blocking.
Just use the normal read to read whatever is available but only send packet for processing in other thread once you have packetData.length bytes in your buffer...