We Keep Coding

Continues Reading Inputstream (serialport/tty)

I am continuously reading data from serialport/tty. Serialport spitting the data every 40 milliseconds. I am using java InputStream to read the data.
static final int buffersize = 208;
buffer = new byte[buffersize];
int dataSize = mInputStream.read(buffer);
if (dataSize > 0)
{
fpgaData = new String(buffer, 0, buffer.length);
}
But most of the time I don't get full data in one read.
E.g. 0000001 0000044 0001BF7 0000091 0000210 0000000 00000FF is the full data.
How to make sure that I get full desire data single read.

That is the serial port specification.
You have to design and program with consideration that one data can be notified separately and it can happen that multiple data are notified at once.

I solved it with BufferReader instread of InputStream reader.
try
{
BufferedReader bufferedReader = serialPort.GetBufferStream();
if (bufferedReader != null)
{
String contents = bufferedReader.readLine();
while (contents != null)
{
contents = bufferedReader.readLine();
}
}
} catch (IOException e)
{
e.printStackTrace();
return;
}
}

Socket using BufferedOutputStream/BufferedInputStream receives bogus data random

I have a client/server application that sends/receives data using BufferedOutputStream / BufferedInputStream . The protocol of communication is the following:
Send part :
first byte is the action to perform
next 4 bytes are the length of the message
next x bytes (x=length of message) are the message itself
Receive part :
read first byte to get the action
read the next 4 bytes to get the message length
read the x (obtained on prev step) bytes to get the message
Now the problem is that sometimes when i sent the length of the message (ex : 23045) on server part when i receive it i get a huge int (ex: 123106847).
A important clue is that this happens only when message exceeds a number of characters (in my case > 10K ) , if i sent a smaller message (ex 4-5k) everything works as expected.
Client send part (outputStream/inputStream are the type BufferedXXXStream):
private String getResponseFromServer( NormalizerActionEnum action, String message) throws IOException{
writeByte( action.id());
writeString( message);
flush(;
return read();
}
private String read() throws IOException{
byte[] msgLen = new byte[4];
inputStream.read(msgLen);
int len = ByteBuffer.wrap(msgLen).getInt();
byte[] bytes = new byte[len];
inputStream.read(bytes);
return new String(bytes);
}
private void writeByte( byte msg) throws IOException{
outputStream.write(msg);
}
private void writeString( String msg) throws IOException{
byte[] msgLen = ByteBuffer.allocate(4).putInt(msg.length()).array();
outputStream.write(msgLen);
outputStream.write(msg.getBytes());
}
private void flush() throws IOException{
outputStream.flush();
}
Server part (_input/_output are the type BufferedXXXStream)
private byte readByte() throws IOException, InterruptedException {
int b = _input.read();
while(b==-1){
Thread.sleep(1);
b = _input.read();
}
return (byte) b;
}
private String readString() throws IOException, InterruptedException {
byte[] msgLen = new byte[4];
int s = _input.read(msgLen);
while(s==-1){
Thread.sleep(1);
s = _input.read(msgLen);
}
int len = ByteBuffer.wrap(msgLen).getInt();
byte[] bytes = new byte[len];
s = _input.read(bytes);
while(s==-1){
Thread.sleep(1);
s = _input.read(bytes);
}
return new String(bytes);
}
private void writeString(String message) throws IOException {
byte[] msgLen = ByteBuffer.allocate(4).putInt(message.length()).array();
_output.write(msgLen);
_output.write(message.getBytes());
_output.flush();
}
....
byte cmd = readByte();
String message = readString();
Any help will be greatly appreciated. If you need additional details let me know.
UPDATE: Due to comments from Jon Skeet and EJP i realized that the read part on the server was having some pointless operations but letting this aside i finally got what the problem was: the key thing is that i keep the streams opened for the full length of the app and the first several times i sent the message length i'm able to read it on the server side BUT as Jon Skeet pointed out the data doesn't arrive all at once so when i try to read the message length again i'm actually reading from the message itself that is why i have bogus message lengths .
~ instead of sending the data length and then reading it all at once i sent it without the length and i read one byte at a time till the end of the string which works perfectly
private String readString() throws IOException, InterruptedException {
StringBuilder sb = new StringBuilder();
byte[] bytes = new byte[100];
int s = 0;
int index=0;
while(true){
s = _input.read();
if(s == 10){
break;
}
bytes[index++] = (byte) (s);
if(index == bytes.length){
sb.append(new String(bytes));
bytes = new byte[100];
index=0;
}
}
if(index > 0){
sb.append(new String(Arrays.copyOfRange(bytes, 0, index)));
}
return sb.toString();
}

Look at this:
byte[] bytes = new byte[len];
s = _input.read(bytes);
while(s==-1){
Thread.sleep(1);
s = _input.read(bytes);
}
return new String(bytes);
Firstly, the loop is pointless: the only time read will return -1 is if it's closed, in which case looping isn't going to help you.
Secondly, you're ignoring the possibility that the data will come in more than one chunk. You're assuming that if you've managed to get any data, you've got all the data. Instead, you should loop something like this:
int bytesRead = 0;
while (bytesRead < bytes.length) {
int chunk = _input.read(bytes, bytesRead, bytes.length - bytesRead);
if (chunk == -1) {
throw new IOException("Didn't get as much data as we should have");
}
bytesRead += chunk;
}
Note that all your other InputStream.read calls also assume that you've managed to read data, and indeed that you've read all the data you need.
Oh, and you're using the platform-default encoding to convert between binary data and text data - not a good idea.
Is there any reason you're not using DataInputStream and DataOutputStream for this? Currently you're reinventing the wheel, and doing so with bugs.

You sending code is bugged:
byte[] msgLen = ByteBuffer.allocate(4).putInt(message.length()).array();
_output.write(msgLen);
_output.write(message.getBytes());
You send the number of characters as the message length, but after that you convert the message to bytes. Depending on the platform encoding String.getBytes() can give you much more bytes than there are characters.
You should never assume that String.length() has any relationship with String.getBytes().length! Those are different concepts and should never be mixed.

Problem: Java-Read socket (Bluetooth) input stream until a string (<!MSG>) is read

I use the following code (from Bluetooth Chat sample app) to read the incoming data and construct a string out of the bytes read. I want to read until this string has arrived <!MSG>. How to insert this condition with read() function?
The whole string looks like this <MSG><N>xxx<!N><V>yyy<!V><!MSG>. But the read() function does not read entire string at once. When I display the characters, I cannot see all the characters in the same line. It looks like:
Sender: <MS
Sender: G><N>xx
Sender: x<V
.
.
.
I display the characters on my phone (HTC Desire) and I send the data using windows hyperterminal.
How to make sure all the characters are displayed in a single line? I have tried using StringBuilder and StringBuffer instead of new String() but the problem is read() function does not read all the characters sent. The length of the input stream (bytes) is not equal to actual length of the string sent. The construction of string from the read bytes is happening alright.
Thank you for any suggestions and time spent on this. Also please feel free to suggest other mistakes or better way of doing below things, if any.
Cheers,
Madhu
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
//Writer writer = new StringWriter();
byte[] buffer = new byte[1024];
int bytes;
//String end = "<!MSG>";
//byte compare = new Byte(Byte.parseByte(end));
// Keep listening to the InputStream while connected
while (true) {
try {
//boolean result = buffer.equals(compare);
//while(true) {
// Read from the InputStream
bytes = mmInStream.read(buffer);
//Reader reader = new BufferedReader(new InputStreamReader(mmInStream, "UTF-8"));
//int n;
//while ((bytes = reader.read(buffer)) != -1) {
//writer.write(buffer, 0, bytes);
//StringBuffer sb = new StringBuffer();
//sb = sb.append(buffer);
//String readMsg = writer.toString();
String readMsg = new String(buffer, 0, bytes);
//if (readMsg.endsWith(end))
// Send the obtained bytes to the UI Activity
mHandler.obtainMessage(BluetoothChat.MESSAGE_READ, bytes, -1, readMsg)
.sendToTarget();
//}
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
break;
}
}
}

The read function does not make any guarantee about the number of bytes it returns (it generally tries to return as many bytes from the stream as it can, without blocking). Therefore, you have to buffer the results, and keep them aside until you have your full message. Notice that you could receive something after the "<!MSG>" message, so you have to take care not to throw it away.
You can try something along these lines:
byte[] buffer = new byte[1024];
int bytes;
String end = "<!MSG>";
StringBuilder curMsg = new StringBuilder();
while (-1 != (bytes = mmInStream.read(buffer))) {
curMsg.append(new String(buffer, 0, bytes, Charset.forName("UTF-8")));
int endIdx = curMsg.indexOf(end);
if (endIdx != -1) {
String fullMessage = curMsg.substring(0, endIdx + end.length());
curMsg.delete(0, endIdx + end.length());
// Now send fullMessage
}
}

java BufferedReader specific length returns NUL characters

I have a TCP socket client receiving messages (data) from a server.
messages are of the type length (2 bytes) + data (length bytes), delimited by STX & ETX characters.
I'm using a bufferedReader to retrieve the two first bytes, decode the length, then read again from the same bufferedReader the appropriate length and put the result in a char array.
most of the time, I have no problem, but SOMETIMES (1 out of thousands of messages received), when attempting to read (length) bytes from the reader, I get only part of it, the rest of my array being filled with "NUL" characters. I imagine it's because the buffer has not yet been filled.
char[] bufLen = new char[2];
_bufferedReader.read(bufLen);
int len = decodeLength(bufLen);
char[] _rawMsg = new char[len];
_bufferedReader.read(_rawMsg);
return _rawMsg;
I solved the problem in several iterative ways:
first I tested the last char of my array: if it wasn't ETX I would read chars from the bufferedReader one by one until I would reach ETX, then start over my regular routine. the
consequence is that I would basically DROP one message.
then, in order to still retrieve that message, I would find the first occurence of the NUL char in my "truncated" message, read & store additional characters one at a time until I reached ETX, and append them to my "truncated" messages, confirming length is ok.
it works also, but I'm really thinking there's something I could do better, like checking if the total number of characters I need are available in the buffer before reading it, but can't find the right way to do it...
any idea / pointer ?
thanks !

The InputStream read method may return short reads; you must check the return value to determine how many characters were read, and continue reading in a loop until you get the number you wanted. The method may block, but it only blocks until some data is available, not necessarily all the data you requested.
Most people end up writing a "readFully" method, like DataInputStream, which reads the amount of data expected, or throws an IOException:
static public int readFully(InputStream inp, byte[] arr, int ofs, int len) throws IOException {
int rmn,cnt;
for(rmn=len; rmn>0; ofs+=cnt, rmn-=cnt) {
if((cnt=inp.read(arr,ofs,rmn))==-1) {
throw new IOException("End of stream encountered before reading at least "+len+" bytes from input stream");
}
}
return len;
}

Here is a sample server that I have used for testing
The main rcv is structured like
while((chars_read = from_server.read(buffer)) != -1)
{
to_user.write(buffer,0,chars_read);
to_user.flush();
}
The actual whole server is below ...
public static void main(String[] args) throws IOException
{
try
{
if (args.length != 2)
throw new IllegalArgumentException("Wrong number of Args");
String host = args[0];
int port = Integer.parseInt(args[1]);
Socket s = new Socket(host,port);
final Reader from_server = new InputStreamReader(s.getInputStream());
PrintWriter to_server = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
BufferedReader from_user = new BufferedReader(new InputStreamReader(System.in));
final PrintWriter to_user = new PrintWriter(new OutputStreamWriter(System.out));
to_user.println("Connected to " + s.getInetAddress() + ":" + s.getPort());
Thread t = new Thread()
{
public void run()
{
char [] buffer = new char[1024];
int chars_read;
try
{
while((chars_read = from_server.read(buffer)) != -1)
{
to_user.write(buffer,0,chars_read);
to_user.flush();
}
}
catch(IOException e)
{
to_user.println(e);
}
to_user.println("Connection closed by server");
to_user.flush();
System.exit(0);
}
};
t.setPriority(Thread.currentThread().getPriority() + 1);
t.start();
String line;
while ((line = from_user.readLine()) != null)
{
to_server.println(line);
to_server.flush();
}
//t.stop();
s.close();
to_user.println("Connection closed by client");
to_user.flush();
}
catch(Throwable e)
{
e.printStackTrace();
System.err.println("Usage : java TCPClient <hostname> <port>");
}
}

Issue in reading data from Socket

I am facing some problem during reading data from socket If there is some null data in socket stream so the DataInputStream would not read the full data and the so at the receiving end there is exception for parsing data.
What is the right way to read the data from socket so there is no loss of data at any time ?
Thanks in advance.

You should post the code being used to read from the socket, but to me the most likely case is that the reading code is incorrectly interpreting a 0 byte as the end of the stream similar to this code
InputStream is = ...;
int val = is.read();
while (0 != (val = is.read()) {
// do something
}
But the end of stream indicator is actually -1
InputStream is = ...;
int val = is.read();
while (-1 != (val = is.read()) {
// do something
}
EDIT: in response to your comment on using isavailable(). I assume you mean available() since there is no method on isavailable() InputStream. If you're using available to detect the end of the stream, that is also wrong. That function only tells you how many bytes can be read without blocking (i.e. how many are currently in the buffer), not how many bytes there are left in the stream.

I personally prefer ObjectInputStream over DataInputStream since it can handle all types including strings, arrays, and even objects.
Yes, you can read an entire object only by 1 line receive.readObject(), but don't forget to type-cast the returned object.
read() mightbe easier since you read the whole thing in 1 line, but not accurate. read the data one by one, like this:
receive.readBoolean()
receive.readInt()
receive.readChar()
etc..

String finalString = new String("");
int finalSize = remainingData;//sizeOfDataN;
int reclen = 0;
int count_for_breaking_loop_for_reading_data = 0;
boolean for_parsing_data = true;
while(allDataReceived == false) {
ByteBuffer databuff = ByteBuffer.allocate(dis.available());
// System.out.println("bis.availbale is "+dis.available());
databuff.clear();
databuff.flip();
dis.read(databuff.array());
String receivedStringN = trimNull(databuff.array());
finalString = finalString + receivedStringN;
System.out.println("final string length "+finalString.length());
if(finalString.length() == finalSize) {
allDataReceived = true;
}
count_for_breaking_loop_for_reading_data++;
if(count_for_breaking_loop_for_reading_data > 1500) {
For_parsing_data = false;
Break;
}
}