I have a Java Modbus/TCP application which is reading constantly data from a device.
This is working normal 99.9% of times, but after a weekend working, it could enter in a strange mode, in which for some seconds, I am getting fake values of my Read Multiple Holding Registers functions.
I have checked by using Modscan application, and fake values appear on the client site, which means that the server device is answering properly.
The answer that I can get is an Byte array filled of 0's, 1's and some times other random values.
Here is my Modbus/TCP answer reading:
private byte[] getModbusReply(){
byte[] reply = null;
int transactionId;
int protocol;
int tcpLen;
int id;
int replyCode;
int mbLen = 1;
try{
InputStream is = socket.getInputStream();
transactionId = (is.read()<<8)+is.read();
protocol = (is.read()<<8)+is.read();
tcpLen = (is.read()<<8)+is.read();
id = is.read();
replyCode = is.read();
if(replyCode>0x3F){
mbLen = 1;
}else{
switch(replyCode){
case 0x03:
case 0x04:
mbLen = is.read();
break;
case 0x10:
case 0x06:
mbLen = 4;
break;
default://unsupported Modbus Methods
return null;
}
}
reply = new byte[mbLen+1];
reply[0] = (byte)replyCode;
for(int i=1;i<reply.length;i++){
int res=is.read();
if(res<0){
//Modbus Stream Reading is returning -1
return null;
}
reply[i] = (byte)res;
}
}catch(Exception e){
e.printStackTrace();
return null;
}
return reply;
}
Return null is processed outside the function as a wrong exception.
I had add 2 protections:
read() method returns -1 after EOF, so I add:
int res=is.read();
if(res<0){
//Modbus Stream Reading is returning -1
return null;
}
Return null for unsupported Modbus/TCP methods:
default:///unsupported Modbus Methods
return null;
Maybe I am missing something more on stream reading to which I have no protection.
It isn't sufficient to just return null when something goes wrong or you get something you don't understand. That way you're just leaving the connection open and in an unsynchronized state where you have no reason to believe the next byte is the beginning of a new message. So, from then on you will read unintelligible junk. You need to close the connection and reopen it. Or, implement the rest of the protocol.
You aren't checking for end of stream at nine out of the ten points you read from it.
You should also have a good look at DataInputStream. It already does everything that you're already doing the hard way. Note particularly readShort() and readFully().
Related
I'm trying to serialize Object between NIO SocketChannel and blocking IO Socket. Since I can't use Serializable/writeObject on NIO, I thought to write code to serialize object into an ByteArrayOutputStream then send array length followed by array.
Sender function is
public void writeObject(Object obj) throws IOException{
ByteArrayOutputStream serializedObj = new ByteArrayOutputStream();
ObjectOutputStream writer = new ObjectOutputStream(serializedObj);
writer.writeUnshared(obj);
ByteBuffer size = ByteBuffer.allocate(4).putInt(serializedObj.toByteArray().length);
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
}
and receiver is:
public Object readObject(){
try {
//Leggi dimensione totale pacchetto
byte[] dimension = new byte[4];
int byteRead = 0;
while(byteRead < 4) {
byteRead += this.getInputStream().read(dimension, byteRead, 4 - byteRead);
}
int size = ByteBuffer.wrap(dimension).getInt(); /* (*) */
System.out.println(size);
byte[] object = new byte[size];
while(size > 0){
size -= this.getInputStream().read(object);
}
InputStream in = new ByteArrayInputStream(object, 0, object.length);
ObjectInputStream ois = new ObjectInputStream(in);
Object res = ois.readUnshared();
ois.close();
return res;
} catch (IOException | ClassNotFoundException e) {
return null;
}
}
The problem is that size (*) is always equals to -1393754107 when serializedObj.toByteArray().length in my test is 316.
I don't understand why casting not works properly.
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
If the result of getSocket() is a SocketChannel in non-blocking mode, the problem is here. You aren't checking the result of write(). In non-blocking mode it can write less than the number of bytes remaining in the ByteBuffer; indeed it can write zero bytes.
So youu aren't writing all the data you think you're writing, so the other end overruns and reads the next length word as part of the data being written, and reads part of the next data as the next length word, and gets a wrong answer. I'm surprised it didn't barf earlier. In fact it probably did, but your deplorable practice of ignoring IOExceptions masked it. Don't do that. Log them.
So you need to loop until all requested data has been written, and if any write() returns zero you need to select on OP_WRITE until it fires, which adds a considerable complication into your code as you have to return to the select loop while remembering that there is an outstanding ByteBuffer with data remaining to be written. And when you get the OP_WRITE and the writes complete you have to deregister interest in OP_WRITE, as it's only of interest after a write() has returned zero.
NB There is no casting in your code.
The problem was write() returned 0 always. This happens because the buffer wasn't flipped before write().
Ive read many tutorials and posts about the java InputStream and reading data. Ive established a client and server implementation but i'm having weird issues where reading a variable length "payload" from the client is not consistent.
What im trying to do is to transfer up 100kB max in one single logical payload. Now i have verified that the TCP stack is not sending one mahousive 100kB packet from the client. I have played about with different read forms as per previous questions about the InputStream reading but ive nearly teared my hair out trying to get it to dump the correct data.
Lets for example say the client is sending a 70k payload.
Now the first observation i've noticed is that if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[]. When free running the byte[] will be different sizes every time i run the code with practically the same payload.
Timing problems?
second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour i.e. i get the correct payload size.
please understand i dont like the way ive had to get this to work and im not happy with the solution.
What experiences, problems have you had/seen respectively that might help me fix this code to be more reliable, easier to read.
public void listenForResponses() {
isActive = true;
try {
// apprently read() doesnt return -1 on socket based streams
// if big stuff comes through, TCP packets are segmented, but the inputstream
// does something odd and doesnt return the correct raw data.
// this is a work around to accept vari-length payloads into one byte[] buffer
byte[] inBuffer = new byte[1];
byte[] buffer = null;
int bytesRead = 0;
byte[] finalbuffer = new byte[0];
boolean isMultichunk = false;
InputStream istrm = currentSession.getInputStream();
while ((bytesRead = istrm.read(inBuffer)) > -1 && isActive) {
buffer = new byte[bytesRead];
buffer = Arrays.copyOfRange(inBuffer, 0, bytesRead);
int available = istrm.available();
if(available < 1) {
if(!isMultichunk) {
finalbuffer = buffer;
}
else {
finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
}
notifyOfResponse(deserializePayload(finalbuffer));
finalbuffer = new byte[0];
isMultichunk = false;
}
else {
if(!isMultichunk) {
isMultichunk = true;
finalbuffer = new byte[0];
}
finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
}
}
} catch (IOException e) {
Logger.consoleOut("PayloadReadThread: " + e.getMessage());
currentSession = null;
}
}
InputStream is working as designed.
if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[].
That's because stepping through the code is slower, so more data drives between reads, enough to fill your buffer.
When free running the byte[] will be different sizes every time i run the code with practically the same payload.
That's because InputStream.read() is contracted to block until at least one byte has been transferred, or EOS or an exception occurs. See the Javadoc. There's nothing in there about filling the buffer.
second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour i.e. i get the correct payload size.
That's the correct behaviour in the case of a 1 byte buffer for exactly the same reason given above. It's not the 'correct behaviour' for any other size.
NB Your copy loop is nonsense. available() has few correct uses, and this isn't one of them.
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
NB (2) read() does indeed return -1 on socket-based streams, but only when the peer has shutdown or closed the connection.
On the one side of the socket I know the data is going into the socket correctly.
I set up a connection:
Connection sr = new Connection();
Server server = new Server("NAME", Interger.parseInt(port));
server.setIp(ip);
sr.setServer(server);
//I know my server connection code is correct because I can send and receive data in
//other areas of my program just fine
InputStream is = null;
try
{
is = sr.getChannel().socket().getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
StringBuffer text = new StringBuffer();
int k =0;
while(k != -1)
{
k = br.read();
text.append((char) k);
}
}
catch(Exception e)
{
//no errors ever get thrown
}
And then I only get about half my data, 10989 bytes out of a total 21398 that I send. The amount of bytes it reads varies but the data always ends with ..., "values": [" which in the data I send over looks like , ..., "values": ["", ""].
Keep reading until you have all the data. This question has been showing up about once a week lately. There's no guarantee that the network is going to have all your data show up at once.
You need to keep reading until you have all your data. How do you know how much data was sent? You should probably build a little protocol between the client/server that defines how much data is going to be sent, the server reads that little header and continues to read until the full message has been received.
Don't know if this could help you :
int k =0;
while((k = br.read()) != -1){
text.append((char) k);
}
1) In your case it is making the check on the next iteration, which may lead to appending of non-representable character(char of -1) to the end of text.
2) Never leave catch block empty, may be there is some execption.
So because my sending side of the socket was in c++ I was accidentally passing in a null ASCII value into the socket. And it is undocumented on the java side of the socket that if the read encounters a null value it treats it as an end of file. So it was prematurely ending the stream because it hit the null.
I am working on a TFTP server application. I managed to process a successful file transfer from server to client however the other way around is bugged.
Client instead of transmitting the entire file simply terminated whit compiler returning no errors. Debugger shows IOBE exception on the marked code referring that the array is out of range.
The whole transfer process goes like so:
Client transmits a file name and requested operation WRQ - Write Request
Server received the packet and determines the operation if WRQ is gives the new file appropriate name.
Server now starts executing receiveData() until it gets a packet < 512 indicationg EOT
Client keeps transferring data it read from the file.
Key code:
Client:
private void sendWRQ() throws Exception
{
String rrq = "WRQ-" + data;
outgoingData = rrq.getBytes();
DatagramPacket output = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
clientSocket.send(output);
//Thread.sleep(50);
sendData();
}
byte outgoingData = new byte[512];
private void sendData() throws Exception
{
DatagramPacket dataTransfer = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
InputStream fis = new FileInputStream(new File(data));
int x;
while((x = fis.read(outgoingData,0,512)) != -1) // << Debugged gives IOBE
{
dataTransfer.setLength(x);
clientSocket.send(dataTransfer);
Thread.sleep(5);
}
fis.close();
}
Server:
private void listen() throws Exception
{
DatagramPacket incTransfer = new DatagramPacket(incomingData, incomingData.length);
serverSocket.receive(incTransfer);
clientAddress = incTransfer.getAddress();
clientPort = incTransfer.getPort();
String output = new String(incTransfer.getData());
if(output.substring(0, 3).equals("RRQ"))
{
File test = new File(output.substring(4));
responseData = output.substring(4);
if(test.exists())
{
sendResponse("Y");
} else {
sendResponse("N");
}
} else if (output.substring(0, 3).equals("WRQ"))
{
File test = new File(output.substring(4));
if(test.exists())
{
Calendar cal = Calendar.getInstance();
SimpleDateFormat prefix = new SimpleDateFormat(date_format);
String date = prefix.format(cal.getTime()).toString();
responseData = date + output.substring(4);
receiveData();
} else {
responseData = output.substring(4);
receiveData();
}
}
}
private void receiveData() throws Exception
{
DatagramPacket receiveData = new DatagramPacket(incomingData, incomingData.length);
OutputStream fos = new FileOutputStream(new File(responseData));
while(true)
{
serverSocket.receive(receiveData);
if(receiveData.getLength() == 512)
{
fos.write(receiveData.getData());
} else {
fos.write(receiveData.getData(), receiveData.getOffset(), receiveData.getLength());
break;
}
}
fos.close();
}
The only way that can happen is if the offset or length parameters violate the constraints specified for InputStream.read(byte[], int, int); in this case probably the buffer isn't 512 bytes long. There's no need to specify the 2nd nd third parameters in this case, just omit them, then it becomes read(buffer, 0, buffer.length) internally, which can't be wrong.
Okay, the way this is coded, the 'outgoingData' field is:
1) Initialized to a length of 512
2) Then, in sendWRQ(), 'outgoingData' is re-initialized to whatever rrq.getBytes() sends back.
3) Then, in sendData(), 'outgoingData' is used as the intermediate buffer to read data from file and put it in the dataTransfer object.
However, since 'outgoingData' is re-initialized in step #2, the assumption in step #3 that 'outgoingData' is still 512 bytes in length is false.
So while EJP was correct in saying that using read(outgoingData, 0, outgoingData.length()) will work, there are some architecture issues that if you address, you'll clean up a lot of potential errors.
For instance:
WIth the code provided, there is seemingly no reason to have outgoingData declared at the class level and shared among two functions. Depending on the rest of the app, this could end up being a Threading issue.
Perhaps byte[] buffer = rrq.getBytes(); in sendWRQ() and byte[] buffer = new byte[1024]; in sendData().
Also, the 'data' parameter is at the class level.... for what reason? Might be better able to be controlled if its a passed in parameter.
Lastly, I've had good luck using the do{} while() loop in network situations. Ensures that the send() gets at least one chance to send the data AND it keeps the code a bit more readable.
For my homework assignment, I have a network of Nodes that are passing messages to each other. Each Node is connected to a set amount of other Nodes (I'm using 4 for testing). Each Link has a weight, and all the Nodes have computed the shortest path for how they want their messages sent. Every Packet that is sent is composed of the message protocol (a hard-coded int), an integer that tells how many messages have passed through the sending Node, and the routing path for the Packet.
Every Node has a Thread for each of its Links. There is an active Socket in each Link. The Packets are sent by adding a 4-byte int to the beginning of the message telling the message's length.
Everything works fine until I stress the network. For my test, there are 10 Nodes, and I get 5 of them to send 10000 packets in a simple while() loop with no Thread.sleep(). Without exception, there is always an error at some point during execution at the if(a!=len) statement.
Please let me know if I can clarify anything. Thanks in advance! Here is the code (from the Link Thread; send() and forward() are called from the Node itself):
protected void listen(){
byte[] b;
int len;
try{
DataInputStream in = new DataInputStream(sock.getInputStream());
while(true){
len = in.readInt();
b = new byte[len];
int a = in.read(b,0,len);
if(a!=len){
System.out.println("ERROR: " + a + "!=" + len);
throw new SocketException(); //may have to fix...this will happen when message is corrupt/incomplete
}
Message m = new Message(b);
int p = m.getProtocol();
switch (p){
case CDNP.PACKET:
owner.incrementTracker();
System.out.print("\n# INCOMMING TRACKER: " + m.getTracker() + "\n>>> ");
owner.forward(m);
}
}
}catch (IOException e){
e.printStackTrace();
}
}
public void send(int tracker){
String[] message = { Conv.is(CDNP.PACKET), Conv.is(tracker), owner.getMST().toString() };
Message m = new Message(message);
forward(m);
}
public synchronized void forward(Message m){
try{
OutputStream out = sock.getOutputStream();
//convert length to byte array of length 4
ByteBuffer bb = ByteBuffer.allocate(4+m.getLength());
bb.putInt(m.getLength());
bb.put(m.getBytes());
out.write(bb.array());
out.flush();
}catch (UnknownHostException e){
System.out.println("ERROR: Could not send to Router at " + sock.getRemoteSocketAddress().toString());
return;
}catch (IOException e1){
}
}
int a = in.read(b,0,len);
if(a!=len){
That won't work. The InputStream may not read all the bytes you want, it may read only what is available right now, and return that much without blocking.
To quote the Javadocs (emphasis mine):
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
You need to continue reading in a loop until you have all the data you want (or the stream is finished).
Or, since you are using a DataInputStream, you can also use
in.readFully(b, 0, len);
which always reads exactly len bytes (blocking until those have arrived, throwing an exception when there is not enough data).