We Keep Coding

How to send packet using binary communication protocol on tcp port

I am working on a device which would work to measure some readings through sensors. Device is operated by an Android app. I have to take readings from TCP layer. This is the code to send data on TCP
TcpClient.java
import android.util.Log;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.Socket;
/**
* Created by shahbaz on 25/4/17.
*/
public class TcpClient {
public static final String SERVER_IP = "192.168.1.76"; //server IP address
public static final int SERVER_PORT = 1800;
// message to send to the server
private String mServerMessage;
// sends message received notifications
private OnMessageReceived mMessageListener = null;
// while this is true, the server will continue running
private boolean mRun = false;
// used to send messages
private PrintWriter mBufferOut;
// used to read messages from the server
private BufferedReader mBufferIn;
/**
* Constructor of the class. OnMessagedReceived listens for the messages received from server
*/
public TcpClient(OnMessageReceived listener) {
mMessageListener = listener;
}
/**
* Sends the message entered by client to the server
*
* #param message text entered by client
*/
public void sendMessage(String message) {
if (mBufferOut != null && !mBufferOut.checkError()) {
mBufferOut.println(message);
mBufferOut.flush();
}
}
/**
* Close the connection and release the members
*/
public void stopClient() {
mRun = false;
if (mBufferOut != null) {
mBufferOut.flush();
mBufferOut.close();
}
mMessageListener = null;
mBufferIn = null;
mBufferOut = null;
mServerMessage = null;
}
public void run() {
mRun = true;
try {
//here you must put your computer's IP address.
InetAddress serverAddr = InetAddress.getByName(SERVER_IP);
Log.e("TCP Client", "C: Connecting...");
//create a socket to make the connection with the server
Socket socket = new Socket(serverAddr, SERVER_PORT);
try {
//sends the message to the server
mBufferOut = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())), true);
//receives the message which the server sends back
mBufferIn = new BufferedReader(new InputStreamReader(socket.getInputStream()));
//in this while the client listens for the messages sent by the server
while (mRun) {
mServerMessage = mBufferIn.readLine();
if (mServerMessage != null && mMessageListener != null) {
//call the method messageReceived from MyActivity class
mMessageListener.messageReceived(mServerMessage);
}
}
Log.e("RESPONSE FROM SERVER", "S: Received Message: '" + mServerMessage + "'");
} catch (Exception e) {
Log.e("TCP", "S: Error", e);
} finally {
//the socket must be closed. It is not possible to reconnect to this socket
// after it is closed, which means a new socket instance has to be created.
socket.close();
}
} catch (Exception e) {
Log.e("TCP", "C: Error", e);
}
}
//Declare the interface. The method messageReceived(String message) will must be implemented in the MyActivity
//class at on asynckTask doInBackground
public interface OnMessageReceived {
public void messageReceived(String message);
}
}
Packet Structure
Packet format contains,
While communicating with the device on TCP, boundaries between packets are not identified, in this case if the packets are out of sequence or if any of the packets is missed one can identify a new packet using ‘header start’.
So, first 2 bytes in the packet represent the start of the packet.
Header start: Two-byte field that indicates the start of every packet. 0x55AA is a 2 bytes number used as header start.
Protocol version: One-byte field to specify the version of the protocol in use. Version specified in the payload will decide the payload structure. At any given moment a device will support single protocol version. Present protocol version is ‘1’.
DSN: Sequence number is 1-byte field which will identify the packet uniquely. Requester of the packet will have to fill this field in request payload; responder has to fill the same unique identifier in the response payload.
Request Id: One-byte field specifies the command id. The parsing of the payload will be done on the basis of the command id. In case of request payload this field will be non zero and in case of response it will be zero.
Payload length: Two-byte field specifies the length of the payload in bytes. It specifies the number of bytes followed payload length field. In the payload length, header length and CRC is not included. Currently, Max payload length supported by gateway device is 512 (bytes).
CRC: 1 byte field which will be calculated by XORing all the bytes and add the XOR count of 0.
And it is working. But according to docs I have to send packet using binary communication protocol. Including header start, payload data, etc. How can I send these params in packet structure? How can I create packet?
Any help is appreciated.

The main mistake was that I was not thinking much about size of primitive data types.
byte = 1 byte
short = 2 bytes
int = 4 bytes
long = 8 bytes
float = 4 bytes
double = 8 bytes
char = 2 byte
After referencing the size of primitive datatypes I realised we should track the size and index of packet because we are dealing with byte array.
TcpPacket.java
public class TcpPacket {
private static int header_start = 0x55AA;
private static int protocol_version = 1;
private PacketUtils packetUtils = new PacketUtils();
public byte[] getHandshakePacket()
{
int request_id = 1;
byte[] header_data = packetUtils.ItoBA2(header_start);
byte[] payload_data = packetUtils.ItoBA4(packetUtils.getDateTime());
byte[] payload_length = packetUtils.ItoBA2(4);
byte[] a_data = new byte[]{header_data[0], header_data[1], (byte) protocol_version, packetUtils.getDSN(), (byte) request_id, payload_length[0], payload_length[1],
payload_data[0], payload_data[1], payload_data[2], payload_data[3]};
byte[] b_data = new byte[]{ packetUtils.getCRC(a_data)};
byte[] packet_data = packetUtils.concatBytes(a_data,b_data);
return packet_data;
}
}
PacketUtils.java
public class PacketUtils {
public byte[] ItoBA4(int value) { // integer to bytes function (return byte array of 4 bytes)
return new byte[] {
(byte)(value >>> 24),
(byte)(value >>> 16),
(byte)(value >>> 8),
(byte)value};
}
public byte[] ItoBA2(int value) { // integer to bytes function (return byte array of 2 bytes)
return new byte[] {
(byte)(value >>> 8),
(byte)value};
}
public byte getDSN() // return one byte random number
{
char[] chars = "1234567890".toCharArray();
StringBuilder sb = new StringBuilder();
Random random = new Random();
for (int i = 0; i < 1; i++) {
char c = chars[random.nextInt(chars.length)];
sb.append(c);
}
byte output = Byte.valueOf(sb.toString());
return output;
}
public byte getCRC(byte[] packet) // required CRC function (return byte)
{
try
{
if (packet == null)
{
//Logger.Error("empty packet received");
return (byte)0;
}
byte XORCheckSum = 0;
byte zeroCount = 0;
byte FFCount = 0;
for (int i = 0; i < packet.length; i++)
{
XORCheckSum ^= packet[i];
if (packet[i] == (byte) 0)
{
zeroCount++;
continue;
}
if (packet[i] == (byte)255)
{
FFCount++;
continue;
}
}
XORCheckSum ^= zeroCount;
XORCheckSum ^= FFCount;
return XORCheckSum;
}
catch (Exception ex)
{
//Logger.Error(ex);
return (byte)0;
}
}
byte[] concatBytes(byte[]...arrays) // concatenate byte arrays
{
// Determine the length of the result array
int totalLength = 0;
for (int i = 0; i < arrays.length; i++)
{
totalLength += arrays[i].length;
}
// create the result array
byte[] result = new byte[totalLength];
// copy the source arrays into the result array
int currentIndex = 0;
for (int i = 0; i < arrays.length; i++)
{
System.arraycopy(arrays[i], 0, result, currentIndex, arrays[i].length);
currentIndex += arrays[i].length;
}
return result;
}
public int getDateTime()
{
int dateInSec = (int) (System.currentTimeMillis() / 1000);
return dateInSec;
}
}

java: Length of data changes after write in stream

I have saved a binary data in FileOutputStream but when I check the length of the data before and after I found that it changes from 72 to 106.
This is my method:
inputStream = new FileInputStream(certificate_file);
/*Certificate file is a Path of a binary file */
pubkey = readFromStream(inputStream, 0, 71);
System.out.println("length of pubkey: "+pubkey.length());
/* This return : length of pubkey: 72 */
writeToStream(path + "pubkey.bin", pubkey);
inputStream = new FileInputStream(path + "pubkey.bin");
pubkey = readFromStream(inputStream);
System.out.println("length of pubkey: "+pubkey.length());
/* This return : length of pubkey: 106 */
writeToStream method to write data into outputstream:
public void writeToStream(String path, String data)
throws FileNotFoundException {
OutputStream os = new FileOutputStream(path);
PrintStream printStream = new PrintStream(os);
printStream.print(data);
}
readFromStream method to read data from stream:
public static String readFromStream(InputStream inputStream, int begin, int end) throws Exception {
int i = 0;
int data = inputStream.read();
String out = "";
while (data != -1) {
if (i >= begin && i <= end) {
out += (char) data;
}
data = inputStream.read();
i++;
}
return out;
}
public static String readFromStream(InputStream inputStream) throws Exception {
int i = 0;
int data = inputStream.read();
String out = "";
while (data != -1) {
out += (char) data;
data = inputStream.read();
i++;
}
return out;
}
Why I have this problem?

I have solved the problem, I transformed the data from String to bytes[] and I changed the read in readFromStream to readAllBytes.

Java - Convert byte[] to char[]. The encoding is UTF-16

I need to convert byte[] to char[] in Java.
The Unicode encoding used is UTF-16.
To be concise, I need a Java equivalent of c#'s
UnicodeEncoding.Unicode.GetChars(byte[] bytes);
Also, I need to convert only a part of byte[] to char[]
public virtual char[] GetChars(byte[] bytes, int index, int count);

You can try this:
byte[] b = ...
char[] c = new String(b, "UTF-16").toCharArray();
From String(byte[] bytes, String charsetName):
Constructs a new String by decoding the specified array of bytes using the specified charset.

C# API:
public virtual char[] GetChars(byte[] bytes);
Java:
byte[] byte = ...
char[] char = new String(byte, "UTF-16").toCharArray();
Java methods to return char[] :
public static char[] getCharsFromBytes(byte[] bytes, String type) {
try {
String byteConvert = new String(bytes, "UTF-16");
return byteConvert.toCharArray();
} catch (UnsupportedEncodingException ex) {
Logger.getLogger(DataTypeUtil.class.getName()).log(Level.SEVERE, null, ex);
return null;
}
}
C# API:
public virtual char[] GetChars(byte[] bytes, int index, int count);
Java:
Iterate through required bytes in the byte array
public static char[] getCharsFromBytes(byte[] bytes, int index, int count) {
char[] charArr = getCharsFromBytes(bytes);
char[] result = new char[count];
for (int i = 0; i < count; i++) {
result[i] = charArr[i + startIndex];
}
return result;
}

public static char[] GetChars(byte[] bytes, int index, int count) {
try {
return new String(java.util.Arrays.copyOfRange(bytes,index,index+count), "UTF-16").toCharArray();
} catch (java.io.UnsupportedEncodingException e) {
assert false: "Your JRE is broken (UTF-16 not supported)";
return null;
}
}

How to split binary data into hex strings when characters are in the start and end of the strings

I want to split data based on character values which are two right parenthesis )) as start of substring and carriage return CR as the end of substring. The data comes in form of bytes Am stuck on how to split it. This is so far what I have come up with.
public class ByteDecoder {
public static void main(String[] args) throws IOException {
InputStream is = null;
DataInputStream dis = null;
try{
is = new FileInputStream("byte.log");
dis = new DataInputStream(is);
int count = is.available();
byte[] bs = new byte[count];
dis.read(bs);
for (byte b:bs)
{
char c = (char)b;
System.out.println(c);
//convert bytes to hex string
// String c = DatatypeConverter.printHexBinary( bs);
}
}catch(Exception e){
e.printStackTrace();
}finally{
if(is!=null)
is.close();
if(dis!=null)
dis.close();
}
}
}

CR (unlucky 13) as end marker of binary data might be a bit dangerous. More dangerous seems how the text and bytes became written: the text must be written as bytes in some encoding.
But considering that, one could wrap the FileInputStream in your own ByteLogInputStream, and there hold the reading state:
/**
* An InputStream converting bytes between ASCII "))" and CR to hexadecimal.
* Typically wrapped as:
* <pre>
* try (BufferedReader in = new BufferedReader(
* new InputStreamReader(
* new ByteLogInputStream(
* new FileInputStream(file), "UTF-8"))) {
* ...
* }
* </pre>
*/
public class ByteLogInputStream extends InputStream {
private enum State {
TEXT,
AFTER_RIGHT_PARENT,
BINARY
}
private final InputStream in;
private State state = State.TEXT;
private int nextHexDigit = 0;
public ByteLogInputStream(InputStream in) {
this.in = in;
}
#Override
public int read() throws IOException {
if (nextHexDigit != 0) {
int hex = nextHexDigit;
nextHexDigit = 0;
return hex;
}
int ch = in.read();
if (ch != -1) {
switch (state) {
case TEXT:
if (ch == ')') {
state = State.AFTER_RIGHT_PARENT;
}
break;
case AFTER_RIGHT_PARENT:
if (ch == ')') {
state = State.BINARY;
}
break;
case BINARY:
if (ch == '\r') {
state = State.TEXT;
} else {
String hex2 = String.format("%02X", ch);
ch = hex2.charAt(0);
nextHexDigit = hex2.charAt(1);
}
break;
}
}
return ch;
}
}
As one binary byte results in two hexadecimal digits, you need to buffer a nextHexDigit for the next digit.
I did not override available (to account for a possible nextHexDigit).
If you want to check whether \r\n follows, one should use a PushBackReader. I did use an InputStream, as you did not specify the encoding.

Converting byte[] to String with new Operator(performance)

Based on the below question,
http://stackoverflow.com/questions/23753342/java-memory-allocation-for-local-variables
I use SerialPortEvent to read data from serial port,
public String logtext = "";
public void serialEvent(SerialPortEvent evt) {
if (evt.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
try {
StringBuilder sBuilder = new StringBuilder();
int length = input.available();
byte[] array = new byte[length];
int numBytes = input.read(array);
for (byte b : array) {
logText = new String(new byte[] {b});
sBuilder.append(logText);
}
//Finally i append the StringBuilder to JtextPane
.......
......
}
Creating new String() every time the serialEvent is called it will create new instance every time which will increase the memory usage.. In certain scenario this serialEvent will be called every second.
Is there any other efficient way to do this with out using new operator??
Please help

You don't need to do this one byte at a time. I would just do all the bytes at once.
int length = input.available();
byte[] array = new byte[length];
String logText = new String(array, 0); // assume ISO-8859-1 encoding