We Keep Coding

write hex string as bytes to socket java

I have a string of hex values that I am trying to write to a socket as bytes.
String confDeliv = "\\x7E\\x01\\x00\\x20\\x37\\x02\\x03\\xF2\\xD5";
I have tried doing this to try and solve my problem
byte [] Delivery_Conf = {(byte)0x7E, (byte)0x01, (byte)0x00, (byte)0x20,
(byte)0x37, (byte)0x02, (byte)0x03, (byte)0xF2, (byte)0xD5};
But I have yet to succeed to write it to the socket. I don't get any errors but when I send it to the device it doesn't do what I need it to do I have tried two different ways of doing this.
Try 1:
DataOutputStream dOut = new DataOutputStream(sock.getOutputStream()); //69.171.154.64
for (int i = 0; i < Delivery_Conf.length-1; i++) {
dOut.writeByte(Delivery_Conf[i]);
}
dOut.flush();
This method I used when I but the values into a byte array.
Try 2:
DataOutputStream dOut = new DataOutputStream(sock.getOutputStream());
dOut.writeBytes(confDeliv);
dOut.flush();
This is the method I used when I tried sending it as the string but still no luck. I am able to make the device work when I use python using its byte string.
eg.
confDel = b"\x7E\x01\x00\x20\x37\x02\x03\xF2\xD5"
I think java changes something when I send it and I think that is why I can get it to work with java. I have looked around for while but I do not seem to find anything that will help me with my problem.

You should use the following:
byte [] Delivery_Conf = {(byte)0x7E, (byte)0x01, (byte)0x00, (byte)0x20,
(byte)0x37, (byte)0x02, (byte)0x03, (byte)0xF2, (byte)0xD5};
// ...
dos.write(Delivery_conf);
The version you had writing a byte at a time should work but it's inefficient, and it's possible that the device has timing constraints.
The version using the String isn't correct. Adding another backslash to make \x compile is not a correct solution: you should change \x to \u00 throughout. Throughout the string, that is, of course.

Setting a Java Byte Array Size from Packet Header

A piece of Java code is residing on a server expecting about 64 bytes of information from a piece of hardware, sent via TCP. The packet has a 10 byte header. The first byte is a protocol identifier, the second two bytes gives the total number of bytes in the packet, including all the header bytes and checksum. The last 7 bytes are a UID.
Server Code:
public void run () throws Exception
{
//Open a socket on localhost at port 11111
ServerSocket welcomeSocket = new ServerSocket(11111);
while(true) {
//Open and Accept on Socket
Socket connectionSocket = welcomeSocket.accept();
//Alt Method
DataInputStream dis = new DataInputStream(connectionSocket.getInputStream());
int len = dis.readInt();
byte[] data = new byte[len];
if (len > 0) {
dis.readFully(data);
}
System.out.println("Recv[HEX]: " + StringTools.toHexString(data));
}
}
The issue is my readInt() line, that takes the first four bytes, however I need to determine the length based on the second two bytes. How can this be achieved?
And secondly, is my stringTools.toHexString(data) correct to dump the received buffer which I know should be readable as a HEX string?
Note: This question has its root here: Java TCP Socket Byte Heap Memory Issue

Only use DataInputStream if the other side is using DataOutputStream or it's exact format. The integers, for example, may be encoded big-endian or little-endian - DataOutputStream uses big-endian notation, if the other side uses different encoding, you cannot use DataInputStream. Using InputStream.read() gives you more control if you need it.
Now, since the format of message as you stated starts with one byte for protocol identifier, you first need to read that as a byte (dis.readByte() or InputStream.read()) and either check that the protocol is what you expect or handle different protocols. Then you read the message length, etc.

You can use ByteBuffer to read the int in the last two bytes
import static java.lang.System.out;
import java.nio.ByteBuffer;
class B {
public static void main( String ... args ) {
// test value
int a = 1238098;
// convert it into an arrays of bytes
ByteBuffer b = ByteBuffer.allocate(4);
b.putInt(a);
byte [] r = b.array();
// read last two
int size = ByteBuffer.wrap(new byte[]{0x0,0x0, r[2], r[3]}).getInt();
// print it
out.println("Original: " + String.format("%32s%n" , Integer.toString(a,2)).replace(' ', '0'));
out.printf("Last two: %32s%n" , Integer.toString(size,2));
out.printf("Decimal : %d%n" , size );
}
}
Output:
Original: 00000000000100101110010001010010
Last two: 1110010001010010
Decimal : 58450
However I would recommend to follow #Jiri answer about read using InputStream.read() instead of DateInputStream

How to send java byte[] on tcp socket to c++ char[] on server?

I want to send a byte[] array from a java client to a server that receives the data in C++. The byte array contains characters and integers that are converted to bytes (its a wave header). The server doesn't receive the values correctly. How can I send the byte[] so that the server socket can write it to a char[]? I am using the following code:
Client.java:
//Some example values in byte[]
byte[] bA = new byte[44];
bA[0]='R';
...
bA[4]=(byte)(2048 & 0xff);
...
bA[16] = 16;
....
//Write byte[] on socket
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.write(bA,0,44);
Server.cpp
int k = 0,n = 0;
char buffer[100];
ofstream wav("out.wav", ios::out | ios::binary);
while(k<44){//receive 44 values
memset(buffer ,0 , 100);
n = recv(sock , buffer , 100 , 0);
k += n;
buffer[99] = '\0';
wav.write(buffer,n);
}

One issue I see is if you receive 100 characters, you're corrupting the data with this line:
buffer[99] = '\0';
If there is a character other than NULL at that position, you've corrupted the data. Since the data is binary, there is no need to null terminate the buffer. Remove that line from your loop.
Instead, rely on the return value of recv to determine the number of characters to copy to the stream. Which brings up another point -- you're not checking if recv returns an error.

Input Stream only returning 1 byte

I am using java comm library to try accomplish a simple read/write to a serial port. I am able to successfully write to the port, and catch the return input from the input stream, but when I read from the input stream I am only able to read 1 byte (when I know there should be 11 returned)
I can write to the port successfully using Putty and am receiving the correct return String there. I am pretty new to Java, buffers and serial i/o and think maybe there is some obvious syntax or understanding of how data is returned to the InputStream. Could someone help me? Thanks!
case SerialPortEvent.DATA_AVAILABLE:
System.out.println("Data available..");
byte[] readBuffer = new byte[11];
try {
System.out.println("We trying here.");
while (inputStream.available() > 0) {
int numBytes = inputStream.read(readBuffer, 1, 11);
System.out.println("Number of bytes read:" + numBytes);
}
System.out.println(new String(readBuffer));
} catch (IOException e) {System.out.println(e);}
break;
}
This code returns the following output:
Data available..
We trying here.
Number of bytes read:1
U

As the documentation states
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read.
This behavior is perfectly legal. I would also expect that a SerialPortEvent.DATA_AVAILABLE does not guarantee that all data is available. It's potentially just 1 byte and you get that event 11 times.
Things you can try:
1) Keep reading until you have all your bytes. E.g. wrap your InputStream into a DataInputStream and use readFully, that's the simplest way around the behavior of the regular read method. This might fail if the InputStream does not provide any more bytes and signals end of stream.
DataInputStream din = new DataInputStream(in);
byte[] buffer = new byte[11];
din.readFully(buffer);
// either results in an exception or 11 bytes read
2) read them as they come and append them to some buffer. Once you have all of them take the context of the buffer as result.
private StringBuilder readBuffer = new StringBuilder();
public void handleDataAvailable(InputStream in) throws IOException {
int value;
// reading just one at a time
while ((value = in.read()) != -1) {
readBuffer.append((char) value);
}
}
Some notes:
inputStream.read(readBuffer, 1, 11)
Indices start at 0 and if you want to read 11 bytes into that buffer you have to specify
inputStream.read(readBuffer, 0, 11)
It would otherwise try to put the 11th byte at the 12th index which will not work.

Efficiently send large int[] over sockets in Java

I am working on a Java application where I need to send an array of 500,000 integers from one Android phone to another Android phone over a socket connection as quickly as possible. The main bottleneck seems to be converting the integers so the socket can take them, whether I use ObjectOutputStreams, ByteBuffers, or a low level mask-and-shift conversion. What is the fastest way to send an int[] over a socket from one Java app to another?
Here is the code for everything I've tried so far, with benchmarks on the LG Optimus V I'm testing on (600 MHz ARM processor, Android 2.2).
Low level mask-and-shift: 0.2 seconds
public static byte[] intToByte(int[] input)
{
byte[] output = new byte[input.length*4];
for(int i = 0; i < input.length; i++) {
output[i*4] = (byte)(input[i] & 0xFF);
output[i*4 + 1] = (byte)((input[i] & 0xFF00) >>> 8);
output[i*4 + 2] = (byte)((input[i] & 0xFF0000) >>> 16);
output[i*4 + 3] = (byte)((input[i] & 0xFF000000) >>> 24);
}
return output;
}
Using ByteBuffer and IntBuffer: 0.75 seconds
public static byte[] intToByte(int[] input)
{
ByteBuffer byteBuffer = ByteBuffer.allocate(input.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(input);
byte[] array = byteBuffer.array();
return array;
}
ObjectOutputStream: 3.1 seconds (I tried variations of this using DataOutPutStream and writeInt() instead of writeObject(), but it didn't make much of a difference)
public static void sendSerialDataTCP(String address, int[] array) throws IOException
{
Socket senderSocket = new Socket(address, 4446);
OutputStream os = senderSocket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream (os);
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(array);
oos.flush();
bos.flush();
os.flush();
oos.close();
os.close();
bos.close();
senderSocket.close();
}
Lastly, the code I used to send byte[]: takes an addition 0.2 seconds over the intToByte() functions
public static void sendDataTCP(String address, byte[] data) throws IOException
{
Socket senderSocket = new Socket(address, 4446);
OutputStream os = senderSocket.getOutputStream();
os.write(data, 0, data.length);
os.flush();
senderSocket.close();
}
I'm writing the code on both sides of the socket so I can try any kind of endianness, compression, serialization, etc. There's got to be a way to do this conversion more efficiently in Java. Please help!

As I noted in a comment, I think you're banging against the limits of your processor. As this might be helpful to others, I'll break it down. Here's your loop to convert integers to bytes:
for(int i = 0; i < input.length; i++) {
output[i*4] = (byte)(input[i] & 0xFF);
output[i*4 + 1] = (byte)((input[i] & 0xFF00) >>> 8);
output[i*4 + 2] = (byte)((input[i] & 0xFF0000) >>> 16);
output[i*4 + 3] = (byte)((input[i] & 0xFF000000) >>> 24);
}
This loop executes 500,000 times. You 600Mhz processor can process roughly 600,000,000 operations per second. So each iteration of the loop will consume roughly 1/1200 of a second for every operation.
Again, using very rough numbers (I don't know the ARM instruction set, so there may be more or less per action), here's an operation count:
Test/branch: 5 (retrieve counter, retrieve array length, compare, branch, increment counter)
Mask and shift: 10 x 4 (retrieve counter, retrieve input array base, add, retrieve mask, and, shift, multiply counter, add offset, add to output base, store)
OK, so in rough numbers, this loop takes at best 55/1200 of a second, or 0.04 seconds. However, you're not dealing with best case scenario. For one thing, with an array this large you're not going to benefit from a processor cache, so you'll introduce wait states into every array store and load.
Plus, the basic operations that I described may or may not translate directly into machine code. If not (and I suspect not), the loop will cost more than I've described.
Finally, if you're really unlucky, the JVM hasn't JIT-ed your code, so for some portion (or all) of the loop it's interpreting bytecode rather than executing native instructions. I don't know enough about Dalvik to comment on that.

Java was IMO never intended to be able efficiently reinterpret a memory region from int[] to byte[] like you could do in C. It doesn't even have such a memory address model.
You either need to go native to send the data or you can try to find some micro optimizations. But I doubt you will gain a lot.
E.g. this could be slightly faster than your version (if it works at all)
public static byte[] intToByte(int[] input)
{
byte[] output = new byte[input.length*4];
for(int i = 0; i < input.length; i++) {
int position = i << 2;
output[position | 0] = (byte)((input[i] >> 0) & 0xFF);
output[position | 1] = (byte)((input[i] >> 8) & 0xFF);
output[position | 2] = (byte)((input[i] >> 16) & 0xFF);
output[position | 3] = (byte)((input[i] >> 24) & 0xFF);
}
return output;
}

I would do it like this:
Socket senderSocket = new Socket(address, 4446);
OutputStream os = senderSocket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
DataOutputStream dos = new DataOutputStream(bos);
dos.writeInt(array.length);
for(int i : array) dos.writeInt(i);
dos.close();
On the other side, read it like:
Socket recieverSocket = ...;
InputStream is = recieverSocket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
DataInputStream dis = new DataInputStream(bis);
int length = dis.readInt();
int[] array = new int[length];
for(int i = 0; i < length; i++) array[i] = dis.readInt();
dis.close();

If you're not adverse to using a library, you might want to check out Protocol Buffers from Google. It's built for much more complex object serialization, but I'd bet that they worked hard to figure out how to quickly serialize an array of integers in Java.
EDIT: I looked in the Protobuf source code, and it uses something similar to your low-level mask and shift.