We Keep Coding

Pass Uint8List from dart to java over a TCPSocket

I need to pass an Uint8List to a byte[] array in java over a TCPSocket. There is size mismatch at both ends. PS: I am a beginner in dart.
I have tried socket.add(buf), socket.write(buf), socket.writeAll(buf) but none of them worked
Code in Flutter side (TCP client)
void readVoiceData(Uint8List buf) {
print("Send data size:"+buf.lengthInBytes.toString());
socket.add(buf);
}
OUTPUT: Send data size: 1280
Code Snippet on java side (TCP server)
in = new DataInputStream(clientSocket.getInputStream());
Log.d(TAG, "****Opened InputStream*******");
while (!isInterrupted()) {
if (mTrack != null) {
try {
in.read(bytes);
Log.d(TAG, "Received data size"+bytes.length);
}
}
OUTPUT: Received data size: 1
I am sure the socket connection has been established correctly as I am able to send Strings and Integers flawlessly over them.

This happens because the read() method of the FilterInputStream (DataInput is his son) reads just the next avaliable byte.
Thou, if you want to get the total size of the InputStream, do something like:
size = in.available(); //returns an estimate of number of bytes avaliable on the stream
buf = new byte[size];
realLength= in.read(buf, 0, size);
This code snippet is taken from here.

TCP sending/receiving missing bytes

I am using C# to create a server software for Windows and Java to create the client software.
It works fine most of the time, except for those few exceptions that I don't understand.
I am generally using .ReadLine() and .WriteLine() on both ends to communicate, unless I try to send binary data. That's when I write and read the bytes directly.
This is how the software is supposed work:
Client requests the binary data
Server responds with the length of the binary data as a string
Client receives the length and converts it into an integer and starts reading (length) bytes
Server starts writing (length) bytes
It works in most cases, but sometimes the client app doesn't receive the full data and blocks. The server always immediately flushes after writing data, so flushing is not the problem.
Furthermore I've noticed this usually happens with larger files, small files (up to ~1 MB) usually are not a problem.
NOTE It seems like the C# server does send the data completely, so the problem is most likely somewhere in the Java code.
EDIT - Here are some logs from the client side
Working download: pastebin.com/hFd5TvrF
Failing download: pastebin.com/Q3zFWRLB
It seems like the client is waiting for 2048 bytes at the end (as it should be, as length - processed = 2048 in this case), but for some reason the client blocks.
Any ideas what I'm doing wrong? Below are the source codes of both server and client:
C# Server:
public void Write(BinaryWriter str, byte[] data)
{
int BUFFER = 2048;
int PROCESSED = 0;
// WriteString sends the String using a StreamWriter (+ flushing)
WriteString(data.Length.ToString());
while (PROCESSED < data.Length)
{
if (PROCESSED + BUFFER > data.Length)
BUFFER = data.Length - PROCESSED;
str.Write(data, PROCESSED, BUFFER);
str.Flush();
PROCESSED += BUFFER;
}
}
Java Client:
public byte[] ReadBytes(int length){
byte[] buffer = new byte[length];
int PROCESSED = 0;
int READBUF = 2048;
TOTAL = length;
progress.setMax(TOTAL);
InputStream m;
try {
m = clientSocket.getInputStream();
while(PROCESSED < length){
if(PROCESSED + READBUF > length)
READBUF = length - PROCESSED;
try {
PROCESSED += m.read(buffer, PROCESSED, READBUF);
} catch (IOException e) {
}
XPROCESSED = PROCESSED;
}
} catch (IOException e1) {
// Removed because of sensitive data
}
return decryptData(buffer);
}

I've found a fix. As of now, the server sends the length and right after sends the byte array. For some reason this does not work.
So what I've changed is:
Send length and wait for the client to respond with "OK"
Start writing bytes
Not sure why, but it works. Ran it in a while(true) loop and it's been sending data 1000 times in 4 minutes straight and no problems, so I guess it's fixed.

Setting a Java Byte Array Size from Packet Header

A piece of Java code is residing on a server expecting about 64 bytes of information from a piece of hardware, sent via TCP. The packet has a 10 byte header. The first byte is a protocol identifier, the second two bytes gives the total number of bytes in the packet, including all the header bytes and checksum. The last 7 bytes are a UID.
Server Code:
public void run () throws Exception
{
//Open a socket on localhost at port 11111
ServerSocket welcomeSocket = new ServerSocket(11111);
while(true) {
//Open and Accept on Socket
Socket connectionSocket = welcomeSocket.accept();
//Alt Method
DataInputStream dis = new DataInputStream(connectionSocket.getInputStream());
int len = dis.readInt();
byte[] data = new byte[len];
if (len > 0) {
dis.readFully(data);
}
System.out.println("Recv[HEX]: " + StringTools.toHexString(data));
}
}
The issue is my readInt() line, that takes the first four bytes, however I need to determine the length based on the second two bytes. How can this be achieved?
And secondly, is my stringTools.toHexString(data) correct to dump the received buffer which I know should be readable as a HEX string?
Note: This question has its root here: Java TCP Socket Byte Heap Memory Issue

Only use DataInputStream if the other side is using DataOutputStream or it's exact format. The integers, for example, may be encoded big-endian or little-endian - DataOutputStream uses big-endian notation, if the other side uses different encoding, you cannot use DataInputStream. Using InputStream.read() gives you more control if you need it.
Now, since the format of message as you stated starts with one byte for protocol identifier, you first need to read that as a byte (dis.readByte() or InputStream.read()) and either check that the protocol is what you expect or handle different protocols. Then you read the message length, etc.

You can use ByteBuffer to read the int in the last two bytes
import static java.lang.System.out;
import java.nio.ByteBuffer;
class B {
public static void main( String ... args ) {
// test value
int a = 1238098;
// convert it into an arrays of bytes
ByteBuffer b = ByteBuffer.allocate(4);
b.putInt(a);
byte [] r = b.array();
// read last two
int size = ByteBuffer.wrap(new byte[]{0x0,0x0, r[2], r[3]}).getInt();
// print it
out.println("Original: " + String.format("%32s%n" , Integer.toString(a,2)).replace(' ', '0'));
out.printf("Last two: %32s%n" , Integer.toString(size,2));
out.printf("Decimal : %d%n" , size );
}
}
Output:
Original: 00000000000100101110010001010010
Last two: 1110010001010010
Decimal : 58450
However I would recommend to follow #Jiri answer about read using InputStream.read() instead of DateInputStream

Input Stream only returning 1 byte

I am using java comm library to try accomplish a simple read/write to a serial port. I am able to successfully write to the port, and catch the return input from the input stream, but when I read from the input stream I am only able to read 1 byte (when I know there should be 11 returned)
I can write to the port successfully using Putty and am receiving the correct return String there. I am pretty new to Java, buffers and serial i/o and think maybe there is some obvious syntax or understanding of how data is returned to the InputStream. Could someone help me? Thanks!
case SerialPortEvent.DATA_AVAILABLE:
System.out.println("Data available..");
byte[] readBuffer = new byte[11];
try {
System.out.println("We trying here.");
while (inputStream.available() > 0) {
int numBytes = inputStream.read(readBuffer, 1, 11);
System.out.println("Number of bytes read:" + numBytes);
}
System.out.println(new String(readBuffer));
} catch (IOException e) {System.out.println(e);}
break;
}
This code returns the following output:
Data available..
We trying here.
Number of bytes read:1
U

As the documentation states
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read.
This behavior is perfectly legal. I would also expect that a SerialPortEvent.DATA_AVAILABLE does not guarantee that all data is available. It's potentially just 1 byte and you get that event 11 times.
Things you can try:
1) Keep reading until you have all your bytes. E.g. wrap your InputStream into a DataInputStream and use readFully, that's the simplest way around the behavior of the regular read method. This might fail if the InputStream does not provide any more bytes and signals end of stream.
DataInputStream din = new DataInputStream(in);
byte[] buffer = new byte[11];
din.readFully(buffer);
// either results in an exception or 11 bytes read
2) read them as they come and append them to some buffer. Once you have all of them take the context of the buffer as result.
private StringBuilder readBuffer = new StringBuilder();
public void handleDataAvailable(InputStream in) throws IOException {
int value;
// reading just one at a time
while ((value = in.read()) != -1) {
readBuffer.append((char) value);
}
}
Some notes:
inputStream.read(readBuffer, 1, 11)
Indices start at 0 and if you want to read 11 bytes into that buffer you have to specify
inputStream.read(readBuffer, 0, 11)
It would otherwise try to put the 11th byte at the 12th index which will not work.

DataInputStream's readFully query

I am using dataInputStream's readFully message to read a fixed length byte array as:
byte[] record = new byte[4660004];
in.readFully(record);
The problem here is that sometimes it takes more than 5 seconds to read these many bytes, which is equal to 20000 records. And I am receiving this data on socket. Client is sending data as byte array of 4660004 bytes. Is there a way to received this data faster as right now it takes about 5 minutes to 1 million such records.
EDIT:: complete data flow :
first I create the stream :
static DataInputStream dIn = null;
dIn = new DataInputStream(connection.getInputStream());
msgType = dIn.readByte();
int msgIntLen = dIn.readInt();
processBatch(msgIntType, msgIntLen, dIn, connector);
.
.
private static void processBatch(int msgIntType, int msgIntLen, DataInputStream in,
Connector connector) throws IOException {
int recordIntLen = in.readInt();
byte[] record = new byte[msgIntLen - 4];
in.readFully(record);
}
where should I include the Buffering if that wudf help ?

Comments are beginning to scroll, so moving to an answer.
Buffer your output on the client side by using a BufferedOutputStream. Make sure to call dlOut.flush() after writing the data, so that unsent bytes don't remain in the buffered output stream.
Buffer your input on the client side by using a BufferedInputStream.
Because you are just sending byte arrays, you probably don't need the DataInputStream/DataOuputStream, unless you are using them for an additional purpose. You could just be using BufferedInputStream/BufferedOutputStream.