We Keep Coding

Java client to C# server TCP

The problem is when i send up to 40 KB everything is okay when i send more sometime half of the data received some time nothing ,is there a limit of the networkstream.Read ,even though i cunked the data ,i can't determine if the problem form the java or the c# from the network stream or the Output stream
C# SERVER
private void ReadData(){
if (networkStream.DataAvailable)
{
int size = GetBufferSize();
Thread.Sleep(340);
byte[] myReadBuffer = new byte[size];
int numberOfBytesRead = 0;
while (true)
{
numberOfBytesRead = networkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
if (numberOfBytesRead >= myReadBuffer.Length)
{
break;
}
}
string str = Encoding.UTF8.GetString(myReadBuffer, 0, size);
dynamic Message = Newtonsoft.Json.JsonConvert.DeserializeObject<dynamic>(str);
// Android Message , JSON String
if (OnAndroidMessage != null)
{
OnAndroidMessage(Message);
}
}
}
private int GetBufferSize()
{
byte[] myReadBuffer = new byte[4];
int numberOfBytesRead = 0;
do
{
numberOfBytesRead = networkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
} while (networkStream.DataAvailable && numberOfBytesRead < myReadBuffer.Length);
if (numberOfBytesRead > 0)
{
// reverse the byte array.
if (BitConverter.IsLittleEndian)
{
Array.Reverse(myReadBuffer);
}
return BitConverter.ToInt32(myReadBuffer, 0);
}
else
{
return 0;
}
}
Java Client // i tested this also without cutting the data to smaller paces ,half of the data received not all of them
mBufferOut = socket.getOutputStream();
private void sendMessage(final String message) {
if (mBufferOut != null && message != null) {
try {
byte[] data = message.getBytes("UTF-8");
Log.d("_TAG", "Sending: " + message);
Log.d("_TAG", "Message length: " + Integer.toString(data.length));
mBufferOut.write(toByteArray(data.length));
mBufferOut.flush();
List<byte[]> divideArray = divideArray(data, 10000);
for (byte[] dataChunk : divideArray) {
Log.e("_TAG","CHUNK SIZE > " + Integer.toString(dataChunk.length));
mBufferOut.write(dataChunk, 0, dataChunk.length);
mBufferOut.flush();
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
private List<byte[]> divideArray(byte[] source, int chunksize) {
List<byte[]> result = new ArrayList<byte[]>();
int start = 0;
while (start < source.length) {
int end = Math.min(source.length, start + chunksize);
result.add(Arrays.copyOfRange(source, start, end));
start += chunksize;
}
return result;
}
Any ideas ?

Solution from this post NetworkStream is reading data that should not be there
static void ReadExact(Stream stream, byte[] buffer, int offset, int count)
{
int read;
while(count > 0 && (read = stream.Read(buffer, offset, count)) > 0) {
offset += read;
count -= read;
}
if(count != 0) throw new EndOfStreamException();
}
the problem is the Read it takes size and want to get that size you need to give it chunks and check each chunk
And also read does not restart from where it stopped until it reads the amount is set to read meaning if i set to read 10 then if it not find the 10 then it will read what it find as example it reads 6 ,it will return 6 and when to loop another time ti read the rest it dose not start from 6 it start from 0 and read until 4 so you overwrite your data ,and if it read 10 from the first try then it set the read to finish so it dose not start from 0 ,it needs to read the amount the has been set to it to re set the read to new buffer location.

Minecraft - Bukkit Sockets

I tried to get MOTD of the remote server, but I can't get colors. When MOTD is colored, the plugin isn't working.
I know why, but I don't know how to resolve it.
public PingServer(String host, int port) {
this.host = host;
this.port = port;
try {
socket.connect(new InetSocketAddress(host, port));
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
out.write(0xFE);
int b;
StringBuffer str = new StringBuffer();
while ((b = in.read()) != -1) {
if (b != 0 && b > 16 && b != 255 && b != 23 && b != 24) {
str.append((char) b);
}
}
data = str.toString().split("§");
data[0] = data[0].substring(1, data[0].length());
} catch (IOException e) {
e.printStackTrace();
}
}
According to the specification, the plugin will get response like this: MOTD§ONLINEPLAYERS§MAXPLAYERS, which should be split on § to get the different portions. However, § is also used for chat messages, and I'm not sure how to differentiate between the two. How can I work around this?

You're currently using the legacy server list ping, designed for beta 1.8 to 1.3. That one is triggered via sending just FE to the server. While current servers still support this ping, it's very old and has several flaws (including the one you found).
You should instead perform the current ping. While this is slightly more complicated, you don't need to implement much of the protocol to actually perform it.
There's only one complicated portion of the protocol you need to know about: VarInts. These are somewhat complicated because they take a varying number of bytes depending on the value. And as such, you have a packet length that can be somewhat hard to calculate.
/** See http://wiki.vg/Protocol_version_numbers. 47 = 1.8.x */
private static final int PROTOCOL_VERSION_NUMBER = 47;
private static final int STATUS_PROTOCOL = 1;
private static final JsonParser PARSER = new JsonParser();
/** Pings a server, returning the MOTD */
public static String pingServer(String host, int port) {
this.host = host;
this.port = port;
try {
socket.connect(new InetSocketAddress(host, port));
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
byte[] hostBytes = host.getBytes("UTF-8");
int handshakeLength =
varIntLength(0) + // Packet ID
varIntLength(PROTOCOL_VERSION_NUMBER) + // Protocol version number
varIntLength(hostBytes.length) + hostBytes.length + // Host
2 + // Port
varIntLength(STATUS_PROTOCOL); // Next state
writeVarInt(handshakeLength, out);
writeVarInt(0, out); // Handshake packet
writeVarInt(PROTOCOL_VERSION_NUMBER, out);
writeVarInt(hostBytes.length, out);
out.write(hostBytes);
out.write((port & 0xFF00) >> 8);
out.write(port & 0xFF);
writeVarInt(STATUS_PROTOCOL, out);
writeVarInt(varIntLength(0));
writeVarInt(0); // Request packet (has no payload)
int packetLength = readVarInt(in);
int payloadLength = readVarInt(in);
byte[] payloadBytes = new int[payloadLength];
int readLength = in.read(payloadBytes);
if (readLength < payloadLength) {
throw new RuntimeException("Unexpected end of stream");
}
String payload = new String(payloadBytes, "UTF-8");
// Now you need to parse the JSON; this is using GSON
// See https://github.com/google/gson
// and http://www.javadoc.io/doc/com.google.code.gson/gson/2.8.0
JsonObject element = (JsonObject) PARSER.parse(payload);
JsonElement description = element.get("description");
// This is a naive implementation; it assumes a specific format for the description
// rather than parsing the entire chat format. But it works for the way the
// notchian server impmlements the ping.
String motd = ((JsonObject) description).get("text").getAsString();
return motd;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
public static int varIntLength(int value) {
int length = 0;
do {
// Note: >>> means that the sign bit is shifted with the rest of the number rather than being left alone
value >>>= 7;
length++;
} while (value != 0);
}
public static void writeVarInt(int value, OutputStream out) {
do {
byte temp = (byte)(value & 0b01111111);
// Note: >>> means that the sign bit is shifted with the rest of the number rather than being left alone
value >>>= 7;
if (value != 0) {
temp |= 0b10000000;
}
out.write(temp);
} while (value != 0);
}
public static int readVarInt(InputStream in) {
int numRead = 0;
int result = 0;
int read;
do {
read = in.read();
if (read < 0) {
throw new RuntimeException("Unexpected end of stream");
}
int value = (read & 0b01111111);
result |= (value << (7 * numRead));
numRead++;
if (numRead > 5) {
throw new RuntimeException("VarInt is too big");
}
} while ((read & 0b10000000) != 0);
return result;
}
The current ping does use JSON, which means you need to use GSON. Also, this implementation makes some assumptions about the chat format; this implementation could break on custom servers that implement chat more completely, but it'll work for servers that embed § into the motd instead of using the more complete chat system (this includes the Notchian server implementation).
If you need to use the legacy ping, you can assume that the 2nd-to-last § marks the end of the MOTD (rather than the 1st §). Something like this:
String legacyPingResult = str.toString();
String[] data = new String[3];
int splitPoint2 = legacyPingResult.lastIndexOf('§');
int splitPoint1 = legacyPingResult.lastIndexOf('§', splitPoint2 - 1);
data[0] = legacyPingResult.substring(0, splitPoint1);
data[1] = legacyPingResult.substring(splitPoint1 + 1, splitPoint2);
data[2] = legacyPingResult.substring(splitPoint2 + 1);
However, I still don't recommend using the legacy ping.

Java SocketChannel Read Entire String

In my current project, I am trying to transmit a string from one computer to another, and after finding and learning from numerous examples I have managed to get a basic form of communication working.
The issue I am having is if one computer tries sending a message that is too long, it seems to get broken up into multiple parts (roughly 3700 characters), and my parsing method fails.
I am using a Selector to iterate through all of the channels. Here is the relevant code:
if(key.isReadable()) {
// Get the channel and read in the data
SocketChannel keyChannel = (SocketChannel)key.channel();
ByteBuffer buffer = buffers.get(keyChannel);
int length = 0;
try {
length = keyChannel.read(buffer);
} catch ( IOException ioe) {
closeChannel(keyChannel);
}
if(length > 0) {
buffer.flip();
// Gather the entire message before processing
while( buffer.remaining() > 0) {
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
fireReceiveEvent(keyChannel, data);//Send the data for processing
}
buffer.compact();
} else if (length < 0) {
closeChannel(keyChannel);
}
}
How can I guarantee that the entire message (regardless of length) is read at once before passing it along?

After talking to numerous people that know more about this than I do. The issue turns out to be that with TCP it is impossible to know when an entire "message" has arrived because there is no such thing as a message since TCP works on a two-way byte stream. The solution is to create your own protocol and implements your own definition of "message".
For my project, every message either starts with a [ or { and ends with a ] or } depending on the starting character. I search through the received data, and if there is a complete message, I grab it and pass it along to the handler. Otherwise skip the channel, and wait for more to arrive.
Here is the final version of my code that handles the message receiving.
if(key.isReadable()) {
// Get the channel and read in the data
SocketChannel keyChannel = (SocketChannel)key.channel();
ByteBuffer buffer = buffers.get(keyChannel);
int length = 0;
try {
length = keyChannel.read(buffer);
} catch ( IOException ioe) {
key.cancel();
closeChannel(keyChannel);
}
if (length > 0) {
buffer.flip();
// Gather the entire message before processing
if (buffer.remaining() > 0) {
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
buffer.rewind();
int index = 0;
int i = 0;
// Check for the beginning of a packet
//[ = 91
//] = 93
//{ = 123
//} = 125
if (data[0] == 91 || data[0] == 123) {
// The string we are looking for
byte targetByte = (byte) (data[0] + 2);
for (byte b : data) {
i += 1;
if (b == targetByte) {
index = i;
break;
}
}
if (index > 0) {
data = new byte[index];
buffer.get(data, 0, index);
fireReceiveEvent(keyChannel, data);
}
} else {
for (byte b : data) {
i += 1;
if (b == 91 || b == 123) {
index = i;
break;
}
}
if (index > 0) {
data = new byte[index];
buffer.get(data, 0, index); // Drain the data that we don't want
}
}
}
buffer.compact();
} else if (length < 0) {
key.cancel();
closeChannel(keyChannel);
}
}

Strange behavior in TCP transmission

My server-client application consists of the following modules:
client (written in Java) connects to server and sends a file;
server (written in C) receives the file and sends a char array message.
The problem is that after receiving the file, the server is unable to send the message or the client is unable to receive it.
Here is my code for server application:
int main(int argc , char *argv[])
{
WSADATA wsa;
SOCKET s , new_socket;
struct sockaddr_in server , client;
int c, bytecount, nr_transf, rest_byte, i, bytesRead;
int recv_size, file_size;
char message[1000];
char buffer[1000];
int buffer_len = 1000;
FILE *f = fopen("out.jpg", "wb");
printf("\nInitialising Winsock...");
if (WSAStartup(MAKEWORD(2, 2), &wsa) != 0)
{
printf("Failed. Error Code : %d", WSAGetLastError());
return 1;
}
printf("Initialised.\n");
//Create a socket
if((s = socket(AF_INET, SOCK_STREAM, 0 )) == INVALID_SOCKET)
{
printf("Could not create socket : %d" , WSAGetLastError());
getch();
return 0;
}
printf("Socket created.\n");
//Prepare the sockaddr_in structure
server.sin_family = AF_INET;
server.sin_addr.s_addr = INADDR_ANY;
server.sin_port = htons(8888);
//Bind
if(bind(s, (struct sockaddr*)&server, sizeof(server)) == SOCKET_ERROR)
{
printf("Bind failed with error code : %d" , WSAGetLastError());
getch();
return 0;
}
puts("Bind done");
//Listen to incoming connections
listen(s, 3);
//Accept and incoming connection
printf("Waiting for incoming connections...");
c = sizeof(struct sockaddr_in);
new_socket = accept(s, (struct sockaddr*)&client, &c);
if (new_socket == INVALID_SOCKET)
{
printf("accept failed with error code : %d", WSAGetLastError());
getch();
return 0;
}
printf("Connection accepted");
//Receive image
while((bytesRead = recv(new_socket, buffer, buffer_len, 0)) > 0)
{
fwrite(buffer, 1, bytesRead, f);
}
fclose(f);
printf("\nReceive finished!");
//Send messsage
char my_message[100];
strcpy(my_message, "Hello World!");
send(new_socket, my_message, strlen(my_message), 0);
closesocket(s);
WSACleanup();
getch();
return 0;
}
And the code for client application:
public class MyClass
{
public static void main(String[] args)
{
String fileName = "1.jpg", receiveMessage;
File a_file = new File(fileName);
int j;
OutputStream output = null;
InputStream input = null;
ObjectInputStream in = null;
Socket socket = null;
try
{
// Create a socket
socket = new Socket("192.168.0.122", 8888);
FileInputStream fileInputStream = new FileInputStream(fileName);
byte[] buffer = new byte[1000];
int bytesRead = 0;
output = socket.getOutputStream();
input = socket.getInputStream();
while((bytesRead = fileInputStream.read(buffer))>0)
{
output.write(buffer,0,bytesRead);
}
fileInputStream.close();
System.out.println("Send finished!");
input.read(buffer);
System.out.println("Receive finished!");
}
catch(Exception e)
{
e.printStackTrace();
}
finally
{
try
{
in.close();
output.close();
socket.close();
}
catch(Exception e)
{
}
}
}
}
Any ideas to solve this problem? Thanks in advance!

It's because you have blocking sockets. When a socket is created (by connecting or by accepting) it is in blocking mode. This means that if there is no data to receive it will not return, effectively blocking the caller.
So after you received the last byte in the loop in the server, the recv call will block indefinitely.
On Windows with winsockets, you use the ioctlsocket function to make a socket blocking or non-blocking. The linked reference have an example which shows hot to make a socket blocking or non-blocking.

ArrayIndexOutOfBoundsException when connecting to WebSocket via Weberknecht

I'm trying to connect to a WebSocket via Weberknecht Java library. When the Java client tries to create the connection, I get the following error message:
W/System.err(805): java.lang.ArrayIndexOutOfBoundsException: length=1000; index=1000
W/System.err(805): at ...WebSocketConnection.connect(WebSocketConnection.java:86)
Here is the part which causes the problem:
public void connect() throws WebSocketException {
try {
if (connected) {
throw new WebSocketException("already connected");
}
socket = createSocket();
input = new DataInputStream(socket.getInputStream());
output = new PrintStream(socket.getOutputStream());
output.write(handshake.getHandshake());
boolean handshakeComplete = false;
int len = 10000;
byte[] buffer = new byte[len];
int pos = 0;
ArrayList<String> handshakeLines = new ArrayList<String>();
while (!handshakeComplete) {
int b = input.read();
buffer[pos] = (byte) b; // THIS LINE CAUSES THE EXCEPTION
pos += 1;
if (buffer[pos - 1] == 0x0A && buffer[pos - 2] == 0x0D) {
String line = new String(buffer, "UTF-8");
if (line.trim().equals("")) {
handshakeComplete = true;
} else {
handshakeLines.add(line.trim());
}
buffer = new byte[len];
pos = 0;
}
}
What could cause this issue? Is it that this client library may not comply with the WebSocket version that the server uses?
The issue over there at Weberknecht has not been solved yet. I want to ask this question here because it seems to be quite a general problem (reading WebSocket headers and handshake data).

The code needs to check for -1 from input.read() to denote the connection hitting EOF.
Also I will assume based on the same bit of code that if a handshake hasn't been reaching after 10000 bytes it is an invalid handshake.
while (!handshakeComplete) {
int b = input.read();
if (b == -1)
throw new WebSocketException ("Connected closed.");
if (pos == buffer.length)
throw new WebSocketException ("Invalid handshake.");
buffer[pos] = (byte) b; // THIS LINE CAUSES THE EXCEPTION
pos += 1;
if (buffer[pos - 1] == 0x0A && buffer[pos - 2] == 0x0D) {
String line = new String(buffer, "UTF-8");
if (line.trim().equals("")) {
handshakeComplete = true;
} else {
handshakeLines.add(line.trim());
}
buffer = new byte[len];
pos = 0;
}
}
Update
Here is a better implementation:
public void connect() throws WebSocketException {
try {
if (connected) {
throw new WebSocketException("already connected");
}
socket = createSocket();
input = new DataInputStream(socket.getInputStream());
output = new PrintStream(socket.getOutputStream());
output.write(handshake.getHandshake());
boolean handshakeComplete = false;
ArrayList<String> handshakeLines = new ArrayList<String>();
BufferedReader reader = new BufferedReader(input);
String handshakeData;
while ((handshakeData = reader.readLine()) != null) {
if (handshakeData.trim().equals("")) {
handshakeComplete = true;
break;
}
handshakeLines.add(handshakeData);
}
if (!handshakeComplete)
throw new WebSocketException ("Failed to establish handshake.");

I am just thinking, why don't you use int read(byte[] b) which reads the the bytes for you. int read(byte[] b) --> Reads some number of bytes from the contained input stream and stores them into the buffer array b.
while (!handshakeComplete) {
int bytesRead = input.read(buffer);//returns number of bytes read
if(bytesRead < 10000 || bytesRead == -1 ||
(buffer[bytesRead - 1] == 0x0A && buffer[bytesRead - 2] == 0x0D)){
//done with reading
handshakeComplete = true;
}
if(bytesRead != -1){
String line = new String(buffer, "UTF-8");
if (!line.trim().equals("")) {
handshakeLines.add(line.trim());
}
}
}

Just by looking briefly, you make no guarantees of stopping before buffer.length == pos. That would be one thing to double check - pos has to stay strictly less than the length of the buffer.