We Keep Coding

Netty doesn't write

When trying to write with netty, the written data never ends up at the remote side, confirmed with Wireshark.
I have tried:
//Directly using writeAndFlush
channel.writeAndFlush(new Packet());
//Manually flushing
channel.write(new Packet());
channel.flush();
// Even sending bytes won't work:
channel.writeAndFlush(new byte[]{1,2,3});
No exception is caught when I wrap it in try{...}catch(Throwable e){e.printStackTrace();}
What can I do to debug this problem?

Netty is asynchronous, meaning that it won't throw exceptions when a write failed. Instead of throwing exceptions, it returns a Future<?> that will be updated when the request is done. Make sure to log any exceptions coming from this as your first debugging steps:
channel.writeAndFlush(...).addListener(new GenericFutureListener<Future<Object>>() {
#Override
public void operationComplete(Future<Object> future) {
// TODO: Use proper logger in production here
if (future.isSuccess()) {
System.out.println("Data written succesfully");
} else {
System.out.println("Data failed to write:");
future.cause().printStackTrace();
}
}
});
Or more simply:
channel.writeAndFlush(...).addListener(ChannelFutureListener.FIRE_EXCEPTION_ON_FAILURE);
After you get the root cause of the exception, there could be multiple problems:
java.lang.UnsupportedOperationException:unsupported message type: <type> (expected: ...)
Notice: This also throws when using an ObjectEncoder, but your object does not implements Serializable
A default Netty channel can only send ByteBufs and FileRegions. You need to convert your objects to these types either by adding more handlers to the pipeline, or converting them manually to ByteBufs.
A ByteBuf is the Netty variant of a byte array, but has the potential for performance because it can be stored in the direct memory space.
The following handlers are commonly used:
To convert a String use a StringEncoder
To convert a Serializable use a ObjectEncoder (warning, not compatible with normal Java object streams)
To convert a byte[] use a ByteArrayEncoder
Notice: Since TCP is a stream based protocol, you usually want some form of packet sizes attached, since you may not receive exact packets that you write. See Dealing with a Stream-based Transport in the Netty wiki for more information.

Java tcp socket does not receive properly

Hi let me get straight to the problem. I have a big JSON packet that the server sends to this client once the client is authenticated
But the packet comes back in a weird way like it's split or something example:
The JSON should be:
Received: {"UserID":1,"PlayerID":2,"EXP":0,"Lvl":1,"Coins":0,"ItemSlots":30}
When it comes through:
Received: {"UserID":1,"PlayerID":2,"EXP":0,"Lvl":1,"Coins":0,
Received: "ItemSlots":30}
Why does it split the packet or something when it comes to the client and how can I fix this anyway?
Java Receive Code:
private class ClientThread extends Thread {
public void run() {
try {
while (selector.select() > 0) {
for (SelectionKey sk : selector.selectedKeys()) {
selector.selectedKeys().remove(sk);
if (sk.isReadable()) {
SocketChannel sc = (SocketChannel)sk.channel();
ByteBuffer buff = ByteBuffer.allocate(1024);
String content = "";
while (sc.read(buff) > 0) {
sc.read(buff);
buff.flip();
content += charset.decode(buff);
buff.clear();
}
System.out.println("Recieved: " + content);
sk.interestOps(SelectionKey.OP_READ);
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Thanks have a wonderful day.

Hi lemme get straight to the problem so i got a big JSON packet that the server sends to this client once the client is authenticated
You mean you have a big JSON message. Packets are things that network protocols used to exchange information.
But the packet comes back in a weird way like its split or something example:
Unless you're looking at the wire, you aren't looking at packets. You're looking at the bytes you got from your end of the TCP connection.
The JSON should be:
Recieved: {"UserID":1,"PlayerID":2,"EXP":0,"Lvl":1,"Coins":0,"ItemSlots":30}
When it comes through:
Recieved: {"UserID":1,"PlayerID":2,"EXP":0,"Lvl":1,"Coins":0,
Recieved: "ItemSlots":30}
Excellent. You got the same bytes. Now make a JSON parser that figures out where the message ends and parses it.
Why does it split the packet or something when it comes to the client
It splits the message into packets because that's how TCP gets the message to the other side. TCP is not a message protocol and it doesn't know or care what the application considers to be a message -- that's the application's job.
and how i can i fix this anyway?
Write a JSON parser to figure out where the messages end. You haven't implemented any code to receive JSON over TCP yet, so that won't work until you do.
TL;DR: If you want an application-level message protocol, you need to implement one. TCP is not one.

TCP protocol does not maintain message boundaries. It is not guaranteed that what the server sends is received as-is by the client and vice-versa.
If the server sends 1000 bytes data, the client application can receive the same across multiple recv or single recv. TCP does not guarantee any behaviour. "Split" can happen, it is upto the application to handle the data coming in multiple chunks, coalesce it to one unit of application data for further processing. One can see this particularly with large data sizes.

It looks like you've got a non-blocking socket channel, meaning that the while (sc.read(buff) > 0) { loop is terminating due to sc.read(buff) returning 0 since only a portion of the data sent has, at this point, been received.
Why does it split the packet or something when it comes to the client
Most likely the data is being split into two or more packets.
and how i can i fix this anyway?
Keep filling your buffer until the socket is closed by the server (read should return -1 rather than 0 in that case). You need to maintain a separate buffer per channel. If the server doesn't close its end after sending the data, you'll need to delineate in some other way; you could prefix the JSON blob with a size header, for instance.

Netty channelRead not getting full message

Why doesn't channelRead() give me the full message I send to the server? Fragmentation sometimes occur when messages are getting above 140 bytes (Roughly, sometimes more and sometimes less). I'm using a TCP socket using the NioServerSocketChannel class.
I'm using 4.1.0.Beta5.
Isn't there a way to read the full message when it has arrived?
this.serverBootstrap = new ServerBootstrap();
this.serverBootstrap.group(new NioEventLoopGroup(1), new NioEventLoopGroup(6))
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>()
{
#Override
public void initChannel(SocketChannel ch) throws Exception
{
ch.pipeline().addLast(new TestServerHandler());
}
})
.option(ChannelOption.SO_BACKLOG, (int)Short.MAX_VALUE)
.option(ChannelOption.SO_RCVBUF, (int) Short.MAX_VALUE)
.option(ChannelOption.SO_KEEPALIVE, true)
.option(ChannelOption.TCP_NODELAY, true);
this.serverBootstrap.bind(this.host, this.port);
And class TestServerHandler extends ChannelInboundHandlerAdapter:
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
String s = buffer.toString(CharsetUtil.UTF_8);
for(int i = 0; i < 20; i++)
{
s = s.replace("[" + ((char)i) + "]", i + "");
}
System.out.println(s.length() + "");
System.out.println();
System.out.println();
System.out.println(s);
}
I need a way to get the full bytebuf / bytearray when it has fully arrived at the server and get notified of that so my application can respond in a correct way according to the data the client has send.
So in short: How can I prevent fragmentation and have the channelRead event output the whole message / bytebuf.

The basic data type used by Netty is Channel Buffers or ByteBuf. This is simply a collection of bytes and nothing else. In your code you have simply used a custom handler to handle the raw incoming data. This is generally not a good practice. A very basic netty pipeline should look something like the following
So a pipeline consists of a decoder / encoder and then we have our custom handlers or logging handlers. We never really handle any raw data as is. TCP is a stream protocol. It does not identify when a specific packet ends and a new packet starts. Even if we send a very very large packet or say two individual packets, they will simply be treated as a set of bytes and when we try to read the raw set of bytes, fragmentation might happen.
So properly implement a channel pipeline which consists of a String decoder / encoder (whatever you need) and this problem will go away.

TCP provides a stream of bytes, so you can't rely on receiving a complete message in one packet. You will need a handler in your pipeline that knows how your messages are framed. Netty provides some built-in handlers that you can adapt for your protocol. See Dealing with a Stream-based Transport in the Netty User Guide.

How to read binary data from socket with Apache MINA?

I know that a server sends MP3 stream after connecting to it and sending few bytes. How to read it's transmission with Apache MINA? Can you provide any examples please?

You need a client to read data from server. If it is possible to make a TCP connection with the server you can get help from this tutorial on Apache MINA TCP client
[UPDATE]
Data will be received in ClientSessionHandler's messageReceived. You can override this function according to you need. You may go through SumUp example to understand it fully.
[UPDATE 2]
To receive bytes in your case, you will have to update messageReceived of your session handler a bit. You can use IoBuffer to read byte. Something like this :
public void messageReceived(IoSession session, Object message) {
if (message instanceof IoBuffer) {
IoBuffer buffer = (IoBuffer) message;
byte[] b = new byte[buffer.remaining()];
buffer.get(b);
}
}

Forwarding incoming TCP "Commands" to a function?

In Java, how would you set up a socket listener that listened to a socket for a series of bytes that represented a command and on recieving called a method which parsed the incoming data and invoked the appropriate command?
Clarification:
My issue is not with handling the commands (Which might also be error codes or responses to commands from the server) but with creating the socket and listening to it.
More Clarification:
What I want to do is mimic the following line of .Net (C#) code:
_stream.BeginRead(_data,0, _data.Length, new
AsyncCallback(this.StreamEventHandler), _stream);
Where:
_stream is a network stream created from a socket
_data is an array of Byte of length 9
this.StreamHandler is a delegate (function pointer) which get executed when data is read.
I am rewriting a library from C# into Java and the component I am currently writing passes commands to a server over TCPIP but also has to be able to bubble up events/responses to the layer above it.
In C# this seems to be trivial and it's looking less and less so in Java.

Starting from my other answer: The specific part you request is the one that goes into the section: "Magic goes here". It can be done in ohh so many ways, but one is:
final InputStream in = socket.getInputStream();
// This creates a new thread to service the request.
new Thread(new Runnable(){
public void run(){
byte[] retrievedData= new byte[ITEM_LENGTH];
in.read(retrievedData, 0, ITEM_LENGTH);
in.close();
// Here call your delegate or something to process the data
callSomethingWithTheData(retrievedData);
}
}).start();

Have a small main method which sets up the socket and listens for incoming connections. Pass each connection to a worker object (possibly in its own thread).
The worker object should have two APIs: The server and the client. The client API gets a connection and reads data from it, the server API takes a connection and writes data to it.
I like to keep these two in a single class because that makes it much more simple to keep the two in sync. Use a helper class to encode/decode the data for transmission, so you have single point to decide how to transmit integers, commands, options, etc.
If you want to go further, define a command class and write code to serialize that to a socket connection and read it from it. This way, you worker objects just need to declare which command class they handle and the server/client API gets even more simple (at the expense of the command class).

I would
put each command into a class of its own, where each class implements a specific interface (e.g. Command)
create a Map<String,Command> which contains a lookup table from each command string to an instance of the class that implements that command

This should help.
Lesson 1: Socket Communications

The TCP connection provides you with one InputStream and one OutputStream. You could just poll the InputStream continuously for the next command (and its inputs) on a dedicated thread. ByteBuffer.wrap(byte[] array) may be useful in interpreting the bytes as chars, ints, longs, etc. You could also pass objects around using serialization.

Any naive approach most likely will not scale well.
Consider using a REST-approach with a suitable small web-server. Jetty is usually a good choice.

To create an listen to a socket, in a very naive way:
mServerSocket = new ServerSocket(port);
listening = true;
while (listening) {
// This call blocks until a connection is made
Socket socket = serverSocket.accept();
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
// Here you do your magic, reading and writing what you need from the streams
// You would set listening to true if you have some command to close the server
// remotely
out.close();
in.close();
socket.close();
}
Normally it is a good idea to delegate the processing of the input stream to some other thread, so you can answer the next request. Otherwise, you will answer all requests serially.
You also need to define some kind of protocol of what bytes you expect on the input and output streams, but from your question it looks like you already have one.

You could create an enum with one member per command
interface Comamnd {
// whatever you expect all command to know to perform their function
void perform(Context context);
}
enum Commands implements Command{
ACTIONONE() {
void perform(Context context) {
System.out.println("Action One");
}
},
ACTIONTWO() {
void perform(Context context) {
System.out.println("Action Two");
}
}
}
// initialise
DataInputStream in = new DataInputStream(socket.getInputStream());
// in a loop
byte[] retrievedData= new byte[ITEM_LENGTH];
in.readFully(retrievedData);
String command = new String(retrievedData, 0);
Commands.valueOf(command).perform(context);