We Keep Coding

Springboot jetty web socket client : How to know relationship between onMessage(String) and onMessage(byte[])

There is a web socket server who sends byte data after sending string data.
And the two data sent in turn are related to each other.
I could get data from the server like below.
#OnWebSocketMessage
public void onMessageString(Session session, String msg){
System.out.println("getRemoteAddress1:"+session.getRemoteAddress());
String vaHost = session.getRemoteAddress().toString().replaceAll("/", "");
System.out.println();
System.out.printf("Got msg: %s%n",msg);
}
#OnWebSocketMessage
public void onMessageBuffer(Session session, byte[] byteArray, int offset, int length) throws IOException {
System.out.println("onMessageBuffer");
System.out.println("getRemoteAddress2:"+session.getRemoteAddress());
FileUtils.writeByteArrayToFile(new File("C:/files/ws/"+System.nanoTime()+".jpg"), byteArray);
}
However, these two functions cannot determine the association between string data and byte data.
How can I correlate and process the data coming in turn from the web socket server?
Thank you all!

Netty server send a byte[] encoded by Protobuf, but C# client Socket.Receive keeps being 0

I am trying to accomplish an Unity game demo with network function, using C# for programming of client, and Java for server.
To be specific, server communication is implemented by Netty.
I also brought in Protobuf, which helps me define protocols of messages.
As I am new to server programming, dealing with packet merging and loss in TCP has not been considered in my code yet.
When I created sockets from client, and sent message to server, everything went well.
Problem happened when server replied:
In the client, an async method is ready to receive message. When I simply sent a string-format message from server, the method were able to get it.
But when I replaced the message with a 4-length byte[], which encoded from a Protobuf Message object, client just showed that it received NOTHING.
when I print what I've sent in the server console, it is like this:
00001000
00000001
00010000
00000001
My server code overrides channelRead and channelReadComplete functions of Netty.
In channelRead, ChannelHandlerContext.write was invoked to write the message to the transmission cache.
And in channelReadComplete, ChannelHandlerContext.flush was invoked, so that the message could be sent finally.
channelRead()
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
Request.MsgPack msgPack = (Request.MsgPack) msg;
Request.MsgPack.MsgType type = msgPack.getType();
switch (type)
{
case GetServerState:
final Request.GetServerState gssbody = msgPack.getGetServerState();
System.out.println("收到类型为" + type + "的消息，内容为：" +
"\nrequestId = " + gssbody.getRequestId()
);
byte[] bytes = ServerStateManager.getState(gssbody.getRequestId());
ctx.write(bytes);
break;
getState(): including Protobuf-encoding procedure
public static byte[] getState(int requestId)
{
ReturnServerState.Message.Builder replyBuilder = ReturnServerState.Message.newBuilder();
replyBuilder.setRequestId(requestId);
replyBuilder.setIsIdle(new ServerStateManager().isIdle());
return replyBuilder.build().toByteArray();
}
channelReadComplete()
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
try
{
ctx.flush();
}
finally
{
ctx.close();
}
}
Client code:
public class ShortLink
{
Socket clientSocket = null;
static byte[] result = new byte[1024];
Task ReceiveAsync<T>(string ip, int port)
{
return Task.Run(() =>
{
T component = default(T);
while (clientSocket.Receive(result) == 0)
{
break;
ReceiveAsync is invoked in the way of:
await ReceiveAsync<ReturnServerState>(ip, port);
when I found clientSocket.Receive(result) always output 0, I tried to log result[0], result[1], result[2], result[3] like this:
Debug.Log(Convert.ToString(result[0]) + ", " +
Convert.ToString(result[1]) + ", " +
Convert.ToString(result[2]) + ", " +
Convert.ToString(result[3]));
And the log turned to be 0,0,0,0.
I will be grateful for any idea of "why the client socket received nothing", and the solution.
Since I come from Asia, there may be a time lag between your reply and mine, and also English is not my mother tongue. However, I will try my best to reply in time.
Thanks a lot!

Okay..I have finally solve it myself
1.The usage "return replyBuilder.build().toByteArray()" is wrong because ProtoEncoder has already do toByteArray() for me:
public class ProtobufEncoder extends MessageToMessageEncoder<MessageLiteOrBuilder> {
public ProtobufEncoder() {
}
protected void encode(ChannelHandlerContext ctx, MessageLiteOrBuilder msg, List<Object> out) throws Exception {
if (msg instanceof MessageLite) {
out.add(Unpooled.wrappedBuffer(((MessageLite)msg).toByteArray()));
} else {
if (msg instanceof Builder) {
out.add(Unpooled.wrappedBuffer(((Builder)msg).build().toByteArray()));
}
}
}
}
So once I registered "new ProtobufEncoder()" in the Netty Channel Pipeline, I can just use "return replyBuilder.build()" - that is correct.
2.In "static byte[] result = new byte[1024];", The length of received message is defined casually, and it doesn't matter - until it really receives a message.
When receiving message, I shall always copy the message bytes to a new byte[] with a correct length firstly - or there will be just a 1024-length bytes[], with the data I need at the beginning, and several zeroes following, which will certainly fail to be decoded.

Netty Server throwing Exception when a simple Java Nio Client sends a message

I have a very simple Server designed in Java using Netty. I have also written a simple client using Java NIO Package. I can connect the Netty Server with the client but when the message is sent to the server I am getting the following exceptions:
My Output:
New client connected: /127.0.0.1:1125 java.io.IOException: An existing connection was forcibly closed by the remote host at sun.nio.ch.SocketDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1100) at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:372) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:123) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Thread.java:745)
My Server Program:
public class EchoServer{
private final int port;
public EchoServer(int port) {
this.port = port;
}
public void start() throws Exception {
EventLoopGroup group = new NioEventLoopGroup();
try {
ServerBootstrap b = new ServerBootstrap();
b.group(group)
.channel(NioServerSocketChannel.class)
.localAddress(new InetSocketAddress(port))
.childHandler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
System.out.println("New client connected: " + ch.localAddress());
ch.pipeline().addLast(newLengthFieldBasedFrameDecoder(Integer.MAX_VALUE,0,4,0,4))
.addLast(new LengthFieldPrepender(4))
.addLast(new EchoServerHandler());
}
});
ChannelFuture f = b.bind().sync();
f.channel().closeFuture().sync();
}
finally {
group.shutdownGracefully().sync();
}
}
public static void main (String [] args) throws Exception {
new EchoServer(1125).start();
}
}
My ServerHandler:
public class EchoServerHandler extends ChannelInboundHandlerAdapter{
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ByteBuf in = (ByteBuf) msg;
System.out.println(in.toString(CharsetUtil.UTF_8));
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
My NIO Java Client:
public class NIOJavaClient {
public static void main(String[] args) throws IOException, InterruptedException {
InetSocketAddress addr = new InetSocketAddress("localhost", 1125);
SocketChannel client = SocketChannel.open(addr);
System.out.println("Connecting to Server on port 1125...");
int i=0;
while(i<10){
byte[] message = new String("Hi Server\n").getBytes();
ByteBuffer buffer = ByteBuffer.wrap(message);
client.write(buffer);
buffer.clear();
i++;
}
client.close();
}
}
Now My required things are:
1) I can't send message from my client to server.Present I am using a Java NIO Client (not Netty). In future I may send message in bytes from a C# client to the netty server. Is it possible and if so how to do it?
2) I am using the LengthFieldBasedFrameDecoder and LengthPrepender to eliminate half strings (the strings that are sent from NETTY client are cut to a size everytime). I am successfully receiving the message when I use a Netty Client but when I use a different Client designed in Mina or C# I am getting an Exception: length frame exceeded. How can I eliminate this?
In a brief manner I want to connect a client designed in any language which sends bytes to the Netty Server and also eliminate the half strings. This is a critical situation for me. Any help would be really appreciable.

Since you are using the LengthFieldBasedFrameDecoder in your code, this means that you need to send your packets from simple program in the format this handler accepts, namely, prepended by a 4 byte value in network order that gives the length of the upcoming data.
Example for your NIO client
byte[] message = new String("Hi Server\n").getBytes();
byte[] fullMessage = new byte[message.length + 4];
fullMessage[0] = (byte)(message.length >> 24);
fullMessage[1] = (byte)(message.length >> 16);
fullMessage[2] = (byte)(message.length >> 8);
fullMessage[3] = (byte)message.length;
System.arraycopy(message, 0, messageFull, 4, message.length);
ByteBuffer buffer = ByteBuffer.wrap(fullMessage);
client.write(buffer);
buffer.clear();
There seems to be a better way, since your protocol work on lines ended with a newline (but this may also be caused by your attempts to debug the issue), instead of prefixing the packets with a length, just wait for a newline character using the DelimiterBasedFrameDecoder handler, use it like this:
ch.pipeline.addLast(new DelimiterBasedFrameDecoder(Delimiters.lineDelimiter()))
// Don't forget to remove your old 2 handlers

Netty ByteToMessageDecoder runs more times than it has to

I am trying to write a simple Client Server application using netty.
I followed this tutorial, and specifically the time server model along with the POJO model. My problem is with the ByteToMessageDecoder : it runs more times than it has to, meaning instead of stopping when it reads a null ByteBuf it reads one more time and for some reason, that I cannot understand, it finds the previous message that my client has sent! I am certain that the client only sends said message only once!
So the idea is this : a simple Client Server model where the Client sends a "DataPacket" with a "hello world" message in it and the server responds with a "DataPacket" with an "ACK". I am using the DataPacket type because in the future I want to pass more things in it, add a header and built something a bit more complicated...But for starters I need to see what am I doing wrong in this one...
The ERROR :
As you can see the Server starts normally, my Client (Transceiver) sends the message, the encoder activates and converts it from DataPacket to ByteBuf, the message is sent and received from the server, the Decoder from the Server activates and converts it from ByteBuf to DataPacket and then the Server handles it accordingly... It should sent the ACK and repeat the same backwards but this is were things go wrong and I cannot understand why.
I have read some posts here and already tried a LengthFieldBasedFrameDecoder, it did not work and I also want to see what is wrong with this one if possible and not use something else...
Code:
Encoder and Decoder class :
package org.client_server;
import java.util.List;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageDecoder;
import io.netty.handler.codec.MessageToByteEncoder;
import io.netty.util.CharsetUtil;
public class EncoderDecoder {
public static class NettyEncoder extends MessageToByteEncoder<DataPacket> {
#Override
protected void encode(ChannelHandlerContext ctx, DataPacket msg, ByteBuf out)
throws Exception {
System.out.println("Encode: "+msg.getData());
out.writeBytes(msg.convertData());
}
}
public static class NettyDecoder extends ByteToMessageDecoder{
#Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in,
List<Object> out) throws Exception {
if((in.readableBytes() < 4) ) {
return;
}
String msg = in.toString(CharsetUtil.UTF_8);
System.out.println("Decode:"+msg);
out.add(new DataPacket(msg));
}
}
}
Server handler :
class DataAvroHandler extends ChannelInboundHandlerAdapter {
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg)
throws Exception {
try {
DataPacket in = (DataPacket)msg;
System.out.println("[Server]: Message received..."+in.getData());
}finally {
ReferenceCountUtil.release(msg);
//ctx.close();
}
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx)
throws Exception {
System.out.println("[Server]: Read Complete...");
DataPacket pkt = new DataPacket("ACK!");
//pkt.setData(Unpooled.copiedBuffer("ACK", CharsetUtil.UTF_8));
ctx.writeAndFlush(pkt);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
throws Exception {
serverLog.warning("[Server]: Error..." + cause.toString());
ctx.close();
}
The Client handler :
class DataAvroHandlerCl extends ChannelInboundHandlerAdapter {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
System.out.println("[Transceiver]: Channel Active!!!");
DataPacket pkt = new DataPacket("Hello World!");
ChannelFuture f = ctx.writeAndFlush(pkt);
//f.addListener(ChannelFutureListener.CLOSE);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
try {
DataPacket in = (DataPacket)msg;
System.out.println("[Transceiver]: Message received..."+in.getData());
}finally {
ReferenceCountUtil.release(msg);
//ctx.close();
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
transLog.warning("[Transceiver] : Error..." + cause.getMessage());
ctx.close();
}
}
The Server and Client pipelines :
ch.pipeline().addLast("Decoder", new EncoderDecoder.NettyDecoder());
ch.pipeline().addLast("Encoder", new EncoderDecoder.NettyEncoder());
ch.pipeline().addLast("DataAvroHandler", new DataAvroHandler());

Your problem arises from using the toString() method of the ByteBuf in in your NettyDecoder.
Quoting from the javadoc (http://netty.io/4.0/api/io/netty/buffer/ByteBuf.html#toString%28java.nio.charset.Charset%29):
This method does not modify readerIndex or writerIndex of this buffer.
Now, the ByteToMessageDecoder doesn't know how many bytes you have actually decoded! It looks like you decoded 0 bytes, because the buffer's readerIndex was not modified and therefore you also get the error messages in your console.
You have to modify the readerIndex manually:
String msg = in.toString(CharsetUtil.UTF_8);
in.readerIndex(in.readerIndex() + in.readableBytes());
System.out.println("Decode:"+msg);

socket messages being split by netty

I wrote a REST server based on netty 4. The client handler looks something like the following.
The bytebuffer capacity in the msg provided by netty varies. When the client message is larger than the buffer the message gets split. What I find is that both channelRead and ChannelReadComplete get called for each fragment. What I usually see is that the ByteBuf is around 512, and the message around 600. I get a channelRead for the first 512 bytes, followed by a ChannelReadComplete for them, and then another channelRead for the remaining 100 bytes and a channelReadComplete for them - 2 messages instead of 1.
I found a few related questions here, but I am wondering what is the point of channelReadComplete? Is it really called after every channelRead? As long as there are bytes available, shouldn't they be read in before channelReadComplete is called?
public class ClientHandler extends ChannelInboundHandlerAdapter {
....
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
Report.debug("Read from client");
ByteBuf buf = (ByteBuf) msg;
String contents = buf.toString(io.netty.util.CharsetUtil.US_ASCII);
ReferenceCountUtil.release(msg);
ClientConnection client = ClientConnection.get(ctx);
if (client != null) {
client.messageText(contents); // adds text to buffer
return;
}
((parse serial number from contents, process registration))
ClientConnection.online(serialNumber, ctx); // register success, create the client object
}
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
ClientConnection client = ClientConnection.get(ctx);
if (client == null)
Report.debug("completed read of message from unregistered client");
else {
Report.debug("completed read of message from client " + client.serialNumber());
String contents = client.messageText();
... ((process message))
}
}
}

channelReadComplete is NOT called after each channelRead. The netty event loop will read from NIO socket and fire multiple channelRead until no more data to read or it should give up, then channelReadComplete is fired.

Yes, channelReadComplete() is called after each channelRead() in the pipeline has finished. If an exception occurs in channelRead() then it will jump to the method ecxeptionCaught().
So you should put code into channelReadComplete() that you only want to have executed on a successful channelRead().
For example this is what our project does:
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
// compute msg
ctx.fireChannelRead(msg); //tells the next handler
//in pipeline (if existing) to read the channel
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
ctx.writeAndFlush("OK");
ctx.fireChannelReadComplete();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
logger.error(Message.RCV_ERROR, cause.getMessage());
ctx.writeAndFlush(cause.getMessage());
ctx.close();
}
If the Client receives something different than "OK" then he doesn't have to send the rest.
If you're looking for a method that gets called after all packages have arrived then:
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
//close the writer that wrote the message to file (for example)
}
EDIT: You could also try sending bigger packages. The message size is controlled by the client, I think.