We Keep Coding

Netty server send a byte[] encoded by Protobuf, but C# client Socket.Receive keeps being 0

I am trying to accomplish an Unity game demo with network function, using C# for programming of client, and Java for server.
To be specific, server communication is implemented by Netty.
I also brought in Protobuf, which helps me define protocols of messages.
As I am new to server programming, dealing with packet merging and loss in TCP has not been considered in my code yet.
When I created sockets from client, and sent message to server, everything went well.
Problem happened when server replied:
In the client, an async method is ready to receive message. When I simply sent a string-format message from server, the method were able to get it.
But when I replaced the message with a 4-length byte[], which encoded from a Protobuf Message object, client just showed that it received NOTHING.
when I print what I've sent in the server console, it is like this:
00001000
00000001
00010000
00000001
My server code overrides channelRead and channelReadComplete functions of Netty.
In channelRead, ChannelHandlerContext.write was invoked to write the message to the transmission cache.
And in channelReadComplete, ChannelHandlerContext.flush was invoked, so that the message could be sent finally.
channelRead()
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
Request.MsgPack msgPack = (Request.MsgPack) msg;
Request.MsgPack.MsgType type = msgPack.getType();
switch (type)
{
case GetServerState:
final Request.GetServerState gssbody = msgPack.getGetServerState();
System.out.println("收到类型为" + type + "的消息，内容为：" +
"\nrequestId = " + gssbody.getRequestId()
);
byte[] bytes = ServerStateManager.getState(gssbody.getRequestId());
ctx.write(bytes);
break;
getState(): including Protobuf-encoding procedure
public static byte[] getState(int requestId)
{
ReturnServerState.Message.Builder replyBuilder = ReturnServerState.Message.newBuilder();
replyBuilder.setRequestId(requestId);
replyBuilder.setIsIdle(new ServerStateManager().isIdle());
return replyBuilder.build().toByteArray();
}
channelReadComplete()
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
try
{
ctx.flush();
}
finally
{
ctx.close();
}
}
Client code:
public class ShortLink
{
Socket clientSocket = null;
static byte[] result = new byte[1024];
Task ReceiveAsync<T>(string ip, int port)
{
return Task.Run(() =>
{
T component = default(T);
while (clientSocket.Receive(result) == 0)
{
break;
ReceiveAsync is invoked in the way of:
await ReceiveAsync<ReturnServerState>(ip, port);
when I found clientSocket.Receive(result) always output 0, I tried to log result[0], result[1], result[2], result[3] like this:
Debug.Log(Convert.ToString(result[0]) + ", " +
Convert.ToString(result[1]) + ", " +
Convert.ToString(result[2]) + ", " +
Convert.ToString(result[3]));
And the log turned to be 0,0,0,0.
I will be grateful for any idea of "why the client socket received nothing", and the solution.
Since I come from Asia, there may be a time lag between your reply and mine, and also English is not my mother tongue. However, I will try my best to reply in time.
Thanks a lot!

Okay..I have finally solve it myself
1.The usage "return replyBuilder.build().toByteArray()" is wrong because ProtoEncoder has already do toByteArray() for me:
public class ProtobufEncoder extends MessageToMessageEncoder<MessageLiteOrBuilder> {
public ProtobufEncoder() {
}
protected void encode(ChannelHandlerContext ctx, MessageLiteOrBuilder msg, List<Object> out) throws Exception {
if (msg instanceof MessageLite) {
out.add(Unpooled.wrappedBuffer(((MessageLite)msg).toByteArray()));
} else {
if (msg instanceof Builder) {
out.add(Unpooled.wrappedBuffer(((Builder)msg).build().toByteArray()));
}
}
}
}
So once I registered "new ProtobufEncoder()" in the Netty Channel Pipeline, I can just use "return replyBuilder.build()" - that is correct.
2.In "static byte[] result = new byte[1024];", The length of received message is defined casually, and it doesn't matter - until it really receives a message.
When receiving message, I shall always copy the message bytes to a new byte[] with a correct length firstly - or there will be just a 1024-length bytes[], with the data I need at the beginning, and several zeroes following, which will certainly fail to be decoded.

Paho MQTT client local address and port

My question is simple.
I want to know how to fix port client has.
According to Eclipse documents and IBM's, User has to fix broker address(this is absolutely natural). But There are no mentions about way of how to fix client site port.
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mq.javadoc.doc/WMQMQxrClasses/com/ibm/micro/client/mqttv3/MqttClient.html
MQTT must be also on TCP Layer, so I think it's possible to fix port.
If you have ideas, let me know.
Thanks

Under normal circumstance you don't set the source port for TCP connections, you just let the OS pick a random free port.
If you fix the source port then you can only ever run 1 instance of the client at a time on a given machine and if you get a connection failure you have to wait for the TCP stack to free that socket up again before you can reconnect.
If for some reason you REALLY NEED to fix the source port then you could probably write a custom javax.net.SocketFactory implementation that you can hard code the source port. Then pass this in as part of the MQTTConnectOptions object, but again I'm really struggling to come up with a reason this is a good idea.

You don't specify the client's port, as it is choosen by the OS, just as with any client to server connection, like you don't need to do that with a HTTP connection either.
You specify the IP address and port of the MQTT broker in the URL you pass in the form of tcp://<address>:<port>, for example tcp://localhost:4922.
The code below shows how I connect a Paho client in an OSGi context, where all connections parameters are retrieved from the bundle context.
private void configureMqtt(BundleContext context) throws IOException, MqttException {
String broker = context.getProperty("mqtt.broker");
if (broker == null) {
throw new ServiceException("Define mqtt.broker");
}
String client = context.getProperty("mqtt.clientname");
if (client == null) {
throw new ServiceException("Define mqtt.clientname");
}
String dir = context.getProperty("mqtt.persistence");
if (dir == null) {
dir = "mqtt";
};
File directory = context.getDataFile(dir);
directory.mkdirs();
logger.config(() -> String.format("MQTT broker: %s, clientname: %s persistence: %s", broker, client, directory.getAbsolutePath()));
connectOptions = new MqttConnectOptions();
connectOptions.setWill(GARAGE + "/door", new byte[0], 0, true);
String ka = context.getProperty("mqtt.keepalive");
if (ka != null) {
connectOptions.setKeepAliveInterval(Integer.valueOf(ka));
}
persistence = new MqttDefaultFilePersistence(directory.getCanonicalPath());
mqttClient = new MqttAsyncClient(broker, client, persistence);
mqttClient.setCallback(this);
connect();
}
private void connect() {
logger.fine("Connecting to MQTT broker");
try {
IMqttToken token = mqttClient.connect(connectOptions);
IMqttActionListener listener = new IMqttActionListener() {
#Override
public void onSuccess(IMqttToken asyncActionToken) {
logger.log(Level.INFO, "Connected to MQTT broker");
}
#Override
public void onFailure(IMqttToken asyncActionToken, Throwable exception) {
logger.log(Level.WARNING, "Could not connect to MQTT broker, retrying in 3 seconds", exception);
service.schedule(this::connect, 3, TimeUnit.SECONDS);
}
};
token.setActionCallback(listener);
} catch (MqttException e) {
logger.log(Level.SEVERE, "Cannot reconnect to MQTT broker, giving up", e);
}
}

Smack 4.1.0 GCM CCS stops responding after a while

Have implemented gcm ccs for chat module and i am able to send and receive messages. Below is the main connection module,
config = XMPPTCPConnectionConfiguration.builder()
.setServiceName("gcm-pesu.googleapis.com")
.setPort(GCM_PORT)
.setHost(GCM_SERVER)
.setCompressionEnabled(false)
.setConnectTimeout(30000)
.setSecurityMode(SecurityMode.ifpossible)
.setSendPresence(false)
.setSocketFactory(SSLSocketFactory.getDefault())
.build();
connection = new XMPPTCPConnection(config);
connection.connect();
Roster roster = Roster.getInstanceFor(connection);
roster.setRosterLoadedAtLogin(false);
connection.addConnectionListener(new LoggingConnectionListener());
// Handle incoming packets
connection.addAsyncStanzaListener(new MyStanzaListener(), new MyStanzaFilter());
// Log all outgoing packets
connection.addPacketInterceptor(new MyStanzaInterceptor(), new MyStanzaFilter());
connection.login(mProjectId + "#gcm.googleapis.com", mApiKey);
logger.info("logged in: " + mProjectId);
PingManager pm = PingManager.getInstanceFor(connection);
pm.setPingInterval(300);
pm.pingMyServer();
pm.registerPingFailedListener(new PingFailedListener() {
#Override
public void pingFailed() {
connection.disconnect();
logger.error("GCM CCS, Ping failed !!");
}
});
The problem i am running into is not receiving any message from GCM, sent by client device after a while. Though, the heartbeat looks normal and i do get pong from GCM even in that case. Is it something to do with SSL ?
Have handled connection draining case as follows,
String controlType = (String) jsonObject.get("control_type");
volatile boolean connectionDraining = false;
if ("CONNECTION_DRAINING".equals(controlType)) {
connectionDraining = true;
try {
connection.disconnect();
connect();
connectionDraining = false;
} catch (Exception e) {
logger.error("Error establishing new connection after draining ", e);
}
}

Implemented queue of channels when one of it is draining.
private Deque<Channel> channels;
protected void handleControlMessage(Map<String, Object> jsonObject) {
logger.info("Control message : " + jsonObject);
String controlType = (String) jsonObject.get("control_type");
if ("CONNECTION_DRAINING".equals(controlType)) {
connectionDraining = true;
}
}
Create new channel while sending message
public void sendDownstreamMessage(String jsonRequest) {
Channel channel = channels.peekFirst();
try {
if (channel.connectionDraining) {
synchronized (channels) {
channel = channels.peekFirst();
if (channel.connectionDraining) {
channels.addFirst(connect());
channel = channels.peekFirst();
}
}
}
channel.send(jsonRequest);
} catch (Exception e) {
logger.error("Message not sent. Error in connecting :", e);
}
}
GCM will take care of closing the other. This resolved the issue.

I believe you're facing a common case using gcm css that is not very visible in the documentation.
If you look in the doc, Control Messages you'll read:
Periodically, CCS needs to close down a connection to perform load balancing. Before it closes the connection, CCS sends a CONNECTION_DRAINING message to indicate that the connection is being drained and will be closed soon. "Draining" refers to shutting off the flow of messages coming into a connection, but allowing whatever is already in the pipeline to continue. When you receive a CONNECTION_DRAINING message, you should immediately begin sending messages to another CCS connection, opening a new connection if necessary. You should, however, keep the original connection open and continue receiving messages that may come over the connection (and ACKing them)—CCS handles initiating a connection close when it is ready.

Netty (4) client server communication in the same TCP Session

I need to build a client that initiates a TCP connection with a server and upon response it sends a hand shake request every 10 seconds and gets a response from the server. The server will be able to send another type of request which my client needs to read and act upon. I am using netty 4.0.26.Final.
I have built a client and a dummy server but I am facing an issue which possibly means that there is something I have not understood.
My Client:
String host = "localhost";
int port = 9884;
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new MyChannelPipeline());
// Start the client.
ChannelFuture f = b.connect(host, port).sync();
String line = "line";
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
while (!line.equals("exit")) {
line = in.readLine();
if (line == null) {
break;
}
}
// Wait until the connection is closed.
f.channel().closeFuture().sync();
} finally {
workerGroup.shutdownGracefully();
}
The ChannelPipleline:
#Override
public void initChannel(Channel ch) throws Exception {
ChannelPipeline channelPipeline = ch.pipeline();
//Encodes every request send from the client to the server
channelPipeline.addLast("clientRequestEncoder", new ClientRequestEncoder());
//Implements channelActive and exceptionCaught
channelPipeline.addLast("initialRequestHandler", new InitialRequestHandler());
channelPipeline.addLast("byteArrayDecoder", new ByteArrayDecoder());
channelPipeline.addLast("serverResponseDecoder", new ServerResponseDecoder());
channelPipeline.addLast("serverRequestDecoder", new ServerRequestDecoder());
//Reads the responses from the client requests AND
//reads the inbound requests from the server - Implements channelRead
//and exceptionCaught
channelPipeline.addLast("myResponseHandler", new MyResponseHandler());
}
The problem is that when I flush the response to the server (in MyResponseHandler) and exception is caught in InitialRequestHandler:
ERROR=java.lang.UnsupportedOperationException:unsupported message type: ServerResponse (expected: ByteBuf, FileRegion)
I don't see why the response is not flushed back to the server while the hand shake request is always properly flushed. In both write and flush I have used a ChannelFuture and onOperationComplete this listener f.addListener(ChannelFutureListener.FIRE_EXCEPTION_ON_FAILURE); is fired on failure.
Can I use two handlers in the same pipeline or is it bad practise? Moreover how should I fire an unregister event triggered by user input?

I solved this using one Handler that overrides channelActive and channelRead and I rearranged the encoders and decoders properly. I also solved the "unregister event triggered by user input" this way:
String line = "line";
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
do {
logger.info("You typed: " + line + ". Please type 'exit' to terminate the program!");
line = in.readLine();
} while (!line.equals("exit"));
logger.info("You typed: " + line + ". Please wait until the application is successfully shutdown...");
f.addListener(new ChannelFutureListener() {
public void operationComplete(ChannelFuture future) throws Exception {
myChannelPipeline.getMyClientHandler().sendDisconnectRequest(future);
}
});
In sendDisconnectReqeust I send the final request and when I get the final response (in channelRead of MyHandler) I call disconnect on the pipeline:
ChannelPromise cp = new DefaultChannelPromise(ctx.channel());
ctx.channel().pipeline().disconnect(cp);
However I still have other issues with inbound requests that are never received by my client.

How can I forward a message that received a server to another server in Netty (Java)?

I have some clients, they communicate with one server and I need that server forward the message to another second server. Then, receive the message from the second server and send to the client.
With this method, I achieve connecting to the second server but it doesn't receive the message and throws me the following exception:
EXCEPTION: java.nio.channels.NotYetConnectedException. java.nio.channels.NotYetConnectedException
public void messageReceived(ChannelHandlerContext ctx, final MessageEvent e) throws IOException, Exception {
response = "hola" + "\r\n";
Main.creaLog("Mensaje recibido del conc: " + e.getMessage().toString());
Main.creaLog("Mensaje enviado al servidor : " + response);
ClientBootstrap bootstrap1 = new ClientBootstrap(
new NioClientSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool()));
// Configure the pipeline factory.
//bootstrap1.setPipelineFactory(new CLIENTE.ClientePipelineFactory());
bootstrap1.setPipelineFactory(new ChannelPipelineFactory() {
public ChannelPipeline getPipeline() {
return Channels.pipeline(new ClienteHandler());
}
});
final ChannelFuture future = bootstrap1.connect(new InetSocketAddress("172.16.10.14", 12355));
Channel channel = future.getChannel();
if (channel.isWritable()) {
ChannelFuture lastWriteFuture = channel.write(e.getMessage().toString() + "\r\n");
}
close = true;
// We do not need to write a ChannelBuffer here.
// We know the encoder inserted at TelnetPipelineFactory will do the conversion.
ChannelFuture future = e.getChannel().write(response + "\r\n");
//CIERRA LA CONEXION
if (close) {
future.addListener(ChannelFutureListener.CLOSE);
}
}
I'm very thanksful if anybody can help me.

Have a look at Netty Proxy example
Right now, you are basically attempting to connect to the remote server on every message that you receive. This probably isn't what you want. You might want to connect to the remote server only once (i.e. outbound channel in Netty proxy example) and forward a new incoming message to that specific channel.