I want to start both TCP echo server and client in one app, client after server.
What I do is:
Start a client in a ChannelFutureListener returned by server.bind().sync() like this:
public void runClientAndServer() {
server.run().addListener((ChannelFutureListener) future -> {
// client.run(); //(1) this doesn't work!
new Thread(()->client.run()).start(); //(2) this works!
});
}
and server.run() is like this:
public ChannelFuture run() {
ServerBootstrap b = new ServerBootstrap();
//doing channel config stuff
return b.bind(6666).sync();
}
and client.run() is like this:
public void run() {
Bootstrap b = new Bootstrap();
//do some config stuff
f = b.connect(host, port).sync(); //wait till connected to server
}
What happens is:
In the statement (2) that works just fine; While in the statement (1), the client sent message, that can be observed in Wireshark, and the server replies TCP ACK segment, but no channelRead() method in server side ChannelInboundHandlerAdapter is invoked, nor any attempt to write message to socket can be observed, like this capture:
wireshark capture
I guess there must be something wrong with Netty threads, but I just cannot figure out
I have prepared an example based on the newest netty version (4.1.16.Final) and the code you posted. This works fine without an extra thread maybe you did something wrong initializing your server or client.
private static final NioEventLoopGroup EVENT_LOOP_GROUP = new NioEventLoopGroup();
private static ChannelFuture runServer() throws Exception {
return new ServerBootstrap()
.group(EVENT_LOOP_GROUP)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<Channel>() {
#Override
protected void initChannel(Channel channel) throws Exception {
System.out.println("S - Channel connected: " + channel);
channel.pipeline().addLast(new SimpleChannelInboundHandler<ByteBuf>() {
#Override
protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception {
System.out.println("S - read: " + msg.toString(StandardCharsets.UTF_8));
}
});
}
})
.bind(6666).sync();
}
private static void runClient() throws Exception {
new Bootstrap()
.group(EVENT_LOOP_GROUP)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<Channel>() {
#Override
protected void initChannel(Channel channel) throws Exception {
System.out.println("C - Initilized client");
channel.pipeline().addLast(new SimpleChannelInboundHandler<ByteBuf>() {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
System.out.println("C - write: Hey this is a test message enjoy!");
ctx.writeAndFlush(Unpooled.copiedBuffer("Hey this is a test message enjoy!".getBytes(StandardCharsets.UTF_8)));
}
#Override
protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception { }
});
}
})
.connect("localhost", 6666).sync();
}
public static void main(String[] args) throws Exception {
runServer().addListener(future -> {
runClient();
});
}
That's what the output should look like:
C - Initilized client
C - write: Hey this is a test message enjoy!
S - Channel connected: [id: 0x1d676489, L:/127.0.0.1:6666 - R:/127.0.0.1:5079]
S - read: Hey this is a test message enjoy!
I also tried this example with a single threaded eventloopgroup which also worked fine but throw me an BlockingOperationException which did not affect the functionality of the program. If this code should work fine you should probably check your code and try to orientate your code on this example (Please don't create inline ChannelHandler for your real project like I did in this example).
Related
I have following binding to handle UDP packets
private void doStartServer() {
final UDPPacketHandler udpPacketHandler = new UDPPacketHandler(messageDecodeHandler);
workerGroup = new NioEventLoopGroup(threadPoolSize);
try {
final Bootstrap bootstrap = new Bootstrap();
bootstrap
.group(workerGroup)
.handler(new LoggingHandler(nettyLevel))
.channel(NioDatagramChannel.class)
.option(ChannelOption.SO_BROADCAST, true)
.handler(udpPacketHandler);
bootstrap
.bind(serverIp, serverPort)
.sync()
.channel()
.closeFuture()
.await();
} finally {
stop();
}
}
and handler
#ChannelHandler.Sharable << note this
#Slf4j
#AllArgsConstructor
public class UDPPacketHandler extends SimpleChannelInboundHandler<DatagramPacket> {
private final MessageP54Handler messageP54Handler;
#Override
public void channelReadComplete(final ChannelHandlerContext ctx) {
ctx.flush();
}
#Override
public void exceptionCaught(final ChannelHandlerContext ctx, final Throwable cause) {
log.error("Exception in UDP handler", cause);
ctx.close();
}
}
At some point I get this exception java.net.SocketException: Network dropped connection on reset: no further information which is handled in exceptionCaught. This triggers ChannelHandlerContext to close. And at this point whole my server stops (executing on finally block from first snippet)
How to correctly handle exception so that I can handle new connections even after such exception occurs?
you shouldn't close the ChannelHandlerContext on an IOException when using a DatagramChannel. As DatagramChannel is "connection-less" the exception is specific to one "receive" or one "send" operation. So just log it (or whatever you want to do) and move on.
I'm trying to create a Netty (4.1) POC which can forward h2c (HTTP2 without TLS) frames onto a h2c server - i.e. essentially creating a Netty h2c proxy service. Wireshark shows Netty sending the frames out, and the h2c server replying (for example with the response header and data), although I'm then having a few issues receiving/processing the response HTTP frames within Netty itself.
As a starting point, I've adapted the multiplex.server example (io.netty.example.http2.helloworld.multiplex.server) so that in HelloWorldHttp2Handler, instead of responding with dummy messages, I connect to a remote node:
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
Channel remoteChannel = null;
// create or retrieve the remote channel (one to one mapping) associated with this incoming (client) channel
synchronized (lock) {
if (!ctx.channel().hasAttr(remoteChannelKey)) {
remoteChannel = this.connectToRemoteBlocking(ctx.channel());
ctx.channel().attr(remoteChannelKey).set(remoteChannel);
} else {
remoteChannel = ctx.channel().attr(remoteChannelKey).get();
}
}
if (msg instanceof Http2HeadersFrame) {
onHeadersRead(remoteChannel, (Http2HeadersFrame) msg);
} else if (msg instanceof Http2DataFrame) {
final Http2DataFrame data = (Http2DataFrame) msg;
onDataRead(remoteChannel, (Http2DataFrame) msg);
send(ctx.channel(), new DefaultHttp2WindowUpdateFrame(data.initialFlowControlledBytes()).stream(data.stream()));
} else {
super.channelRead(ctx, msg);
}
}
private void send(Channel remoteChannel, Http2Frame frame) {
remoteChannel.writeAndFlush(frame).addListener(new GenericFutureListener() {
#Override
public void operationComplete(Future future) throws Exception {
if (!future.isSuccess()) {
future.cause().printStackTrace();
}
}
});
}
/**
* If receive a frame with end-of-stream set, send a pre-canned response.
*/
private void onDataRead(Channel remoteChannel, Http2DataFrame data) throws Exception {
if (data.isEndStream()) {
send(remoteChannel, data);
} else {
// We do not send back the response to the remote-peer, so we need to release it.
data.release();
}
}
/**
* If receive a frame with end-of-stream set, send a pre-canned response.
*/
private void onHeadersRead(Channel remoteChannel, Http2HeadersFrame headers)
throws Exception {
if (headers.isEndStream()) {
send(remoteChannel, headers);
}
}
private Channel connectToRemoteBlocking(Channel clientChannel) {
try {
Bootstrap b = new Bootstrap();
b.group(new NioEventLoopGroup());
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.remoteAddress("localhost", H2C_SERVER_PORT);
b.handler(new Http2ClientInitializer());
final Channel channel = b.connect().syncUninterruptibly().channel();
channel.config().setAutoRead(true);
channel.attr(clientChannelKey).set(clientChannel);
return channel;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
When initializing the channel pipeline (in Http2ClientInitializer), if I do something like:
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(Http2MultiplexCodecBuilder.forClient(new Http2OutboundClientHandler()).frameLogger(TESTLOGGER).build());
ch.pipeline().addLast(new UserEventLogger());
}
Then I can see the frames being forwarded correctly in Wireshark and the h2c server replies with the header and frame data, but Netty replies with a GOAWAY [INTERNAL_ERROR] due to:
14:23:09.324 [nioEventLoopGroup-3-1] WARN
i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was
fired, and it reached at the tail of the pipeline. It usually means
the last handler in the pipeline did not handle the exception.
java.lang.IllegalStateException: Stream object required for
identifier: 1 at
io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.requireStream(Http2FrameCodec.java:587)
at
io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.onHeadersRead(Http2FrameCodec.java:550)
at
io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.onHeadersRead(Http2FrameCodec.java:543)...
If I instead try making it have the pipeline configuration from the http2 client example, e.g.:
#Override
public void initChannel(SocketChannel ch) throws Exception {
final Http2Connection connection = new DefaultHttp2Connection(false);
ch.pipeline().addLast(
new Http2ConnectionHandlerBuilder()
.connection(connection)
.frameLogger(TESTLOGGER)
.frameListener(new DelegatingDecompressorFrameListener(connection, new InboundHttp2ToHttpAdapterBuilder(connection)
.maxContentLength(maxContentLength)
.propagateSettings(true)
.build() ))
.build());
}
Then I instead get:
java.lang.UnsupportedOperationException: unsupported message type:
DefaultHttp2HeadersFrame (expected: ByteBuf, FileRegion) at
io.netty.channel.nio.AbstractNioByteChannel.filterOutboundMessage(AbstractNioByteChannel.java:283)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:882)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1365)
If I then add in a HTTP2 frame codec (Http2MultiplexCodec or Http2FrameCodec):
#Override
public void initChannel(SocketChannel ch) throws Exception {
final Http2Connection connection = new DefaultHttp2Connection(false);
ch.pipeline().addLast(
new Http2ConnectionHandlerBuilder()
.connection(connection)
.frameLogger(TESTLOGGER)
.frameListener(new DelegatingDecompressorFrameListener(connection, new InboundHttp2ToHttpAdapterBuilder(connection)
.maxContentLength(maxContentLength)
.propagateSettings(true)
.build() ))
.build());
ch.pipeline().addLast(Http2MultiplexCodecBuilder.forClient(new Http2OutboundClientHandler()).frameLogger(TESTLOGGER).build());
}
Then Netty sends two connection preface frames, resulting in the h2c server rejecting with GOAWAY [PROTOCOL_ERROR]:
So that is where I am having issues - i.e. configuring the remote channel pipeline such that it will send the Http2Frame objects without error, but also then receive/process them back within Netty when the response is received.
Does anyone have any ideas/suggestions please?
I ended up getting this working; the following Github issues contain some useful code/info:
Generating a Http2StreamChannel, from a Channel
A Http2Client with Http2MultiplexCode
I need to investigate a few caveats further, although the gist of the approach is that you need to wrap your channel in a Http2StreamChannel, meaning that my connectToRemoteBlocking() method ends up as:
private Http2StreamChannel connectToRemoteBlocking(Channel clientChannel) {
try {
Bootstrap b = new Bootstrap();
b.group(new NioEventLoopGroup()); // TODO reuse existing event loop
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.remoteAddress("localhost", H2C_SERVER_PORT);
b.handler(new Http2ClientInitializer());
final Channel channel = b.connect().syncUninterruptibly().channel();
channel.config().setAutoRead(true);
channel.attr(clientChannelKey).set(clientChannel);
// TODO make more robust, see example at https://github.com/netty/netty/issues/8692
final Http2StreamChannelBootstrap bs = new Http2StreamChannelBootstrap(channel);
final Http2StreamChannel http2Stream = bs.open().syncUninterruptibly().get();
http2Stream.attr(clientChannelKey).set(clientChannel);
http2Stream.pipeline().addLast(new Http2OutboundClientHandler()); // will read: DefaultHttp2HeadersFrame, DefaultHttp2DataFrame
return http2Stream;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
Then to prevent the "Stream object required for identifier: 1" error (which is essentially saying: 'This (client) HTTP2 request is new, so why do we have this specific stream?' - since we were implicitly reusing the stream object from the originally received 'server' request), we need to change to use the remote channel's stream when forwarding our data on:
private void onHeadersRead(Http2StreamChannel remoteChannel, Http2HeadersFrame headers) throws Exception {
if (headers.isEndStream()) {
headers.stream(remoteChannel.stream());
send(remoteChannel, headers);
}
}
Then the configured channel inbound handler (which I've called Http2OutboundClientHandler due to its usage) will receive the incoming HTTP2 frames in the normal way:
#Sharable
public class Http2OutboundClientHandler extends SimpleChannelInboundHandler<Http2Frame> {
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
super.exceptionCaught(ctx, cause);
cause.printStackTrace();
ctx.close();
}
#Override
public void channelRead0(ChannelHandlerContext ctx, Http2Frame msg) throws Exception {
System.out.println("Http2OutboundClientHandler Http2Frame Type: " + msg.getClass().toString());
}
}
I am trying to use the following code which is an implementation of web sockets in Netty Nio. I have implment a JavaFx Gui and from the Gui I want to read the messages that are received from the Server or from other clients. The NettyClient code is like the following:
public static ChannelFuture callBack () throws Exception{
String host = "localhost";
int port = 8080;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
}));
}
});
ChannelFuture f = b.connect(host, port).sync();
//f.channel().closeFuture().sync();
return f;
}
finally {
//workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
ChannelFuture ret;
ClientHandler obj = new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
});
ret = callBack();
int connected = connectedClients;
if (connected != 2) {
System.out.println("The number if the connected clients is not two before locking");
synchronized (lock) {
while (true) {
connected = connectedClients;
if (connected == 2)
break;
System.out.println("The number if the connected clients is not two");
lock.wait();
}
}
}
System.out.println("The number if the connected clients is two: " + connected );
ret.channel().read(); // can I use that from other parts of the code in order to read the incoming messages?
}
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages? Do I need to call again callBack, or how can I received the updated message of the channel? Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
By reading that code,the NettyClient is used to create connection(ClientHandler ),once connect done,ClientHandler.channelActive is called by Netty,if you want send data to server,you should put some code here. if this connection get message form server, ClientHandler.channelRead is called by Netty, put your code to handle message.
You also need to read doc to know how netty encoder/decoder works.
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages?
share those ClientHandler created by NettyClient(NettyClient.java line 29)
Do I need to call again callBack, or how can I received the updated message of the channel?
if server message come,ClientHandler.channelRead is called.
Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
yes you could,but not a netty way,to play with netty,you write callbacks(when message come,when message sent ...),wait netty call your code,that is : the driver is netty,not you.
last,do you really need such a heavy library to do network?if not ,try This code,it simple,easy to understanding
There is a middleware in between of two other softwares. In the middleware I'm routing Apache ActiveMQ messages by Apache Camel.
the first software uses middleware to send message to the 3rd software and the 3rd one reply the message to the first(using middleware).
1stSoftware <<=>> Middleware <<=>> 3rdSoftware
Problem:
when with the first one I send message to the middleware, middleware sends that message directly to ActiveMQ.DLQ and the 3rd one can not consume it!(Interesting point is this: when I copy that message to the main queue with the Admin panel of ActiveMQ, software can consume it properly!)
What's the problem?! It was working until I changed the Linux date!!!!!!!
Middleware is like this:
#SuppressWarnings("deprecation")
public class MiddlewareDaemon {
private Main main;
public static void main(String[] args) throws Exception {
MiddlewareDaemon middlewareDaemon = new MiddlewareDaemon();
middlewareDaemon.boot();
}
public void boot() throws Exception {
main = new Main();
main.enableHangupSupport();
//?wireFormat.maxInactivityDuration=0
main.bind("activemq", activeMQComponent("tcp://localhost:61616")); //ToGet
main.bind("activemq2", activeMQComponent("tcp://192.168.10.103:61616")); //ToInOut
main.addRouteBuilder(new MyRouteBuilder());
System.out.println("Starting Camel(MiddlewareDaemon). Use ctrl + c to terminate the JVM.\n");
main.run();
}
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
intercept().to("log:Midlleware?level=INFO&showHeaders=true&showException=true&showCaughtException=true&showStackTrace=true");
from("activemq:queue:Q.Midlleware")
.process(new Processor() {
public void process(Exchange exchange) {
Map<String, Object> header = null;
try {
Message in = exchange.getIn();
header = in.getHeaders();
} catch (Exception e) {
log.error("Exception:", e);
header.put("Midlleware_Exception", e.getMessage() + " - " + e);
}
}
})
.inOut("activemq2:queue:Q.Comp2")
}
}
}
And the 3rd software(Replier): (this is a daemon like above, i just copied the RouteBuilder part)
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() {
intercept().to("log:Comp2?level=INFO&showHeaders=true&showException=true&showCaughtException=true&showStackTrace=true");
from("activemq:queue:Q.Comp2")
.process(new Processor() {
public void process(Exchange exchange) {
Message in = exchange.getIn();
Map<String, Object> headers = null;
try {
headers = in.getHeaders();
in.setBody(ZipUtil.compress(/*somResults*/));
} catch (Exception e) {
log.error("Exception", e);
in.setBody(ZipUtil.compress("[]"));
in.getHeaders().put("Comp2_Exception", e.getMessage() + " - " + e);
}
}
})
;
}
}
If the only thing you changed is the time on one of the servers, then this might well be the problem.
For communications over MQ to work properly, it is essential that all involved host systems have their clock set in sync. In the case of ActiveMQ there is a default message time-to-live (30 seconds, I think) for response queues. If the responding system is more then 30 seconds in the future relative to the host running ActiveMQ, then ActiveMQ will immediately expire the message and move it to the DLQ.
I'm trying to create a TCP proxy that forwards request to many other TCP endpoints using Netty/Java.
For example:
/--> SERVER A
Client A --> PROXY --
\--> SERVER B
If Client A sends a TCP command through the proxy, the proxy opens two TCP connections to Server A and Server B, and concurrently proxies the request sent by Client A to both of them.
If Client A subsequently sends another command, the proxy theoretically has previously cached the two connections in a pool, so without opening two new connections again, proxies the request to the two servers.
Regarding the response handling, I would like to have two options:
Show two responses one after the other to Client A.
Or completely ignore the response.
If a connection is lost or closed, the proxy should be able to automatically recreate it and add it back to the connection pool.
I've been looking at the Netty examples, and tried to use ChannelGroup to handle the connection pool, but without success. Also, in my code below, after sending the first request the proxy stops working. Any tips?
import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.Unpooled;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelOption;
import io.netty.channel.group.ChannelGroup;
import io.netty.channel.group.DefaultChannelGroup;
import io.netty.util.concurrent.GlobalEventExecutor;
import java.util.LinkedList;
import java.util.List;
public class TcpProxyHandler extends ChannelInboundHandlerAdapter {
private static List<String> hosts = new LinkedList<>();
private static List<String> connected = new LinkedList<>();
static {
hosts.add("127.0.0.1:10000");
hosts.add("127.0.0.1:20000");
}
static final ChannelGroup channels = new DefaultChannelGroup(
GlobalEventExecutor.INSTANCE);
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
final Channel inboundChannel = ctx.channel();
for (String host : hosts) {
if (!connected.contains(host)) {
String address = host.split(":")[0];
int port = Integer.parseInt(host.split(":")[1]);
Channel outboundChannel = ConnectionPool.getConnection(address,
port);
if (outboundChannel == null) {
Bootstrap b = new Bootstrap();
b.group(inboundChannel.eventLoop())
.channel(ctx.channel().getClass())
.handler(new TcpProxyBackendHandler(inboundChannel))
.option(ChannelOption.AUTO_READ, false);
ChannelFuture f = b.connect(address, port);
outboundChannel = f.channel();
f.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future)
throws Exception {
if (future.isSuccess()) {
// connection complete start to read first data
inboundChannel.read();
} else {
// Close the connection if the connection
// attempt
// has failed.
inboundChannel.close();
}
}
});
channels.add(outboundChannel);
connected.add(host);
System.out.println("Connected to " + host);
}
}
}
}
#Override
public void channelRead(final ChannelHandlerContext ctx, Object msg)
throws Exception {
channels.flushAndWrite(msg);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
throws Exception {
cause.printStackTrace();
ctx.close();
}
static void closeOnFlush(Channel ch) {
if (ch.isActive()) {
ch.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
static class TcpProxyBackendHandler extends ChannelInboundHandlerAdapter {
private final Channel inboundChannel;
public TcpProxyBackendHandler(Channel inboundChannel) {
this.inboundChannel = inboundChannel;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
ctx.read();
ctx.write(Unpooled.EMPTY_BUFFER);
}
#Override
public void channelRead(final ChannelHandlerContext ctx, Object msg)
throws Exception {
inboundChannel.writeAndFlush(msg).addListener(
new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future)
throws Exception {
if (future.isSuccess()) {
ctx.channel().read();
} else {
future.channel().close();
}
}
});
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
TcpProxyHandler.closeOnFlush(inboundChannel);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
throws Exception {
cause.printStackTrace();
TcpProxyHandler.closeOnFlush(ctx.channel());
}
}
}
You can try to call connect() and read() in another thread to give a chance to ChannelGrop's worker to do its work.
If you haven't tried yet, enable AUTO_READ again and remove the manual invokations to read(). You may initialize your server also with auto read set to false, you could try to change this too.