We Keep Coding

spring boot handling for TCP/IP server

Have to implement a server for handling the following protocol through Ethernet connection:
Establishing a connection
The client connects to the configured server via TCP / IP.
After the connection has been established, the client initially sends a heartbeat message to the
Server:
{
"MessageID": "Heartbeat"
}
Response:
{
"ResponseCode": "Ok"
}
Communication process
To maintain the connection, the client sends every 10 seconds when inactive
Heartbeat message.
Server and client must close the connection if they are not receiving a message for longer than 20 seconds.
An answer must be given within 5 seconds to request.
If no response is received, the connection must also be closed.
The protocol does not contain numbering or any other form of identification.
Communication partner when sending the responses makes sure that they are in the same sequence.
Message structure:
The messages are embedded in an STX-ETX frame.
STX (0x02) message ETX (0x03)
An `escaping` of STX and ETX within the message is not necessary since it is in JSON format
Escape sequence are following:
JSON.stringify ({"a": "\ x02 \ x03 \ x10"}) → "{" a \ ": " \ u0002 \ u0003 \ u0010 \ "}"
Not only heartbeat messages should be used. A typical message should be like:
{
"MessageID": "CheckAccess"
"Parameters": {
"MediaType": "type",
"MediaData": "data"
}
}
And the appropriate response:
{
"ResponseCode": "some-code",
"DisplayMessage": "some-message",
"SessionID": "some-id"
}
It should be a multi-client server. And protocol doesn't have any identification.
However, we have to identify the client at least the IP address from which it was sent.
Could not find some solution on how to add such server to Spring Boot application and enable on startup & handle input and output logic for it.
Any suggestions are highly appreciated.
Solution
Configured following for TCP server:
#Slf4j
#Component
#RequiredArgsConstructor
public class TCPServer {
private final InetSocketAddress hostAddress;
private final ServerBootstrap serverBootstrap;
private Channel serverChannel;
#PostConstruct
public void start() {
try {
ChannelFuture serverChannelFuture = serverBootstrap.bind(hostAddress).sync();
log.info("Server is STARTED : port {}", hostAddress.getPort());
serverChannel = serverChannelFuture.channel().closeFuture().sync().channel();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
#PreDestroy
public void stop() {
if (serverChannel != null) {
serverChannel.close();
serverChannel.parent().close();
}
}
}
#PostConstruct launches server during startup of an application.
Configuration for it as well:
#Configuration
#RequiredArgsConstructor
#EnableConfigurationProperties(NettyProperties.class)
public class NettyConfiguration {
private final LoggingHandler loggingHandler = new LoggingHandler(LogLevel.DEBUG);
private final NettyProperties nettyProperties;
#Bean(name = "serverBootstrap")
public ServerBootstrap bootstrap(SimpleChannelInitializer initializer) {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup(), workerGroup())
.channel(NioServerSocketChannel.class)
.handler(loggingHandler)
.childHandler(initializer);
bootstrap.option(ChannelOption.SO_BACKLOG, nettyProperties.getBacklog());
bootstrap.childOption(ChannelOption.SO_KEEPALIVE, nettyProperties.isKeepAlive());
return bootstrap;
}
#Bean(destroyMethod = "shutdownGracefully")
public NioEventLoopGroup bossGroup() {
return new NioEventLoopGroup(nettyProperties.getBossCount());
}
#Bean(destroyMethod = "shutdownGracefully")
public NioEventLoopGroup workerGroup() {
return new NioEventLoopGroup(nettyProperties.getWorkerCount());
}
#Bean
#SneakyThrows
public InetSocketAddress tcpSocketAddress() {
return new InetSocketAddress(nettyProperties.getTcpPort());
}
}
Initialization logic:
#Component
#RequiredArgsConstructor
public class SimpleChannelInitializer extends ChannelInitializer<SocketChannel> {
private final StringEncoder stringEncoder = new StringEncoder();
private final StringDecoder stringDecoder = new StringDecoder();
private final QrReaderProcessingHandler readerServerHandler;
private final NettyProperties nettyProperties;
#Override
protected void initChannel(SocketChannel socketChannel) {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new DelimiterBasedFrameDecoder(1024 * 1024, Delimiters.lineDelimiter()));
pipeline.addLast(new ReadTimeoutHandler(nettyProperties.getClientTimeout()));
pipeline.addLast(stringDecoder);
pipeline.addLast(stringEncoder);
pipeline.addLast(readerServerHandler);
}
}
Properties configuration:
#Getter
#Setter
#ConfigurationProperties(prefix = "netty")
public class NettyProperties {
#NotNull
#Size(min = 1000, max = 65535)
private int tcpPort;
#Min(1)
#NotNull
private int bossCount;
#Min(2)
#NotNull
private int workerCount;
#NotNull
private boolean keepAlive;
#NotNull
private int backlog;
#NotNull
private int clientTimeout;
}
and a snippet from application.yml:
netty:
tcp-port: 9090
boss-count: 1
worker-count: 14
keep-alive: true
backlog: 128
client-timeout: 20
And handler is quite trivial.
Checked locally by running at the console:
telnet localhost 9090
It works fine there. I hope it will be fine for access from clients.

Since the protocol is NOT based on top of HTTP (unlike WebSocket which piggyback on HTTP in the first place), your only option is to use TCP server yourself and wire it up within spring context to gain full advantage of spring along with.
Netty is best known for low-level TCP/IP communication and it's easy to wrap up Netty server within spring app.
In fact, spring boot provides Netty HTTP server out of the box but this is not what you need.
TCP communication server with Netty And SpringBoot project is a simple and effective example of what you need.
Take a look at TCPServer from this project which uses Netty's ServerBootstrap for starting custom TCP server.
Once you have the server, you can wire up either Netty codecs OR Jackson OR any other message converter as you seem fit for your application domain data marshalling/unmarshalling.
[Update - July 17, 2020]
Against the updated understanding of the question (both HTTP and TCP requests are terminating on the same endpoint), following is the updated solution proposal
----> HTTP Server (be_http)
|
----> HAProxy -
|
----> TCP Server (be_tcp)
Following changes/additions are required for this solution to work:
Add Netty based listener in your existing spring boot app OR create a separate spring boot app for TCP server. Say this endpoint is listening for TCP traffic on port 9090
Add HAProxy as the terminating endpoint for ingress traffic
Configure HAProxy so that it sends all HTTP traffic to your existing spring boot HTTP endpoint (mentioned as be_http) on port 8080
Configure HAProxy so that all non HTTP traffic is sent to the new TCP spring boot endpoint (mentioned as be_tcp) on port 9090.
Following HAProxy configuration will suffice. These are excerpt which are relevant for this problem, please add other HAProxy directives as applicable for normal HAProxy setup:
listen 443
mode tcp
bind :443 name tcpsvr
/* add other regular directives */
tcp-request inspect-delay 1s
tcp-request content accept if HTTP
tcp-request content accept if !HTTP
use-server be_http if HTTP
use-server be_tcp if !HTTP
/* backend server definition */
server be_http 127.0.0.1:8080
server be_tcp 127.0.0.1:9090 send-proxy
Following HAProxy documentation links are particularly useful
Fetching samples from buffer contents - Layer 6
Pre-defined ACLs
tcp-request inspect-delay
tcp-request content
Personally I'll play around and validate tcp-request inspect-delay and adjust it as per actual needs since this has potential for adding delay in request in worst case scenario where connection has been made but no contents are available yet to evaluate if request is HTTP or not.
Addressing the need for we have to identify the client at least the IP address from which it was sent, you have an option to use Proxy Protocol while sending it back to backend. I have updated the sample config above to include proxy protocol in be_tcp (added send_proxy). I have also removed send_proxy from be_http since it's not needed for spring boot, instead you'll likely rely upon regular X-Forwarded-For header for be_http backend instead.
Within be_tcp backend, you can use Netty's HAProxyMessage to get the actual source IP address using sourceAddress() API. All in all, this is a workable solution. I myself have used HAProxy with proxy protocol (at both ends, frontend and backend) and it's much more stable for the job.

push messages to active mq via proxy using http transport

Hi I am trying to push messages to active mq via http transport.
The active mq is behind the proxy server which need authentication before pushing the message to external active mq.
is there a way I can set the proxy details to active mq connection.
I read some article where mentioned we can use HttpClientTrasport.
But i am not sure how to set the HttpClientTrasport to ActiveMQConnection object.
Thanks in advance.

if i understand you want to set the proxy config for org.apache.activemq.ActiveMQConnection ?
this can be set on org.apache.activemq.ActiveMQConnectionFactory.ActiveMQConnectionFactory(String brokerURL) level by passing the url with proxy config like this :
brokerURL = "http://localhost:8080?proxyHost=proxy&proxyPort=8080&proxyUser=user&proxyPassword=pwd"
if you have specials characters on user or pwd you need to change like this :
brokerURL = "http://localhost:8080?"+ URLEncoder.encode("proxyHost=proxy&proxyPort=8080&proxyUser=user&proxyPassword=pwd", "UTF-8");
hope this help

add transportConnection configuration in activemq;
nano ~/apache-activemq-5.11.1/conf/activemq.xml
<transportConnector name="http" uri="http://0.0.0.0:8888?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
use broker_url
private static final String DEFAULT_BROKER_URL = "http://localhost:8888";
add dependency in project
<dependency><groupId>org.apache.activemq</groupId><artifactId>activemq-http</artifactId><version>5.11.1</version></dependency>

Restlet framework: how to bind to localhost only?

I need to build a (standalone Java) restlet-based service that only listens on localhost, i.e. no requests from network are allowed.
I was trying to do the obvious:
Server srv = new Server(Protocol.HTTPS, "localhost", httpsPort);
component.getServers().add(srv);
But the service still listens on 0.0.0.0. :-(
I went into the code and found that HttpsServerHelper ignores the hostname when creating the service:
this.server = HttpsServer.create(new InetSocketAddress(getHelped().getPort()), 0);
Similar code exists in plain HTTP's HttpServerHelper, where it is even more clear.
My question then is this:
How can I configure Restlet component/service to only listen on localhost?

I don't know which server you use under the hood within your standalone Restlet application. You should use a server connector other than the default one and I recommend you to use the Jetty one.
To do that, simply put the jar of the extension org.restlet.ext.jetty in your classpath.
In this case, using the following code should correspond to your needs:
component.getServers().add(Protocol.HTTP, "localhost", 8182);
Here is the corresponding trace at application startup:
2015-09-03 09:47:22.180:INFO::jetty-7.1.6.v20100715
2015-09-03 09:47:22.211:INFO::Started SelectChannelConnector#localhost:8182
In addition, here is the link in the Restlet documentation regarding Restlet connectors: http://restlet.com/technical-resources/restlet-framework/guide/2.3/core/base/connectors.
Hope it helps you,
Thierry

The easier way to achieve that is to use virtual hosts.
Virtual hosts are the first routing barrier when handling a request, especially it helps routing on a domain.
Here is a sample code that illustrates this:
Component c = new Component();
c.getServers().add(Protocol.HTTP, 8182);
VirtualHost host = new VirtualHost();
host.setHostDomain("localhost");
c.getHosts().add(host);
host.attach(new Restlet() {
#Override
public void handle(Request request, Response response) {
response.setEntity("hello, world", MediaType.TEXT_PLAIN);
}
});
c.start();
Usually, applications are attached on the default host of a component. This default host does nothing, except routing requests based on the context path of the attached application:
c.getDefaultHost().attach("/contextPath1", new Test1Application());
c.getDefaultHost().attach("/contextPath2", new Test2Application());
When you would like to filter calls based on other data than the request's path, virtual host may be the solution.
Here is a diagram that may help you:
http://restlet.com/technical-resources/restlet-framework/tutorials/2.3#part05

org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed

I am using apache's activemq for queueing. We have started to see the following exception very often when writing things to the queue:
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed:
at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:282)
at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:271)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1366)
I can't figure out what could be causing this-- or even, frankly, where to start debugging what is causing this.
Here is the queue set up code:
camelContext = new DefaultCamelContext();
camelContext.setErrorHandlerBuilder(new LoggingErrorHandlerBuilder());
camelContext.getShutdownStrategy().setTimeout(SHUTDOWN_TIMEOUT_SECONDS);
routePolicy = new RoutePolicy();
routePolicy.setCamelContext(camelContext);
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(queueUri);
// use a pooled connection factory between the module and the queue
pooledConnectionFactory = new PooledConnectionFactory(connectionFactory);
// how many connections should there be in the session pool?
pooledConnectionFactory.setMaxConnections(this.maxConnections);
pooledConnectionFactory.setMaximumActiveSessionPerConnection(this.maxActiveSessionPerConnection);
pooledConnectionFactory.setCreateConnectionOnStartup(true);
pooledConnectionFactory.setBlockIfSessionPoolIsFull(false);
JmsConfiguration jmsConfiguration = new JmsConfiguration(pooledConnectionFactory);
jmsConfiguration.setDeliveryPersistent(false); // do not store a copy of the messages on the queue
ActiveMQComponent activeMQComponent = ActiveMQComponent.activeMQComponent(queueUri);
activeMQComponent.setConfiguration(jmsConfiguration);
camelContext.addComponent("activemq", activeMQComponent);
Component activemq = camelContext.getComponent("activemq");
// register endpoints for queues and topics
Endpoint queueEndpoint = activemq.createEndpoint("activemq:queue:polaris.*");
Endpoint topicEndpoint = activemq.createEndpoint("activemq:topic:polaris.*");
producerTemplate = camelContext.createProducerTemplate();
camelContext.start();
queueEndpoint.start();
topicEndpoint.start();
Like I said, the error doesn't suggest any directions for debugging, and it doesn't happen in 100% of cases where I can be sure my configuration is not set up correctly.

Recently I ran into the same problem. I found this https://issues.apache.org/jira/browse/AMQ-6600
Apache ActiveMQ client throws InactivityIOException when one of the jars is missing in classpath. In my case it was hawtbuf-1.11.jar. When I added this jar to classpath it started to work without errors.

management port: 61616 (default)
service port : 8161(default)
change your broker url port to 61616 and run
refer this

Check, if there is a non-Jms client pinging your JMS broker. This may be an external monitoring tool, a load balancing tool such as keepalived, or another one.

javax.jms.InvalidDestinationException: Cannot use a Temporary destination from another Connection

I have a producer which connects to ActiveMQ broker to send me messages to the client.
Since it expects some response from the client, it first creates a temp queue and associates it to the JMS replyto header.
It then sends the message over to the broker and waits for the response on temp queue from the client.
Receives the response from the client over the temp queue, performs required actions and then exits.
This works fine most of the times, but sporadically the application throws error messsages saying " Cannot use queue created from another connection ".
I am unable to identify what could cause this to happen as the temp queue is being created from the current session itself.
Did anyone else come across this situation and knows how to fix it?
Code snippet:
Connection conn = myJmsTemp. getConnectionFactory().createConnection();
ses = conn.createSession(transacted,ackMode);
responseQueue = ses.createTemporaryQueue();
...
MyMessageCreator msgCrtr = new MyMessageCreator(objects,responseQueue);
myJmsTemp.send(dest, msgCrtr);
myJmsTemp.setReceiveTimeout(timeout);
ObjectMessage response = (ObjectMessage)myJmsTemplate.receive(responseQueue);
Here MyMessageCreator implements MessageCreator interface.
All am trying to do is send a message to the broker and wait for a response from the client over the temp queue. Also am using a pooled connection factory to get the connection.

You get an error like this if you have a client that is trying to subscribe as a consumer on a temporary destination that was created by a different connection instance. The JMS spec defines that only the connection that created the temp destination can consume from it, so that's why the limitation exists. As for the reason you are seeing it its hard to say without seeing your code that encounters the error.
Given that your update says you are using the Pooled connection factory I'd guess that this is the root of you issue. If the consume call happens to use a different connection from the Pool than the one that created the temp destination then you would see the error that you mentioned.