I know spring integration has TcpInboundGateway and ByteArrayStxEtxSerializer to handle data coming through TCP port.
ByteArrayStxEtxSerializer works great if the TCP server needs to read all the data sent from the client and then processes it. (request and response model) I am using single-use=false so that multiple requests can be processed in the same connection.
For example if the client sends 0x02AAPL0x03 then Server can send the AAPL price.
My TCP Server is working if the client sends 0x02AAPL0x030x02GOOG0x03. It sends the price of AAPL and GOOG price.
Sometimes clients can send EOT (0x04). If the client sends EOT, I would like to close the socket connection.
For example: Client request can be 0x02AAPL0x030x02GOOG0x03 0x020x040x03. Note EOT came in the last packet.
I know ByteArrayStxEtxSerializer deserializer can be customized to read the bytes sent by the client.
is deserializer good place to close socket connection? if not, how should spring integration framework be notified to close socket connection?
Please help.
Here is my spring configuration:
<int-ip:tcp-connection-factory id="crLfServer"
type="server"
port="${availableServerSocket}"
single-use="false"
so-timeout="10000"
using-nio="false"
serializer="connectionSerializeDeserialize"
deserializer="connectionSerializeDeserialize"
so-linger="2000"/>
<bean id="connectionSerializeDeserialize" class="org.springframework.integration.ip.tcp.serializer.ByteArrayStxEtxSerializer"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf"
connection-factory="crLfServer"
request-channel="serverBytes2StringChannel"
error-channel="errorChannel"
reply-timeout="10000"/> <!-- reply-timeout works on inbound-gateway -->
<int:channel id="toSA" />
<int:service-activator input-channel="toSA"
ref="myService"
method="prepare"/>
<int:object-to-string-transformer id="serverBytes2String"
input-channel="serverBytes2StringChannel"
output-channel="toSA"/>
<int:transformer id="errorHandler"
input-channel="errorChannel"
expression="payload.failedMessage.payload + ':' + payload.cause.message"/>
UPDATE:
Adding throw new SoftEndOfStreamException("Stream closed") to close the stream in serializer works and I can see the CLOSED log entry in EventListener. When the server closes the connection, I expect to receive java.io.InputStream.read() as -1 in the client. But the client is receiving the
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:107)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:93)
at java.io.InputStreamReader.read(InputStreamReader.java:151)
is there anything else to close the connection on the server side and propagate it to client?
I appreciate your help.
Thank you
The deserializer doesn't have access to the socket, just the input stream; closing it would probably work, but you will likely get a lot of noise in the log.
The best solution is to throw a SoftEndOfStreamException; that signals that the socket should be closed and everything cleaned up.
EDIT
Add a listener to detect/log the close...
#SpringBootApplication
public class So40471456Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So40471456Application.class, args);
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("foo\r\n".getBytes());
socket.close();
Thread.sleep(10000);
context.close();
}
#Bean
public EventListener eventListener() {
return new EventListener();
}
#Bean
public TcpNetServerConnectionFactory server() {
return new TcpNetServerConnectionFactory(1234);
}
#Bean
public TcpReceivingChannelAdapter inbound() {
TcpReceivingChannelAdapter adapter = new TcpReceivingChannelAdapter();
adapter.setConnectionFactory(server());
adapter.setOutputChannelName("foo");
return adapter;
}
#ServiceActivator(inputChannel = "foo")
public void syso(byte[] in) {
System.out.println(new String(in));
}
public static class EventListener implements ApplicationListener<TcpConnectionCloseEvent> {
private final Log logger = LogFactory.getLog(getClass());
#Override
public void onApplicationEvent(TcpConnectionCloseEvent event) {
logger.info(event);
}
}
}
With XML, just add a <bean/> for your listener class.
Result:
foo
2016-11-07 16:52:04.133 INFO 29536 --- [pool-1-thread-2] c.e.So40471456Application$EventListener : TcpConnectionCloseEvent
[source=org.springframework.integration.ip.tcp.connection.TcpNetConnection#118a7548],
[factory=server, connectionId=localhost:50347:1234:b9fcfaa9-e92c-487f-be59-1ed7ebd9312e]
**CLOSED**
EDIT2
It worked as expected for me...
#SpringBootApplication
public class So40471456Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So40471456Application.class, args);
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("foo\r\n".getBytes());
try {
System.out.println("\n\n\n" + socket.getInputStream().read() + "\n\n\n");
context.getBean(EventListener.class).latch.await(10, TimeUnit.SECONDS);
}
finally {
socket.close();
context.close();
}
}
#Bean
public EventListener eventListener() {
return new EventListener();
}
#Bean
public TcpNetServerConnectionFactory server() {
TcpNetServerConnectionFactory server = new TcpNetServerConnectionFactory(1234);
server.setDeserializer(is -> {
throw new SoftEndOfStreamException();
});
return server;
}
#Bean
public TcpReceivingChannelAdapter inbound() {
TcpReceivingChannelAdapter adapter = new TcpReceivingChannelAdapter();
adapter.setConnectionFactory(server());
adapter.setOutputChannelName("foo");
return adapter;
}
public static class EventListener implements ApplicationListener<TcpConnectionCloseEvent> {
private final Log logger = LogFactory.getLog(getClass());
private final CountDownLatch latch = new CountDownLatch(1);
#Override
public void onApplicationEvent(TcpConnectionCloseEvent event) {
logger.info(event);
latch.countDown();
}
}
}
Result:
2016-11-08 08:27:25.964 INFO 86147 --- [ main] com.example2.So40471456Application : Started So40471456Application in 1.195 seconds (JVM running for 1.764)
-1
2016-11-08 08:27:25.972 INFO 86147 --- [pool-1-thread-2] c.e.So40471456Application$EventListener : TcpConnectionCloseEvent [source=org.springframework.integration.ip.tcp.connection.TcpNetConnection#fee3774], [factory=server, connectionId=localhost:54984:1234:f79a6826-0336-4823-8844-67054903a094] **CLOSED**
Related
I have following binding to handle UDP packets
private void doStartServer() {
final UDPPacketHandler udpPacketHandler = new UDPPacketHandler(messageDecodeHandler);
workerGroup = new NioEventLoopGroup(threadPoolSize);
try {
final Bootstrap bootstrap = new Bootstrap();
bootstrap
.group(workerGroup)
.handler(new LoggingHandler(nettyLevel))
.channel(NioDatagramChannel.class)
.option(ChannelOption.SO_BROADCAST, true)
.handler(udpPacketHandler);
bootstrap
.bind(serverIp, serverPort)
.sync()
.channel()
.closeFuture()
.await();
} finally {
stop();
}
}
and handler
#ChannelHandler.Sharable << note this
#Slf4j
#AllArgsConstructor
public class UDPPacketHandler extends SimpleChannelInboundHandler<DatagramPacket> {
private final MessageP54Handler messageP54Handler;
#Override
public void channelReadComplete(final ChannelHandlerContext ctx) {
ctx.flush();
}
#Override
public void exceptionCaught(final ChannelHandlerContext ctx, final Throwable cause) {
log.error("Exception in UDP handler", cause);
ctx.close();
}
}
At some point I get this exception java.net.SocketException: Network dropped connection on reset: no further information which is handled in exceptionCaught. This triggers ChannelHandlerContext to close. And at this point whole my server stops (executing on finally block from first snippet)
How to correctly handle exception so that I can handle new connections even after such exception occurs?
you shouldn't close the ChannelHandlerContext on an IOException when using a DatagramChannel. As DatagramChannel is "connection-less" the exception is specific to one "receive" or one "send" operation. So just log it (or whatever you want to do) and move on.
Trying use agi to listen asterisk.
But after startup method is running, my application is freezing. There is no error appears...
My spring bean:
#Bean(name = "agi")
public DefaultAgiServer getAsteriskAgi() throws Exception {
DefaultAgiServer agiServer = new DefaultAgiServer();
agiServer.startup();
return agiServer;
}
My mapping
public class AsteriskAgi extends BaseAgiScript{
#Override
public void service(AgiRequest agiRequest, AgiChannel agiChannel) throws AgiException {
// Answer the channel...
answer();
// ...say hello...
streamFile("welcome");
streamFile("tt-monkeys");
// ...and hangup.
hangup();
}
}
my properties file
fastagi-mapping.properties
located in resources folder
Why is this happening?
UPD
Last two console output:
2018-05-21 15:19:53 DEBUG DefaultAgiServer:81 - Using channelFactory org.asteriskjava.fastagi.internal.DefaultAgiChannelFactory
2018-05-21 15:19:53 INFO DefaultAgiServer:315 - Listening on *:4573.
Tomcat hang on startup because the AGI server is blocking it and waits for incoming AGI data from socket connection link.
To solve this you should wrap your AgiServer in a separate thread so that it runs in the background or use AgiServerThread.
In result my asterisk agi configuration looks like this:
#Bean
public AgiServerThread agiServerThread(){
AgiServerThread agiServerThread = new AgiServerThread(getDefaultAgiServer());
agiServerThread.startup();
return agiServerThread;
}
#Bean
public DefaultAgiServer getDefaultAgiServer(){
return new DefaultAgiServer(getAsteriskAgiScript());
}
#Bean
public AgiScript getAsteriskAgiScript(){
return new AsteriskAgi();
}
Gary Russell kindly answered a previous question of mine about Spring Integration udp flows. Moving from there, I have stumbled upon an issue with ports.
The Spring Integration documentation says that you can put 0 to the inbound channel adapter port, and the OS will select an available port for the adapter, which can be retrieved at runtime invoking getPort() on the adapter object. The problem is that at runtime I just get a 0 if I try to retrieve the port programmatically.
Here's "my" code (i.e. a slightly modified version of Russel's answer to my previous question for Spring Integration 4.3.12, which I am currently using).
#SpringBootApplication
public class TestApp {
private final Map<Integer, IntegrationFlowRegistration> registrations = new HashMap<>();
#Autowired
private IntegrationFlowContext flowContext;
public static void main(String[] args) {
SpringApplication.run(TestApp.class, args);
}
#Bean
public PublishSubscribeChannel channel() {
return new PublishSubscribeChannel();
}
#Bean
public TestData test() {
return new TestData();
}
#Bean
public ApplicationRunner runner() {
return args -> {
UnicastReceivingChannelAdapter source;
source = makeANewUdpInbound(0);
makeANewUdpOutbound(source.getPort());
Thread.sleep(5_000);
channel().send(MessageBuilder.withPayload("foo\n").build());
this.registrations.values().forEach(r -> {
r.stop();
r.destroy();
});
this.registrations.clear();
makeANewUdpInbound(1235);
makeANewUdpOutbound(1235);
Thread.sleep(5_000);
channel().send(MessageBuilder.withPayload("bar\n").build());
this.registrations.values().forEach(r -> {
r.stop();
r.destroy();
});
this.registrations.clear();
};
}
public UnicastSendingMessageHandler makeANewUdpOutbound(int port) {
System.out.println("Creating an adapter to send to port " + port);
UnicastSendingMessageHandler adapter = new UnicastSendingMessageHandler("localhost", port);
IntegrationFlow flow = IntegrationFlows.from(channel())
.handle(adapter)
.get();
IntegrationFlowRegistration registration = flowContext.registration(flow).register();
registrations.put(port, registration);
return adapter;
}
public UnicastReceivingChannelAdapter makeANewUdpInbound(int port) {
System.out.println("Creating an adapter to receive from port " + port);
UnicastReceivingChannelAdapter source = new UnicastReceivingChannelAdapter(port);
IntegrationFlow flow = IntegrationFlows.from(source)
.<byte[], String>transform(String::new)
.handle(System.out::println)
.get();
IntegrationFlowRegistration registration = flowContext.registration(flow).register();
registrations.put(port, registration);
return source;
}
}
The output I read is
Creating an adapter to receive from port 0
Creating an adapter to send to port 0
Creating an adapter to receive from port 1235
Creating an adapter to send to port 1235
GenericMessage [payload=bar, headers={ip_packetAddress=127.0.0.1/127.0.0.1:54374, ip_address=127.0.0.1, id=c95d6255-e63a-433d-3723-c389fe66b060, ip_port=54374, ip_hostname=127.0.0.1, timestamp=1517220716983}]
I suspect the library did create adapters on OS-chosen free ports, but I am unable to retrieve the assigned port.
The port is assigned asynchronously; you need to wait until the port is actually assigned. Something like...
int n = 0;
while (n++ < 100 && ! source.isListening()) {
Thread.sleep(100;
}
if (!source.isListening()) {
// failed to start in 10 seconds.
}
We should probably enhance the adapter to emit an event when the port is ready. Feel free to open an 'Improvement' JIRA Issue.
I am building an application using Spring Integration which is used to send files from one FTP server (source) to another FTP server (target). I first send files from source to the local directory using the inbound adapter and then send files from the local directory to the target using the outbound adapter.
My code seems to be working fine and I am able to achieve my goal but my problem is when the connection is reset to the target FTP server during the transfer of files, then the transfer of files don't continue after the connection starts working.
I used the Java configurations using inbound and outbound adapters. Can anyone please tell me if it is possible to resume my transfer of files somehow after the connection reset?
P.S: I am a beginner at Spring, so correct me if I have done something wrong here. Thanks
AppConfig.java:
#Configuration
#Component
public class FileTransferServiceConfig {
#Autowired
private ConfigurationService configurationService;
public static final String FILE_POLLING_DURATION = "5000";
#Bean
public SessionFactory<FTPFile> sourceFtpSessionFactory() {
DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
sf.setHost(configurationService.getSourceHostName());
sf.setPort(Integer.parseInt(configurationService.getSourcePort()));
sf.setUsername(configurationService.getSourceUsername());
sf.setPassword(configurationService.getSourcePassword());
return new CachingSessionFactory<FTPFile>(sf);
}
#Bean
public SessionFactory<FTPFile> targetFtpSessionFactory() {
DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
sf.setHost(configurationService.getTargetHostName());
sf.setPort(Integer.parseInt(configurationService.getTargetPort()));
sf.setUsername(configurationService.getTargetUsername());
sf.setPassword(configurationService.getTargetPassword());
return new CachingSessionFactory<FTPFile>(sf);
}
#MessagingGateway
public interface MyGateway {
#Gateway(requestChannel = "toFtpChannel")
void sendToFtp(Message message);
}
#Bean
public FtpInboundFileSynchronizer ftpInboundFileSynchronizer() {
FtpInboundFileSynchronizer fileSynchronizer = new FtpInboundFileSynchronizer(sourceFtpSessionFactory());
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setRemoteDirectory(configurationService.getSourceDirectory());
fileSynchronizer.setFilter(new FtpSimplePatternFileListFilter(
configurationService.getFileMask()));
return fileSynchronizer;
}
#Bean
#InboundChannelAdapter(channel = "ftpChannel",
poller = #Poller(fixedDelay = FILE_POLLING_DURATION ))
public MessageSource<File> ftpMessageSource() {
FtpInboundFileSynchronizingMessageSource source =
new FtpInboundFileSynchronizingMessageSource(ftpInboundFileSynchronizer());
source.setLocalDirectory(new File(configurationService.getLocalDirectory()));
source.setAutoCreateLocalDirectory(true);
source.setLocalFilter(new AcceptOnceFileListFilter<File>());
return source;
}
#Bean
#ServiceActivator(inputChannel = "ftpChannel")
public MessageHandler targetHandler() {
FtpMessageHandler handler = new FtpMessageHandler(targetFtpSessionFactory());
handler.setRemoteDirectoryExpression(new LiteralExpression(
configurationService.getTargetDirectory()));
return handler;
}
}
Application.java:
#SpringBootApplication
public class Application {
public static ConfigurableApplicationContext context;
public static void main(String[] args) {
context = new SpringApplicationBuilder(Application.class)
.web(false)
.run(args);
}
#Bean
#ServiceActivator(inputChannel = "ftpChannel")
public MessageHandler sourceHandler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
Object payload = message.getPayload();
System.out.println("Payload: " + payload);
if (payload instanceof File) {
File file = (File) payload;
System.out.println("Trying to send " + file.getName() + " to target");
}
MyGateway gateway = context.getBean(MyGateway.class);
gateway.sendToFtp(message);
}
};
}
}
First of all it isn't clear what is that sourceHandler for, but you really should be sure that it is subscribed (or targetHandler) to proper channel.
I somehow believe that in your target code the targetHandler is really subscribed to the toFtpChannel.
Anyway that isn't related.
I think the problem here is exactly with the AcceptOnceFileListFilter and error. So, filter work first during directory scan and load all the local files to the in-memory queue for performance reason. Then all of them are sent to the channel for processing. When we reach the targetHandler and got an exception, we just silently got away to the global errorChannel loosing the fact that file hasn't been transferred. And this happens with all the remaining files in memory. I think the transfer is resumed anyway but it is going work already only for new files in the remote directory.
I suggest you to add ExpressionEvaluatingRequestHandlerAdvice to the targetHandler definition (#ServiceActivator(adviceChain)) and in case of error call the AcceptOnceFileListFilter.remove(File):
/**
* Remove the specified file from the filter so it will pass on the next attempt.
* #param f the element to remove.
* #return true if the file was removed as a result of this call.
*/
boolean remove(F f);
This way you remove the failed files from the filter and it will be picked up on the next poll task. You have to make AcceptOnceFileListFilter to be able to get an access to it from the onFailureExpression. The file is the payload of request message.
EDIT
The sample for the ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public Advice expressionAdvice() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnFailureExpressionString("#acceptOnceFileListFilter.remove(payload)");
advice.setTrapException(true);
return advice;
}
...
#ServiceActivator(inputChannel = "ftpChannel", adviceChain = "expressionAdvice")
Everything rest you can get from their JavaDocs.
I'm trying to send messages to my topic, but the problem is when I send the message nothing happens... I'm using apache tomcat 7.0.53
UPDATE: 04/15: Link to test:
http://ec2-54-187-72-145.us-west-2.compute.amazonaws.com:8080/kupo
Login: admin
Password: admin
LINK TO ACCESS TOMCAT LOG:
http://ec2-54-187-72-145.us-west-2.compute.amazonaws.com:28778/
P.S: You need to checked the combobox on the sidebar to start watch the messages
Github Link: https://github.com/tiarebalbi/kupo
LOG:
DEBUG - gWebSocketHandlerDecorator - Connection established, SockJS session id=_mqg8qer, uri=/kupo/application/807/_mqg8qer/websocket
DEBUG - StompDecoder - Decoded [Payload byte[0]][Headers= {stompCommand=CONNECT, nativeHeaders={heart-beat=[10000,10000], accept-version=[1.1,1.0]}, simpMessageType=CONNECT, id=e79a615e-5522-a0f9-aecf-6ea5a54b3d9b, timestamp=1397013491497}]
DEBUG - StompEncoder - Encoded STOMP command=CONNECTED headers={user-name=[balbi], heart-beat=[0,0], version=[1.1]}
DEBUG - StompDecoder - Decoded [Payload byte[0]][Headers={stompCommand=SUBSCRIBE, nativeHeaders={id=[sub-0], destination=[/topic/greetings]}, simpMessageType=SUBSCRIBE, simpSubscriptionId=sub-0, simpDestination=/topic/greetings, id=42c2019d-96a0-95f0-29aa-2bcc62d6d721, timestamp=1397013491501}]
CODE:
#Service
public class ExampleServiceImpl implements ApplicationListener<BrokerAvailabilityEvent> {
private AtomicBoolean brokerAvailable = new AtomicBoolean();
#Autowired
private MessageSendingOperations<String> messagingTemplate;
#Override
public void onApplicationEvent(BrokerAvailabilityEvent event) {
this.brokerAvailable.set(event.isBrokerAvailable());
}
#Scheduled(fixedDelay=3000)
public void testing() {
if (this.brokerAvailable.get()) {
this.messagingTemplate.convertAndSend("/topic/greetings", "Testing....");
}
}
Javascript Connect:
var socket = new SockJS('/kupo/application'); // <!-- My endpoint
var stompClient = Stomp.over(socket);
stompClient.connect({}, function(frame) {
var username = frame.headers['user-name'];
console.log("User connected: " + username);
stompClient.subscribe("/topic/greetings", function(message) { // <-- Topic where I want to received the message
console.log("TOPIC:",message);
});
} , function(error) {
console.log("STOMP protocol error " + error);
});
Browser Console:
Opening Web Socket... stomp.min.js:8
Web Socket Opened... stomp.min.js:8
>>> CONNECT
accept-version:1.1,1.0
heart-beat:10000,10000
<<< CONNECTED
user-name:balbi
heart-beat:0,0
version:1.1
connected to server undefined stomp.min.js:8
User connected: balbi
>>> SUBSCRIBE
id:sub-0
destination:/topic/greetings
Websocket Context Configuration:
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketApplicationContext extends AbstractWebSocketMessageBrokerConfigurer {
#Autowired
private Environment env;
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
if (env.acceptsProfiles("test.tomcat")) {
registry.addEndpoint("/application")
.setHandshakeHandler(
new DefaultHandshakeHandler(new TomcatRequestUpgradeStrategy()))
.withSockJS();
} else {
registry.addEndpoint("/application").withSockJS();
}
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.enableSimpleBroker("/queue/", "/topic/");
registry.setApplicationDestinationPrefixes("/app");
}
}
When connecting to your application, I managed to send a message to a topic from the JavaScript console and get a message back in the web page.
var socket = new SockJS('/kupo/application');
var stompClient = Stomp.over(socket);
stompClient.send('/topic/greetings',{},"hello");
And I received:
<<< MESSAGE
subscription:sub-0
content-length:5
message-id:ts3oov6b-1
destination:/topic/greetings
content-length:5
hello
Is your Scheduling task being called as expected?
Why are you, in your main configuration, importing some Configurations and still scanning the configuration package? Shouldn't it be one or the other?