I want to achieve the following using spring-integration: having a singleton open socket that constantly receives and writes data, asyncrhon!
This means I have to open a socket that constantly reads from the single socket, dispatches each message for async processing, and return the responses over the socket also async.
How can I achieve that asynchron pattern?
Especially: how can I use Serializer/Deserializer? As far as I understood, a serializer is only invoked on a new socket connection, so in my case only once at start of the first message?
#Configuration
public class SocketConfig {
#Bean
public TcpConnectionFactoryFactoryBean tcpFactory(MyConverter converter) {
TcpConnectionFactoryFactoryBean fact = new TcpConnectionFactoryFactoryBean();
fact.setType("server");
fact.setPort(PORT);
fact.setUsingNio(true); //should I use true or false?
fact.setSingleUse(false); //keep socket constantly open
fact.setSerializer(converter);
fact.setDeserializer(converter);
return fact;
}
#Bean
public TcpInboundGateway serverGateway(
#Qualifier("tcpFactory") TcpConnectionFactoryFactoryBean factory,
#Qualifier("serverChannel") MessageChannel serverChannel) throws Exception {
TcpInboundGateway g = new TcpInboundGateway();
g.setConnectionFactory(factory.getObject());
g.setRequestChannel(serverChannel);
return g;
}
}
#MessageEndpoint
public class SocketEndpoint {
#ServiceActivator(inputChannel = "serverChannel")
public Object run(Object obj) {
}
}
#Service
public class MyConverter implements Serializer<Object>, Deserializer<Object> {
//read from socket
#Override
public Object deserialize(InputStream inputStream) {
}
//send back to socket
#Override
public void serialize(Object message, OutputStream outputStream) {
}
}
A gateway is used for individual request/response pairs.
If you need to send multiple responses for a single request, you must use collaborating channel adapters as described in the documentation.
Collaborating adapters can also be used (server-side or client-side) for totally asynchronous communication (rather than with request/reply semantics).
On the server side, care must be taken to populate the ip_connectionId header because it is used to correlate the message to a connection. Messages that originate at the inbound adapter will automatically have the header set. If you wish to construct other messages to send, you will need to set the header. The header value can be captured from an incoming message.
Related
I have been trying to send messages to external systems(using rest template POST, PUT etc) from the service activators as below.
Below is my pubsub consumer class
public class MyConsumer{
#Autowired
ExternalService externalService;
#Bean
public PubSubInboundChannelAdapter messageChannelAdapter(final #Qualifier("myInputChannel") MessageChannel inputChannel,
PubSubTemplate pubSubTemplate)
{
PubSubInboundChannelAdapter adapter = new PubSubInboundChannelAdapter(pubSubTemplate, pubSubSubscriptionName);
adapter.setOutputChannel(inputChannel);
adapter.setAckMode(AckMode.AUTO_ACK);
adapter.setErrorChannelName("pubsubErrors");
return adapter;
}
#ServiceActivator(inputChannel = "pubsubErrors")
public void pubsubErrorHandler(Message<MessagingException> exceptionMessage) {
BasicAcknowledgeablePubsubMessage originalMessage = (BasicAcknowledgeablePubsubMessage) exceptionMessage
.getPayload().getFailedMessage().getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE);
originalMessage.nack();
}
#Bean
public MessageChannel myInputChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "myInputChannel")
public MessageHandler messageReceiver_AddCustomer() {
return message -> {
externalService.postDataTOExternalSystems(new String((byte[]) message.getPayload());
};
}
#Bean
#ServiceActivator(inputChannel = "myInputChannel")
public MessageHandler messageReceiver_DeleteCustomer() {
return message -> {
externalService.deleteCustomer(new String((byte[]) message.getPayload());
BasicAcknowledgeablePubsubMessage originalMessage =
message.getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE, BasicAcknowledgeablePubsubMessage.class);
originalMessage.ack();
};
}
}
ExternalService below is the service which sends data to the external systems.
public class ExternalService{
void postDataTOExternalSystems(Object obj){
// RequestEntity object formed with HttpEntity object using obj(in json) and headers
restTemplate.exchange("https://externalsystems/",HttpMethod.POST,requestEntity,Object.class);
}
void deleteDatafromExternalSystems(Object obj){
// RequestEntity object formed with HttpEntity object using obj(in json) and headers
restTemplate.exchange("https://externalsystems/",HttpMethod.Detele,requestEntity,Object.class);
}
}
Since both the methods messageReceiver_AddCustomer and messageReceiver_deleteCustomer are using same channel whats happening is when I try to just addcustomer, the deleteCustomer is also called by default.
I was thinking of creating a seperate channel for deleteCustomer, but creating in this way leads to creating channels for every usecase.
Hence would like to know three things here.
Is there is any other approach of sending through Spring integration through which I can send data to external systems using a single Channel or a different utilization of Channels.
If any error in the external service calls leads to unending of failure logs in the console
message_id: "6830962001745961"
publish_time {
seconds: 1675783352
nanos: 547000000
}
}, timestamp=1675783353720}]': error occurred in message handler
It's not clear what is your expectation for such a logic. You have two contradicting subscribers for the same input channel. It sounds more like you need a router first to determine where to proceed next with an input message from Pub/Sub: https://docs.spring.io/spring-integration/reference/html/message-routing.html#messaging-routing-chapter.
but creating in this way leads to creating channels for every usecase.
Sure! You can go without Spring Integration and just do everything with the plain if..else. So, what's a difference? Therefore I don't see a reasonable argument in your statement. Yo have different HTTP methods, and that is OK to map them via Spring Integration router to respective sub-flows. The MessageChannel is really first-class citizen in Spring Integration. It makes a solution as loosely-coupled as possible.
Several subscribers on your current DirectChannel will lead to a round-robin logic by default, so you'll be surprised that one message from Pub/Sub creates a customer, another deletes and so on in rounds. The PublishSubscribeChannel will make it bad as well: both of your subscribers are going to be called, so created first, then deleted immediately.
I was wondering if there was a way to create a bi-direction stream (or allow multiple HTTP body sends over a single long-polled connection) in OKHTTP3 that does not require data to be constantly flowing between the client and the server.
For context, I am trying to implement a system where there can be intermittent data pushes can occur from either the client or the server over a persistent connection. The application is data-use sensitive, so I don't want the client sending requests to the server to see if there is data ready, I just want the server to push it.
A Websocket connection is the ideal solution to your problem. This creates a persistent connection between the client and the server and both parties can start sending data at any time.
in OKHTTP you can implement this by
adding the library to your build gradle file compile 'com.squareup.okhttp3:okhttp:3.6.0'
Create a class that implements the okhttp WebsocketListener interface
private final class MyWebSocketListener extends WebSocketListener {
private static final int CLOSE_STATUS = 1000;
#Override
public void onOpen(WebSocket webSocket, Response response) {
webSocket.send("Hello");
webSocket.close(CLOSE_STATUS, "Goodbye");
}
#Override
public void onMessage(WebSocket webSocket, String text) {
log(text);
}
#Override
public void onMessage(WebSocket webSocket, ByteString bytes) {
log(bytes.hex());
}
#Override
public void onClosing(WebSocket webSocket, int code, String reason) {
webSocket.close(CLOSE_STATUS, null);
log("Closing");
}
#Override
public void onFailure(WebSocket webSocket, Throwable t, Response response) {
log(t.getMessage());
}
}
Create a method to initiate the connection
private void connect() {
Request request = new Request.Builder().url("ws://my.websocket.url").build();
MyWebSocketListener listener = new MyWebSocketListener();
WebSocket ws = client.newWebSocket(request, listener);
\\ to shutdown the connection client.dispatcher().executorService().shutdown();
}
This should establish a connection with the server and should persist as long as the application is alive. I recommend reading more on websockets if you are the same person responsible for the backend.
Note: see update at the bottom of the question for what I eventually concluded.
I need to send multiple responses to a request over the web socket that sent the request message, the first one quickly, and the others after the data is verified (somewhere between 10 and 60 seconds later, from multiple parallel threads).
I am having trouble getting the later responses to stop broadcasting over all open web sockets. How do I get them to only send to the initial web socket? Or should I use something besides Spring STOMP (because, to be honest, all I want is the message routing to various functions, I don't need or want the ability to broadcast to other web sockets, so I suspect I could write the message distributor myself, even though it is reinventing the wheel).
I am not using Spring Authentication (this is being retrofitted into legacy code).
On the initial return message, I can use #SendToUser, and even though we don't have a user, Spring only sends the return value to the websocket that sent the message. (see this question).
With the slower responses, though, I think I need to use SimpMessagingTemplate.convertAndSendToUser(user, destination, message), but I can't, because I have to pass in the user, and I can't figure out what user the #SendToUser used. I tried to follow the steps in this question, but didn't get it to work when not authenticated (principal.getName() returns null in this case).
I've simplified this considerably for the prototype, so don't worry about synchronizing threads or anything. I just want the web sockets to work correctly.
Here is my controller:
#Controller
public class TestSocketController
{
private SimpMessagingTemplate template;
#Autowired
public TestSocketController(SimpMessagingTemplate template)
{
this.template = template;
}
// This doesn't work because I need to pass something for the first parameter.
// If I just use convertAndSend, it broacasts the response to all browsers
void setResults(String ret)
{
template.convertAndSendToUser("", "/user/topic/testwsresponse", ret);
}
// this only sends "Testing Return" to the browser tab hooked to this websocket
#MessageMapping(value="/testws")
#SendToUser("/topic/testwsresponse")
public String handleTestWS(String msg) throws InterruptedException
{
(new Thread(new Later(this))).start();
return "Testing Return";
}
public class Later implements Runnable
{
TestSocketController Controller;
public Later(TestSocketController controller)
{
Controller = controller;
}
public void run()
{
try
{
java.lang.Thread.sleep(2000);
Controller.setResults("Testing Later Return");
}
catch (Exception e)
{
}
}
}
}
For the record, here is the browser side:
var client = null;
function sendMessage()
{
client.send('/app/testws', {}, 'Test');
}
// hooked to a button
function test()
{
if (client != null)
{
sendMessage();
return;
}
var socket = new SockJS('/application-name/sendws/');
client = Stomp.over(socket);
client.connect({}, function(frame)
{
client.subscribe('/user/topic/testwsresponse', function(message)
{
alert(message);
});
sendMessage();
});
});
And here is the config:
#Configuration
#EnableWebSocketMessageBroker
public class TestSocketConfig extends AbstractWebSocketMessageBrokerConfigurer
{
#Override
public void configureMessageBroker(MessageBrokerRegistry config)
{
config.setApplicationDestinationPrefixes("/app");
config.enableSimpleBroker("/queue", "/topic");
config.setUserDestinationPrefix("/user");
}
#Override
public void registerStompEndpoints(StompEndpointRegistry registry)
{
registry.addEndpoint("/sendws").withSockJS();
}
}
UPDATE: Due to the security issues involved with the possibility of information being sent over other websockets than the originating socket, I ended up recommending to my group that we do not use the Spring 4.0 implementation of STOMP over Web Sockets. I understand why the Spring team did it the way they did it, and it is more power then we needed, but the security restrictions on our project were severe enough, and the actual requirements were simple enough, that we decided to go a different way. That doesn't invalidate the answers below, so make your own decision based on your projects needs. At least we have hopefully all learned the limitations of the technology, for good or bad.
Why don't you use a separate topic for each client?
Client generates a session id.
var sessionId = Math.random().toString(36).substring(7);
Client subscribes to /topic/testwsresponse/{sessionId}, then sends a message to '/app/testws/{sessionId}'.
In your controller you use #MessageMapping(value="/testws/{sessionId}") and remove #SendToUser. You can use #DestinationVariable to access sessionId in your method.
The controller sends further responses to /topic/testwsresponse/{sessionId}.
Essentially Spring does a similar thing internally when you use user destinations. Since you don't use Spring Authentication you cannot rely on this mechanism but you can easily implement your own as I described above.
var client = null;
var sessionId = Math.random().toString(36).substring(7);
function sendMessage()
{
client.send('/app/testws/' + sessionId, {}, 'Test');
}
// hooked to a button
function test()
{
if (client != null)
{
sendMessage();
return;
}
var socket = new SockJS('/application-name/sendws/');
client = Stomp.over(socket);
client.connect({}, function(frame)
{
client.subscribe('/topic/testwsresponse/' + sessionId, function(message)
{
alert(message);
});
// Need to wait until subscription is complete
setTimeout(sendMessage, 1000);
});
});
Controller:
#Controller
public class TestSocketController
{
private SimpMessagingTemplate template;
#Autowired
public TestSocketController(SimpMessagingTemplate template)
{
this.template = template;
}
void setResults(String ret, String sessionId)
{
template.convertAndSend("/topic/testwsresponse/" + sessionId, ret);
}
#MessageMapping(value="/testws/{sessionId}")
public void handleTestWS(#DestinationVariable String sessionId, #Payload String msg) throws InterruptedException
{
(new Thread(new Later(this, sessionId))).start();
setResults("Testing Return", sessionId);
}
public class Later implements Runnable
{
TestSocketController Controller;
String sessionId;
public Later(TestSocketController controller, String sessionId)
{
Controller = controller;
this.sessionId = sessionId;
}
public void run()
{
try
{
java.lang.Thread.sleep(2000);
Controller.setResults("Testing Later Return", sessionId);
}
catch (Exception e)
{
}
}
}
}
Just tested it, works as expected.
This is not full answer. Just general consideration and suggestion.
You cannot do different stuff or type of connection via the same socket. Why not have different sockets for different work? Some with authentication and some without. Some for quick task and some for long execution.
I did a simple web socket communication with spring 4, STOMP and sock.js, following this https://spring.io/guides/gs/messaging-stomp-websocket/
Now I want to upgrade it to simple chat. My problem is that when user subscribes to new chat room, he should get past messages. I don't know how to capture the moment when he subscribed to send him the list of the messages.
I tried using #MessageMapping annotation, but didn't reach any success:
#Controller
public class WebSocketController {
#Autowired
private SimpMessagingTemplate messagingTemplate;
#MessageMapping("/chat/{chatId}")
public void chat(ChatMessage message, #DestinationVariable String chatId) {
messagingTemplate.convertAndSend("/chat/" + chatId, new ChatMessage("message: " + message.getText()));
}
#SubscribeMapping("/chat")
public void chatInit() {
System.out.println("worked");
int chatId = 1; //for example
messagingTemplate.convertAndSend("/chat/" + chatId, new ChatMessage("connected"));
}
}
Then I created that:
#Controller
public class ApplicationEventObserverController implements ApplicationListener<ApplicationEvent> {
#Override
public void onApplicationEvent(ApplicationEvent applicationEvent) {
System.out.println(applicationEvent);
}
}
It works, but captures all possible events, I don't think it is a good practice.
So, my question can be rephrased: how to send initial data when user subscried to sth?
You can return anything directly to a client when it subscribes to a destination using a #SubscribeMapping handler method. The returned object won't go to the broker but will be sent directly to the client:
#SubscribeMapping("/chat")
public Collection<ChatMessage> chatInit() {
...
return messages;
}
On the client side:
socket.subscribe("/app/chat", function(message) {
...
});
Check out the chat example on GitHub, which shows this exact scenario.
I am using SpringBoot to start a SpringAMQP application that connect to RabbitMQ queues. I would like to be able to send a message from the producer, specifying the reply-queue so that the consumer would only need to send without having to investigate the destination (hence not having to pass the reply data in the message itself).
this is the configuration I have (shared between producer and consumer)
private static final String QUEUE_NAME = "testQueue";
private static final String ROUTING_KEY = QUEUE_NAME;
public static final String REPLY_QUEUE = "replyQueue";
private static final String USERNAME = "guest";
private static final String PASSWORD = "guest";
private static final String IP = "localhost";
private static final String VHOST = "/";
private static final int PORT = 5672;
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
amqpAdmin().declareQueue(new Queue(QUEUE_NAME));
amqpAdmin().declareQueue(new Queue(REPLY_QUEUE));
return template;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(IP);
connectionFactory.setUsername(USERNAME);
connectionFactory.setPassword(PASSWORD);
connectionFactory.setVirtualHost(VHOST);
connectionFactory.setPort(PORT);
return connectionFactory;
}
I am sending a message as follows :
public Object sendAndReply(String queue, String content){
return template.convertSendAndReceive(queue, new Data(content), new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setReplyTo(ReplyTester.REPLY_QUEUE);
return message;
}
});
}
and awaiting a reply as follows:
public void replyToQueue(String queue){
template.receiveAndReply(queue, new ReceiveAndReplyCallback<Data, Data>() {
#Override
public Data handle(Data payload) {
System.out.println("Received: "+payload.toString());
return new Data("This is a reply for: "+payload.toString());
}
});
}
When sending however, I get the following exception:
Exception in thread "main" org.springframework.amqp.UncategorizedAmqpException: java.lang.IllegalArgumentException: Send-and-receive methods can only be used if the Message does not already have a replyTo property.
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:66)
at org.springframework.amqp.rabbit.connection.RabbitAccessor.convertRabbitAccessException(RabbitAccessor.java:112)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:841)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:820)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doSendAndReceiveWithTemporary(RabbitTemplate.java:705)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doSendAndReceive(RabbitTemplate.java:697)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertSendAndReceive(RabbitTemplate.java:673)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertSendAndReceive(RabbitTemplate.java:663)
at prodsend.Prod.sendAndReply(ReplyTester.java:137)
at prodsend.ReplyTester.sendMessages(ReplyTester.java:49)
at prodsend.ReplyTester.main(ReplyTester.java:102)
Caused by: java.lang.IllegalArgumentException: Send-and-receive methods can only be used if the Message does not already have a replyTo property.
at org.springframework.util.Assert.isNull(Assert.java:89)
at org.springframework.amqp.rabbit.core.RabbitTemplate$6.doInRabbit(RabbitTemplate.java:711)
at org.springframework.amqp.rabbit.core.RabbitTemplate$6.doInRabbit(RabbitTemplate.java:705)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:835)
... 8 more
the line ReplyTest.137 points to the return line in the sendAndReply method above.
EDIT:
Here is the Data class that is mentioned above :)
class Data{
public String d;
public Data(String s){ d = s; }
public String toString() { return d; }
}
From the documentation:
Basic RPC pattern. Send a message to a default exchange with a specific routing key and attempt to receive a response. Implementations will normally set the reply-to header to an exclusive queue and wait up for some time limited by a timeout.
So the method convertSendAndReceive handles setting the replyTo header and returns a Messaage - the response. This is a synchronous pattern - RPC.
If you want to do this asynchronously - which you seem to - do not use this method. Use the appropriate convertAndSend method and use the appropriate MessagePostProcessor to add your replyTo header.
As this is asynchronous, you need to register a separate handler for receiving the reply. This needs to be done before sending the message to the other party. This handler will then be called at some point after sending the message - when is unknown. Read section 3.5.2 Asynchronous Consumer of the Spring AQMP Documentation.
So, asynchronous process flow:
sender registers a handler on replyTo queueue
sender sends message with replyTo set
client calls receiveAndReply, processes the message, and sends a reply to the replyTo
sender callback method is triggered
The synchronous process flow is:
sender sends message using sendAndReceive and blocks
client calls receiveAndReply, processes the message, and sends a reply to the replyTo
sender receives the reply, wakes and processes it
So the latter case requires the sender to wait. As you are using receiveXXX rather than registering asynchronous handlers, the sender could be waiting a very long time if the client takes a while to get around to calling receiveXXX.
Incidentally, if you want to use the synchronous approach but use a specific replyTo you can always call setReplyQueue. There is also a setReplyTimeout for the case I mention where the client either doesn't bother to read the message or forgets to reply.