I have just started experimenting with Spring and rabbitMQ.
I would like to create a microsevice infrastructure with rabbit and spring,
I have been following Spring boot tutorial
But it is very simplistic. As well I am looking at the documentation (springs, Rabbit) for how to create an RPC, i understand the Rabbits approach, but i would like to leverage Spring template to save me the boilerplate.
I just cant seem to understand where to register the reciveAndReplay callback at.
I tried doing this:
sending
System.out.println("Sending message...");
Object convertSendAndReceive = rabbitTemplate.convertSendAndReceive("spring-boot", "send and recive: sent");
System.out.println("GOT " + convertSendAndReceive); //is null
receiving
#Component
public class Receiver {
#Autowired
RabbitTemplate rabbitTemplate;
public void receiveMessage(String message) {
this.rabbitTemplate.receiveAndReply("spring-boot", (Message)->{
return "return this statement";
});
}
}
But its not a big surprise this doesn't work the message is received but nothing comes back. I assume that this needs to be registered somewhere in the factory/template at the bean creation level but i don't seem to understand where and sadly the documentation is unclear.
First, please use the Spring AMQP Documentation.
You would generally use a SimpleMessageListenerContainer wired with a POJO listener for RPC.
The template receiveAndReply method is intended for "scheduled" server-side RPC - i.e. only receive (and reply) when you want to, rather than whenever a message arrives in the queue. It does not block waiting for a message.
If you want to use receiveAndReply(), there's a test case that illustrates it.
EDIT:
This code...
this.template.convertAndSend(ROUTE, "test");
sends a message to the queue.
This code...
this.template.setQueue(ROUTE);
boolean received = this.template.receiveAndReply(new ReceiveAndReplyMessageCallback() {
#Override
public Message handle(Message message) {
message.getMessageProperties().setHeader("foo", "bar");
return message;
}
});
Receives a message and from that queue; adds a header and returns the same messsage to the reply queue. received will be false if there was no message to receive (and reply to).
This code:
Message receive = this.template.receive();
receives the reply.
This test is a bit contrived because the reply is sent to the same queue as the request. We can't use sendAndReceive() on the client side in this test because the thread would block waiting for the reply (and we need to execute the receiveAndReply()).
Another test in that class has a more realistic example where it does the sendAndReceive()s on different threads and the receiveAndReply()s on the main thread.
Note that that test uses a listener container on the client side for replies; that is generally no longer needed since the rabbit broker now supports direct reply-to.
receiveAndReply() was added for symmetry - in most cases, people use a listener container and listener adapter for server-side RPC.
Related
We try to publish and subscribe to MQTT protocol using smallrye reactive messaging. We managed to actually publish a message into a specific topic/channel through the following simple code
import io.smallrye.mutiny.Multi;
import org.eclipse.microprofile.reactive.messaging.Outgoing;
import javax.enterprise.context.ApplicationScoped;
import java.time.Duration;
#ApplicationScoped
public class Publish {
#Outgoing("pao")
public Multi<String> generate() {
return Multi.createFrom().ticks().every(Duration.ofSeconds(1))
.map(x -> "A Message in here");
}
}
What we want to do is to call whenever we want the generate() method somehow with a dynamic topic, where the user will define it. That one was our problem but then we found these classes from that repo in github. Package name io.smallrye.reactive.messaging.mqtt
For example we found that there is a class that says it makes a publish call to a MQTT broker(Mosquitto server up).
Here in that statement SendingMqttMessage<String> message = new SendingMqttMessage<String>("myTopic","A message in here",0,false);
We get the a red underline under the SendingMqttMessage<String> saying 'SendingMqttMessage(java.lang.String, java.lang.String, io.netty.handler.codec.mqtt.MqttQoS, boolean)' is not public in 'io.smallrye.reactive.messaging.mqtt.SendingMqttMessage'. Cannot be accessed from outside package
UPDATE(Publish done)
Finally made a Publish request to the mqtt broker(a mosquitto server) and all this with a dynamic topic configured from user. As we found out the previous Class SendingMqttMessage was not supposed to be used at all. And we found out that we also needed and emitter to actually make a publish request with a dynamic topic.
#Inject
#Channel("panatha")
Emitter<String> emitter;
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response createUser(Device device) {
System.out.println("New Publish request: message->"+device.getMessage()+" & topic->"+device.getTopic());
emitter.send(MqttMessage.of(device.getTopic(), device.getMessage()));
return Response.ok().status(Response.Status.CREATED).build();
}
Now we need to find out about making a Subscription to a topic dynamically.
first to sett us to the same page:
Reactive messaging does not work with topics, but with channels.
That is important to note, because you can exclusively read or write to a channel. So if you want to provide both, you need to configure two channels pointing at the same topic, one incoming and one outgoing
To answer your question:
You made a pretty good start with Emitters, but you still lack the dynamic nature you'd like.
In your example, you acquired that Emitter thru CDI.
Now that is all we need, to make this dynamic, since we cann dynamically inject Beans at runtime using CDI like this:
Sending Messages
private Emitter<byte[]> dynamicEmitter(String topic){
return CDI.current().select(new TypeLiteral<Emitter<byte[]>>() {}, new ChannelAnnotation(topic)).get();
}
please also note, that i am creating a Emitter of type byte[], as this is the only currently supportet type of the smallrye-mqtt connector (version 3.4.0) according to its documentation.
Receiving Messages
To read messages from a reactive messaging channel, you can use the counterpart of the Emitter, which is the Publisher.
It can be used analog:
private Publisher<byte[]> dynamicReceiver(String topic){
return CDI.current().select(new TypeLiteral<Publisher<byte[]>>() {}, new ChannelAnnotation(topic)).get();
}
You can then process these Date in any way you like.
As demo, it hung it on a simple REST Endpoint
#GET
#Produces(MediaType.SERVER_SENT_EVENTS)
public Multi<String> stream(#QueryParam("topic") String topic) {
return Multi.createFrom().publisher(dynamicReceiver(topic)).onItem().transform(String::new);
}
#GET
#Path("/publish")
public boolean publish(#QueryParam("msg") String msg, #QueryParam("topic") String topic) {
dynamicEmitter(topic).send(msg.getBytes());
return true;
}
One more Thing
When creating this solution I hit a few pitfalls you should know about:
Quarkus removes any CDI-Beans that are "unused". So if you want to inject them dynamically, you need to exclude those, or turne off that feature.
All channels injected that way must be configured. Otherwise the injection will fail.
For some Reason, (even with removal completely disabled) I was unable to inject Emitters dynamically, unless they are ever injected elsewhere.
I need to show off an example of server sent events. I learned about it in a spring talk. People used Webflux there to show the reactive principles. I understood the part on how this will free thread resources because the request thread won't be blocked until the job is done and the server returns the response.
I have an example here but actually I don't really know how I can make this thread resource example be clear enough.
I do not want to use the WebFlux framework here. Just need to know what to put into a separate thread here - if necessary at all?!
As you can see I have a GetMapping to subscribe to the event stream. And then I have a GetMapping to launch or fire an event. This example is fast for sure but should be considered as heavy database call.
So I actually need to have the whole logic be separated in another thread right? So the request thread is free as soon as possible?
#RestController
public class EventStreamRequestHandler {
#Autowired
ObjectMapper objectMapper;
SseEmitter sseEmitter = new SseEmitter(1000000L);
#GetMapping("/get/event/stream")
public SseEmitter getStream() {
return this.sseEmitter;
}
#GetMapping("/launch/event")
public void fireEvent() throws IOException {
Person peter = new Person("Peter", "25");
String valueAsString = objectMapper.writeValueAsString(peter);
SseEmitter.SseEventBuilder sseEventBuilder = SseEmitter.event()
.id("foo")
.name("awesome-event")
.data(valueAsString);
sseEmitter.send(sseEventBuilder);
}
}
Yes, Server sent events are supposed to send messages to the client asynchronously without client keep on polling for message.
The sending of messages from client to server needs to be done asynchronously. With the way you have done it. When a GET request is sent to /get/event/stream an SseEmitter will be created but messages will only be sent when a GET request is sent to /launch/event. And the GET request thread for /launch/event will be used to send the message.
Sometime back I wrote post to send SSE messages using a different thread. I hope this helps.
But I don't recommend storing the SseEmitter in an instance variable as it will overridden by multiple requests. You must at least make it ThreadLocal
I'm trying to create a Spring Cloud Stream Source Bean inside a Spring Boot Application that simply sends the results of a method to a stream (underlying Kafka topic is bound to the stream).
Most of the Stream samples I've seen use #InboundChannelAdapter annotation to send data to the stream using a poller. But I don't want to use a poller. I've tried setting the poller to an empty array but the other problem is that when using #InboundChannelAdapter you are unable to have any method parameters.
The overall concept of what I am trying to do is read from an inbound stream. Do some async processing, then post the result to an outbound stream. So using a processor doesn't seem to be an option either. I am using #StreamListener with a Sink channel to read the inbound stream and that works.
Here is some code i've been trying but this doesn't work at all. I was hoping it would be this simple because my Sink was but maybe it isn't. Looking for someone to point me to an example of a source that isn't a Processor (i.e. doesn't require listening on an inbound channel) and doesn't use #InboundChannelAdapter or to give me some design tips to accomplish what I need to do in a different way. Thanks!
#EnableBinding(Source.class)
public class JobForwarder {
#ServiceActivator(outputChannel = Source.OUTPUT)
#SendTo(Source.OUTPUT)
public String forwardJob(String message) {
log.info(String.format("Forwarding a job message [%s] to queue [%s]", message, Source.OUTPUT));
return message;
}
}
Your orginal requirement can be achieved through the below steps.
Create your custom Bound Interface (you can use the default #EnableBinding(Source.class) as well)
public interface CustomSource {
String OUTPUT = "customoutput";
#Output(CustomSource.OUTPUT)
MessageChannel output();
}
Inject your bound channel
#Component
#EnableBinding(CustomSource.class)
public class CustomOutputEventSource {
#Autowired
private CustomSource customSource;
public void sendMessage(String message) {
customSource.output().send(MessageBuilder.withPayload(message).build());
}
}
Test it
#RunWith(SpringRunner.class)
#SpringBootTest
public class CustomOutputEventSourceTest {
#Autowired
CustomOutputEventSource output;
#Test
public void sendMessage() {
output.sendMessage("Test message from JUnit test");
}
}
So if you don't want to use a Poller, what causes the forwardJob() method to be called?
You can't just call the method and expect the result to go to the output channel.
With your current configuration, you need an inputChannel on the service containing your inbound message (and something to send a message to that channel). It doesn't have to be bound to a transport; it can be a simple MessageChannel #Bean.
Or, you could use a #Publisher to publish the result of the method invocation (as well as being returned to the caller) - docs here.
#Publisher(channel = Source.OUTPUT)
Thanks for the input. It took me a while to get back to the problem. I did try reading the documentation for #Publisher. It looked to be exactly what I needed but I just couldn't get the proper beans initialized to get it wired properly.
To answer your question the forwardJob() method is called after some async processing of the input.
Eventually I just implemented using spring-kafka library directly and that was much more explicit and felt easier to get going. I think we are going to stick to kafka as the only channel binding so I think we'll stick with that library.
However, we did eventually get the spring-cloud-stream library working quite simply. Here was the code for a single source without a poller.
#Component
#EnableBinding(Source.class)
public class JobForwarder {
private Source source;
#Autowired
public ScheduledJobForwarder(Source source) {
this.source = source;
}
public void forwardScheduledJob(String message) {
log.info(String.format("Forwarding a job message [%s] to queue [%s]", message, Source.OUTPUT));
source.output().send(MessageBuilder.withPayload(message).build());
}
}
I'm using Spring MVC, ActiveMQ and WebSocket(via sock.js and stomp.js) to build a real-time data delivery application.
As we know, when a producer(another desktop application) push a message to ActiveMQ, and the onMessage() method will catch it.
public class MessageReceiver implements MessageListener {
public void onMessage(Message message) {
System.out.println(message);
// How to push the message to a Controller in Spring?
}
}
Most of the tutorials just print the message to the console.
I have another controller called WebSocketController:
#Controller
public class WebSocketController {
#SubscribeMapping("/getRealTimeResult.action/{submissionId}")
public Message getRealTimeResultAction(
#DestinationVariable long submissionId) {
return new Message("Message content from onMessage() method");
}
}
I want to push the message received in onMessage() method to the getRealTimeResultAction() method. Can you tell me how to do it?
I know that the ActiveMQ can communicate with the browser using stomp via the port 61613.
I don't want to do this because I think the MQ should be transparent to the user. Also I need to do some authorization in the WebSocketController.
Generally speaking an #Controller with #SubscribeMapping and #MessageMapping methods can handle subscriptions and messages from STOMP clients connected over WebSocket.
From your description it's not clear what you're trying to do. Was the message pushed to ActiveMQ via STOMP from a browser client or was it produced by some other back-end JMS client? Also the MessageReceiver receives an actual message while the #Controller method has an #SubscribeMapping method for handling a subscription from a STOMP client. It's not clear what you're trying to do. Please provide more information so I can provide a better answer.
I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}
This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.