The only thing I've found in the API is localConsumer so my idea was to register the consumer twice - once using consumer and once using localConsumer with the address prefixed by local:.
Here's an example:
public class VertxLocal {
public static void main(String[] args) {
final UUID id = UUID.randomUUID();
Config config = new Config();
config.getNetworkConfig().setPortAutoIncrement(true);
HazelcastClusterManager cm = new HazelcastClusterManager(config);
// This is io.vertx.rxjava.core.Vertx
Vertx.clusteredVertxObservable(new VertxOptions().setClusterManager(cm))
.subscribe(vertx -> {
EventBus bus = vertx.eventBus();
// Register the "public" consumer
bus.consumer("test")
.toObservable()
.subscribe(message -> {
message.reply(id.toString());
});
// Register the local consumer
bus.localConsumer("local:test")
.toObservable()
.subscribe(message -> {
message.reply(id.toString());
});
// Periodically retrieve replies from the local consumer
vertx.setPeriodic(1, i -> {
bus.sendObservable("local:test", id.toString())
.subscribe(reply -> {
UUID fromId = UUID.fromString((String)reply.body());
if(fromId.equals(id)) {
System.out.println("Received message from local handler");
} else {
System.out.println("Received message from remote handler");
}
});
});
});
}
}
This works, but it's rather ugly to register each consumer twice - I'd rather only choose if should be a local message or not when sending the message.
Is there any way of doing this without registering a local consumer - something like localSend (though I can't see anything like that in the API)?
Related
I had configured SQS listener to consume messages in List of Messages but I am only getting a single message at a time and getting error as cannot convert model.StudentData to the instance of java.util.ArrayList<com.amazonaws.services.sqs.model.Message>
my code is :-
#SqsListener(value = "${queueName}", deletionPolicy = SqsMessageDeletionPolicy.NEVER)
public void receiveMessage(final StudentData studentData,
#Header("SenderId") final String senderId, final Acknowledgment acknowledgment) {
// business logic
acknowledgment.acknowledge();
}
Any suggestion on how to configure sqs listener to consume multiple messages
any help will be appreciated
solution for the above issue is :-
final ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(() -> {
while (true) {
final String queueUrl = amazonSqs.getQueueUrl("enter your queue name").getQueueUrl();
final var receiveMessageRequest = new ReceiveMessageRequest(queueUrl)
.withWaitTimeSeconds(20);
List<Message> messages = amazonSqs.receiveMessage(receiveMessageRequest).getMessages();
while (messages.size() > 0) {
for (final Message queueMessage : messages) {
try {
String message = queueMessage.getBody();
amazonSqs.deleteMessage(new DeleteMessageRequest(queueUrl, queueMessage
.getReceiptHandle()));
} catch (Exception e) {
log.error("Received message with errors " + e);
}
}
messages = amazonSqs.receiveMessage(new ReceiveMessageRequest(queueUrl)).getMessages();
}
}
});
executorService.shutdown();
The SQS listener annotation provides the most simple configuration, it will consume messages one by one. This limitation comes directly from spring's
QueueMessagingTemplate.
To consume batches you could use AmazonSQS client directly.
#Autowire AmazonSQSAsync amazonSqs;
...
String queueUrl = amazonSqs.getQueueUrl("queueName").getQueueUrl();
ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest();
receiveMessageRequest.setQueueUrl(queueUrl);
receiveMessageRequest.setWaitTimeSeconds(10); // Listener for messages in the next 10 seconds
receiveMessageRequest.setMaxNumberOfMessages(1000); // If 10000 messages are read stop listening
ReceiveMessageResult receiveMessageResult = amazonSqs.receiveMessage(receiveMessageRequest);
receiveMessageResult.getMessages(); // batch of messages
I am using netty-socket.io and I implemented the server like the demo.
I receive onConnect event both on server and client, but when I sent a message {message: message} I don't get anything on the server event though I see the message being sent in the network tab.
Configuration config = new Configuration();
config.setHostname("localhost");
config.setPort(9092);
final SocketIOServer server = new SocketIOServer(config);
server.addConnectListener(socketIOClient -> System.out.println("Connection test"));
server.addEventListener("messageevent", MessageEventObject.class, new DataListener<MessageEventObject>() {
#Override
public void onData(SocketIOClient socketIOClient, MessageEventObject messageEventObject, AckRequest ackRequest) throws Exception {
System.out.println("message received!");
}
});
server.start();
My MessageEventObject has String message property, constructor getters and setters, looking the same as client-sided.
And this is my websocket service client-sided:
export class WebsocketService {
private socket;
private subject = new Subject < any > ();
constructor() {
console.log('test!');
}
public connect(host: string, port: number) {
this.socket = io(`http://${host}:${port}`, {
'reconnection': false
});
this.socket.on('connect', this.onConnected);
this.socket.on('connect_error', this.onConnectionFailure);
}
public getConnectionStateUpdate(): Observable < any > {
return this.subject.asObservable();
}
public sendMessage(message: string) {
console.log('test');
this.socket.emit('messageevent', {
message: message
});
}
private onConnected = () => {
this.subject.next({
connected: true
});
}
private onConnectionFailure = () => {
this.subject.next({
connected: false
});
}
}
Is there anything that I did wrong?
I would love to answer my own question after tons of debugging and breaking my head, my laziness to use Engine IO with tomcat or jetty, and just wanting to use that awesome netty package which does not require any servlets, I tried to fix it and figure out.
At first I thought it was the client's protocol version, so I used the exact same client as the demo shows on their github page here but that didn't work so the problem is server-sided.
It appears that your object (MessageEventObject) must have a default empty constructor aswell in addition to your other constructors, probably because netty tries to build an empty object and it fails which causes an exception that you don't see.
I've used the subscriber example from the google documentation for Google PubSub
the only modification I've made is commenting out the acknowledgement of the messages.
The subscriber doesn't add messages to the queue anymore while messages should be resent according to the interval set in the google cloud console.
Why is this happening or am I missing something?
public class SubscriberExample {
use the default project id
private static final String PROJECT_ID = ServiceOptions.getDefaultProjectId();
private static final BlockingQueue<PubsubMessage> messages = new LinkedBlockingDeque<>();
static class MessageReceiverExample implements MessageReceiver {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
messages.offer(message);
//consumer.ack();
}
}
/** Receive messages over a subscription. */
public static void main(String[] args) throws Exception {
// set subscriber id, eg. my-sub
String subscriptionId = args[0];
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(
PROJECT_ID, subscriptionId);
Subscriber subscriber = null;
try {
// create a subscriber bound to the asynchronous message receiver
subscriber = Subscriber.newBuilder(subscriptionName, new MessageReceiverExample()).build();
subscriber.startAsync().awaitRunning();
// Continue to listen to messages
while (true) {
PubsubMessage message = messages.take();
System.out.println("Message Id: " + message.getMessageId());
System.out.println("Data: " + message.getData().toStringUtf8());
}
} finally {
if (subscriber != null) {
subscriber.stopAsync();
}
}
}
}
When you do not acknowledge a messages, the Java client library calls modifyAckDeadline on the message until maxAckExtensionPeriod passes. By default, this value is one hour. Therefore, if you don't ack/nack the message or change this value, it is likely the message will not be redelivered for an hour. If you want to change the max ack extension period, set it on the builder:
subscriber = Subscriber.newBuilder(subscriptionName, new MessageReceiverExample())
.setMaxAckExtensionPeriod(Duration.ofSeconds(60))
.build();
It is also worth noting that when you don't ack or nack messages, then flow control may prevent the delivery of more messages. By default, the Java client library allows up to 1000 messages to be outstanding, i.e., waiting for ack or nack or for the max ack extension period to pass.
Gary Russell kindly answered a previous question of mine about Spring Integration udp flows. Moving from there, I have stumbled upon an issue with ports.
The Spring Integration documentation says that you can put 0 to the inbound channel adapter port, and the OS will select an available port for the adapter, which can be retrieved at runtime invoking getPort() on the adapter object. The problem is that at runtime I just get a 0 if I try to retrieve the port programmatically.
Here's "my" code (i.e. a slightly modified version of Russel's answer to my previous question for Spring Integration 4.3.12, which I am currently using).
#SpringBootApplication
public class TestApp {
private final Map<Integer, IntegrationFlowRegistration> registrations = new HashMap<>();
#Autowired
private IntegrationFlowContext flowContext;
public static void main(String[] args) {
SpringApplication.run(TestApp.class, args);
}
#Bean
public PublishSubscribeChannel channel() {
return new PublishSubscribeChannel();
}
#Bean
public TestData test() {
return new TestData();
}
#Bean
public ApplicationRunner runner() {
return args -> {
UnicastReceivingChannelAdapter source;
source = makeANewUdpInbound(0);
makeANewUdpOutbound(source.getPort());
Thread.sleep(5_000);
channel().send(MessageBuilder.withPayload("foo\n").build());
this.registrations.values().forEach(r -> {
r.stop();
r.destroy();
});
this.registrations.clear();
makeANewUdpInbound(1235);
makeANewUdpOutbound(1235);
Thread.sleep(5_000);
channel().send(MessageBuilder.withPayload("bar\n").build());
this.registrations.values().forEach(r -> {
r.stop();
r.destroy();
});
this.registrations.clear();
};
}
public UnicastSendingMessageHandler makeANewUdpOutbound(int port) {
System.out.println("Creating an adapter to send to port " + port);
UnicastSendingMessageHandler adapter = new UnicastSendingMessageHandler("localhost", port);
IntegrationFlow flow = IntegrationFlows.from(channel())
.handle(adapter)
.get();
IntegrationFlowRegistration registration = flowContext.registration(flow).register();
registrations.put(port, registration);
return adapter;
}
public UnicastReceivingChannelAdapter makeANewUdpInbound(int port) {
System.out.println("Creating an adapter to receive from port " + port);
UnicastReceivingChannelAdapter source = new UnicastReceivingChannelAdapter(port);
IntegrationFlow flow = IntegrationFlows.from(source)
.<byte[], String>transform(String::new)
.handle(System.out::println)
.get();
IntegrationFlowRegistration registration = flowContext.registration(flow).register();
registrations.put(port, registration);
return source;
}
}
The output I read is
Creating an adapter to receive from port 0
Creating an adapter to send to port 0
Creating an adapter to receive from port 1235
Creating an adapter to send to port 1235
GenericMessage [payload=bar, headers={ip_packetAddress=127.0.0.1/127.0.0.1:54374, ip_address=127.0.0.1, id=c95d6255-e63a-433d-3723-c389fe66b060, ip_port=54374, ip_hostname=127.0.0.1, timestamp=1517220716983}]
I suspect the library did create adapters on OS-chosen free ports, but I am unable to retrieve the assigned port.
The port is assigned asynchronously; you need to wait until the port is actually assigned. Something like...
int n = 0;
while (n++ < 100 && ! source.isListening()) {
Thread.sleep(100;
}
if (!source.isListening()) {
// failed to start in 10 seconds.
}
We should probably enhance the adapter to emit an event when the port is ready. Feel free to open an 'Improvement' JIRA Issue.
Changed:
How to response http request from socket.
Web code:
public void start() {
Router router = Router.router(vertx);
router.route("/api/getdata").handler(this::getData);
vertx.createHttpServer().requestHandler(router::accept).listen(8080);
}
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ServerVerticle.ADDRESS, pricemessage, reply -> {
});
}
Socket code:
public void start() {
final EventBus eb = vertx.eventBus();
NetClient netClient = vertx.createNetClient();
if (ar.succeeded()) {
socket.handler(this::onDataReceived);
eb.consumer(ServerVerticle.ADDRESS, message -> {
socket.write(buffer); // request from the getData method
message.reply(data);// no data here, it's in the handler
}
}
}
private void onDataReceived(Buffer buffer) {
// buffer changed to JsonObject here
vertx.eventBus().send("some address here", jsonObject);
}
The socket handler has no return value. Just a eventbus send.
And I don't know how to response this jsonObject to the http request.
========================================================================
Old question, maybe not clear.
I have a vertex that handles the socket write and response.
public void start() { // 1
NetClient netClient = vertx.createNetClient();
netClient.connect(port, host, ar -> {
socket = ar.result(); // NetSocket
socket.handler(this::doSocketHandleMethod);
socket.write(BYTEBUFFER);// buffer here
})
}
private void doSocketHandleMethod(Buffer buffer){ // socket handler
// process data here and send
vertx.eventBus().send(ADDRESS, data here);
}
I use the below code to fetch the response from the http request.
public void start() {
Router router = Router.router(vertx);
router.route(API_GET).handler(this::getData);
vertx.eventBus().consumer(ADDRESSHERE, msg -> {
// get data from the socket send. 2
});
vertx.createHttpServer().requestHandler(router::accept).listen(8080, result -> {
});
}
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ADDRESS, message); // send message to the top // 1 verticle
// 3
}
The question is that the second code mention above gets the the data, but not sure how to fetch the response from the http reqest 3.
The (HttpServerRequest) is passed to the route (requestHandler(router::accept)) and is contained in the RoutingContext. "As HTTP requests are received by the server, instances of [...].HttpServerRequest will be created and passed to this handler." - JavaDoc
So, if the data arrives at 2 and you want to do a response to a HttpServerRequest (as a third step), you can use routingContext.response() in the getData() method, to get a HttpServerResponse.
If you want to handle a http server request, by sending a message to a consumer that is getting some data from a socket and want to send this result as a reply to the specific http server request, then you may do something like this:
// Send a message and get the response via handler
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ADDRESS, message, handler -> {
if(handler.succeded()) {
routingContext.response().end(handler.result());
}
else {
// error
}
});
}
// To reply to a message do
vertx.eventBus().consumer(ADDRESSHERE, msg -> {
// get data from the socket send. 2
msg.reply(data); // you can only do a reply once. Put data into reply
});
As far as I know, the event bus only knows "send and reply" and not a concept like a socket. It looks like you want to send data everytime new data is available through the socket.
You can write something to a httpResponse mutliple times, so you need to save a reference to the response object.
But I do not know, if that is such a good idea. I would recommend to encapsulate the socket-get-data process. The "socket" verticle only answers once, with the whole buffer it got. Here are two examples on what I mean.
// open socket
vertx.eventBus().consumer("ADRRESS", message -> {
// execute this on worker thread to not block the event bus thread
vertx.executeBlocking(future -> {
Buffer buffer = Buffer.buffer();
socket.handler(buff -> buffer.appendBuffer(buff)) // read data
.endHandler(endHandler -> {
// no more data to read
socket.pause();
future.complete(buffer);
})
.resume() // socket was paused, now read the data
.exceptionHandler(err -> future.fail(err)); // handle exception
}, result -> {
if(result.succeeded()) {
message.reply(result.result()); // reply with the buffer content
}
else {
message.reply(result.cause()); // may want to send error later
}
});
});
// connect and get a new socket every time
vertx.eventBus().consumer("ADRRESS", message -> {
// execute this on worker thread to not block the event bus thread
vertx.executeBlocking(future -> {
netClient.connect(1, "", netSocketHandler -> {
if(netSocketHandler.succeeded()) {
Buffer buffer = Buffer.buffer();
netSocketHandler.result().handler(buff -> buffer.appendBuffer(buff)) // read data
.endHandler(endHandler -> {
// no more data to read
future.complete(buffer);
netSocketHandler.result().close(); // close the NetSocket once finished
})
.exceptionHandler(err -> {
netSocketHandler.result().close();
future.fail(err);
}); // handle exceptions
}
else {
future.fail(netSocketHandler.cause());
}
});
}, result -> {
if(result.succeeded()) {
message.reply(result.result()); // reply with the buffer content
}
else {
message.reply(result.cause()); // may want to send error later
}
});
});
If this realy does not help you, I'm sorry, and maybe this is not the concept you are looking for.