I am trying to connect to a message queue running on a docker container using Reactor Netty. I am doing this as standalone, not using SpringFlux because of dependency issues.
From the examples in the Reactor Netty documentation I saw there is a way to connect to the Server and get a response:
public static void main(String[] args) {
String response =
HttpClient.create()
.headers(h -> h.add("my header", my_header)
.get()
.uri(my_uri)
.responseContent()
.aggregate()
.asString()
.block();
}
but when I try afterwards to display the output via System.out.println() nothing happens.
I also tried to understand how to use:
Flux<V> response(BiFunction<HttpClientResponse,ByteBufFlux,Publisher<V>> receiver)
But I am not sure exactly what to do.
I saw in the documentation there is a class called Connection, which uses a TCPClient and has a method subscribe.
I am kind of lost, can you possibly point me in the right direction of implementing this in Reactor Netty without the use of spring-flux?
Thank you
EDIT:
After some experimentation i got this:
private Disposable subscribe() {
return HttpClient.create()
.headers(h -> h.add("my header", my_header)
.get()
.uri(my_uri)
.response((res, bytes) - > {
System.out.println(bytes.asString());
return bytes.asString();})
.subscribe();
}
This gives me a FluxHandle, how can I use that to actually read the body of the response?
So I figured out how to subscibe and read the data recieved from the Server and even transform the data to JSON, using the jackson library, to be more easily read by my code.
private Disposable subscribe() {
return HttpClient.create()
.headers(h -> h.add("my header", my_header)
.get()
.uri(my_uri)
.response((resp, bytes) -> {
return bytes.asString();
})
.subscribe(response -> {
try {
consumeData(new ObjectMapper()
.readValue(response, MyData.class));
} catch (IOException ex) {
System.out.println("ERROR converting to json: " + ex);
}
});
}
it seems that when using the subscribe() method i can listen to incoming responses and do something with them. I still need to add a way for the connection to close when the server stops, or the message queue is shut down, so the client doesn't hang on non-existent message queue.
Related
I am trying to achieve parallel process with completableFuture
i have 2 downstreams in completableFuture, where i need the response of 1 downstream.
Am able to my downstream calls with completableFuture and fetch the results as well.
But I am unable to read response and act on that , which also includes handling exception.
I tried with .completedExceptionally, handle in completableFuture (after .get() which waits for process to get completed) . . Can you please help me where am going wrong here
//no response needed
CompletableFuture<Void> cf0
= CompletableFuture.runAsync(() -> sdk1.method1(param1));
//response needed
CompletableFuture<Response> cf1
= CompletableFuture.supplyAsync(() -> service1.callapi(
param1, param2));
final CompletableFuture<Void> cf3 = CompletableFuture.allOf(
cf0,
cf1);
try {
//wait to get both the calls completed.
cf3.get();
response1 = cf1.get();
}catch (Exception e) {
throw new Exception(e.getMessage(), e.getMessage(), e.getMessage(),
500);
}
My question is
I have processingResponse method in different java class for the second downstream, how to call that method with param irrespective of happyPath or exception
if that processingResponse throws some error, how do i wrap or throw same error with completableFuture.
when the api throws some error my catch block is throwing default exception(which i dont want, the processResponse method exception should be thrown and stopped there). in my case i see the catch block exception immediately when the api call fails
Appreciate your help in this, Thanks
I recently changed from using a standard Rabbit Template, in my Spring Boot application, to using an Async Rabbit Template. In the process, I switched from the standard send method to using the sendAndReceive method.
Making this change does not seem to affect the publishing of messages to RabbitMQ, however I do now see stack traces as follows when sending messages:
org.springframework.amqp.core.AmqpReplyTimeoutException: Reply timed out
at org.springframework.amqp.rabbit.AsyncRabbitTemplate$RabbitFuture$TimeoutTask.run(AsyncRabbitTemplate.java:762) [spring-rabbit-2.3.10.jar!/:2.3.10]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.3.9.jar!/:5.3.9]
I have tried modifying various settings including the reply and receive timeouts but all that changes is the time it takes to receive the above error. I have also tried setting useDirectReplyToContainer to true as well as setting useChannelForCorrelation to true.
I have managed to recreate the issue in a main method, included bellow, using a RabbitMQ broker running in docker.
public static void main(String[] args) {
com.rabbitmq.client.ConnectionFactory cf = new com.rabbitmq.client.ConnectionFactory();
cf.setHost("localhost");
cf.setPort(5672);
cf.setUsername("<my-username>");
cf.setPassword("<my-password>");
cf.setVirtualHost("<my-vhost>");
ConnectionFactory connectionFactory = new CachingConnectionFactory(cf);
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setExchange("primary");
rabbitTemplate.setUseDirectReplyToContainer(true);
rabbitTemplate.setReceiveTimeout(10000);
rabbitTemplate.setReplyTimeout(10000);
rabbitTemplate.setUseChannelForCorrelation(true);
AsyncRabbitTemplate asyncRabbitTemplate = new AsyncRabbitTemplate(rabbitTemplate);
asyncRabbitTemplate.start();
System.out.printf("Async Rabbit Template Running? %b\n", asyncRabbitTemplate.isRunning());
MessageBuilderSupport<MessageProperties> props = MessagePropertiesBuilder.newInstance()
.setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN)
.setMessageId(UUID.randomUUID().toString())
.setHeader(PUBLISH_TIME_HEADER, Instant.now(Clock.systemUTC()).toEpochMilli())
.setDeliveryMode(MessageDeliveryMode.NON_PERSISTENT);
asyncRabbitTemplate.sendAndReceive(
"1.1.1.csv-routing-key",
new Message(
"a,test,csv".getBytes(StandardCharsets.UTF_8),
props.build()
)
).addCallback(new ListenableFutureCallback<>() {
#Override
public void onFailure(Throwable ex) {
System.out.printf("Error sending message:\n%s\n", ex.getLocalizedMessage());
}
#Override
public void onSuccess(Message result) {
System.out.println("Message successfully sent");
}
});
}
I am sure that I am just missing a configuration option but any help would be appricated.
Thanks. :)
asyncRabbitTemplate.sendAndReceive(..) will always expect a response from the consumer of the message, hence the timeout you are receiving.
To fire and forget use the standard RabbitTemplate.send(...) and catching any exceptions in a try/catch block:
try {
rabbitTemplate.send("1.1.1.csv-routing-key",
new Message(
"a,test,csv".getBytes(StandardCharsets.UTF_8),
props.build());
} catch (AmqpException ex) {
log.error("failed to send rabbit message, routing key = {}", routingKey, ex);
}
Set reply timeout to some bigger number and see the effect.
rabbitTemplate.setReplyTimeout(60000);
https://docs.spring.io/spring-amqp/reference/html/#reply-timeout
I am trying to register a simple REST service on int port,
to ZooKeeper server at localhost:2181.
I checked path ls / using zooClient too.
Any ideas?
private static void registerInZookeeper(int port) throws Exception {
CuratorFramework curatorFramework = CuratorFrameworkFactory
.newClient("localhost:2181", new RetryForever(5));
curatorFramework.start();
ServiceInstance<Object> serviceInstance = ServiceInstance.builder()
.address("localhost")
.port(port)
.name("worker")
.uriSpec(new UriSpec("{scheme}://{address}:{port}"))
.build();
ServiceDiscoveryBuilder.builder(Object.class)
.basePath("myNode")
.client(curatorFramework)
.thisInstance(serviceInstance)
.build()
.start();
Optional.ofNullable(curatorFramework.checkExists().forPath("/zookeeper")).ifPresent(System.out::println);
Optional.ofNullable(curatorFramework.checkExists().forPath("/myNode")).ifPresent(System.out::println);
}
I kept receiving Received packet at server of unknown type 15 from Zoo Server, because of compatibility issues
the registration code here looks correct. In order to print registered instances the following code can be executed:
Optional.ofNullable(curatorFramework.getChildren().forPath("/myNode/worker"))
.orElse(Collections.emptyList())
.forEach(childNode -> {
try {
System.out.println(childNode);
System.out.println(new String(curatorFramework.getData().forPath("/myNode/worker/" + childNode)));
} catch (Exception e) {
e.printStackTrace();
}
});
The result will be like
07:23:12.353 INFO [main-EventThread] ConnectionStateManager:228 - State change: CONNECTED
48202336-e89b-4724-912b-89620f7c9954
{"name":"worker","id":"48202336-e89b-4724-912b-89620f7c9954","address":"localhost","port":1000,"sslPort":null,"payload":null,"registrationTimeUTC":1515561792319,"serviceType":"DYNAMIC","uriSpec":{"parts":[{"value":"scheme","variable":true},{"value":"://","variable":false},{"value":"address","variable":true},{"value":":","variable":false},{"value":"port","variable":true}]}}
Creating your curator framework with zk34 (the version used by kafka) compatibility should fix your problem
private CuratorFramework buildFramework(String ip) {
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
return CuratorFrameworkFactory.builder().zk34CompatibilityMode(true).connectString(ip + ":2181")
.retryPolicy(retryPolicy).build();
}
Please note that curator will just try its best and some new methods (eg. creatingParentsIfNeeded (ok) vs creatingParentContainersIfNeeded (ko)) will fail.
Changed:
How to response http request from socket.
Web code:
public void start() {
Router router = Router.router(vertx);
router.route("/api/getdata").handler(this::getData);
vertx.createHttpServer().requestHandler(router::accept).listen(8080);
}
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ServerVerticle.ADDRESS, pricemessage, reply -> {
});
}
Socket code:
public void start() {
final EventBus eb = vertx.eventBus();
NetClient netClient = vertx.createNetClient();
if (ar.succeeded()) {
socket.handler(this::onDataReceived);
eb.consumer(ServerVerticle.ADDRESS, message -> {
socket.write(buffer); // request from the getData method
message.reply(data);// no data here, it's in the handler
}
}
}
private void onDataReceived(Buffer buffer) {
// buffer changed to JsonObject here
vertx.eventBus().send("some address here", jsonObject);
}
The socket handler has no return value. Just a eventbus send.
And I don't know how to response this jsonObject to the http request.
========================================================================
Old question, maybe not clear.
I have a vertex that handles the socket write and response.
public void start() { // 1
NetClient netClient = vertx.createNetClient();
netClient.connect(port, host, ar -> {
socket = ar.result(); // NetSocket
socket.handler(this::doSocketHandleMethod);
socket.write(BYTEBUFFER);// buffer here
})
}
private void doSocketHandleMethod(Buffer buffer){ // socket handler
// process data here and send
vertx.eventBus().send(ADDRESS, data here);
}
I use the below code to fetch the response from the http request.
public void start() {
Router router = Router.router(vertx);
router.route(API_GET).handler(this::getData);
vertx.eventBus().consumer(ADDRESSHERE, msg -> {
// get data from the socket send. 2
});
vertx.createHttpServer().requestHandler(router::accept).listen(8080, result -> {
});
}
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ADDRESS, message); // send message to the top // 1 verticle
// 3
}
The question is that the second code mention above gets the the data, but not sure how to fetch the response from the http reqest 3.
The (HttpServerRequest) is passed to the route (requestHandler(router::accept)) and is contained in the RoutingContext. "As HTTP requests are received by the server, instances of [...].HttpServerRequest will be created and passed to this handler." - JavaDoc
So, if the data arrives at 2 and you want to do a response to a HttpServerRequest (as a third step), you can use routingContext.response() in the getData() method, to get a HttpServerResponse.
If you want to handle a http server request, by sending a message to a consumer that is getting some data from a socket and want to send this result as a reply to the specific http server request, then you may do something like this:
// Send a message and get the response via handler
private void getData(RoutingContext routingContext) {
vertx.eventBus().send(ADDRESS, message, handler -> {
if(handler.succeded()) {
routingContext.response().end(handler.result());
}
else {
// error
}
});
}
// To reply to a message do
vertx.eventBus().consumer(ADDRESSHERE, msg -> {
// get data from the socket send. 2
msg.reply(data); // you can only do a reply once. Put data into reply
});
As far as I know, the event bus only knows "send and reply" and not a concept like a socket. It looks like you want to send data everytime new data is available through the socket.
You can write something to a httpResponse mutliple times, so you need to save a reference to the response object.
But I do not know, if that is such a good idea. I would recommend to encapsulate the socket-get-data process. The "socket" verticle only answers once, with the whole buffer it got. Here are two examples on what I mean.
// open socket
vertx.eventBus().consumer("ADRRESS", message -> {
// execute this on worker thread to not block the event bus thread
vertx.executeBlocking(future -> {
Buffer buffer = Buffer.buffer();
socket.handler(buff -> buffer.appendBuffer(buff)) // read data
.endHandler(endHandler -> {
// no more data to read
socket.pause();
future.complete(buffer);
})
.resume() // socket was paused, now read the data
.exceptionHandler(err -> future.fail(err)); // handle exception
}, result -> {
if(result.succeeded()) {
message.reply(result.result()); // reply with the buffer content
}
else {
message.reply(result.cause()); // may want to send error later
}
});
});
// connect and get a new socket every time
vertx.eventBus().consumer("ADRRESS", message -> {
// execute this on worker thread to not block the event bus thread
vertx.executeBlocking(future -> {
netClient.connect(1, "", netSocketHandler -> {
if(netSocketHandler.succeeded()) {
Buffer buffer = Buffer.buffer();
netSocketHandler.result().handler(buff -> buffer.appendBuffer(buff)) // read data
.endHandler(endHandler -> {
// no more data to read
future.complete(buffer);
netSocketHandler.result().close(); // close the NetSocket once finished
})
.exceptionHandler(err -> {
netSocketHandler.result().close();
future.fail(err);
}); // handle exceptions
}
else {
future.fail(netSocketHandler.cause());
}
});
}, result -> {
if(result.succeeded()) {
message.reply(result.result()); // reply with the buffer content
}
else {
message.reply(result.cause()); // may want to send error later
}
});
});
If this realy does not help you, I'm sorry, and maybe this is not the concept you are looking for.
I am writing some kind of integration test on my REST controller using MockRestServiceServer to mock backend behaviour.
What I am trying to achieve now is to simulate very slow response from backend which would finally lead to timeout in my application. It seems that it can be implemented with WireMock but at the moment I would like to stick to MockRestServiceServer.
I am creating server like this:
myMock = MockRestServiceServer.createServer(asyncRestTemplate);
And then I'm mocking my backend behaviour like:
myMock.expect(requestTo("http://myfakeurl.blabla"))
.andExpect(method(HttpMethod.GET))
.andRespond(withSuccess(myJsonResponse, MediaType.APPLICATION_JSON));
Is it possible to add some kind of a delay or timeout or other kind of latency to the response (or maybe whole mocked server or even my asyncRestTemplate)? Or should I just switch to WireMock or maybe Restito?
You can implement this test functionality this way (Java 8):
myMock
.expect(requestTo("http://myfakeurl.blabla"))
.andExpect(method(HttpMethod.GET))
.andRespond(request -> {
try {
Thread.sleep(TimeUnit.SECONDS.toMillis(1));
} catch (InterruptedException ignored) {}
return new MockClientHttpResponse(myJsonResponse, HttpStatus.OK);
});
But, I should warn you, that since MockRestServiceServer simply replaces RestTemplate requestFactory any requestFactory settings you'd make will be lost in test environment.
If you control timeout in your http client and use for example 1 seconds you can use mock server delay
new MockServerClient("localhost", 1080)
.when(
request()
.withPath("/some/path")
)
.respond(
response()
.withBody("some_response_body")
.withDelay(TimeUnit.SECONDS, 10)
);
If you want to drop connection in Mock Server use mock server error action
new MockServerClient("localhost", 1080)
.when(
request()
.withPath("/some/path")
)
.error(
error()
.withDropConnection(true)
);
Approach that you can go for:
Specifying the responsebody either with Class Path resource or normal string content. More detailed version of what Skeeve suggested above
.andRespond(request -> {
try {
Thread.sleep(TimeUnit.SECONDS.toMillis(5)); // Delay
} catch (InterruptedException ignored) {}
return withStatus(OK).body(responseBody).contentType(MediaType.APPLICATION_JSON).createResponse(request);
});
In Restito, there is a buil-in function to simulate timeout:
import static com.xebialabs.restito.semantics.Action.delay
whenHttp(server).
match(get("/something")).
then(delay(201), stringContent("{}"))
In general, you can define your custom request handler, and do a nasty Thread.sleep() there.
This would be possible in Restito with something like this.
Action waitSomeTime = Action.custom(input -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return input;
});
whenHttp(server).match(get("/asd"))
.then(waitSomeTime, ok(), stringContent("Hello World"))
Not sure about Spring, however. You can easily try. Check DefaultResponseCreator for inspiration.