I have a reactive core WebClient to post to a given endpoint. The payload is a flux of JsonNodes and the content-type is application/stream+json
JsonNode response = localEP.post().uri( "/createItem" )
.contentType(MediaType.APPLICATION_STREAM_JSON)
.body( BodyInserters.fromPublisher(itemData, JsonNode.class ))
.retrieve()
.bodyToMono( JsonNode.class )
.block();
On the server end I have tried both a Spring Controller style and Spring Web Reactive FunctionHandler to process the payload of the above call with a payload that is a Flux.
#PostMapping(path = "/dev/jobad/dynamo", consumes = MediaType.APPLICATION_STREAM_JSON_VALUE)
#ResponseStatus(HttpStatus.CREATED)
public Flux<JsonNode> loadItems (#RequestBody Flux<JsonNode> items) {
items.subscribe(storage::add);
JsonNode response = new ObjectMapper().createObjectNode().put( "shady", "shade" );
return Flux.just( response );
}
The return to the client is always ok, however the server reports that the content of the flux is empty. If I change (#RequestBody Flux<JsonNode> items to (#RequestBody JsonNode items The payload is received fine. The WebClient logs appear to indicate that it has written the data on the wire and processed the response. However the body seems to empty
Reactor.core.Exceptions$ErrorCallbackNotImplemented: org.springframework.web.server.ServerWebInputException: Response status 400 with reason "Request body is missing: public reactor.core.publisher.Flux<com.fasterxml.jackson.databind.JsonNode> com.talroo.rest.JobResource.loadJobs(reactor.core.publisher.Flux<com.fasterxml.jackson.databind.JsonNode>)"
Caused by: org.springframework.web.server.ServerWebInputException:
Response status 400 with reason "Request body is missing: public
reactor.core.publisher.Flux<com.fasterxml.jackson.databind.JsonNode>
What do I need to do to be able to handle the request body of a post as a Flux?
First, I don't think Spring officially supports reading/writing Jackson JsonNode instances directly from Controllers. Your application is supposed to ask for a domain object or something like a Map<String, String>.
Now in Jackson's model, a JsonNode represents any node in the JSON tree - as it is a tree, you can expect to get a Flux of nodes, but you are apparently able to get the root node - which explains the behavior you're seeing.
So I think your application should rather rely on higher-level classes and let Jackson deserialize them for you.
Note that your controller implementation is also breaking a few rules:
you should not call blocking operators, such as block, within a method that returns a reactive type (your controller is not breaking this one, but close)
you should not break the reactive pipeline and decouple the reading of the request and the writing of the response; chances are the HTTP exchange could be closed before your controller has a chance to read the whole request. Calling subscribe just does that.
Related
I have a service which calls a dozen other services. This reads from a Kafka topic using a #StreamListener in a controller class. For traceability purposes, the same headers(original request ID) from the Kafka message need to be forwarded to all the other services as well
Traditionally, with a #PostMapping("/path") or GetMapping, a request context is generated, and one can access the headers from anywhere using RequestContextHolder.currentRequestAttributes() and I would just pass the HttpHeaders object into a RequestEntity whenever I need to make an external call
However in a StreamListener, no request context is generated and trying to access the RequestContextHolder results in an exception
Here's an example of what I tried to do, which resulted in an exception:
public class Controller {
#Autowired Service1 service1
#Autowired Service2 service2
#StreamListener("stream")
public void processMessage(Model model) {
service1.execute(model);
service2.execute(model);
}
}
public class Service {
RestTemplate restTemplate;
public void execute(Model model){
// Do some stuff
HttpHeaders httpHeaders = RequestContextHolder.currentRequestAttributes().someCodeToGetHttpHeaders();
HttpEntity<Model> request = new HttpEntity(model, httpHeaders);
restTemplate.exchange(url, HttpMethod.POST, request, String.class);
}
}
My current workaround is to change the StreamListener to a PostMapping and have another PostMapping which calls that so a request context can be generated. Another option was to use a ThreadLocal but it seems just as janky
I'm aware of the #Headers MessageHeaders annotation to access the stream headers, however, this isn't accessible easily without passing the headers down to each and every service and would affect many unit tests
Ideally, I need a way to create my own request context (or whatever the proper terminology is) to have a place to store request scoped objects (the HttpHeader) or another thread safe way to have request headers passed down the stack without adding a request argument to service.execute
I've found a solution and am leaving it here for anyone else trying to achieve something similar
If your goal is to forward a bunch of headers end-to-end through REST controllers and Stream listeners, you might want to consider using Spring Cloud Sleuth
Add it to your project through your maven or gradle configuration:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
Specifically, in Spring Cloud Sleuth there is a feature to forward headers or "baggage" by setting the property spring.sleuth.propagation-keys in your application.properties. These key-value pairs are persisted through the entire trace, including any downstream http or stream calls which also implement the same propagation keys
If these fields need to be accessed on a code level, you can get and set them using the ExtraFieldPropagation static functions:
ExtraFieldPropagation.set("country-code", "FO"); // Set
String countryCode = ExtraFieldPropagation.get("country-code"); // Get
Note that the ExtraFieldPropagation setter cannot set a property not present in the defined spring.sleuth.propagation-keys so arbitrary keys won't be accepted
You can read up on the documentation for more information
I have a webservice which calls another WS and returns the response from the second WS. It looks like so:
// MyController
public ResponseEntity<Foo> requestFooController(#RequestBody #Valid Bar request) {
return this.myService.requestFooService(request);
}
//MyService
ResponseEntity<Foo> requestFooService(Bar request) {
Buzz improvedRequest = ...
return this.secondWS.secondRequestFoo(improvedRequest);
}
When I call the API through Postman, I receive a HTTP OK response with an empty body. Yet, when I'm in debug mode I can see that the service is returning a ResponseEntity with a body. The headers are not lost though.
I changed my code like so and it works fine:
// MyController
public ResponseEntity<Foo> requestFooController(#RequestBody #Valid Bar request) {
ResponseEntity<Foo> tmp = this.myService.requestFooService(request);
return ResponseEntity.status(tmp.getStatusCode()).body(tmp.getBody());
}
Now through Postman I do have the expected body. However, I don't understand the behaviour. I thought that maybe it's due to the fact that the body is some kind of stream that can be read once or something similar. But from reading the source code I don't see anything that could explain this behaviour.
I'm using the Netflix-stack (so HTTP calls between the two WS are made through a Feign client).
Any idea why I'm getting this result?
EDIT:
More details on my stask:
SpringBoot 1.5.3.RELEASE
Feign 2.0.5
There is a bug that causes the named body of an HTTP MultiPart POST to fail. The symptom of this is that you make a POST request with a body, and Spring-Boot can't match it up to an endoint. The exception I see is:
2019-01-23 15:22:45.046 DEBUG 1639 --- [io-8080-exec-10] .w.s.m.m.a.ServletInvocableHandlerMethod : Failed to resolve argument 3 of type 'org.springframework.web.multipart.MultipartFile'
org.springframework.web.multipart.support.MissingServletRequestPartException: Required request part 'file' is not present
Zuul is doing caching of the request in order to re-try multiple times. In this process, it fails to preserve the named field for the binary body. You may find it working if you preface the request with zuul. So instead of http://myserver.com/myservice/endpoint use zuul in the path: http://myserver.com/zuul/myservice/endpoint
That will effectively avoid the saving of the request and the retry mechanism.
More details are available on this issue in Zuul's GitHub Bug List.
I have a question regarding Spring Webflux. I wanted to create a reactive endpoint that consumes content type text/event-stream. Not produce but consume. One of our services needs to send a lot of small objects to another one and we thought that streaming it this way might be a good solution.
#PostMapping(value = "/consumeStream", consumes = MediaType.TEXT_EVENT_STREAM_VALUE)
public Mono<Void> serve(#RequestBody Flux<String> data) {
return data.doOnNext(s -> System.out.println("MessageReceived")).then();
}
I am trying to use Spring WebClient to establish a connection to the endpoint and stream data to it. For example using code:
WebClient.builder().baseUrl("http://localhost:8080")
.clientConnector(new ReactorClientHttpConnector())
.build()
.post()
.uri("/test/serve")
.contentType(MediaType.TEXT_EVENT_STREAM)
.body(BodyInserters.fromPublisher(flux, String.class))
.exchange()
.block();
The flux is a stream that produces a single value every 1 sec.
The problem I have is that the WebClient fully reads the publisher and then sends the data as a whole and not streams it one by one.
Is there anything I can do to do this using this client or any other ? I do not want to go the websockets way.
SSE standard does not allow POST. There is no way to specify method even in browser API https://www.w3.org/TR/eventsource/
Server Side Events as name states are designed for delivering events from the server to the client.
I have a Spring REST application that accepts JSON messages, written like
#RequestMapping(value = "/myhook", method = RequestMethod.POST,
produces = JSON, consumes = JSON)
public #ResponseBody MyResponse doIt
(#Valid #RequestBody(required = true) MyContractRequest request) {
MyResponse response;
...
return response;
}
This works really well with almost no code to support, but now I have a requirement to sign both response and request.
I started from simply computing the shared signature of all message fields at Java level and assigning it to the dedicated signature field. However this requires to have and maintain code for computing the signatures:
public void update(java.security.Signature sign) throws Exception {
sign.update(name);
sign.update(value);
sign.update(etc);
}
Some people around me expressed opinion that the need to write and maintain this signing code may not be the best design, and it may be better to sign the whole message as a single JSON string. I could fetch the request as a string manually, and then process JSON manually, but I really would like to preserve the Spring controller concepts.
Also, I cannot longer have the signature field in the message itself because the value of this field obviously also changes the signature of the JSON string.
Is there any way to compute the signature of the whole JSON message body on the message departure and arrival, and where to place the signature so it could be passed together with the message? One of the idea is to use the custom HTTP header for the signature. Anyway, how to compute it first?
You can use a servlet filter with Spring MVC and modified your content whatever you want in request and response as well
Example :
http://www.mkyong.com/spring-mvc/how-to-register-a-servlet-filter-in-spring-mvc/
or you can use Spring 3 MVC Interceptor
http://viralpatel.net/blogs/spring-mvc-interceptor-example/
Follwing the examples from Krams. What does the below return, will the Person object be converted to JSON or XML based on the header of submitted request ?
#RequestMapping(value = "/person",
method = RequestMethod.POST,
headers="Accept=application/xml, application/json")
public #ResponseBody Person addPerson(#RequestBody Person person) {
logger.debug("Provider has received request to add new person");
// Call service to here
return personService.add(person);
}
So when I submit data as json I get json back, and the same for xml. Or is something else going on ?
It depends on Accept http request header. If it's json, you get json, if it's xml you get xml.
That's exactly what this part of your code says.:
....
headers="Accept=application/xml, application/json")
...
You can send one mime type and receive another without problems.
edit
Both headers and produces/consumes parameters only say what can be produced/consumed by the requestmapping. They don't force any particular serialization. The type of request/response is decided entirely in request headers. If the dispatcher doesn't find mapping with produces and consumes (or headers) matching the request headers you'll get an error.
The 'headers', 'produces', and 'consumes' parameters to #RequestMapping are one piece to the puzzle. They help the framework route incoming requests, and differentiate between requests based on values in the header.
The other part to this puzzle is the set of MessageConverters that are configured in the application. The controller handler method in the example just returns an object, and the framework needs to know how to convert the object to some text representation (i.e. XML or JSON). MessageConverters are used on the inbound side (on parameters annotated with #RequestBody) and on the outbound side (when the method return value is annotated with #ResponseBody). The appropriate MessageConverter is chosen by the framework based on the media type of the incoming request.
When using the <mvc:annotation-driven> namespace in a Spring XML config file, or when using the #EnableMvc annotation on a Java Config class, MessageConverters supporting JSON and XML and configured automatically.