We Keep Coding

Custom-Webclient Spock test throws unwanted NullPointerException in GET call

Using the custom WebClient below:
#Slf4j
#RequiredArgsConstructor
#Component
public class TransitApiClient {
private final TransitApiClientProperties transitApiClientProperties;
private final WebClient transitApiWebClient;
private final OAuth2CustomClient oAuth2CustomClient;
public ResponseEntity<Void> isOfficeOfTransitValidAndNational(String officeId){
try {
final String url = UriComponentsBuilder.fromUriString(transitApiClientProperties.getFindOfficeOfTransit())
.queryParam("codelistKey", "CL173")
.queryParam("itemCode", officeId)
.build()
.toUriString();
return transitApiWebClient.get()
.uri(url)
.header(AUTHORIZATION, getAccessTokenHeaderValue(oAuth2CustomClient.getJwtToken()))
.retrieve()
.onStatus(status -> status.value() == HttpStatus.NO_CONTENT.value(),
clientResponse -> Mono.error( new InvalidOfficeException(null,
"Invalid Office exception occurred while invoking :" + transitApiClientProperties.getFindOfficeOfTransit() + officeId)))
.toBodilessEntity()
.block();
} catch (WebClientResponseException webClientResponseException) {
log.error("Technical exception occurred while invoking :" + transitApiClientProperties.getFindOfficeOfTransit(), webClientResponseException);
throw new TechnicalErrorException(null, "Technical exception occurred while trying to find " + transitApiClientProperties.getFindOfficeOfTransit(), webClientResponseException);
}
}
with its intended usage to hit an endpoint, and check if it returns a 200 code with a body or 204 NoContent code, and react accordingly with some custom exceptions.
I've implemented the groovy-spock test below :
class TransitApiClientSpec extends Specification {
private WebClient transitApiWebClient
private TransitApiClient transitApiClient
private OAuth2CustomClient oAuth2CustomClient
private TransitApiClientProperties transitApiClientProperties
private RequestBodyUriSpec requestBodyUriSpec
private RequestHeadersSpec requestHeadersSpec
private RequestBodySpec requestBodySpec
private ResponseSpec responseSpec
private RequestHeadersUriSpec requestHeadersUriSpec
def setup() {
transitApiClientProperties = new TransitApiClientProperties()
transitApiClientProperties.setServiceUrl("https://test-url")
transitApiClientProperties.setFindOfficeOfTransit("/transit?")
transitApiClientProperties.setUsername("username")
transitApiClientProperties.setPassword("password")
transitApiClientProperties.setAuthorizationGrantType("grantType")
transitApiClientProperties.setClientId("clientId")
transitApiClientProperties.setClientSecret("clientSecret")
oAuth2CustomClient = Stub(OAuth2CustomClient)
oAuth2CustomClient.getJwtToken() >> "token"
transitApiWebClient = Mock(WebClient)
requestHeadersSpec = Mock(RequestHeadersSpec)
responseSpec = Mock(ResponseSpec)
requestHeadersUriSpec = Mock(RequestHeadersUriSpec)
transitApiClient = new TransitApiClient(transitApiClientProperties, transitApiWebClient, oAuth2CustomClient)
}
def "request validation of OoTra and throw InvalidOfficeException"(){
given :
def officeId = "testId"
def uri = UriComponentsBuilder
.fromUriString(transitApiClientProperties.getFindOfficeOfTransit())
.queryParam("codelistKey", "CL173")
.queryParam("itemCode", officeId)
.build()
.toUriString()
1 * transitApiWebClient.get() >> requestHeadersUriSpec
1 * requestHeadersUriSpec.uri(uri) >> requestHeadersSpec
1 * requestHeadersSpec.header(HttpHeaders.AUTHORIZATION, "Bearer token") >> requestHeadersSpec
1 * requestHeadersSpec.retrieve() >> responseSpec
1 * responseSpec.onStatus() >> Mono.error( new InvalidOfficeException(null,null) )
when :
def response = transitApiClient.isOfficeOfTransitValidAndNational(officeId)
then :
thrown(InvalidOfficeException)
}
But instead of an InvalidOfficeException being thrown, a java.lang.NullPointerException is thrown.
It seems to be triggered when during the test run, the program calls the following :
return transitApiWebClient.get()
.uri(url)
.header(AUTHORIZATION, getAccessTokenHeaderValue(oAuth2CustomClient.getJwtToken()))
.retrieve()
.onStatus(status -> status.value() == HttpStatus.NO_CONTENT.value(),
clientResponse -> Mono.error( new InvalidOfficeException(null,
"Invalid Office exception occurred while invoking :" + transitApiClientProperties.getFindOfficeOfTransit() + officeId)))
.toBodilessEntity() <---------------------- **HERE**
.block();
I understand that I haven't mocked its behavior but seems to me that some other mock hasn't been done correctly.

I can only recommend not to mock WebClient calls, as the necessary steps are a pain to mock, as you have seen yourself, requiring a lot of intermediary mocks without actually adding much value. This basically repeats the implementation, thus locking it in, which is not a good thing.
What I usually do is to extract all code that interacts with WebClient into a client class, and only mock this class interactions in my code. From the looks of it this is what you are already doing with TransitApiClient. For these client classes, I would recommend testing them with MockServer, WireMock, or any of the other frameworks. This way you actually make sure that the correct request/responses are sent/received, and you don't have to awkwardly deal with the WebClient interface.

How can I increase the in memory session store to maximum

Please note this is not about concurrent user session. This is about the total sessions can be stored in the in memory.
Here is the log :
java.lang.IllegalStateException: Max sessions limit reached: 10000
at org.springframework.web.server.session.InMemoryWebSessionStore$InMemoryWebSession.checkMaxSessionsLimit(InMemoryWebSessionStore.java:276)
at org.springframework.web.server.session.InMemoryWebSessionStore$InMemoryWebSession.save(InMemoryWebSessionStore.java:251)
at org.springframework.web.server.session.DefaultWebSessionManager.save(DefaultWebSessionManager.java:123)
at org.springframework.web.server.session.DefaultWebSessionManager.lambda$null$0(DefaultWebSessionManager.java:88)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:272)
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:230)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:236)
You can find the spring doc here https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/server/session/InMemoryWebSessionStore.html
I created a bean like below but not sure about this :
#Bean
public InMemoryWebSessionStore inMemoryWebSessionStore() {
InMemoryWebSessionStore inMemoryWebSessionStore = new InMemoryWebSessionStore();
inMemoryWebSessionStore.setMaxSessions(-1);
return inMemoryWebSessionStore;
}

I finally found a solution that worked for me.
#Bean
public WebSessionManager webSessionManager () {
DefaultWebSessionManager webSessionManager = new DefaultWebSessionManager();
InMemoryWebSessionStore inMemoryWebSessionStore = new InMemoryWebSessionStore();
inMemoryWebSessionStore.setMaxSessions(2147483647); //2147483647 - MAX_VALUE of Integer
webSessionManager.setSessionStore(inMemoryWebSessionStore);
return webSessionManager;
}

Sending output of one websocket client as input to another

Quick search on SO failed to find me a similar question so here we go
I basically want RSocket's requestChannel syntax with Webflux so I am able to process the received Flux outside of WebSocketClient.execute() method and write something like this (with session being opened only when the returned flux is subscribed to, proper error propagation, automatic completion and closing of the WS session when both inbound and outbound fluxes are complete -
either completed by the server side or cancelled by the consumer)
service /f wraps its received string messages in 'f(...)': 'str' -> 'f(str)'
service /g does the same with 'g(...)' and the following test passes:
private final DataBufferFactory dataBufferFactory = new DefaultDataBufferFactory();
private WebSocketMessage serializeString(final String text) {
return new WebSocketMessage(Type.TEXT, dataBufferFactory.wrap(text.getBytes(StandardCharsets.UTF_8)));
}
#Test
void test() {
var requests = 5;
var input = Flux.range(0, requests).map(String::valueOf);
var wsClient = new ReactorNettyWebSocketClient(
HttpClient.from(TcpClient.create(ConnectionProvider.newConnection())));
var f = requestChannel(wsClient, fUri, input.map(this::serializeString))
.map(WebSocketMessage::getPayloadAsText);
var g = requestChannel(wsClient, gUri, f.map(this::serializeString))
.map(WebSocketMessage::getPayloadAsText);
var responses = g.take(requests);
var expectedResponses = Stream.range(0, requests)
.map(i -> "g(f(" + i + "))")
.toJavaArray(String[]::new);
StepVerifier.create(responses)
.expectSubscription()
.expectNext(expectedResponses)
.verifyComplete();
}

And this seems to work for me... so far
public static Flux<WebSocketMessage> requestChannel(
WebSocketClient wsClient, URI uri, Flux<WebSocketMessage> outbound) {
CompletableFuture<Flux<WebSocketMessage>> recvFuture = new CompletableFuture<>();
CompletableFuture<Integer> consumerDoneCallback = new CompletableFuture<>();
var executeMono = wsClient.execute(uri,
wss -> {
recvFuture.complete(wss.receive().log("requestChannel.receive " + uri, Level.FINE));
return wss.send(outbound)
.and(Mono.fromFuture(consumerDoneCallback));
}).log("requestChannel.execute " + uri, Level.FINE);
return Mono.fromFuture(recvFuture)
.flatMapMany(recv -> recv.doOnComplete(() -> consumerDoneCallback.complete(1)))
.mergeWith(executeMono.cast(WebSocketMessage.class));
}
Rather interested if there're any flaws with this solution I haven't stumbled upon yet

Combining many ReactiveX streams into one result stream

I am trying to understand ReactiveX using RxJava but I can't get the whole Reactive idea. My case is the following:
I have Task class. It has perform() method which is executing an HTTP request and getting a response through executeRequest() method. The request may be executed many times (defined number of repetitions). I want to grab all the results of executeRequest() and combine them into Flowable data stream so I can return this Flowable in perform() method. So in the end I want my method to return all results of the requests that my Task executed.
executeRequest() returns Single because it executes only one request and may provide only one response or not at all (in case of timeout).
In perform() I create Flowable range of numbers for each repetition. Subscribed to this Flowable I execute a request per repetition. I additionally subscribe to each response Single for logging and gathering responses into a collection for later. So now I have a set of Singles, how can I merge them into Flowable to return it in perform()? I tried to mess around with operators like merge() but I don't understand its parameters types.
I've read some guides on the web but they all are very general or don't provide examples according to my case.
public Flowable<HttpClientResponse> perform() {
Long startTime = System.currentTimeMillis();
List<HttpClientResponse> responses = new ArrayList<>();
List<Long> failedRepetitionNumbers = new ArrayList<>();
Flowable.rangeLong(0, repetitions)
.subscribe(repetition -> {
logger.debug("Performing repetition {} of {}", repetition + 1, repetitions);
Long currentTime = System.currentTimeMillis();
if (durationCap == 0 || currentTime - startTime < durationCap) {
Single<HttpClientResponse> response = executeRequest(method, url, headers, body);
response.subscribe(successResult -> {
logger.info("Received response with code {} in the {}. repetition.", successResult
.statusCode(), repetition + 1);
responses.add(successResult);
},
error -> {
logger.error("Failed to receive response from {}.", url);
failedRepetitionNumbers.add(repetition);
});
waitInterval(minInterval, maxInterval);
} else {
logger.info("Reached duration cap of {}ms for task {}.", durationCap, this);
}
});
return Flowable.merge(???);
}
And executeRequest()
private Single<HttpClientResponse> executeRequest(HttpMethod method, String url, LinkedMultiValueMap<String, String>
headers, JsonNode body) {
CompletableFuture<HttpClientResponse> responseFuture = new CompletableFuture<>();
HttpClient client = vertx.createHttpClient();
HttpClientRequest request = client.request(method, url, responseFuture::complete);
headers.forEach(request::putHeader);
request.write(body.toString());
request.setTimeout(timeout);
request.end();
return Single.fromFuture(responseFuture);
}

Instead of subscribing to each observable(each HTTP request) within your perform method, Just keep on chaining the observables like this. Your code can be reduced to something like.
public Flowable<HttpClientResponse> perform() {
// Here return a flowable , which can emit n number of times. (where n = your number of HTTP requests)
return Flowable.rangeLong(0, repetitions) // start a counter
.doOnNext(repetition -> logger.debug("Performing repetition {} of {}", repetition + 1, repetitions)) // print the current count
.flatMap(count -> executeRequest(method, url, headers, body).toFlowable()) // get the executeRequest as Flowable
.timeout(durationCap, TimeUnit.MILLISECONDS); // apply a timeout policy
}
And finally, you can subscribe to the perform at the place where you actually need to execute all this, As shown below
perform()
.subscribeWith(new DisposableSubscriber<HttpClientResponse>() {
#Override
public void onNext(HttpClientResponse httpClientResponse) {
// onNext will be triggered each time, whenever a request has executed and ready with result
// if you had 5 HTTP request, this can trigger 5 times with each "httpClientResponse" (if all calls were success)
}
#Override
public void onError(Throwable t) {
// any error during the execution of these request,
// including a TimeoutException in case timeout happens in between
}
#Override
public void onComplete() {
// will be called finally if no errors happened and onNext delivered all the results
}
});

Spring DeferredResult timeout

I have an observable and as it may take a long time i return spring's DeferredResult.
This is the controller logic I'm using:
public DeferredResult<ResponseEntity<InputStreamResource>> getSomeFile() {
DeferredResult<ResponseEntity<InputStreamResource>> deferredResult = new DeferredResult<>(TIMEOUT);
Observable<File> observableFile = fileService.getSomeFile();
observableFile
.map(this::fileToInputStreamResource)
.map(resource -> ResponseEntity.ok().cacheControl(CacheControl.maxAge(1, TimeUnit.HOURS).cachePublic()).body(resource))
.subscribe(deferredResult::setResult, ex -> {
deferredResult.setErrorResult(ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(null));
});
return deferredResult;
}
And this is my test case:
#Test
public void getSomeFile() throws Exception {
this.mockMvc.perform(get("/somefile").accept(MediaType.parseMediaType("application/xml;charset=UTF-8")))
.andExpect(request().asyncStarted())
.andExpect(status().isOk())
.andDo(result -> {
ResponseEntity<InputStreamResource> fileResource = (ResponseEntity<InputStreamResource>) result.getAsyncResult();
InputStream fileResourceInputStream = fileResource.getBody().getInputStream();
FooBar foobar = (FooBar) jaxb2Marshaller.unmarshal(new StreamSource(fileResourceInputStream));
assertThat(foobar.getFoos(), is(not(empty())));
});
}
This test fails when I instantiate DeferredResult without timeout set as it uses generic timeout that is set for rest template (10 sec). If i explicitly set the timeout new DefferredResult(Long.MAX_VALUE) it fails with same exception:
java.lang.IllegalStateException: Async result for handler [public org.springframework.web.context.request.async.DeferredResult<org.springframework.http.ResponseEntity<org.springframework.core.io.InputStreamResource>> com.example.controller.FileController.getSomeFile()] was not set during the specified timeToWait=9223372036854775807
at org.springframework.test.web.servlet.DefaultMvcResult.getAsyncResult(DefaultMvcResult.java:145)
at org.springframework.test.web.servlet.DefaultMvcResult.getAsyncResult(DefaultMvcResult.java:121)
at org.springframework.test.web.servlet.result.RequestResultMatchers$4.match(RequestResultMatchers.java:114)
at org.springframework.test.web.servlet.MockMvc$1.andExpect(MockMvc.java:171)
So how timeout should be configured or where's a problem ?