I call below code within my application. The first request is always working fine. My issue is that every following request is not sent, it runs into timeout, when I specify a timeout value. Otherwise it seems to wait endlessly. It seems the first request blocks the connection for every following attempt. How can I ensure the connection is properly released again? Maybe some headers? Maybe some properties (defaults are used for http https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/doc-files/net-properties.html)?
HttpRequest request = HttpRequest.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.GET()
.uri(URI.create(url))
.build();
try {
HttpResponse<Path> response = client.send(request, HttpResponse.BodyHandlers.ofFile(Paths.get(outfile)));
} catch (IOException | InterruptedException e) {
// ...
}
used: java.net.http.HttpClient (AdoptOpenJDK 11.0.7_10)
The send method is a Sync method, so the request get block until you get a response, in this case maybe you are not getting the response.
send(HttpRequest, BodyHandler) blocks until the request has been sent and the response has been received.
Try using the Async method
sendAsync(HttpRequest, BodyHandler) sends the request and receives the response asynchronously.
The sendAsync method returns immediately with a CompletableFuture. The CompletableFuture completes when the response becomes available. The returned CompletableFuture can be combined in different ways to declare dependencies among several asynchronous tasks.
Example of an async request (taken from the apidoc):
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://example.com/"))
.timeout(Duration.ofMinutes(2))
.header("Content-Type", "application/json")
.POST(BodyPublishers.ofFile(Paths.get("file.json")))
.build();
client.sendAsync(request, BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.thenAccept(System.out::println);
Related
I developed a Java library for Twitter API here using OkHttp3 4.8.1.
Unfortunately, it looks like after having sent a request, once everything is finished, the program never stops and is stuck in SocketInputStream.
When not using cache, it is stuck in waitForReferencePendingList method of Reference class instead :
I tried everything, closing connection explicitly in my code like this, updating the version of OkHttp, but still the same. Any idea ?
If needed, here is the full code where the request is done, in summary :
Request request = new Request.Builder()
.url(url)
.get()
.headers(Headers.of("Authorization", "Bearer " + bearerToken))
.build();
OkHttpClient client = new OkHttpClient.Builder().build()
Response response = client.newCall(request).execute();
String stringResponse = response.body().string();
return Optional.ofNullable(TwitterClient.OBJECT_MAPPER.readValue(stringResponse, classType));
Finally adding client.connectionPool().evictAll(); elsewhere (in my post request to get a bearer token) solved the problem !
how to re-use webclient client response? I am using webclient for synchronous request and response. I am new to webclient and not sure how to extract response body in multiple places
WebClient webClient = WebClient.builder().baseUrl("http://localhost:8080").build();
below is my call to API which returns valid response
ClientResponse clientResponse;
clientResponse = webClient.get()
.uri("/api/v1/data")
.accept(MediaType.APPLICATION_JSON)
.exchange()
.block();
How to use clientResponse in multiple places? only one time I am able to extract response body
String response = clientResponse.bodyToMono(String.class).block(); // response has value
When I try to extract the response body second time (in a different class), it's null
String response = clientResponse.bodyToMono(String.class).block(); // response is null
So, can someone explain why response is null second time and how to extract the response body multiple times?
WebClient is based on Reactor-netty and the buffer received is one time thing.
One thing you could do is to cache the result at the first time and then reuse it.
You can refer to this issue in spring cloud gateway: https://github.com/spring-cloud/spring-cloud-gateway/issues/1861
Or refer to what Spring Cloud gateway do for caching request body: https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/AdaptCachedBodyGlobalFilter.java
Or you can write your code like:
String block = clientResponse.bodyToMono(String.class).block();
And next time you can use this body:
Mono.just(block);
I'm trying to make a POST call to an api through WebClient without a request body (the api does not need a body):
webClient.post()
.uri("url")
.retrieve()
.bodyToMono(CustomResponse.class);
But the call returns an exception:
The underlying HTTP client completed without emitting a
response.java.lang.IllegalStateException: The underlying HTTP client
completed without emitting a response.
For another API which requires a request body, I can successfully make a POST call with no issue:
webClient.post()
.uri("url")
.bodyValue("value")
.retrieve()
.bodyToMono(CustomResponse.class);
Is the problem related to the WebClient’s request body? How do I fix it?
UPDATE
The error happens because I added a ContentType header for Webclient through .defaultHeader("ContentType", JSON).
The problem has gone after I've removed the header.
It looks like the downstream webservice responding too slow and exceeding connection timeout hence you're getting this issue.
In case you don't need (or response doesn't have body) you can use .toBodilessEntity() instead. So you code snippet can be rewritten:
webClient.post()
.uri("url")
.retrieve()
.toBodilessEntity();
Why should you use it? Because you're always responsible for resources release. Please, read NOTE section.
When using exchange(), it is the responsibility of the application to
consume any response content regardless of the scenario (success,
error, unexpected data, etc). Not doing so can cause a memory leak.
See ClientResponse for a list of all the available options for
consuming the body.
I am seeing some strange behaviour with the following client-server interaction and I am wondering whether I am running into HTTP/1.1 semantics or my Reactive programming skills need work (or both).
I am attempting to create a client-server interaction where both the request and response bodies are long-running streams of data.
The client is a Spring Reactive WebClient that sends an infinite stream in the request body. It expects to receive (and log) an infinite stream of results.
Flux<Long> requests = Flux.interval(Duration.ofSeconds(2));
return WebClient.create()
.post()
.uri("/instructions")
.contentType(MediaType.APPLICATION_STREAM_JSON)
.body(requests, Long.class)
.retrieve()
.bodyToFlux(Object.class)
.map(response -> {
log.info("Received Response Object {}", response);
return response;
});
The server is a spring-boot-starter-webflux application with a route handler to log the request objects as they are received and provide an infinite stream of results:
public Mono<ServerResponse> instructions(ServerRequest request) {
// Log the request objects as they are received
Flux<Object> requestStream = request.bodyToFlux(Object.class)
.map(r -> {
log.info("Received Request Object: {}", r);
return r;
});
requestStream.subscribe();
// Infinite stream of responses
Flux<Long> responses = Flux.interval(Duration.ofSeconds(5));
return ServerResponse.ok()
.contentType(MediaType.APPLICATION_STREAM_JSON)
.body(responses, Long.class);
}
When the above code is run, the server logs the infinite stream of request objects but the client never logs any response objects.
If I bound the request stream by doing something like so: Flux<Long> requests = Flux.interval(Duration.ofSeconds(2)).take(20); then the client begins to log the responses after all requests are received.
What are the problems here?
* Is there something wrong with the reactive code?
* Is this part of the HTTP/1.1 specification where a response header should not be sent until the request body is completely received?
See this question: WebClient doesn't read response until request write is completed.
Apparently, Netty-based WebClient starts processing the response only after it is finished sending the request
Netty server might also behave similarly (only start sending the response after it is finished reading the request body), I'm not sure
Interestingly, Jetty-based WebClient is able to process request and response simultaneously
I am using a HttpURLConnection instance in order to connect to a URL.
I am then calling the getResponseCode method in order to determine the connection state.
I am polling the connection with this method, until the returned response-code is NOT 1xx:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
int responseType = con.getResponseCode()/100;
while (responseType == 1)
{
Thread.sleep(10);
responseType = con.getResponseCode()/100;
}
switch (responseType)
{
...
}
What is the recommended time to sleep inside the loop? (here set to 10ms).
Is there any time period (min, max, average) defined in the HTTP standard?
The status code "100 continue" is usually sent by the server after it got the request header of the POST request, to signal the client that it can follow up with the POST body. This way the server can reject requests early based on the request header (e.g. authorization required) without wasting resources on the client. The final response will then be sent after the server received the request body from the client, which might be immediately or hours later, depending on how fast the clients sends the request body.