I'm trying do some logging in my middleware application, where I want to log request and response of backend system. I would like to use Netty (because we have implemented difficult SslContext).
I'm trying to learn it by this tutorial: https://www.baeldung.com/spring-log-webclient-calls (chapter 4.2. Logging with Netty HttpClient).
For better looking logs, there is part of code:
HttpClient httpClient = HttpClient
.create()
.tcpConfiguration(
tc -> tc.bootstrap(
b -> BootstrapHandlers.updateLogSupport(b, new CustomLogger(HttpClient.class))))
.build()
but bootstrap(...) method is deprecated. What can I use instead?
If you need CustomLogger for request/response (as per comment above) then you can do the following:
CustomLogger customLogger = new CustomLogger(HttpClient.class);
HttpClient httpClient = HttpClient
.create()
.doOnRequest((request, connection) -> {
connection.addHandlerFirst(customLogger);
});
Related
Once I've created an Apache HttpClient, how do I get the configuration values from it (what used to be HttpParams)?
For example, if I do something like this to create a client:
RequestConfig requestConfig = RequestConfig.custom()
.setCircularRedirectsAllowed(true)
.setAuthenticationEnabled(true)
.setExpectContinueEnabled(true)
.setConnectionRequestTimeout(10000);
.build();
HttpClient client = HttpClientBuilder
.create()
.setConnectionManager(new PoolingHttpClientConnectionManager(createSchemeRegistry()))
.setDefaultRequestConfig(requestConfig)
.build();
How would I get the value of the connection request timeout from the org.apache.http.client.HttpClient instance? other configuration items?
For AbstractHttpClient there was a method getParams (or similar) but that's all gone away in favour of the HttpClient interface. There's an InternalHttpClient instance that's generated along the way but I think that's not exposed.
The reason I'm interested in this is firstly for testing - I want to verify a HTTP Client was configured correctly, the other case is to present that information in a UI when debugging HTTP traffic - think of something like Charles proxy.
Does anyone know if there's a way to disable cookie management in Spring WebClient using Reactor Netty HttpClient?
I noticed both WebClient.Builder and HttpClient APIs provide a means to add cookies to an outbound request but I am looking for a way to inhibit them altogether if such exists. That is akin to disableCookieManagement on Apache's HttpComponentClientBuilder.
It looks like there's no way to disable the cookie handling per se but this seems to work: Create your own HttpClient, then use HttpClient.doOnRequest to register a callback to be called before sending the request. In the callback, call HttpClientRequest.requestHeaders() to retrieve the request headers, then delete the Cookie header.
Sample code that removes User-Agent header before sending the request.
public class Main {
public static void main(String[] args) {
HttpClient httpClient = HttpClient.create().doOnRequest((request, connection) -> {
request.requestHeaders().remove("User-Agent");
});
WebClient client = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
Mono<String> r = client.get().uri("https://www.google.com").retrieve()
.bodyToMono(String.class);
System.out.println(r.block());
}
}
In my testing, I'm not seeing that Reactor Netty HttpClient is doing anything at all with the cookies that come back in the response. I was actually looking to enable a cookie store like the Apache HttpClient supports, but to all indications the Netty HttpClient has no such feature.
In the entire codebase of Reactor Netty, there is no usage of the "set-cookie" header except in one of their unit tests. This tells me cookies are ignored by default unless the client code chooses to look for them.
When I use a SimpleRequestFactory with my AsyncRestTemplate I can easily configure an HTTP proxy server.
I can either do (sample code in Kotlin):
#Bean
open fun asyncRestTemplate(): AsyncRestTemplate {
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("127.0.0.1", 8008))
val requestFactory = SimpleClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setProxy(proxy)
this.setTaskExecutor(taskExecutor())
}
return AsyncRestTemplate(requestFactory)
}
Or I can simply set the corresponding system properties: -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8008.
However, in the moment that I switch from the SimpleClientHttpRequestFactory to a Netty4ClientHttpRequestFactory there is no evident way to configure the proxy directly and it seems this client does not respect the system properties either.
val requestFactory = Netty4ClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
//this.setProxy(proxy) //???
}
Once I change for the netty client, I have no clue on how to make it go through the proxy.
My interest in using the netty client was that I not only wanted to make async requests, but also I wanted this to be non-blocking. I hope I'm not making a wrong assumption here.
Does anyone know how can I use a proxy server when using the Netty4ClientHttpRequestFactory or perhaps know of an alternative non-blocking client supported by Spring that I could use?
The Netty4ClientHttpRequestFactory (source) and related classes such as Netty4ClientHttpRequest (source) use SimpleChannelInboundHandler for the channel and do not use the proxy handler. Everything is private and unable to be overridden within the source, so there is no way to change it to support Proxies. You would have to almost rewrite the whole thing.
You have other async client options that will work very well and allow you more configuration options. The included Netty one is fairly basic anyway. OkHttpClientHttpRequestFactory and HttpComponentsAsyncClientHttpRequestFactory both let you pass in your own configured client.
To your interest, AsyncRestTemplate's different implementation:
SimpleClientHttpRequestFactory -> simple thread pool, blocking api, proxy supported
OkHttpClient (OkHttp3) -> blocking api, proxy supported
CloseableHttpAsyncClient -> non-blocking nio api, proxy supported
Netty4ClientHttpRequestFactory -> non-blocking nio api, proxy not supported
you can visit https://github.com/wuxudong/VariousAsyncHttpClientPerformance for more details
Is it possible to add/remove Authenticators and/or Interceptors to an existing Okhttp instance? If yes, how?
No, it is not possible.
However, you can create a builder from an existing client, and make changes to that. This will share the dispatcher, connectionPool etc.
OkHttpClient.Builder clientBuilder = client1.newBuilder();
clientBuilder.networkInterceptors().add(0, serviceInterceptor);
OkHttpClient client2 = clientBuilder.build();
There is an example for adjusting the timeout of a client in the javadoc https://square.github.io/okhttp/3.x/okhttp/okhttp3/OkHttpClient.html
In httpclient 4.3, if you respect all the deprecations, you must build and configure your HttpClient using an HttpClientBuilder (documentation here). The methods are explicit, they seem easy to use, and HttpClient's interface is clear. But maybe a tad too much.
In my own case, I have to inherit Spring's HttpComponentsHttpInvokerRequestExecutor (documentation here). As a consequence, I can easily get the HttpClient, but all I know about this object is that it implements the interface.
Since the client is already built, and I do not know anything about its implementation, I cannot access methods such as AbstractHttpClient's setHttpRequestRetryHandler or addRequestInterceptor (though yes, I know, they are deprecated).
So, what would be the cleanest way of updating the settings of this HttpClient (the retry handler and request interceptor are those I am most concerned with at the moment)? Should I...
... savagely cast my client to AbstractHttpClient, hoping I will always receive this implementation?
... create a new HttpClient in my HttpInvokerRequestExecutor's constructor, and get something like the example reproduced below? I might add that Spring's constructor (in 3.2.4 at least) uses methods deprecated in httpclient 4.3 too. Would there be any side effect I missed using this strategy?
... do something I have not proposed yet?
Custom constructor example:
public CustomHttpInvokerRequestExecutor() {
super(); // will create an HttpClient
// Now overwrite the client the super constructor created
setHttpClient(HttpClientBuilder.custom(). ... .build());
}
do something I have not proposed yet?
My recommendation would be to re-think the entire approach. One should not be removing / adding protocol interceptors at runtime but rather should be making use of HttpContext instance to update request execution context and pass configuration to protocol interceptors
http://hc.apache.org/httpcomponents-core-4.4.x/tutorial/html/fundamentals.html#d5e306
http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/fundamentals.html#d5e223
CloseableHttpClient client = HttpClientBuilder.create()
.addInterceptorFirst(new HttpRequestInterceptor() {
#Override
public void process(
final HttpRequest request,
final HttpContext context) throws HttpException, IOException {
boolean b = (Boolean) context.getAttribute("my-config");
if (b) {
// do something useful
}
}
})
.build();
HttpClientContext context = HttpClientContext.create();
context.setAttribute("my-config", Boolean.TRUE);
CloseableHttpResponse response1 = client.execute(new HttpGet("/"), context);
response1.close();
context.setAttribute("my-config", Boolean.FALSE);
CloseableHttpResponse response2 = client.execute(new HttpGet("/"), context);
response2.close();