We Keep Coding

What is the right way to use HttpClientConnectionManager with CloseableHttpClient in Apache?

Context
I have an instance of org.apache.hc.client5.http.impl.classic.CloseableHttpClient
and a singleton instance of org.apache.hc.client5.http.io.HttpClientConnectionManager to communicate with some remote API.
I have created a singleton pool of HTTP connections to create an instance of the HTTP client.
What is the right way to use HttpClientConnectionManager?
Should I really use try-with-resource (or old-style try-finally) to work with CloseableHttpClient?
If I close the HTTP client, a connection will be closed also from this pool. And then I can't use this pool to communicate with a remote API.
Of course, I've read the documentation https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
Maybe something I missed. Could anyone explain me?

You should also be using CloseableHttpClient as a singleton (on a per distinct service basis).
If you want to continue creating short lived instances of CloseableHttpClient with the same HttpClientConnectionManager please make sure to mark it as shared when creating CloseableHttpClient instances with HttpClientBuilder.

How to configure and build a custom RestTemplate for each client in Spring?

I am using Spring RestTemplate for executing HTTP request from my application. There are several services we need to call, some on the internet and some on intranet, some fast and some slow. I have been instructed to configure custom settings for each service, basically, connection timeout, read timeout.
These setting will be extremely specific for example, the services hosted on intranet would have a timeout of ~2-5s while they provide an SLA of 1000ms for 99.9% of requests. While the other third party services with ~10-60s.
As these parameters can be set in the factory for the template only, i am creating a number of beans with different factories differing in timeouts only. Something like this:
#Bean
RestTemplate restTemplate() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory(httpClient);
factory.setReadTimeout(20000);
factory.setConnectTimeout(5000);
RestTemplate restTemplate = new RestTemplate(factory);
}
I am afraid this will create a maintenance nightmare eventually. Can it be solved in a better ways?
PS: The application is a monolith calling various services.

You will have to create multiple RestTemplates and assign timeouts, connection pool size. Connection pool will improve the performance drastically
I have hard-coded the connection properties, you can pick it from application.properties file
#Configuration
class RestTemplateConfigs {
#Bean
public HttpClient httpClient() {
return HttpClientBuilder.create()
.setMaxConnPerRoute(200)
.setMaxConnTotal(50)
.setConnectionTimeToLive(10L, TimeUnit.SECONDS)
.build();
}
#Bean(name = "restTemplate1")
RestTemplate restTemplate() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory(httpClient());
RestTemplate restTemplate = new RestTemplate(factory);
return restTemplate;
}
}
You can create multiple RestTemplates and Autowire it using Qualifier name.

Disclaimer: my answer suggests to work with another Http client than Rest Template - If you must use Rest template my answer would be irrelevant.
I dealt with a similar design issue and here is what I did. I wrote my own HttpClient class. It is much simpler in use then most known Http clients. This class could be used on its own or (and this is relevant for your case) it could be used as a parent class for a group of classes (implementing the same interface) where each class will be an Http client for specific Rest Service. On this class you can pre-set the target URL and all the parameters (such as read and connection timeouts etc). Once this class is preset, all you need to do is to invoke sendHttpRequestMethod(). Just to expand a bit - lets say you have a User Rest service with CRUD API implemented by particular URL calls with different HTTP methods and may be different URLs. (say in addition to create (POST) update (PUT) read (GET) and delete (DELETE) methods that are located at HTTP://www.myserver.com:8083/user say you will also have methods activate and deactivate (say both GET) at URLs HTTP://www.myserver.com:8083/user/activate/ and HTTP://www.myserver.com:8083/user/deactivate.
So, in this case, your Http client will set all required timeouts and other configurations and it will also have pre-set target URL HTTP://www.myserver.com:8083/user. and it will have six methods as mentioned above, where each one will simply invoke the parent class method sendHttpRequest(). Of course, for activate and deactivate methods you will need to append "activate" and "deactivate" suffixes for pre-set base URL. So, for each REST service, you can create a dedicated Http client with very minimal effort since the base class already does most of the job. In addition to it, I wrote a self-populating factory for any group of classes that implement the same interface. With that factory, all you will have to do is to write your additional Http client and the factory will detect it and will make it available on its own by predefined name or by the name of the class (based on your choice). All this worked so well for me, that I packaged it into Open Source library called MgntUtils and published it on Maven and Github (with source code and Javadoc. JavaDoc is available here). A detailed explanation on the Self-populating factory can be seen in Javadoc here. Also, the general article about the library can be found here and specific article about the idea and the implementation of self-populating Factory can be found here. Package com.mgnt.lifecycle.management.example in the source code contains a working example. I hope this helps.

Using parameterized construction of the RestTemplate bean solved my problem.
The bean is now configured as:
#Bean
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
public RestTemplate getRestTemplate(String configParam){
int connectionTimeout; //get from config using configParam
int readTimeout; //get from config using configParam
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setConnectTimeout(connectionTimeout);
factory.setReadTimeout(readTimeout);
return new RestTemplate(factory);
}
Instead of #Autowiring this field, it can be injected in may be #PostConstruct method like depicted below. The dependent bean can do following:
#Autowire
BeanFactory beanFactory;
RestTemplate restTemplate;
#PostConstruct
public void init(){
restTemplate = beanFactory.getBean(RestTemplate.class, configParam);
}
Here you can have your bean with custom settings injected in restTemplate.

Resteasy Client Register filter for all services

It`s possible to register one filter in RestEasy that will works for any instance? Today I have a lot of services thats use one client
for example:
Client client = ClientBuilder.newClient();
WebTarget target = client.target(ulrBase);
ResteasyWebTarget rtarget = (ResteasyWebTarget) target;
this.servico = rtarget.proxy(UsuarioServiceClient.class);
but now I have to pass one header prop, so I create this:
public class AuthHeadersRequestFilter implements ClientRequestFilter {
#Override
public void filter(ClientRequestContext requestContext) throws IOException {
requestContext.getHeaders().add("xx-Authorization", ((IntegraUI) UI.getCurrent()).getSessionToken());
}
}
but for all Client I need to insert this line:
client.register(new AuthHeadersRequestFilter());
So, can I configure to every place i create one client, this filter will be registered automatic?
tks

A step back
How many Client instances are you playing with?
Client instances are heavy-weight objects and you are supposed to have only a small number of instances and reuse them when possible. Have a look at the documentation:
Clients are heavy-weight objects that manage the client-side communication infrastructure. Initialization as well as disposal of a Client instance may be a rather expensive operation. It is therefore advised to construct only a small number of Client instances in the application. Client instances must be properly closed before being disposed to avoid leaking resources.
Registering filters and interceptors
RESTEasy won't register filters and interceptors automatically on your Client. It happens on server side when a filter or interceptor is annotated with #Provider. But on client side, you must register the filters and interceptors manually.
Instead of registering filters and interceptors in the Client instance, you could register them in a Configuration instance and use it to create the Client instance:
Configuration config = new Configuration();
config.register(MyClientResponseFilter.class);
config.register(new AnotherClientFilter());
Client client = ClientBuilder.newClient(config);
It might be useful when creating more than one client with the same configuration.

How to set proxy for Spring AsyncRestTemplate using Netty4ClientHttpRequestFactory?

When I use a SimpleRequestFactory with my AsyncRestTemplate I can easily configure an HTTP proxy server.
I can either do (sample code in Kotlin):
#Bean
open fun asyncRestTemplate(): AsyncRestTemplate {
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("127.0.0.1", 8008))
val requestFactory = SimpleClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setProxy(proxy)
this.setTaskExecutor(taskExecutor())
}
return AsyncRestTemplate(requestFactory)
}
Or I can simply set the corresponding system properties: -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8008.
However, in the moment that I switch from the SimpleClientHttpRequestFactory to a Netty4ClientHttpRequestFactory there is no evident way to configure the proxy directly and it seems this client does not respect the system properties either.
val requestFactory = Netty4ClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
//this.setProxy(proxy) //???
}
Once I change for the netty client, I have no clue on how to make it go through the proxy.
My interest in using the netty client was that I not only wanted to make async requests, but also I wanted this to be non-blocking. I hope I'm not making a wrong assumption here.
Does anyone know how can I use a proxy server when using the Netty4ClientHttpRequestFactory or perhaps know of an alternative non-blocking client supported by Spring that I could use?

The Netty4ClientHttpRequestFactory (source) and related classes such as Netty4ClientHttpRequest (source) use SimpleChannelInboundHandler for the channel and do not use the proxy handler. Everything is private and unable to be overridden within the source, so there is no way to change it to support Proxies. You would have to almost rewrite the whole thing.
You have other async client options that will work very well and allow you more configuration options. The included Netty one is fairly basic anyway. OkHttpClientHttpRequestFactory and HttpComponentsAsyncClientHttpRequestFactory both let you pass in your own configured client.

To your interest, AsyncRestTemplate's different implementation:
SimpleClientHttpRequestFactory -> simple thread pool, blocking api, proxy supported
OkHttpClient (OkHttp3) -> blocking api, proxy supported
CloseableHttpAsyncClient -> non-blocking nio api, proxy supported
Netty4ClientHttpRequestFactory -> non-blocking nio api, proxy not supported
you can visit https://github.com/wuxudong/VariousAsyncHttpClientPerformance for more details

Is there something like DefaultMqttPahoClientFactory for creating an async client?

I would like to use an async Paho client in Spring Integration:
#Bean
public DefaultMqttPahoClientFactory clientFactory() {
return new DefaultMqttPahoClientFactory();
}
<int-mqtt:message-driven-channel-adapter id="mqttInbound"
client-id="${mqtt.default.client.id}.src"
url="${mqtt.url}"
topics="sometopic"
client-factory="clientFactory"
channel="output"/>
Do I have to write one on my own? I can only find examples for sync client factories.

Correct; at this time, only the sync client is supported.
Feel free to open an improvement JIRA issue here and, of course contributions are always welcome - guidelines here.
That said, it's not clear to me the benefit of using an async client in the inbound adapter as in your example; it is message-driven and arriving messages are delivered asynchronously on the client's thread. Obviously, the outbound adapter could gain some benefit from async operations.
The outbound adapter is pretty simple and shouldn't be too hard to write one that uses the async client - it would probably need to send the delivery tokens to some channel - but we should abstract away from the IMqttDeliveryToken object so we don't leak Paho into the upper layers.

I'm pretty sure that the DefaultMqttPahoClientFactory() provided by Spring Integration is using the "synchronous" mode of operation in Paho, so the likelihood is that you would need an alternative ClientFactory implementation.