We Keep Coding

Java Specific webclient connect timeout per request

I have to call different urls with different connection timeout with webclient. But i found that we can set connect timeout globally only and not per resquest .. what to do please to set this timeout on a request without creating a new weblient each time.

You need to instantiate one webclient per url. For each webclient, you can set the connection timeout :
HttpClient httpClient = HttpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000);
WebClient client = WebClient.builder()
.baseUrl("http://yourendpoint:8080")
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
Then reuse this webclient for all the calls to the configured url.

Assign a different proxy to thread - Java

Hi Guys I am making a bot which can use a an other REST API to create a user but API only supports one user at a time and I dont want to change it. So I am using Multithreading to call the API multiple times, but I want to use proxies in it. So like different proxies for different threads, and all the threads use HttpClient and I tried its #proxy method and gave it the ip and port of the proxy but when I tried to call the API it returned a null response and I tried without the proxy and it did return a valid response.
So is there any other way to assign an proxy to a thread
My Code
HttpClient client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.followRedirects(HttpClient.Redirect.NORMAL)
//THIS LINE BELOW IS HOW I USED PROXIES
.proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))
.connectTimeout(Duration.ofSeconds(30))
.build();
HttpRequest discordAccNoCaptcha = HttpRequest.newBuilder()
.uri(URI.create("https://myapiserver.com/api/v9/auth/register"))
.timeout(Duration.ofMinutes(2))
.POST(HttpRequest.BodyPublishers.ofString(body))
.build();
String response = client.send(discordAccNoCaptcha, HttpResponse.BodyHandlers.ofString()).body(); /* This is null when using ProxySelector and valid response when using no ProxySelector */

Best way for apache HttpClients using in a multithreaded environment

I need to create a service on the server-side for sending HTTP requests. It works like this:
1. Client sends a request to the server
2. Server uses singleton service for calling 3rd-party API via HTTP.
3. API returns the response
4. Server process this response and return it to the client
It is a synchronized process.
I created service:
public class ApacheHttpClient {
private final static CloseableHttpClient client = HttpClients.createDefault();
public String sendPost(String serverUrl, String body) {
try {
HttpPost httpPost = new HttpPost(serverUrl);
httpPost.setEntity(new StringEntity(body));
CloseableHttpResponse response = client.execute(httpPost);
String responseAsString = EntityUtils.toString(response.getEntity());
response.close();
return responseAsString;
} catch (IOException e) {
throw new RestException(e);
}
}
ApacheHttpClient is a singleton in my system. CloseableHttpClient client is a singleton too.
Question 1: Is it correct to use one instance of CloseableHttpClient client or I should create a new instance for each request?
Question 2: When I should close the client?
Question 3: This client can process only 2 connections in one time period. Should I use an executor?

The use of HttpClient instance as a singleton per distinct HTTP service is correct and is in line with the Apache HttpClient best practices. It should not be static, though.
http://hc.apache.org/httpcomponents-client-5.1.x/migration-guide/preparation.html
One should close HttpClient when releasing the HTTP service. In your case ApacheHttpClient should implement Closeable and close the internal instance of CloseableHttpClient in its #close method.
You probably should not, but it really depends on how exactly your application deals with request execution.

Default read and connection timeouts for reactor-netty HttpClient

I know that a read and a connection timeouts can be configured in reactor-netty HttpClient, like:
public WebClient xsdWebClient() {
HttpClient httpClient = createHttpClient(config.getConnectionTimeout(), config.getReadTimeout());
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient.followRedirect(true)))
.baseUrl(config.getHost())
.build();
}
private static HttpClient createHttpClient(int connectionTimeout, int readTimeout) {
return HttpClient.create()
.option(
ChannelOption.CONNECT_TIMEOUT_MILLIS,
(int) TimeUnit.SECONDS.toMillis(connectionTimeout))
.doOnConnected(c -> c.addHandlerLast(new ReadTimeoutHandler(readTimeout)));
}
But what are the default read and connection timeouts for reactor-netty HttpClient?

I would recommend you to use the response timeout configuration provided by Reactor Netty instead of ReadTimeoutHandler. You can configure the response timeout either globally on HttpClient level for all requests or per request.
The default values for various timeouts provided by Reactor Netty you can find in the reference documentation.

Referring to answer given by one of the devs of reactor-netty, the read default time is 10 seconds.

HttpClient can't get response from server

This problem has blocked our whole team half a day!
We use apache httpclient 4.3.x to post and get data from an storage server which provides http api. In order to improve performance, we used PoolingHttpClientConnectionManager:
public HttpClient createHttpClient() {
Registry registry = RegistryBuilder.create()....build();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
connectionManager.setMaxTotal(50);
connectionManager.setDefaultMaxPerRoute(50);
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.build();
return httpClient;
}
Then we hold an instance of the httpClient in our program, reuse it with every http request:
Global httpClient:
HttpClient httpClient = createHttpClient();
Post some data:
HttpPost httpPut = new HttpPost("...");
HttpResponse response = httpClient.execute(httpPut);
// Notice we get the response content here!
String content = EntityUtils.toString(response.getEntity());
System.out.println(content);
httpPut.releaseConnection();
response.close();
Then get:
HttpGet httpGet = new HttpGet("...");
// Blocked at this line !!!!
HttpResponse response = httpClient.execute(httpGet);
String content = EntityUtils.toString(response.getEntity());
System.out.println(content);
httpPut.releaseConnection();
response.close();
Please notice the line: // Blocked at this line !!!!
The program has blocked at that line and never go to next line. In debugging mode, I can see it has been blocked at:
SocketInputStream.socketRead0()
I've searched for a lot of questions and documents, but no lucky.
My colleage just fix it by setting NoConnectionReuseStrategy.INSTANCE:
HttpClients.custom()
.setConnectionManager(connectionManager)
// Following line fixed the problem, but why?
.setConnectionReuseStrategy(NoConnectionReuseStrategy.INSTANCE)
.build();
Now it doens't blocked, but why?
What does "reuse connection" mean? And is there performance issue by using NoConnectionReuseStrategy?
Thank you, guys~

I tried to reproduce the blocking http-get (also as an exercise for myself) but even without closing responses I could not get it to block. The ONLY time I managed to make the http-get block is by doing a response.getEntity().getContent() without reading from the returned InputStream and without closing the returned InputStream.
For my tests I used Tomcat 7.0.47 with two very simple servlets (one responding "OK" to a get, the other echoing a post) as a server. The client started 50 threads with each thread performing 30 alternating http-get and http-post request (total of 1500 requests). The client did not use the RegistryBuilder, instead the default one is used (created by the PoolingHttpClientConnectionManager itself).
About the NoConnectionReuseStrategy: by default (HttpClient created with HttpClients.createDefault(), I used org.apache.httpcomponents:httpclient:4.3.1) a connection pool is used with a maximum of 2 connections to 1 server. E.g. even if 5 threads are doing all kinds of requests at the same time to 1 server, the connection pool opens only 2 connections, re-uses them for all requests and ensures that 1 connection is used by 1 thread at any given time. This can have a very positive impact on client performance and significantly reduces load on the server. The only thing you must make sure is to call response.close() in a finally-block (this ensures the connection is returned to the connection pool). By using the NoConnectionReuseStrategy you basically disable the connection pool: for each request a new connection will be created. I recommend you enable debug-logging for category org.apache.http.impl.conn.PoolingHttpClientConnectionManager, it is very informative.
A note about httpPut.releaseConnection(): this does not actually release a connection, it only ensures that you can re-use the "httpPut" object in a next request (see the apidocs, follow the shown link). Also note that in your code for the "httpGet", you call releaseConnection() on "httpPut" instead of "httpGet".

Ran into this problem just a while back. In case someone else comes across this problem, this post might be useful.
I am using a Java Servlet to service my requests. When I wrote to the response stream using the PrintWriter instance my client blocked. Tried writing to the OutputStream directlyresponse.getOutputStream.write("myresponse") and it worked.