Hi Guys I am making a bot which can use a an other REST API to create a user but API only supports one user at a time and I dont want to change it. So I am using Multithreading to call the API multiple times, but I want to use proxies in it. So like different proxies for different threads, and all the threads use HttpClient and I tried its #proxy method and gave it the ip and port of the proxy but when I tried to call the API it returned a null response and I tried without the proxy and it did return a valid response.
So is there any other way to assign an proxy to a thread
My Code
HttpClient client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.followRedirects(HttpClient.Redirect.NORMAL)
//THIS LINE BELOW IS HOW I USED PROXIES
.proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))
.connectTimeout(Duration.ofSeconds(30))
.build();
HttpRequest discordAccNoCaptcha = HttpRequest.newBuilder()
.uri(URI.create("https://myapiserver.com/api/v9/auth/register"))
.timeout(Duration.ofMinutes(2))
.POST(HttpRequest.BodyPublishers.ofString(body))
.build();
String response = client.send(discordAccNoCaptcha, HttpResponse.BodyHandlers.ofString()).body(); /* This is null when using ProxySelector and valid response when using no ProxySelector */
Related
I use a httpClient of apache in java to call REST APIs which is configured as below in a static block of class, what I want is to change the connection timeout per request. is it possible? how?
static {
PoolingHttpClientConnectionManager pool = new PoolingHttpClientConnectionManager();
pool.setMaxTotal(ChatSettings.HTTP_CLIENT_THREAD_POOL_SIZE);
httpClient = HttpClients
.custom()
.setConnectionManager(pool)
.setDefaultRequestConfig(
RequestConfig
.custom()
.setConnectTimeout(DEFAULT_HTTP_TIMEOUT)
.setSocketTimeout(DEFAULT_HTTP_TIMEOUT)
.build()
)
.build();
}
One thing you can implement for sure is to follow this example: https://github.com/apache/httpcomponents-client/blob/5.1.x/httpclient5/src/test/java/org/apache/hc/client5/http/examples/ClientAbortMethod.java
It demonstrates how a client can fire a request, but eventually change it's mind and tell the server it no longer wants to receive the response.
Combine this with a 'watchdog' thread that fires after some time/when the time runs out.
I do have the following scenario:
1) The Client sends a HTTP request with an enclosing entity to a Server, via a socket.
2) The Server uploads the enclosing entity to another location, let's call it Storage.
I am required to implement only the Server.
So far, I was able to implement it using Apache HTTP Components library using something like:
// The request from the client
org.apache.http.HttpRequest request = ...;
// The org.apache.http.entity.InputStreamEntity will
// read bytes from the socket and write to the Storage
HttpEntity entity = new InputStreamEntity(...)
BasicHttpEntityEnclosingRequest requestToStorage = new ......
requestToStorage.setEntity(entity);
CloseableHttpClient httpClient = ...
CloseableHttpResponse response = httpClient.execute(target, requestToStorage );
So far so good. Problem is, the Storage server requires authentication. When the Server makes the first request (via Apache Http Client API), the Storage responds with 407 Authentication Required. The Apache Http Client makes the initial handshake then resends the request, but now there is no entity since it has already been consumed for the first request.
One solution is to cache the entity from the Client, but it can be very big, over 1 GB.
Question Is there a better solution, like pre-sending only the request's headers?
Use the expect-continue handshake.
CloseableHttpClient client = HttpClients.custom()
.setDefaultRequestConfig(
RequestConfig.custom()
.setExpectContinueEnabled(true)
.build())
.build();
I'm trying to consume a REST API that requires a body with a GET request. But as a GET usually doesn't have a body, I can't find a way to attach a body in my request. I am also building the REST API, but the professor won't allow us to change the method to POST (he gave us a list of the endpoints we are to create, no more, no less).
I'm trying to do it like this:
Response r = target.request().method(method, Entity.text(body));
Where I set the method to GET and the body to my get body. However, using this approach I get an exception:
javax.ws.rs.ProcessingException: RESTEASY004565: A GET request cannot have a body.
Is there any way to do this with JAX-RS? We learned to use JAX-RS so I would prefer a solution using this, as I'm not sure my professor would allow us to use any other REST client. I'm currently using RESTEasy, provided by the WildFly server.
(This is not a duplicate of HTTP GET with request body because I'm asking on how to create a GET request with body in JAX-RS, not if it should be done.)
This depends on what is your JAX-RS implementation. This check can be disabled in Jersey 2.25 using SUPPRESS_HTTP_COMPLIANCE_VALIDATION property:
ClientConfig config = new ClientConfig();
config.property(ClientProperties.SUPPRESS_HTTP_COMPLIANCE_VALIDATION, true);
JerseyClient client = JerseyClientBuilder.createClient(config);
WebTarget target = client.target(URI.create("https://www.stackoverflow.com"));
Response response = target.request().method("GET", Entity.text("BODY HERE"));
Instead of exception you will get an INFO log
INFO: Detected non-empty entity on a HTTP GET request. The underlying HTTP transport connector may decide to change the request method to POST.
However in RESTEasy 3.5.0.Final there is a hardcoded check in both URLConnectionEngine and ApacheHttpClient4Engine:
if (request.getEntity() != null)
{
if (request.getMethod().equals("GET")) throw new ProcessingException(Messages.MESSAGES.getRequestCannotHaveBody());
You would have to create your own implementation of the ClientHttpEngine to skip this. Then you need to supply it when building the client:
ClientHttpEngine engine = new MyEngine();
ResteasyClient client = new ResteasyClientBuilder().httpEngine(engine).build();
I am having a problem getting the Apache HttpClient to connect to a service external to my virtualised development environment.
To access the internet (e.g. api.twitter.com) I need to call a local URL (e.g. api.twitter.com.dev.mycompany.net), which then forwards the request to real host.
The problem is, that to whatever request I send, I get a 404 Not Found response.
I have tried debugging it using wget, and it appears the problem is, that the destination server identifies the desired resource by using both the request URL and the hostname in the Host header. Since the hostname does not match, it is unable to locate the resource.
I have (unsuccessfully) tried to override the Host header by setting the http.virtual-host parameter on the client like this:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, "api.twitter.com");
}
Technical details:
Client is used as an executor in RESTeasy to call the REST API. So "manually" setting the virtual host (as described here) is not an option.
Everything is done via HTTPS/SSL - not that I think it makes a difference.
Edit 1: Using a HttpHost instead of a String does not have the desired effect either:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
HttpHost realHost = new HttpHost("api.twitter.com", port, scheme);
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, realHost);
}
Edit 2: Further investigation has revealed, that the parameter needs to be set on the request object. The following is the code v. 4.2-aplha1 of HttpClient setting the virtual host:
HttpRequest orig = request;
RequestWrapper origWrapper = wrapRequest(orig);
origWrapper.setParams(params);
HttpRoute origRoute = determineRoute(target, origWrapper, context);
virtualHost = (HttpHost) orig.getParams().getParameter(
ClientPNames.VIRTUAL_HOST);
paramsare the parameters passed from the client. But the value for 'virtualHost' is read from the request parameters.
So this changes the nature of the question to: How do I set the VIRTUAL_HOST property on the requests?
ClientPNames.VIRTUAL_HOST is the right parameter for overriding physical host name in HTTP requests. I would just recommend setting this parameter on the request object instead of the client object. If that does not produce the desired effect please post the complete wire / context log of the session (see logging guide for instructions) either here or to the HttpClient user list.
Follow-up
OK. Let's take a larger sledge hammer. One can override content of the Host header using an interceptor.
DefaultHttpClient client = new DefaultHttpClient();
client.addRequestInterceptor(new HttpRequestInterceptor() {
public void process(
final HttpRequest request,
final HttpContext context) throws HttpException, IOException {
request.setHeader(HTTP.TARGET_HOST, "www.whatever.com");
}
});
One can make the interceptor clever enough to override the header selectively, only for specific hosts.
I have a JSP that dynamically sets the page header of my application.
However, I want to be able to call the REST Service that gets user details based on the system user. I already have the system user value but need to call the backend service to get the details from the database. This is already implemented but I don't know how to setup the JSP to do this.
I do not want to use javascript as this is being used for the extjs side of things.
In order to call REST from JSP, you could utilize Apache HTTPClient. Once you have that you could walk through the samples as well as the HTTPClient Tutorial. HTTPClient supports all REST API Call including GET/POST and others.
Check also this following HTTPClient template to see how HTTPClient can be used with REST. You need to call a similar code from your JSP.
In particular to REST GET Service, you want to look the following block from the template in the above link
final HttpClient httpClient = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(httpClient.getParams(), 10000);
HttpGet httpget = new HttpGet(SERVER_URL + url);
HttpResponse response = httpClient.execute(httpget);