I'm using httpclient 4.2.5 to make http requests which have to handle redirects as well.
Here is a little example to understand the context:
A sends http request (using httpclient 4.2.5) to B
B sends 302 redirect (containing url to C) back to A
A follows redirect to C
C retrieves request URL and do some work with it
If C parses the request URL by request.getRequestURL() (HttpServlet API) it contains e.g. host and port of the original request from step 1, which is wrong.
The problem exists in step 2, where httpclient handles the redirect. It just copies all headers from the original request (step 1) to the current request (step 3). I already had a look at the responsible code, via grepcode:
DefaultRequestDirector
HttpUriRequest redirect = redirectStrategy.getRedirect(request, response, context);
HttpRequest orig = request.getOriginal();
redirect.setHeaders(orig.getAllHeaders());
I don't really understand why all headers of the original request are copied to the current request.
E.g. using cURL for a simple test is doing it as expected, C would receive the correct host and port.
Implementing my own redirect strategy does not help because the original headers are copied after it.
I had the same problem when trying to download files from bitbucket's download section using HttpClient. After the first request bitbucket sends a redirect to CDN which then complains if the Authorization header is set.
I worked around it by changing DefaultRedirectStrategy.getRedirect() method to return redirect object which does not allow Authorization headers to be set.
I work with Scala so here is the code:
val http = new DefaultHttpClient()
http.setRedirectStrategy(new DefaultRedirectStrategy() {
override def getRedirect(
request: HttpRequest, response: HttpResponse, context: HttpContext
): HttpRequestBase = {
val uri: URI = getLocationURI(request, response, context)
val method: String = request.getRequestLine.getMethod
if (method.equalsIgnoreCase(HttpHead.METHOD_NAME)) {
new HttpHead(uri) {
override def setHeaders(headers: Array[Header]) {
super.setHeaders(headers.filterNot(_.getName == "Authorization"))
}
}
}
else {
new HttpGet(uri) {
override def setHeaders(headers: Array[Header]) {
super.setHeaders(headers.filterNot(_.getName == "Authorization"))
}
}
}
}
})
Please note orig.getAllHeaders() returns an array of headers explicitly added to the message by the caller. The code from DefaultRequestDirector posted above does not copy request headers automatically generated by HttpClient such as Host, Content-Length, Transfer-Encoding and so on.
You post a wire log of the session exhibiting the problem I may be able to tell why redirects do not work as expected.
Related
I'm new to the java rest CXF client. I will make various requests to a remote server, but first I need to create a Ticket Granting Ticket (TGT). I looked through various sources but I could not find a solution. The server requests that I will create a TGT are as follows:
Content-Type: text as parameter, application / x-www-form-urlencoded as value
username
password
I create TGT when I make this request with the example URL like below using Postman. (URL is example). But in the code below, I'm sending the request, but the response is null. Could you help me with the solution?
The example URL that I make a request with POST method using Postman: https://test.service.com/v1/tickets?format=text&username=user&password=pass
List<Object> providers = new ArrayList<Object>();
providers.add(new JacksonJsonProvider());
WebClient client = WebClient.create("https://test.service.com/v1/tickets?format=text&username=user&password=pass", providers);
Response response = client.getResponse();
You need to do a POST, yet you did not specify what your payload looks like?
Your RequestDTO and ResponseDTO have to have getters/setters.
An example of using JAX-RS 2.0 Client.
Client client = ClientBuilder.newBuilder().register(new JacksonJsonProvider()).build();
WebTarget target = client.target("https://test.service.com/v1/tickets");
target.queryParam("format", "text");
target.queryParam("username", "username");
target.queryParam("password", "password");
Response response = target.request().accept(MediaType.APPLICATION_FORM_URLENCODED).post(Entity.entity(yourPostDTO,
MediaType.APPLICATION_JSON));
YourResponseDTO responseDTO = response.readEntity(YourResponseDTO.class);
int status = response.getStatus();
Also something else that can help is if you copy the POST request from POSTMAN as cURL request. It might help to see the differences between your request and POSTMAN. Perhaps extra/different headers are added by postman?
Documentation: https://cxf.apache.org/docs/jax-rs-client-api.html#JAX-RSClientAPI-JAX-RS2.0andCXFspecificAPI
Similar Stackoverflow: Is there a way to configure the ClientBuilder POST request that would enable it to receive both a return code AND a JSON object?
When using the java.net.http.HttpClient classes in Java 11 and later, how does one tell the client to follow through an HTTP 303 to get to the redirected page?
Here is an example. Wikipedia provides a REST URL for getting the summary of a random page of their content. That URL redirects to the URL of the randomly-chosen page. When running this code, I see the 303 when calling HttpResponse#toString. But I do not know how to tell the client class to follow along to the new URL.
HttpClient client = HttpClient.newHttpClient();
HttpRequest request =
HttpRequest
.newBuilder()
.uri( URI.create( "https://en.wikipedia.org/api/rest_v1/page/random/summary" ) )
.build();
try
{
HttpResponse < String > response = client.send( request , HttpResponse.BodyHandlers.ofString() );
System.out.println( "response = " + response ); // ⬅️ We can see the `303` status code.
String body = response.body();
System.out.println( "body = " + body );
}
catch ( IOException e )
{
e.printStackTrace();
}
catch ( InterruptedException e )
{
e.printStackTrace();
}
When run:
response = (GET https://en.wikipedia.org/api/rest_v1/page/random/summary) 303
body =
Problem
You're using HttpClient#newHttpClient(). The documentation of that method states:
Returns a new HttpClient with default settings.
Equivalent to newBuilder().build().
The default settings include: the "GET" request method, a preference of HTTP/2, a redirection policy of NEVER [emphasis added], the default proxy selector, and the default SSL context.
As emphasized, you are creating an HttpClient with a redirection policy of NEVER.
Solution
There are at least two solutions to your problem.
Automatically Follow Redirects
If you want to automatically follow redirects then you need to use HttpClient#newBuilder() (instead of #newHttpClient()) which allows you to configure the to-be-built client. Specifically, you need to call HttpClient.Builder#followRedirects(HttpClient.Redirect) with an appropriate redirect policy before building the client. For example:
HttpClient client =
HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.NORMAL) // follow redirects
.build();
The different redirect policies are specified by the HttpClient.Redirect enum:
Defines the automatic redirection policy.
The automatic redirection policy is checked whenever a 3XX response code is received. If redirection does not happen automatically, then the response, containing the 3XX response code, is returned, where it can be handled manually.
There are three constants: ALWAYS, NEVER, and NORMAL. The meaning of the first two is obvious from their names. The last one, NORMAL, behaves just like ALWAYS except it won't redirect from https URLs to http URLs.
Manually Follow Redirects
As noted in the documentation of HttpClient.Redirect you could instead manually follow a redirect. I'm not well versed in HTTP and how to properly handle all responses so I won't give an example here. But I believe, at a minimum, this requires you:
Check the status code of the response.
If the code indicates a redirect, grab the new URI from the response headers.
If the new URI is relative then resolve it against the request URI.
Send a new request.
Repeat 1-4 as needed.
Obviously configuring the HttpClient to automatically follow redirects is much easier (and less error-prone), but this approach would give you more control.
Please find below code where i was calling another api from my REST APi in java.
To note I am using java version 17. This will solve error code 303.
#GetMapping(value = "url/api/url")
private String methodName() throws IOException, InterruptedException {
var url = "api/url/"; // remote api url which you want to call
System.out.println(url);
var request = HttpRequest.newBuilder().GET().uri(URI.create(url)).setHeader("access-token-key", "accessTokenValue").build();
System.out.println(request);
var client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL).build();
System.out.println(client);
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response);
System.out.println(response.body());
return response.body();
}
I am experiencing a weird issue when working with RestTemplate. I'm using a certain REST API and there I want to update something using PUT.
Thus, in e.g. Postman I am sending this request:
PUT http://fake/foobar/c/123 with a certain body
This update via Postman is successful. If I now execute the same call in Java via a RestTemplate, I am getting a 405 Method Not Allowed:
HttpHeaders headers = createHeader();
HttpEntity<Offer> httpEntity = new HttpEntity<>(bodyEntity, headers);
String url = "http://fake/foobar/c/123"; //Created dynamically, but here pasted for sake of simplicity
RestTemplate restTemplate = new RestTemplate(...);
ResponseEntity<OfferResponse> response = restTemplate.exchange(url, HttpMethod.PUT, httpEntity, OfferResponse.class);
...
I compared the URL again and again. If I copy the URL logged in the console and copy it to Postman, I can do the update successfully. I also compared the headers and everything. Everything is equal to how it is done via Postman.
Is there any potential other reason for such a behavior (another reason than I am too stupid comparing the headers etc. and missing something)? Other PUT, POST calls etc. against this API are working fine, otherwise I would have assumed that there is a general problem with my usage of RestTemplate
Code 405 Method Not Allowed means the HTTP verb (GET, POST, PUT, etc.) you use against this end-point is known but not accepted by the API.
If you can't post the details of your API as #Dinesh Singh Shekhawat suggested, I will first try to use Postman Code feature and get an automatically generated code for Java (OkHTTP or UniRest) of the request. You can find this option on the right part below the Send button. Copy this code and try to perform the request.
Then compare this request with yours.
You can always use HttpPut instead of RestTemplate if it's not a requirement:
HttpClient client = HttpClientBuilder.create().build();
String url = "http://fake/foobar/c/123";
HttpHeaders headers = createHeader();
HttpEntity<Offer> httpEntity = new HttpEntity<>(bodyEntity, headers);
HttpPut httpPut = new HttpPut(url);
httpPut.setEntity(httpEntity);
HttpResponse response = client.execute(httpPut);
I was facing the same problem. Later I printed the request and URL in the logs.
I found that I was using a wrong endpoint.
Can you please try to print the URL and the request in the logs and check if those are expected and correct?
Just in case it helps someone else: I was encountering the same issue and for me it was just the issue of a trailing slash / on the URL. In insomnia (similar to postman) I had a trailing slash, in code I didn't. When I added the slash to my code everything worked.
failure: http://localhost:8080/api/files
success: http://localhost:8080/api/files/
Of course it could also be the other way around, so just double check the actual api definition.
I'm using HttpClient to send a PUT request with a body to Azure.
The body is represented by a StringEntity.
I need to add the Azure authentication signature to the request, and in order to compute it correctly I need the values of the Content-Type and Content-Length headers.
When call the setEntity() method on the HttpPost request, no headers are added to the request, but using a HTTP debugging proxy I can see that they are correctly sent with the request.
From the Apache documentation (here) I saw I could use entity.getContentLength() and entity.getContentType() to compute those, but I would prefer to extract the data directly from the HttpPost if possible.
Anyone knows a way to force the entity to add the headers to the request, before the request is executed?
This is the code I'm using
HttpPut createBlob = new HttpPut(createBlobUrl);
createBlob.addHeader("x-ms-blob-type", "BlockBlob");
createBlob.addHeader("x-ms-date", utcTime);
createBlob.addHeader("x-ms-version", "2015-04-05");
HttpEntity body = new StringEntity("test blob", "UTF-8");
createBlob.setEntity(body);
// h is missing Content-Length and Content-Type
Header[] h = createBlob.getAllHeaders();
resp = httpclient.execute(createBlob);
I found a solution for it.
Adding an interceptor to the client, I can get the complete request with all the headers just before it's executed, so I don't need to deal with special cases where the headers are added by the entity.
HttpClientBuilder builder = HttpClientBuilder.create();
builder = builder.disableContentCompression().disableConnectionState();
builder.addInterceptorLast((HttpRequestInterceptor) (request, context) -> {
try {
SignRequest(request, account, secret);
}
catch (Exception e) {
}
});
HttpClient httpclient = builder.build();
Official tutorial about interceptor can be found at this link
I am having a problem getting the Apache HttpClient to connect to a service external to my virtualised development environment.
To access the internet (e.g. api.twitter.com) I need to call a local URL (e.g. api.twitter.com.dev.mycompany.net), which then forwards the request to real host.
The problem is, that to whatever request I send, I get a 404 Not Found response.
I have tried debugging it using wget, and it appears the problem is, that the destination server identifies the desired resource by using both the request URL and the hostname in the Host header. Since the hostname does not match, it is unable to locate the resource.
I have (unsuccessfully) tried to override the Host header by setting the http.virtual-host parameter on the client like this:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, "api.twitter.com");
}
Technical details:
Client is used as an executor in RESTeasy to call the REST API. So "manually" setting the virtual host (as described here) is not an option.
Everything is done via HTTPS/SSL - not that I think it makes a difference.
Edit 1: Using a HttpHost instead of a String does not have the desired effect either:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
HttpHost realHost = new HttpHost("api.twitter.com", port, scheme);
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, realHost);
}
Edit 2: Further investigation has revealed, that the parameter needs to be set on the request object. The following is the code v. 4.2-aplha1 of HttpClient setting the virtual host:
HttpRequest orig = request;
RequestWrapper origWrapper = wrapRequest(orig);
origWrapper.setParams(params);
HttpRoute origRoute = determineRoute(target, origWrapper, context);
virtualHost = (HttpHost) orig.getParams().getParameter(
ClientPNames.VIRTUAL_HOST);
paramsare the parameters passed from the client. But the value for 'virtualHost' is read from the request parameters.
So this changes the nature of the question to: How do I set the VIRTUAL_HOST property on the requests?
ClientPNames.VIRTUAL_HOST is the right parameter for overriding physical host name in HTTP requests. I would just recommend setting this parameter on the request object instead of the client object. If that does not produce the desired effect please post the complete wire / context log of the session (see logging guide for instructions) either here or to the HttpClient user list.
Follow-up
OK. Let's take a larger sledge hammer. One can override content of the Host header using an interceptor.
DefaultHttpClient client = new DefaultHttpClient();
client.addRequestInterceptor(new HttpRequestInterceptor() {
public void process(
final HttpRequest request,
final HttpContext context) throws HttpException, IOException {
request.setHeader(HTTP.TARGET_HOST, "www.whatever.com");
}
});
One can make the interceptor clever enough to override the header selectively, only for specific hosts.