We Keep Coding

How to follow-through on HTTP 303 status code when using HttpClient in Java 11 and later?

When using the java.net.http.HttpClient classes in Java 11 and later, how does one tell the client to follow through an HTTP 303 to get to the redirected page?
Here is an example. Wikipedia provides a REST URL for getting the summary of a random page of their content. That URL redirects to the URL of the randomly-chosen page. When running this code, I see the 303 when calling HttpResponse#toString. But I do not know how to tell the client class to follow along to the new URL.
HttpClient client = HttpClient.newHttpClient();
HttpRequest request =
HttpRequest
.newBuilder()
.uri( URI.create( "https://en.wikipedia.org/api/rest_v1/page/random/summary" ) )
.build();
try
{
HttpResponse < String > response = client.send( request , HttpResponse.BodyHandlers.ofString() );
System.out.println( "response = " + response ); // ⬅️ We can see the `303` status code.
String body = response.body();
System.out.println( "body = " + body );
}
catch ( IOException e )
{
e.printStackTrace();
}
catch ( InterruptedException e )
{
e.printStackTrace();
}
When run:
response = (GET https://en.wikipedia.org/api/rest_v1/page/random/summary) 303
body =

Problem
You're using HttpClient#newHttpClient(). The documentation of that method states:
Returns a new HttpClient with default settings.
Equivalent to newBuilder().build().
The default settings include: the "GET" request method, a preference of HTTP/2, a redirection policy of NEVER [emphasis added], the default proxy selector, and the default SSL context.
As emphasized, you are creating an HttpClient with a redirection policy of NEVER.
Solution
There are at least two solutions to your problem.
Automatically Follow Redirects
If you want to automatically follow redirects then you need to use HttpClient#newBuilder() (instead of #newHttpClient()) which allows you to configure the to-be-built client. Specifically, you need to call HttpClient.Builder#followRedirects(HttpClient.Redirect) with an appropriate redirect policy before building the client. For example:
HttpClient client =
HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.NORMAL) // follow redirects
.build();
The different redirect policies are specified by the HttpClient.Redirect enum:
Defines the automatic redirection policy.
The automatic redirection policy is checked whenever a 3XX response code is received. If redirection does not happen automatically, then the response, containing the 3XX response code, is returned, where it can be handled manually.
There are three constants: ALWAYS, NEVER, and NORMAL. The meaning of the first two is obvious from their names. The last one, NORMAL, behaves just like ALWAYS except it won't redirect from https URLs to http URLs.
Manually Follow Redirects
As noted in the documentation of HttpClient.Redirect you could instead manually follow a redirect. I'm not well versed in HTTP and how to properly handle all responses so I won't give an example here. But I believe, at a minimum, this requires you:
Check the status code of the response.
If the code indicates a redirect, grab the new URI from the response headers.
If the new URI is relative then resolve it against the request URI.
Send a new request.
Repeat 1-4 as needed.
Obviously configuring the HttpClient to automatically follow redirects is much easier (and less error-prone), but this approach would give you more control.

Please find below code where i was calling another api from my REST APi in java.
To note I am using java version 17. This will solve error code 303.
#GetMapping(value = "url/api/url")
private String methodName() throws IOException, InterruptedException {
var url = "api/url/"; // remote api url which you want to call
System.out.println(url);
var request = HttpRequest.newBuilder().GET().uri(URI.create(url)).setHeader("access-token-key", "accessTokenValue").build();
System.out.println(request);
var client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL).build();
System.out.println(client);
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response);
System.out.println(response.body());
return response.body();
}

Apache HttpClient custom dynamic header on every request

I am writing a java client for a protected api service using Apache's HttpClient. I was wondering if it is possible to add a dynamic header to each request automatically instead of having to add the header on every HttpGet or HttpPost instance. The header needs to take the request URL and the request method (GET or POST), because of this requirement I cannot just simply add it to the default request headers when building the HttpClient. Thanks

Use custom request interceptor
CloseableHttpClient client = CachingHttpClients.custom()
.addInterceptorLast((HttpRequestInterceptor) (request, context) -> {
String method = request.getRequestLine().getMethod();
String requestUri = request.getRequestLine().getUri();
request.addHeader("x-my-header", doSomethingClever(method, requestUri));
})
.build();

Handling HTTP request redirect in java

I'm writing a network android application that uses http requests to get data. The data is HTML format. I use Apache HttpClient and JSoup.
When I'm out of traffic with my mobile internet provider, I am always redirected to the providers' page saying that I should pay some money. Of course, it is a bad idea to parse this page.
How to detect occured page substitution?

This code will help you to know with is the final target of your request, if isn't the page that you asked for, is the provider page.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://www.google.com/");
HttpResponse response = httpclient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);// this is the final page of the request
System.out.println("Final target: " + target);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thanks

If your provider is lying to you by immediately returning a 200 OK but not giving you the resource you've requested, your best option is probably to set a custom HTTP response header that your client can check before continuing.

Header values overwritten on redirect in HTTPClient

I'm using httpclient 4.2.5 to make http requests which have to handle redirects as well.
Here is a little example to understand the context:
A sends http request (using httpclient 4.2.5) to B
B sends 302 redirect (containing url to C) back to A
A follows redirect to C
C retrieves request URL and do some work with it
If C parses the request URL by request.getRequestURL() (HttpServlet API) it contains e.g. host and port of the original request from step 1, which is wrong.
The problem exists in step 2, where httpclient handles the redirect. It just copies all headers from the original request (step 1) to the current request (step 3). I already had a look at the responsible code, via grepcode:
DefaultRequestDirector
HttpUriRequest redirect = redirectStrategy.getRedirect(request, response, context);
HttpRequest orig = request.getOriginal();
redirect.setHeaders(orig.getAllHeaders());
I don't really understand why all headers of the original request are copied to the current request.
E.g. using cURL for a simple test is doing it as expected, C would receive the correct host and port.
Implementing my own redirect strategy does not help because the original headers are copied after it.

I had the same problem when trying to download files from bitbucket's download section using HttpClient. After the first request bitbucket sends a redirect to CDN which then complains if the Authorization header is set.
I worked around it by changing DefaultRedirectStrategy.getRedirect() method to return redirect object which does not allow Authorization headers to be set.
I work with Scala so here is the code:
val http = new DefaultHttpClient()
http.setRedirectStrategy(new DefaultRedirectStrategy() {
override def getRedirect(
request: HttpRequest, response: HttpResponse, context: HttpContext
): HttpRequestBase = {
val uri: URI = getLocationURI(request, response, context)
val method: String = request.getRequestLine.getMethod
if (method.equalsIgnoreCase(HttpHead.METHOD_NAME)) {
new HttpHead(uri) {
override def setHeaders(headers: Array[Header]) {
super.setHeaders(headers.filterNot(_.getName == "Authorization"))
}
}
}
else {
new HttpGet(uri) {
override def setHeaders(headers: Array[Header]) {
super.setHeaders(headers.filterNot(_.getName == "Authorization"))
}
}
}
}
})

Please note orig.getAllHeaders() returns an array of headers explicitly added to the message by the caller. The code from DefaultRequestDirector posted above does not copy request headers automatically generated by HttpClient such as Host, Content-Length, Transfer-Encoding and so on.
You post a wire log of the session exhibiting the problem I may be able to tell why redirects do not work as expected.

How to emulate a browser HTTPS POST request with Java (Apache HTTP Client)?

There is a website with an AJAX API. I have opened Firebug to look into the details of the login HTTPS POST request.
Then I have tried to do the same POST request from my Java program using Apache HTTP Client. But somehow the server identified my request as a non browser request. It sends a security exception message, which tells me that.
When all request headers are the same, what else could identify my client as not a browser?

My guess is that it's a cookie issue (e.g. JSESSIONID the browser has stored). Include the session information with your POST. Have a look at the cookies of this site. Try disabling cookies for this site a have a look a the request again.

user-agent header? "httpclient.useragent" property
Use debug mode to see full wire logging and compare the request with firebug's one.

Dont know about the POST request but there is this for a multipart request
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
see if its of any help
EDIT: Code sample for a multipart request
String createOrderUrl = Constants.CREATE_ORDER_URL;
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(createOrderUrl);
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
// add the information to the multipart request
entity.addPart("msisdn", new StringBody("something"));
entity.addPart("recipientname", new StringBody("something"));
entity.addPart("recipientnumber", new StringBody("something"));
entity.addPart("recipientaddress", new StringBody("something"));
// add the images
for (String imagePath : selectedImages)
{
FileBody bin = new FileBody(new File(imagePath));
entity.addPart("image", bin);
}
httpPost.setEntity(entity);
return httpClient.execute(httpPost);