We Keep Coding

Java Apache HTTP 401 response returned with correct credentials

I'm trying to hit a REST API link using Apache HttpClient but I keep getting a 401 error returned. I can login when I go to the URL in browser, after being prompted for a password. The code I'm using is below:
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(creds.get(0), creds.get(1));
provider.setCredentials(AuthScope.ANY, credentials);
AuthCache authCache = new BasicAuthCache();
authCache.put(new HttpHost(uri.getHost(), uri.getPort(), "https"), new BasicScheme());
BasicHttpContext context = new BasicHttpContext();
context.setAttribute(ClientContext.CREDS_PROVIDER, provider);
context.setAttribute(ClientContext.AUTH_CACHE, authCache);
DefaultHttpClient client = new DefaultHttpClient();
client.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler());
client.setCredentialsProvider(provider);
HttpResponse response = null;
try
{
// response = client.execute(new HttpGet(uri));
response = client.execute(new HttpGet(uri), context);
}
catch(IOException e)
{
logger.error("Error running authenticated get request: " + e);
}
I'm using HttpClient 4.2.3 and unfortunately I'm not able to upgrade this.
Any help would be appreciated! Thanks!
EDIT: turns out I need to supply the certificate, like using -cacert in curl, however I can't find an example of this!

Since you need to provide a certificate maybe this can help:
http://hc.apache.org/httpcomponents-client-4.2.x/httpclient/examples/org/apache/http/examples/client/ClientCustomSSL.java
I think that example complies with 4.2.3 .

How to fix HTTP 504 Gateway timeout exception

I am attempting a GET request to another API to get a json response.
When I make the request with the code below, I am getting HTTP 504 Error (Gateway timeout error).
However, when I tried it through rest client tool, the request does not throw any error.
How do I increase the time gap in my code to avoid the timeout error?
This is how the call looks like:
HttpClient httpClient = getBasicAuthDefaultHttpClient();
String url= "http://XXXXX";
HttpGet httpGet = new HttpGet(url);
httpGet.addHeader("id", id);
httpGet.addHeader("secret", secret);
httpGet.addHeader("network_val", networkval);
HttpResponse response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
if (entity != null && response.getStatusLine().getStatusCode() == HttpStatus.OK.value()) {
ObjectMapper objectMapper = new ObjectMapper();
restInfo = objectMapper.readValue(entity.getContent(), MyClass.class);
} else {
logger.error("Call to API failed: response code = {}", response.getStatusLine().getStatusCode());
}
Please note:
Could this be something to do with 'https'?
When I try with 'http' through my Insomnia REST Client I get the ERROR: error: Failure when receiving data from the peer.
https works fine without any error (https://XXXXX)

This is what I tried.
public HttpClient getBasicAuthDefaultHttpClient() {
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials creds = new UsernamePasswordCredentials(user,
password);
provider.setCredentials(AuthScope.ANY, creds);
//Fix to avoid HTTP 504 ERROR (GATEWAY TIME OUT ERROR)
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setConnectTimeout(30 * 1000);
requestBuilder.setConnectionRequestTimeout(30 * 1000);
HttpClientBuilder builder = HttpClientBuilder.create();
builder.setDefaultRequestConfig(requestBuilder.build());
builder.setDefaultCredentialsProvider(provider).build();
return builder.build();
}

NTLM authentication java via HttpClient

My problem is i'm trying to get into scopus using a crawler but it requires my crawler to enter the site through my school proxy server. I tried authenticating but it keep responding with 401 status.
public void testConnection() throws ClientProtocolException, IOException {
DefaultHttpClient httpclient = new DefaultHttpClient();
List<String> authpref = new ArrayList<String>();
authpref.add(AuthPolicy.NTLM);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authpref);
NTCredentials creds = new NTCredentials("username","password","ezlibproxy1.ntu.edu.sg","ntu.edu.sg");//this is correct
httpclient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);
HttpHost target = new HttpHost("ezlibproxy1.ntu.edu.sg", 443, "https");//this is correct
// Make sure the same context is used to execute logically related requests
HttpContext localContext = new BasicHttpContext();
// Execute a cheap method first. This will trigger NTLM authentication
HttpGet httpget = new HttpGet("http://www-scopus-com.ezlibproxy1.ntu.edu.sg/authid/detail.url?authorId=14831850700");
HttpResponse response = httpclient.execute(target, httpget, localContext);
HttpEntity entity = response.getEntity();
System.out.println(EntityUtils.toString(entity));
int statusCode = response.getStatusLine().getStatusCode();
System.out.println("Status Code:" + statusCode);
}
The status code respond is 401 (unauthorised).
Any suggestion on this?

Apache HttpAsyncClient

I am testing the Apache HttpAsyncClient, in particular I want to make an asynchronous HTTP POST Request where authentication is needed. I use this example as reference. So far I found out how to set Application type and body but can't find out how to set the credentials.
I try to add Authentication credentials with
HttpAsyncClientBuilder create = HttpAsyncClientBuilder.create();
create.setTargetAuthenticationStrategy(new TargetAuthenticationStrategy());
BasicCredentialsProvider basicCredentialsProvider = new BasicCredentialsProvider();
Credentials defaultcreds = new UsernamePasswordCredentials("user", "password");
basicCredentialsProvider.setCredentials(new AuthScope("http://localhost", 7351), defaultcreds);
create.setDefaultCredentialsProvider(basicCredentialsProvider);
final CloseableHttpAsyncClient httpclient = create.build();
httpclient.start();
...
But I always get
Sep 11, 2013 4:21:35 PM org.apache.http.impl.auth.HttpAuthenticator handleAuthChallenge
WARNING: Malformed challenge: Authentication challenge is empty
I have not found an example which explains how to set authentication data for the CloseableHttpAsyncClient. Anyone can help me out?

You can set a credentials provider either at the client level if you want it to be shared by all requests by default
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
CloseableHttpAsyncClient httpclient = HttpAsyncClients.custom()
.setDefaultCredentialsProvider(credentialsProvider)
.build();
httpclient.start();
try {
HttpGet request = new HttpGet("http://www.apache.org/");
Future<HttpResponse> future = httpclient.execute(request, null);
HttpResponse response = future.get();
System.out.println("Response: " + response.getStatusLine());
System.out.println("Shutting down");
} finally {
httpclient.close();
}
System.out.println("Done");
or set it at the request level, if you want it to apply to a particular request only
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
httpclient.start();
try {
HttpGet request = new HttpGet("http://www.apache.org/");
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credentialsProvider);
Future<HttpResponse> future = httpclient.execute(request, context, null);
HttpResponse response = future.get();
System.out.println("Response: " + response.getStatusLine());
System.out.println("Shutting down");
} finally {
httpclient.close();
}
System.out.println("Done");
Please also note that Malformed challenge: Authentication challenge is empty warning is likely caused by the server sending a malformed (empty) auth challenge rather than HttpClient configuration. Providing user credentials for the request may not necessarily resolve the issue.

Getting Unexpected 401 from Apache HTTPClient Basic Auth

I'm attempting to do basic auth with Apache HTTPClient 4.x using the example from the site, the only change being that I've extracted some details out into constants, however I'm not getting the results I was hoping for.
Namely, with the logging turned up to debug, I'm getting: "DEBUG main client.DefaultHttpClient:1171 - Credentials not found", followed by a 401 error from the server.
I've manually validated that the credentials I've configured are correct, and the "Credentials not found" message leads me to believe the credentials were never passed in the request.
Any thoughts on what I might be doing wrong?
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getCredentialsProvider().setCredentials(
new AuthScope(API_HOST, API_PORT),
new UsernamePasswordCredentials(API_USERNAME, API_PASSWORD));
HttpGet httpget = new HttpGet(API_TEST_URL);
System.out.println("executing request" + httpget.getRequestLine());
HttpResponse response = httpClient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if (entity != null) {
System.out.println("Response content length: " + entity.getContentLength());
}
if (entity != null) {
entity.consumeContent();
}
httpClient.getConnectionManager().shutdown();

Are you sure the AuthScope is set correctly? Try setting it like this just to see if the problem is there
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT)