We Keep Coding

How do I get the full URL in Apache httpclient of a page I am redirected to after authentication?

I am writing a program that needs to access the cPanel API (https://api.docs.cpanel.net). To use the API I need to be authenticated and have the cPanel session id so that I can build my URL. For example to add a new FTP user I would use the following URL - https://hostname.example.com:2083/cpsess##########/execute/Ftp/add_ftp?user=username
When authenticating in cPanel in a browser, the user is redirected to the cPanel home page and the cPanel session id is displayed in the URL.
I am using Apache HTTP client version 4.5.13. I have no problem authenticating using the following code:
HttpHost targetHost = new HttpHost(domain, 2083, "https");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(user, password));
AuthCache authCache = new BasicAuthCache();
authCache.put(targetHost, new BasicScheme());
// Add AuthCache to the execution context
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);
// disable the redirect
//HttpClient client = HttpClientBuilder.create().disableRedirectHandling().build();
// dont disable the redirect
HttpClient client = HttpClientBuilder.create().build();
HttpResponse response = client.execute(
new HttpGet(url), context);
int statusCode = response.getStatusLine().getStatusCode();
System.out.println(statusCode);
Header[] headers = response.getAllHeaders();
for (Header header : headers) {
System.out.println(header.toString());
}
HttpEntity entity = response.getEntity();
if (entity != null) {
// return it as a String
String result = EntityUtils.toString(entity);
System.out.println(result);
}
However when I print out my headers I do not have a Location header so I can not get the URL that contains the session Id.
If I disable the redirect using the line that is commented out above:
HttpClient client = HttpClientBuilder.create().disableRedirectHandling().build();
Then I do get a Location header. However it only provides me with the end of the URl like:
Location: /frontend/paper_lantern/index.html
I need the full URL like:
https://someURL.com:2083/cpsess3886765014/frontend/paper_lantern/index.html
So that I can get the cPanel session Id (cpsess3886765014) in the above URL.
Any help would be much appreciated. Thanks!

OK I figured this out. I can use the following code:
List<URI> redirectLocations = context.getRedirectLocations();
for (URI uri : redirectLocations){
System.out.println(uri.toASCIIString());
}
And I can get the redirects

Java Apache HTTP 401 response returned with correct credentials

I'm trying to hit a REST API link using Apache HttpClient but I keep getting a 401 error returned. I can login when I go to the URL in browser, after being prompted for a password. The code I'm using is below:
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(creds.get(0), creds.get(1));
provider.setCredentials(AuthScope.ANY, credentials);
AuthCache authCache = new BasicAuthCache();
authCache.put(new HttpHost(uri.getHost(), uri.getPort(), "https"), new BasicScheme());
BasicHttpContext context = new BasicHttpContext();
context.setAttribute(ClientContext.CREDS_PROVIDER, provider);
context.setAttribute(ClientContext.AUTH_CACHE, authCache);
DefaultHttpClient client = new DefaultHttpClient();
client.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler());
client.setCredentialsProvider(provider);
HttpResponse response = null;
try
{
// response = client.execute(new HttpGet(uri));
response = client.execute(new HttpGet(uri), context);
}
catch(IOException e)
{
logger.error("Error running authenticated get request: " + e);
}
I'm using HttpClient 4.2.3 and unfortunately I'm not able to upgrade this.
Any help would be appreciated! Thanks!
EDIT: turns out I need to supply the certificate, like using -cacert in curl, however I can't find an example of this!

Since you need to provide a certificate maybe this can help:
http://hc.apache.org/httpcomponents-client-4.2.x/httpclient/examples/org/apache/http/examples/client/ClientCustomSSL.java
I think that example complies with 4.2.3 .

Consuming soap service with NTLM Authentication

I am trying to consume a SOAP service with NTLM authentication by creating a NTLM engine (following instructions on http://hc.apache.org/httpcomponents-client-4.3.x/ntlm.html ) implemented AuthSchemeFactory and finally registered the AuthSchemeFactory to my HTTP Client. When I hit the service using my HTTP Client I get a reponse that "Status code - 415 , Message - The server cannot service the request because the media type is unsupported."
Can anybody tell how can I fix this issue of unsupported media to consume a NTLM-protected SOAP web service on Java platform. Is using JCIFS a correct option to conmsume NTLM protected service or are there any better approach(s). Thanks in advance.
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getAuthSchemes().register(AuthSchemes.NTLM,
new JCIFSNTLMSchemeFactory());
CredentialsProvider credsProvider = new BasicCredentialsProvider();
NTCredentials ntcred = new NTCredentials("USERNAME", "PASSWORD",
"HOST", "DOMAIN");
credsProvider.setCredentials(new AuthScope("HOST", 443,
AuthScope.ANY_REALM, "NTLM"), ntcred);
httpclient.setCredentialsProvider(credsProvider);
httpclient.getParams().setParameter(
CoreProtocolPNames.HTTP_CONTENT_CHARSET, "UTF-8");
Writer writer = new StringWriter();
writer.write("MY SOAP REQUEST BODY");
HttpPost httppost = new HttpPost(
"https://<HOST_NAME>/XiPay30WS.asmx");
httppost.setEntity(new StringEntity(writer.toString()));
httppost.setHeader("Content-Type",
"application/x-www-form-urlencoded");
HttpResponse httpresponse = httpclient.execute(
new HttpHost("HOST", 443, "https"),
httppost, new BasicHttpContext());
String statusCode = httpresponse.getStatusCode();

If you use Spring WS support:
Check this Solution
http://dolszewski.com/spring/sharepoint-web-services-spring-and-ntlm-authentication/
#Bean("navisionMessageSender")
public HttpComponentsMessageSender httpComponentsMessageSender() {
HttpComponentsMessageSender httpComponentsMessageSender = new HttpComponentsMessageSender();
String user = env.getProperty("navision.endpoint.user");
String password = env.getProperty("navision.endpoint.password");
String domain = env.getProperty("navision.endpoint.domain");
NTCredentials credentials = new NTCredentials(user, String.valueOf(password), null, domain);
httpComponentsMessageSender.setCredentials(credentials);
return httpComponentsMessageSender;
}

Sample python implementation with NTLM Auth with FLASK.
If you want to use with java , run the standalone flask code below and call the url (e.g POST request /dora/httpWithNTLM ) from java code by http request
from flask import Flask, render_template, flash, request, url_for, redirect, session , Response
import requests,sys,json
from requests_ntlm import HttpNtlmAuth
app = Flask(__name__)
#app.route("/dora/httpWithNTLM",methods=['POST'])
def invokeHTTPReqWithNTLM():
url =""
reqData = json.loads(request.data)
reqxml=request.data
headers = {}
headers["SOAPAction"] = "";
headers["Content-Type"] = "text/xml"
headers["Accept"] = "text/xml"
print("req headers "+str(request.headers))
r = requests.Request("POST",url,auth=HttpNtlmAuth('domain\\username','password'), data=reqxml, headers=headers)
prepared = r.prepare()
s = requests.Session()
resp = s.send(prepared)
print (resp.status_code)
return Response(resp.text.replace("<","<").replace(">",">"),resp.status_code)
if __name__ == '__main__':
app.run(host="0.0.0.0",port=5001)

How to get error message when connect to server in Android

I use GET method to connect to the server, and the server responses http status code 403.
When I paste the url of my GET method to browser, I'm received "some text" and http status code 403. But when I send a GET request with the same url to the server by HttpURLConnection of Java(Android), I'm just received http status code 403, and response text is null.
So anyone can tell me how to get the "some text" when server return code 403.
Thanks in advance.

Just do as Zoombie wrote and add line:
String reasonPhrase = httpResponse.getStatusLine().getReasonPhrase();
If it doesn't work, your server doesn't set the reason. Then you should map codes to standard reason phrases:
List of HTTP status codes

Try to use DefaultHttpClient class,
client = new DefaultHttpClient(httpParameters);
httpResponse = client.execute(request);
responseCode = httpResponse.getStatusLine().getStatusCode();
HttpEntity entity = httpResponse.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
response = convertStreamToString(instream);
instream.close();
}

Getting Unexpected 401 from Apache HTTPClient Basic Auth

I'm attempting to do basic auth with Apache HTTPClient 4.x using the example from the site, the only change being that I've extracted some details out into constants, however I'm not getting the results I was hoping for.
Namely, with the logging turned up to debug, I'm getting: "DEBUG main client.DefaultHttpClient:1171 - Credentials not found", followed by a 401 error from the server.
I've manually validated that the credentials I've configured are correct, and the "Credentials not found" message leads me to believe the credentials were never passed in the request.
Any thoughts on what I might be doing wrong?
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getCredentialsProvider().setCredentials(
new AuthScope(API_HOST, API_PORT),
new UsernamePasswordCredentials(API_USERNAME, API_PASSWORD));
HttpGet httpget = new HttpGet(API_TEST_URL);
System.out.println("executing request" + httpget.getRequestLine());
HttpResponse response = httpClient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if (entity != null) {
System.out.println("Response content length: " + entity.getContentLength());
}
if (entity != null) {
entity.consumeContent();
}
httpClient.getConnectionManager().shutdown();

Are you sure the AuthScope is set correctly? Try setting it like this just to see if the problem is there
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT)