We Keep Coding

How to determine authentication requirements given 401 response?

I'm using the following code on the client side to take response of a protected website, which works pretty fine whenever there is no authentication request from server side:
final String targetURL = "http://************";
final String username = "********";
final String password = "******";
final int connectTimeout = 30000;
final String username_password = username + ":" + password;
final byte[] usernamepassword = (byte[])username_password.getBytes();
final String encoded_username_password = java.util.Base64.getEncoder().encodeToString(usernamepassword);
final URL url = new URL(targetURL);
final URLConnection connection = url.openConnection();
connection.setConnectTimeout(connectTimeout);
connection.setRequestProperty("Authorization", "Basic " + encoded_username_password);
connection.setRequestProperty("Accept", "text/html");
// response headers
final Map<String, List<String>> headerFields = connection.getHeaderFields();
for (final Map.Entry<String, List<String>> entry : headerFields.entrySet()) {
System.out.println("Key: " + entry.getKey() + ", Value: " + entry.getValue());
}
However, when I try to connect with a specific (protected) webserver, the connection is refused with the 401 http status code.
Key: null, Value: [HTTP/1.1 401 Unauthorized]
Key: WWW-Authenticate, Value: [Digest realm="Login to ********", qop="auth", nonce="*******", opaque=""]
Key: Connection, Value: [close] Key: Content-Length, Value: [0]
Is it possible to infer from the above logged response what I'm missing ?
I mean, the snippet above is a well known Java basic-authentication template, there are many examples everywhere with the same structure, so the issue seems likely related to the parameters issued as argument of the setRequestProperty() method.
This is what I took on request header by using Mozilla >> Web Developer >> Debug (where some fields are hidden by me):
Host: xxx.xxx.xxx.xxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache
Authorization: Digest username="", realm="Login to xxxxxxx", nonce="xxx-xxx-xxx-xxx-xxx", uri="xxxxxxx", response="xxxxx", qop=auth, nc=00000001, cnonce="b2b43c8d9631354d"

I am not sure to understand your question. My guess is you would like to filter the request based on the parameters and url so you can chhose to either serve content or reject and return 401.
In this case your question should be: How to filter which http requests requires authentication.
Depending on what framework your are using, in spring-boot, there is something called WebSecurityConfigurerAdapter. It basically helps you with your problem. You can filter all the requests coming to your server.

It Simply because you are not authenticated to their server,
you need the use that protected server authentication, if you want to enable yourself to access their resource,
maybe u can check their website, or docs to get some some api key or
some token (basic, bearer or whatever token)
then follow their guideline where to put that authentication in your request (usually if token will be in request payload, if api key or basic auth will be in request header)
hope this helps

Send the API Headers in Rest Assured using java

API Headers have two parameter Content-Type=application/json and also accesstoken = "some_token"
I trying to automate the API using Rest assured but not successful.
Below is the code
RestAssured.baseURI = prop.getProperty("serviceurl1");
//2. define the http request:
RequestSpecification httpRequest = RestAssured.given()
.filter(new ResponseLoggingFilter())
.filter(new RequestLoggingFilter());
JSONObject requestParams = new JSONObject();
requestParams.put("longitude", eLongitude);
requestParams.put("latitude", eLaititude);
requestParams.put("country", eCity);
httpRequest.headers("Content-Type", "application/json");
httpRequest.headers("accesstoken", "some_token.");
httpRequest.body(requestParams.toJSONString());
int statusCode = response.getStatusCode();
System.out.println("the status code is: "+ statusCode);
Assert.assertEquals(statusCode, TestUtil.RESPONSE_CODE_200);
System.out.println("the status line is: "+ response.getStatusLine());
//6. get the headers:
Headers headers = response.getHeaders();
System.out.println(headers);
String contentType = response.getHeader("Content-Type");
System.out.println("the value of content-type header is: "+ contentType);
String contentLength = response.getHeader("Content-Length");
System.out.println("the value of Content-Length header is: "+ contentLength);
Getting error message as "Provide Application Token" and 404 error code display.

Your httpRequest.headers("accesstoken", "kggkgkgkgketdfgxgcccvcdftfty."); is wrong. It should be:
httpRequest.headers("Authorization", "Bearer "+token);

can you try this once
Response resp = given().when().contentType(ContentType.JSON).header("accesstoken", "token").body(body).put("url");
You can pass the HashMap as body

These are the issues I can think of
This might be an internal API and it is expecting "Provide Application Token" and not the "accesstoken"
The error code you are getting is 404. So either the service is down or the URL you are using is not correct.
Hope this helps :)

How to abort HttpRequest?

I'm using ThreadSafeClientConnManager to manage a pool of client connections on HttpClient 4.0.1 environment.
Usually my system works fine when authenticating with NTLM, but sometimes it doesn't work properly.
When authenticating with NTLM, we receive result codes in the same order as below: HTTP/1.1 401 > HTTP/1.1 401 > HTTP/1.1 200.
It is correct process.
But, I don't know why, sometimes MS EWS returns 401 result code 3 times: HTTP/1.1 401 > HTTP/1.1 401 > HTTP/1.1 401.
After that, EWS only returns HTTP/1.1 200 for every request.
to help you understand better..
User A: 401 > 401 > 200 (Good)
User B: 401 > 401 > 200 (Good)
User C: 401 > 401 > 401 (Issue arises)
User D: 200
User E: 200
User F: 200
So, when the issue arises, UserC,D,E,F... can use our system properly.
Here are my questions
Why does EWS only return 200 code after returning 401 three times?
How can I abort the connection when I keep receiving the result code, HTTP/1.1 401?
In more detail,
Why does EWS only return 200 code after returning 401 three times? It doesn't happen often, though. The username and password for authentication is correct.
How can I abort the connection when I receive the third HTTP/1.1 401?
I use request.abort(), but it doesn't work.
My code is as below.
private Response processRequest(AbstractHttpClient httpClient, HttpRequestBase httpRequest, HttpContext httpContext){
Response response = new Response();
try {
// execute request
HttpResponse httpResponse = null;
if(httpContext != null){
httpResponse = httpClient.execute(httpRequest, httpContext);
// retrieve cookie
CookieStore cookieStore = (CookieStore)httpContext.getAttribute(ClientContext.COOKIE_STORE);
List<Cookie> cookieList = cookieStore.getCookies();
for(Cookie cookie : cookieList){
com.XXX.mo.connectivity.info.http.Cookie newCookie = new com.XXX.mo.connectivity.info.http.Cookie(cookie.getName(), cookie.getValue());
newCookie.setVersion(cookie.getVersion());
newCookie.setDomain(cookie.getDomain());
newCookie.setPath(cookie.getPath());
response.addCookie(newCookie);
}
}else{
httpResponse = httpClient.execute(httpRequest);
}
// retrieve header
response.setStatusCode(httpResponse.getStatusLine().getStatusCode());
Header [] headers = httpResponse.getAllHeaders();
for(Header header : headers){
response.addHeader(new com.XXX.mo.connectivity.info.http.Header(header.getName(), header.getValue()));
}
// retrieve body
HttpEntity httpEntity = httpResponse.getEntity();
if(httpEntity != null){
response.setContentCharSet(EntityUtils.getContentCharSet(httpEntity));
response.setBody(EntityUtils.toByteArray(httpEntity));
httpEntity.consumeContent();
}
} catch (Exception e) {
e.printStackTrace();
httpRequest.abort();
}
return response;
}

Jetty POST Request returning error code 411

Here is my code to run the POST request
// Instantiate HttpClient
HttpClient httpClient = new HttpClient();
// Configure HttpClient, for example:
httpClient.setFollowRedirects(false);
// Start HttpClient
httpClient.start();
ContentResponse response = httpClient.POST("url")
.agent("Mozilla/5.0")
.method(HttpMethod.POST)
.param("do","login")
.param("url","")
.param("vb_login_md5password", this.password)
.param("vb_login_md5password_utf", this.password)
.param("s", "")
.param("vb_login_username", this.username)
.param("vb_login_password", "")
.send();
for(HttpField h : response.getHeaders())
System.out.println(h.getName() + " = " + h.getValue());
Here is the output
Date = Mon, 14 Dec 2015 08:17:08 GMT
Content-Type = text/html
Transfer-Encoding = chunked
Connection = keep-alive
Set-Cookie = __cfduid=dbb11c371945319b1d5943aad06c1977e1450081028; expires=Tue, 13-Dec-16 08:17:08 GMT; path=/; domain=.sythe.org; HttpOnly
Server = cloudflare-nginx
CF-RAY = 2548787ba9dc21b6-EWR
I look at the ContentResponse text of the post request by doing
response.getContentAsString()
However for some reason I keep getting a 411 error which tells me my Content-Length is zero or not specified. Is there a way to set the content length header of the post?
Thanks!
EDIT : Doing this makes no difference in my post request
.header(HttpHeader.CONTENT_LENGTH, "0")

With .param("do","login") you specify a query param to be added. For the POST to actually contain a body I'd think you need to set content() on the request. Maybe a FormContentProvider will work for you?

Problem displaying request headers with apache httpclient 4

I've been trying to retrieve the headers sent by a HttpMethod, using HttpClient 4, but without any success...
here is my code :
HttpClient httpClient = new DefaultHttpClient();
HttpParams httpParams = httpClient.getParams();
HttpGet httpGet = new HttpGet("http://www.google.fr");
HttpResponse response = httpClient.execute(httpGet);
log.info("*** Request headers ***");
Header[] requestHeaders = httpGet.getAllHeaders();
for(Header header : requestHeaders) {
log.info(header.toString());
}
log.info("***********************");
log.info("*** reponse ***");
log.info(response.getStatusLine());
Header[] headers = response.getAllHeaders();
for(Header header : headers) {
log.info(header.toString());
}
but the result is :
00:27:57,368 INFO - *** Request headers ***
00:27:57,368 INFO - ***********************
00:27:57,368 INFO - *** reponse ***
00:27:57,368 INFO - HTTP/1.1 200 OK
00:27:57,368 INFO - Date: Sun, 15 Aug 2010 22:28:09 GMT
00:27:57,368 INFO - Expires: -1
00:27:57,368 INFO - Cache-Control: private, max-age=0
00:27:57,368 INFO - Content-Type: text/html; charset=ISO-8859-1
00:27:57,368 INFO - Set-Cookie:
[..]
Aka the response headers are good, but not the request's. ( Same result if I move the log request headers block before the execute statement ).
(and NO, I dont want to simply see them, so setting the log level to debug isnt acceptable )
Anyone can help ?

To get all headers including those which the HTTPclient sets, use
HttpCoreContext. This class allows to read out all the headers.
HttpClient client = HttpClients.createDefault();
HttpCoreContext localContext = new HttpCoreContext();
HttpResponse response = client.execute(request,localContext);
Header[] headers = localContext.getRequest().getAllHeaders();
for (Header header : headers) {
System.out.println(header.toString());
}

Things may have changed since 2010, however this can be done by using a request interceptor (to inspect the request at a lower level) with amazingly similar code.
// So we can get all the headers (not just the ones we explicitly set).
httpClient.addRequestInterceptor(new HttpRequestInterceptor() {
public void process(
final HttpRequest request,
final HttpContext context)
throws HttpException, IOException {
// Start Debug
System.out.println("*** Request headers ***");
Header[] requestHeaders = request.getAllHeaders();
for(Header header : requestHeaders) {
System.out.println(header.toString());
}
System.out.println("***********************");
// End Debug
}
});
In my case, I get the following output (having only explicitly set two of these).
*** Request headers ***
Accept: application/xml
Authorization: Basic bmV3Omd1ZXN0
Content-Length: 772
Content-Type: application/xml; charset=UTF-8
Host: rest3api.sifassociation.org:80
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.2.1 (java 1.5)
***********************
May this help those who travel here.

It will only display the request headers you've set yourself.
If you want to log the request headers which HttpClient has set, then you need to configure HttpClient's builtin logging by Commons Logging. Also see this document.
As an alternative, you can also use an external tool like Fiddler.