This question already has answers here:
Apache HttpClient Digest authentication
(4 answers)
Closed 4 years ago.
When I try to connect to a webserver using DIGEST authentification, the connection will be refused (401, Not Authenticated). The Answers I could find related to this topic where mostly deprecated, so I wanted to start a discussion about resolving this problem with the current version:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.4.1</version>
</dependency>
The following code allows to set credentials and connect to my target rest-service using Apache HttpClient. The credentials I am using are correct, so there must be something wrong with my configuration of the credentials or the way I'm using the HTTPClient.
HttpGet getArticles = new HttpGet("http://myurl.xx/api/");
Registry<AuthSchemeProvider> authSchemeRegistry =
RegistryBuilder.<AuthSchemeProvider>create().register(AuthSchemes.DIGEST,new DigestSchemeFactory()).build();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope("http://myurl.xx/api/",80),
new UsernamePasswordCredentials(username,pw));
CloseableHttpClient client = HttpClients.custom()
.setDefaultAuthSchemeRegistry(authSchemeRegistry)
.setDefaultCredentialsProvider(credentialsProvider).build();
try {
CloseableHttpResponse response = client.execute(getArticles);
logger.info(String.valueOf(response.getStatusLine().getStatusCode()));
} catch (IOException e) {
logger.error(e.getMessage(),e);
}finally {
try {
client.close();
} catch (IOException e) {
logger.error(e.getMessage(),e);
}
}
This will return
HTTP/1.1 401 Unauthorized
I'm not an expert with digest auth but I know you have to connect twice, because the servers will send you some auth data in the first place. But I believe that, when I register DIGEST as the AuthentificationScheme, this should be handled automatically?
When I check the Client
client.authSchemeRegistry says:
{digest=org.apache.http.impl.auth.DigestSchemeFactory#481a996b}
So it's successfully registered.
It may or may not be the reason, but the scope of credentials is wrong. The AuthScope constructor takes a host name, not a URL as the first parameter.
credentialsProvider.setCredentials(
new AuthScope("myurl.xx",80),
new UsernamePasswordCredentials(username,pw));
Related
This question already has answers here:
How to set Cookies at Http Get method using Java
(3 answers)
Closed 3 years ago.
I am trying to perform the same task in Java, but can't seem to figure out how to particularly set the cookies.
I know how to do this in python:
response = requests.get(app_url,cookies=cookies,allow_redirects=False)
How do I do the equivalent in Java?
Open a URLConnection(HttpURLConnection or HttpsURLConnection, set the cookie and connect.
HttpURLConnection con;
InputStream is;
try{
con=((HttpURLConnection)new URL(app_url).openConnection());
con.setRequestProperty("Cookie",cookie);
is=con.openStream();
//recv code
}finally{
if(is!=null){try{is.close();}catch(IOException e){}}
if(con!=null){try{con.close();}catch(IOException e){}
}
The typical solution is using Apache HttpClient. If you need a less engineered and/or third-party library free solution, I'd suggest URLConnection or the new Java 11 HttpClient.
private final CloseableHttpClient client = HttpClients.createDefault();
...
public String get(String appUrl, String cookie) {
HttpGet request = new HttpGet(appUrl);
request.setHeader("Cookie", cookie);
try (CloseableHttpResponse response = client.execute(request)) {
HttpEntity entity = response.getEntity();
return EntityUtils.toString(entity);
}
}
Using Java11's HttpClient it could look like follows:
HttpResponse<String> response = HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.NEVER)
.build()
.send(
HttpRequest.newBuilder(URI.create(url))
.header("cookieName", "cookieValue")
.GET()
.build(),
HttpResponse.BodyHandlers.ofString());
I assume you can reliably do the request. If not, this could help you.
How to get HTTP response code for a URL in Java?
To handle cookies, you may want to look at this
How to set Cookies at Http Get method using Java
Both implementations use the basic Java HttpURLConnection class.
I'm using the most recent apache http:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient-osgi</artifactId>
<version>4.5.6</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore-osgi</artifactId>
<version>4.4.10</version>
</dependency>
I have the following operation:
public void store(InputStream input) throws IOException {
HttpClientBuilder builder = HttpClientBuilder.create();
if (StringUtils.isNotBlank(username)) {
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(username.trim(), StringUtils.trimToEmpty(password));
provider.setCredentials(AuthScope.ANY, credentials);
builder.setDefaultCredentialsProvider(provider);
}
HttpClient client = builder.build();
HttpPost post = new HttpPost(uri);
post.setEntity(new InputStreamEntity(input));
HttpResponse response = client.execute(post);
}
Until basic auth was active, everything was working fine, however, after adding basic auth I get the following error:
Caused by: org.apache.http.client.NonRepeatableRequestException:
Cannot retry request with a non-repeatable request entity. at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:226)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
... 6 more
I've found a following bug report: https://github.com/http-builder-ng/http-builder-ng/issues/10, however it is assigned to another problem.
What is causing the error? How to use basic auth with apache httpclient? I have no idea what is 'repeatable HTTP request', from what I know all the client need to set is Authorization header. Is it possible that I've misconfigured something on the server so that it requires 'repeatable' HTTP request?
It seems that basic authentication model is broken in apache httpclient. The library tries to sneak around authentication and sends request without Authorization header, which of course fails. Then the library tries to resend the request, which of course fails, because InputStream can't be rewinded.
The solution is to forget BasicCredentialsProvider and use HttpRequestInterceptor to set headers:
HttpClientBuilder builder = HttpClientBuilder.create();
if (StringUtils.isNotBlank(username)) {
builder.addInterceptorFirst(new HttpRequestInterceptor()
{
#Override
public void process(HttpRequest request, HttpContext context) throws HttpException, IOException
{
String token = Base64.getEncoder().encodeToString((username + ":" + password).getBytes(StandardCharsets.UTF_8));
request.addHeader("Authorization", "Basic "+token);
}
});
}
I do not agree with the OP's own solution, because it is somehow very hackish and circumvents the Credential Mechanism of the library.
There are several entity types for HTTP Entities, as documented here. So now that you know, you cannot use a repeatable entity in your scenario, how about using a self-contained one or a wrapper which uses a buffer.
You can achieve this with a one-liner. Without having tried it out, I think the correct solution is:
post.setEntity(new BufferedHttpEntity(new InputStreamEntity(input)));
I am trying to get some data (json data -> restful server) from a HTTPS server with basic authentification using the Apache httpclient. The SSL certificate is selfsigned.
The server is responding very well to a browser call and also when using curl.
However using the java Apache httpclient, that's another story.
Client side :
The basic authentification is working : the server sends me 401 errors if forget the authorization header or if I set the wrong base64 encoded login:password.
The https/SSL part is working : I am successfully getting data from online restful json server but sadly no way to find an online restful json server with basic authentification for testing purpose...
try {
CloseableHttpClient httpClient = null;
try {
httpClient = HttpClients.custom()
.setSSLSocketFactory(new SSLConnectionSocketFactory(SSLContexts.custom()
.loadTrustMaterial(null, new TrustSelfSignedStrategy())
.build()
)
)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.build();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyManagementException e) {
e.printStackTrace();
} catch (KeyStoreException e) {
e.printStackTrace();
}
HttpGet getRequest = new HttpGet("https://localhost:5050/getdata");
getRequest.addHeader("Accept", "*/*");
getRequest.addHeader("Authorization", "Basic UBJ0aHVyOmFo0XElYHU=");
HttpResponse response = httpClient.execute(getRequest);
Debugging is telling me :
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
True! It's not a valid HTTP response that I would like to get, it's a valid HTTPS response!
I guess that I am missing something...
Solved!
The error was from the server side : my response did not include any headers....
httpclient seems to like well made responses from a server. That's not true for a browser or for curl : garbage they can receive , display they will !
trying to access http://forum.worldoftanks.eu/index.php?app=members using apache HttpClient but keep getting 403. Can anyone help out?
Been fiddling with this piece as a starting point:
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpRequestBase method = new HttpGet(theUrl);
String s = httpClient.execute(method, new BasicResponseHandler());
System.out.println(s);
httpClient.getConnectionManager().shutdown();
I don't think this is related to HttpClient. I tried this
$ wget http://forum.worldoftanks.eu/index.php?app=members
--2011-08-08 23:17:52-- http://forum.worldoftanks.eu/index.php?app=members
Resolving forum.worldoftanks.eu (forum.worldoftanks.eu)... 213.252.177.21, 213.2
52.177.20
Connecting to forum.worldoftanks.eu (forum.worldoftanks.eu)|213.252.177.21|:80..
. connected.
HTTP request sent, awaiting response... 403 Forbidden
2011-08-08 23:17:56 ERROR 403: Forbidden.
with no luck.
Yet I can hit it in the browser. It might be that there is some server logic returning 403s when an appropriate browser headers aren't sent. My next step would be to use FireBug and try to replicate the request as your browser makes it.
Also, try catching the exceptino
} catch (HttpResponseException e) {
System.err.println(e.response.parseAsString());
}
Does anyone have a good example of how to do https over http (or a socket)?
The embedded java platform doesn't support https out of the box, but I'm sure it should be possible using some 3rd party libraries or source code snippets.
If anyone can point me to the correct direction I'd be very grateful
What profile are you using? MIDP comes with HTTS handler. I just checked the code. This package,
com.sun.midp.io.j2me.https
implements HttpsConnection interface.
EDIT:
I think your best bet is to find an old version of BouncyCastle JCE that works with your version of Java. BC JCE comes with this class to handle SSL,
http://www.bouncycastle.org/docs/docs1.3/org/bouncycastle/crypto/tls/TlsProtocolHandler.html
This test shows you how to make a simple HTTPS request.
http://www.java2s.com/Open-Source/Java-Document/Security/Bouncy-Castle/org/bouncycastle/crypto/tls/test/BasicTlsTest.java.htm
As bitover has said, you can use apache http components.
Simple example where https page need user credentials:
public static void main (String[] args){
HttpClient httpclient = new DefaultHttpClient();
// Note that the specified port 443 corresponds with the SSL service
((AbstractHttpClient) httpclient).getCredentialsProvider().setCredentials(
new AuthScope("www.moneytrackin.com", 443),
new UsernamePasswordCredentials("user", "password"));
// Https page to access
HttpGet httpget = new HttpGet("https://www.moneytrackin.com/api/rest/getBalance");
HttpResponse response;
try {
response = httpclient.execute(httpget);
System.out.println("State: "+response.getStatusLine().toString());
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
System.out.println("Data: "+result);
instream.close();
}
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
More examples in this blog.
Apache HttpCore is what you need. See more: http://hc.apache.org/httpcomponents-core/index.html
Br,
Gabi
From what i can remember, with HttpComponents you just set the socket factory to use the SSLSocketFactory and it works; however, the SSLSocketFactory was added in Java 1.4.
Honestly, I would not think the Apache HttpComponents API would even work on the limited environment you described... you would most likely need an older version, from back when it was called HttpClient (I think) and part of the Jakarta Commons.
Good luck
JBoss netty has an example of HTTP client, you just need to add the SSlHandler in HTTPRequestHandler and HTTPResponseHandler.