We Keep Coding

Unable to download Azure blob from Java

This is a rather strange issue. I need to generate a signed url that can be shared with others for downloading the file within a certain time limit. Obviously, since I am saying signed, you should not require any prior permissions to download. Following is the code I am using to upload and download
private final BlobSasPermission blobSasPermission = new BlobSasPermission()
.setReadPermission(true);
public String uploadAndGenerateSignedUrl(String filePath, String uploadPath) {
BlobClient blobClient = containerClient.getBlobClient(uploadPath);
blobClient.uploadFromFile(filePath);
BlockBlobClient blockBlobClient = blobClient.getBlockBlobClient();
BlobServiceSasSignatureValues blobServiceSasSignatureValues = new BlobServiceSasSignatureValues(
OffsetDateTime.now().plusMinutes(azureConfiguration.getExpiryMin()), blobSasPermission);
return blockBlobClient.getBlobUrl() + "?" + blockBlobClient
.generateSas(blobServiceSasSignatureValues);
}
public void downloadFromUrl(String signedUrl, File file) throws IOException {
OkHttpClient client = SharedClient.getNewSharedClientBuilder().build();
Request request = new Request.Builder()
.url(signedUrl)
.build();
okhttp3.Response response = client.newCall(request).execute();
InputStream inputStream = response.body().byteStream();
FileUtils.copyInputStreamToFile(inputStream, file);
}
Now, when I open the generated url in the browser, the file is successfully downloaded. Similarly, running the GET in PostMan for this url, works fine. However, when I send the same url to the download method, it fails with error 403 stating Server failed to authenticate the request. I am clueless about the reason for this. I tried adding both http and https protocols to signed url but that also did not help.
Edit:
Adding the error message
Response{protocol=http/1.1, code=403, message=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature., url=https://{account}.blob.core.windows.net/{container}/MDConnector/MDTest/Test_16269425/2021-07-24/1627125196450/NormalFile.md?sv=2019-07-07&se=2021-07-24T12:13:18Z&sr=b&sp=r&sig=BoTaZ9iEA8Cdcbscf6zpWTol32+52rfVyLNDMBlLr1Q=}

I'm able to reproduce this issue. Essentially the issue is coming because of + sign in your SAS token signature. When a URL is created, + sign is interpreted as a space and because of that your authorization is failing.
What you have to do is URL encode your SAS token. Once you do that, your request should work just fine. Please see the sample code below:
package com.company;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import java.io.IOException;
import java.io.InputStream;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
public class Main {
public static void main(String[] args) throws IOException {
String url = "https://account.blob.core.windows.net/container/blob.png";
String sasToken = "sv=2020-04-08&st=2021-07-24T14:33:27Z&se=2021-07-31T14:33:00Z&sr=b&sp=rac&sig=QytPc/+0z/eHd+u4WO0HGOFDOZjVfB+vbQdbR6FFrl4=";//Notice the "+" sign in SAS token.
sasToken = URLEncoder.encode(sasToken, StandardCharsets.UTF_8.toString());
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url(url+"?"+ sasToken).build();
okhttp3.Response response = client.newCall(request).execute();
InputStream inputStream = response.body().byteStream();
}
}

Kerberos authentication for web request in Java?

I'm trying to implement exactly this: https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/lab/part6.html
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;
public class RunHttpSpnego {
static final String kuser = "username"; // your account name
static final String kpass = "password"; // your password for the account
static class MyAuthenticator extends Authenticator {
public PasswordAuthentication getPasswordAuthentication() {
// I haven't checked getRequestingScheme() here, since for NTLM
// and Negotiate, the usrname and password are all the same.
System.err.println("Feeding username and password for "
+ getRequestingScheme());
return (new PasswordAuthentication(kuser, kpass.toCharArray()));
}
}
public static void main(String[] args) throws Exception {
Authenticator.setDefault(new MyAuthenticator());
URL url = new URL(args[0]);
InputStream ins = url.openConnection().getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(ins));
String str;
while((str = reader.readLine()) != null)
System.out.println(str);
}
}
I've already reduced to trying just to execute the sample code snippet, but I don't know how to overcome this issue I'm facing.
As I understand, the flow for such requests should look like, as detailed in this document as well:
1. The clients sends a HTTP GET request to the server
2. The server responds with a HTTP 401, and a WWW Authenticate: Negotiate header
3. Then the client sends another HTTP GET request, this time with the proper Negotiate header in the request.
My problem is, when I try to run that sample, the request fails with an IOException obviously, because of the server responding with a 401, and then I don't know how to issue the second request with the header. I was trying to add a header as with Basic Authentication, but in that case I don't know where to get the Negotiate value for that from.
I did a packet capture, I can see the TGS ticket being requested from the KDC, so I think that part is OK, I just don't know how it should be applied to my request. I did the same packet capture with other scripts connecting to the same web service, and I can see the exact same flow what I've described above.
So, to summarize my question: how should I add the authentication header to my request? Do I need to do it explicitely in my code, if so, how do I do it?

URLConnection in java using username, password, token

I am trying to Connect to url using URLConnection, in java with username and password.
This is the following code I am using:
package com.nivi.org.client;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import com.sun.jersey.core.util.Base64;
public class GetURLContent {
public static void main(String[] args) {
URL url;
try {
url = new URL("http://sampleurl.co.uk");
URLConnection conn = url.openConnection();
String username = "username";
String password = "password";
String Token = "zzzzzzzzzzzzzzzzzzzzz";
System.setProperty("http.proxyHost", "proxyhostname");
System.setProperty("http.proxyPort", "8080");
String userpass = username + ":" + password;
String basicAuth = "Basic " + new String(new Base64().encode(userpass.getBytes()));
conn.setRequestProperty ("Authorization", basicAuth);
conn.setRequestProperty ("Token", Token);
BufferedReader br = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
System.out.println("br............."+br.toString());
br.close();
System.out.println("Done");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
=======================================
I am getting the following error
java.io.IOException: Server returned HTTP response code: 403 for URL:
http://sampleurl.co.uk
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1625)
at com.nivi.org.client.GetURLContent.main(GetURLContent.java:63)
=======================================
First of all my question is
Are these following lines in the code are correct?
conn.setRequestProperty ("Authorization", basicAuth);
conn.setRequestProperty ("Token", Token);

Are these following lines in the code are correct?
It depends on how the actual website you are accessing implements its user authentication.
What you appear to be doing is a combination of HTTP Basic Authentication (i.e. the Authorization header), and something involving a non-standard header called Token. This may be sufficient, if the website supports Basic Authentication.
You should probably read the website's programmer documentation of how their web APIs work. If there isn't any such documentation available to you, use your browsers web development tools to identify the mechanism that the site is using when you log in via a web browser ... and attempt to get your client to behave the same way.
One thing to note is that sending a Basic authorization response in an HTTP request is insecure. Use an HTTPS request if that is an option.

IIS windows authentication error (401.1)

I have configure IIS 8.0 with PHP 5.3 , along with SQL Server.
I am not sure what is going wrong. I am able to see my JSON string in web browser , when I type in URL in the browser,
But When I passed it to the code below, it is return me
IIS 8.0 Detailed Error - 401.2 - Unauthorized
401.2 Error page in my eclipse.
I am connecting to the database via Windows Authentication. Therefore , I have set Windows Authentication enabled in my IIS. I cannot able any other authentication other than Windows authentication , then my JSONstring is not return at all from the browser as well., in that case.
I have tried the solution mentioned in the below URL
http://support.microsoft.com/kb/942043
The problem still persists
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.CoreProtocolPNames;
public class JSONTester {
/**
* #param args
* #throws Exception
*/
public static void main(String[] args)
{
InputStream is = null;
String json = "";
try
{
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://localhost/hive/get_all_products.php");
httpPost.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
System.out.println(e.toString());
}
System.out.println(json);
}
}
Any help would be great .
Thanks !!!

Internet Explorer is the only browser that does the NTLM authentication using integrated mode out-of-the-box, this is why your URL works in your 'browser'. Your Java HttpClient must authenticate explicitly and answer the NTLM challenge from the server. See HttpClient NTLM Authentication:
NTLM is the most complex of the authentication protocols supported by
HttpClient. It is a proprietary protocol designed by Microsoft with no
publicly available specification... NTLM
authentication requires an instance of NTCredentials be available for
the domain name of the server or the default credentials. Note that
since NTLM does not use the notion of realms HttpClient uses the
domain name of the server as the name of the realm. Also note that the
username provided to the NTCredentials should not be prefixed with the
domain - ie: "adrian" is correct whereas "DOMAIN\adrian" is not
correct...
HttpClient provides limited support for what is known as NTLMv1, the early version of the NTLM protocol. HttpClient does not support NTLMv2 at all.
The important thing to take home is that HttpClient does not support integrated authentication. You must provide explicit credentials, which makes it always probe to bad practices because of the requirement to provide the account password explicitly. Make sure you do not store it but ask it from the user, much the same way Firefox does when visiting a site that requests an NTLM challenge.
Read The NTLM Authentication Protocol and Security Support Provider for more details.

Make sure the account you are authenticating with has rights to the files in the website directory.

Connection to a URL from within an applet using Apache's HttpClient vs using the JDK's URLConnection

In the following code, I have verified that connecting to a URL from within an applet preserves the browser's session if JDK's URLConnection class is used. However, this is not the case if Apache's HttpClient library is used. Does anyone know why? Alternatively, is there a way for me to set the connection instance to be used by an HttpClient instance?
import java.applet.Applet;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URISyntaxException;
import java.net.URL;
import javax.net.ssl.SSLException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
public class HttpClientTesterApplet extends Applet {
private static final long serialVersionUID = -1599714556710568947L;
public void testHttpClient() throws ClientProtocolException, IOException,
URISyntaxException {
URL url = new URL(String.format("%s://localhost:%s/%s/testHttpClient",
getParameter("protocol"), getParameter("port"),
getParameter("context")));
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url.toURI());
System.out.println("Executing request " + post.getURI());
try {
System.out
.println(client.execute(post, new BasicResponseHandler()));
} catch (SSLException e) {
System.out.println(e.getMessage());
}
System.out.println("Executed request " + post.getURI());
System.out.println("Opening connection " + url);
HttpURLConnection urlConnection = (HttpURLConnection) url
.openConnection();
System.out.println("Opened connection " + url);
urlConnection.setRequestMethod("POST");
System.out.println("Connecting");
urlConnection.connect();
System.out.println("Connected");
InputStream inputStream = urlConnection.getInputStream();
try {
while (inputStream.read() != -1) {
System.out.println("Reading");
}
} finally {
inputStream.close();
}
}
}

This is a common problem with libraries implementing their own URL connection via Socket. Apparently, the JRE implementation of the URLConnection class can get to the browser information directly. We had to employ the technique as mentioned by oscargm above, i.e. on the appserver writing the request cookies to be the parameters to the applet AND getting to the browser's document cookies using JavaScript (this is for the case of SSO where the set of cookies may not be the same because of the intermediate agent -- proxy servers). Note that if the cookies are HttpOnly -- the javascript code will fail.

You must send the jsessionid cookie or rewrite your URL to use the jsessionid.
That's the way the server knows your session.
If you generate the applet tag in a JSP page dynamically you can pass the jsessionidvalue to the applet as a parameter and then use it.
post.setHeader("Cookie", "jsessionid=" + jsessionidValue );

I think that you're using an older version of HttpClient. Check out HttpClient's website.
In the current API, you can use HttpState in the execute method, so that your code could look like this:
HttpClient client = new HttpClient();
HttpMethod method = new PostMethod(url.toURI());
HttpState state = new HttpState();
client.executeMethod(HttpConfiguration.ANY_HOST_CONFIGURATION, method, state);
In the next execution, pass the same "state" object, and you'll get the credentials and cookies preserved.

Possible causes, is that you have not done a disconnect() when using URLConnection, however, the apache library will close the connection when you are done with it.

Thi is an important issue.
The standard java.net.URLConnection class integrates seamlessly with the java plugin and the web browser, can inherit session, HTTP authentication tokens, proxy connectors, etc.
The guys at Apache Commons made a gross mistake when they decided to implement HttpClient from Socket (ie, from scratch) instead of just developing on top of the standard java.net.URL* classes. HttpClient does not inherit from java.net.URLConnection so it cannot inherit its advanced enterprise features.
Maybe OpenSource projects are not so smart as they think.

I could make it work without passing cookies as arguments from the Web Page with this code:
private String retrieveCookies(URL url) throws IOException, URISyntaxException
{
String cookieValue = null;
CookieHandler handler = CookieHandler.getDefault();
if (handler != null) {
Map<String, List<String>> headers = handler.get(url.toURI(), new HashMap<String, List<String>>());
List<String> cookiesList = headers.get("Cookie");
if (cookiesList != null)
{
for (String v : cookiesList) {
if (cookieValue == null)
cookieValue = v;
else
cookieValue = cookieValue + ";" + v;
}
}
}
return cookieValue;
}
...
httppost.addHeader("Cookie", retrieveCookies(new URL(uploadUrl)));
JDK's class CookieHandler can fortunately get the cookies from the "system" store. In this case it's the browser store, accesed via the Java Plugin.
Sort of "manual work", but it works.
NOTE: I found the code here