We Keep Coding

Unable to download Azure blob from Java

This is a rather strange issue. I need to generate a signed url that can be shared with others for downloading the file within a certain time limit. Obviously, since I am saying signed, you should not require any prior permissions to download. Following is the code I am using to upload and download
private final BlobSasPermission blobSasPermission = new BlobSasPermission()
.setReadPermission(true);
public String uploadAndGenerateSignedUrl(String filePath, String uploadPath) {
BlobClient blobClient = containerClient.getBlobClient(uploadPath);
blobClient.uploadFromFile(filePath);
BlockBlobClient blockBlobClient = blobClient.getBlockBlobClient();
BlobServiceSasSignatureValues blobServiceSasSignatureValues = new BlobServiceSasSignatureValues(
OffsetDateTime.now().plusMinutes(azureConfiguration.getExpiryMin()), blobSasPermission);
return blockBlobClient.getBlobUrl() + "?" + blockBlobClient
.generateSas(blobServiceSasSignatureValues);
}
public void downloadFromUrl(String signedUrl, File file) throws IOException {
OkHttpClient client = SharedClient.getNewSharedClientBuilder().build();
Request request = new Request.Builder()
.url(signedUrl)
.build();
okhttp3.Response response = client.newCall(request).execute();
InputStream inputStream = response.body().byteStream();
FileUtils.copyInputStreamToFile(inputStream, file);
}
Now, when I open the generated url in the browser, the file is successfully downloaded. Similarly, running the GET in PostMan for this url, works fine. However, when I send the same url to the download method, it fails with error 403 stating Server failed to authenticate the request. I am clueless about the reason for this. I tried adding both http and https protocols to signed url but that also did not help.
Edit:
Adding the error message
Response{protocol=http/1.1, code=403, message=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature., url=https://{account}.blob.core.windows.net/{container}/MDConnector/MDTest/Test_16269425/2021-07-24/1627125196450/NormalFile.md?sv=2019-07-07&se=2021-07-24T12:13:18Z&sr=b&sp=r&sig=BoTaZ9iEA8Cdcbscf6zpWTol32+52rfVyLNDMBlLr1Q=}

I'm able to reproduce this issue. Essentially the issue is coming because of + sign in your SAS token signature. When a URL is created, + sign is interpreted as a space and because of that your authorization is failing.
What you have to do is URL encode your SAS token. Once you do that, your request should work just fine. Please see the sample code below:
package com.company;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import java.io.IOException;
import java.io.InputStream;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
public class Main {
public static void main(String[] args) throws IOException {
String url = "https://account.blob.core.windows.net/container/blob.png";
String sasToken = "sv=2020-04-08&st=2021-07-24T14:33:27Z&se=2021-07-31T14:33:00Z&sr=b&sp=rac&sig=QytPc/+0z/eHd+u4WO0HGOFDOZjVfB+vbQdbR6FFrl4=";//Notice the "+" sign in SAS token.
sasToken = URLEncoder.encode(sasToken, StandardCharsets.UTF_8.toString());
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url(url+"?"+ sasToken).build();
okhttp3.Response response = client.newCall(request).execute();
InputStream inputStream = response.body().byteStream();
}
}

Read from URL if content is image [duplicate]

I am trying to write a java program that will automatically download and name some of my favorite web comics. Since I will be requesting multiple objects from the same domain, I wanted to have a persistent http connection that I could keep open until all the comics have been downloaded. Below is my work-in-progress. How do I make another request from the same domain but different path without opening a new http connection?
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class ComicDownloader
{
public static void main(String[] args)
{
URL url = null;
HttpURLConnection httpc = null;
BufferedReader input = null;
try
{
url = new URL("http://www.cad-comic.com/cad/archive/2002");
httpc = (HttpURLConnection) url.openConnection();
input = new BufferedReader(new InputStreamReader(httpc.getInputStream()));
String inputLine;
while ((inputLine = input.readLine()) != null)
{
System.out.println(inputLine);
}
input.close();
httpc.disconnect();
}
catch (IOException ex)
{
System.out.println(ex);
}
}
}

According to the documentation here, HTTP persistence is being handled transparently in Java, although it gives you the options to control it too via http.keepAlive and http.maxConnections system properties.
However,
The current implementation doesn't
buffer the response body. Which means
that the application has to finish
reading the response body or call
close() to abandon the rest of the
response body, in order for that
connection to be reused. Furthermore,
current implementation will not try
block-reading when cleaning up the
connection, meaning if the whole
response body is not available, the
connection will not be reused.
Take a look at the link and see if it really helps you.

According to this link http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html, HTTP connection reuse is enabled by default, you can use Wireshark to check the interactions between your client and server. The first request contains TCP and SSL handshakes(if your request is https), the subsequent requests fired in the keep-alive time, contains no TCP and SSL handshakes, just application data transfers.

Even though HttpURLConnection enable keep-alive by default, it is not guaranteed that HttpURLConnection uses same TCP connection for multiple HTTP requests. I faced same kind of issue when writing HTTPS client application. Solved this issue by using single instance of SSLContext, SSLSocketFactory and HttpsURLConnection.
public class MyHTTPClient {
private SSLContext mSSLContext = null;
private SSLSocketFactory mSSLSocketFactory = null;
private HttpsURLConnection mConnection = null;
public void init() {
//Setup SSL context and Socket factory here
}
pubblic void sendRequest() {
URL url = new URL("https://example.com/request_receiver");
mConnection = (HttpsURLConnection) url.openConnection();
mConnection.setSSLSocketFactory(mSSLSocketFactory);
// Setup request property and send request
// Open input stream to read response
// Close output, input streams
mConnection.disconnect();
}
}

JBPM Console Remote authetication through java

I have deployed jbpm-cosole on my localhost, and I have created a java application to make REST calls for process transaction. So, my question is.
As jbpm-console uses spring-security, how to send authentication request to JBPM?
After successful authentication I want to store authentication token as a java object to make further transactions without login.
or is there any another way to do that please let me know.

This is a sample code which returns the tasks list of a user. You just need to set the authentication details in request header as shown in this code. This works with jbpm-6.1.0.Final.
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
public class JBPMRest {
public static void getTaskSummaryList() throws Exception {
String status = "Reserved";
String actor = "krisv";
String addr = "http://localhost:8080/jbpm-console/rest/task/query?status=" + status + "&potentialOwner=" + actor;
try {
HttpClient client = HttpClientBuilder.create().build();
HttpGet get = new HttpGet(addr);
String authData = "krisv" + ":" + "krisv";
String encoded = new sun.misc.BASE64Encoder().encode(authData.getBytes());
get.setHeader("Content-Type", "application/json");
get.setHeader("Authorization", "Basic " + encoded);
get.setHeader("ACCEPT", "application/xml");
HttpResponse cgResponse = client.execute(get);
String content = EntityUtils.toString(cgResponse.getEntity());
System.out.println(content);
} catch (Exception e) {
throw new Exception("Error consuming service.", e);
}
}
}

You can send basic authentication information (username + password) in the header of your REST call. The remote Java client implementation for example does this as well. If you use the remote Java client, you can reuse the same engine for subsequent calls as well.

IIS windows authentication error (401.1)

I have configure IIS 8.0 with PHP 5.3 , along with SQL Server.
I am not sure what is going wrong. I am able to see my JSON string in web browser , when I type in URL in the browser,
But When I passed it to the code below, it is return me
IIS 8.0 Detailed Error - 401.2 - Unauthorized
401.2 Error page in my eclipse.
I am connecting to the database via Windows Authentication. Therefore , I have set Windows Authentication enabled in my IIS. I cannot able any other authentication other than Windows authentication , then my JSONstring is not return at all from the browser as well., in that case.
I have tried the solution mentioned in the below URL
http://support.microsoft.com/kb/942043
The problem still persists
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.CoreProtocolPNames;
public class JSONTester {
/**
* #param args
* #throws Exception
*/
public static void main(String[] args)
{
InputStream is = null;
String json = "";
try
{
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://localhost/hive/get_all_products.php");
httpPost.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
System.out.println(e.toString());
}
System.out.println(json);
}
}
Any help would be great .
Thanks !!!

Internet Explorer is the only browser that does the NTLM authentication using integrated mode out-of-the-box, this is why your URL works in your 'browser'. Your Java HttpClient must authenticate explicitly and answer the NTLM challenge from the server. See HttpClient NTLM Authentication:
NTLM is the most complex of the authentication protocols supported by
HttpClient. It is a proprietary protocol designed by Microsoft with no
publicly available specification... NTLM
authentication requires an instance of NTCredentials be available for
the domain name of the server or the default credentials. Note that
since NTLM does not use the notion of realms HttpClient uses the
domain name of the server as the name of the realm. Also note that the
username provided to the NTCredentials should not be prefixed with the
domain - ie: "adrian" is correct whereas "DOMAIN\adrian" is not
correct...
HttpClient provides limited support for what is known as NTLMv1, the early version of the NTLM protocol. HttpClient does not support NTLMv2 at all.
The important thing to take home is that HttpClient does not support integrated authentication. You must provide explicit credentials, which makes it always probe to bad practices because of the requirement to provide the account password explicitly. Make sure you do not store it but ask it from the user, much the same way Firefox does when visiting a site that requests an NTLM challenge.
Read The NTLM Authentication Protocol and Security Support Provider for more details.

Make sure the account you are authenticating with has rights to the files in the website directory.

Connection to a URL from within an applet using Apache's HttpClient vs using the JDK's URLConnection

In the following code, I have verified that connecting to a URL from within an applet preserves the browser's session if JDK's URLConnection class is used. However, this is not the case if Apache's HttpClient library is used. Does anyone know why? Alternatively, is there a way for me to set the connection instance to be used by an HttpClient instance?
import java.applet.Applet;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URISyntaxException;
import java.net.URL;
import javax.net.ssl.SSLException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
public class HttpClientTesterApplet extends Applet {
private static final long serialVersionUID = -1599714556710568947L;
public void testHttpClient() throws ClientProtocolException, IOException,
URISyntaxException {
URL url = new URL(String.format("%s://localhost:%s/%s/testHttpClient",
getParameter("protocol"), getParameter("port"),
getParameter("context")));
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url.toURI());
System.out.println("Executing request " + post.getURI());
try {
System.out
.println(client.execute(post, new BasicResponseHandler()));
} catch (SSLException e) {
System.out.println(e.getMessage());
}
System.out.println("Executed request " + post.getURI());
System.out.println("Opening connection " + url);
HttpURLConnection urlConnection = (HttpURLConnection) url
.openConnection();
System.out.println("Opened connection " + url);
urlConnection.setRequestMethod("POST");
System.out.println("Connecting");
urlConnection.connect();
System.out.println("Connected");
InputStream inputStream = urlConnection.getInputStream();
try {
while (inputStream.read() != -1) {
System.out.println("Reading");
}
} finally {
inputStream.close();
}
}
}

This is a common problem with libraries implementing their own URL connection via Socket. Apparently, the JRE implementation of the URLConnection class can get to the browser information directly. We had to employ the technique as mentioned by oscargm above, i.e. on the appserver writing the request cookies to be the parameters to the applet AND getting to the browser's document cookies using JavaScript (this is for the case of SSO where the set of cookies may not be the same because of the intermediate agent -- proxy servers). Note that if the cookies are HttpOnly -- the javascript code will fail.

You must send the jsessionid cookie or rewrite your URL to use the jsessionid.
That's the way the server knows your session.
If you generate the applet tag in a JSP page dynamically you can pass the jsessionidvalue to the applet as a parameter and then use it.
post.setHeader("Cookie", "jsessionid=" + jsessionidValue );

I think that you're using an older version of HttpClient. Check out HttpClient's website.
In the current API, you can use HttpState in the execute method, so that your code could look like this:
HttpClient client = new HttpClient();
HttpMethod method = new PostMethod(url.toURI());
HttpState state = new HttpState();
client.executeMethod(HttpConfiguration.ANY_HOST_CONFIGURATION, method, state);
In the next execution, pass the same "state" object, and you'll get the credentials and cookies preserved.

Possible causes, is that you have not done a disconnect() when using URLConnection, however, the apache library will close the connection when you are done with it.

Thi is an important issue.
The standard java.net.URLConnection class integrates seamlessly with the java plugin and the web browser, can inherit session, HTTP authentication tokens, proxy connectors, etc.
The guys at Apache Commons made a gross mistake when they decided to implement HttpClient from Socket (ie, from scratch) instead of just developing on top of the standard java.net.URL* classes. HttpClient does not inherit from java.net.URLConnection so it cannot inherit its advanced enterprise features.
Maybe OpenSource projects are not so smart as they think.

I could make it work without passing cookies as arguments from the Web Page with this code:
private String retrieveCookies(URL url) throws IOException, URISyntaxException
{
String cookieValue = null;
CookieHandler handler = CookieHandler.getDefault();
if (handler != null) {
Map<String, List<String>> headers = handler.get(url.toURI(), new HashMap<String, List<String>>());
List<String> cookiesList = headers.get("Cookie");
if (cookiesList != null)
{
for (String v : cookiesList) {
if (cookieValue == null)
cookieValue = v;
else
cookieValue = cookieValue + ";" + v;
}
}
}
return cookieValue;
}
...
httppost.addHeader("Cookie", retrieveCookies(new URL(uploadUrl)));
JDK's class CookieHandler can fortunately get the cookies from the "system" store. In this case it's the browser store, accesed via the Java Plugin.
Sort of "manual work", but it works.
NOTE: I found the code here