Unencrypted HTTP protocol error when creating new URL() object in Java - java

I'm facing issue while resolving DAST issue 'USAGE OF
UNENCRYPTED HTTP PROTOCOL [SAST] [M3] [CWE-
319]' in this it's pointing about new URL() used but as
per documentation to create url in java we mainly use
URL object and then open the connection.
I want to know the reason why it's showing issue
regarding URL, while I'm using HttpsURLconnection.
Below reference is provided in report to solve issue but it did not worked.
https://developer.android.com/reference/java/sql/PreparedStatement

Related

Android Fresco SimpleDraweeView.setImageURI not working for this http url

mySimpleDraweeView.setImageURI(Uri.parse("http://cdn.cnn.com:80/cnn/.e1mo/img/4.0/logos/menu_politics.png"));
This is a url from link preview for cnn. It is an http url and not an https url. It has a port :80 in the path.
If I throw that url into a webbrowser, it gets resolved as "http://cdn.cnn.com/cnn/.e1mo/img/4.0/logos/menu_politics.png" and an image is there.
If I try to call setImageURI with "http://cdn.cnn.com/cnn/.e1mo/img/4.0/logos/menu_politics.png" from the web browser, it still doesn't work.
If I try to call setImageURI with "https://cdn.cnn.com/cnn/.e1mo/img/4.0/logos/menu_politics.png" it works.
I'm using a custom OKHttp3 OkHttpClient for Fresco. I tried using the getUnsafeOkHttpClient described here https://stackoverflow.com/a/25992879/1578222, but did not see a change in behavior with it.
I also tried setting the OkHttpClient.followRedirects but it did not fix it either:
httpClient.followSslRedirects(true);
httpClient.followRedirects(true);
I found a log message from the OKHttp3 client and that helped me figure out the problem:
<-- HTTP FAILED: java.net.UnknownServiceException: CLEARTEXT communication to cdn.cnn.com not permitted by network security policy
Solved my changing the Android Manifest file's Application object to include:
android:usesCleartextTraffic="true"
you can use any external library like Picasso and then show the image by using it

Hadoop WebHDFS Java Client API enable SSL and Basic Authentication

I have a Spring Boot application that uses spring-yarn-boot:2.2.0.RELEASE to get access to a Hadoop filesystem (HDFS). Operations that I do are LISTSTATUS, GETFILESTATUS and OPEN (to read a file). HDFS URI is specified through application.properties:
spring.hadoop.fsUri=webhdfs://127.0.0.1:50070/webhdfs/v1/
I make a bean to which I provide Hadoop Configuration (that Spring somehow automagically prepares for me on startup):
SimplerFileSystem fs = new SimplerFileSystem(FileSystem.get(configuration));
FsShell shell = new FsShell(configuration);
And everything works well as expected, but the problems came when I got two new requirements.
First thing is that HDFS will be protected with SSL from now on. I can't seem to find any way to tell my application that the fsURI that starts with webhdfs:// is actually a https connection. And if I will give the https URL directly, I'll get an exception:
java.io.IOException: No FileSystem for scheme: https
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
... which is caused by that code: FileSystem.get(configuration).
This thing is driving me crazy, I don't seem to find a way to get pass this.
Second requirement is, that I need to authenticate myself against the WebHDFS with basic authentication. For this I also can't find any means in the client API.
Has anyone done it before and have any instructions to share? Or maybe anyone knows a different client API that I can use to accomplish this?
One option is to implement the REST calls myself with RestTemplate or any other REST service consumer API, but this looks like not-so-special use case so I'm really hoping that there is something that has been done already.
EDIT:
Found a solution to the HTTPS problem. One should use swebhdfs:// as url prefix and everything will work. Still havent found a solution to the Basic Auth problem.

Http 502 Error when trying to get InputStream in my Spring Project

I am using a 3rd party Java API to get results from Google Search Appliance in my Spring based web project. The URL the API constructed seem correct. I've sent a request on web browser with generated URL by API and it worked (search results returned correctly). Moreover, I've tried the URL with standalone Java project that I created to get InputStream and it worked too. The problem comes out when I try to get results in my project. It crashes on the stage of creating InputStream and gives this error:
java.io.IOException: Server returned HTTP response code: 502 for URL: http://myIP:80/search?access=p&output=xml&client=default_frontend&lr=lang_en&num=100&requiredfields=gsaentity_language%3AEnglish&site=default_site
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
at java.net.URL.openStream(URL.java:1037)
at net.sf.gsaapi.GSAClient.search(Unknown Source)
at net.sf.gsaapi.GSAClient.getGSAResponse(Unknown Source)
at net.sf.gsaapi.GSAClient.getGSAResponse(Unknown Source)
The things I've tried that didn't work for me so far:
I've used the API's interface (overriding) to handle HttpConnection and creating InputStream manually. Via this interface, some proxy options can be added to HttpConnection. (The interface named GSAClientDelegate with method getResponseStream)
I've checked API's codes in case of finding something that causing proxy trouble etc. I couldn't find any something like that.
As stated on an answer, I've checked collection, frontend,proxystylesheet names and they are ok. Otherwise I couldn't reach the results via web browser or standalone Java project right?
It's been almost 1 week I keep facing this annoying and mysterious error and I'm starting to go mad.
What can I do about this issue?
I've finally figured out the base of the problem. It is because of the java.net.HttpUrlConnection API. It uses shared connection factory, which may cause the errors like this. I've replaced it with Apache HttpClient. And no problem showed up:
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
.........
public InputStream getConnectionStream(String fullUrl){
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(fullUrl);
HttpResponse response = client.execute(request);
InputStream is = response.getEntity().getContent();
return is;
}
I also had this problem and it turned out that I just forgot to encode a space in the URL (which a browser does for you silently). Replacing the space by %20 did the trick ...
I did not try HttpClient but it may encode correctly without asking.
First, this is a 5xx error, so the issue does not (generally) lie on any system that you control. If you're consistently getting this error, then the third-party API provider you're interfacing with is having technical difficulties, and you should reach out to them as soon as possible.
Since you may be going through a gateway or proxy, consider using a direct connection to the resource that you need.
You most likely made a typo in a parameter of your query.
A 502 error usually means you try to load a frontend (so check client / proxystylesheet parameter) or a collection (check parameter site. default_site ? Real default is default_collection if you didn't made a new one) that doesn't exist.

EWSHttpException: Connection not established

I'm using EWS Java API to connect Exchange server and retrieve information about mail, calendar appointment and task.
It's working well with a lot of user, except for one account.
I got the following error :
microsoft.exchange.webservices.data.EWSHttpException: Connection not established
at microsoft.exchange.webservices.data.HttpClientWebRequest.throwIfConnIsNull(HttpClientWebRequest.java:394)
at microsoft.exchange.webservices.data.HttpClientWebRequest.getResponseHeaders(HttpClientWebRequest.java:280)
at microsoft.exchange.webservices.data.ExchangeServiceBase.processHttpResponseHeaders(ExchangeServiceBase.java:1045)
at microsoft.exchange.webservices.data.SimpleServiceRequestBase.internalExecute(SimpleServiceRequestBase.java:58)
at microsoft.exchange.webservices.data.MultiResponseServiceRequest.execute(MultiResponseServiceRequest.java:144)
at microsoft.exchange.webservices.data.ExchangeService.bindToFolder(ExchangeService.java:350)
at microsoft.exchange.webservices.data.ExchangeService.bindToFolder(ExchangeService.java:374)
Here the code to establish the connection :
ExchangeService service = new ExchangeService();
ExchangeCredentials credentials = new WebCredentials(<user>, <password>);
service.setCredentials(credentials);
service.setUrl(new URI(url));
I suspect a specific account configuration for explaining this error but I'm unable to determine which parameter.
That exception is almost certainly due to a bug. I've seen it many, many times. The problem lies in the SimpleServiceRequest class. If there's an error when reading the response, it will close the response in a finally block in readResponse(). It will go back up to internalExecute(), where the catch block will try to process the headers...and it tries to read the response that has been closed. The closing won't null out the response, but it does null out some data in the response, which EWS tries to read as to display errors. Then you get another exception because the connection is null due to the response being closed earlier.
The solution is to either fix the bug yourself or enable tracing and look at the response to see what kind of error you're dealing with. Also, for good measure, make sure the Strings class is reading in the Strings.properties file or it'll throw a different exception when it can't find certain error messages.
After working with the debugger and Fiddler, one way I've seen this error coming in is from an HTTP 302 error (the server says the link has been moved permanently to an https: location instead of the almost identical http: location).
I'm going to guess that the Java EWS API is not using the Secure Sockets Layer correctly (and is attempting to send to an HTTP url instead of an HTTPS url).
EDIT
If you get past the 302 error, then you may very well have a problem with handling the SSL certificate properly later on. If you debug the API, you may be able to see one of those
"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
errors in on the stack in ServiceRequestBase.java. That means that a Truststore, somewhere cannot find a certificate, or it's not looking in the right place.
UPDATE
Check which NTLM flags are getting set in the EwsJCIFSNTLMScheme class. I've seen connections fail because these flags were getting set wrong.
Use something like Fiddler to automatically (and successfully) authenticate into your EWS instance, check and see what NTLM flags are getting set (by decoding the Authorization: Negotiate headers with Fiddler in the "Inspectors", "Auth" tab - it's a 32-bit hex number), and send those hex-valued flags into the Type1 and Type3 message constructors.
I have the same error in china.I think it's ews-java-api bug .so I check the github.com,I see the author Victor Boctor update the scrip.so I think maybe it can fix this bug.so try to compile the source code,and sure ,it fix this bug. ~_~ thanks for Victor Boctor

Uploading to Youtube via a proxy using the Java Youtube API

So I want to write a servlet which uploads a video to a youtube channel using the Java API, but I can't seem to find a way of specifying that I want to go through a proxy server. I've seen an example on this site where someone managed to do this using C#, but the Classes they used don't seem to exist in the Java API. Has anybody managed to successfully do this?
YouTubeService service = new YouTubeService(clientID, developerKey);
I'm new here so I'm unable to comment on posts (and a little late on this topic), but Jesper, I believe this is the C# sample that the original poster was talking about: How to upload to YouTube using the API via a Proxy Server
I can see no "direct" way of porting that example to Java though, since the GDataRequestFactory doesn't seem to have any proxy-related fields.
I was also having issues with the Java client Library with proxy in our application. Basically, the library picks up the global Java proxy settings:
System.getProperty("http.proxyHost");
System.getProperty("http.proxyPort");
but for some reason not everywhere. To be more precise, even with a proxy server properly configured in Java, YouTube authentication (calling service.setUserCredentials("login", "pwd")) would use a direct connection and ignore the proxy. But a video upload (calling service.insert(...)) would use the proxy correctly.
With the help of folks at the official YouTube API mailing list, I was able to nail this down. The issue is that the authentication is performed using SSL (HTTPS) and since there is a different set of properties for the HTTPS proxy, this didn't work. The fix is to simply set https.proxy* properties as well (in addition to http.proxy*), so that these point to a valid proxy server too:
System.getProperty("https.proxyHost");
System.getProperty("https.proxyPort");

Categories