I'm having issues retrieving the content from some HTTPS sites using the Java 11 HTTPClient. Here's the code I'm using:
HttpClient client = HttpClient.newBuilder().build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://en.wikipedia.org/wiki/Main_Page"))
.build();
HttpResponse<byte[]> response = client.send(request, HttpResponse.BodyHandlers.ofByteArray());
System.out.println(response.statusCode());
System.out.println(new String(response.body()));
When trying to download the source of Wikipedia, I get the following error:
java.io.IOException: Received fatal alert: handshake_failure
If I use the same code but change the source URL to https://www.google.com, it works fine. From a bit of Googling, this makes it sound to me like it's probably an issue with differing crypto algorithms (although I don't know that), but it's not clear to me how to fix it. I've tried a few solutions that have been suggested on other StackOverflow questions, but haven't found one that works for me or a reliable guide to debugging the problem.
Can someone please point me in the direction of what I can try to fix the issue?
$ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04)
OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)
I don't know what caused problem on your environment, but just to test the URL I ran a different Http client using Java 8, and it worked just fine. Just in case you would want to try this Http client Here is the code and the info where to get the client:
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setConnectionUrl("https://en.wikipedia.org/wiki/Main_Page");
String result;
try {
ByteBuffer buff = client.sendHttpRequestForBinaryResponse(HttpMethod.GET);
result = new String(buff.array());
// result = client.sendHttpRequest(HttpMethod.GET);
System.out.println(result);
} catch (IOException e) {
result = TextUtils.getStacktrace(e, "com.mot.");
}
}
This code worked with both options: you can comment the two lines
ByteBuffer buff = client.sendHttpRequestForBinaryResponse(HttpMethod.GET);
result = new String(buff.array());
and uncomment the line
// result = client.sendHttpRequest(HttpMethod.GET);
And it also works.
The HttpClient used here comes from MgntUtils Open Source library. That library is available as Maven Artifacts Maven Central here And also on GitHub (including source code and Javadoc). Javadoc for The library could be seen here.
Disclaimer: The library is written by me
Related
I have written a J2ME application using WTK 2.5.2 with JDK 1.8.0_361 and JDK
1.5.0_22.
The following code snippet is used to connect to an HTTP server and it works as expected.
try{
http = (HttpConnection)Connector.open(url);
http.setRequestMethod(HttpConnection.POST);
OutputStream os = http.openOutputStream();
String params = "XXXXXXXX";
os.write(params.getBytes());
respCode = http.getResponseCode();
}catch(Exception e){
System.out.println(e);
}
However, when trying to connect to an HTTPS site, it throws one of these two errors:
Google:
java.io.IOException: Bad record type (21) or version (3.1)
Wikipedia:
java.io.IOException: Alert (2,40)
I have tried using HttpsConnection and switching to another WTK and JRE version.
Exporting the application to device shows another error:
java.io.IOException: Alert (2, 70)
Thanks in advance.
I’m just learning Java; please bear with my current lack of knowledge. I’m trying to write a program to scrape publicly available data from various web sites. I’m using a simple Java class to start the development of this web scraper (see SimpleWebScraper class below). This class works well for some sites (e.g., https://www.codetriage.com/), but others (e.g., https://forum.mrmoneymustache.com/ask-a-mustachian/) return an SSL handshake fatal alert (see below).
I’m running on macOS V10.14.6, Java SE Runtime Environment (build 1.7.0_51-b13), Java HotSpot 64-Bit Server VM (build 24.51-b03, mixed mode), and using IntelliJ IDEA 2019.2 Community Edition (runtime version 11.0.3+12-b304.10 x86_64).
Can you please suggest changes I should make to this code or my operating environment to address this fatal alert message?
Thanks in advance for your help.
Simple Web Scraper:
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
public class SimpleWebScraper {
public static void main(String[] args)
{
// Determine the number of command line arguments passed to the program...
int numberOfCommandLineArguments = args.length;
if (numberOfCommandLineArguments < 1) {
System.out.println("\nYou failed to include the URL for the web site you want scraped in the command line.\n");
} else {
// Initialize the string that holds the URL for the web site we want to scrape
String webSiteURL = args[0];
System.out.printf("\nTrying to retrieve the title from the URL: %s\n", webSiteURL);
try {
// Create a document object and use JSoup to fetch the website
Document webSiteDocument = Jsoup.connect(webSiteURL).get();
// Use JSoup's title() method to fetch the title
System.out.printf("\tTitle: %s\n\n", webSiteDocument.title());
// In case of any IO errors, we want the messages written to the console
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Fatal Alert Message:
Trying to retrieve the title from the URL: https://forum.mrmoneymustache.com/ask-a-mustachian/
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1959)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1077)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:746)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:722)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:306)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:295)
at SimpleWebScraper.main(SimpleWebScraper.java:22)
Process finished with exit code 0
I updated the JDK to v1.8, changed the required macOS environment variables to point to the V1.8 of the JDK, and the set the parameters for the IntelliJ IDEA IDE to use the V1.8 JDK. Together, these changes eliminated the error message and the code I included in my questions now functions as designed. :-)
I am trying to make a HTTP POST request to a server with https. When I was testing making the call I did it on a standalone java program (not deployed on a server) and initaially got the error:
unable to find valid certification path to requested target
however I fixed this with creating a cacert using a tool online called InstallCert and pointing java there with:
System.setProperty("javax.net.ssl.trustStore", "C:\\Users\\...\\jssecacerts");
However, when I implement the same fix in the code that I deploy on Apache Tomcat it doesnt work and I get the same error as before. So my question is do .war files deployed on Apache Tomcat use a different trust store than the java one? Or do they not use the system properties? How can I get Apache Tomcat to use the cacert?
Thanks
EDIT:
So I've tried to do some more research and get it fixed with no success. I tried following http://andyarismendi.blogspot.co.uk/2012/01/changing-tomcats-ca-trust-keystore-file.html to try and get it fixed.
Changing the server.xml file made no difference. However, changing the Tomcat Java Options with
-Djavax.next.ssl.trustStore="myFile"
Altered the response that I got, with this I get:
java.io.IOException: Server returned response code 500 for URL websitesURL
I also saw in some places that copying the jssecacerts file into my jre\lib\security directory could fix this. However this just cause the response code 500 error...
This is the code that is being used to make the POST Request:
//System.setProperty("javax.net.ssl.trustStore", "C:\\file\\path\\jssecacerts");
try
{
OAuthClient client = new OAuthClient(new URLConnectionClient());
//grant_type=password
OAuthClientRequest.TokenRequestBuilder trb = new OAuthClientRequest.TokenRequestBuilder(TOKEN_URL)
.setGrantType(GrantType.PASSWORD)
.setClientId(CLIENT_ID)
.setUsername(username)
.setPassword(password);
OAuthClientRequest ocr = trb.buildBodyMessage();
ocr.setHeader("Authorization", base64EncodedBasicAuthentication(CLIENT_ID, CLIENT_SECRET));
String responseBody = client.accessToken(ocr, OAuthJSONAccessTokenResponse.class).getBody();
return responseBody;
}
catch(Exception e)
{
e.printStackTrace();
return e.getMessage();
}
I am getting "peer not authenticated" using google map api's after creating javafx package.
It is working fine when directly run from code or from executable jar file, The following is the exception I am getting,
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:397)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:561)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
at com.precisionhawk.flightplanner.utils.LocationUtils.performAPICall(LocationUtils.java:109)
at com.precisionhawk.flightplanner.utils.LocationUtils.getAutocompleteResults(LocationUtils.java:74)
at com.precisionhawk.flightplanner.controls.AddressTextField$1$1.run(AddressTextField.java:103)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
I am using http client for calling the api. I have tried bypassing the X509 certificate but still no luck
below is my code
SSLContext ctx = null;
try {
SSLUtilities.trustAllHostnames();
SSLUtilities.trustAllHttpsCertificates();
HttpClient httpClient = new DefaultHttpClient();
HttpGet getRequest = new HttpGet(url.toString());
getRequest.addHeader("accept", "application/json");
HttpResponse response = httpClient.execute(getRequest);
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ response.getStatusLine().getStatusCode());
}
BufferedReader br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
String inputLine;
while ((inputLine = br.readLine()) != null) {
result = result + inputLine;
}
br.close();
LOG.info("Autocomplete API call Response: " + result);
}
catch (Exception e) {
LOG.error("Autocomplete Exception: "+e.getMessage());
}
If by "creating javafx package" you mean creating a Self-Contained Application, then a reason for this is that the sun cryptography extensions are not included with the default self-contained application packaging as detailed in the JavaFX deployment blog.
Quoting from the deployment blog, there are instructions on including the sun jce provider with the self-contained application for JavaFX 2.2. Likely for future releases of the JavaFX deployment tools, a more convenient way will be provided to do this.
Fine tuning application bundle
If you are using packaging tools to produce an installable package there could be a need to tweak the application image before it is wrapped into the installer. Why? For example you may want to sign the application, so it does not appear to be untrusted to the OS (for example to please Mac OS X Gatekeeper).
Also by default a self-contained application does not contain full copy of Java Runtime. We only include set of mandatory components. Part of the reason why this approach was taken is that we want to reduce the package size. However, there are situations where your application may depend on these optional components and in that case you will need a way to add them to the private runtime. For example https connections will not work if jre/lib/ext/sunjce_provider.jar is missing.
Currently this can be achieved by providing a custom config script that is executed after application image is populated. Like in the example above with the icon, you need to enable verbose output to find the name of the script file and then drop it to the location where packaging tools will find it. Note that scripting language is platform specific too. Currently we only support shell for Mac/Linux and Windows Script on windows.
How do you find out where the application image is located? Currently custom scripts are run in the directory where config files are stored but application image can be accessed using relative platform specific path. You can derive this path from verbose output or by setting environment variable JAVAFX_ANT_DEBUG to true to keep intermediate build artifacts.
Here is sample script (contributed by John Petersen) you can use to add jre/lib/ext/sunjce_provider.jar to the application package of MyApp on the Windows platform. Script using Javascript but you could also use VBScript for Windows scripting.
<?xml version="1.0" ?>
<package>
<job id="postImage">
<script language="JScript">
<![CDATA[
var oFSO = new ActiveXObject("Scripting.FileSystemObject");
var oFolder = oFSO.getFolder(".");
var from = oFolder.path + "\\MyApp\\app\\sunjce_provider.jar";
var to = oFolder.path + "\\MyApp\\runtime\\jre\\lib\\ext";
if (!oFSO.FolderExists(to)) {
oFSO.CreateFolder(to);
}
to += "\\";
oFSO.CopyFile(from, to);
]]>
</script>
</job>
</package>
I am using maven. so how should i call this script from pom.xml that will create the jar within jre/lib/ext folder?
I am not a maven expert. The solution above is for apache ant. You will need to work out your own solution to this issue if you wish to continue using maven.
I create web service
#WebService(serviceName = "DynamipsService2")
#Stateless()
public class DynamipsService2 {
#WebMethod(operationName = "StartSession")
public static String StartSession(#WebParam(name = "key") String key) {
try {
return "100-Session started";
} catch (Exception ex) {
return null;
}
}
}
I want to test but on the page
http://localhost:8080/DynamipsService2/DynamipsService2?Tester crash bug
Error generating artifacts for the
following WSDL
http://localhost:8080/DynamipsService2/DynamipsService2?WSDL
Possible causes can be invoking https
when the application is not configured
for security
I created other Web services in the same assembly and it works.
I just experienced this problem as well. The solution for me was to use my hostname rather than localhost in the URL for the Tester.
So in my case, the following, which is what NetBeans/Glassfish did by default when I clicked Test Web Service in the NetBeans UI, did not work:
http://localhost:8080/Calculator/Calculator?Tester
However, when I paste the following into a browser, it does work:
http://david-pc:8080/Calculator/Calculator?Tester
I couldn't figure out how to change the hostname that NetBeans uses for the built-in Test dialog (nor could I cut+paste the URL from the error dialog). So I had to visually copy the URL from the error message into a browser, replacing the hostname along the way.
I had the same problem and the reason apeared in the Server's log. I'm useing Glassfish 3.1 with netBeans 7. And the error I got in the Glassfish output was:
INFO: [ERROR] com.sun.tools.javac.Main is not available in the classpath, requires Suns JDK version 5.0 or latter.
I googled a bit and it appeared to be because the glassfish server was working with the openjdk that came with ubuntu. If your problem is the same the solution I found was to remove the openjdk jre, like this:
sudo apt-get remove openjdk-6-jre
sudo apt-get autoremove
Hope this is useful.
PS: I assigned /usr/lib/jvm/java-6-sun/bin/java in the java tab at the servers configuration wizard in netBeans but don't know if that was part of the solution (I'm afraid to change it back :p)
I had the same problem with Glassfish
but it was more compilcated because of the NAT
I have GF in NAT - do MYDOMAIN and port is redirected to internal machine
the problem in GF is that it tries to connect to itself by the domain name which again redirects to inside network
(the ?wsdl works properly)
I have made temp solution adding to /etc/hosts (127.0.0.1 domainname)
be aware that it's only a temp solution
try to check if you have "localhost" in your hosts file (in windows c:/windows/system32/drivers/etc/hosts )
and ping localhost
I will add GF log - maybe someone in future will search it by google :) :)
moreover I looked GF logs there was something like ->
Connection timed out
Failed to read the WSDL document: http://MYDOMAIN:8080/MYSERVICE?WSDL, because 1) could not find the document; /2) the document could not be read; 3) the root element of the document is not <wsdl:definitions>.
failed.noservice=Could not find wsdl:service in the provided WSDL(s):
At least one WSDL with at least one service definition needs to be provided.
Failed to parse the WSDL.
Which web server do you use? If you use glassfish you can open server admin page and select Configurations===>server-config===>Security
and make Security Manager enabled
Check your domains/{YOUR_DOMAIN}/config/domain.xml
If you setup Glassfish using Eclipse, all will be done for you automatically.
Now I am surprised if I start the domain from the command line, it gave me that error, but starting Glassfish 4 from Eclipse, it is not showing any problem.
One cause could be that you don't have correctly configured the environment variable JAVA_HOME (with the correct path) and the JAVA_HOME/bin directory added to global PATH environment variable as well. For some processes the glassfish look for the classpath of the JDK installed.
Hope this help.
I had the exact same problem and it is because you have a static method, I realised after debugging for some while. Just remove static from the method and it should work.
#WebMethod(operationName = "StartSession")
public String StartSession(#WebParam(name = "key") String key) {
try {
return "100-Session started";
} catch (Exception ex) {
return null;
}
}