I intend to write a method to download images. All implementations failed so far, so I boiled it down to the code that produces the error and removed everything else, so the code just shows the important parts. The point is: the getinputstream() method causes a connection error when I use a HTTPS URL. How can I circumvent that? Could it be because of certificates? I'm just surprised that similar code I used in C# worked, but Java needs to circumvent certificates for HTTPS connections?
public static void img_DL(String s) throws IOException {
URL url = new URL(s);
InputStream inputStream;
String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36";
URLConnection con = url.openConnection();
con.setRequestProperty("User-Agent", USER_AGENT);
inputStream = con.getInputStream();
inputStream.close();
}
Here the stacktrace:
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at application.Img.img_DL(Img.java:48)
at application.FXMLController.download_img(FXMLController.java:215)
... 58 more
You can check the details of the exception.
Run by setting -Djavax.net.debug=all to debug the error. See
Debugging SSL/TLS Connections
In case of timeout, check timeout settings. You may increase timeout by setting the system property.
System.setProperty("sun.net.client.defaultConnectTimeout", "10000");
Related
I am trying to execute a crawler program from my office. A very basic one which is available in internet and which works fine in my home PC. However while I am trying to run the same program in my office PC i am getting connect timed out error. I thought it was proxy problem and tried accessing some site from eclipse internal browser and it worked fine also.
Document doc = Jsoup.connect("http://flipkart.com/").timeout(0).get();
Please find below my stack trace
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:449)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:170)
at org.syntel.crawler.Crawler.processPage(Crawler.java:44)
at org.syntel.crawler.Crawler.main(Crawler.java:20)
How can I fix this problem?
#alkis made the suggestion:
Try setting a user agent. ff you are using a proxy check this other question:
How to add proxy support to Jsoup (HTML parser)?
Try using:
System.out.println("Testing JSOUP\n--------------");
Proxy proxy = new Proxy( //
Proxy.Type.HTTP, //
InetSocketAddress.createUnresolved("www.yourPROXY.com", 80) //
);
Document doc = Jsoup.connect("http://en.wikipedia.org/").proxy(proxy).get();
Elements newsHeadlines = doc.select("#mp-itn b a");
System.out.println(newsHeadlines.html());
how are you?
Well, the problem is that we are using Axis1 to consume a wsdl based webservice which works fine when the URL where the WSDL is located uses plain old HTTP connection, but when it uses a SSL secured conection it brings a ConnectionException when Axis1 tries to download the WSDL document content.
Even reading comments on XMLUtils.class the Axis developers aren't even sure if it will work with HTTPS as it reads on line 810.
Is there any way to solve this? Whe tried to install the certificates on the computer, on ...jre7/lib/security/cacerts and tried to trust all certificates but the problem persists...
Thanks in advance.
Edit:
You can reproduce the Exception with this code:
InputSource source = new InputSource(urlWSDL);
DocumentBuilder db = DocumentBuilderFactoryImpl.newInstance().newDocumentBuilder();
Document doc = (Document) db.parse(source);
The Exception is:
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
The problem was the Company Proxy, so I've appended:
System.setProperty("https.proxySet", "true");
System.setProperty("https.proxyHost", this.getConfiguracionProxy().getUrlProxy());
System.setProperty("https.proxyPort", this.getConfiguracionProxy().getPuertoProxy());
To:
System.setProperty("http.proxySet", "true");
System.setProperty("http.proxyHost", this.getConfiguracionProxy().getUrlProxy());
System.setProperty("http.proxyPort", this.getConfiguracionProxy().getPuertoProxy());
I've found the solution looking at Wireshark. When I was getting the file on SoapUI or on a web browser, the IP was other than the IP used by our application (the true IP). Then I realized that I was behind a proxy.
I've never used Wireshark... I've learned a lot, which is a good thing.
This sets as a System property the stored proxy configuration.
Thanks everybody.
I have tried adding proxy, nonProxyHost etc. in Eclipse but not able to pass through this error. I also read lot of pages around java.net.UnknownHostException but no luck. Below is the snippet and exception.
URL url = new URL("https://xyz.com/Something");
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection)connection;
ByteArrayOutputStream bout = new ByteArrayOutputStream();
OutputStream out = httpConn.getOutputStream(); //Fails with
//Exception in thread "main" java.net.UnknownHostException: xyz.com
Full stack trace:
Exception in thread "main" java.net.UnknownHostException: xyz.com
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
I have tried setting below proxies through code:
System.setProperty("useDefaultWebProxy", "true");
System.setProperty("https.proxyHost", "my_proxy_server");
System.setProperty("https.proxyPort", "8080");
System.setProperty("https.nonProxyHosts", "xyz.com");
Also, in Eclipse I have set proxy and proxy bypass in Window -> Preferences -> General -> Network Connection
I'm using this simple code below to send an http request:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static void main(String[] args) throws Exception {
URL yahoo = new URL("http://www.yahoo.com/");
URLConnection yc = yahoo.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
}
But it doesn't work. I've got always the same error:
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
My OS: Windows 7.
I can't find where is the problem.
I've no internet connection problem.
I've already tried by getting off my firewall but It does not solve the problem.
I think it's not a network problem because I've a similar c# code that works.
Check with your firewall setting.
If your problem was like mine, the issue may be a network security setup on the other end (wherever you're trying to talk to), not your own firewall.
I am using the method openStream with Java
in = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"));
How long is the method waiting of a response from the service where the URL is sended?
I am becoming same times errors messages like this and i don't know why.
java.net.UnknownHostException: dev.virtualearth.net
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
Thanks
openStream() method is a shortcut for URL.openConnection().getInputStream()
you can set the timeout by yourself:
URLConnection con = url.openConnection();
con.setConnectTimeout(XXX);
From the JDK:
"If the timeout expires before the connection can be established, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout."
The default timeout is 0
Your Exception "Thrown to indicate that the IP address of a host could not be determined."
It seems that you might be behind a firewall, so you can add:
System.setProperty("java.net.useSystemProxies", "true"); //if you have set system proxy
or
System.setProperty("http.proxySet","true");
System.setProperty("proxyPort","port");
System.setProperty("proxyHost","proxyhost");
System.setProperty("http.proxyUser", "user" );
System.setProperty("http.proxyPassword", "password" );
to the related java class
java.net.UnknownHostException: dev.virtualearth.net implies that your DNS configuration is incorrect since Java doesn't know how to resolve dev.virtualearth.net to an IP address. What is the string representation of the URL you are opening?