I am using the method openStream with Java
in = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"));
How long is the method waiting of a response from the service where the URL is sended?
I am becoming same times errors messages like this and i don't know why.
java.net.UnknownHostException: dev.virtualearth.net
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
Thanks
openStream() method is a shortcut for URL.openConnection().getInputStream()
you can set the timeout by yourself:
URLConnection con = url.openConnection();
con.setConnectTimeout(XXX);
From the JDK:
"If the timeout expires before the connection can be established, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout."
The default timeout is 0
Your Exception "Thrown to indicate that the IP address of a host could not be determined."
It seems that you might be behind a firewall, so you can add:
System.setProperty("java.net.useSystemProxies", "true"); //if you have set system proxy
or
System.setProperty("http.proxySet","true");
System.setProperty("proxyPort","port");
System.setProperty("proxyHost","proxyhost");
System.setProperty("http.proxyUser", "user" );
System.setProperty("http.proxyPassword", "password" );
to the related java class
java.net.UnknownHostException: dev.virtualearth.net implies that your DNS configuration is incorrect since Java doesn't know how to resolve dev.virtualearth.net to an IP address. What is the string representation of the URL you are opening?
Related
I am trying to execute a crawler program from my office. A very basic one which is available in internet and which works fine in my home PC. However while I am trying to run the same program in my office PC i am getting connect timed out error. I thought it was proxy problem and tried accessing some site from eclipse internal browser and it worked fine also.
Document doc = Jsoup.connect("http://flipkart.com/").timeout(0).get();
Please find below my stack trace
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:449)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:170)
at org.syntel.crawler.Crawler.processPage(Crawler.java:44)
at org.syntel.crawler.Crawler.main(Crawler.java:20)
How can I fix this problem?
#alkis made the suggestion:
Try setting a user agent. ff you are using a proxy check this other question:
How to add proxy support to Jsoup (HTML parser)?
Try using:
System.out.println("Testing JSOUP\n--------------");
Proxy proxy = new Proxy( //
Proxy.Type.HTTP, //
InetSocketAddress.createUnresolved("www.yourPROXY.com", 80) //
);
Document doc = Jsoup.connect("http://en.wikipedia.org/").proxy(proxy).get();
Elements newsHeadlines = doc.select("#mp-itn b a");
System.out.println(newsHeadlines.html());
I have created a service that opens a lots of remote resources; for a lot, i mean over 200k / day, but i think that less than 1000 connections are opened at the same time. After i connect to the remote resource, i open an input stream to get the file content.
Sometimes i get the following exception
java.net.BindException: Address already in use: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.followRedirect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sitodove.varie.ExternalLinksChecker.getWebPage(ExternalLinksChecker.java:121)
at sitodove.varie.ExternalLinksChecker.run(ExternalLinksChecker.java:37)
This service starts correctly, opening just a few ports for limited needs because it needs to offer some services. While it is running, i get the above exception, what may be the cause?
A port is already being used and you try to bind it again. Check in your application if your are not trying to use a port twice.
Probably, you're trying to open a port that is already open.
open the terminal an type telnet to see if that port is already open. ie.
telnet localhost 9889
Trying ::1...
telnet: connect to address ::1: Connection refused
if this command returns Connection refused, that means the port is free.
how are you?
Well, the problem is that we are using Axis1 to consume a wsdl based webservice which works fine when the URL where the WSDL is located uses plain old HTTP connection, but when it uses a SSL secured conection it brings a ConnectionException when Axis1 tries to download the WSDL document content.
Even reading comments on XMLUtils.class the Axis developers aren't even sure if it will work with HTTPS as it reads on line 810.
Is there any way to solve this? Whe tried to install the certificates on the computer, on ...jre7/lib/security/cacerts and tried to trust all certificates but the problem persists...
Thanks in advance.
Edit:
You can reproduce the Exception with this code:
InputSource source = new InputSource(urlWSDL);
DocumentBuilder db = DocumentBuilderFactoryImpl.newInstance().newDocumentBuilder();
Document doc = (Document) db.parse(source);
The Exception is:
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
The problem was the Company Proxy, so I've appended:
System.setProperty("https.proxySet", "true");
System.setProperty("https.proxyHost", this.getConfiguracionProxy().getUrlProxy());
System.setProperty("https.proxyPort", this.getConfiguracionProxy().getPuertoProxy());
To:
System.setProperty("http.proxySet", "true");
System.setProperty("http.proxyHost", this.getConfiguracionProxy().getUrlProxy());
System.setProperty("http.proxyPort", this.getConfiguracionProxy().getPuertoProxy());
I've found the solution looking at Wireshark. When I was getting the file on SoapUI or on a web browser, the IP was other than the IP used by our application (the true IP). Then I realized that I was behind a proxy.
I've never used Wireshark... I've learned a lot, which is a good thing.
This sets as a System property the stored proxy configuration.
Thanks everybody.
I have tried adding proxy, nonProxyHost etc. in Eclipse but not able to pass through this error. I also read lot of pages around java.net.UnknownHostException but no luck. Below is the snippet and exception.
URL url = new URL("https://xyz.com/Something");
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection)connection;
ByteArrayOutputStream bout = new ByteArrayOutputStream();
OutputStream out = httpConn.getOutputStream(); //Fails with
//Exception in thread "main" java.net.UnknownHostException: xyz.com
Full stack trace:
Exception in thread "main" java.net.UnknownHostException: xyz.com
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
I have tried setting below proxies through code:
System.setProperty("useDefaultWebProxy", "true");
System.setProperty("https.proxyHost", "my_proxy_server");
System.setProperty("https.proxyPort", "8080");
System.setProperty("https.nonProxyHosts", "xyz.com");
Also, in Eclipse I have set proxy and proxy bypass in Window -> Preferences -> General -> Network Connection
I'm pretty new to java , actually I have just started learning it
I tried to do an exercise and the exercise was to read the first five lines of a webpage
for start I wrote this code :
import java.io.* ;
import java.net.URL ;
class testcode {
public static void main(String[] args) throws Exception {
URL address = new URL("http://www.yahoo.com/") ;
InputStream is = address.openStream() ;
InputStreamReader isr = new InputStreamReader(is) ;
BufferedReader reader = new BufferedReader(isr) ;
String line = reader.readLine() ;
}
}
but when I run this piece of code through Eclipse , I get this :
Exception in thread "main" java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at test.testcode.main(testcode.java:10)
Why is this happening !?
and ofcourse when I dont put the throws Exception part at the begining I get the malformed url exception !
PS : my internet connection works just fine !
Can Somebody please help me and explain why is this happening while doing that ?
I have a pretty good c++ background so feel free to explain as deep as you can :D
It seems like your program can't connect to the URL. Are u using internet behind proxy? If so, then make sure ur program is configured accordingly. One way is to use this code:
System.setProperty("http.proxyHost", "proxy.mydomain.com");
System.setPropery("http.proxyPort", "8080");