Given the code:
HttpURLConnection huc = (HttpURLConnection) new URL( url ).openConnection();
huc.setConnectTimeout( 10000 );
huc.connect();
how exactly the connection timeout is processed? Some HTTP header gets set or what? Or the connection status is being checked in a loop for connectionTimeout time?
I tried to find it in the source code, but there is only the long connectionTimout field...
Think of it as:
Inside connect first a parallel timer is run for the the connection timeout.
If the timer ends before the actual connection is established (response received), then fail.
In reality on most platforms the operating system can be parametrised with a timeout and will handle it oneself - in the same manner.
Not having seen the java native code, but there are POSIX methods like setsocketopt with which to set timeouts. POSIX connect will give a timeout.
In java the timeout was a later much desired addition to utilize these available timeouts.
Related
So I have a problem with a Java program I have. The program's basic functionality includes basically connecting to a web API for data. The function that does that is something like this:
public static Object getData(String sURL) throws IOException {
URL url = new URL(sURL);
URLConnection request = url.openConnection();
request.connect();
return request.getContent();
}
The code works fine as it is, but recently, after my house changed ISPs, I have found that sometimes the connections take an unreasonably long amount of time, something like 10 seconds or more in about 10% of attempts, while the other 90% takes only around 200ms. I have found it to be faster to ask my program to call the function again in a different thread than to wait for some of these connections to finally connect.
Therefore, I want to change the function so that if after 500ms, the connection did not establish, it would disconnect and a new connection would be attempted. How could I do this?
Somewhere online I read that HttpURLConnection might help, but I am not sure how.
URLConnection allows you to specify the connect and read timeout prior to calling connect():
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URLConnection.html#setConnectTimeout(int)
Sets a specified timeout value, in milliseconds, to be used when
opening a communications link to the resource referenced by this
URLConnection. If the timeout expires before the connection can be
established, a java.net.SocketTimeoutException is raised. A timeout of
zero is interpreted as an infinite timeout.
With 500ms timeout:
try {
URLConnection request = url.openConnection();
request.setConnectTimeout(500); // 500 ms
request.connect();
// on successful connection
} catch (SocketTimeoutException ex) {
// on request timeout
}
This you can pack into a loop, but I recommend limiting the number of attempts made.
Java's URLConnection doesn't have retry capabilities in Java 8 therefore the best way here to achieve this - use an appropriate standalone 3-party library such as Apache HttpClient.
This is by far the best standalone 3-party HTTP client with advanced capabilities as of 2020 and it's still maintained.
By default as of version 5.2.x Apache Http Client, Apache Http Client uses the default implementation of org.apache.http.client.HttpRequestRetryHandler, which retries 3 times, but you can use a custom implementation instead.
The configuration might look like this(full imports are for example's sake):
org.apache.http.client.HttpClient httpClient = org.apache.http.impl.client.HttpClients.custom()
.setRetryHandler(YourCustomImplOfTheRetryHandlerClass)
//other config
.build();
There is no way I can reproduce that problem using my ISP.
I suggest you dig deeper into the problem and find a better solution. Sending another request just doesn't seem good enough to me. Maybe try a different way to get the data and see if that works for you. Can't say for sure as I can't reproduce the problem.
I'm trying to properly configure the timeouts for my connections using HttpURLConnection.
My problem is that after the getResponseCode() call It always timeouts after 60 seconds instead of the value I set. My code:
URL url = new URL(uri.toString());
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setConnectTimeout(15000);
connection.setReadTimeout(15000);
int responseCode = connection.getResponseCode();
What do I am missing?
I also encountered the same kind of problem a few days back, So did some research.
My problem is that after the getResponseCode() call It always timeouts after 60 seconds instead of the value I set.
this is because InetAddress.getByName(String) does a DNS lookup. That lookup is not part of the connection
timeout.
The JDK doesn't let you specify a timeout here. It simply uses the timeouts of the underlying
name resolution mechanism.
Anyway, I suspect the effect is not limited to Java. You should be able to observe the same
timeouts using nslookup or the host command from a terminal. In a "normal" environment DNS
lookup timeouts should be of the order of 1-3 seconds, but not 20 seconds. So I strongly suspect
your network setup is broken.
Several things can lead to such insane timeouts:
DNS server not reachable (UDP port 53), but ICMP is filtered, so the client cannot fail fast
local firewall on DNS server dropping packets on closed TCP ports instead of sending RST
intermediate firewalls blocking ICMP messages
lookups performed over IPv6, but missing IPv6 connectivity
AAAA record lookups before A record lookup
your DNS server performs full recursion but no caching. Clients should always query a DNS cache, never a recursor only.
Workaround: You may perform the lookup before sending the request, so the result is already
pre-cached.
I'm in the process of writing two Junit testcases for testing the timemouts
#Test
public void connectionTimeoutTest()
{
String myurl = "http://serverip:serverport/context";
URL url = new URL(myurl);
URLConnection con = url.openConnection();
//how to check connectionTimeout but not socket time out
//what I think
//is it good enough to check with a server ip that does not exist ?
}
#Test
public void socketTimeoutTest()
{
String myurl = "http://localhost:serverport/context";
URL url = new URL(myurl);
URLConnection con = url.openConnection();
//how to check successful connection and timesout at socket (or port)
//what I think
//IP should exist (so it is localhost) but should not be listening on the port
}
I'm confused here because I'm not sure if my approach is right. What are the exceptions in each case ? Also, is it possible to differentiate based on Exceptions ?
Are there any other time outs that I have missed ?
Thanks in advance
You need to call URLConnection.setConnectTimeout() before opening any streams or getting the response code. It will throw a ConnectException with the text 'connection timed out' if a connect timeout happens.
Note that, contrary to the Javadoc, the default connect timeout is about a minute (platform-dependent), not infinity, and that you can decrease the default connection timeout this way, but not increase it.
For a read timeout, which I assume is what you mean by 'socket timeout', you have to call URLConnection.setReadTimeout(). If it fires, you will get a SocketTimeoutException. In this case zero does mean infinity as per the Javadoc.
I am not too clear on your unit test requirements. If you are to test the connection (and socket read) timeout values of the server, the approach mentioned above will only allow you to validate a certain "maximum" threshold value. But you still won't be able to confirm TCP server side timeout settings. The TCP server's connection (and read) timeouts depend upon factors such as platform (OS) and TCP stack implementation.
Again, the timeout setting via setConnectTimeout or setReadTimeout() are only on the client socket, when communicating with the resource referenced by this URLConnection.
In the first test case, I think it will get java.net.UnknownHostException, because of non-exist ip address.
If you wanna cause a socket connect timeout exception (exactly get messages like "java.net.SocketTimeoutException: connect timed out") , just use URLConnection.setConnectTimeout(10), set the parameter as smaller as possible ,but positive;
The same in the second test case, you can use URLConnection.setReadTimeout(10) to cause a read timeout exception ("java.net.SocketTimeoutException: Read timed out").
I'm using the XMLReader to simply read a feed like below.
URLConnection urlConnection = url.openConnection();
XmlReader reader = new XmlReader(urlConnection);
When this is called I receive within 5 seconds an IOException "Timeout
while fetching". So I tried to set the timeouts to the max. (10 sec) but still no luck and still an IOExeption in 5 sec.
urlConnection.setConnectTimeout(10000);
(the max is stated in documentation: http://code.google.com/intl/nl-NL/appengine/docs/java/urlfetch/overview.html)
Seems that the size of the feed is too large. When I call a smaller feed
it works properly. Is there any workaround or solution for this? I need to be able to call larger feeds.
You should use setReadTimeout method that sets the read deadline:
urlConnection.setReadTimeout(10000); //10 Sec
You should be able to download larger feeds in 10 seconds.
If you still have problem, try to fiddle with this different approach.
The reason is:
If no data is available for the read timeout period, exception can be thrown. From the doc of Oracle
A SocketTimeoutException can be thrown when reading from the returned input stream if the read timeout expires before data is available for read.
By the way, ReadTimeout is different with ConnectTimeout, the read timeout is the timeout to get data from the host, see different connection timeout and read timeout
So as #systempuntoout answer, need to set read timeout.
I'm getting a ConnectException: Connection timed out with some frequency from my code. The URL I am trying to hit is up. The same code works for some users, but not others. It seems like once one user starts to get this exception they continue to get the exception.
Here is the stack trace:
java.net.ConnectException: Connection timed out
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:516)
at java.net.Socket.connect(Socket.java:466)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
at sun.net.www.http.HttpClient.New(HttpClient.java:287)
at sun.net.www.http.HttpClient.New(HttpClient.java:299)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:796)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:748)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:673)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:840)
Here is a snippet from my code:
URLConnection urlConnection = null;
OutputStream outputStream = null;
OutputStreamWriter outputStreamWriter = null;
InputStream inputStream = null;
try {
URL url = new URL(urlBase);
urlConnection = url.openConnection();
urlConnection.setDoOutput(true);
outputStream = urlConnection.getOutputStream(); // exception occurs on this line
outputStreamWriter = new OutputStreamWriter(outputStream);
outputStreamWriter.write(urlString);
outputStreamWriter.flush();
inputStream = urlConnection.getInputStream();
String response = IOUtils.toString(inputStream);
return processResponse(urlString, urlBase, response);
} catch (IOException e) {
throw new Exception("Error querying url: " + urlString, e);
} finally {
IoUtil.close(inputStream);
IoUtil.close(outputStreamWriter);
IoUtil.close(outputStream);
}
Connection timeouts (assuming a local network and several client machines) typically result from
a) some kind of firewall on the way that simply eats the packets without telling the sender things like "No Route to host"
b) packet loss due to wrong network configuration or line overload
c) too many requests overloading the server
d) a small number of simultaneously available threads/processes on the server which leads to all of them being taken. This happens especially with requests that take a long time to run and may combine with c).
If the URL works fine in the web browser on the same machine, it might be that the Java code isn't using the HTTP proxy the browser is using for connecting to the URL.
The error message says it all: your connection timed out. This means your request did not get a response within some (default) timeframe. The reasons that no response was received is likely to be one of:
a) The IP/domain or port is incorrect
b) The IP/domain or port (i.e service) is down
c) The IP/domain is taking longer than your default timeout to respond
d) You have a firewall that is blocking requests or responses on whatever port you are using
e) You have a firewall that is blocking requests to that particular host
f) Your internet access is down
g) Your live-server is down i.e in case of "rest-API call".
Note that firewalls and port or IP blocking may be in place by your ISP
I'd recommend raising the connection timeout time before getting the output stream, like so:
urlConnection.setConnectTimeout(1000);
Where 1000 is in milliseconds (1000 milliseconds = 1 second).
try to do the Telnet to see any firewall issue
perform tracert/traceroute to find number of hops
I solved my problem with:
System.setProperty("https.proxyHost", "myProxy");
System.setProperty("https.proxyPort", "80");
or http.proxyHost...
Why would a “java.net.ConnectException: Connection timed out”
exception occur when URL is up?
Because the URLConnection (HttpURLConnection/HttpsURLConnection) is erratic. You can read about this here and here.
Our solution were two things:
a) set the ContentLength via setFixedLengthStreamingMode
b) catch any TimeoutException and retry if it failed.
This can be a IPv6 problem (the host publishes an IPv6 AAAA-Address and the users host thinks it is configured for IPv6 but it is actually not correctly connected). This can also be a network MTU problem, a firewall block, or the target host might publish different IP addresses (randomly or based on originators country) which are not all reachable. Or similliar network problems.
You cant do much besides setting a timeout and adding good error messages (especially printing out the hosts' resolved address). If you want to make it more robust add retry, parallel trying of all addresses and also look into name resolution caching (positive and negative) on the Java platform.
There is a possibility that your IP/host are blocked by the remote host, especially if it thinks you are hitting it too hard.
The reason why this happened to me was that a remote server was allowing only certain IP addressed but not its own, and I was trying to render the images from the server's URLs... so everything would simply halt, displaying the timeout error that you had...
Make sure that either the server is allowing its own IP, or that you are rendering things from some remote URL that actually exists.