I have set up a local proxy server for request logging but my java code ignores it and connects directly (Windows XP, JDK 1.7). Web browsers work with it. So I wrote test code for discussion that seems to connect directly even if a (bogus) proxy is specified. With the bogus proxy, I would expect connection failure but the code succeeds, connecting directly:
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "12345");
System.setProperty("http.nonProxyHosts", "noNonProxyHost.com");
URL url = new URL("http://docs.oracle.com/javase/7/docs/technotes/guides/net/proxies.html");
InputStream in = url.openStream();
System.out.println("Connection via bogus proxy succeeded");
The code is run as standalone Java, no Maven, no applet, no container. I have a direct internet connection.
In your case using java.net.URL(), if the proxy server cannot be reached at http.proxyHost and http.proxyPort then it simply falls back and tries to do a direct connect. If that succeeds, you'll see no exception thrown which is why your code works without error. You should see a pause while it tries to find the proxy though.
This sample code below happily fetches the URL and displays it, without error, even when run with bogus proxy settings. -Dhttp.proxyHost=bogus -Dhttp.proxyPort=2345 but will talk to my local proxy localhost port 8888 if set correctly
import java.io.*;
import java.net.URL;
import java.util.*;
public class URLClient {
private static String sUrl = "http://www.apache.org/";
public static void main(String[] args) {
try {
URL url = new URL(sUrl);
InputStream is = url.openStream();
java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\\A");
String output = s.hasNext() ? s.next() : "";
System.out.println(output);
} catch(Throwable e) {
System.err.println("exception");
}
}
}
The problem I was originally having with http.proxyHost and http.proxyPort being ignored (Google led me to your question) was that those settings are completely ignored by apache.commons.httpClient because it uses its own sockets, as described here.
http://cephas.net/blog/2007/11/14/java-commons-http-client-and-http-proxies/
I have faced a similar problem recently. First of all, one part of the above answer from Daemon42 explains pretty well, why the bogus proxy server didn't lead to a failure of the program:
if the proxy server cannot be reached at http.proxyHost and http.proxyPort then it simply falls back and tries to do a direct connect. If that succeeds, you'll see no exception thrown which is why your code works without error. You should see a pause while it tries to find the proxy though.
Still, your actual question was, why the proxy server configured via the operating system is not used by the Java application. As stated in the Oracle documentation (https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html), the system proxy settings are not evaluated by Java by default. To do so, you have to set the value of the system property "java.net.useSystemProxies" to the value "true".
You can set that system property on the command line, or you can edit the JRE installation file jre/lib/net.properties, that way you have to change it only once on a given system.
Related
I'm facing strange behavior when i request my own ping server with java HttpComponent client.
Sometimes, an unknownHostException is thrown with no reason.
This exception is principally thrown after switching network, for example when i changed the default network route (from eth0 to wifi or from wifi to other mobile NIC)
For information, my wifi connection is enabled through mobile access point. (Could this point reason of my issue ? )
I'm running on a linux OS, in embededded context with limited linux command.
Below, code example:
HttpGet get = new HttpGet("someUri");
if (networkInterfaceName != null) { // used to ping through specific route
get.setConfig(RequestConfig.custom()
.setLocalAddress(networkInterfaceToInetAdress(networkInterfaceName ))
.setConnectionRequestTimeout(15000)
.setConnectTimeout(15000)
.setSocketTimeout(15000).build());
}
HttpResponse response = HttpClientBuilder.create().useSystemProperties().build().execute(get);
So, if a specific interface is provided, local address is set in the requestConfig, otherwise we use the default route.
I'm currently not abled to identify the root cause and some actions are tried to identify the root cause.
First action, i checked on each httpRequest, route and /etc/resolv.conf and everything seems ok. Ping cmd result is also ok when unknown host exception is thrown.
I checked the httpClient code, and it seems that it create a new client on each call, so there is no httpClient cache for me.I used HttpClientBuilder class and it seems that create method call return always a new builder. Also, cloaseableHttpClient is not closed explicitly, should it impact next call ?
I checked httpClient method, it enable setting a dnsResolver, but in my case, but i've no control to remote/targeted ip.
I set JVM system properties in code (and not as Java ARGS directly):
java.security.Security.setProperty("networkaddress.cache.ttl", "0");
java.security.Security.setProperty("networkaddress.cache.negative.ttl", "0");
System.setProperty("java.net.preferIPv4Stack", "true");
System.setProperty("sun.net.inetaddr.ttl", "0");
Also, i'm going to use dnsjava lib when unknowhostexception is thrown as dig/dnslookup are not available in my limited OS.
Any hint about this issue ? or other thing that i need to ckeck ? may be it's not a DNS issue but something in Java socket or connection that i passed by ?
UPDATE
I tried to run same logic in another JVM and httpGet request are ok while httpGet fail in my original program that is running in another JVM
Thanks for your help.
i need to send some messages from my java web application to some servers using Diameter protocol, in particular CCR-CCA scenario. I had a look at jdiameter opensource project, but my usecase does not require such complexity, since that i just need to send a single request and log the response (actually i don't even need the CER-CEA part).
So i thought i could just have used Seagull running under my webapp. I downloaded Seagull (for Windows), and what i'm trying to do is basically to run the .bat file coming from Seagull for the diameter environment from my java environment.
That's what i've done till now..
1) A simple test to invoke the client.. Here wrapper simply sets working dir and starts the process
public static void main(String[] args) {
List<String> cmd=new ArrayList<>();
cmd.add("cmd.exe");
cmd.add("/c");
cmd.add("my_start_client.bat");
JavaProcessBuilderWrapper wrapper = new JavaProcessBuilderWrapper();
Process p = wrapper.createProcess(RedirectErrorsTo.STDERR,
new HashMap<>(), new File("my_working_dir"), cmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
StringBuilder output = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
output.append(line);
}
System.out.println(line);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
2) I modified the client's and server's .bat files coming from Seagull to use CCR-CCA protocol.
Running Java main with this configuration caused a
Fatal: Keyboard saved configuration failure error
on my logs.
3) So, as mentioned here i further modified my client's .bat file to run in background mode, adding -bg at the end. Now my client's bat look like this
#ECHO OFF
rem
"Diameter Start Script Sample"
"Local env"
SET RUN_DIR=C:\Program Files\Seagull
set PATH=%PATH%;%RUN_DIR%
set LD_LIBRARY_PATH=%RUN_DIR%
set RUN_DIR=%RUN_DIR%\diameter-env\run
cd %RUN_DIR%
cls
mode 81,25
echo "Seagull Diameter Client Sample Start"
seagull -conf ..\config\conf.client.xml -dico ..\config\base_ro_3gpp.xml -scen ..\scenario\ccr-cca.ro.client.xml -log ..\logs\ccr-cca.client.log -llevel ETM -bg
pause
Since i was facing some troubles, to keep things simple, i just tried to make it work at least via cmd (not using my java method), but i think background mode is messing around, because now when i start my server and then my client in bg mode, sometimes i get a
Fatal: Forking error
but the most of the times, the client send a single message and then on my console i see that my software is causing connection abort (error code -1), and from the log i see that the channel just get closed, and my client does not even receive an answer. (NB for now i left the configuration files untouched)
Has any of you faced this behaviour? Is something else closing the connection (firewall perhaps)? Do i have to provide other configurations to make this work?
Once i can get this working, can i use my java web app (with a method similar to the one i already mentioned) to make diameter calls?
Thanks in advance, any help is really welcomed.
I am using code that is similar to the code in this question.
A copy of the code form the question is with properties commented out, as I have mine commented out.
import javax.ws.rs.client.Client;
public static void main(String[] args)
{
Client client = ClientBuilder.newClient();
//client.property(ClientProperties.CONNECT_TIMEOUT, 1000);
//client.property(ClientProperties.READ_TIMEOUT, 1000);
WebTarget target = client.target("http://1.2.3.4:8080");
target = target.queryParam("paramname", "paramvalue");
target = target.queryParam("paramname2", "paramvalue2");
try
{
String responseMsg;
for (int i = 0; i < 50; i++)
responseMsg = target.request(MediaType.APPLICATION_XML).get();
System.out.println("responseMsg: " + responseMsg);
}
catch (ProcessingException pe)
{
pe.printStackTrace();
}
}
I modified the original code slightly by adding in a for-loop. The idea is that Jersey only generates one connection, not 50.
The problem that I have is that the daemon with which I communicate reports that I create a new connection with each call.
How can I have just one connection and then use that for each communication transaction?
At worst, I would like to close the connection, but that seems silly. There is a lot of overhead to creating a connection (on the daemon if nothing else and closing it).
The daemon reports "connection allowed" on the terminal window (CENTOS 7, but does not matter). I run the client usually off of my Windows 7 desktop. I am using Java 8 with Eclipse Luna. What happens quite frequently is that the daemon will say "maximum number of connections reached" and the proceed to do not nice things.
I have not tested fully yet, however the answer is in this other StackOverflow ticket.
I have to use an ApacheConnectorProvider object.
The Jersey help documentation, section 5.5 states:
In a simple environment, setting the property before creating the first target is sufficient, but in complex
environments (such as application servers), where some poolable connections might exist before your
application even bootstraps, this approach is not 100% reliable and we recommend using a different client
transport connector, such as Apache Connector. These limitations have to be considered especially when
invoking CORS (Cross Origin Resource Sharing) requests.
I am doing cross original resource sharing, so the simple method that I used is not stable. Using the Apache Connector on my small applet worked. I was able to use a for-loop with an iteration of 500 and no issues, just have to try the real code now.
I'm trying to write a small java program that connects to a twitter search URL (which returns a JSON list of tweets) using the URL connection libary.
My code which is taken from the java tutorials looks like :
public static void main(String[] args) throws Exception {
URL oracle = new URL("http://search.twitter.com/search.json?q=hi");
URLConnection yc = oracle.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
but for some reason I keep getting the following Exception:
in thread "main" java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
I don't know if this is something due to the way I've written the code, and eclipse setting or something to do with my network. I do have a proxy server configured for internet access. As far as I know this is properly configured because I am getting updates and can install new software through eclipse. Do I need to put the proxy information in the URL method somehow or is something else the problem.
URL rely on System properties for proxy, try setting proxy like this:
System.setProperty("http.proxyHost", "yourproxyserver");
System.setProperty("http.proxyPort", "portnumber");
Unfortunately, a correct proxy setup in Eclipse seems not to help with proxying Java programs started in Eclipse. Similarly, setting the Java system settings to use the systemwide proxy settings doesn't either. Not when you have a proxy that requires authentication, anyway.
As Thomas Johan Eggum said, if you have a "normal," non-authenticating proxy then setting the two JVM variables http.proxyHost and http.proxyPort either in the command line via -D or programmatically (see below) will take care of things.
For an authenticating proxy server, i.e. one that wants to see a user ID and password, many people recommend setting http.proxyUser and http.proxyPassword. That's bad advice, because these don't work. Apparently they are not defined in the Java docs.
Unfortunately, it looks like the way to "do" authentication is to use an Authenticator, programmatically. If you're going to do that, you might as well do the whole thing programmatically, i.e. including host and port. Here's how I got that to work:
public static void main(String[] args) {
try {
System.setProperty("http.proxyHost", "my.proxy.host");
System.setProperty("http.proxyPort", "8080-or-whatever-proxy-port");
Authenticator.setDefault(new DummyAuthenticator());
/* do your main program stuff */
} catch (Exception e) {
e.printStackTrace();
}
}
private static class DummyAuthenticator extends Authenticator {
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(
"my-user-id", "my-password".toCharArray()
);
}
}
I have a very simple code that uses HttpURLConnection to access some web site via proxy
System.setProperty("java.net.useSystemProxies", "true");
System.out.println("Proxy: " + ProxySelector.getDefault().select(new URI(urlS)));
URL url = new URL(urlS);
HttpURLConnection ic = (HttpURLConnection)url.openConnection();
ic.connect();
For some reason, Java thinks that I need SOCKS proxy, not http, throwing the following exception:
ERROR: Can't connect to SOCKS proxy:Connection timed out: connect
If you are having this issues on Windows, you may run into a Java bug.
Java treats any system proxy setting as SOCKS. You have to either disable useSystemProxies or don't use proxy in Windows.
If proxy is needed, try to uncheck "Use the same proxy server for all protocols", making sure the field for the SOCKS proxy is blank. That fixed our problem.
The real problem is that Java assumes that the "Use the same proxy server for all protocols" check affects SOCKS proxy too (I don't know the logic behind this dialog in Windows, but it is, at least, confusing)
If the check is set, you get proxies enabled for both HTTP and SOCKS, wich is very unlikely to be the desired configuration.
One way to solve it is unchecking the check and leaving blank the SOCKS field.
I finally solved it creating a ProxySelector wich first calls the default selector and if it finds the same configuration for HTTP and SOCKS connections, it omits the SOCKS proxy.
public class SocksFixerProxySelector extends ProxySelector {
ProxySelector base;
public SocksFixerProxySelector() {
base = ProxySelector.getDefault();
}
#Override
public List<Proxy> select(URI uri) {
List<Proxy> baseList = base.select(uri);
try {
if (uri.getScheme().equals("socket")) {
Proxy socksProxy = findByType(baseList, Type.SOCKS);
if (socksProxy != null) {
URI httpTestUri = new URI("http", uri.getHost(), uri.getPath(), uri.getFragment());
Proxy httpProxy = findByType(base.select(httpTestUri), Type.HTTP);
if (httpProxy != null && socksProxy.address().equals(httpProxy.address())) {
// Quitamos SOCKS
List<Proxy> filteredList = new ArrayList<>(baseList);
filteredList.remove(socksProxy);
return filteredList;
}
}
}
} catch (Exception e) {
}
return baseList;
}
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
base.connectFailed(uri, sa, ioe);
}
private Proxy findByType(List<Proxy> proxies, Proxy.Type type) {
for (Proxy proxy : proxies) {
if (proxy.type() == type)
return proxy;
}
return null;
}
Maybe a better solution would be to inspect the registry and detect the right settings, but I didn't want to mess with Windows specific code (and all those script settings looked bad, too )
You need to use the http.proxyHost system property instead. See http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html for details.
java -Dhttp.proxyHost=webcache.mydomain.com GetURL
Check that something has not set the "socksProxyHost" property in the Systems properties.
EDIT
The "useSystemProxies" property is described thus:
"On recent Windows systems and on Gnome 2.x platforms it is possible to tell the default ProxySelector to use the system proxy settings (both recent versions of Windows and Gnome 2.x let you set proxies globally through their user interface). If the system property java.net.useSystemProxies is set to true (by default it is set to false for compatibility sake), then the default ProxySelector will try to use these settings."
So, assuming that you have not supplied your own ProxySelector class, you should also check the system proxy settings to ensure that they don't say to use SOCKS.