How to make all Java connections to use proxy provided via JAVA_TOOL_OPTIONS environment variable?
The simple app I'm using as a test is taken from GitHub:
package to.noc.sslping;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import java.io.OutputStream;
public class SSLPing {
public static void main(String[] args) {
if (args.length != 2) {
System.out.println("Usage: java -jar SSLPing.jar <host> <port>");
System.exit(1);
}
try {
String hostname = args[0];
int port = Integer.parseInt(args[1]);
System.out.println("About to connect to '" + hostname + "' on port " + port);
SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket(hostname, port);
// Hostname verification is not done by default in Java with raw SSL connections.
// The next 3 lines enable it.
SSLParameters sslParams = new SSLParameters();
sslParams.setEndpointIdentificationAlgorithm("HTTPS");
sslsocket.setSSLParameters(sslParams);
// we only send 1 byte, so don't buffer
sslsocket.setTcpNoDelay(true);
// Write a test byte to trigger the SSL handshake
OutputStream out = sslsocket.getOutputStream();
out.write(1);
// If no exception happened, we connected successfully
System.out.println("Successfully connected");
} catch (Exception e) {
e.printStackTrace();
}
}
}
What I want is to be able to provide the PROXY settings via environment variables without having to configure it in the Java code.
I found that it is possible to provide some settings via the JAVA_TOOL_OPTIONS env.
JAVA_TOOL_OPTIONS="-Dhttp.proxyHost=161.xxx.xxx.xxx
-Dhttp.proxyPort=8080
-Dhttp.proxySet=true
-Dhttps.proxyHost=161.xxx.xxx.xxx
-Dhttps.proxyPort=8080
-Dhttps.proxySet=true"
It is correctly seen by the command
java -jar SSLPing.jar google.com 443
Picked up JAVA_TOOL_OPTIONS: -Dhttp.proxyHost=161.xxx.xxx.xxx
-Dhttp.proxyPort=8080
-Dhttp.proxySet=true
-Dhttps.proxyHost=161.xxx.xxx.xxx
-Dhttps.proxyPort=8080
-Dhttps.proxySet=true
About to connect to 'google.com' on port 443
Successfully connected
However when I need to reach a particular URL that requires the proxy, it fails to connect.
How do I make any socket to use the proxy from JAVA_TOOL_OPTIONS env?
How to check if the sockets are using the proxy?
I have been asked to make the secure FTP Server connection. FTP over explicit TLS/SSL. In order to achieve this, I have added a below line in the existing implementation. This is my FTPClient lib - it.sauronsoftware.ftp4j - version 1.6.1
ftpClient.setSecurity(FTPClient.SECURITY_FTPES);
// Existing code
ftpClient.setConnector(new HTTPTunnelConnector(proxyHost, proxyPort));
ftpClient.connect(getFtpHost(), getFtpPort());
ftpClient.login(getUsername(), getPassword());
ftpClient.setAutoNoopTimeout(30000);
When I deployed the code on JBOSS 5.1, I am getting successful connection, but I am unable to list the files in the root directory. we only have permission to access the root directory.
On the other hand, I have written a standalone client (java main program) - through which I can print the files present at FTP location, by this way I have ensured the secure connection and the files availability at FTP location. Here, I have used the
My problem is, when I make a secure connection through the deployed application I am getting the unable to locate any files at remote location. ie: FTPFiles.length is 0
Any help would be much appreciated, thank you!
Adding few more logs and info,
Normal Flow without FTPES security parameter added (current Implementation)
printing ftpClient.serverStatus()
msg: Status of 'FTP Service'
msg: Connected from ec2-xyz
msg: Logged in as <user>
msg: TYPE: BINARY, STRUcture: File, Mode: Stream
msg: Total bytes transferred for session: 10511
msg: No data connection
msg: End of status
printing ftpClient.serverStatus() With FTPES added
msg: Status of 'FTP Service'
msg: Connected from ec2-xyz
msg: Logged in as <user>
msg: TYPE: ASCII, STRUcture: File, Mode: Stream
msg: No data connection
msg: End of status
Few questions which I need to know answers (may be that give a lead to fix):
Here, TYPE: BINARY is changed to ASCII and can someone tell how to modify back to BINARY? Note tehre is no explicit setting made, got changed post FTPES set
Is this issue related to proxy / port. if so - I wont be able to provide those information.
As of now, I am testing with same proxy used which is used in the application.
Is there any certificates to be imported to get files viewed / downloaded.
Found this on net which is exactly same as my issue, http://www.proftpd.org/docs/howto/TLS.html ( search for -"I can login using FTPS, but I cannot see any directories. Why not?") but I need to explain my third party to make the required change... what should I tell?
stand alone client code
import it.sauronsoftware.ftp4j.*;
import javax.net.ssl.*;
import java.io.File;
import java.security.*;
import java.util.List;
public class FTPWithSSL {
private static FTPClient ftpClient;
private static FTPConfig ftpConfig;
private DailyFTPConfig config;
public static void main(String args[]) throws Exception {
ftpConfig = new FTPConfig("username", "password", "FTPServer.net", 21, setupConnector());
FTPDownloader ftpDownloader = new FTPDownloader(ftpConfig, new FTPDownloadController() {
#Override
public List<FTPFile> download(FTPClient ftpClient) throws Exception {
downloadFile(ftpClient);
System.out.println("download success");
return null;
}
});
try {
openConnection();
List<FTPFile> ftpFileList = ftpDownloader.download();
closeConnection();
} catch (Exception e) {
e.printStackTrace();
}
}
private static void openConnection() throws Exception {
System.out.println("OpenConnection method");
if (ftpClient == null) {
ftpClient = new FTPClient();
// Even if the below line is commented, the code works fine
ftpClient.setSSLSocketFactory(getSSLSocketFactory());
System.out.println("setting FTPES here");
ftpClient.setSecurity(FTPClient.SECURITY_FTPES);
ftpClient.setConnector(ftpConfig.getConnector());
ftpClient.connect(ftpConfig.getFtpHost(), ftpConfig.getFtpPort());
ftpClient.login(ftpConfig.getUsername(), ftpConfig.getPassword());
ftpClient.setAutoNoopTimeout(30000);
System.out.println("ftpClient.isConnected() " + ftpClient.isConnected());
}
}
private static SSLSocketFactory getSSLSocketFactory() throws Exception {
TrustManager[] trustManager = new TrustManager[] { new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
} };
SSLContext sslContext = null;
try {
sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustManager, new SecureRandom());
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyManagementException e) {
e.printStackTrace();
}
SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
return sslSocketFactory;
}
private static void closeConnection() throws Exception {
System.out.println("ftpClient.serverStatus() -- ");
for (String serverStatus : ftpClient.serverStatus()) {
System.out.println(" msg: " + serverStatus);
}
if (ftpClient != null && ftpClient.isConnected()) {
ftpClient.disconnect(false);
}
}
private static void downloadFile(FTPClient ftpClient) throws Exception {
System.out.println("ftpsClient.currentDirectory() " + ftpClient.currentDirectory());
FTPFile[] ftpFile = ftpClient.list();
System.out.println("Name " + ftpFile[0].getName());
System.out.println("Link " + ftpFile[0].getLink());
System.out.println("Modified Date " + ftpFile[0].getModifiedDate());
String[] listnames = ftpClient.listNames();
System.out.println("ftpsClient.listNames() " +listnames);
System.out.println("ftpsClient.currentDirectory() " + ftpClient.list());
File file = new File( "C:\\opt\\copycat\\a1234.zip");
System.out.println("Downloading File: [" + file.getName() + "] has started.");
ftpClient.download("a1234.zip", file);
System.out.println("Downloading File: has Completed");
}
private static FTPConnector setupConnector() {
FTPConnector connector = new DirectConnector();
String proxyHost = "amazonaws.com";
if (proxyHost != null) {
System.out.println("proxy host NOT NULL");
int proxyPort = Integer.parseInt("123");
connector = new HTTPTunnelConnector(proxyHost, proxyPort);
}
return connector;
}
}
Finally, I am able to make the secure connection and able to download the latest files from FTP server using FTPES (FTP over explicit TLS/SSL).
I have made only 2 line code changes (below), in the actual implementation. earlier it was having the list() method to get the files from FTP server, and its return type is FTPFile[]
This is the code changes which I made, and other line of codes are modified / removed.
ftpClient.setSecurity(FTPClient.SECURITY_FTPES);
// get the FTP connection
String[] fileNames = ftpClient.listNames();
// Exisitng implementation
FTPFile[] ftpFiles = ftpClient.list("*.zip");
// after that there is an implementation for getting the latest files using the Comparator<FTPFile>
There was an issue with the LIST command's output, ie: LIST *.zip is not giving the correct output (I guess)
i need to be able to access certain urls programmatically . I am using URLConnection as follows
URL url = new URL(http,
myProxy.com, // I need to know this parameter
-1,
http://www.example.com/);
How do I get the name of the proxy server used in a managed network.
when i use a browser like chrome it connects me with the proxy server that makes requests to the internet . How do i get the name of the proxy server ?
You can try to use a java ProxySelector class to do it, her is short example of it'usage from java proxy configuration guide:
private Proxy findProxy(URI uri)
{
try
{
ProxySelector selector = ProxySelector.getDefault();
List<Proxy> proxyList = selector.select(uri);
if (proxyList.size() > 1)
return proxyList.get(0);
}
catch (IllegalArgumentException e)
{
}
return Proxy.NO_PROXY;
}
To get a host name and IP address, you can use an InetSocketAddress, which you can get from Proxy instance:
InetSocketAddress addr = (InetSocketAddress) proxy.address();
if(addr != null) {
System.out.println("proxy hostname : " + addr.getHostName());
System.out.println("proxy port : " + addr.getPort());
}
But as I know, it's needed to set a system property to do it:
System.setProperty("java.net.useSystemProxies","true");
One more solution is to use a proxy-vole library to do it. Here is some usage examples.
We are calling URL using proxy settings via java standalone sample code on our weblogic servers (node1/node2).
This code works fine on node 1 but same code doesn't work on node2 server.
We already checked proxy settings and credentials all are fine but still we get following error :
java.net.ProtocolException: Server redirected too many times (20)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at ProxyCode.start2(ProxyCode.java:54)
at ProxyCode.main(ProxyCode.java:23)
Exception in thread "Main Thread" java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:61)
at java.io.InputStreamReader.<init>(InputStreamReader.java:55)
at ProxyCode.readFromInputStream(ProxyCode.java:65)
at ProxyCode.start2(ProxyCode.java:59)
at ProxyCode.main(ProxyCode.java:22)
Also , Please find my code snippet below :
SimpleAuthenticator.java
import java.net.Authenticator;
import java.net.PasswordAuthentication;
public class SimpleAuthenticator extends Authenticator
{
private String username;
private String password;
public SimpleAuthenticator(String username,String password)
{
this.username = username;
this.password = password;
}
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication(
username,password.toCharArray());
}
}
Main class :
String url = "http://www.oracle.com/technetwork/java/readme-2-149793.txt";
String proxy = "proxyserver";
String port = "8080";
String username = "username";
String password = "password";
Authenticator.setDefault(new SimpleAuthenticator(username,password));
URL server = null;
try {
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));
server = new URL(url);
}
catch (MalformedURLException e) {
e.printStackTrace();
}
Properties systemProperties = System.getProperties();
systemProperties.setProperty("http.proxyHost", proxy);
systemProperties.setProperty("http.proxyPort", port);
InputStream in = null;
URLConnection connection = null;
try {
connection = (URLConnection) server.openConnection();
connection.connect();
in = connection.getInputStream();
}
catch (IOException e) {
e.printStackTrace();
}
System.out.println(readFromInputStream(in));
}
public static String readFromInputStream(InputStream in) {
StringBuffer strBuf = new StringBuffer();
char ac[];
BufferedReader buf = new BufferedReader(new InputStreamReader(in));
try
{
while (buf.ready()) {
ac = new char[10000];
buf.read(ac);
strBuf.append(ac);
}
buf.close();
}
catch (IOException e) {
e.printStackTrace();
}
We are stuck in this since months now and not able to get any useful information anywhere.
kindly help.thanks
You will get this error if you provide wrong credentials (username or password).
This happened to me with a Glassfish based web app. I was expecting a 401 response however.
I think, the Authenticator tries the same credentials many times.
Found this open issue, still unfixed it would seem. Your user doesn't have access rights, but instead of prompting again it retries with the same user over and over
Update: The issue had another ticket open - this is apparently expected behaviour and
"To overcome this, your implementation of
Authenticator::getPasswordAuthentication needs to provide a means to
collect the correct password if the initial authentication attempt was
not successful."
See the ticket or the java.net.Authenticator documentation for more info.
How do you check if you can connect to the internet via java? One way would be:
final URL url = new URL("http://www.google.com");
final URLConnection conn = url.openConnection();
... if we got here, we should have net ...
But is there something more appropriate to perform that task, especially if you need to do consecutive checks very often and a loss of internet connection is highly probable?
You should connect to the place that your actual application needs. Otherwise you're testing whether you have a connection to somewhere irrelevant (Google in this case).
In particular, if you're trying to talk to a web service, and if you're in control of the web service, it would be a good idea to have some sort of cheap "get the status" web method. That way you have a much better idea of whether your "real" call is likely to work.
In other cases, just opening a connection to a port that should be open may be enough - or sending a ping. InetAddress.isReachable may well be an appropriate API for your needs here.
The code you basically provided, plus a call to connect should be sufficient. So yeah, it could be that just Google's not available but some other site you need to contact is on but how likely is that? Also, this code should only execute when you actually fail to access your external resource (in a catch block to try and figure out what the cause of the failure was) so I'd say that if both your external resource of interest and Google are not available chances are you have a net connectivity problem.
private static boolean netIsAvailable() {
try {
final URL url = new URL("http://www.google.com");
final URLConnection conn = url.openConnection();
conn.connect();
conn.getInputStream().close();
return true;
} catch (MalformedURLException e) {
throw new RuntimeException(e);
} catch (IOException e) {
return false;
}
}
People have suggested using INetAddress.isReachable. The problem is that some sites configure their firewalls to block ICMP Ping messages. So a "ping" might fail even though the web service is accessible.
And of course, the reverse is true as well. A host may respond to a ping even though the webserver is down.
And of course, a machine may be unable to connect directly to certain (or all) web servers due to local firewall restrictions.
The fundamental problem is that "can connect to the internet" is an ill-defined question, and this kind of thing is difficult to test without:
information on the user's machine and "local" networking environment, and
information on what the app needs to access.
So generally, the simplest solution is for an app to just try to access whatever it needs to access, and fall back on human intelligence to do the diagnosis.
If you're on java 6 can use NetworkInterface to check for available network interfaces.
I.e. something like this:
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
while (interfaces.hasMoreElements()) {
NetworkInterface interf = interfaces.nextElement();
if (interf.isUp() && !interf.isLoopback())
return true;
}
Haven't tried it myself, yet.
This code should do the job reliably.
Note that when using the try-with-resources statement we don't need to close the resources.
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.UnknownHostException;
public class InternetAvailabilityChecker
{
public static boolean isInternetAvailable() throws IOException
{
return isHostAvailable("google.com") || isHostAvailable("amazon.com")
|| isHostAvailable("facebook.com")|| isHostAvailable("apple.com");
}
private static boolean isHostAvailable(String hostName) throws IOException
{
try(Socket socket = new Socket())
{
int port = 80;
InetSocketAddress socketAddress = new InetSocketAddress(hostName, port);
socket.connect(socketAddress, 3000);
return true;
}
catch(UnknownHostException unknownHost)
{
return false;
}
}
}
This code:
"127.0.0.1".equals(InetAddress.getLocalHost().getHostAddress().toString());
Returns - to me - true if offline, and false, otherwise. (well, I don't know if this true to all computers).
This works much faster than the other approaches, up here.
EDIT: I found this only working, if the "flip switch" (on a laptop), or some other system-defined option, for the internet connection, is off. That's, the system itself knows not to look for any IP addresses.
InetAddress.isReachable sometime return false if internet connection exist.
An alternative method to check internet availability in java is : This function make a real ICMP ECHO ping.
public static boolean isReachableByPing(String host) {
try{
String cmd = "";
if(System.getProperty("os.name").startsWith("Windows")) {
// For Windows
cmd = "ping -n 1 " + host;
} else {
// For Linux and OSX
cmd = "ping -c 1 " + host;
}
Process myProcess = Runtime.getRuntime().exec(cmd);
myProcess.waitFor();
if(myProcess.exitValue() == 0) {
return true;
} else {
return false;
}
} catch( Exception e ) {
e.printStackTrace();
return false;
}
}
I usually break it down into three steps.
I first see if I can resolve the domain name to an IP address.
I then try to connect via TCP (port 80 and/or 443) and close gracefully.
Finally, I'll issue an HTTP request and check for a 200 response back.
If it fails at any point, I provide the appropriate error message to the user.
URL url=new URL("http://[any domain]");
URLConnection con=url.openConnection();
/*now errors WILL arise here, i hav tried myself and it always shows "connected" so we'll open an InputStream on the connection, this way we know for sure that we're connected to d internet */
/* Get input stream */
con.getInputStream();
Put the above statements in try catch blocks and if an exception in caught means that there's no internet connection established. :-)
The code using NetworkInterface to wait for the network worked for me until I switched from fixed network address to DHCP. A slight enhancement makes it work also with DHCP:
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
while (interfaces.hasMoreElements()) {
NetworkInterface interf = interfaces.nextElement();
if (interf.isUp() && !interf.isLoopback()) {
List<InterfaceAddress> adrs = interf.getInterfaceAddresses();
for (Iterator<InterfaceAddress> iter = adrs.iterator(); iter.hasNext();) {
InterfaceAddress adr = iter.next();
InetAddress inadr = adr.getAddress();
if (inadr instanceof Inet4Address) return true;
}
}
}
This works for Java 7 in openSuse 13.1 for IPv4 network. The problem with the original code is that although the interface was up after resuming from suspend, an IPv4 network address was not yet assigned. After waiting for this assignment, the program can connect to servers. But I have no idea what to do in case of IPv6.
1) Figure out where your application needs to be connecting to.
2) Set up a worker process to check InetAddress.isReachable to monitor the connection to that address.
This code is contained within a jUnit test class I use to test if a connection is available. I always receive a connection, but if you check the content length it should be -1 if not known :
try {
URL url = new URL("http://www.google.com");
URLConnection connection = url.openConnection();
if(connection.getContentLength() == -1){
fail("Failed to verify connection");
}
}
catch (IOException e) {
fail("Failed to open a connection");
e.printStackTrace();
}
public boolean checkInternetConnection()
{
boolean status = false;
Socket sock = new Socket();
InetSocketAddress address = new InetSocketAddress("www.google.com", 80);
try
{
sock.connect(address, 3000);
if(sock.isConnected()) status = true;
}
catch(Exception e)
{
status = false;
}
finally
{
try
{
sock.close();
}
catch(Exception e){}
}
return status;
}
You can simply write like this
import java.net.InetAddress;
import java.net.UnknownHostException;
public class Main {
private static final String HOST = "localhost";
public static void main(String[] args) throws UnknownHostException {
boolean isConnected = !HOST.equals(InetAddress.getLocalHost().getHostAddress().toString());
if (isConnected) System.out.println("Connected");
else System.out.println("Not connected");
}
}
There are (nowadays) APIs for this, but they are platform specific:
On Android ConnectivityManager (https://developer.android.com/training/basics/network-ops/reading-network-state) does everything you need.
On Windows INetworkListManager::GetConnectivity (for which you'll need a JNI)
On generic Linux, you are probably stuck with testing if you have access to a DNS server and Google, as above.
there is probably an Apple way to do this as well
(I'd use the specific tools where available)
This have worked well for me.
try{
InetAddress addr = InetAddress.getByName("google.com" );
}catch(IOException e){
JOptionPane.showMessageDialog(new JFrame(),"No Internet connection.\nTry again later", "Network Error", JOptionPane.ERROR_MESSAGE);
}
There is also a gradle option --offline which maybe results in the behavior you want.
The following piece of code allows us to get the status of the network on our Android device
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
TextView mtv=findViewById(R.id.textv);
ConnectivityManager connectivityManager=
(ConnectivityManager) this.getSystemService(Context.CONNECTIVITY_SERVICE);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
if(((Network)connectivityManager.getActiveNetwork())!=null)
mtv.setText("true");
else
mtv.setText("fasle");
}
}
}