We Keep Coding

Unable to load data from URL using HTTP Proxy Android

I'm using java.net.URL and java.net.Proxy to load my app's data from the desired URL. When I am using Proxy.Type.SOCKS proxy, everything is OK, but I need to use Proxy.Type.HTTP. My code is the following:
URL url = new URL("https://google.com/");
final Proxy proxy = getProxy();
HttpURLConnection connection = (HttpURLConnection) url.openConnection(proxy);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
Where getProxy (simplified) looks as follows:
private static Proxy getProxy(){
if(!StringUtils.isNullOrBlank(username)){
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
});
} else {
Authenticator.setDefault(null);
}
return new Proxy(Proxy.Type.SOCKS HTTP, new InetSocketAddress(address, port));
}
I am getting java.io.IOException: unexpected end of stream on null, Caused by: java.io.EOFException: \n not found: size=0 content=....
System.setProperty option doesn't seem to work, as well.
Is there any way to use HTTP proxy correctly on Android?

How can I set https.proxyHost and https.proxyPort for individual HttpsURLConnections?

I'm trying to make HTTPS requests through a proxy. Here's what I've got so far, based on code from this question:
try {
HttpsURLConnection connection = (HttpsURLConnection) new URL("https://proxylist.geonode.com/api/proxy-list?limit=1&page=1&sort_by=speed&sort_type=asc&protocols=https").openConnection();
connection.setRequestMethod("GET");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestProperty("user-agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36");
connection.setConnectTimeout(30000);
connection.connect();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String rawJSON = reader.readLine();
if(rawJSON == null) throw new IOException("No data");
JSONObject data = new JSONObject(rawJSON).getJSONArray("data").getJSONObject(0);
String ipAddress = data.getString("ip"), port = data.getString("port");
System.setProperty("https.proxyHost", ipAddress);
System.setProperty("https.proxyPort", port);
SSLContext sslContext = SSLContext.getInstance("SSL");
// set up a TrustManager that trusts everything
sslContext.init(null, new TrustManager[] { new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() { return null; }
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
} }, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory());
HttpsURLConnection.setDefaultHostnameVerifier((arg0, arg1) -> true);
HttpsURLConnection testConnection = (HttpsURLConnection) new URL("https://example.com").openConnection();
testConnection.connect();
StringBuilder result = new StringBuilder();
String line;
try(BufferedReader reader2 = new BufferedReader(new InputStreamReader(testConnection.getInputStream()))) {
while ((line = reader2.readLine()) != null) result.append(line);
}
System.out.println(result);
} catch(Exception e) {
e.printStackTrace();
}
The code works, but there's a problem. My application (https://encyclosearch.org) is multithreaded, and I need to make some requests through a proxy, and some directly. Since system properties are global, if I set https.proxyHost and https.proxyPort using System.setProperty, some requests that aren't supposed to go through the proxy will go through the proxy.
I can use java.net.Proxy like this:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ipAddress, Integer.parseInt(port)));
HttpsURLConnection testConnection = (HttpsURLConnection) new URL("http://example.com").openConnection(proxy);
But that only works for HTTP proxies, not HTTPS ones, so I can't make HTTPS requests. There's no Proxy.Type.HTTPS.
Any help would be greatly appreciated. Thanks in advance.

If you choose which connections go through a proxy and which do not by the destination url then you can use the property http.nonProxyHosts. This property is used for http and https as the documentation states:
for the "non proxy hosts" list, the HTTPS protocol handler will use the same as the http handler (i.e. http.nonProxyHosts).
You set the property value by adding patterns of urls separated by | For example:
System.setProperty("http.nonProxyHosts", ”localhost|host.example.com”)
You could also use ProxySelector class for choosing which connections go through a proxy. for more info (a bit old):
https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
As for the absence of Proxy.Type.HTTPS, it is because a proxy in general is not the final destination so the secure connection will be with the final destination not the proxy itself. There is such thing as SSL tunneling through a proxy, but i am not well informed about it.

With #Bashi's help, I figured it out. For direct connections, I used:
url.openConnection(Proxy.NO_PROXY);
This works for Jsoup, too:
Document document = Jsoup.connect("https://example.com").proxy(Proxy.NO_PROXY).get();
Explanation from https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html:
Now, this guarantees you that this specific URL will be retrieved though a direct connection bypassing any other proxy settings, which can be convenient.

Proxy returns "HTTP/1.1 407 Proxy Authorization Required" in production but works in local

Getting error while calling a api in production but works in local.
Used same proxy as in local 10.235.88.30 and port 8080
calling through jersey client:-
com.sun.jersey.api.client.Client client = new Client(
new URLConnectionClientHandler(new HttpURLConnectionFactory() {
#Override
public HttpURLConnection getHttpURLConnection(final URL url) throws IOException {
Proxy proxy;
proxy = new Proxy(Proxy.Type.HTTP,
new InetSocketAddress("10.235.88.30", 8080));
return (HttpURLConnection) url.openConnection(proxy);
}
}), new DefaultClientConfig());
Should return response but getting proxy error.
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authorization Required"

Unable to get Proxy working with Socket

I am trying to get Proxy working with Socket. But everytime I tried, it would returned a "Exception in thread "pool-1-thread-1" java.lang.IllegalArgumentException: Invalid Proxy" exception error
at java.net.Socket.(Socket.java:131)
But if its Proxy.Type.SOCKS, it works.
public void Test()
{
Socket s = null;
SocketAddress addr = null;
Proxy proxy = null;
addr = new InetSocketAddress("127.0.0.1", 8080);
proxy = new Proxy(Proxy.Type.HTTP, addr);
socket = new Socket(proxy); // This is the line that is triggering the exception
}

Sadly this is a bug in (Oracle) Java - only DIRECT and SOCKS proxy is supported for Socket. See http://bugs.sun.com/view_bug.do?bug_id=6370908.

How to use a custom socketfactory in Apache HttpComponents

I have been trying to use a custom SocketFactory in the httpclient library from the Apache HTTPComponents project. So far without luck. I was expecting that I could just set a socket factory for a HttpClient instance, but it is obviously not so easy.
The documentation for HttpComponents at http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html does mention socket factories, but does not say how to use them.
Does anybody know how this is done?

oleg's answer is of course correct, I just wanted to put the information directly here, in case the link goes bad. In the code that creates a HttpClient, I use this code to let it use my socket factory:
CustomSocketFactory socketFactory = new CustomSocketFactory();
Scheme scheme = new Scheme("http", 80, socketFactory);
httpclient.getConnectionManager().getSchemeRegistry().register(scheme);
CustomSocketFactory is my own socket factory, and I want to use it for normal HTTP traffic, that's why I use "http" and 80 as parameters.
My CustomSchemeSocketFactory looks similar to this:
public class CustomSchemeSocketFactory implements SchemeSocketFactory {
#Override
public Socket connectSocket( Socket socket, InetSocketAddress remoteAddress, InetSocketAddress localAddress, HttpParams params ) throws IOException, UnknownHostException, ConnectTimeoutException {
if (localAddress != null) {
socket.setReuseAddress(HttpConnectionParams.getSoReuseaddr(params));
socket.bind(localAddress);
}
int connTimeout = HttpConnectionParams.getConnectionTimeout(params);
int soTimeout = HttpConnectionParams.getSoTimeout(params);
try {
socket.setSoTimeout(soTimeout);
socket.connect(remoteAddress, connTimeout );
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " + remoteAddress + " timed out");
}
return socket;
}
#Override
public Socket createSocket( HttpParams params ) throws IOException {
// create my own socket and return it
}
#Override
public boolean isSecure( Socket socket ) throws IllegalArgumentException {
return false;
}
}

We use a custom socket factory to allow HttpClient connections to connect to HTTPS URLs with untrusted certificates.
Here is how we did it:
We adapted implementations of both the 'EasySSLProtocolSocketFactory' and 'EasyX509TrustManager' classes from the examples source directory referenced by Oleg.
In our HttpClient startup code, we do the following to enable the new socket factory:
HttpClient httpClient = new HttpClient();
Protocol easyhttps = new Protocol("https", new EasySSLProtocolSocketFactory(), 443);
Protocol.registerProtocol("https", easyhttps);
So that any time we request an https:// URL, this socket factory is used.