I am trying to build a proxy server and recently I am working on https. As specified in this post. I've tried to tunnel Connect request. My Code is as:
private boolean handleConnect(HttpServletRequest req,HttpServletResponse response){
String uri=req.getRequestURI();
String port="";
String host="";
int c=uri.indexOf(":");
if (c >= 0){
port = uri.substring(c + 1);
host = uri.substring(0,c);
if (host.indexOf('/') > 0)
host = host.substring(host.indexOf('/') + 1);
}
// Make Asyncronous connection
try{
InetSocketAddress inetAddress = new InetSocketAddress(host,Integer.parseInt(port));
{
InputStream in=req.getInputStream();
OutputStream out=response.getOutputStream();
if(true){
Socket sock=new Socket(host,Integer.parseInt(port));
IO.copy(in, sock.getOutputStream());
IO.copy(sock.getInputStream(), out);
if(!sock.getKeepAlive()){
sock.close();
}
}
}
}
catch(Exception ex){
ex.printStackTrace();
return false;
}
return true;
}
The code results java.net.UnknownHostException: google.com.np for https://google.com.np and Timeouts for https://Facebook.com . Why is that ??
Please suggest best way to tunnel Connect HTTP request.
Your UnknownHostException is due either to a non-existent host or a misconfigured DNS, and your connect timeout to a network connectivity problem, neither of which are on-topic here, but you can't really write a proper proxy this way. You need to start two threads per connection, one to copy bytes in each direction.
Related
I am trying to make a proxy server in java and I was able to make a working proxy which handles http requests properly. After searching a lot I was also able to extend my program for https requests by following this answer to a similar question: https://stackoverflow.com/a/9389125/5309299
Here's my code after a TCP connection is established between client and proxy:
String request = "";
byte[] requestByteArr;
//read the complete request
while(true){
String requestLine = bufferedReaderFromClient.readLine() + "\r\n";
if (requestLine.trim().length()==0 && !request.equals("")){
request+=requestLine;
requestByteArr = request.getBytes();
System.out.println(request);
break;
} else {
request+=requestLine;
}
}
String hostname = getHostFromRequest(request);
int remoteport = getRemotePortFromRequest(request);
if (request.startsWith("CONNECT")){
//establish connection between host and proxy
final Socket hostSocket = new Socket(hostname, remoteport);
//tell client that connection was successful
String statusLine = "HTTP/1.1 200 Connection established \n" + "Proxy-agent: ProxyServer/1.0\n" + "\r\n";
outToClient.write(statusLine.getBytes());
outToClient.flush();
//new thread to handle incoming responses from host
new Thread(){
public void run(){
try{
InputStream inFromHost = hostSocket.getInputStream();
while(true){
byte[] bufread = new byte[128];
int bytes_received;
while ((bytes_received = inFromHost.read(bufread)) > 0){
outToClient.write(bufread, 0, bytes_received);
outToClient.flush();
}
}
} catch (IOException e){
e.printStackTrace();
}
}
}.start();
//main thread handles incoming requests from client
OutputStream outToHost = hostSocket.getOutputStream();
while (true){
byte[] bufread = new byte[128];
int bytes_received;
while ((bytes_received = inFromClient.read(bufread)) > 0){
outToHost.write(bufread, 0, bytes_received);
outToHost.flush();
}
}
}
Obviously, this only works for one host, i.e. when a client (e.g. chrome browser) sends a CONNECT request for one host (e.g. "www.google.com:443"). I want my client to be able to connect with multiple hosts. The problem is that since all the requests that come after CONNECT request are encrypted, my proxy server will not be able to determine which request is meant for which host, so it cannot forward the requests.
I try to implement a Java Proxy for Http (Https will be the extension after Http works). I found a lot of resources on the Internet and try to solve all problems on my own so far. But now I come to a point where I stuck.
My Proxoy does not load the full http websites. I get a lot of error messages with the socket is already closed. So I think I try to send something over a Socket that is closed.
My Problem is now. I can not see why it is like this. I think a lot over the problem but I can not find the mistake. From my side The Sockets only get closed when the server close the connection to my Proxy Server. This happen when I read a -1 on the input stream from the server.
I would be happy for any help :-)
greetings
Christoph
public class ProxyThread extends Thread {
Socket client_socket;
Socket server_socket;
boolean thread_var = true;
int buffersize = 32768;
ProxyThread(Socket s) {
client_socket = s;
}
public void run() {
System.out.println("Run Client Thread");
try {
// Read request
final byte[] request = new byte[4096];
byte[] response = new byte[4096];
final InputStream in_client = client_socket.getInputStream();
OutputStream out_client = client_socket.getOutputStream();
in_client.read(request);
System.out.println("---------------------- Request Info --------------------");
System.out.println(new String(request));
Connection conn = new Connection(new String(request));
System.out.println("---------------------- Connection Info --------------------");
System.out.println("Host: " + conn.host);
System.out.println("Port: " + conn.port);
System.out.println("URL: " + conn.URL);
System.out.println("Type: " + conn.type);
System.out.println("Keep-Alive:" + conn.keep_alive);
server_socket = new Socket(conn.URL, conn.port);
InputStream in_server = server_socket.getInputStream();
final OutputStream out_server = server_socket.getOutputStream();
out_server.write(request);
out_server.flush();
Thread t = new Thread() {
public void run() {
int bytes_read;
try {
while ((bytes_read = in_client.read(request)) != -1) {
out_server.write(request, 0, bytes_read);
out_server.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
t.start();
int bytes_read;
while ((bytes_read = in_server.read(response)) != -1) {
out_client.write(response, 0, bytes_read);
out_client.flush();
//System.out.println("---------------------- Respone Info --------------------");
//System.out.println(new String(response));
}
//System.out.println("EIGENTLICH FERTIG");
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client_socket.close();
server_socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
EDIT:
My HTTP Proxy now works. The Answer is pretty helpfull once you understand what is ryl going on. If you come hear to find a solution this questions may help you:
Does the client send a request only to one Website / Webserver? Means do we always have the same port / hostname?
The Loop from the answer is very usefull but think where to place it?
Last think: Thanks #EJP its working your reply was very usefull. It only tooks a time to understand it!
You are making all the usual mistakes, and a few more.
The entire request is not guaranteed to arrive in a single read. You can't assume more than a single byte has arrived. You have to loop.
You aren't checking for end of stream at this stage.
You need a good knowledge of RFC 2616 to implement HTTP, specifically the parts about Content-length and transfer encoding.
You cannot assume that the server will close the connection after sending the response.
Closing either the input or the output stream or a socket closes the socket. This is the reason for your SocketException: socket closed.
When you get to HTTPS you will need to look at the CONNECT verb.
Flushing a socket output stream does nothing, and flushing inside a loop is to be avoided,
Currently I am using URL()
public boolean isInternetAvailable(){
try {
URL url = new URL("http://www.google.com");
HttpURLConnection urlConnect = (HttpURLConnection)url.openConnection();
urlConnect.setConnectTimeout(5000);
Object objData = urlConnect.getContent();
return true;
} catch (Exception e) {}
return false;
}
But In the requirement, we don't want to use any URL. We want to ping localhost if connection is available than return true otherwise flase.
For nslookup I am using
try
{
InetAddress inetAddress = InetAddress.getByName("localhost");
System.out.println("Host: " +inetAddress.getHostName());
System.out.println("IP Address: " +inetAddress.getHostAddress());
System.out.println("IP Address: " +inetAddress.isSiteLocalAddress());
}
catch (UnknownHostException e)
{
e.printStackTrace();
}
But I am not understand how to check the connection availability with nslookup.
Please suggest best approach for it. Thanks
There are several catches to this question:
Starting with the most specific one - connecting to localhost: even if your computer does not have a network card, it will be able to resolve localhost on a loopback interface and connect to itself (if there is an open port).
Your DNS may be down/misconfigured, so you cannot resolve example.com but you can connect to it by IP (93.184.216.34) - does that mean than "internet is not available"?
The firewall in your company may be blocking certain sites, but allowing other - does that mean than "internet is not available"?
The server of example.com is down while all the other sites in the world work fine. Does that mean than "internet is available" or not?
The firewall in your company may be allowing HTTP connections only on standard ports 80 and 443 and disallowing other. Thus, http://example.com connects, but http://example.com:12345 does not. Does that mean than "internet is available" or not?
So the only question you can actually ask is whether you can connect to a particular host on a particular port using its domain name and/or its IP address.
Figured out a final solution using NetworkInterface:
Enumeration<NetworkInterface> eni = NetworkInterface.getNetworkInterfaces();
while(eni.hasMoreElements()) {
Enumeration<InetAddress> eia = eni.nextElement().getInetAddresses();
while(eia.hasMoreElements()) {
InetAddress ia = eia.nextElement();
if (!ia.isAnyLocalAddress() && !ia.isLoopbackAddress() && !ia.isSiteLocalAddress()) {
if (!ia.getHostName().equals(ia.getHostAddress()))
return true;
}
}
}
I have got solution from here:- Java Quickly check for network connection
I am trying to send data from a C#.NET (Windows Application) program to a Java (Android App) program and vice versa, via TCP connection through Wifi. Till now I am success to send data from Java to C#, but unable to do so from C# to Java.
Following is the Java code, I used to create a connection and receive data:
ServerSocket serverSocket = null;
DataInputStream socketInputStream;
while (true) {
try {
String localIPAddr = getLocalIPAddress();
InetSocketAddress ipEndPoint = new InetSocketAddress(
InetAddress.getByName(localIPAddr), 8222);
serverSocket = new ServerSocket();
serverSocket.bind(ipEndPoint, 4);
workerSocket = serverSocket.accept();
socketInputStream = new DataInputStream(
workerSocket.getInputStream());
inputText.setText(socketInputStream.readUTF());
} catch (Exception ex) {
throw ex;
}
}
Here getLocalIPAddress() method returns the IP Address of the Android Device.
Following is the C# code in Windows Application to connect to the Android's IP Address (192.168.1.6) and send data to it:
Socket clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
void button1_Click(object sender, EventArgs e)
{
try
{
if (!clientSocket.Connected)
clientSocket.Connect(IPAddress.Parse("192.168.1.6"), 8222);
clientSocket.Send(Encoding.UTF8.GetBytes(txtInput.Text));
}
catch (Exception ex)
{
throw ex;
}
}
Well, client (C#) is failing to connect to the server (Java) That means data is not leaving from client. But it will, if it get connected. Please tell me what am I missing and where am I mistaken. :)
After you have launched your android app and it is connected to the wifi, did you try to do a ping to the ip where the application is launched.
ping 192.168.1.6
If the IP is accessible from the workstation where C# app is running, try to perform a telnet on the IP and Port of the android ip, to see whether it works or not.
telnet 192.168.1.6 8222
If either of the two steps fail then could be a problem in the wifi network. As i have noticed many times the firewall of the routers filters out all the ports except 8080 and 80. So you would need to open the ports on the router.
Did you try to do this?
Runnable showmessage = new Runnable() {
public void run() {
myTextView.setText(membervariabletext);
}
};
and from your thread, after the readUTF(), call
runOnUiThread(showmessage);
Found this here
Well, I have solved this myself, but of course Dilberted has helped me a little bit. I thank him for what he has provided. :)
Check out the solved Java code below:
ServerSocket serverSocket = null;
Socket workerSocket;
DataInputStream socketInputStream;
try {
if (serverSocket == null) {
// No need to get local IP address and to bind InetSocketAddress.
// Following single line make it very simple.
serverSocket = new ServerSocket(8222, 4);
workerSocket = serverSocket.accept();
}
// When data are accepted socketInputStream will be invoked.
socketInputStream = new DataInputStream(
workerSocket.getInputStream());
/* Since data are accepted as byte, all of them will be collected in the
following byte array which initialised with accepted data length. */
byte[] rvdMsgByte = new byte[socketInputStream.available()];
// Collecting data into byte array
for (int i = 0; i < rvdMsgByte.length; i++)
rvdMsgByte[i] = socketInputStream.readByte();
// Converting collected data in byte array into String.
String rvdMsgTxt = new String(rvdMsgByte);
// Setting String to the text view.
receivedMsg.setText(rvdMsgTxt);
} catch (Exception ex) {
throw ex;
}
Note that a separate thread is to be used to run this code in background.
I need a monitor class that regularly checks whether a given HTTP URL is available. I can take care of the "regularly" part using the Spring TaskExecutor abstraction, so that's not the topic here. The question is: What is the preferred way to ping a URL in java?
Here is my current code as a starting point:
try {
final URLConnection connection = new URL(url).openConnection();
connection.connect();
LOG.info("Service " + url + " available, yeah!");
available = true;
} catch (final MalformedURLException e) {
throw new IllegalStateException("Bad URL: " + url, e);
} catch (final IOException e) {
LOG.info("Service " + url + " unavailable, oh no!", e);
available = false;
}
Is this any good at all (will it do what I want)?
Do I have to somehow close the connection?
I suppose this is a GET request. Is there a way to send HEAD instead?
Is this any good at all (will it do what I want?)
You can do so. Another feasible way is using java.net.Socket.
public static boolean pingHost(String host, int port, int timeout) {
try (Socket socket = new Socket()) {
socket.connect(new InetSocketAddress(host, port), timeout);
return true;
} catch (IOException e) {
return false; // Either timeout or unreachable or failed DNS lookup.
}
}
There's also the InetAddress#isReachable():
boolean reachable = InetAddress.getByName(hostname).isReachable();
This however doesn't explicitly test port 80. You risk to get false negatives due to a Firewall blocking other ports.
Do I have to somehow close the connection?
No, you don't explicitly need. It's handled and pooled under the hoods.
I suppose this is a GET request. Is there a way to send HEAD instead?
You can cast the obtained URLConnection to HttpURLConnection and then use setRequestMethod() to set the request method. However, you need to take into account that some poor webapps or homegrown servers may return HTTP 405 error for a HEAD (i.e. not available, not implemented, not allowed) while a GET works perfectly fine. Using GET is more reliable in case you intend to verify links/resources not domains/hosts.
Testing the server for availability is not enough in my case, I need to test the URL (the webapp may not be deployed)
Indeed, connecting a host only informs if the host is available, not if the content is available. It can as good happen that a webserver has started without problems, but the webapp failed to deploy during server's start. This will however usually not cause the entire server to go down. You can determine that by checking if the HTTP response code is 200.
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setRequestMethod("HEAD");
int responseCode = connection.getResponseCode();
if (responseCode != 200) {
// Not OK.
}
// < 100 is undetermined.
// 1nn is informal (shouldn't happen on a GET/HEAD)
// 2nn is success
// 3nn is redirect
// 4nn is client error
// 5nn is server error
For more detail about response status codes see RFC 2616 section 10. Calling connect() is by the way not needed if you're determining the response data. It will implicitly connect.
For future reference, here's a complete example in flavor of an utility method, also taking account with timeouts:
/**
* Pings a HTTP URL. This effectively sends a HEAD request and returns <code>true</code> if the response code is in
* the 200-399 range.
* #param url The HTTP URL to be pinged.
* #param timeout The timeout in millis for both the connection timeout and the response read timeout. Note that
* the total timeout is effectively two times the given timeout.
* #return <code>true</code> if the given HTTP URL has returned response code 200-399 on a HEAD request within the
* given timeout, otherwise <code>false</code>.
*/
public static boolean pingURL(String url, int timeout) {
url = url.replaceFirst("^https", "http"); // Otherwise an exception may be thrown on invalid SSL certificates.
try {
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setConnectTimeout(timeout);
connection.setReadTimeout(timeout);
connection.setRequestMethod("HEAD");
int responseCode = connection.getResponseCode();
return (200 <= responseCode && responseCode <= 399);
} catch (IOException exception) {
return false;
}
}
Instead of using URLConnection use HttpURLConnection by calling openConnection() on your URL object.
Then use getResponseCode() will give you the HTTP response once you've read from the connection.
here is code:
HttpURLConnection connection = null;
try {
URL u = new URL("http://www.google.com/");
connection = (HttpURLConnection) u.openConnection();
connection.setRequestMethod("HEAD");
int code = connection.getResponseCode();
System.out.println("" + code);
// You can determine on HTTP return code received. 200 is success.
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (connection != null) {
connection.disconnect();
}
}
Also check similar question How to check if a URL exists or returns 404 with Java?
Hope this helps.
You could also use HttpURLConnection, which allows you to set the request method (to HEAD for example). Here's an example that shows how to send a request, read the response, and disconnect.
The following code performs a HEAD request to check whether the website is available or not.
public static boolean isReachable(String targetUrl) throws IOException
{
HttpURLConnection httpUrlConnection = (HttpURLConnection) new URL(
targetUrl).openConnection();
httpUrlConnection.setRequestMethod("HEAD");
try
{
int responseCode = httpUrlConnection.getResponseCode();
return responseCode == HttpURLConnection.HTTP_OK;
} catch (UnknownHostException noInternetConnection)
{
return false;
}
}
public boolean isOnline() {
Runtime runtime = Runtime.getRuntime();
try {
Process ipProcess = runtime.exec("/system/bin/ping -c 1 8.8.8.8");
int exitValue = ipProcess.waitFor();
return (exitValue == 0);
} catch (IOException | InterruptedException e) { e.printStackTrace(); }
return false;
}
Possible Questions
Is this really fast enough?Yes, very fast!
Couldn’t I just ping my own page, which I want
to request anyways? Sure! You could even check both, if you want to
differentiate between “internet connection available” and your own
servers beeing reachable What if the DNS is down? Google DNS (e.g.
8.8.8.8) is the largest public DNS service in the world. As of 2013 it serves 130 billion requests a day. Let ‘s just say, your app not
responding would probably not be the talk of the day.
read the link. its seems very good
EDIT:
in my exp of using it, it's not as fast as this method:
public boolean isOnline() {
NetworkInfo netInfo = connectivityManager.getActiveNetworkInfo();
return netInfo != null && netInfo.isConnectedOrConnecting();
}
they are a bit different but in the functionality for just checking the connection to internet the first method may become slow due to the connection variables.
Consider using the Restlet framework, which has great semantics for this sort of thing. It's powerful and flexible.
The code could be as simple as:
Client client = new Client(Protocol.HTTP);
Response response = client.get(url);
if (response.getStatus().isError()) {
// uh oh!
}