We Keep Coding

Can't connect to HTTPS sites using Socket HTTP-proxy

I have written a simple class extends Thread, used to receive user request, send it to HTTP-proxy server, runs on my Android device, get response and send it back to user:
public class ProxyThread extends Thread {
private Socket clientSocket = null;
public ProxyThread(Socket socket) {
super("ProxyThread");
this.clientSocket = socket;
}
public void run() {
try {
// Read request
InputStream incomingIS = clientSocket.getInputStream();
byte[] b = new byte[8196];
int len = incomingIS.read(b);
if (len > 0) {
System.out.println("REQUEST"
+ System.getProperty("line.separator") + "-------");
System.out.println(new String(b, 0, len));
// Write Request
Socket socket = new Socket("192.168.136.96", 8080);
OutputStream outgoingOS = socket.getOutputStream();
outgoingOS.write(b, 0, len);
// Send back
OutputStream incomingOS = clientSocket.getOutputStream();
InputStream outgoingIS = socket.getInputStream();
for (int length; (length = outgoingIS.read(b)) != -1; ) {
incomingOS.write(b, 0, length);
System.out.println("Answer"
+ System.getProperty("line.separator") + "-------");
System.out.println(new String(b, 0, len));
}
incomingOS.close();
outgoingIS.close();
outgoingOS.close();
incomingIS.close();
socket.close();
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
clientSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
System.out.println("closed");
}
I used FireFox to connect to the server (localhost:80). So it works fine with sites using HTTP (example http://info.cern.ch/), but it is not for sites using HTTPS.
For example i try to connect https://facebook.com.
FireFox sends a request:
CONNECT www.facebook.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: www.facebook.com:443
Proxy sends back a response:
HTTP/1.1 200 Connection Established
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: www.facebook.com:443
I don't know why FireFox does show nothing, but maybe it just can't recive an response from my server (localhost:80) and i don't know why.

Trying to make send a message when the enter key is pressed

I am trying to send a message to the server. My problem is that the client keeps waiting for input and never sends it to the server until the last message that tells it to terminate (which is "bye"). After client terminates, the server will receive the message. But I want the server to receive the message everytime the client hit's the enter key.
I am not sure, but I think that the problem is with the Bufferedwriter in the client, because if I connect to the server using the browser, the server receives all the browser information. An example is shown below:
Connect to this server name: localhost and port: 121
Connection from client: Socket[addr=/0:0:0:0:0:0:0:1,port=61942,localport=121]
From client > GET / HTTP/1.1
From client > Host: localhost:121
From client > Connection: keep-alive
From client > Cache-Control: max-age=0
From client > Upgrade-Insecure-Requests: 1
From client > User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36
...
Relevant server code:
try {
sers = new ServerSocket(port);
client = sers.accept();
reader = new BufferedReader(new InputStreamReader(client.getInputStream()));
writer = new BufferedWriter(new OutputStreamWriter(client.getOutputStream()));
boolean clientIsOnline = true;
while (clientIsOnline) {
print("Connection from client: " + client);
while((msg = reader.readLine()) != null) {
print("From client > " + msg);
if(msg.toLowerCase().equals("bye"))
print("Client left, server closed");
clientIsOnline=false;
}
ss.close();
reader.close();
writer.close();
}
} catch(Exception e) {
print("Error starting server: ");
e.printStackTrace();
}
Relevant client code:
public Client(String adr, int port) {
this.adr=adr;
this.port=port;
try {
c=new Socket(adr, port);
r = new BufferedReader(new InputStreamReader(c.getInputStream()));
w = new PrintWriter(new OutputStreamWriter(c.getOutputStream()));
} catch (IOException e) {
e.printStackTrace();
}
}
Client method for sending the message
private void msgToServer(String msg) {
try {
w.write(msg);
w.flush();
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
if(msg.toLowerCase().equals("bye")) {
print("I am out, bye");
try {
c.close();
r.close();
w.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Client main method:
public static void main(String[] args) {
int port=121;
String msg =null;
Scanner sc = new Scanner(System.in);
Client c = new Client("localhost", port);
System.out.println("Write something to server");
while(true) {
c.msgToServer(sc.next());
}
}
I also tried using the PrintWriter instead of BufferedWriter, and also tried adding the true keyword as parameter, like
new PrintWriter(new OutputStreamWriter(c.getOutputStream()), true);
Just to give an example of what it looks like using my client.
I type hello hit enter (I expect it to send this, but it doesn't). Then I type my name is john hit enter again. Finally I type bye
I excpect:
From client > hello
From client > my name is john
From client > bye
What I get: From client > hellomynameisjohnbye

I believe the newline is being consumed when the client reads the input, so it needs to be added when sent to the server. In the client method:
private void msgToServer(String msg) {
try {
// re-introduce newline:
w.println(msg);
[SNIP]

Java Server-Client Connection reset

I have server and client applications running on my local machine.
Client takes file, changes it and sends to server, then server responds if the file is correct. Client does it multiple times, sending one file at a time.
I send two files from client and on the second file I get Connection reset
Server snippet:
private void initServer() throws IOException {
while (true) {
ServerSocket server = new ServerSocket(55555);
Socket fromclient = server.accept();
InputStream sin = fromclient.getInputStream();
OutputStream sout = fromclient.getOutputStream();
DataInputStream in = new DataInputStream(sin);
DataOutputStream out = new DataOutputStream(sout);
String line = in.readUTF();
if (line.equals("new file")) {
long fileSize = in.readLong();
tempSavedFile = new File("/home/evgeniy/Files/Downloads/temp");
tempSavedFile.createNewFile();
try (FileOutputStream fos = new FileOutputStream(tempSavedFile)) {
int t;
for (int i = 0; i < fileSize; i++) {
t = sin.read();
fos.write(t);
}
}
if (checkPadding(tempSavedFile)) {
out.writeInt(PADDING_OK_RESPONSE);
} else {
out.writeInt(PADDING_ERROR_RESPONSE);
}
out.flush();
}
out.close();
in.close();
sout.close();
sin.close();
fromclient.close();
server.close();
}
}
Client class that calls new thread in for loop
for (byte i = 0; i < 2; i++) {
Callable callable = new FileSender(tempFile);
FutureTask<Integer> ftask = new FutureTask<>(callable);
Thread thread = new Thread(ftask);
thread.start();
int response = 3244;
try {
response = ftask.get();
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(FXMLController.class.getName()).log(Level.SEVERE, null, ex);
}
putMessage(String.valueOf(response));
Client Callable thread:
public Integer call() throws Exception {
Socket socket = new Socket(address, serverPort);
InputStream sin = socket.getInputStream();
OutputStream sout = socket.getOutputStream();
DataInputStream in = new DataInputStream(sin);
DataOutputStream out = new DataOutputStream(sout);
out.writeUTF("new file");
out.flush();
out.writeLong(file.length);
out.flush();
sout.write(file);
sout.flush();
System.out.println(socket.isConnected());
int response = in.readInt();
System.out.println("--------RESP="+response);
out.close();
in.close();
sin.close();
sout.close();
socket.close();
return response;
}
As you can see I send two files, and get this console output on client app:
true
--------RESP=200
true
ноя 20, 2018 5:16:36 PM com.evgeniy_mh.paddingoracle.FXMLController SendFileToServer
SEVERE: null
java.util.concurrent.ExecutionException: java.net.SocketException: Connection reset
Also, I don't understand why socket is ok, but
int response = in.readInt();
raising Connection reset exception.

Your code creates a new ServerSocket and later closes the created server socket for every single request that it processes. The code does not use the same ServerSocket instance to serve multiple requests.
The connection reset failure appears to be because of this, as when the second request is creating a socket connection with the server, the existing ServerSocket is closed and recreated on the same port leading to the connection being reset. For a start try taking out the ServerSocket creation outside the while loop.
private void initServer() {
try (ServerSocket server = new ServerSocket(5555)) {
while (true) {
Socket fromclient = server.accept();
... // remaining code
fromclient.close();
}
} catch (IOException ioe) {
// handle failures
}
}
The try-with-resources handles the .close() call on the AutoCloseable server socket.
Also, note that the requests would be processed serially on a single thread in your code. Usually the Socket obtained for a connection from the ServerSocket#accept() (or the streams derived from it) is passed to a separate thread for processing unlike your code that processes the requests serially.

HTTP and Java - modified headers in gzip encoding

I've got strange issue with my http communication. I wrote following very simple http server and client.
Http server:
public class SimpleHttpServer {
public void run() throws IOException {
final ServerSocket serverSocket = new ServerSocket(8080);
while(true)
{
final Socket connectionSocket = serverSocket.accept();
new Thread(new Runnable() {
#Override
public void run() {
try {
try(BufferedReader in = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream()));
DataOutputStream out = new DataOutputStream(connectionSocket.getOutputStream())){
System.out.println(in.readLine());
String line = null;
while((line = in.readLine()) != null && !line.trim().isEmpty()){
System.out.println(line);
}
byte[] message = loadFile(); //load some gzipped jpg image from disk
out.write(ascii("HTTP/1.1 200 OK\n"));
out.write(ascii("Content-Type: image/jpg\n"));
out.write(ascii(String.format("Content-Length: %s\n", message.length)));
out.write(ascii("content-encoding: gzip\n"));
out.write(ascii("\n"));
out.write(message);
}
}catch(Exception e){
e.printStackTrace();
}
}
}).start();
}
}
}
Http client:
public class SimpleHttpClient {
public void run() throws IOException {
String host = "localhost";
int port = 8080;
Socket socket = new Socket(host, port);
try (OutputStream out = socket.getOutputStream()) {
out.write(ascii("GET / HTTP/1.1\n"));
out.write(ascii("User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\n"));
out.write(ascii("Accept-Encoding: gzip, deflate, sdch\n"));
out.write(ascii("Accept-Language: pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4\n"));
out.write(ascii("\n"));
try (BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()))) {
String line = null;
while ((line = in.readLine()) != null && !line.trim().isEmpty()) {
System.out.println(line);
}
}
}
}
}
So, when I remove line with user agent header, I get result as I expect:
HTTP/1.1 200 OK
Content-Type: image/jpg
Content-Length: 378632
content-encoding: gzip
And this is clear and obvious.
But when this header will appear, the answer is different:
HTTP/1.1 200 OK
Content-Type: image/jpg
Transfer-Encoding: chunked
I didn't change anything on server side. On every request I send the same response. The user agent was copied from chrome browser.
Both implementations are in plain java socket, not on third party libriares.
So my question is: how it happend? Something was cached? But where?
Please don't look at code quality. This is just an example. I don't want to use it on production :)
EDIT 1:
private byte[] ascii(String value) throws UnsupportedEncodingException {
return value.getBytes(StandardCharsets.US_ASCII);
}
EDIT 2:
I've done some research. It happens when user agent contains "AppleWebKit", protocol name is HTTP and port is 80 or 8080.
Moreover it happens on Windows 10 (I checked on two different computers). On Linux (Ubuntu 16.04 and Debian 8) everything is fine.
Maybe I should change the title of question? But how can I explain the problem in one short sentence?

connecting to google host and return nothing

I was wrote fowling code then set HTTP proxy on 127.0.0.1:9090 and try to reach google.com but it print following output and nothing happend.
Output:
Waiting for clients on port 9090
Got connection from /127.0.0.1:11827
Active Connections = 1
Waiting for clients on port 9090
connect started
connect finished
***Got connection from /74.125.232.131:80
writed
GET http://google.com/ HTTP/1.1
Host: google.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: ****
Connection: keep-alive
null
read line startes
Code:
import java.net.*;
import java.io.*;
import java.util.*;
public class SocketGoogler
{
public static void main(String[] args)
{
int port = 9090;
try {
ServerSocket server = new ServerSocket(port);
//System.out.println(google());
while(true) {
System.out.println("Waiting for clients on port " + port);
Socket client = server.accept();
ConnectionHandler handler = new ConnectionHandler(client);
handler.start();
}
} catch(Exception ex) {
System.out.println("Connection error: "+ex);
}
}
}
class ConnectionHandler extends Thread {
private Socket client;
BufferedReader reader;
PrintWriter writer;
static int count;
public ConnectionHandler(Socket client) {
this.client = client;
System.out.println("Got connection from "+client.getInetAddress()
+":"+client.getPort());
count++;
System.out.println("Active Connections = " + count);
}
public void run() {
String message;
String totalMessage;
try {
reader = new BufferedReader(new
InputStreamReader(client.getInputStream()));
writer = new PrintWriter(client.getOutputStream());
writer.flush();
message = reader.readLine();
totalMessage=message+"\n";
while (message != null) {
message = reader.readLine();
totalMessage+=message+"\n";
}
client.close();
//System.out.println(totalMessage);
google(totalMessage);
count--;
System.out.println("Active Connections = " + count);
} catch (Exception ex) {
count--;
System.out.println("Active Connections = " + count);
}
}
public String google(String w) throws IOException
{
String message , totalMessage;
int port = 80;
//ServerSocket server = new ServerSocket(port,10,InetAddress.getByName("google.com"));
//Socket googler=server.accept();
Socket googler=new Socket();
InetSocketAddress endpoint=new InetSocketAddress(InetAddress.getByName("google.com").getHostAddress(), port);
System.out.println("connect started");
googler.connect(endpoint);
System.out.println("connect finished");
System.out.println("***Got connection from "+googler.getInetAddress()+":"+googler.getPort());
BufferedReader reader = new BufferedReader(new
InputStreamReader(googler.getInputStream()));
PrintWriter writer=new PrintWriter(googler.getOutputStream());
//String w="GET http://google.com/ HTTP/1.1\nHost: google.com\nUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\nAccept-Language: en-US,en;q=0.5\nAccept-Encoding: gzip, deflate\nConnection: keep-alive"+null;
writer.print(w);
System.out.println("writed\n"+w);
System.out.println("read line startes");
message = reader.readLine();
System.out.println("read line finished");
totalMessage=message+"\n";
while (message != null) {
message = reader.readLine();
System.out.println("*");
totalMessage+=message+"\n";
}
googler.close();
System.out.println("close");
return totalMessage;
}
}
Why this problems happens?I already connected too google.com and send request but nothing respond from this host.

Use writer.flush(); It will send your request.
:)