I try to implement a Java Proxy for Http (Https will be the extension after Http works). I found a lot of resources on the Internet and try to solve all problems on my own so far. But now I come to a point where I stuck.
My Proxoy does not load the full http websites. I get a lot of error messages with the socket is already closed. So I think I try to send something over a Socket that is closed.
My Problem is now. I can not see why it is like this. I think a lot over the problem but I can not find the mistake. From my side The Sockets only get closed when the server close the connection to my Proxy Server. This happen when I read a -1 on the input stream from the server.
I would be happy for any help :-)
greetings
Christoph
public class ProxyThread extends Thread {
Socket client_socket;
Socket server_socket;
boolean thread_var = true;
int buffersize = 32768;
ProxyThread(Socket s) {
client_socket = s;
}
public void run() {
System.out.println("Run Client Thread");
try {
// Read request
final byte[] request = new byte[4096];
byte[] response = new byte[4096];
final InputStream in_client = client_socket.getInputStream();
OutputStream out_client = client_socket.getOutputStream();
in_client.read(request);
System.out.println("---------------------- Request Info --------------------");
System.out.println(new String(request));
Connection conn = new Connection(new String(request));
System.out.println("---------------------- Connection Info --------------------");
System.out.println("Host: " + conn.host);
System.out.println("Port: " + conn.port);
System.out.println("URL: " + conn.URL);
System.out.println("Type: " + conn.type);
System.out.println("Keep-Alive:" + conn.keep_alive);
server_socket = new Socket(conn.URL, conn.port);
InputStream in_server = server_socket.getInputStream();
final OutputStream out_server = server_socket.getOutputStream();
out_server.write(request);
out_server.flush();
Thread t = new Thread() {
public void run() {
int bytes_read;
try {
while ((bytes_read = in_client.read(request)) != -1) {
out_server.write(request, 0, bytes_read);
out_server.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
t.start();
int bytes_read;
while ((bytes_read = in_server.read(response)) != -1) {
out_client.write(response, 0, bytes_read);
out_client.flush();
//System.out.println("---------------------- Respone Info --------------------");
//System.out.println(new String(response));
}
//System.out.println("EIGENTLICH FERTIG");
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client_socket.close();
server_socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
EDIT:
My HTTP Proxy now works. The Answer is pretty helpfull once you understand what is ryl going on. If you come hear to find a solution this questions may help you:
Does the client send a request only to one Website / Webserver? Means do we always have the same port / hostname?
The Loop from the answer is very usefull but think where to place it?
Last think: Thanks #EJP its working your reply was very usefull. It only tooks a time to understand it!
You are making all the usual mistakes, and a few more.
The entire request is not guaranteed to arrive in a single read. You can't assume more than a single byte has arrived. You have to loop.
You aren't checking for end of stream at this stage.
You need a good knowledge of RFC 2616 to implement HTTP, specifically the parts about Content-length and transfer encoding.
You cannot assume that the server will close the connection after sending the response.
Closing either the input or the output stream or a socket closes the socket. This is the reason for your SocketException: socket closed.
When you get to HTTPS you will need to look at the CONNECT verb.
Flushing a socket output stream does nothing, and flushing inside a loop is to be avoided,
Related
I can set up client Socket to send request to server (sendData() method) and read the received message (readData() method) correctly, but I only received message each time I send the request to server by using MOBILE_REQUEST string, through these codes:
#Override
protected Boolean doInBackground(String... params) {
try {
mSocket = new Socket(
// PC Ip is 192.168.1.199
// It is the other device, Not be local host : 127.0.0.1
Pas.pas.getPcIP(), 17001);
DataOutputStream mDos = new DataOutputStream(mSocket.getOutputStream());
String RESPONSE = null;
String MOBILE_BLOCK = "MobileBlock#";
// Converting collected data in byte array into String.
RESPONSE = sendData(mDos, MOBILE_BLOCK);
/**
* The result response from PC app in here
*/
// Log : response - #WindowsResp#192.168.1.199#
Log.i("", "response '" + RESPONSE + "'");
}
} catch (SocketTimeoutException e) {
IS_SOCKET_TIME_OUT = true;
e.printStackTrace();
} catch (ConnectException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return false;
}
sendData() method - Client send the request to server and wait to get response data - String data.
private String sendData(DataOutputStream mDos, String MOBILE_REQUEST) {
try {
// Log : MOBILE_REQUEST.getBytes() - [B#82f10f8
mDos.write(MOBILE_REQUEST.getBytes());
// todo I should set this sleep, bcs TCP has delay time,
// so i need set the delay time for client should receive data
// otherwise, sometimes I did not receive anything
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
// Log : #WindowsResp#192.168.1.199#
return new String(readData(mSocket));
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
readData() method - Read data after received from server.
public static byte[] readData(Socket mSocket) {
/* Since data are accepted as byte, all of them will be collected in the
following byte array which initialised with accepted data length. */
DataInputStream mDis = null;
try {
mDis = new DataInputStream(mSocket.getInputStream());
// Log : mDis.available() - 23
byte[] data = new byte[mDis.available()];
// Collecting data into byte array
for (int i = 0; i < data.length; i++)
data[i] = mDis.readByte();
// Log : data - [B#30c044a4
return data;
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
What I want is every time server send to my socket client the message by clicking button, the socket client should receive it. But in above codes, it's not.
Please help me how to set up socket client always listen from server?
p/s : Or do I need set up ServerSocket? If use ServerSocket I cannot use same port, right? Because when I open ServerSocket for listen first (ex. at port 17001), I can not use client socket to send request via port 17001 because that port is already used.
UPDATED
The way server (PC app - laptop device) send to client (Mobile device - Android) is via Socket TCP, through these steps :
1 - Client (Android device) set up TCP socket connection to Server (PC app) (this connection never closed until exit app in onDestroy() method).
2 - Client send request to server, ex. MOBILE_REQUEST = "MobileID#MobileIP#"
3 - Server received the request from client, It replied to client via Socket connection, actually client received data correctly. ex. "WindowsRep#WindowsIP"
This way not work for me, even socket TCP connection not closed, and getInputStream() not shut down yet. In this case :
Server send string data to client via Socket connection, client received data correctly.
What I want is every time "Server send string data to client via Socket connection, client received data correctly". But in my case, client only receive data after sent request to server.
C# Server
Server socket
IPEndPoint ipe = new IPEndPoint("192.168.1.199", 17001);
svSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
svSocket.Bind(ipe);
Server Send data
string data_send = "#WBroad#" + "192.168.1.199" + "#";
byte[] byteData = Encoding.UTF8.GetBytes(data_send);
c.ClientSocket.Send(byteData);
As you haven't posted the sending code it is impossible to tell why you're not receiving data, but here is a quick critique of what you have posted:
// mSocket.setReuseAddress(true);
You've commmented this out, but it would be pointless to call this method now. You would have to construct the socket as new Socket(), with no parameters, then call this method, then call connect(). And as you aren't providing a source port or IP address to be re-used, it would still be pointless.
byte[] data = new byte[mDis.available()];
This is a complete misuse of available(). It does not provide a message length. See the Javadoc. There is no reason to believe that whatever data has arrived at this point, if any, is a complete message, or only one message. If you want messages you won't get any help from TCP: you will have to implement them yourself. As your protocol appears to be text-based I suggest you just use lines and readLine(), with BufferedReader and BufferedWriter instead of the DataInput/OutputStreams. And construct those once for the life of the socket, not once per application message, otherwise you will lose data.
// Collecting data into byte array
for (int i = 0; i < data.length; i++)
data[i] = mDis.readByte();
The huge problem with this is that it won't block, because, most of the time, available() will be zero, so this method will do nothing except return an empty byte[] array.
In any case this is entirely equivalent to mDis.readFully(data);, only several times less efficient, but you shouldn't be doing this anyway: see above.
return data;
} catch (IOException e) {
e.printStackTrace();
}
return null;
This is poor practice. You should let the IOException be thrown by this method and let the caller deal with it.
mDos.write(MOBILE_REQUEST.getBytes());
See above. This should include a line terminator, a length-word prefix, or some other way of delimiting the message.
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
This sleep is literally a complete and utter waste of time. And space. Remove it. Sleeping in networking code is just cargo-cult programming.
return new String(readData(mSocket));
This will throw a NullPointerException if readData() returns null, which it does if there was an IOException, which is another reason to let that method propagate that exception instead of catching it internally and returning null.
private boolean splitData(int mobile_send_request_case, String DATA) {
This method is entirely irrelevant to the problem and should not have been posted.
Or do I need set up ServerSocket?
No. Why do you think that?
If use ServerSocket I cannot use same port, right?
Wrong.
Because when I open ServerSocket for listen first (ex. at port 17001), I can not use client socket to send request via port 17001 because that port is already used.
Wrong again. It isn't.
As I said above, it's impossible to help you further when you don't post all the relevant code, but there's enough wrong with this already that you really need to start again.
I want a proxy in java that get response of a server in socket and change the html code and then pass the client as a response in socket connection.
Now My program can get request from client and can get response that from server and pass that to client Properly.
but I can not change the html code.This is my code:
Thread thread = new Thread() {
public void run() {
int bytesRead;
try {
while ((bytesRead = streamFromClient.read(request)) != -1) {
streamToServer.write(request, 0, bytesRead);
streamToServer.flush();
}
Logging incomingLog = new Logging("Incoming", tmpClient.toString());
incomingLog.doLog();
} catch (IOException e) {
Logging IOExceptionLog = new Logging("Error", "Proxy cannot read client request - Client: "
+ tmpClient.toString() + ".\nException : " + e.getMessage() + "\n");
try {
IOExceptionLog.doLog();
} catch (IOException e1) {
//Ignore me!!!
}
}
// the client closed the connection to us, so close our
// connection to the server.
try {
streamToServer.close();
} catch (IOException e) {
Logging log = new Logging("Error", "Proxy could not close connection to server.");
try {
log.doLog();
} catch (IOException e1) {
//Ignore me!!!
}
}
}
};
thread.start();// Start the client-to-server request thread running;
// Read the server's responses
// and pass them back to the client;
int bytesRead;
InputStream tempStreamFromServer = streamFromServer;
/*ConvertStream convertor = new ConvertStream();
String htmlCode = convertor.getStringFromInputStream(tempStreamFromServer);*/
try {
while ((bytesRead = streamFromServer.read(response)) != -1) {
streamToClient.write(response, 0, bytesRead);
streamToClient.flush();
}
Logging incomingLog = new Logging("OutComing", tmpClient.toString());
incomingLog.doLog();
} catch (IOException e) {
Logging IOExceptionLog = new Logging("Error", "Proxy cannot send client response - Client: "
+ tmpClient.toString() + ".\nException : " + e.getMessage() + "\n");
IOExceptionLog.doLog();
}
// The server closed its connection to us, so we close our
// connection to our client.
streamToClient.close();
Can anyone help me?
I finally can find problem.
problem was in threads.I had two thread in my program:
1.Thread A: get response of server.
2.Thread B: change the html code and pass them to client.
Now the problem was that when thread A had a loop that took responses from servers and pass them to thread B, and thread B also had a loop that wait for responses from thread A.
Now when thread B want change html code, this action give a delay to thread B and thread A don't wait for thread B.
I solved this problem with wait thread A after pass one response to thread B and notify thread A from thread B after process html code.
for more information read : http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html
I got to implement a chat in my application. Connection to a server is made using sockets. I should register to that server and the server will aknowledge that with a reply.
I have implemented this in a single method where I send the command using a BufferedWriter, and then start reading from the input stream until it tells me there is no more data.
I read properly the server reply. However, I never get the negative value from the second in.read call and thus my method stays blocked in the while loop (in the conditionnal statement where I make that call).
How should this be done with sockets? I usually do that with files or other input streams without problem.
If I should read only the bytes I am supposed to read, does that mean that I either have to:
Know in advance the length of the server response?
or make the server send a code to notify it has finished to send its response?
Currently I am doing the following:
private String sendSocketRequest(String request, boolean skipResponse) throws ChatException {
if (!isConnected()) openConnection();
try {
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
socket.getOutputStream()), 2048);
out.append(request);
out.flush();
out = null;
} catch (IOException e) {
LogHelper.error("Unable to send socket request: " + request, e);
throw new ChatException("Unable to send socket request: " + request, e);
}
try {
BufferedReader in = new BufferedReader(new InputStreamReader(
socket.getInputStream()), 2048);
StringBuffer response = new StringBuffer();
char[] buffer = new char[2048];
int charsRead = -1;
// >>>>>>>> This is where it gets blocked <<<<<<<<<
while ((charsRead = in.read(buffer)) >= 0) {
if (charsRead > 0) response.append(new String(buffer, 0, charsRead));
}
return response.toString();
} catch (IOException e) {
LogHelper.error("Unable to read socket response: " + request, e);
throw new ChatException("Unable to read socket response: " + request, e);
}
}
Connection to the server is made with the following method:
public synchronized void openConnection() throws ChatException {
try {
socket = new Socket(Constants.API_CHAT_SERVER_ADDRESS, Constants.API_CHAT_SERVER_PORT);
socket.setKeepAlive(true);
LogHelper.debug("CHAT >> Connected to the chat server: " + Constants.API_CHAT_SERVER_ADDRESS);
} catch (UnknownHostException e) {
LogHelper.error("Unable to open chat connection", e);
throw new ChatException("Unable to open chat connection", e);
} catch (IOException e) {
LogHelper.error("Unable to open chat connection", e);
throw new ChatException("Unable to open chat connection", e);
}
}
The amount of data to be sent/received over a socket based connection is protocol dependend and not known to the TCP/IP stack, but only to the application layer.
The protocol used is developer dependend ... ;-) so coming to your questions:
If I should read only the bytes I am supposed to read, does that mean that I either have to:
Know in advance the length of the server response?
Yes, this is one possibility.
or make the server send a code to notify it has finished to send its response?
Also yes, as this is another possibility. Common markers are \n or \r\n. The NUL/'\0' character also might make sense.
A third option is to prefix each data chunk with a constant number of bytes describing the amount of bytes to come.
Instead of dealing with bytes, maybe it's simpler handling instances of ad-hoc classes, like - for instance - a Message class:
The server:
// Streams
protected ObjectInputStream fromBuffer = null;
protected ObjectOutputStream toBuffer = null;
// Listening for a new connection
ServerSocket serverConn = new ServerSocket(TCP_PORT);
socket = serverConn.accept();
toBuffer = new ObjectOutputStream(socket.getOutputStream());
fromBuffer = new ObjectInputStream(socket.getInputStream());
// Receiving a new Message object
Message data = (Message)fromBuffer.readObject();
The client then sends a message by simply:
// Sending a message
Message data = new Message("Hello");
toBuffer.writeObject(data);
Message can be as complex as needed as long as its members implement Serializable interface.
I'm reading a string from a buffer and writing it to a server. The problem I'm having is that the string never gets received by the server when I leave the socket open and write in a loop.
When I use this:
try {
Socket send = new Socket("localhost", 1490);
DataOutputStream out = new DataOutputStream(send.getOutputStream());
String message = null;
while ((message = buffer.get()) != null){
out.writeBytes(message);
}
out.close();
send.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
the server doesn't receive the string, but when I do this it works properly:
try {
String message = null;
while ((message = buffer.get()) != null){
Socket send = new Socket("localhost", 1490);
DataOutputStream out = new DataOutputStream(send.getOutputStream());
out.writeBytes(message);
out.close();
send.close();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
Obviously I don't want to keep opening and closing the socket, though. What is the problem?
You need to flush your socket every time you want to send a data packet.
Closing a socket forces an automatic flush and that explains why your data is getting sent on socket close.
The data is not being written to the socket even when you close it? (in your first snippet that is)
Also, have you tried to use the flush method? You can read about it here: http://docs.oracle.com/javase/1.4.2/docs/api/java/io/DataOutputStream.html#flush() and your code will look like:
try {
Socket send = new Socket("localhost", 1490);
DataOutputStream out = new DataOutputStream(send.getOutputStream());
String message = null;
while ((message = buffer.get()) != null){
out.writeBytes(message);
out.flush();
}
out.close();
send.close();
} catch (IOException ex) {
ex.printStackTrace();
}
Let me make a guess.
Does the buffer.get() method block? If so, then the problem is that out.writeBytes(message) does not guarantee that the entire byte representation to be pushed to the server. Instead. there is a good chance that your client has buffered bytes waiting to be flushed through to the server.
If this is what is going on, then calling flush after each call to writeBytes will fix the problem.
But if the buffer.get() method doesn't block, then calling flush won't make any difference. In fact, it will just increase the network traffic. So adding the flush "just in case" is a bad idea.
Another possibility is that there is something wrong with the server-side code.
I have a PC server and an android client; my android client start a socket connection to server.
While I am connected to server, I also receive data from server to android client;
Here is my code:
Socket socket = null;
DataOutputStream out = null;
DataInputStream in = null;
InputStream inputStream = null;
OutputStream outputStream = null;
...
public void connectToTCP()
{
try
{
socket = new Socket(HOST_ADDRESS, PORT);
socket.setSoTimeout(30000);
outputStream = socket.getOutputStream();
out = new DataOutputStream(outputStream);
inputStream = socket.getInputStream();
in = new DataInputStream(inputStream);
Log.e("TCP-", "Connected");
while (socket.isConnected()){readBytes();}
}
catch (UnknownHostException e)
{
Log.e("Error in tcp connection","Unknown Host");
}
catch (IOException e)
{
Log.e("Error in tcp connection", "Couldn't get I/O for the connection");
}
}
public void readBytes() throws IOException
{
if (in.available() > 0)
{
byte[] buffer = new byte[in.available()];
if (buffer.length > 0)
{
if (mListener != null)
{
int numberOfBytes = in.read(buffer);
mListener.tcpConnectionDataReceived(buffer, numberOfBytes);
}
}
}
}
but my problem is in performance. I tested the code on the device and I noticed (from task manager) that the app consume a lot of resources (CPU usage is more than 50%) but when I stop reading from socket by deleting this while loop while (socket.isConnected()){readBytes();} CPU usage becomes less than 1%.
Any ideas to solve this?
You readBytes() method will return immediately if no data is available. Since it's in a tight loop, you're essentially continuously checking if there is something available, wasting a lot of CPU power.
With the code you show, you would be better off doing a plain blocking read (i.e. remove the available() check altogether, and use a reasonable, fixed-size buffer).
You should sleep between calls to readBytes() - you basically created an endless loop if no data is available and thus in.available() > 0 is false.
Or if this is in its own background thread, just do blocking reads when you know that more data is expected.