I'm trying to write a Java HTTP Proxy Tunnelling program, and I need an experts advice about the best and fastest stream to use for the communication.
I've implemented the basic functionality and everything works fine. The only matter is communication speed or performance. My HTTP proxy system consists of a server program, running on a remote server and a client program running on the local machine. So far, the program looks like this:
Listener.java :
/**
* Listens and accepts connection requests from the browser
*/
ServerSocket listener = null;
try {
listener = new ServerSocket(port, 128);
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
ExecutorService executor = Executors.newCachedThreadPool();
Socket connection;
while (!shutdown) {
try {
connection = listener.accept();
executor.execute(new ProxyTunnel(connection));
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
}
ProxyTunnel.java :
try {
byte[] buffer = new byte[8192]; // 8-KB buffer
InputStream browserInput = browser.getInputStream();
OutputStream browserOutput = browser.getOutputStream();
// Reading browser request ...
StringBuilder request = new StringBuilder(2048);
do {
int read = browserInput.read(buffer);
logger.log(read + " bytes read from browser.");
if (read > 0) {
request.append(new String(buffer, 0, read));
}
} while (browserInput.available() > 0 && read > 0);
// Connecting to proxy server ...
Socket server = new Socket(SERVER_IP, SERVER_PORT);
server.setSoTimeout(5000); // Setting 5 sec read timeout
OutputStream serverOutput = server.getOutputStream();
InputStream serverInput = server.getInputStream();
// Sending request to server ...
serverOutput.write(request.toString().getBytes());
serverOutput.flush();
// Waiting for server response ...
StringBuilder response = new StringBuilder(16384);
do {
try {
read = serverInput.read(buffer);
} catch (SocketTimeoutException ex) {
break; // Timeout!
}
if (read > 0) {
// Send response to browser.");
response.append(new String(buffer, 0, read));
browserOutput.write(buffer, 0, read);
browserOutput.flush();
}
} while (read > 0);
// Closing connections ...
server.close();
} catch (IOException ex) {
ex.printStackTrace(System.err);
} finally {
try {
browser.close();
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
}
The server program uses a similar fashion and sends the HTTP request to the destination server (e.g. www.stackoverflow.com) and forwards the response to the client program, where the client program forwards the response to the local browser.
How can I improve the performance of these TCP/HTTP communications?
Does using buffered streams such as BufferedInputSream and BufferedOutputStream improve communication performance?
Will I gain any performance improvements if I use java.nio Channels and Buffers, instead of java.net Sockets and java.io Stream?
Don't do it yourself
Advice 0: there are plenty of proxy servers out there, much more scalable, stable and mature. Do you really need to write your own?
Don't use StringBuilder/String to buffer request
byte[] buffer = new byte[8192]; // 8-KB buffer
//...
browserInput.read(buffer);
//...
request.append(new String(buffer, 0, read));
//...
serverOutput.write(request.toString().getBytes());
This is flawed for several reasons:
you are assuming your HTTP calls are text (ASCII) only, binary data will be malformed after transforming to String and back to byte[], see: String, byte[] and compression
even if the protocol is text-based, you are using system's default encoding. I bet this is not what you want
finally, the most important part: do not buffer the whole request. Read chunk of data from incoming request and forward it immediately to target server in one iteration. There is absolutely no need for the extra memory overhead and latency. Immediately after receiving few bytes dispatch them and forget about them.
Don't use Executors.newCachedThreadPool()
This pool can grow inifinitely, creating thousands of threads during peak. Essentially you create one thread per connection (except that the pool reuses free threads, but creates new if none available). Consider Executors.newFixedThreadPool(100) - 100-200 threads should be enough in most cases. Above that you'll most likely burn your CPU barely in context switching, without doing much work. Don't be afraid of latency, scale out.
Use non-blocking netty stack
Which brings us to the final advice. Drop blocking sockets altogether. They are handy, but don't scale well due to thread-per-connection requirement. Too much memory is spent to hold stack, too much CPU is wasted for context switching. netty is great and it builds powerful abstraction over NIO.
Check out the examples, they include HTTP client/server code. There is a bit of a learning curve, but you can expect performance growth by several order of magnitued.
Related
When I try to call .read method of inputstream with multiple parallel request,time response increases. I think it is blocked till it read previous one response.
What will be the alternative for InputStream ?
i m using UNIX domain socket and from there i m trying to read data from inputstream,Here the code i m using----------
-
UnixDomainSocketClient socket = new UnixDomainSocketClient(sockFileName, JUDS.SOCK_STREAM);
InputStream in = socket.getInputStream();
OutputStream out = socket.getOutputStream();
out.write(inputStreamData.getBytes());
String modelResponse = "";
while (true) {
try {
modelResponse += (char) in.read();
} catch (Exception e) {
break;
}
}
out.flush();
socket.close();
in.close();
out.close();
}
When I try to call .read method of inputstream with multiple parallel request,time response increases. I think it is blocked till it read previous one response.
No. It is blocked until data arrives on this socket. It has nothing to do with the other sockets, except for the overall load on the machine. Reads are not sequentialized as between different input streams.
What will be the alternative for InputStream?
Non-blocking I/O or asynchronous I/O via the NIO package, but you haven't correctly identified your problem. You may have some undue synchronization in your application. Or maybe your diagnosis is incorrect.
I have a big problem. I have developped a client-server application. A client thread sends a serialized object to the server and the server sends back a serialized object. Currently I'm using one server and 10 client threads and after about 30 seconds I get the error message from each client thread (IOException):
No buffer space available (maximum connections reached?): connect
If I'm looking in netstat then I see that there are a lot of connections created and it is growing and growing and all connections are in TIME_WAIT state.
I don't know why. I close the sockets in the server and in the clients everytime in a finally block. Here is some code:
In the server I have in socketHandlerThread:
ServerSocket serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(5000);
while(true) {
Socket socket = serverSocket.accept();
}
The new socket is then put on a LinkedBlockingQueue and a worker thread takes the socket and makes the following:
try {
outputStream = new ObjectOutputStream(new BufferedOutputStream(socket.getOutputStream()));
outputStream.flush();
inStream = new ObjectInputStream(new BufferedInputStream(socket.getInputStream()));
ClientRequest clientRequest = (ClientRequest) inStream.readObject();
...
outputStream.writeObject(serverResponse);
outputStream.flush();
} catch....
} finally {
if (inStream != null) {
inStream.close();
}
if (outputStream != null) {
outputStream.close();
}
if (socket != null) {
socket.close();
}
}
On the client side I have the following code:
try {
socket = new Socket(host, port);
outputStream = new ObjectOutputStream(new BufferedOutputStream(socket.getOutputStream()));
outputStream.flush();
inputStream = new ObjectInputStream(new BufferedInputStream(socket.getInputStream()));
outputStream.writeObject(request);
outputStream.flush();
Object serverResponse = inputStream.readObject();
} catch....
} finally {
if (inputStream != null) {
inputStream.close();
}
if (outputStream != null) {
outputStream.close();
}
if (socket != null) {
socket.close();
}
}
Can somebody help? I really don't know what mistake I made. I seems that the sockets get no closed but I don't know why.
Could it be the problem that I put the sockets on a queue on the server side so that the socket is somehow copied?
Edit: If I put the client and the server each on a different Amazon EC2 classic instance running Linux AMI then it works. Could it be a problem with Windows or is the problem simply that I was running the Clients and servers on the same machine (my local pc)?
Does somebody see a bug in my code?
Edit2: As said above on EC2 instances it works but if I use netstat it shows still a lot of lines saying TIME_WAIT.
Here are screenshots:
https://drive.google.com/file/d/0BzERdJrwWrNCWjhReGhpR2FBMUU/view?usp=sharing
https://drive.google.com/file/d/0BzERdJrwWrNCOG1TWGo5YmxlaTg/view?usp=sharing
First screenshot is from windows. "WARTEND" means "WAITING" (it is german).
The second screenshot is from Amazon EC2 (to the left the client machine, to the right the server machine).
TIME-WAIT is entered after the connection is closed at both ends. It lasts for a couple of minutes, for data integrity reasons.
If the buffer problem is due to TIME-WAIT states at the server, the solution is to make the server be the peer that first receives the close. That will shift the TIME-WAIT state to the client, where it is benign.
You can do that by putting your server-side request handling into a loop, so that it can handle multiple requests per connection, and so that the server only closes the socket when it reaches end of stream on it.
for (;;)
{
try
{
ClientRequest clientRequest = (ClientRequest) inStream.readObject();
...
outputStream.writeObject(serverResponse);
outputStream.flush();
}
catch (EOFException exc)
{
break;
}
}
If you then implement connection-pooling at the client, you will massively reduce the number of connections, which will further reduce the incidence of the buffer problem.
I have a client socket connected to the server socket, the server will send data to the client from time to time while its connected. currently my client uses a while loop to keep receiving data from the server even the server is not sending anything.
my question is, is there any more efficient way to listen for input?
i am thinking maybe create a thread for the socket connection, and put it to sleep when there is no incoming data, and sends an interrupt when there is data coming in. would that work? if putting the thread to sleep, would it break the socket connection?
i cannot modify the server socket and it doesnt initiate a connection.
import java.io.*;
import java.net.Socket;
public class core_socket {
public static void main(String[] args) {
String host = ("192.168.100.206");
int port = 4025;
try {
Socket socket = new Socket(host, port);
System.out.println("created socket\n");
OutputStream os = socket.getOutputStream();
boolean autoflush = true;
PrintWriter out = new PrintWriter(socket.getOutputStream(), autoflush);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// read the response
boolean loop = true;
StringBuilder sb = new StringBuilder(8096);
while (loop) {
if (in.ready()) {
int i = 0;
while (i != -1) {
i = in.read();
sb.append((char) i);
}
loop = false;
}
}
// display the response to the out console
System.out.println(sb.toString());
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
See multi-user chat application example at http://cs.lmu.edu/~ray/notes/javanetexamples/ - basically, you should consider spawning off a new worker thread for each incoming connection and then go back to listen for any new incoming requests.
A long time ago I wrote one of the first application servers (say in 1997 when most people didn't know what an app server is) - it was deployed at one of the largest higher-ed institutions and processed couple million requests per day during peak times - that's not the institution in the link by the way. The reason I mention this is... multi-threading gives you a tremendous scalability with very little effort - even if scalability is not what you are looking for, worker thread model is still a good practice.
Maybe what you want is to use asynchronous sockets. Basically, it spins off another thread that's job is to listen for any data on the socket. Once data does come in, a "callback" method is called, which can then begin to process your data.
I've never done sockets in Java before though, just C#, so I'm not sure how it compares, but the concept should remain the same.
I have created a server by using ServerSocket. After that, I have created Client using Socket, and connect to this server.
After that, I do "some stuff" with InputStream and OutputStream is taken from Socket Object. But, I don't really understand inputStream and outputStream so much. Here is my simple code :
private Socket sock = null;
private InputStream sockInput = null;
private OutputStream sockOutput = null;
...
String msg = "Hello World";
byte[] buffer = null;
try {
sockOutput.write(msg.getBytes(), 0, test.length());
sockOutput.write("Hello StackOverFlow".getBytes(), 0, test.length());
buffer = new byte[test.length()];
sockInput.read(buffer, 0, test.length());
System.out.println(new String(buffer));
sockInput.read(buffer, 0, test.length());
System.out.println(new String(buffer));
} catch (IOException e1) {
e1.printStackTrace();
}
The result will be : "Hello World" and "Hello StackOverFlow".
Here is server side code :
private int serverPort = 0;
private ServerSocket serverSock = null;
public VerySimpleServer(int serverPort) {
this.serverPort = serverPort;
try {
serverSock = new ServerSocket(this.serverPort);
}
catch (IOException e){
e.printStackTrace(System.err);
}
}
// All this method does is wait for some bytes from the
// connection, read them, then write them back again, until the
// socket is closed from the other side.
public void handleConnection(InputStream sockInput, OutputStream sockOutput) {
while(true) {
byte[] buf=new byte[1024];
int bytes_read = 0;
try {
// This call to read() will wait forever, until the
// program on the other side either sends some data,
// or closes the socket.
bytes_read = sockInput.read(buf, 0, buf.length);
// If the socket is closed, sockInput.read() will return -1.
if(bytes_read < 0) {
System.err.println("Server: Tried to read from socket, read() returned < 0, Closing socket.");
return;
}
System.err.println("Server: Received "+bytes_read
+" bytes, sending them back to client, data="
+(new String(buf, 0, bytes_read)));
sockOutput.write(buf, 0, bytes_read);
// This call to flush() is optional - we're saying go
// ahead and send the data now instead of buffering
// it.
sockOutput.flush();
}
catch (Exception e){
System.err.println("Exception reading from/writing to socket, e="+e);
e.printStackTrace(System.err);
return;
}
}
}
public void waitForConnections() {
Socket sock = null;
InputStream sockInput = null;
OutputStream sockOutput = null;
while (true) {
try {
// This method call, accept(), blocks and waits
// (forever if necessary) until some other program
// opens a socket connection to our server. When some
// other program opens a connection to our server,
// accept() creates a new socket to represent that
// connection and returns.
sock = serverSock.accept();
System.err.println("Server : Have accepted new socket.");
// From this point on, no new socket connections can
// be made to our server until we call accept() again.
sockInput = sock.getInputStream();
sockOutput = sock.getOutputStream();
}
catch (IOException e){
e.printStackTrace(System.err);
}
// Do something with the socket - read bytes from the
// socket and write them back to the socket until the
// other side closes the connection.
handleConnection(sockInput, sockOutput);
// Now we close the socket.
try {
System.err.println("Closing socket.");
sock.close();
}
catch (Exception e){
System.err.println("Exception while closing socket.");
e.printStackTrace(System.err);
}
System.err.println("Finished with socket, waiting for next connection.");
}
}
public static void main(String argv[]) {
int port = 54321;
VerySimpleServer server = new VerySimpleServer(port);
server.waitForConnections();
}
My question is :
When I use sockOutput.write and I can get back those message back by sockInput.read. So, those message has been saved, right? If this true, does it saved on Server I have created or just saved in some other thing such as Socket Object.
If I have written to socket String A1, A2,... An so I will receive A1, A2, ... An String respectively, right?
A socket is an abstraction that you use to talk to something across the network. See diagram below...
In Java, to send data via the socket, you get an OutputStream (1) from it, and write to the OutputStream (you output some data).
To read data from the socket, you get its InputStream, and read input from this second stream.
You can think of the streams as a pair of one-way pipes connected to a socket on the wall. What happens on the other side of the wall is not your problem!
In your case, the server has another socket (the other end of the connection) and another pair of streams. It uses its InputStream (2) to read from the network, and its OutputStream (3) to write the same data back across the network to your client, which reads it again via its InputStream (4) completing the round trip.
Client Server
1. OutputStream -->\ /--> 2. InputStream -->
Socket <--> network <--> ServerSocket |
4. InputStream <--/ \<--3. OutputStream <--
Updated: in reply to comment:
Note that the streams and sockets just send raw bytes; they have no notion of a "message" at this level of abstraction. So if you send X bytes and another X bytes, then read X bytes and read another X bytes, then your system behaves as if there are two messages, because that's how you've divided up the bytes.
If you send X bytes, and another X bytes, then read a reply of length 2X, then you might be able to read a single combined "message", but as you've noticed, the underlying implementation of the streams can choose when to deliver chunks of bytes, so it might return X bytes, then X bytes, later, or 2X at once, or 0.5X four times...
InputStream and OutputStream are two completely separate streams. What you write into one has no a priori relation to what you read from the other. The InputStream gives you whatever data the server decides to send to you. I would also like to comment on this piece of your code:
sockOutput.write(msg.getBytes(), 0, test.length());
sockOutput.write("Hello StackOverFlow".getBytes(), 0, test.length());
You use the length of a string test (not shown in your code), which has nothing to do with the byte array you are passing as the first argument. This can cause an ArrayIndexOutOfBoundsException or truncation of your intended message.
Additional comments to your updated question
Reviewing your server-side code, it is not quite correctly written. You need to have try { handleConnection(...); } finally { socket.close(); } to ensure proper cleanup after an error, as well as when completing normally. Your code never closes anything on the server side.
Finally, and most critically, your entire code is written in a way that can result in a deadlock. Normally you need a separate thread to read and to write; otherwise the following may happen:
You attempt to write some data to the output;
The server reads it and tries to respond with data in your input;
But, since the buffers are too small, you don't manage to send everything because the server wants to first send something to you, then receive the rest; but you don't get to the receiving part before you have sent everything you've got.
I rewrote a simple example of file transfer code between server and client.
And it works.
But i want to make it able to transfer multiple files in a particular directory. User will write the file names (which sits in that particular directory) and client will download them from the server. How can i do that? Any ideas? Thank you.
Client code:
import java.net.*;
import java.io.*;
public class Client {
static String hostname = "127.0.0.1";
static int port = 4588;
static int processedByte;
static byte[] theByte = new byte[1];
static Socket client = null;
static InputStream inuputSt = null;
public static void main(String[] args) throws InterruptedException {
System.out.println("connecting...");
Thread.sleep(500);
try {
client = new Socket(hostname, port);
inuputSt = client.getInputStream();
} catch (IOException ex) {
System.out.println("connection error.");
}
ByteArrayOutputStream arrayOutput = new ByteArrayOutputStream();
if (inuputSt != null) {
FileOutputStream fileOutput = null;
BufferedOutputStream bufferedOutput = null;
try {
System.out.println("downloading target file...");
Thread.sleep(800);
fileOutput = new FileOutputStream("file1_downloaded.txt");
bufferedOutput = new BufferedOutputStream(fileOutput);
processedByte = inuputSt.read(theByte, 0, theByte.length);
do {
arrayOutput.write(theByte);
processedByte = inuputSt.read(theByte);
} while (processedByte != -1);
bufferedOutput.write(arrayOutput.toByteArray());
bufferedOutput.flush();
bufferedOutput.close();
System.out.println("file downloaded");
client.close();
} catch (IOException ex) {
System.out.println("file transfer error.");
}
}
}
}
Server code:
import java.net.*;
import java.io.*;
public class Server {
static int port = 4588;
public static void main(String[] args) {
while (true) {
ServerSocket server = null;
Socket connection = null;
BufferedOutputStream bufferedOutput = null;
try {
server = new ServerSocket(port);
connection = server.accept();
bufferedOutput = new BufferedOutputStream(connection.getOutputStream());
} catch (IOException ex) {
// Do exception handling
}
if (bufferedOutput != null) {
File fileToSend = new File("files\\file1.txt");
byte[] mybytearray = new byte[(int) fileToSend.length()];
FileInputStream fileInputSt = null;
try {
fileInputSt = new FileInputStream(fileToSend);
} catch (FileNotFoundException ex) {
// exception stuff
}
BufferedInputStream bufferedInput = new BufferedInputStream(fileInputSt);
try {
bufferedInput.read(mybytearray, 0, mybytearray.length);
bufferedOutput.write(mybytearray, 0, mybytearray.length);
bufferedOutput.flush();
bufferedOutput.close();
connection.close();
//file1.txt has been downloaded
return;
} catch (IOException ex) {
// exception stuff
}
}
}
}
}
You suggest HTTP as a protocol for your clients and servers -- HTTP is a fine protocol but may be a large implementation hurdle if you want to do the whole thing yourself. The HTTP PUT verb can be used to upload a file, and the benefit of using HTTP in this fashion is that your client and server could communicate with other tools designed to use PUT requests. (PUT is less-used than other HTTP verbs, so not all HTTP tools will work; the curl(1) program does support PUT via the -T command line option. This will be a great implementation aid, should you chose HTTP.)
There are a variety of REST Frameworks that can assist you in writing HTTP software; I have heard good things about Restlet, it would be my recommended starting point.
But you don't have to pick HTTP as your protocol. I think you can learn a lot about networking programming if you implement your own protocol -- it will teach you a lot about API design and sockets programming in a way that would be difficult to learn by using pre-written HTTP protocol tools (and frustrating if you tried to implement HTTP in its entirety yourself).
Consider this conversation:
client -> server: upload file named "flubber" sized 200000 bytes
server -> client: ok
client -> server: flubber contents
server -> client: ok
client -> server: upload file named "blort" sized 10 bytes
server -> client: error, file exists
...
You might want to add new commands to provide for hashing the file on both ends to ensure the file transfer succeeded, commands for sending specific byte ranges (either to append to an existing file or re-start a failed transfer), commands to list existing file names on the server, commands to overwrite existing files, commands to delete files from the server, and so forth. The best part of writing your own protocol is you get to decide what your programs will support. The downside is that you get to test the features you write, and testing some cases may be difficult. (Say, consider that a client may send each character of a command in a different TCP packet. Implementing the buffering to store up an entire command isn't trivial, but it is already done for you with a tool such as Restlet.)
Juan's advice to use multiple TCP sessions isn't strictly necessary -- though it may be the easiest path forward for you. You'll need to add some mechanism to provide the filename to the remote peer, and that might be best done through the "control" channel (the first session running -- similar to FTP) or it might be something you send immediately before the file's contents (similar to HTTP).
I'd like to suggest avoiding multiple connections, though -- each connection requires three times the round-trip time between systems to set up and start transferring bytes. This delay can be extremely annoying, especially when you're trying to transfer hundreds of small files. You can even spend more time setting up connections than you do actually sending data.
But it's your tool -- you get to design it as you wish. Have fun.
I think you need to create a new connection for each file so in that situation you'll be able to transfer files simultaneously.
You may have to modify your server to create a new thread (or get one from a thread pool) for each client connection so it can work with many at the same time.
Then you can run the client once per file.
Cheers
Ok, can you transfer multi files making a ArrayList or List files. Getting into in the array after get out in a filesystem path. I hope help you.