I'm trying to allow multiple connections to a little Java server type app. It works fine as is, but if one connection opens and then hangs, all subsequent connections will hang. I'm not sure how to go about handling each connection, up to about 20 concurrent ones in their own thread while keeping track of which thread belongs to which client etc. The code I have so far is:
private void init() {
try {
// Create the server socket channel
ServerSocketChannel server = ServerSocketChannel.open();
// nonblocking I/O
server.configureBlocking(false);
// host-port
server.socket().bind(new InetSocketAddress(host, port));
System.out.println("Server connected on " + host + ":" + port);
// Create the selector
Selector selector = Selector.open();
// Recording server to selector (type OP_ACCEPT)
server.register(selector, SelectionKey.OP_ACCEPT);
// Infinite server loop
for (;;) {
// Waiting for events
selector.select();
// Get keys
Set keys = selector.selectedKeys();
Iterator i = keys.iterator();
// For each keys...
while (i.hasNext()) {
SelectionKey key = (SelectionKey) i.next();
// Remove the current key
i.remove();
// if isAccetable = true
// then a client required a connection
if (key.isAcceptable()) {
// get client socket channel
SocketChannel client = server.accept();
// Non Blocking I/O
client.configureBlocking(false);
// recording to the selector (reading)
client.register(selector, SelectionKey.OP_READ);
continue;
}
// then the server is ready to read
if (key.isReadable()) {
SocketChannel client = (SocketChannel) key.channel();
// Read byte coming from the client
int BUFFER_SIZE = 32;
ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE);
try {
client.read(buffer);
} catch (Exception e) {
// client is no longer active
e.printStackTrace();
continue;
}
buffer.flip();
Charset charset = Charset.forName("ISO-8859-1");
CharsetDecoder decoder = charset.newDecoder();
CharBuffer charBuffer = decoder.decode(buffer);
Handler dataHandler = new Handler();
client.write(ByteBuffer.wrap(dataHandler.processInput(charBuffer.toString()).getBytes()));
client.socket().close();
continue;
}
}
}
} catch (IOException ex) {
Logger.getLogger(Server.class.getName()).log(Level.SEVERE, null, ex);
}
}
Whenever I need to write a socket server I avoid using low-level JVM classes because of the need to handle all the nitty-gritty details.
Instead I use Apache Mina. This ia a Java library for writing high-performance non-blocking multi-threaded socket servers.
An added benefit of using Mina is that it enforces clean architecture (IoFilters, protocol decoders) which makes your code more modular and makes it more maintainable.
Unless you really want to write an NIO server as a learning exercise I would recommend using Netty. Like Mina that Peter mentioned it is also a library for writing high performance servers.
I recently moved from using my own NIO code to this library and it has made my code so much cleaner.
My solution i Netty and Executor which creates ThreadPool.
You simply add handler do Netty's pipeline, which call executor witch ChannelBuffer as a parameter.
Than every client request will be processed by separate thread.
Look at the examples
Related
I am being asked to design the following programming problem.
Previously when a client sent a request to my application I was required to route it towards a specific port on another server. The server would response in a timely manner and a response would be sent from their server to my server and I would send the information to my client. This worked fine until the load increased and we realized that the other server did not handle multi threaded calls.
I have been given a limited set of ports by the other app so that whenever a load of client requests enters my application I must balance the load among those ports in such a manner that if port A is not free I send the client request to port B and if B is not free it goes to port C. [I know that this is not a correct solution, management wants it to work that way]
All calls handled are synchronous and all ports (requirement from application X) must always be kept open.
My biggest problems at the moment are:
Knowing when a port is free. (not waiting for a response)
How to push the load to the other ports.
I need some pointers to where to head to.
Could you please help?
What I have done so far is load multiple sockets into an array of SocketAddress.
Selector socketSelector = SelectorProvider.provider().openSelector();
int interestSet = SelectionKey.OP_CONNECT | SelectionKey.OP_WRITE | SelectionKey.OP_READ;
List<SocketAddress> socketAddresses = FactorySockectConnection.getSocketInstances();
for (SocketAddress socketAddress : socketAddresses) {
SocketChannel socket = SocketChannel.open(socketAddress);
socket.configureBlocking(false);
socket.register(socketSelector, interestSet);
}
System.out.println("Start");
while (true) {
try {
socketSelector.select();
Set<SelectionKey> keys = socketSelector.selectedKeys();
Iterator<SelectionKey> keyIterator = keys.iterator();
while (keyIterator.hasNext()) {
SelectionKey selectedKey = keyIterator.next();
if (selectedKey.isConnectable()) {
SocketChannel connectChannel = (SocketChannel) selectedKey.channel();
connectChannel.finishConnect();
}
if (selectedKey.isWritable() && requestMessageByte != null) {
SocketChannel writeChannel = (SocketChannel) selectedKey.channel();
ByteBuffer buf = ByteBuffer.wrap(requestMessageByte.getBytes());
while (buf.hasRemaining()) {
writeChannel.write(buf);
}
requestMessageByte = null;
}
if (selectedKey.isReadable() && responseMessage == null) {
SocketChannel readChannel = (SocketChannel) selectedKey.channel();
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
readChannel.read(readBuffer);
responseMessage = new String(readBuffer.array());
}
keyIterator.remove();
}
I am planning on using a load balancer. I will keep you posted.
I want to write a HTTP server with a long time response.
I prepared a loop:
serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.socket().bind(new InetSocketAddress(InetAddress.getByName("127.0.0.1"), 8000));
while (serverSocketChannel.isOpen()) {
try {
System.out.println("Waiting...");
SocketChannel socketChannel = serverSocketChannel.accept();
System.out.println("The new connection is open ;)");
} catch (IOException ex) {
System.out.println(ex.getMessage());
Logger.getLogger(ServerHttp.class.getName()).log(Level.SEVERE, null, ex);
}
}
I want to catch new connection from a user's browser, put it into a list, and then another thread will take it and will parse it.
But I have problem on the beginning: the code above is freezing in the method serverSocketChannel.accept().
When I add socketChannel.close() everything is working correctly, but I don't want to close the connection in this place, because I need a long time response.
Look at the javadoc for ServerSocketChannel:
"If this channel is in non-blocking mode then this method will immediately return null if there are no pending connections. Otherwise it will block indefinitely until a new connection is available or an I/O error occurs."
This seems to describe the "freezing" you are experiencing.
You can open your ServerSocketChannel in non-blocking mode:
serverSocketChannel.configureBlocking(false);
I have a client socket connected to the server socket, the server will send data to the client from time to time while its connected. currently my client uses a while loop to keep receiving data from the server even the server is not sending anything.
my question is, is there any more efficient way to listen for input?
i am thinking maybe create a thread for the socket connection, and put it to sleep when there is no incoming data, and sends an interrupt when there is data coming in. would that work? if putting the thread to sleep, would it break the socket connection?
i cannot modify the server socket and it doesnt initiate a connection.
import java.io.*;
import java.net.Socket;
public class core_socket {
public static void main(String[] args) {
String host = ("192.168.100.206");
int port = 4025;
try {
Socket socket = new Socket(host, port);
System.out.println("created socket\n");
OutputStream os = socket.getOutputStream();
boolean autoflush = true;
PrintWriter out = new PrintWriter(socket.getOutputStream(), autoflush);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// read the response
boolean loop = true;
StringBuilder sb = new StringBuilder(8096);
while (loop) {
if (in.ready()) {
int i = 0;
while (i != -1) {
i = in.read();
sb.append((char) i);
}
loop = false;
}
}
// display the response to the out console
System.out.println(sb.toString());
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
See multi-user chat application example at http://cs.lmu.edu/~ray/notes/javanetexamples/ - basically, you should consider spawning off a new worker thread for each incoming connection and then go back to listen for any new incoming requests.
A long time ago I wrote one of the first application servers (say in 1997 when most people didn't know what an app server is) - it was deployed at one of the largest higher-ed institutions and processed couple million requests per day during peak times - that's not the institution in the link by the way. The reason I mention this is... multi-threading gives you a tremendous scalability with very little effort - even if scalability is not what you are looking for, worker thread model is still a good practice.
Maybe what you want is to use asynchronous sockets. Basically, it spins off another thread that's job is to listen for any data on the socket. Once data does come in, a "callback" method is called, which can then begin to process your data.
I've never done sockets in Java before though, just C#, so I'm not sure how it compares, but the concept should remain the same.
I'm trying to write a Java HTTP Proxy Tunnelling program, and I need an experts advice about the best and fastest stream to use for the communication.
I've implemented the basic functionality and everything works fine. The only matter is communication speed or performance. My HTTP proxy system consists of a server program, running on a remote server and a client program running on the local machine. So far, the program looks like this:
Listener.java :
/**
* Listens and accepts connection requests from the browser
*/
ServerSocket listener = null;
try {
listener = new ServerSocket(port, 128);
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
ExecutorService executor = Executors.newCachedThreadPool();
Socket connection;
while (!shutdown) {
try {
connection = listener.accept();
executor.execute(new ProxyTunnel(connection));
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
}
ProxyTunnel.java :
try {
byte[] buffer = new byte[8192]; // 8-KB buffer
InputStream browserInput = browser.getInputStream();
OutputStream browserOutput = browser.getOutputStream();
// Reading browser request ...
StringBuilder request = new StringBuilder(2048);
do {
int read = browserInput.read(buffer);
logger.log(read + " bytes read from browser.");
if (read > 0) {
request.append(new String(buffer, 0, read));
}
} while (browserInput.available() > 0 && read > 0);
// Connecting to proxy server ...
Socket server = new Socket(SERVER_IP, SERVER_PORT);
server.setSoTimeout(5000); // Setting 5 sec read timeout
OutputStream serverOutput = server.getOutputStream();
InputStream serverInput = server.getInputStream();
// Sending request to server ...
serverOutput.write(request.toString().getBytes());
serverOutput.flush();
// Waiting for server response ...
StringBuilder response = new StringBuilder(16384);
do {
try {
read = serverInput.read(buffer);
} catch (SocketTimeoutException ex) {
break; // Timeout!
}
if (read > 0) {
// Send response to browser.");
response.append(new String(buffer, 0, read));
browserOutput.write(buffer, 0, read);
browserOutput.flush();
}
} while (read > 0);
// Closing connections ...
server.close();
} catch (IOException ex) {
ex.printStackTrace(System.err);
} finally {
try {
browser.close();
} catch (IOException ex) {
ex.printStackTrace(System.err);
}
}
The server program uses a similar fashion and sends the HTTP request to the destination server (e.g. www.stackoverflow.com) and forwards the response to the client program, where the client program forwards the response to the local browser.
How can I improve the performance of these TCP/HTTP communications?
Does using buffered streams such as BufferedInputSream and BufferedOutputStream improve communication performance?
Will I gain any performance improvements if I use java.nio Channels and Buffers, instead of java.net Sockets and java.io Stream?
Don't do it yourself
Advice 0: there are plenty of proxy servers out there, much more scalable, stable and mature. Do you really need to write your own?
Don't use StringBuilder/String to buffer request
byte[] buffer = new byte[8192]; // 8-KB buffer
//...
browserInput.read(buffer);
//...
request.append(new String(buffer, 0, read));
//...
serverOutput.write(request.toString().getBytes());
This is flawed for several reasons:
you are assuming your HTTP calls are text (ASCII) only, binary data will be malformed after transforming to String and back to byte[], see: String, byte[] and compression
even if the protocol is text-based, you are using system's default encoding. I bet this is not what you want
finally, the most important part: do not buffer the whole request. Read chunk of data from incoming request and forward it immediately to target server in one iteration. There is absolutely no need for the extra memory overhead and latency. Immediately after receiving few bytes dispatch them and forget about them.
Don't use Executors.newCachedThreadPool()
This pool can grow inifinitely, creating thousands of threads during peak. Essentially you create one thread per connection (except that the pool reuses free threads, but creates new if none available). Consider Executors.newFixedThreadPool(100) - 100-200 threads should be enough in most cases. Above that you'll most likely burn your CPU barely in context switching, without doing much work. Don't be afraid of latency, scale out.
Use non-blocking netty stack
Which brings us to the final advice. Drop blocking sockets altogether. They are handy, but don't scale well due to thread-per-connection requirement. Too much memory is spent to hold stack, too much CPU is wasted for context switching. netty is great and it builds powerful abstraction over NIO.
Check out the examples, they include HTTP client/server code. There is a bit of a learning curve, but you can expect performance growth by several order of magnitued.
I am writing a Client/Server set of programs
Depending on the operation requested by the client, I use make TCP or UDP request.
Implementing the client side is straight-forward, since I can easily open connection with any protocol and send the request to the server-side.
On the servers-side, on the other hand, I would like to listen both for UDP and TCP connections on the same port. Moreover, I like the server to open new thread for each connection request.
I have adopted the approach explained in: link text
I have extended this code sample by creating new threads for each TCP/UDP request.
This works correctly if I use TCP only, but it fails when I attempt to make UDP bindings.
Please give me any suggestion how can I correct this.
tnx
Here is the Server Code:
public class Server {
public static void main(String args[]) {
try {
int port = 4444;
if (args.length > 0)
port = Integer.parseInt(args[0]);
SocketAddress localport = new InetSocketAddress(port);
// Create and bind a tcp channel to listen for connections on.
ServerSocketChannel tcpserver = ServerSocketChannel.open();
tcpserver.socket().bind(localport);
// Also create and bind a DatagramChannel to listen on.
DatagramChannel udpserver = DatagramChannel.open();
udpserver.socket().bind(localport);
// Specify non-blocking mode for both channels, since our
// Selector object will be doing the blocking for us.
tcpserver.configureBlocking(false);
udpserver.configureBlocking(false);
// The Selector object is what allows us to block while waiting
// for activity on either of the two channels.
Selector selector = Selector.open();
tcpserver.register(selector, SelectionKey.OP_ACCEPT);
udpserver.register(selector, SelectionKey.OP_READ);
System.out.println("Server Sterted on port: " + port + "!");
//Load Map
Utils.LoadMap("mapa");
System.out.println("Server map ... LOADED!");
// Now loop forever, processing client connections
while(true) {
try {
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
// Iterate through the Set of keys.
for (Iterator<SelectionKey> i = keys.iterator(); i.hasNext();) {
SelectionKey key = i.next();
i.remove();
Channel c = key.channel();
if (key.isAcceptable() && c == tcpserver) {
new TCPThread(tcpserver.accept().socket()).start();
} else if (key.isReadable() && c == udpserver) {
new UDPThread(udpserver.socket()).start();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
} catch (Exception e) {
e.printStackTrace();
System.err.println(e);
System.exit(1);
}
}
}
The UDPThread code:
public class UDPThread extends Thread {
private DatagramSocket socket = null;
public UDPThread(DatagramSocket socket) {
super("UDPThread");
this.socket = socket;
}
#Override
public void run() {
byte[] buffer = new byte[2048];
try {
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
socket.receive(packet);
String inputLine = new String(buffer);
String outputLine = Utils.processCommand(inputLine.trim());
DatagramPacket reply = new DatagramPacket(outputLine.getBytes(), outputLine.getBytes().length,
packet.getAddress(), packet.getPort());
socket.send(reply);
} catch (IOException e) {
e.printStackTrace();
}
socket.close();
}
}
I receive:
Exception in thread "UDPThread" java.nio.channels.IllegalBlockingModeException
at sun.nio.ch.DatagramSocketAdaptor.receive(Unknown Source)
at server.UDPThread.run(UDPThread.java:25)
10x
It should work. One of the problems with this code, it seems, is that the ByteBuffer size is set to 0, meaning that the datagram is discarded (as it mentions in the comments). If you need to receive any information over UDP and you are on a reliable network, you can set the size quite big and receive big datagrams made up of multiple packets. Otherwise, on an unreliable network, set this to the MTU size. Make sure you flip() the ByteBuffer after receiving anything in it.
Also, creating new threads for each request is a bad idea, create a 'session' thread for each different IP you receive in a HashMap or something, and then do a guarded block on the session object. Wake up the thread sleeping on that object when you receive a message after passing in new information. The selector code you have is designed to avoid the creation of threads in this way.
Edit: based on the above code, you're create a datagram channel and then using the socket to receive datagrams directly? That's doesn't make sense. Use the channel methods only after binding the channel. Also, don't do this in a separate thread. Your code isn't thread-safe and will bust itself up. Hand the received information off to the separate 'session' thread as mentioned earlier. The selector is designed to tell you what channels can be read from without blocking (although blocking is disabled anyway, so it will tell you what channels have data to be read from).
AFAIK, you should be able to listen for both TCP connections and UDP messages on the same port. It would help if you posted your UDP code, and the exception + stacktrace that you are seeing.
You can't use DatagramSocket.receive() in non-blocking mode. You have to use the read() or receive() methods of your DatagramChannel directly.
In fact as you are using non-blocking mode and a Selector, it is quite impossible to see why you're also using a UDPThread at all. Just call udpserver.receive() instead of starting the thread.