I'm attempting to move my networking over from standard IO to NIO and following the few tutorials that there are to attempt to figure it out, while me, myself, and I decided it'd be a great idea to spend my first week rewriting the core for all of the applications logic handling, I never would have imagined that I wouldn't be able to set up the basic networking.
Currently the networking is at a very basic stage, everything is tossed inside of a while-loop and I can't honestly say I've put any attempt into it to make it look nice, considering I haven't a clue what I'm doing my goal was to figure out how to do it first, then go back and give it a makeover.
Here's the code I use to initialize my server:
// Initializes the TCP Server and all of its components.
private void initTcpServer(int port) {
try {
// Create a new selector
Selector socketSelector = SelectorProvider.provider()
.openSelector();
// Create a new non-blocking server socket channel;
this.serverSocketChannel = ServerSocketChannel.open();
this.serverSocketChannel.configureBlocking(false);
// Bind the server socket to the specified address and port
this.serverSocketChannel.socket().bind(
new InetSocketAddress("127.0.0.1", port));
// Register the server socket channel, indicating an interest in
// accepting new connections
this.serverSocketChannel.register(socketSelector,
SelectionKey.OP_ACCEPT);
// Set the selector for the server instance.
this.selector = socketSelector;
} catch (IOException e) {
e.printStackTrace();
}
}
Then this class implements the Runnable interface, and a new thread is started directly after this method completes, in this thread we contain the following code:
public void run() {
while (isRunning) {
try {
selector.selectNow();
} catch (IOException io) {
return;
}
Iterator<SelectionKey> it = selector.selectedKeys().iterator();
while (it.hasNext()) {
SelectionKey key = it.next();
if (!key.isValid()) {
it.remove();
continue;
}
try {
if (key.isAcceptable()) {
this.handleConnection(key);
} else if (key.isReadable()) {
Connection connection = (Connection) key.attachment();
if (connection != null) {
try {
connection.getMasterProtocol()
.decode(connection,
connection.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
}
}
} finally {
it.remove();
}
}
}
}
This, from my understanding is what allows us to handle our connections and data based off of a SelectionKey.. and is what all of the NIO based networking runs from, you'll see that I'm calling two different methods to make this not so much of a mess, the first one is #handleConnection and the other one is a decode function.
The handle connection method creates a new instance of my Connection class and attatches it to the SelectionKey, like so:
public Connection(SelectionKey key) {
try {
// For an accept to be pending the channel must be a server socket channel.
ServerSocketChannel serverSocketChannel = (ServerSocketChannel)key.channel();
// Accept the connection and make it non-blocking.
this.socketChannel = serverSocketChannel.accept();
this.socketChannel.configureBlocking(false);
// Set up other user data.
this.inputStream = new DataInputStream(socketChannel.socket().getInputStream());
this.masterProtocol = new MasterProtocol();
// Register the new SocketChannel with our Selector, indicating
// we'd like to be notified when there's data waiting to be read.
key = this.socketChannel.register(OGServer.getInstance().getSelector(), SelectionKey.OP_READ);
key.attach(this);
// Add the current <SelectorKey, Connection> to the current connections collection.
connections.put(key, this);
Log.debug(getClass(), "Connection constructed successfully.");
} catch(IOException e) {
e.printStackTrace();
}
}
The error is called when I attempt to call the MasterProtocol#decode method, which looks like this:
public Object decode(Connection connection, DataInputStream dataInputStream) throws IOException {
if(connection.getState() == ConnectionState.CONNECTED) {
byte[] bytes = ByteStreams.toByteArray(dataInputStream);
if(bytes.length < 4) {
System.out.println("Not enough bytes read.");
return null;
}
int bufferSize = dataInputStream.readInt();
System.out.println("Buffer Size: " + bufferSize);
while(bytes.length < bufferSize) {
return null;
}
int test = dataInputStream.readInt();
System.out.println("Test: " + test);
return null;
}
return null;
}
The error seems to be called when the DataInputStream tries to read from the network, more specifically on this line of code:
byte[] bytes = ByteStreams.toByteArray(dataInputStream);
The error:
Exception in thread "Thread-0" java.nio.channels.IllegalBlockingModeException
at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:190)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.io.DataInputStream.read(DataInputStream.java:100)
at com.google.common.io.ByteStreams.copy(ByteStreams.java:70)
at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:115)
at net.ogserver.framework.net.protocol.MasterProtocol.decode(MasterProtocol.java:29)
at net.ogserver.framework.net.OGServer.run(OGServer.java:146)
at java.lang.Thread.run(Thread.java:745)
The 'IllegalBlockingModeException' exception is what's throwing me off, as all of the information I've found was for setting up a Non-blocking server, but the DataInputStream implementation was my own, so I must have done something wrong somewhere. NIO is a completely different world from IO, but learning is learning, eh?
EDIT: I guess it'd help to know how I'm sending the data from the client, it's just a very basic test application that does this:
socket = new Socket("127.0.0.1", 5055);
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
dos.writeBoolean(false);
If you're moving to NIO in non-blocking you can't keep using streams. If you want to use streams there is no advantage to using NIO at all. I would just stop the migration project now.
Related
I am trying to improve the speed at which the sockets transfer information but i am unsure how to do so. the pourpose of the code is to transfer a number, the date, and a short xml which is being sent in the form of a string.
this is the server code
import java.net.*;
import java.io.*;
public class SSocket extends Thread
{
private ServerSocket serverSocket;
public SSocket(int port) throws IOException
{
serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(100000);
}
public void run()
{
System.out.println("Waiting for client on port " + serverSocket.getLocalPort() + "...");
while(true)
{
try
{
Socket server = serverSocket.accept();
DataInputStream in = new DataInputStream(server.getInputStream());
int cor=in.readInt();
int i=0;
String transaccion = in.readUTF();
String fecha = in.readUTF();
System.out.println(cor);
System.out.println(transaccion);
System.out.println(fecha);
DataOutputStream out =
new DataOutputStream(server.getOutputStream());
if(transaccion!=null && fecha != null && cor>0){
out.writeInt(cor);
}
else {
out.writeInt(-1);
}
if (i==100){
out.flush();
i=0;
}
i++;
server.close();
}catch(SocketTimeoutException s)
{
System.out.println("Socket timed out!");
break;
}catch(IOException e)
{
e.printStackTrace();
break;
}
}
}
public static void main(String [] args)
{
int port = 1337;
try
{
Thread t = new SSocket(port);
t.start();
}catch(IOException e)
{
e.printStackTrace();
}
}
}
the code for the client is
import java.net.*;
import java.io.*;
public class ClientSocket
{
public static void send(int correl, String transaccion, String fecha)
{
String serverName = "localhost";
int port = 1337;
try
{
Socket client = new Socket(serverName, port);
int i=0;
OutputStream outToServer = client.getOutputStream();
DataOutputStream out =
new DataOutputStream(outToServer);
out.writeInt(correl);
out.writeUTF(transaccion);
out.writeUTF(fecha);
InputStream inFromServer = client.getInputStream();
DataInputStream in =
new DataInputStream(inFromServer);
int corin=in.readInt();
if(corin>0){
Envio.updater(corin);
}
else {
}
if (i==100){
out.flush();
i=0;
}
i++;
client.close();
}catch(IOException e)
{
e.printStackTrace();
}
}
}
i have done some reading on the mater and it seems that posible solutions are to use either a buffer or swich to a datagram. however my experience on working with sockets is rather limited and i am unsure which would be best to use for this situation or if there is another option i havent yet considered. this code will be moving many transactions and i wish to do it in as short time as posible.
thanks in advance
ps. sorry for my bad english it is not my first language
Datagrams imply UDP, which is an unreliable delivery protocol so you're not guaranteed to get all content. That's probably not what you want; I'd stay with plain Sockets (which use TCP, which has reliable delivery).
Will the same client be calling send() repeatedly and connecting to the same server each time? That is, will there be many messages going across a single connection, or will each message be to a different server, with only a single message (or only a few) going to each of the many servers? If there's just one server that a client is going to connect to and if a given client is going to send lots of messages, you should keep the Socket open between send() calls; setting up and tearing down Sockets is expensive, so you're paying a high price for making a new connection each time.
Also, your server appears to only be able to handle a single connection at a time: you accept a connection, read from it, and then close it and accept a new one. So to make this work for more than one client, you'll need to separate the logic for accepting connections onto a different thread from the logic that reads data. If you'll only have a few clients at a time, you can just start a new thread to read from each socket as you create it for a new client; if you'll have lots of clients (thousands), you'll probably need to look at NIO for its ability to service multiple sockets from a single thread. But I suspect you're a long way from having that problem, if you ever do, so I'd just spawn a new thread for each socket.
EDIT: I have corrected the mistake below in the code, by adding a line into the server code
I'm trying to write some socket code that will allow me to send data from one computer to another for a game (which for simplicity's sake, we can think of as tic-tac-toe, not much data needs to be sent, just a couple of numbers). In order to achieve this I have written two classes, Server and Client. At the moment I am testing through the localhost using port 1234, and I am only using one single instance of the program (though the same problem occurs when trying to use two instances).
Firstly here's the code, and then I can go into more depth about the problem, and what testing I've done to attempt to work out what is going wrong:
public class Server
{
private ServerSocket server;
private Socket socket;
private Client socketHandler;
private static final int DEFAULT_PORT = 1234;
public Server() { this(DEFAULT_PORT); }
public Server(int port)
{
Thread thread = new Thread()
{
public void run()
{
try
{
System.out.println("Attempting to Establish Connection");
server = new ServerSocket(port);
socket = server.accept();
socketHandler = new Client(port, socket); //THIS LINE ADDED
System.out.println("Server Online!");
}
catch (Exception e)
{
e.printStackTrace();
}
}
};
thread.setDaemon(true);
thread.start();
}
//ADJUSTED
Client getSocketHandler()
{
return socketHandler;
}
public void kill()
{
try
{
if (socket != null) socket.close();
if (server != null) server.close();
}
catch(IOException e)
{
e.printStackTrace();
}
finally
{
socket = null;
server = null;
}
}
}
public class Client
{
public static final int DEFAULT_PORT = 1234;
public static final String DEFAULT_HOST = "localhost";
private static final String THUMP_THUMP = "thump thump";
private static final int PULSE = 1000;
private int port;
private String ip;
private Socket socket;
private BufferedReader input = null;
private PrintWriter output = null;
boolean closed = true;
String data = "";
public Client() { this(DEFAULT_PORT, DEFAULT_HOST, null); }
public Client(int port) { this(port, DEFAULT_HOST, null); }
public Client(int port, String ip) { this(port, ip, null); }
public Client(int port, Socket server) { this(port, DEFAULT_HOST, server); }
public Client(String ip) { this(DEFAULT_PORT, ip, null); }
public Client(String ip, Socket server) { this(DEFAULT_PORT, ip, server); }
public Client(Socket server) { this(DEFAULT_PORT, DEFAULT_HOST, server); }
public Client(int port, String ip, Socket server)
{
socket = server;
this.ip = ip;
this.port = port;
Thread thread = new Thread()
{
public void run()
{
try
{
initialise(server);
String line;
startHeartbeat();
while (isClosed()) {} //first it is closed, lets wait for it to open before we start waiting for it to close!
System.out.println("We are about to listen!");
while (!isClosed())
{
System.out.println("pre-read"); //this line was used to determine that the code was hanging on the next line
line = input.readLine(); //offending line
System.out.println("post-read"); //this line was used to determine when the block was lifted
if (line != null)// || line != THUMP_THUMP)
{
System.out.println(line);
data += line + "\n";
}
}
System.out.println(data);
kill();
System.out.println("Connection Closed!");
}
catch (SocketException e)
{
e.printStackTrace();
System.out.println("Server closed!");
}
catch (Exception e)
{
e.printStackTrace();
}
}
};
thread.setDaemon(true);
thread.start();
}
private void initialise(Socket server)
{
try
{
if (server == null) socket = new Socket(ip, port);
input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
output = new PrintWriter(new OutputStreamWriter(socket.getOutputStream()));
}
catch (IOException e) { e.printStackTrace(); }
}
public boolean post(String text)
{
synchronized(this)
{
output.println(text);
output.flush();
return !output.checkError();
}
}
public void kill()
{
try
{
if (input != null) input.close();
if (socket != null) socket.close();
}
catch(IOException e) { e.printStackTrace(); }
finally
{
input = null;
socket = null;
}
}
public void killOutputStream()
{
try
{
if (output != null) output.close();
}
catch (Exception e) { e.printStackTrace(); }
finally
{
output = null;
}
}
//////////////////////////////////
///////// Socket Control /////////
//////////////////////////////////
synchronized boolean isClosed()
{
return closed;
}
synchronized void setClosed(boolean b)
{
closed = b;
}
//We need to make sure that the socket is still online, to ensure the reading stops when the connection closes.
void startHeartbeat()
{
Thread heartbeat = new Thread()
{
public void run()
{
while (output != null)
{
setClosed(post(THUMP_THUMP) ? false : true); //post returns true on success
synchronized(this)
{
try
{
this.wait(PULSE);
}
catch (InterruptedException e) {}
}
}
setClosed(true);
}
};
heartbeat.setDaemon(true);
heartbeat.start();
}
}
The Problem
When the client is started (after having created the server) it fails to read any data sent through (or even the heartbeat), in fact the code does not go past line = input.readLine() in the reading thread (which is from now on called the offending line), except it seems, until the server is disconnected (see below).
Here is the order of regular testing:
Server() is called and the resulting Server is stored in the serverConnection variable then
Client(serverConnection != null ? serverConnection.getSocket() : null) is called and the new Client is stored in clientConnection.
Because we can test whether it is working using the heartbeat no other data needs to be sent, and the server is terminated by calling serverConnection.kill() and then clientConnection.killOutputStream() after letting some time elapse.
and this is the result:
Attempting to Establish Connection Server Online!
We are about to listen!
Connection Closed!
where the empty line represents the non null data received over the course of the connection, ie that there is none.
I expect this:
Attempting to Establish Connection
Server Online!
We are about to listen!
thump thump
thump thump
thump thump (and so on, every second)
Connection closed!
I spent time performing different tests by commenting out or changing the code slightly with the same testing format (except for the special case, which is number 6) and made these observations:
Observations
Only when the socket is closed and the output stream is closed, does the program move past the offending line.
When the readline() method starts to process (shortly before the heartbeat cuts it off) it detects nothing in the stream, not even THUMP_THUMP.
When the socket is closed, but the output stream is not, the readline() method starts to process, only to detect nothing, heartbeat cuts it off. No SocketException even though it would be expected.
If the socket is NOT closed, and only the output stream is closed, a SocketException is triggered, suggesting the socket is closed.
I used netstat -an in command prompt, and when the server is started the port 1234 is LISTENING. When the client connects, it is still LISTENING, implying that there is no connection.
I set up some python code to connect to itself over port 1234,
however I made a mistake in the python code, and as such the server
didn't close, and was still open. So I decided to connect the java
client to the server and see what happens. I did this by running
Client(null) which is the client code for the non-host. It
resulted in the port reading ESTABLISHED, and the python server was
echoing back the "thump thump", and the java code was successfully
reading it. No hanging, it worked perfectly.
This leads me to believe that the problem lies in the server code, as the python server was able to communicate sucessfully with the Java client, but the Java client is unable to communicate with the Java server.
Before performing this testing I had been concentrating on the Client code, believing that it was at fault. All the questions I have found here with similar symptoms (see here, here and here, among others) have turned up blank for me, having written in their solutions (most were due to the output stream not flushing, or the \n ommitted, which I have not failed to do, or the solution not fixing my problem, and so having been removed in favor of the heartbeat in this case). I originally based my code off of this article.
After 4 days of trying to figure out this problem I am at a loss for what to do... What am I missing here? Why is the Server code not working as I expect it to? If anybody needs any more clarification on my code then please ask!
As an after-note, the testing code is run through a simple minimalistic GUI written in javafx (not fxml though), whether that would be a problem or not I'm sure, I would think not, due to it working with the Python server. This code is compiled in Java 8
I'm a little confused about why you think it would go any furthur than input.readLine() considering there is no handling of inputs/outputs on the server side....
Client/Server connections are like a game of tennis, as one side serves the other must receive the ball and then serve it back(maybe with different information). Your server side must handle the input it recieves from the start heartbeat method, and then send you back a response. the input.readLine() function blocks the thread until it receives data from the other end, so yes the code stops there and waits for your server to send the "tennis ball" back. In the server class you should add an input and output stream that handle the heart beat inputs and send back a string of data to the client.
Server:
OutputStream os = socket.getOutputStream();
InputStream is = socket.getInputStream();
String response = "thump thump";
while(true){
is.read();
os.write(response.getBytes());
os.flush();
}
with this example, the client should remain unchanged and just add the above code to your server.
There is a component in my application that listens to a server via TCP (so it only receives data, the output-stream is never used). The only reason for a potential disconnect are technical issues. From a logical point of view, the connection should stay open forever.
I know that I have to implement some kind of ping/pong strategy if I want to detect a connection failure immediately. But in my case, it is not necessary to detect a dropped connection immediately as long as it gets detected at all (let's say some minutes or hours later).
My questions:
If I don't use some kind of pingpong/alive-check strategy and the connection drops, will I get an IOException in my application logic some time later (it would be okay if it took some hours) or is it possible that the dropped connection isn't detected at all?
Would the code below fit my requirements? It's a bit ugly (many try-catch/while(true) and even sleep, but I'm wondering if a timed out connection could be recognized after a certain amount of time (e.g. due to an IOException in the blocking BufferedReader.readLine method).
Apart from the questions above, what could I do better?
public class Receiver implements Runnable {
private Socket socket;
private final String host;
private final int port;
private final int connectionRetryAfter = 10* 1000;
public Receiver(String host, int port) { // assignments... }
#Override
public void run() {
tryCreateSocket();
listenToServer();
}
private void listenToServer() {
String receivedLine;
BufferedReader buf;
while(true) {
try {
buf = new BufferedReader(new InputStreamReader(socket.getInputStream()));
while ((receivedLine = buf.readLine()) != null) {
// do something with 'inputLine'
}
} catch (IOException e) {
// logging
} finally {
closeSocket();
}
// At this point, either an exception occured or the stream equals null (which means it's closed?)
tryCreateSocket();
}
}
private void tryCreateSocket() {
try {
socket = new Socket(host, port);
} catch (IOException e) {
// logging
try {
Thread.sleep(connectionRetryAfter);
} catch(InterruptedException ex) {
// logging
Thread.currentThread().interrupt();
}
// retry
tryCreateSocket();
}
}
private void closeSocket() {
if (socket != null) {
try {
socket.close();
} catch (IOException e) {
// logging
}
}
}
}
listenToServer() should certainly throw an IOException if the connection/reconnection attempt fails. Consider the case when the server is down. Do you really want to loop inside this method forever?
One problem you might need to avoid is that you tryCreateSocket() makes a recursive call. If your client is disconnected for a very long time you might run out of memory. Further more when you do reestablish connection the memory stack is not freed.
I would recommend an iterative while loop calling the tryCreateSocket() to avoid this problem.
Just to give the background of my issue; I'm writing a client server application. when a client initially connects, the server accesses the database and send all the items in a particular table to the client by writing the objects to an objectoutputstream in a for loop.
The issue is, the client reads only the first object that is being sent. Even if I create a new object and send it to the server for verification the client does not pick it up when the server sends it back after inserting to database... extract from code:
Server:
public void loadClients()
{
ArrayList <Client> list = dBCon.loadClients();
try
{
for (int i =0; i<list.size();i++)
{
sendtoClient(list.get(i));
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
public void sendtoClient(Object obj) throws IOException
{
_out.writeObject(obj);
_out.flush();
_out.reset();
}
Client:
socket = new Socket("localhost", 4447);
ois = new ObjectInputStream(socket.getInputStream());
oos = new ObjectOutputStream(socket.getOutputStream());
Object objIn = ois.readObject();
Client client = (Client)objIn;
switch(client.actionType)
{
case ("ADD"):
{
if(clientFrame.tbDeals!=null)
{
clientFrame.tbDeals.addClient(client);
break;
}
else
{
clientList.add(client);
System.out.println(client.clientName);
break;
}
}
}
Edit: I have tried with a while loop on the client side as well. then ois.available() returns 0....
You're reading exactly one object on the client side. That's all you're going to get.
available() is rarely if ever used when doing basic IO in Java, and generally isn't going to do what you think it does.
Reading from a ObjectInputStream is blocking, just like all other basic Java IO. Using readObject() you will either get an object, or an exception as noted in the javadoc
The most simplistic example of how you would handle this in a loop if you have no idea the number of objects you'll be receiving and expect the connection to stay open is:
while (true) {
try {
Object o = ois.readObject();
/// do something with the object you just read
} catch ( EOFException e) {
System.out.println("remote connection closed!");
e.printStackTrace();
break;
}
}
I'm working on a web server for Android and even though I've spent days trying to fix it, I'm at my wits' end with this particular bug. I'm trying to read the request from the browser, and the code works fine most of the time, but something like 5% of the requests fail and it throws random SocketTimeoutExceptions without even reading a single character from the Socket.
I have tested this with different browsers and it happens with all of them, so chances are the problem is on my end. Here's the relevant code, stripped down as far as possible:
public class ServerThread extends Thread {
private ServerSocket ss = null;
private boolean isRunning;
private ExecutorService threadPool = new ThreadPoolExecutor(2, 12,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(),
Executors.defaultThreadFactory(),
new ThreadPoolExecutor.CallerRunsPolicy());
public ServerThread() {
}
public synchronized void run() {
ss = new ServerSocket(8080, 1);
isRunning = true;
while (isRunning) {
Socket clientSocket = null;
try {
if (ss != null) {
clientSocket = ss.accept();
if (isRunning) {
this.threadPool.execute(new HTTPSession(clientSocket));
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
And:
public class HTTPSession implements Runnable {
private Socket mSocket = null;
public HTTPSession (Socket s) {
mSocket = s;
}
public void run() {
InputStream ips = null;
try {
mSocket.setSoTimeout(15000);
ips = mSocket.getInputStream();
ips.read();
}
catch (Exception e) {
e.printStackTrace();
Log.v("HTTPSession", "Socket connected: " + mSocket.isConnected() + ", Socket closed: " + mSocket.isClosed() + ", InputShutdown: " + mSocket.isInputShutdown());
}
finally {
try { ips.close(); } catch (IOException ioe) { }
try { mSocket.close(); } catch (IOException ioe) { }
}
}
}
So ServerThread accepts the connection, HTTPSession tries to read from the Socket and sometimes it throws the SocketTimeoutException after the 15 seconds are up.
The output from the Log statement in the catch in this case is:
Socket connected: true, Socket closed: false, InputShutDown: false
What gives? Surely 15 seconds is enough of a wait and it seems unlikely that mainstream web browsers just aren't sending any data, so why can't I read it?
I would appreciate any input on this problem.
SocketTimeoutException only means one thing: no data was available within the timeout period. So yes maybe your timeout is too short, and yes the browser didn't send it within the timeout period, or at least it didn't arrive at the server's socket receive buffer within the timeout period.
I would say 15 seconds is a bit aggressive for a server side timeout. 30s to a couple of minutes would be more like it.
I don't see any reason this code would fail in that way unless, like you said, a browser just wasn't sending anything. You could change the ips.read(); to System.out.println(ips.read()); to be sure of that. If you see a byte show up on stdout, then the browser did send something. My guess would be that in your full code, you're not properly recognizing the end of a request and continuing to wait for more data. After 15 seconds, you'll time out. But that's just a guess. If you post some code that demonstrates the problem, someone might be able to give you a definitive answer.