Java/Android: Socket closed when offloading work to a thread pool - java

I'm experiencing a very puzzling error writing a thread-pooled TCP server on Android. Basically, my code is structured as follows:
Standard server loop (blocking call to socket.accept() in a loop within its own thread), calling a handler upon an incoming connection:
socket = mServerSocket.accept();
myHandler.onIncomingConnection(socket);
The handler offloads all further processing of the incoming connection(s) to a thread pool:
public class X {
private final ExecutorService receiveThreadPool = Executors.newSingleThreadExecutor();
[...]
private final ConnectionHandler mHandler = new MyServer.ServerHandler() {
#Override
public void onIncomingConnection(final Socket socket) {
MLog.vv(TAG, "Socket status: " + (socket.isBound() ? "bound" : "unbound") + ", "
+ (socket.isConnected() ? "connected" : "unconnected") + ", "
+ (socket.isClosed() ? "closed" : "not closed") + ", "
+ (socket.isInputShutdown() ? "input shutdown" : "input not shut down") + ".");
// Process result
receiveThreadPool.execute(new Runnable() {
#Override
public void run() {
MLog.vv(TAG, "Socket status: " +
(socket.isBound() ? "bound" : "unbound") + ... ); // same code as above
BufferedOutputStream out = null;
try {
out = new BufferedOutputStream(socket.getOutputStream());
out.write(HELLO_MESSAGE);
out.flush();
[rest omitted...]
} catch (IOException e) {
[...]
} finally {
[close resources...]
}
}
Note, that the socket is defined as final in the handler method's signature, making it accessible from within the anonymous inner Runnable class. However, the first write out.write(HELLO_MESSAGE); to the output stream fails due to a closed socket exception. logcat output:
02-16 17:49:26.383 14000-14057/mypackage:remote V/ManagementServer﹕ Incoming connection from /192.168.8.33:47764
02-16 17:49:26.383 14000-14057/mypackage:remote V/ManagementServer﹕ Socket status: bound, connected, not closed, input not shut down.
02-16 17:49:26.393 14000-14077/mypackage:remote V/ManagementServer﹕ Socket status: bound, unconnected, closed, input not shut down.
02-16 17:49:26.398 14000-14077/mypackage:remote E/ManagementServer﹕ Error communicating with client:
java.net.SocketException: Socket is closed
at java.net.Socket.checkOpenAndCreate(Socket.java:665)
at java.net.Socket.getInputStream(Socket.java:359)
at net.semeion.tusynctest.network.ManagementServer$1$1.run(ManagementServer.java:79)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
As shown in the log output, somehow the socket changes its status from connected to unconnected/closed right after offloading the Runnable into the thread pool. If I just remove the ThreadPool.execute lines, everything works as expected. I have also tried to create my own static Runnable class within the outer class X, passing the socket as a parameter to the Runnable's constructor. However, this triggers the same problem.
Am I missing something here? In theory, this should work like a charm; but somehow it seems, that upon starting the thread and terminating the handler's incomingConnection method, something happens to the socket instance.
Further facts:
It's not the client's fault either - to the client, it looks like the server closed the socket. And as said, commenting out the thread pool on the server side fixes the problem.)
You may notice the ":remote" in the logcat output - the server thread is created from within a background service that itself runs in a separate process.
At the moment, I'm employing a SingleThreadExecutor for the pool, just for testing. The type of executor should make no difference IMHO.
I tried to use the socket's OutputStream directly (unbuffered), which has not made a difference. The log statements show that the socket itself changes its status.
If I initialise the BufferedOutputstream in the handler right before executing the thread, this yields a strange "bad file number" SocketException at the out.flush(); line (possibly this is an IPC problem?!):
java.net.SocketException: sendto failed: EBADF (Bad file number)
at libcore.io.IoBridge.maybeThrowAfterSendto(IoBridge.java:546)
at libcore.io.IoBridge.sendto(IoBridge.java:515)
at java.net.PlainSocketImpl.write(PlainSocketImpl.java:504)
at java.net.PlainSocketImpl.access$100(PlainSocketImpl.java:37)
at java.net.PlainSocketImpl$PlainSocketOutputStream.write(PlainSocketImpl.java:266)
at java.io.BufferedOutputStream.flushInternal(BufferedOutputStream.java:185)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:85)
at net.semeion.tusynctest.network.ManagementServer$1$1.run(ManagementServer.java:88)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
Caused by: android.system.ErrnoException: sendto failed: EBADF (Bad file number)
at libcore.io.Posix.sendtoBytes(Native Method)
at libcore.io.Posix.sendto(Posix.java:206)
at libcore.io.BlockGuardOs.sendto(BlockGuardOs.java:278)
at libcore.io.IoBridge.sendto(IoBridge.java:513)
Thanks for any hints :)

I've found the problem now. Embarrassingly, I've overlooked the most obvious source for trouble, the server loop in my Server class:
new Thread(new Runnable() {
#Override
public void run() {
while (mReceiving) {
Socket recSocket = null;
try {
recSocket = mServerSocket.accept();
// Process connection
mTcpServerHandler.onIncomingConnection(recSocket);
} catch (IOException e) {
// ...
} finally {
if (recSocket != null) {
try {
recSocket.close();
} catch (IOException e) {
// log, ignore...
}
}
}
}
}
}).start();
So, in case the handler offloads the processing to a new thread, the method returns immediately, and the finally block of the server loop closes the socket (which was intended originally, but doesn't fit the thread pool approach). Too obvious, sorry for bothering :)

Related

How to properly use setSoTimeout to block running accept() on a socket in Java?

I'm struggling with sockets in Java. I need to set a timeout so that my process will give up and stop running after 1000ms. I tried following the documentation, and some posts here on stackoverflow, but the process keeps waiting, blocked on the call of the accept() function. What am I doing wrong?
private static void statusRequest(String destAddr) throws ClassNotFoundException {
try {
ServerSocket serverSocket = new ServerSocket(PORT_NUMBER2);
serverSocket.setSoTimeout(1000);
Socket socket = serverSocket.accept(); // Blocking function
// [... Expected working flow ...]
// [... do some work with the received object ...]
} catch (SocketException s) {
System.out.println("No message received");
}
}

Java NIO client causes file descriptor leakage only when remote TCP server is down

The below program acts as TCP client and uses NIO to open socket to a remote server, as below
private Selector itsSelector;
private SocketChannel itsChannel;
public boolean getConnection(Selector selector, String host, int port)
{
try
{
itsSelector = selector;
itsChannel = SocketChannel.open();
itsChannel.configureBlocking(false);
itsChannel.register(itsSelector, SelectionKey.OP_CONNECT);
itsChannel.connect(new InetSocketAddress(host, port));
if (itsChannel.isConnectionPending())
{
while (!itsChannel.finishConnect())
{
// waiting until connection is finished
}
}
itsChannel.register(itsSelector, SelectionKey.OP_WRITE);
return (itsChannel != null);
}
catch (IOException ex)
{
close();
if(ex instanceof ConnectException)
{
LOGGER.log(Level.WARNING, "The remoteserver cannot be reached");
}
}
}
public void close()
{
try
{
if (itsChannel != null)
{
itsChannel.close();
itsChannel.socket().close();
itsSelector.selectNow();
}
}
catch (IOException e)
{
LOGGER.log(Level.WARNING, "Connection cannot be closed");
}
}
This program runs on Red Hat Enterprise Linux Server release 6.2 (Santiago)
When number of concurrent sockets are in establishment phase, file descriptor limit reaches a max value and I see below exception while trying to establish more socket connections.
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
This happens only when the remote Node is down, and while it is up, all is fine.
When the remote TCP server is down, below exception is thrown as is handled as IOException in the above code
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
Is there any way to forcefully close the underlying file descriptor in this case.
Thanks in advance for all the help.
private Selector itsSelector;
I cannot see the point of this declaration. You can always get the selector the channel is registered with, if you need it, which you never do. Possibly you are leaking Selectors?
itsChannel.configureBlocking(false);
itsChannel.register(itsSelector, SelectionKey.OP_CONNECT);
Here you are registering for OP_CONNECT but never making the slightest use of the facility.
itsChannel.connect(new InetSocketAddress(host, port));
Here you are starting a pending connection.
if (itsChannel.isConnectionPending())
It is. You just started it. The test is pointless.
{
while (!itsChannel.finishConnect())
{
// waiting until connection is finished
}
}
This is just a complete waste of time and space. If you don't want to use the selector to detect when OP_CONNECT fires, you should call connect() before setting the channel to non-blocking, and get rid of this pointless test and loop.
itsChannel.register(itsSelector, SelectionKey.OP_WRITE);
return (itsChannel != null);
itsChannel cannot possibly be null at this point. The test is pointless. You would be better off allowing the IOExceptions that can arise to propagate out of this method, so that the caller can get some idea of the failure mode. That also places the onus on the caller to close on any exception, not just the ones you're catching here.
catch (IOException ex)
{
close();
if(ex instanceof ConnectException)
{
LOGGER.log(Level.WARNING, "The remoteserver cannot be reached");
}
}
See above. Remove all this. If you want to distinguish ConnectException from the other IOExceptions, catch it, separately. And you are forgetting to log anything that isn't a ConnectException.
public void close()
{
try
{
if (itsChannel != null)
{
itsChannel.close();
itsChannel.socket().close();
itsSelector.selectNow();
The second close() call is pointless, as the channel is already closed.
catch (IOException e)
{
LOGGER.log(Level.WARNING, "Connection cannot be closed");
}
I'm glad to see you finally logged an IOException, but you're not likely to get any here.
Don't write code like this.

BluetoothServerSocket doesn't return from accept()

I'm trying to build a little Bluetooth-Android-App for a project in school.
I'm quite new to Android (got my phone since 2 days). I'm experimenting since 2 weeks with android programming on my laptop. Installed a VirtualBox with Android x86 (eeepc) so I can use the BluetoothAdapter of the laptop. Emulator doesn't support Bluetooth and is quite slow. That's about the project...
The problem/question:
A Bluetoothconnection has 2 devices - a connecting and a listening one. The listening device has a BluetoothServerSocket, that loops accept() method until accept() returns a BluetoothSocket.
In my case the accept() method doesn't return so I get stuck and the app freezes with blackscreen asking mit to stop the app or just to wait. When I pass a timeout to accept() --> accept(10000) I get an IOException after the timeout.
listening device:
private class AcceptThread extends Thread {
private BluetoothSocket tSocket;
private BluetoothServerSocket bss = null;
public void run() {
try {
Log.d(TAG, "erzeuge ServerSocket");
bss = BluetoothAdapter.getDefaultAdapter().listenUsingInsecureRfcommWithServiceRecord("BluetoothChatInsecure", MainActivity.BT_UUID);
Log.d(TAG, "ServerSocket OK");
} catch (IOException e) {
e.printStackTrace();
Log.e(TAG, "Fehler Serversocket");
}
while (true) {
Log.d(TAG, "Versuche zu akzeptieren");
try {
Log.d(TAG, "Akzeptieren Anfang");
tSocket = bss.accept(10000);
//this line is never reached
Log.d(TAG, "Akzeptieren Ende");
if (tSocket != null){
//Hier wollen wir hin!
Log.d(TAG, "Verbindung akzeptiert");
ConnectedThread conThread = new ConnectedThread(tSocket);
conThread.run();
bss.close();
break;
} else {
Log.e(TAG, "Fehler, keine Verbindung");
}
} catch (IOException e) {
Log.e(TAG, "IOException währent accept-loop");
//this exception is triggered every 10 sec, when the accept(10000) times out
e.printStackTrace();
}
}
Log.i(TAG, "Acceptthread hat fertig");
}
}
connecting device:
try {
socket = device.createInsecureRfcommSocketToServiceRecord(MainActivity.BT_UUID);
outstr = socket.getOutputStream();
instr = socket.getInputStream();
ois = new ObjectInputStream(instr);
oos = new ObjectOutputStream(outstr);
} catch (IOException e) {
e.printStackTrace();
}
I've read a lot of threads on stackoverflow and some other forums about this topic, but I didn't got a solution for the problem.
Sorry about my English, but I am not a native speaker.
Thanks for any help!
EDIT:
I forgot to write, that I test the app with 2 devices. My laptop does accept-loop, while I use my phone and try to connect.
This is just the normal behavior: accept() will "wait" (block) until a connection has been made from another device. Then it returns the socket representing that connection for further data transfer.
As you have seen, the timeout is signalled via an IOException. The contract of accept() is that it never returns null but always a valid socket, or fails with an exception thrown.
Therefore, thejh is right in saying that you should have a dedicated thread which waits for connections in accept().
When accept() returns a new socket, you may want to spawn another thread to handle further communication over that socket, while the accept() thread loops to wait for the next connection.
N.b.: You cannot shut down a thread blocked in IO (as in accept()) via Thread.interrupt(), but you have to close the ServerSocket from another thread to cause an IOException to 'wake up' the blocked thread.
I've been facing this problem for a couple of days. Finally, I realized why:
I was creating the Thread that accepts incoming connections in the server twice. Thus, the ServerSocket was being created to times, although only the second time the accept() method was called.
This leads to server not accepting any connection!!
It seems that you didn't call socket.connect() from client side in the shown codes.
Today I continued work on project. I got IOException after failing connect() from connecting device.
Now I managed the devices to have a socket, after pairing them before running the app.
EDIT: accept() returns a socket now, but it isn't connected when asking with isConnected().
Socket of the connecting device is connected.

Netty Connection Retries

Retry Connection in Netty
I am building a client socket system. The requirements are:
First attemtp to connect to the remote server
When the first attempt fails keep on trying until the server is online.
I would like to know whether there is such feature in netty to do it or how best can I solve that.
Thank you very much
This is the code snippet I am struggling with:
protected void connect() throws Exception {
this.bootstrap = new ClientBootstrap(new NioClientSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool()));
// Configure the event pipeline factory.
bootstrap.setPipelineFactory(new SmpPipelineFactory());
bootstrap.setOption("writeBufferHighWaterMark", 10 * 64 * 1024);
bootstrap.setOption("sendBufferSize", 1048576);
bootstrap.setOption("receiveBufferSize", 1048576);
bootstrap.setOption("tcpNoDelay", true);
bootstrap.setOption("keepAlive", true);
// Make a new connection.
final ChannelFuture connectFuture = bootstrap
.connect(new InetSocketAddress(config.getRemoteAddr(), config
.getRemotePort()));
channel = connectFuture.getChannel();
connectFuture.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future)
throws Exception {
if (connectFuture.isSuccess()) {
// Connection attempt succeeded:
// Begin to accept incoming traffic.
channel.setReadable(true);
} else {
// Close the connection if the connection attempt has
// failed.
channel.close();
logger.info("Unable to Connect to the Remote Socket server");
}
}
});
}
Assuming netty 3.x the simplest example would be:
// Configure the client.
ClientBootstrap bootstrap = new ClientBootstrap(
new NioClientSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool()));
ChannelFuture future = null;
while (true)
{
future = bootstrap.connect(new InetSocketAddress("127.0.0.1", 80));
future.awaitUninterruptibly();
if (future.isSuccess())
{
break;
}
}
Obviously you'd want to have your own logic for the loop that set a max number of tries, etc. Netty 4.x has a slightly different bootstrap but the logic is the same. This is also synchronous, blocking, and ignores InterruptedException; in a real application you might register a ChannelFutureListener with the Future and be notified when the Future completes.
Add after OP edited question:
You have a ChannelFutureListener that is getting notified. If you want to then retry the connection you're going to have to either have that listener hold a reference to the bootstrap, or communicate back to your main thread that the connection attempt failed and have it retry the operation. If you have the listener do it (which is the simplest way) be aware that you need to limit the number of retries to prevent an infinite recursion - it's being executed in the context of the Netty worker thread. If you exhaust your retries, again, you'll need to communicate that back to your main thread; you could do that via a volatile variable, or the observer pattern could be used.
When dealing with async you really have to think concurrently. There's a number of ways to skin that particular cat.
Thank you Brian Roach. The connected variable is a volatile and can be accessed outside the code or further processing.
final InetSocketAddress sockAddr = new InetSocketAddress(
config.getRemoteAddr(), config.getRemotePort());
final ChannelFuture connectFuture = bootstrap
.connect(sockAddr);
channel = connectFuture.getChannel();
connectFuture.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future)
throws Exception {
if (future.isSuccess()) {
// Connection attempt succeeded:
// Begin to accept incoming traffic.
channel.setReadable(true);
connected = true;
} else {
// Close the connection if the connection attempt has
// failed.
channel.close();
if(!connected){
logger.debug("Attempt to connect within " + ((double)frequency/(double)1000) + " seconds");
try {
Thread.sleep(frequency);
} catch (InterruptedException e) {
logger.error(e.getMessage());
}
bootstrap.connect(sockAddr).addListener(this);
}
}
}
});

Client Exception: Software caused connection abort: recv failed

I've been playing around with my Java client app. Behaviour is very similar to this post (and others):
java.net.SocketException: Software caused connection abort: recv failed
As yet I've not found the answer - hence this post. I have a idea what is wrong but I'm not sure what to do about it.
The protocol is simple - read some data in, send back an ACK and repeat (until terminted).
I've been looking at what is going on with WireShark, and is seems the TCP window is filling up. I'm performing a flush() on the DataOutputStream (For the ACKs) but doesn't change the fact at after a while I get this exception (I can see on WireShark that there is always a window problem right before the Java exception).
So how to I make sure in Java that my TCP windows/buffers are clear (Which I think is the root cause of the problem?) seems to be no flush on the DataInputStream. Make be wonder that whilst I might be reading it the TCP stack is filling up.
Many Thanks
Mark
I've attached the basic code calls below:
public void connectToServer()
{
//Create socket connection
try
{
if (lSocket == null)
{
lSocket = new Socket("localhost", 7651);
lSocket.setSoTimeout(0);
lInDataStream = new DataInputStream(lSocket.getInputStream());
lOutDataStream = new DataOutputStream(lSocket.getOutputStream());
}
}
catch (UnknownHostException e)
{
System.out.println("Unknown host: localhost");
}
catch (IOException e)
{
System.out.println("No I/O");
}
}
public void readSocket() throws IOException
{
//Receive data from ROS SerialtoNetwork server
if (lSocket != null)
{
lInDataStream.readInt();
lInDataStream.readFully(cbuf);
lOutDataStream.writeBytes("Ack");
lOutDataStream.flush();
//String lstr = new String(cbuf);
//System.out.print(lstr);
//System.out.println("");
}
}
public String getDataBuffer()
{
String lstr = new String(cbuf);
return lstr;
}
This indicates persistent network errors. There are several (repetitive) MSDN article on this error, e.g. this one.

Categories