I use getInputStream().read() to check if client has disconnected from the server. It works but the read() function, as it states in documentation, blocks the first letter of message. So instead of printing "Hello", it prints "ello". How can I make the read() function 'let go' of the first letter?
Here is the following server code to wait for input:
while(true)
{
if(socket.getInputStream().read()==-1)//if no response
break;
String msg = bufferedReader.readLine();
if(msg!=null) {
System.out.println("Received message: " + msg);
}
}
If there is no way to go around that, what's the alternative to getInputStream().read() to detect client disconnection?
The right approach is to not use the separate read() at all. Simply handle readLine() telling you the underlying connection is gone.
For a robust solution, there are two cases you should handle:
readLine() returns null; this is 'normal end of stream' and I'd expect it to happen if the client closed the connection cleanly.
readLine() throws an IOException, which you need try-catch to deal with. This can happen if the connection is terminated abruptly, possibly if the client exits without closing the connection.
It's for you to decide whether those two possibilities are treated identically, based in your program requirements.
Related
Before anything, allow me to show you my client code:
String command;
socket.setSoTimeout(5000);
while(true) {
try {
final byte[] targetArray = new byte[is.available()];
final int x = is.read(targetArray);
if (x == -1) {
System.out.println("end");
break;
}
if (x == 0) continue;
command = new String(targetArray);
handleCommand(command).start();
}catch (Exception e){
e.printStackTrace();
}
}
once connected to the actual socket, the client sends out some authentication data, but doesn't recieve any data, now, it waits to recieve data from the server, and when it does, it processes it fine etc, except, when I stop the server (literally shutting down the program), nothing happens, when in reality, im expecting it to send EOF (-1). Instead it just spams out 0 consistently.
According to available method's documentation (emphasis mine):
Returns an estimate of the number of bytes that can be read ...
and
It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream ...
So according to the minimal code you posted, you shouldn't use available to allocate the buffer, because it may always return 0 which in turn makes the read operation always return 0. But according to your question you only see this behaviour when the sending program is closed which should mean that:
The available method correctly returns the buffered number of bytes in the stream, so this is why you receive the data. Even if you consume received data faster than the sending end sends (so some zeros may show up in available) you just continue and never handle that case, which makes some other loop in the future to capture the (eventually) received non-zero length data.
Once the sending end is closed, there will be a point after which it will always return 0 since it has no data. So you allocate zero length buffer, and the read method first checks that your buffer is zero length, rather than EOF.
So make sure the allocated buffer is of size at least 1.
Also, according to the Socket#getInputStream method's documentation:
If there are no bytes buffered on the socket, and the socket has not been closed using close, then available will return 0.
Right now, I'm trying to write a GUI based Java tic-tac-toe game that functions over a network connection. It essentially works at this point, however I have an intermittent error in which several chars sent over the network connection are lost during gameplay. One case looked like this, when println statements were added to message sends/reads:
Player 1:
Just sent ROW 14 COLUMN 11 GAMEOVER true
Player 2:
Just received ROW 14 COLUMN 11 GAMEOV
Im pretty sure the error is happening when I read over the network. The read takes place in its own thread, with a BufferedReader wrapped around the socket's InputStream, and looks like this:
try {
int input;
while((input = dataIn.read()) != -1 ){
char msgChar = (char)input;
String message = msgChar + "";
while(dataIn.ready()){
msgChar = (char)dataIn.read();
message+= msgChar;
}
System.out.println("Just received " + message);
this.processMessage(message);
}
this.sock.close();
}
My sendMessage method is pretty simple, (just a write over a DataOutputStream wrapped around the socket's outputstream) so I don't think the problem is happening there:
try {
dataOut.writeBytes(message);
System.out.println("Just sent " + message);
}
Any thoughts would be highly appreciated. Thanks!
As it turns out, the ready() method guaruntees only that the next read WON'T block. Consequently, !ready() does not guaruntee that the next read WILL block. Just that it could.
I believe that the problem here had to do with the TCP stack itself. Being stream-oriented, when bytes were written to the socket, TCP makes no guarantees as to the order or grouping of the bytes it sends. I suspect that the TCP stack was breaking up the sent string in a way that made sense to it, and that in the process, the ready() method must detect some sort of underlying break in the stream, and return false, in spite of the fact that more information is available.
I refactored the code to add a newline character to every message send, then simply performed a readLine() instead. This allowed my network protocol to be dependent on the newline character as a message delimiter, rather than the ready() method. I'm happy to say this fixed the problem.
Thanks for all your input!
Try flushing the OutputStream on the sender side. The last bytes might remain in some intenal buffers.
It is really important what types of streamed objects you use to operate with data. It seems to me that this troubleshooting is created by the fact that you use DataOutputStream for sending info, but something else for receiving. Try to send and receive info by DataOutputStream and DataInputStream respectively.
Matter fact, if you send something by calling dataOut.writeBoolean(b)
but trying to receive this thing by calling dataIn.readString(), you will eventually get nothing. DataInputStream and DataOutputStream are type-sensitive. Try to refactor your code keeping it in mind.
Moreover, some input streams return on invocation of read() a single byte. Here you try to convert this one single byte into char, while in java char by default consists of two bytes.
msgChar = (char)dataIn.read();
Check whether it is a reason of data loss.
First, some background. There is a worker which expands/resolves bunch of short URLS:
http://t.co/example -> http://example.com
So, we just follow redirects. That's it. We don't read any data from the connection. Right after we got 200 we return the final URL and close InputStream.
Now, the problem itself. On a production server one of the resolver threads hangs inside the InputStream.close() call:
"ProcessShortUrlTask" prio=10 tid=0x00007f8810119000 nid=0x402b runnable [0x00007f882b044000]
java.lang.Thread.State: RUNNABLE
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.skip(BufferedInputStream.java:352)
- locked <0x0000000561293aa0> (a java.io.BufferedInputStream)
at sun.net.www.MeteredStream.skip(MeteredStream.java:134)
- locked <0x0000000561293a70> (a sun.net.www.http.KeepAliveStream)
at sun.net.www.http.KeepAliveStream.close(KeepAliveStream.java:76)
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.close(HttpURLConnection.java:2735)
at ru.twitter.times.http.URLProcessor.resolve(URLProcessor.java:131)
at ru.twitter.times.http.URLProcessor.resolve(URLProcessor.java:55)
at ...
After a brief research, I understood that skip() is called to clean up the stream before sending it back to the connections pool (if keep-alive is set on?). Still I don't understand how to avoid this situation. Moreover, I doubt if there is some bad design in our code or there is problem in JDK.
So, the questions are:
Is it possible to avoid hanging on close()? Guarantee some reasonable
timeout, for example.
Is it possible to avoid reading data from connection at all?
Remember I just want the final URL. Actually, I think, I don't want
skip() to be called at all ...
Update:
KeepAliveStream, line 79, close() method:
// Skip past the data that's left in the Inputstream because
// some sort of error may have occurred.
// Do this ONLY if the skip won't block. The stream may have
// been closed at the beginning of a big file and we don't want
// to hang around for nothing. So if we can't skip without blocking
// we just close the socket and, therefore, terminate the keepAlive
// NOTE: Don't close super class
try {
if (expected > count) {
long nskip = (long) (expected - count);
if (nskip <= available()) {
long n = 0;
while (n < nskip) {
nskip = nskip - n;
n = skip(nskip);} ...
More and more it seems to me that there is a bug in JDK itself. Unfortunately, it's very hard to reproduce this ...
The implementation of KeepAliveStream that you have linked, violates the contract under which available() and skip() are guaranteed to be non-blocking and thus may indeed block.
The contract of available() guarantees a single non-blocking skip():
Returns an estimate of the number of bytes that can be read (or
skipped over) from this input stream without blocking by the next
caller of a method for this input stream. The next caller might be
the same thread or another thread. A single read or skip of this
many bytes will not block, but may read or skip fewer bytes.
Wheres the implementation calls skip() multiple times per single call to available():
if (nskip <= available()) {
long n = 0;
// The loop below can iterate several times,
// only the first call is guaranteed to be non-blocking.
while (n < nskip) {
nskip = nskip - n;
n = skip(nskip);
}
This doesn't prove that your application blocks because KeepAliveStream incorrectly uses InputStream. Some implementations of InputStream may possibly provide stronger non-blocking guarantees, but I think it is a very likely suspect.
EDIT: After a bit more research, this is a very recently fixed bug in JDK: https://bugs.openjdk.java.net/browse/JDK-8004863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel. The bug report says about an infinite loop, but a blocking skip() could also be a result. The fix seems to address both issues (there is only a single skip() per available())
I guess this skip() on close() is intended for Keep-Alive support.
See http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html.
Prior to Java SE 6, if an application closes a HTTP InputStream when
more than a small amount of data remains to be read, then the
connection had to be closed, rather than being cached. Now in Java SE
6, the behavior is to read up to 512 Kbytes off the connection in a
background thread, thus allowing the connection to be reused. The
exact amount of data which may be read is configurable through the
http.KeepAlive.remainingData system property.
So keep alive can be effectively disabled with http.KeepAlive.remainingData=0 or http.keepAlive=false.
But this can negatively affect performance if you always address to the same http://t.co host.
As #artbristol suggested, using HEAD instead of GET seems to be the preferable solution here.
I was facing a similar issue when I was trying to make a "HEAD" request. To fix it, I removed the "HEAD" method because I just wanted to ping the url
I'm playing around with the NIO library. I'm attempting to listen for a connection on port 8888 and once a connection is accepted, dump everything from that channel to somefile.
I know how to do it with ByteBuffers, but I'd like to get it working with the allegedly super efficient FileChannel.transferFrom.
This is what I got:
ServerSocketChannel ssChannel = ServerSocketChannel.open();
ssChannel.socket().bind(new InetSocketAddress(8888));
SocketChannel sChannel = ssChannel.accept();
FileChannel out = new FileOutputStream("somefile").getChannel();
while (... sChannel has not reached the end of the stream ...) <-- what to put here?
out.transferFrom(sChannel, out.position(), BUF_SIZE);
out.close();
So, my question is: How do I express "transferFrom some channel until end-of-stream is reached"?
Edit: Changed 1024 to BUF_SIZE, since the size of the buffer used, is irrelevant for the question.
There are few ways to handle the case. Some background info how trasnferTo/From is implemented internally and when it can be superior.
1st and foremost you should know how many bytes you have to xfer, i.e. use FileChannel.size() to determine the max available and sum the result. The case refers to FileChannel.trasnferTo(socketChanel)
The method does not return -1
The method is emulated on Windows. Windows doesn't have an API function to xfer from filedescriptor to socket, it does have one (two) to xfer from the file designated by name - but that's incompatible with java API.
On Linux the standard sendfile (or sendfile64) is used, on Solaris it's called sendfilev64.
in short for (long xferBytes=0; startPos + xferBytes<fchannel.size();) doXfer() will work for transfer from file -> socket.
There is no OS function that transfers from socket to file (which the OP is interested in). Since the socket data is not int he OS cache it can't be done so effectively, it's emulated. The best way to implement the copy is via standard loop using a polled direct ByteBuffer sized with the socket read buffer. Since I use only non-blocking IO that involves a selector as well.
That being said: I'd like to get it working with the allegedly super efficient "? - it is not efficient and it's emulated on all OSes, hence it will end up the transfer when the socket is closed gracefully or not. The function will not even throw the inherited IOException, provided there was ANY transfer (If the socket was readable and open).
I hope the answer is clear: the only interesting use of File.transferFrom happens when the source is a file. The most efficient (and interesting case) is file->socket and file->file is implemented via filechanel.map/unmap(!!).
Answering your question directly:
while( (count = socketChannel.read(this.readBuffer) ) >= 0) {
/// do something
}
But if this is what you do you do not use any benefits of non-blocking IO because you actually use it exactly as blocking IO. The point of non-blocking IO is that 1 network thread can serve several clients simultaneously: if there is nothing to read from one channel (i.e. count == 0) you can switch to other channel (that belongs to other client connection).
So, the loop should actually iterate different channels instead of reading from one channel until it is over.
Take a look on this tutorial: http://rox-xmlrpc.sourceforge.net/niotut/
I believe it will help you to understand the issue.
I'm not sure, but the JavaDoc says:
An attempt is made to read up to count bytes from the source channel
and write them to this channel's file starting at the given position.
An invocation of this method may or may not transfer all of the
requested bytes; whether or not it does so depends upon the natures
and states of the channels. Fewer than the requested number of bytes
will be transferred if the source channel has fewer than count bytes
remaining, or if the source channel is non-blocking and has fewer than
count bytes immediately available in its input buffer.
I think you may say that telling it to copy infinite bytes (of course not in a loop) will do the job:
out.transferFrom(sChannel, out.position(), Integer.MAX_VALUE);
So, I guess when the socket connection is closed, the state will get changed, which will stop the transferFrom method.
But as I already said: I'm not sure.
allegedly super efficient FileChannel.transferFrom.
If you want both the benefits of DMA access and nonblocking IO the best way is to memory-map the file and then just read from the socket into the memory mapped buffers.
But that requires that you preallocate the file.
This way:
URLConnection connection = new URL("target").openConnection();
File file = new File(connection.getURL().getPath().substring(1));
FileChannel download = new FileOutputStream(file).getChannel();
while(download.transferFrom(Channels.newChannel(connection.getInputStream()),
file.length(), 1024) > 0) {
//Some calculs to get current speed ;)
}
transferFrom() returns a count. Just keep calling it, advancing the position/offset, until it returns zero. But start with a much larger count than 1024, more like a megabyte or two, otherwise you're not getting much benefit from this method.
EDIT To address all the commentary below, the documentation says that "Fewer than the requested number of bytes will be transferred if the source channel has fewer than count bytes remaining, or if the source channel is non-blocking and has fewer than count bytes immediately available in its input buffer." So provided you are in blocking mode it won't return zero until there is nothing left in the source. So looping until it returns zero is valid.
EDIT 2
The transfer methods are certainly mis-designed. They should have been designed to return -1 at end of stream, like all the read() methods.
Building on top of what other people here have written, here's a simple helper method which accomplishes the goal:
public static void transferFully(FileChannel fileChannel, ReadableByteChannel sourceChannel, long totalSize) {
for (long bytesWritten = 0; bytesWritten < totalSize;) {
bytesWritten += fileChannel.transferFrom(sourceChannel, bytesWritten, totalSize - bytesWritten);
}
}
(Forgive me because I do not write in Java very often.)
I'm writing a client-side network application in Java and I'm having an interesting issue. Every call to readInt() throws an EOFException. The variable is of type DataInputStream (initialized as: DataInputStream din = new DataInputStream(new BufferedInputStream(sock.getInputStream())); where sock is of type Socket).
Now, sock.isInputShutdown() returns false and socket.isConnected() returns true. I'm assuming that this means that I have a valid connection to the other machine I'm connecting to. I've also performed other checks to ensure that I'm properly connected to the other machine.
Is it possible that the DataInputStream was not set up correctly? Are there any preconditions that I have missed?
Any help is greatly appreciated.
#tofubeer: I actually wrote 17 bytes to the socket. The socket is connected to another machine and I'm waiting on input from that machine (I'm sorry if this was unclear). I successfully read from the stream (to initiate a handshake) first and this worked just fine. I'm checking now to see if my sent-requests are malformed, but I don't think they are. Also, I tried reading a single byte from the stream (via read()) and it returned -1.
Are you writing 4 bytes to the socket? According to the JavaDoc it will throw an EOFException if this stream reaches the end before reading all the bytes.
Try calling readByte() 4 times in a row instead of readInt() and see what happens (likely not all of them will work).
Edit (given your edit).
Find out how many times you can call read() before you get the -1.
When read() returns -1 it means that it has hit the end of file.
Also find out what each read() returns to make sure what you are reading in is what you actually wrote out.
It sounds like a problem either with the read code reading more than you thing while doing the handshake or the other side not writing what you think it is writing.
Some things to check:
Did the handshake consume more than 13 bytes, leaving less than four for the readInt()?
Was the integer you want to read written via DataOutputStream.writeInt()?
Did you flush the stream from the sender?
Edit: I took a look at the Java sources (I have the 1.4 sources on my desktop, not sure which version you're using) and the problem might be in BufferedInputStream. DataInputStream.readInt() is just calling BufferedInputStream.read() four times. BufferedInputStream.read() is calling BufferedInputStream.fill() if its buffer is exhausted (e.g., if its first read only got 16 bytes). BufferedInputStream.fill() calls the underlying InputStream's read(byte[], int, int) method, which by contract might not actually read anything! If this happens, BufferedInputStream.read() will return an erroneous EOF.
This is all assuming that I'm reading all of this correctly, which might not be the case. I only took a quick peek at the sources.
I suspect that your BufferedInputStream is only getting the first 16 bytes of the stream in its first read. I'd be curious what your DataInputStream's available() returns right before the readInt. If you're not already, I'd suggest you flush your OutputStream after writing the int you can't read as a possible workaround.