is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully? In my case I'm using it for sending a simple string through a java.net.Socket. In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock. What I would like to do is wait for the output to end, and then start reading the input.
Socket sk = new Socket("192.168.0.112", 3000);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write(message.getBytes());
bo.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
if (line.equals("ack")) {
System.out.println("ack");
}
sk.close();
Update
ServerSocket:
ServerSocket ss = new ServerSocket(3000);
System.out.println("server socket open");
while (true) {
Socket sk = ss.accept();
System.out.println("new connection");
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
System.out.println("received line: " + line);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write("ack".getBytes()); bo.flush();
sk.close();
}
Update:
#Global Variable - the reason that read was blocking the socket is that it was waiting for the \n, indeed. Using
bo.write("ack\n".getBytes());
instead of
bo.write("ack".getBytes());
made it work.
Regarding the initial question, is there a way of knowing if flush() method has finished successfully, #Stephen C provided the answer:
there is no way to know that based on the Socket or OutputStream APIs.
The normal way to get that sort of assurance is to have the remote
application send an "reply" in response, and read it in the local
side.
This "reply" is implemented in the code sample and it works.
Is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully?
It depends on what you mean by "finished successfully".
The flush() method ensures that all unsent data in the pipeline has been pushed as far as the operating system network stack. When that is done, then you could say that flush() has finished successfully. The way that you know that that has happened is that the flush() call returns.
On the other hand, if you want some assurance that the data has (all) been delivered to the remote machine, or that the remote application has read it (all) ... there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.
In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock.
The code that you are talking about is basically the correct approach. The way to wait for the response is to read it like that.
If it is not working, then you need to compare what the client and server side are doing:
Is the server waiting for the client to send something more? Maybe an end of line sequence?
Did the server sends the response?
Did it flush() the response?
A mismatch between what the client and server are doing can lead to a form or deadlock, but the solution is to fix the mismatch. Waiting for some kind of hypothetical confirmation of the flush() is not the answer.
UPDATE
The problem is indeed a mismatch. For example, the server writes "ack" but the client expects "ack\n". The same happens in the client -> server case ... unless message always ends with a newline.
Your code is reading reader.readLine() . Are your writing \n when writing? You may want to append \n to the string your are writing.
I tried to reproduce your problem. First, I ran in to some kind of blocking state too, until I realized, I was using readLine at Server-side, too. But the message I was sending did not have a concluding \n. Therefore, the serversocket was still waiting at its InputStream without sending the client the ACK through its OutputStream. I think, #Global Variable is right.
Related
Hello stack overflow world, I've been struggling with the most straight forward and common problem within Java IO, for some time, and now need your help to tackle it.
Check out this piece of code I have in a try block, within a thread.run():
// connect to client socket, and setup own server socket
clientSocket = new Socket(serverHostname, CLIENT_PORT);
//send a test command to download a file
String downloadFileName = "sample.txt";
DataOutputStream dataOutputStream = new DataOutputStream(clientSocket.getOutputStream());
System.out.println("Sending a request to download file : " + downloadFileName + " from user: Arsa node"); //todo: replace with node user later
dataOutputStream.writeUTF("D/sample.txt");
//close socket if host isn't detected anymore, and if socket doesn't become null suddenly
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("****File has been sent****");
in = new DataInputStream(clientSocket.getInputStream());
byte[] retrievedFileData = new byte[8036];
if (in.readInt() > 0) {
System.out.println("Starting file download!");
in.read(retrievedFileData);
System.out.println("File data has been read, converting to file now");
//closing input stream will close socket also
in.close();
}
clientSocket.close();
2 Main questions that have been confusing me to death:
Why does dataOutputStream.close() need to be run for writeUTF to actually send my string to the server socket, I find that when I don't have dos.close(), data isn't retrieved on the other side, further because I close it, I no longer can read from the socket - as it seems the socket connection becomes closed when the Output Stream is previously closed...
What's a better way, following some sort of pattern to do this? For context, all I'm trying to do is write the filename I'm looking to download to my client, then read the response right away, which I expect to be bytes with the file, any error handling I will consider as a part of my development.
Overall, it shouldn't be complicated to write something to a socket, then read and ingest it's response...which doesn't seem to be the case here,
any help would be greatly appreciated! If the ServerSocket code snippet is needed I'm happy to share.
The observed behavior is just a side-effect of close(), as it calls flush() before closing to make sure any buffered data is sent. To solve your problem, you need to call the flush() method instead of closing.
This behavior is not unique to DataOutputStream: a lot of other OutputStream (or Writer) implementations apply buffering, and you will need to flush when you want to ensure the data is sent to the client, written to disk or otherwise processed.
BTW: The DataOutputStream and DataInputStream is for a very specific type of data serialization protocol that is particular to Java. You may want to consider carefully if this is the right protocol to use.
I'm using Erlang with SSL,
My server socket listens to incoming client connections and spawns a new thread for every incoming connection (assume the looping function called clientroutine())
This thread is designed based on this tutorial I found on web: http://erlycoder.com/89/erlang-ssl-sockets-example-ssl-echo-server-ssl-client-
so basically clientroutine() waits in receive, gets data from client, does some action based on received data and recursively calls itself again
Now, the problem is that when I issue ssl:send(Socket, Data), the client (Java-based) does not get anything from inputstream
Interestingly, this happens only when I recursively call clientroutine() after ssl:send like this (I skip socket close and default cases for simplicity):
clientroutine(Socket) ->
ssl:setopts(Socket, [{active, once}]),
receive
{ssl, Sock , Data} ->
ok = ssl:send(Sock, "~100 bytes list goes to client"),
clientroutine(Socket)
end.
The following works correctly (i.e. no recursion takes places and thread finishes) and my Java-client receives the string from inputstream:
clientroutine(Socket) ->
ssl:setopts(Socket, [{active, once}]),
receive
{ssl, Sock , Data} ->
ok = ssl:send(Sock, "~100 bytes list goes to client")
end.
Java-client launches inputstream listener in a separate thread like this (BufferedReader in has been declared above, among class fields):
new Thread(new Runnable(){
#Override
public void run() {
String msg;
try {
while((msg=in.readLine())!=null)
System.out.println("user received: " + msg);
} catch (IOException e) {
System.out.println("user: exception occured - inputstream reader");
}
}}).start();
I haven't yet checked if this works with Erlang client or not, I will update my post when I check it as well, but anyhow I need it to work with Java client
Any ideas why this happens?
Probably I should use some other BufferedReader routine instead of readLine(), or maybe BufferedReader requires some special character to be pushed into outputstream after the transferred message?
UPDATE. Erlang client receives everything correctly, with and without recursive call. Seems that this is somewhat related to Java inputstream
I found out that the newline character required for readLine() to fetch a line from inputstream, was (strangely) not included when I was sending my message in "recursive" version of the program, so everything goes well after I append \n to the transferred message
I'm writing a Java client/server application. It should allow clients to send text data to the server. This kind of communication should be repeatable many times using the same connection.
I write it like this:
// On a server:
ServerSocket serverSocket = new ServerSocket(port);
Socket socket = serverSocket.accept();
socket.setKeepAlive(true);
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
if (reader.ready()) {
for (String line = reader.readLine(); line != null; line = reader.readLine()) {
// do something with line
}
}
// On a client:
Socket socket = new Socket(host, port);
socket.setKeepAlive(true);
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
writer.write("Some data from client to server");
writer.flush();
The problem is: I can't read on a server before I close OutputStream on a client. Or I can't open OutputStream on a client again, if it was already closed. How can I do continuous sending and reading of data?
You need two threads at both ends, one for reading data and other one for writing data.
The problem is: I can't read on a server before I close OutputStream on a client.
Yes you can. You just can't get to the case where readLine() returns null. It isn't the same thing.
Or I can't open OutputStream on a client again, if it was already closed.
Of course not. You have to create a new Socket.
How can I do continuous sending and receiving of data?
I don't understand the question. The code you posted doesn't attempt that.
If your goal is to send many mesages over the same socket connection, these messages will have to be delimited by an application-level protocol. In other words, you won't be able to rely on any system calls like reader.ready() or reader.readLine() == null to detect the end of the message on te server.
One way to achieve this is to begin each message with its length in characters. The server will then read exactly that number of charecters, and then stop and wait for a new message. Another is to define a special character sequence which concludes each message. The server will react to reading that particular sequence by ending the reading of the current message and returning to the "wait for new message" state. You must ensure that this sequence never appears in the message itself.
I am using the below code to send data to a tcp server. I am assuming that I need to use socket.shutdownOutput() to properly indicate that the client is done sending the request. Is my assumption correct? If not please let me know the purpose of shutdownOutput(). Also appreciate any further optimizations I can make.
Client
def address = new InetSocketAddress(tcpIpAddress, tcpPort as Integer)
clientSocket = new Socket()
clientSocket.connect(address, FIVE_SECONDS)
clientSocket.setSoTimeout(FIVE_SECONDS)
// default to 4K when writing to the server
BufferedOutputStream outputStream = new BufferedOutputStream(clientSocket.getOutputStream(), 4096)
//encode the data
final byte[] bytes = reqFFF.getBytes("8859_1")
outputStream.write(bytes,0,bytes.length)
outputStream.flush()
clientSocket.shutdownOutput()
Server
ServerSocket welcomeSocket = new ServerSocket(6789)
while(true)
{
println "ready to accept connections"
Socket connectionSocket = welcomeSocket.accept()
println "accepted client req"
BufferedInputStream inFromClient = new BufferedInputStream(connectionSocket.getInputStream())
BufferedOutputStream outToClient = new BufferedOutputStream(connectionSocket.getOutputStream())
ByteArrayOutputStream bos=new ByteArrayOutputStream()
println "reading data byte by byte"
byte b=inFromClient.read()
while(b!=-1)
{
bos.write(b)
b=inFromClient.read()
}
String s=bos.toString()
println("Received request: [" + s +"]")
def resp = "InvalidInput"
if(s=="hit") { resp = "some data" }
println "Sending resp: ["+resp+"]"
outToClient.write(resp.getBytes());
outToClient.flush()
}
I am using the below code to send data to a tcp server. I am assuming
that I need to use socket.shutdownOutput() to properly indicate that
the client is done sending the request. Is my assumption correct?
YES Your assumption is correct. And this output ShutDown is known as half close . Using half close the TCP provides the ability for one end of the connection to terminate its output, while still receiving data from the other end. Let me walk you through the effects of socket.shutdownOutput() method :
Locally, the local socket and its input stream behave normally for reading
purposes, but for writing purposes the socket and its output stream behave
as though the socket had been closed by this end: subsequent writes to the
socket will throw an IOException
TCP’s normal connection-termination sequence (a - FIN acknowledged by
an ACK) is queued to be sent after any pending data has been sent and acknowledged.
Remotely, the remote socket behaves normally for writing purposes, but for
reading purposes the socket behaves as though it had been closed by this
end: further reads from the socket return an EOF condition, i.e. a read count
of -1 or an EOFException , depending on the method being called.
When the local socket is finally closed, the connection-termination sequence
has already been sent, and is not repeated; if the other end has already
done a half-close as well, all protocol exchanges on the socket are now
complete.
Hence we see that When the EOF is received, that end is assured that the other end has done the output shutdown. And this scenario is perfectly achieved by socket.shutDownOutput() on the other side.
Source: Fundamental Networking in Java, Esmond Pitt
Socket.shutdownOutput() means that the client is finished sending any data through the TCP connection. It will send the remaining data followed by a termination sequence which will completely close its OUTGOING connection. It is not possible to send any further data, which will also indicate to your program that the request is completely finished. So its recommended if you are sure you don't have to send any more data.
But it's not needed to indicate that the request is finished (you don't have to open/close the output all the time if you have multiple requests), there are other ways.
I'm working on a network app written in Java, using ObjectOutputStream and ObjectInputStream on top of Sockets to exchange messages. My code looks like this:
Sender:
ObjectOutputStream out;
ObjectInputStream in;
try{
Socket socket=new Socket(address, port);
socket.setSoLinger(true, socketLingerTime);
out=new ObjectOutputStream(socket.getOutputStream());
out.writeObject(message);
out.flush();
out.close();
}catch (variousExceptions)...
Receiver:
Object incoming;
try{
incoming=myObjectInputStream.readObject();
}catch (SocketException socketError)
{
if (socketError.getMessage().equals("Connection reset"))
{
//this is the exception I get
}
}
Sometimes the message goes through ok, but other times I get the marked exception instead of an object. Isn't flush supposed to force the message through to the other side? Am I somehow using the function incorrectly? Or is this some sort of bug in the underlying Java/OS network code?
Thanks!
UPDATE:
I've done some more snooping on this, and it seems to only happen when the system's resources are being taxed by something. I've not been able to replicate it outside the VirtualBox, but that could just be because the VirtualBox doesn't have many resources to begin with. I'll keep this question updated as I look into it further.
It turns out the issue was caused by Nagle's Algorithm; the output buffer is within the OS, so it wasn't affected by flush. The solution is to turn Nagle's Algorithm off using Socket.setTcpNoDelay(true), and buffer messages at the user level using BufferedOutputStream.
For my case, it's a silly problem but wasting me 4 hours.
Just have to use outStream.writeln(""); or outStream.write(mess + "\n");
Since reader.readLine() reads until it finds '\n' character. So write() alone won't work.
You should be able to send one object per connection.
To ensure resources are cleaned up in an orderly manner it is best to close the socket as well as the output stream.
close() will call flush so it should be redundant.
What happens if you don't set the SO Linger?
What is the actual exception you are getting?
It sounds like a firewall in one of the routers in the path from client to server is sending an RST for some reason. I don't believe there's anything wrong with your code. I tried to replicate the problem, but couldn't.
Connection resets can be caused by writing to a connection that is already closed at the other end. Detection can occur at the next I/O or a subsequent one, e.g. a read. In other words it can be caused by a bug in your application protocol. SO_LINGER won't help, don't mess with this.