I am using the below code to send data to a tcp server. I am assuming that I need to use socket.shutdownOutput() to properly indicate that the client is done sending the request. Is my assumption correct? If not please let me know the purpose of shutdownOutput(). Also appreciate any further optimizations I can make.
Client
def address = new InetSocketAddress(tcpIpAddress, tcpPort as Integer)
clientSocket = new Socket()
clientSocket.connect(address, FIVE_SECONDS)
clientSocket.setSoTimeout(FIVE_SECONDS)
// default to 4K when writing to the server
BufferedOutputStream outputStream = new BufferedOutputStream(clientSocket.getOutputStream(), 4096)
//encode the data
final byte[] bytes = reqFFF.getBytes("8859_1")
outputStream.write(bytes,0,bytes.length)
outputStream.flush()
clientSocket.shutdownOutput()
Server
ServerSocket welcomeSocket = new ServerSocket(6789)
while(true)
{
println "ready to accept connections"
Socket connectionSocket = welcomeSocket.accept()
println "accepted client req"
BufferedInputStream inFromClient = new BufferedInputStream(connectionSocket.getInputStream())
BufferedOutputStream outToClient = new BufferedOutputStream(connectionSocket.getOutputStream())
ByteArrayOutputStream bos=new ByteArrayOutputStream()
println "reading data byte by byte"
byte b=inFromClient.read()
while(b!=-1)
{
bos.write(b)
b=inFromClient.read()
}
String s=bos.toString()
println("Received request: [" + s +"]")
def resp = "InvalidInput"
if(s=="hit") { resp = "some data" }
println "Sending resp: ["+resp+"]"
outToClient.write(resp.getBytes());
outToClient.flush()
}
I am using the below code to send data to a tcp server. I am assuming
that I need to use socket.shutdownOutput() to properly indicate that
the client is done sending the request. Is my assumption correct?
YES Your assumption is correct. And this output ShutDown is known as half close . Using half close the TCP provides the ability for one end of the connection to terminate its output, while still receiving data from the other end. Let me walk you through the effects of socket.shutdownOutput() method :
Locally, the local socket and its input stream behave normally for reading
purposes, but for writing purposes the socket and its output stream behave
as though the socket had been closed by this end: subsequent writes to the
socket will throw an IOException
TCP’s normal connection-termination sequence (a - FIN acknowledged by
an ACK) is queued to be sent after any pending data has been sent and acknowledged.
Remotely, the remote socket behaves normally for writing purposes, but for
reading purposes the socket behaves as though it had been closed by this
end: further reads from the socket return an EOF condition, i.e. a read count
of -1 or an EOFException , depending on the method being called.
When the local socket is finally closed, the connection-termination sequence
has already been sent, and is not repeated; if the other end has already
done a half-close as well, all protocol exchanges on the socket are now
complete.
Hence we see that When the EOF is received, that end is assured that the other end has done the output shutdown. And this scenario is perfectly achieved by socket.shutDownOutput() on the other side.
Source: Fundamental Networking in Java, Esmond Pitt
Socket.shutdownOutput() means that the client is finished sending any data through the TCP connection. It will send the remaining data followed by a termination sequence which will completely close its OUTGOING connection. It is not possible to send any further data, which will also indicate to your program that the request is completely finished. So its recommended if you are sure you don't have to send any more data.
But it's not needed to indicate that the request is finished (you don't have to open/close the output all the time if you have multiple requests), there are other ways.
Related
Hello stack overflow world, I've been struggling with the most straight forward and common problem within Java IO, for some time, and now need your help to tackle it.
Check out this piece of code I have in a try block, within a thread.run():
// connect to client socket, and setup own server socket
clientSocket = new Socket(serverHostname, CLIENT_PORT);
//send a test command to download a file
String downloadFileName = "sample.txt";
DataOutputStream dataOutputStream = new DataOutputStream(clientSocket.getOutputStream());
System.out.println("Sending a request to download file : " + downloadFileName + " from user: Arsa node"); //todo: replace with node user later
dataOutputStream.writeUTF("D/sample.txt");
//close socket if host isn't detected anymore, and if socket doesn't become null suddenly
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("****File has been sent****");
in = new DataInputStream(clientSocket.getInputStream());
byte[] retrievedFileData = new byte[8036];
if (in.readInt() > 0) {
System.out.println("Starting file download!");
in.read(retrievedFileData);
System.out.println("File data has been read, converting to file now");
//closing input stream will close socket also
in.close();
}
clientSocket.close();
2 Main questions that have been confusing me to death:
Why does dataOutputStream.close() need to be run for writeUTF to actually send my string to the server socket, I find that when I don't have dos.close(), data isn't retrieved on the other side, further because I close it, I no longer can read from the socket - as it seems the socket connection becomes closed when the Output Stream is previously closed...
What's a better way, following some sort of pattern to do this? For context, all I'm trying to do is write the filename I'm looking to download to my client, then read the response right away, which I expect to be bytes with the file, any error handling I will consider as a part of my development.
Overall, it shouldn't be complicated to write something to a socket, then read and ingest it's response...which doesn't seem to be the case here,
any help would be greatly appreciated! If the ServerSocket code snippet is needed I'm happy to share.
The observed behavior is just a side-effect of close(), as it calls flush() before closing to make sure any buffered data is sent. To solve your problem, you need to call the flush() method instead of closing.
This behavior is not unique to DataOutputStream: a lot of other OutputStream (or Writer) implementations apply buffering, and you will need to flush when you want to ensure the data is sent to the client, written to disk or otherwise processed.
BTW: The DataOutputStream and DataInputStream is for a very specific type of data serialization protocol that is particular to Java. You may want to consider carefully if this is the right protocol to use.
In my app I'm using socket connection to communicate with a server. Everything works fine, I'm able to send/receive from/to the server with no issues. None, but one. There's a certain option/response from the server than leaves the connection open for around 30 seconds. Server sends the reply right away, but it keeps the connection open and as such the app hangs there showing the 'waiting' message, because I can't seem to figure out how to close the connection/inputStream without it waiting for the server to close it.
Is there a way to read each character received by the buffer and as soon as a character is found ('*' for example) the buffer should close and also the connection to the server.
Any help?
you need to manage it between client and server. One approach is HTTP chunked. HTTP chunked send first length of message, then message data. Or, if as it is your case, if you know a "magic" byte notifies client that connection can close, you can read data one by one, and when you reach the finalizer byte, you can complete your reading and close connection. Otherwise, application hangs in input.read() until connection reaches timeout or new byte arrives
InputStream input = ....;
ByteArrayOutputStream bo = new ByteArrayOutputStream();
while (true){
int singlebyte = input.read();
if (singlebyte == -1) break; //stream ends
bo.write(singlebyte);
if (singlebyte == '*'){
//the byte you are wating. at this point, you can break loop. or continue to read
bo.write(singlebyte);
byte data[] = bo.toByteArray();
}
}
You do 'read each character as it is found'. Your problem here is not reading characters, it is the server not closing the connection, so you're blocked waiting for end of stream. Such a protocol is either broken, in which case it needs redesigning, or else it includes a length indicator or some other means of knowing when the message is complete,mwhich you're not taking proper notice of.
is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully? In my case I'm using it for sending a simple string through a java.net.Socket. In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock. What I would like to do is wait for the output to end, and then start reading the input.
Socket sk = new Socket("192.168.0.112", 3000);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write(message.getBytes());
bo.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
if (line.equals("ack")) {
System.out.println("ack");
}
sk.close();
Update
ServerSocket:
ServerSocket ss = new ServerSocket(3000);
System.out.println("server socket open");
while (true) {
Socket sk = ss.accept();
System.out.println("new connection");
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
System.out.println("received line: " + line);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write("ack".getBytes()); bo.flush();
sk.close();
}
Update:
#Global Variable - the reason that read was blocking the socket is that it was waiting for the \n, indeed. Using
bo.write("ack\n".getBytes());
instead of
bo.write("ack".getBytes());
made it work.
Regarding the initial question, is there a way of knowing if flush() method has finished successfully, #Stephen C provided the answer:
there is no way to know that based on the Socket or OutputStream APIs.
The normal way to get that sort of assurance is to have the remote
application send an "reply" in response, and read it in the local
side.
This "reply" is implemented in the code sample and it works.
Is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully?
It depends on what you mean by "finished successfully".
The flush() method ensures that all unsent data in the pipeline has been pushed as far as the operating system network stack. When that is done, then you could say that flush() has finished successfully. The way that you know that that has happened is that the flush() call returns.
On the other hand, if you want some assurance that the data has (all) been delivered to the remote machine, or that the remote application has read it (all) ... there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.
In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock.
The code that you are talking about is basically the correct approach. The way to wait for the response is to read it like that.
If it is not working, then you need to compare what the client and server side are doing:
Is the server waiting for the client to send something more? Maybe an end of line sequence?
Did the server sends the response?
Did it flush() the response?
A mismatch between what the client and server are doing can lead to a form or deadlock, but the solution is to fix the mismatch. Waiting for some kind of hypothetical confirmation of the flush() is not the answer.
UPDATE
The problem is indeed a mismatch. For example, the server writes "ack" but the client expects "ack\n". The same happens in the client -> server case ... unless message always ends with a newline.
Your code is reading reader.readLine() . Are your writing \n when writing? You may want to append \n to the string your are writing.
I tried to reproduce your problem. First, I ran in to some kind of blocking state too, until I realized, I was using readLine at Server-side, too. But the message I was sending did not have a concluding \n. Therefore, the serversocket was still waiting at its InputStream without sending the client the ACK through its OutputStream. I think, #Global Variable is right.
I'm writing a Java client/server application. It should allow clients to send text data to the server. This kind of communication should be repeatable many times using the same connection.
I write it like this:
// On a server:
ServerSocket serverSocket = new ServerSocket(port);
Socket socket = serverSocket.accept();
socket.setKeepAlive(true);
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
if (reader.ready()) {
for (String line = reader.readLine(); line != null; line = reader.readLine()) {
// do something with line
}
}
// On a client:
Socket socket = new Socket(host, port);
socket.setKeepAlive(true);
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
writer.write("Some data from client to server");
writer.flush();
The problem is: I can't read on a server before I close OutputStream on a client. Or I can't open OutputStream on a client again, if it was already closed. How can I do continuous sending and reading of data?
You need two threads at both ends, one for reading data and other one for writing data.
The problem is: I can't read on a server before I close OutputStream on a client.
Yes you can. You just can't get to the case where readLine() returns null. It isn't the same thing.
Or I can't open OutputStream on a client again, if it was already closed.
Of course not. You have to create a new Socket.
How can I do continuous sending and receiving of data?
I don't understand the question. The code you posted doesn't attempt that.
If your goal is to send many mesages over the same socket connection, these messages will have to be delimited by an application-level protocol. In other words, you won't be able to rely on any system calls like reader.ready() or reader.readLine() == null to detect the end of the message on te server.
One way to achieve this is to begin each message with its length in characters. The server will then read exactly that number of charecters, and then stop and wait for a new message. Another is to define a special character sequence which concludes each message. The server will react to reading that particular sequence by ending the reading of the current message and returning to the "wait for new message" state. You must ensure that this sequence never appears in the message itself.
it's not my first time trying to understand this issue but i hope it will be the last one:
some background:
i have a Java SocketChannel NIO server working in non-blocking mode.
this server has multiple clients which send and receive messages from it.
each client maintain its connection to the server with "keepalive" messages every once in a while.
The main idea with the server is that the clients will remain connect "all the time" and receive messages from it in "push" mode.
now to my question:
in Java NIO read() function - when the read() return -1 - it means that its EOS.
in the question i've asked here i realized that it means that the socket has finished its current stream and doesn't need to be closed..
when searching in google a bit more about this i found out that it does mean that the connection is closed on the other side..
what does the word "stream" exactly means? is it the current message being sent from the client? is it the ability of the client side connection to send anymore messages ?
why would a SocketChannel be closed on the client side if the client never told him to be closed ?
what is the difference between read() return -1 and connection reset by peer I/O error ?
this is how i read from SocketChannel:
private JSONObject readIncomingData(SocketChannel socketChannel)
throws JSONException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException, IOException {
JSONObject returnObject = null;
ByteBuffer buffer = ByteBuffer.allocate(1024);
Charset charset = Charset.forName("UTF-8");
String endOfMesesage = "\"}";
String message = "";
StringBuilder input = new StringBuilder();
boolean continueReading = true;
while (continueReading && socketChannel.isOpen())
{
buffer.clear();
int bytesRead = socketChannel.read(buffer);
if (bytesRead == -1)
{
continueReading = false;
continue;
}
buffer.flip();
input.append(charset.decode(buffer));
message = input.toString();
if (message.contains(endOfMesesage))
continueReading = false;
}
if (input.length() > 0 && message.contains(endOfMesesage))
{
JSONObject messageJson = new JSONObject(input.toString());
returnObject = new JSONObject(encrypter.decrypt(messageJson.getString("m")));
}
return returnObject;
}
What does the word "stream" exactly means? is it the current message being sent from the client? is it the ability of the client side connection to send anymore messages ?
The stream means the data that is flowing between two locations, usually between the client and the server but effectively it's any kind of data flowing. E.g. if you read a file from your hard disc you use a FileInputStream which represents data flowing from the file on disc to your program. It's a very generic concept. Think of it as a river where the water is the data. Plus it's a very cool kind of river which allows you to control how the water/data is flowing.
Why would a SocketChannel be closed on the client side if the client never told him to be closed ?
That can happen if the connection between client and server is reset or interrupted. Your program should never assume that connections just live and are never interrupted. Connections are interrupted for all kinds of reasons, may it be a flaky network component, someone pulling a plug that should better be left where it was or the wireless network is going down. Also the server might close the connection, e.g. if the server program goes down, has a bug or the connection runs into a timeout. Always remember that open connections are a limited resource so servers might decide to close them if they are idle for too long.
What is the difference between read() return -1 and connection reset by peer I/O error ?
When the read() returns -1 this simply means that there is currently no more data in the stream. A connection reset means, there was probably more data, but the connection no longer exists and therefore this data cannot be read anymore. Again taking the river anology: Think of the data as some quantity of water being sent from a village upstream (aka Serverville) to a village downstream (aka Clientville) using a riverbed that connects the two villages (the connection). Now someone at Serverville pulls the big lever and the water (the data) flows down from Serverville to Clientville. After Serverville has sent all the water it wanted to send, it closes the lever and the riverbed will be empty again (and actually destroyed as the connection got closed). This is where Clientville get's the -1. Now imagine some bulldozer interrupting the riverbed and some of the water never makes it to Clientville. This is the "connection reset" situation.
Hope this helps :)
what does the word "stream" exactly means? is it the current message being sent from the client?
It is a stream of bytes, not messages. You can use those bytes to form a message but the stream has no idea you are doing this, nor does it support messages in any way.
why would a SocketChannel be closed on the client side if the client never told him to be closed ?
It can only be closed with a -1 if the other end closed it.
what is the difference between read() return -1 and connection reset by peer I/O error ?
You can close or drop a connection other ways such as closing it from the same side, or a timeout in the connection e.g.you pulled out the network cable.
BTW: The way you have written the code is better suited to blocking NIO. For example, if you receive more than one whole message, anything after the first one is discarded. If you use blocking IO and keep everything you read you will not get corrupted or dropped messages.
What does the word "stream" exactly means? is it the current message being sent from the client?
It basically means one side of the connection, which is full-duplex. TCP is a byte-stream protocol, providing two independent byte streams, one in each direction.
Why would a SocketChannel be closed on the client side if the client never told him to be closed?
It wouldn't. The client did close the connection. That's what read() returning -1 means.
What is the difference between read() return -1 and connection reset by peer I/O error ?
read() returning -1 means the peer closed the connection properly. 'Connection reset by peer' indicates a protocol error of some kind, usually that you have written data to a connection that had already been closed by the peer.
Re your code, if read() returns -1 you must close the channel. There is no other sensible way to proceed.