We Keep Coding

java.net.Socket > outputStream > BufferedOutputStream flush() confirmation

is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully? In my case I'm using it for sending a simple string through a java.net.Socket. In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock. What I would like to do is wait for the output to end, and then start reading the input.
Socket sk = new Socket("192.168.0.112", 3000);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write(message.getBytes());
bo.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
if (line.equals("ack")) {
System.out.println("ack");
}
sk.close();
Update
ServerSocket:
ServerSocket ss = new ServerSocket(3000);
System.out.println("server socket open");
while (true) {
Socket sk = ss.accept();
System.out.println("new connection");
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
System.out.println("received line: " + line);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write("ack".getBytes()); bo.flush();
sk.close();
}
Update:
#Global Variable - the reason that read was blocking the socket is that it was waiting for the \n, indeed. Using
bo.write("ack\n".getBytes());
instead of
bo.write("ack".getBytes());
made it work.
Regarding the initial question, is there a way of knowing if flush() method has finished successfully, #Stephen C provided the answer:
there is no way to know that based on the Socket or OutputStream APIs.
The normal way to get that sort of assurance is to have the remote
application send an "reply" in response, and read it in the local
side.
This "reply" is implemented in the code sample and it works.

Is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully?
It depends on what you mean by "finished successfully".
The flush() method ensures that all unsent data in the pipeline has been pushed as far as the operating system network stack. When that is done, then you could say that flush() has finished successfully. The way that you know that that has happened is that the flush() call returns.
On the other hand, if you want some assurance that the data has (all) been delivered to the remote machine, or that the remote application has read it (all) ... there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.
In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock.
The code that you are talking about is basically the correct approach. The way to wait for the response is to read it like that.
If it is not working, then you need to compare what the client and server side are doing:
Is the server waiting for the client to send something more? Maybe an end of line sequence?
Did the server sends the response?
Did it flush() the response?
A mismatch between what the client and server are doing can lead to a form or deadlock, but the solution is to fix the mismatch. Waiting for some kind of hypothetical confirmation of the flush() is not the answer.
UPDATE
The problem is indeed a mismatch. For example, the server writes "ack" but the client expects "ack\n". The same happens in the client -> server case ... unless message always ends with a newline.

Your code is reading reader.readLine() . Are your writing \n when writing? You may want to append \n to the string your are writing.

I tried to reproduce your problem. First, I ran in to some kind of blocking state too, until I realized, I was using readLine at Server-side, too. But the message I was sending did not have a concluding \n. Therefore, the serversocket was still waiting at its InputStream without sending the client the ACK through its OutputStream. I think, #Global Variable is right.

purpose of socket.shutdownOutput()

I am using the below code to send data to a tcp server. I am assuming that I need to use socket.shutdownOutput() to properly indicate that the client is done sending the request. Is my assumption correct? If not please let me know the purpose of shutdownOutput(). Also appreciate any further optimizations I can make.
Client
def address = new InetSocketAddress(tcpIpAddress, tcpPort as Integer)
clientSocket = new Socket()
clientSocket.connect(address, FIVE_SECONDS)
clientSocket.setSoTimeout(FIVE_SECONDS)
// default to 4K when writing to the server
BufferedOutputStream outputStream = new BufferedOutputStream(clientSocket.getOutputStream(), 4096)
//encode the data
final byte[] bytes = reqFFF.getBytes("8859_1")
outputStream.write(bytes,0,bytes.length)
outputStream.flush()
clientSocket.shutdownOutput()
Server
ServerSocket welcomeSocket = new ServerSocket(6789)
while(true)
{
println "ready to accept connections"
Socket connectionSocket = welcomeSocket.accept()
println "accepted client req"
BufferedInputStream inFromClient = new BufferedInputStream(connectionSocket.getInputStream())
BufferedOutputStream outToClient = new BufferedOutputStream(connectionSocket.getOutputStream())
ByteArrayOutputStream bos=new ByteArrayOutputStream()
println "reading data byte by byte"
byte b=inFromClient.read()
while(b!=-1)
{
bos.write(b)
b=inFromClient.read()
}
String s=bos.toString()
println("Received request: [" + s +"]")
def resp = "InvalidInput"
if(s=="hit") { resp = "some data" }
println "Sending resp: ["+resp+"]"
outToClient.write(resp.getBytes());
outToClient.flush()
}

I am using the below code to send data to a tcp server. I am assuming
that I need to use socket.shutdownOutput() to properly indicate that
the client is done sending the request. Is my assumption correct?
YES Your assumption is correct. And this output ShutDown is known as half close . Using half close the TCP provides the ability for one end of the connection to terminate its output, while still receiving data from the other end. Let me walk you through the effects of socket.shutdownOutput() method :
Locally, the local socket and its input stream behave normally for reading
purposes, but for writing purposes the socket and its output stream behave
as though the socket had been closed by this end: subsequent writes to the
socket will throw an IOException
TCP’s normal connection-termination sequence (a - FIN acknowledged by
an ACK) is queued to be sent after any pending data has been sent and acknowledged.
Remotely, the remote socket behaves normally for writing purposes, but for
reading purposes the socket behaves as though it had been closed by this
end: further reads from the socket return an EOF condition, i.e. a read count
of -1 or an EOFException , depending on the method being called.
When the local socket is finally closed, the connection-termination sequence
has already been sent, and is not repeated; if the other end has already
done a half-close as well, all protocol exchanges on the socket are now
complete.
Hence we see that When the EOF is received, that end is assured that the other end has done the output shutdown. And this scenario is perfectly achieved by socket.shutDownOutput() on the other side.
Source: Fundamental Networking in Java, Esmond Pitt

Socket.shutdownOutput() means that the client is finished sending any data through the TCP connection. It will send the remaining data followed by a termination sequence which will completely close its OUTGOING connection. It is not possible to send any further data, which will also indicate to your program that the request is completely finished. So its recommended if you are sure you don't have to send any more data.
But it's not needed to indicate that the request is finished (you don't have to open/close the output all the time if you have multiple requests), there are other ways.

File transfer through Socket in java

I'm making a Network File Transfer System for transfering any kind of file over a network in java. The size also could be of any kind. Therefore I've used UTF-8 protocol for the task.
I'm providing the codes which I've made but the problem is some times the file gets transfered as it is, with no problem at all. But sometimes few kb's of data is just skipped at the receiving end, which actually restricts the mp3/video/image file to be opened correctly. I think the problem is with BUFFER. I'm not creating any buffer which, right now, I think may be of some use to me.
I would really appreciate if anyone could provide any help regarding the problem, so that the file gets transferred fully.
Client side : --->> File Sender
Socket clientsocket = new Socket(host,6789); // host contains the ip address of the remote server
DataOutputStream outtoserver = new DataOutputStream(clientsocket.getOutputStream());
try
{
int r=0;
FileInputStream fromFile1 = new FileInputStream(path); // "path" is the of the file being sent.
while(r!=-1)
{
r = fromFile1.read();
outtoserver.writeUTF(r+"");
}
}
catch(Exception e)
{
System.out.println(e.toString());
}
clientsocket.close();
Server side: --->> File Receiver
ServerSocket welcome = new ServerSocket(6789);
Socket conn = welcome.accept();
try
{
String r1 = new String();
int r=0;
FileOutputStream toFile1 = new FileOutputStream(path); // "path" is the of the file being received.
BufferedOutputStream toFile= new BufferedOutputStream(toFile1);
DataInputStream recv = new DataInputStream(conn.getInputStream());
while(r!=-1)
{
r1 = recv.readUTF();
r = Integer.parseInt(r1);
toFile.write(r);
}
}
catch(Exception e)
{
System.out.println(e.toString());
}

I don't understand why you are encoding binary data as text.
Plain sockets can send and receive streams of bytes without any problems. So, just read the file as bytes using a FileInputStream and write the bytes to the socket as-is.
(For the record, what you are doing is probably sending 3 to 5 bytes for each byte of the input file. And you are reading the input file one byte at a type without any buffering. These mistakes and others you have made are likely to have a significant impact on file transfer speed. The way to get performance is to simply read and write arrays of bytes using a buffer size of at least 1K bytes.)
I'm not sure of this, but I suspect that the reason that you are losing some data is that you are not flushing or closing outtoserver before you close the socket on the sending end.
FOLLOW UP
I also noticed that you are not flushing / closing toFile on the receiver end, and that could result in you losing data at the end of the file.

The first problem I see is that you're using DataInputStream and DataOutputStream. These are for reading/writing primitive Java types (int, long etc), you don't need them for just binary data.
Another problem is that you're not flushing your file output stream - this could be causing the lost bytes.

An explicit flush might help the situation.

Java ObjectOutputStream on Socket not flush()ing

I'm working on a network app written in Java, using ObjectOutputStream and ObjectInputStream on top of Sockets to exchange messages. My code looks like this:
Sender:
ObjectOutputStream out;
ObjectInputStream in;
try{
Socket socket=new Socket(address, port);
socket.setSoLinger(true, socketLingerTime);
out=new ObjectOutputStream(socket.getOutputStream());
out.writeObject(message);
out.flush();
out.close();
}catch (variousExceptions)...
Receiver:
Object incoming;
try{
incoming=myObjectInputStream.readObject();
}catch (SocketException socketError)
{
if (socketError.getMessage().equals("Connection reset"))
{
//this is the exception I get
}
}
Sometimes the message goes through ok, but other times I get the marked exception instead of an object. Isn't flush supposed to force the message through to the other side? Am I somehow using the function incorrectly? Or is this some sort of bug in the underlying Java/OS network code?
Thanks!
UPDATE:
I've done some more snooping on this, and it seems to only happen when the system's resources are being taxed by something. I've not been able to replicate it outside the VirtualBox, but that could just be because the VirtualBox doesn't have many resources to begin with. I'll keep this question updated as I look into it further.

It turns out the issue was caused by Nagle's Algorithm; the output buffer is within the OS, so it wasn't affected by flush. The solution is to turn Nagle's Algorithm off using Socket.setTcpNoDelay(true), and buffer messages at the user level using BufferedOutputStream.

For my case, it's a silly problem but wasting me 4 hours.
Just have to use outStream.writeln(""); or outStream.write(mess + "\n");
Since reader.readLine() reads until it finds '\n' character. So write() alone won't work.

You should be able to send one object per connection.
To ensure resources are cleaned up in an orderly manner it is best to close the socket as well as the output stream.
close() will call flush so it should be redundant.
What happens if you don't set the SO Linger?
What is the actual exception you are getting?

It sounds like a firewall in one of the routers in the path from client to server is sending an RST for some reason. I don't believe there's anything wrong with your code. I tried to replicate the problem, but couldn't.

Connection resets can be caused by writing to a connection that is already closed at the other end. Detection can occur at the next I/O or a subsequent one, e.g. a read. In other words it can be caused by a bug in your application protocol. SO_LINGER won't help, don't mess with this.

Problem with Sending and Receiving Files with SPP over Bluetooth

I am attempting to transfer files (MP3s about six megabytes in size) between two PCs using SPP over Bluetooth (in Java, with the BlueCove API). I can get the file transfer working fine in one direction (for instance, one file from the client to the server), but when I attempt to send any data in the opposite direction during the same session (i.e., send a file from the server to the client), the program freezes and will not advance.
For example, if I simply:
StreamConnection conn;
OutputStream outputStream;
outputStream = conn.openOutputStream();
....
outputStream.write(data); //Data here is an MP3 file converted to byte array
outputStream.flush();
The transfer works fine. But if I try:
StreamConnection conn;
OutputStream outputStream;
InputStream inputStream;
ByteArrayOutputStream out = new ByteArrayOutputStream();
outputStream = conn.openOutputStream();
inputStream = conn.openInputStream();
....
outputStream.write(data);
outputStream.flush();
int receiveData;
while ((receiveData = inputStream.read()) != -1) {
out.write(receiveData);
}
Both the client and the server freeze, and will not advance. I can see that the file transfer is actually happening at some point, because if I kill the client, the server will still write the file to the hard drive, with no issues. I can try to respond with another file, or with just an integer, and it still will not work.
Anyone have any ideas what the problem is? I know OBEX is commonly used for file transfers over Bluetooth, but it seemed overkill for what I needed to do. Am I going to have to use OBEX for this functionality?

It could be as simple as both programs stuck in blocking receive calls, waiting for the other end to say something... try adding a ton of log statements so you can see what "state" each program is in (ie, so it gives you a running commentary such as "trying to recieve", "got xxx data", "trying to reply", etc), or set up debugging, wait until it gets stuck and then stop one of them and single step it.

you can certainly use SPP to transfer file between your applications (assuming you are sending and receiving at both ends using your application). From the code snippet it is difficult to tell what is wrong with your program.
I am guessing that you will have to close the stream as an indication to the other side that you are done with sending the data .. Note even though you write the whole file in one chunk, SPP / Bluetooth protocol layers might fragment it and the other end could receive in fragments, so you need to have some protocol to indicate transfer completion.

It is hard to say without looking at the client side code, but my guess, if the two are running the same code (i.e. both writing first, and then reading), is that the outputStream needs to be closed before the reading occurs (otherwise, both will be waiting for the other to close their side in order to get out of the read loop, since read() only returns -1 when the other side closes).
If the stream should not be closed, then the condition to stop reading cannot be to wait for -1. (so, either change it to transmit the file size first, or some other mechanism).

Why did you decide to use ByteArrayOutputStream? Try following code:
try {
try {
byte[] buf = new byte[1024];
outputstream = conn.openOutputStream();
inputStream = conn.openInputStream();
while ((n = inputstream.read(buf, 0, 1024)) > -1)
outputstream.write(buf, 0, n);
} finally {
outputstream.close();
inputstream.close();
log.debug("Closed input streams!");
}
} catch (Exception e) {
log.error(e);
e.printStackTrace();
}
And to convert the outputStream you could do something like this:
byte currentMP3Bytes[] = outputStream.toString().getBytes();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(currentMP3Bytes);