I'm communicating with a server and ideally I would want an input stream and an output stream always up. I receive unsolicited responses, so I always have to be ready to receive data on the input stream.
Before I dive in much further, I should say that whatever connection I make must be capable of supporting SSL.
The problem I'm encountering is when I open an HttpUrLConnection and send stuff from the Output stream, it won't send the actual output until I open an input stream. If I open an input stream, it kills the output stream and there seems no was of recovering it once the stream is closed.
What would be the best way to implement my desired behavior? If maintaining both streams at once on one port is impossible, is there a way I could have two separate streams that share the connection via a sort of port lock/switch?
Is there a library for sending output without having to kill the output stream to signal the end of my request?
Http is a single request protocol.
You cannot (unless websocket) create a persistent connection using http(s). You need another protocol.
A basic socket can be used. It works mostly like an http connection except the url starts with socket://.
You simply open a connection, and then you can get the input stream and the output stream and use them.
- My best bet will be using the java.nio package.
- Use SocketChannel from java.nio package, we can set a SocketChannel into non-blocking mode. When you do so, you can call connect(), read() and write() in asynchronous mode.
Related
I'm writing a toy Java NIO server paired with a normal Java client. The client sends a string message to the server using plain Socket. The server receives the message and dumps the content to terminal.
I've noticed that the same message from client is broken up into bytebuffers differently every single time. I understand this is intended behaviour of NIO, but would like to find out roughly how the NIO decides to chop up a message?
Example: Sending string "this is a test message" to server. The following are excerpts of server loggings (each line represents 1 bytebuffer received).
Run 1:
Server receiving: this is a test message
Run 2:
Server receiving: t
Server receiving: his is a test message
Run 3:
Server receiving: this is
Server receiving: a test message
UPDATE - Issue Resolved
I have installed Wireshark to analyse the packets and it has become apparent that the random "break up" was due to me using DataOutputStream for the writer, which sends the message character by character! So there was a packet for each character...
After changing the writer to BufferedWriter, my short message is now sent as a single packet, as expected. So the truth is Java NIO actually did the clever thing and merged my tiny packets to 1 to 2 bytebuffers!
UPDATE2 - Clarification
Thank you all for your replies. Thank you #StephenC for pointing out that unless I encode the message myself(yes, I did call flush() after writing to BufferedWriter), there's always the possiblity of my message arriving across multiple packets.
So the truth is Java NIO actually did the clever thing and merged my tiny
Actually, no. The merging is happening in the BufferedWriter layer. The buffered writer will only deliver a "bunch" of bytes to the NIO layer when either the application flushes or closes the DataOutputStream or the BufferdWriters buffer fills up.
I was in fact referring to my first attempt with DataOutputStream (I got it from an example online, which obviously is incorrect use of the class now that you've pointed it out). BufferedWriter was not involved. My simple writer in that case went like
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.writeBytes("this is a test message");
Wireshark confirmed that this message was sent(server on localhost) 1 character a packet(22 packets in total for the actual message not including all the ACK and etc).
I'm probably wrong, but this behaviour seems to suggest that the NIO server combined these 22 packets into 1-2 bytebuffers?
The end game I'm trying to achieve here is a simple Java NIO server capable of receiving request and data stream using TCP from various clients, some may be written in C++ or C# by third party. It's not time critical so the clients can send all data in one go and the server can process them at its own pace. That's why I've written a toy client in Java using plain Socket rather than a NIO client. Therefore the client in this case can't really manipulate the ByteBuffer directly, so I probably need some sort of message format. Could I make this work?
If you are sending data over a TCP/IP socket, then there are no "messages" as such. What you send and receive is a stream of bytes.
If you are asking if you can send a chunk of N bytes, and have the receiver get exactly N bytes in a single read call, then the answer is that there is no guarantee that will happen. However, it is the TCP/IP stack that is "breaking up" the "messages". Not NIO. Not Java.
Data sent over a TCP/IP connection is ultimately broken into network packets for transmission. This typically erases any "message" structure based on the original write request sizes.
If you want a reliable message structure over the top of the TCP/IP byte stream, you need to encode it in the stream itself; e.g. using an "end-of-message" marker or prefixing each message with a byte count. (If you want to use fancy words, you need to implement a "message protocol" over the top of the TCP/IP stream.)
Concerning your update, I think there are still some misconceptions:
... it became apparent that the random "break up" was due to me using DataOutputStream for the writer, which sends the message character by character! So there was a packet for each character...
Yes, lots of small writes to a socket stream may result in severe fragmentation at the network level. However, it won't always. If there is sufficient "back pressure" due to either network bandwidth constraints or the receiver reading slowly, then this will lead to larger packets.
After changing the writer to BufferedWriter, my short message is now sent as a single packet, as expected.
Yes. Adding buffering to the stack is good. However, you are probably doing something else; e.g. calling flush() after each message. If you didn't then I would expect a network packet to contain a sequence of messages and partial messages.
What is more, if the messages are too large to fit into a single network packet, or if there is severe back-pressure (see above) then you are liable to get multiple / partial messages in a packet anyway. Either way, the receiver should not rely on getting one (whole) message each time it reads.
In short, you may not have really resolved your issue!!
So the truth is Java NIO actually did the clever thing and merged my tiny
Actually, no. The merging is happening in the BufferedWriter layer. The buffered writer will only deliver a "bunch" of bytes to the NIO layer when either the application flushes or closes the DataOutputStream or the BufferdWriters buffer fills up.
FWIW - given your description of what you are doing, it is unlikely using NIO is helping performance. If you wanted to maximize performance, you should stop using BufferedWriter and DataOutputStream. Instead do your message encoding "by hand", putting the bytes or characters directly into the ByteBuffer or CharBuffer.
(Also DataOutputStream is for binary data, not text. Putting one in front of a Writer doesn't seem right ... if that is what you are really doing.)
As the title says - if I want to close a socket connection is it enough if I just call mySocket.close()? What will happen on the other side?
Or, if it's not the way to go, what is? Do I have to send some kind of "finish" message through the socket so the other side knows it should end as well?
Bonus question: java documentations says
Closing this socket will also close the socket's InputStream and OutputStream.
So is closing the socket alone enough or do I have to explicitly close the streams as well (assuming I'm not using try-with-resources, of course)?
Closing the socket won't do any thing on the other side. The remote host will not know that the connection has been closed.
The only way to check whether the connection is closed is to try to read or write from/to the input/output stream. If -1 is returned by the InputStream#read() method, then the second side knows that the first side closed the connection. A thrown IOException also indicates a closed connection.
Normally you don't need to close the in/output stream as they are closed when invoking Socket#close(). Socket#close() is the only thing you need to do to close a connection.
But if you have a wrapper stream (e.g. BufferdInputStream/BufferedOutputStream) you should explicitly close these wrapper streams to release resources used by these wrapper streams (e.g. byte buffer array). You only need to close the top level wrapper stream as closing the wrapper closes the wrapped stream which closes the wrapped stream of the wrapped stream, too. And so on...
socket.close() is a clean way to close the connection. It causes a TCP FIN packet to be sent to the peer and starts the normal TCP shutdown sequence.
It also closes the socket's InputStream and OutputStream, I don't see any ambiguity in the documentation on that point.
A tcp socket is an endpoint which has bidirectional read and write capabilities. In java we can aquire InputStream and OutputStream of the Socket.
is it safe to use those streams concurrently?
As far as i know there is a single connection that is capable to send or recieve from one endpoint to other data at any given time.
I'm implementing nio transport layer based on SocketChannels, and i want to keep one thread for all writes and one thread for accepting and reads, but i'm not sure what will happen if my threads concurrently try to read and write at the same time on the same socket...
As far as I know there is a single connection that is capable to send or recieve from one endpoint to other data at any given time.
Or both at the same time. It's a full-duplex connection. You can send and receive at the same time.
I am a new java socket developer. In my solution has three sockets for sending and receiving. I want to receive three socket's data in a single thread. For this reason, how i get notification which socket get data from remote.
Handling multiple streams (those of the sockets) within a single thread is possible. It requires the use of socket channels (from java.nio.channels) and of a (single) Selector.
You create a Selector and register the SocketChannels.
To learn about any new possibility for an i/o operation, you call the Selector's select() method, which returns whenever one of the channels is ready for reading, writing or accepting. You'll have to learn the ready channel (i.e., obtain its "key"), and call its appropriate data transfer method.
There is some sample code to be found on the net.
PS: It might be easier to use threads.
I have a Java socket server and the the connection socket working just fine. What I need help with is streaming a response back to the client.
I get the output stream with socket.getOutputStream(). How can I make it so that when I write to the output stream it is immediately sent, but in the future on the same connection I can send another chunk of data.
I tried simply using write and write in conjunction with flush, but I don't really know what I am doing...
Depending on native implementation, the socket may have a buffer, and not send the bytes the second you call write(). flush() however, will force the bytes to be sent. Typically it is good practice to send larger chunks rather than byte by byte (for streaming you generally start by building up a buffer on the receiver's side). Optimal network usage is likely to be to send as large packets as possible (limited by MTU). To have a local buffer in java, wrap the socket outputstream in a BufferedOutputStream.
flush() will force the data to be sent to the OS. The OS can buffer the data and so can the OS on the client. If you want the OS to send data earlier, I suggest you try turning Nagle's Algorithm off. socket.setTcpNoDelay(true); However, you will find that OS/driver parameters can still introduce some buffering/packet coelesing.
If you look at Sun's JDK 6 java.net.SocketOutputStream you will see the flush() method does nothing. This is not guarenteed to be the case on all platforms and a flush() may be required.
Another solution could be DataOutputStream
DataOutputStream dataOut = new DataOutputStream(socket.getOutputStream());
dataOut.writeInt(1)