This is one for the tomcat / network experts. I would benchmark / wireshark it but this is pretty demanding and perhaps someone knows the answer offhand.
Comparing these two methods for generating servlet output, which one would be the fastest from a user's perspective:
Writing direct to the servlet output stream:
for( int i=0; i<10000; i++ ) {
servletOutputStream.write( "a" );
/* a little bit of delay */
}
Creating a buffer and write it in one turn
for( int i=0; i<10000; i++ ) {
stringbuffer.append( "a" );
}
servletOutputStream.write( stringBuffer.toString() )
I can imagine the PROs of method 1 would be that the response can start sending stuff quickly while in method 2 the sending starts later.
On the other hand method 1 could generate more / smaller TCP packets which in turn could take longer to transmit completely?
Regards
PS: Please, don't tell me this is premature optimization. In the case at hand I have an object which offers both toString and write(Appendable a) methods. I've simply have to choose which one to use here. Additionally I find this very interesting from a theoretical point of view and regarding the general design of servlets.
EDIT: Thanks all for the answers. But it seems I was to unclear in my question or oversimplified my example.
I'm not worried about not buffering at all. I know that there must be buffering at least in one place in the sending queue. Probably it is in multiple places (Java,OS,Hardware). I think the real question I have is this: When is are these buffers flushed?
So to make it more clear lets assume we have a MTU of 1000 and sending of consecutive packets is triggered by a buffer-empty interrupt by the hardware. Then in the first case it could look like:
. packet( "a" ) //triggered by the first write( "a" ),
. packet( "aaaaaaa" ) // triggered by buffer-empty, sending the amount of "a"s which have been written in the meantime
. packet( "aaaa" ) // and so on
. packet( "aaaaaaaaaaa" )
...x1000 // or so in this example
While for the second case there are all 10000 bytes already available when sending starts and so the result would be:
. packet( "aaaa....a(x1000)" )
. packet( "aaaa....a(x1000)" )
...x10
Even for smaller data sizes (smaller than MTU, lets say 100 "a"s) and creating the output faster then it could be send the result could look like:
. packet( "a" ) // first write
. packet( "aaaa...a(x99) ) // all remaining data available when buffer-empty interrupt.
Of course all this would be quiet different if the buffer(s) where working differently. E.g. if they would be waiting for more data to send or waiting for a flush to send anything at all ... (but this in turn would slow down sending in some respect, too)
So this is what I don't know: How exactly is this buffering within tomcat working and what would be the the best strategy of using it?
(And I'm not worrying or expecting larger speed gains. I just like to know how things work.)
I expect that the ServletOutputStream is actually an instance of
org.apache.tomcat.core.BufferedServletOutputStream
which is (as the name suggests) is a buffered stream. That will mean that it is better to write characters directly to the stream rather than assembling them in a StringBuffer or StringBuilder and writing the result. Writing directly will avoid at least one copy of the characters.
If it turns out that your ServletOutputStream is not buffered already, then you can wrap it in a BufferedOutputStream, and you will get the same result.
Assuming now that you are talking about the streams now. (Flushing a StringBuffer has no meaning.)
When is are these buffers flushed?
When they are full, when you call flush on the stream, or when the stream is closed.
... and what would be the the best strategy of using it?
In general, write the data and when you are finished, close the file. Don't flush explicitly, unless there is a good reason to do so. There rarely is, if you are delivering ordinary HTTP responses. (A flush is liable to cause the network stack to transmit the same amount of information by sending more network packets. That could impact on overall network throughput.)
In the case of the servlet framework, I recall that the Servlet specification says that a ServletOutputStream will automatically be flushed and closed when the request/response processing is finished. Provided that you didn't wrap the ServletOutputStream, you don't even need to close the stream. (It does no harm though.)
There's no doubt that writing directly to the output stream will be faster for a number of reasons:
The output buffer is fixed
The output buffer will be flushed automatically when it's full (and I'd argue that it doesn't matter when this happens, so stop worrying about it)
The output buffer will be re-used
Your StringBuilder can grow very large, taking up lots of heap space
Your StringBuilder will re-allocate its space at intervals, causing new objects to be created, data copied all over the place, etc
All that memory activity will create "garbage" that the GC will have to deal with
However
I would argue that your analysis isn't taking into account a ver important factor: detection and recovery from errors.
If you have a semi-complex procedure that your servlet is performing, it could fail at any time. If it fails after rendering half of the output, you will be unable to do any of the following things:
Issue an "error" HTTP status code (e.g. 500 Server Error)
Redirect the user to another page (error page?)
Show a nice error message on the screen without ruining/interrupting the page
So, even though the manually-buffered approach (based upon the StringBuilder) is less efficient, I believe it gives you a great deal of flexibility for handling errors.
This is more of a religious argument than anything else, but you'll find many web application programmers who would say that your servlet should produce no output at all, and the task of generating responses should be delegated to another component more suited to the task (e.g. JSP, Velocity, FreeMarker, etc.).
If you are, however, writing a servlet with an eye towards raw speed, then by all means: write directly to the output stream. It will give you the best performance in both micro-benchmarks and overall speed under load.
EDIT 2016-01-26
When [are] these buffers flushed?
The servlet spec makes no guarantees about whether or not the ServletOutputStream is buffered or not, but not using a buffer would be a practical mistake: sending TCP packets one-character-at-a-time would certainly be awful for performance.
If you absolutely need to make sure that the response is buffered, you must use your own BufferedOutputStream, because the servlet container could change its implementation at any time and, as mentioned, is not guaranteed to buffer your response for you.
How exactly is this buffering within Tomcat working?
The buffering currently implemented in Tomcat works just like buffering in the standard JDK classes: when the buffer fills, it's flushed to the lower stream and then the balance of bytes remains in the buffer after the call is made.
If you manually call flush on the stream, you'll force the use of Transfer-Encoding: chunked which means that additional data will need to be sent over the wire, because there is no Content-Length (unless you manually set one before you start filling the buffer). If you can avoid chunked-encoding, you can save yourself some network traffic. Also, if the client knows the Content-Length of the response, they can show an accurate progress bar when downloading the resource. With chunked encoding, the client never knows how much data is coming until it's all been downloaded.
Wrap you servletOutputStream in a BufferedOutputStream (unless it already is) and you don't need to worry about silly things like that.
I would definitely use the first one. The servlet output stream is buffered, so you don't have to worry about sending it too fast. Also you allocate a new string everytime with the second one, which might impose a GC overhead overtime. Use the first one and call flush after the loop.
It's already buffered, and in some cases it is written to a ByteArrayOutputStream so that Tomcat can prepend the Content-Length header. Don't worry about it.
Related
I need to empty the buffer of a inputstream from TcpSocket connection.
I tried this:
public void emptyReadBuffer(){
try {
while((DataInputStream)inFromServer.read()>=0) {}
} catch (IOException e) {}
}
But it waits for some input until the timeout... I just want to empty the buffer because I found that sometimes I read dirty data for previous connection.
Your code snippet will loop forever, until either an exception happens, or the socket closes.
What you want is not possible, depending on your definition of 'buffer'.
If you're talking about the actual buffers on the local hardware (So, the network card, etcetera), you can sort of do this, though the actual specification of java's TCP support don't quite guarantee you can do it. However, this is pointless: A new TCP socket simply does not hold stray bytes from previous connections (it would be a major security leak if it was, so, yes, people would care, a lot, it'd be public information).
So, I assume what you really mean is that you have a single, long-lived TCP connections which is used for multiple 'sessions', and you're still picking up bytes from the previous session.
If the latter is indeed what's going on, your use of the word 'buffer' is misleading. You have no idea WHERE those 'stray bytes' are right now: Perhaps they are still stuck somewhere in a router, or in the middle of a cable across the atlantic. The notion of 'just throw away any bytes that have arrived at the local machine' just won't cover it.
What you really need is a unique message along the lines of 'Hello, this is the start of session ', and then your 'emptyReadBuffer' method should instead be: "ignoreAllUntilStartOfSession(String sessionId)" instead. That CAN be done, and will reliably get rid of the previous session.
What your snippet is trying to do and failing would be best done like so:
inFromServer.skip(inFromServer.available());
But as I said, this doesn't accomplish anything reliably. You really, really shouldn't do this. It WILL fail to do what you want one day because networks are varied and unreliable.
I am currently using java.net.Socket to send messages from the client and reading messages from the server. All my messages are fairly short so far, and I have never had any problems.
One of my friends noticed that I was not handling message fragmentation, where the data could come in pieces, and has advised that I should create a buffer to handle this. I insisted that TCP handles this for me, but I'm not 100% sure.
Who is right?
Also, I plan on creating a client in C as well in the future. Do Berkeley sockets handle message fragmentation?
Details: Currently, in Java, the server creates a socket and reads the first byte from the message with InputStream#read(). That first byte determines the length of the entire message, and creates a byte array of the appropriate length, and calls InputStream#read(byte[]) once and assumes that the entire message has been read.
If you are talking about WebSockets,you may be mixing different concepts.
One thing is TCP/IP message fragmentation.
Other thing is how buffering works. You read buffers of data, and you need a framing protocol that tells you when you have a complete "message" (or frame). Basically you:
Read buffer.
Has complete header? No-> Goto 1, Yes-> continue
Read until having all the bytes that the head indicates as message
length.
Has complete message? No-> Goto 3, Yes -> continue
Yield message.
Goto 1.
Other different thing is WebSocket message fragmentation. WebSocket has already a framing protocol and messages can be split in different data frames, and control frames can be interleaved with data frames: https://developer.mozilla.org/en-US/docs/WebSockets/Writing_WebSocket_servers#Message_Fragmentation
If you are writing a WebSocket client or server you have to be ready for this situation.
Expanding on what nos said, TCP will break up large messages into smaller chunks, if the message is large enough. Often, it isn't. Often, the data you write is already split into parts (by you), into meaningful chunks like discrete messages.
The stuff about the reads/writes taking different amounts of calls comes from how the data is written, how it travels over the wire, and how you read it.
If you write 2 bytes 100 times, and then 20 seconds later go to read, it will say there is 200 bytes to be read, which you can read all at once if you want. If you pass a massive 2mb buffer to be written (I dont even know if thats possible), it would take longer to write out, giving more of a chance to the reading program to get different read calls.
Details: Currently, in Java, the server creates a socket and reads the first byte from the message with InputStream#read(). That first byte determines the length of the entire message, and creates a byte array of the appropriate length, and calls InputStream#read(byte[]) once and assumes that the entire message has been read.
That won't work. Have a look at the contract for InputStream.read(byte[]). It isn't obliged to transfer more than one byte. The correct technique is to read the length byte and then use DataInputStream.readFully(), which has the obligation to fill the buffer.
In the following scenario
ObjectOutputStream output = new ObjectOutputStream(socket.getOutputStream());
output.flush();
// Do stuff with it
Why is it always necessary to flush the buffer after initial creation?
I see this all the time and I don't really understand what has to be flushed. I kind of expect newly created variables to be empty unless otherwise is specified.
Kind of like buying a trash-can and finding a tiny pile of trash inside that came with it.
In over 15 years of writing Java on a professional level I've never once encountered a need to flush a stream before writing to it.
The flush operation would do nothing at all, as there's nothing to flush.
You want to flush the stream before closing it, though the close operation should do that for you it is often considered best practice to do it explicitly (and I have encountered situations where that did make a difference, where apparently the close operation did not actually do a flush first.
Maybe you are confused with that?
When you write data out to a stream, some amount of buffering will occur, and you never know for sure exactly when the last of the data will actually be sent. You might perform many rite operations on a stream before closing it, and invoking the flush()method guarantees that the last of the data you thought you had already written actually gets out to the file. Whenever you're done using a file, either reading it or writing to it, you should invoke the close()method. When you are doing file I/O you're using expensive and limited operating system resources, and so when you're done, invoking close()will free up those resources.
This is needed when using either ObjectInputStream and ObjectOutputStream, because they send a header over the stream before the first write is called. The call to flush() will send that header to the remote side.
According to the spec, the header exists of the following contents:
magic version
If the header doesn't arrive at the moment a ObjectInputStream is build, this call will hang until it received the header bytes.
This means that if the protocol in question is written with ObjectStreams, it should flush after creating a ObjectOutputStream.
To be more specific, i have written a server with java NIO, and it works quiet well, after some testing i have found out that for some reason, in average a call to the SocketChannels write method takes 1ms, the read method on the other hand takes 0.22ms in average.
Now at first i was thinking that setting the sent/receive buffer values on Socket might help a bit, but after thinking about it, all the messages are very short(a few bytes) and i send a message about every 2 seconds on a single connection. Both sent and receive buffers are well over 1024 bytes in size so this can't really be the problem, i do have several thousand clients connected at once thou.
Now i am a bit out of ideas on this, is this normal and if it is, why ?
I would start by using Wireshark to eliminate variables.
#Nuoji i am using nonblocikng-io and yes i am using a Selector, as for when i write to a channel i do the following:
Since what i wrote in the second paragraph in my post is true, i assume that the channel is ready for writing in most cases, hence i do not at first set the interest set on the key to write, but rather try to write to the channel directly. In case however that, i can not write everything to the channel (or anything at all for that matter), i set the interest set on the key to write(that way the next time i try to write to the channel it is ready to write). Although in my testing where i got the results mentioned in the original post, this happens very rarely.
And yes i can give you samples of the code, although i didn't really want to bother anyone with it. What parts in particular would you like to see, the selector thread or the write thread ?
I'm coding a tool, that, given any URL, would periodically fetch its output. The problem is that an output could be not a simple and lightweight HTML page (expected in most cases), but some heavy data stream (i.e. straight from /dev/urandom, possible DoS attack).
I'm using java.net.URL + java.net.URLConnection, setting connection and read timeouts to 30sec. Currently input is being read by java.io.BufferedReader, using readLine().
Possible solutions:
Use java.io.BufferedReader.read() byte by byte, counting them and closing connection after limit has been reached. The problem is that an attacker may transmit one byte every 29sec, so that read/connection timeout would almost never occur (204800B * 29sec = 68 days)
Limit Thread execution to 1-5min and use java.io.BufferedReader.readLine(). Any problems here?
I feel like trying to reinvent the wheel and the solution is very straightforward, just doesn't come to my mind.
Thanks in advance.
You could encapsulatebhhis by writing yourself a FilterInputStream that enforces whatever you want to enforce and placing it at the bottom of the stack, around the connection output stream
However this and the remedies you suggest only work if the output is arriving in chunked transfer mode. Otherwise HttpURLConnection can buffer the entire response before you read any of it. The usual solution to this is a filter in the firewall.
There seems to be a number of avenues for denial of service here.
A huge big line that gobbles memory. Probably the easiest is to use a MeteredInputStream before even hitting the character decoding. Reading char by char will be extremely slow in any circumstance. You could read a long char[] at a time, but that will likely over complicate the code.
Dealing with an adversary (or bug) keeping many connections alive at once. You probably want non-blocking I/O reading the whole message, and then proceed normally.