Some requests are rejected by Tomcat with an empty HTTP 400 response.
A couple of examples:
A request url containing unencoded characters (e.g. '[' or ']' since Tomcat 8.5.x) triggers:
INFO o.a.c.h.Http11Processor Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
A 400 error page is also returned for example when the header size is too large:
INFO: Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Request header is too large
Is it possible to have a custom error page for those errors? More generally for when Tomcat triggers this HTTP 400 response. Delivering an empty response is the worst UX. I am aware that the creation of such requests should be avoided, but I am nonetheless looking for a fallback.
I have set up a custom error page in my (embedded) Tomcat context with ctx.addErrorPage(...) for the error code 400.
It works properly when triggered from my webapp.
E.g. when delegating the error handling to the servlet error handling mechanism with res.sendError(SC_BAD_REQUEST); - res being a HttpServletResponse.
Unfortunately for the kind of tomcat errors described at the top, the custom error page is not used.
Thanks!
This is a nuisance to me as well. Unfortunately, from having a look at the sources, it seems to be wired deep in Tomcat's internals, and can't be changed easily.
In particular, the exceptions you note are thrown in org.apache.coyote.http11.Http11InputBuffer, which is part of one of Tomcat's component called the Coyote HTTP/1.1 Connector (old docs, newer docs don't have this):
The Coyote HTTP/1.1 Connector element represents a Connector component
that supports the HTTP/1.1 protocol. It enables Catalina to function
as a stand-alone web server, in addition to its ability to execute
servlets and JSP pages.
Also, the exceptions end up in catalina.log and are very short - compare this to when you get an exception from the JSP processor, which are several times that size.
So I think it isn't trivial to patch this - at least not without knowledge about Tomcat internals, which I don't have :(
Related
Is there a possibility to force tomcat to ignore "Expect" header or to use only HTTP 1.0?
I am sending curl request curl POST -vv 'http://127.0.0.1:8080/test' -d "#requests/test.xml" -H "SOAPAction: abc/test" -H "Content-Type: text/xml; charset=UTF-8" and curl adds by default "Expect" header which causes server waits for another request which never come.
Tech stack: Springboot 2.1.3.RELEASE, Tomcat 9.0.16
I cannot modify clients and to their requests empty header "Expect:".
I don't think there is a way to get Tomcat to respond only with an HTTP 1.0 response. That would be a protocol violation. If server gets an HTTP 1.1 request it must either respond with a HTTP 1.1 compatible response, or signal that it can't with a 505 HTTP Version Not Supported error.
Likewise, I don't think there is a way to get Tomcat to ignore a (well-formed / non-empty) Expect header. That would also be a protocol violation.
(While it is possible in theory for a server to violate the spec, I couldn't find a way to configure Tomcat to do that. Obviously you could download the Tomcat source code and modify it, but then you have the problem of maintaining your "fork".)
So what are the alternatives:
The HTTP spec says that server must ignore an Expect header in an HTTP 1.0 request. So you could add the --http1.0 option to the curl command.
The curl command uses a defaults file - ~/.curlrc - to get default overrides for various things. You could add an empty default for the Expect header into this file; see How to setup default "Expect" header for curl.
However, I think you may be worried unduly:
If you are worried that this will slow up the client (curl), don't. By default it only waits for 1 second for the 100 Continue response. (And I don't think that is what you are talking about ...)
If you are worried that this might tie up server side resources (in Tomcat), I can't see how that could be significant. The Tomcat server will timeout and close a client connection if there is no activity. This should deal with a curl command that sends the initial request but doesn't follow up.
I am getting below error while executing the code
Error 404--Not Found
From RFC 2068 Hypertext Transfer Protocol -- HTTP/1.1:
10.4.5 404 Not Found
The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent.
If the server does not wish to make this information available to the client, the status code 403 (Forbidden) can be used instead.
The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address.
As I can see URI is correct and still getting the above mentioned error.
I am using tomcat8.0.43 as my server.
When reviewing my logs, occasionally I see:
[...]INFO[...] org.apache.coyote.http11.AbstractHttp11Processor.process
Error parsing HTTP request header
Note: further occurrences of HTTP
header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in the
HTTP protocol
Or:
java.lang.IllegalArgumentException: Invalid character found in the
request target. The valid characters are defined in RFC 7230 and RFC
3986
If I look at my access logs, I see that the urls that were requested to yield these exceptions were things like:
"GET /scripts/index.php?OPT_Session= null" 400
or:
"GET null null" 400
Was I correct in identifying the requests that caused the exceptions to be thrown?
Is there anything that I can do to stop these exceptions from being thrown or restrict these requests from being made?
A normal browser doesn't even allow a client to enter a url with a space in it. It appears these requests do have spaces in them though.
Thanks.
The requests are most probably attacks. If you are running an Internet-facing web server you have to live with them. It is fairly common to put a web server such as Apache in front of Tomcat, possibly configured with mod_security (https://modsecurity.org). In addition you could use fail2ban or a similar solution in order to ban IPs based on errors in the log. However, in my recent experience attackers tend to use a wide range of IP addresses, so fail2ban may not be very effective.
When I make a rest request and server throws an application exception like IllegalArgumentException, I get response with http status 404.
Wouldn't a response with http status 500 be better?
Or what would actually be the expected response when an application exception is thrown? Is there some default behaviour in resteasy, spring or tomcat itself?
I know I can use an ExceptionMapper for resteasy, but is this really best practice or is there a better alternative?
I'm using following setup:
spring
resteasy
tomcat 7
Update:
The problem I'm facing is that I have 2 web applications, one is returning http status 500 and the other is returning http status 400 when an IllegalArgumentException is thrown. I can't figure out why they behave different. It seems to me, that both web application have the same spring and resteasy configuration.
When I'm debugging, I see that resteasy is transforming the IllegalArgumentException to a org.jboss.resteasy.spi.UnhandledException, but the response and the response status code, respectively, is not touched.
So besides not knowing which http status code would be the expected one (400, as Jon Skeet and Stefano Cazzola already pointed out), I didn't know either why the 2 web applications behaves different.
I couldn't find any resteasy ExceptionMapper in both web application.
So is there some default behavior in resteasy, spring or tomcat, which is mapping an IllegalArgumentException to http status 404 or http status 500? Or how can this happen?
The response code is correlated to the error happened in the server. If the error is related to an invalid input received from the client, then the formally correct response status is 400 (Bad Request). The difference with 500 is that returning 400 is the correct behaviour for the server: means, the server processed correctly and responded with an error because the request was wrong. If the same request will ever be resubmitted, it will receive the same error response. The server is not supposed to correct this error, it is up to the client.
You can take this link as a reference
I have a tomcat server and java client with java.net.HttpUrlConnection sending messages. Usually all works fine, but there some requests the client sends out which are responded with a 400 http error.
The problem is that nothing is shown on tomcat logs. I even tried to put .level=fine and other log level changes, and still no exception or error was shown.
I'm suspecting bad encoding in request or something like that, but tomcat doesn't give me any clues.
By the way, the URL is OK. I also see the request reaches the server but not to my servlet.
It appears tomcat gets the message, declares it bad and doesn't log what's wrong.
Can anybody think of a way to see the 400 reason?
Thank you
I finally figured out -
I saw that the size of the total headers + size of data written to OutputStream - was exceptionally high - and so I defined in the Tomcat HTTP connector:
Connector port="8080" protocol="HTTP 1/1" maxPostSize="10485760" maxHttpHeaderSize="10485760"/
It is a shame Tomcat doesn't just tell me this, and I had to guess my - I will report this as a bug.
Thank you
Indeed, this was also the solution to the "http 400 bad request" error on Internet Explorer 11.
Adding
maxHttpHeaderSize="10485760" maxPostSize="10485760"
to the Connector-node of Tomcat's server.xml.
Seems my ajax-request (extjs) on IE11 are using a bigger request header then the default (8192 = 8 KB) max http header size on Tomcat 6?
With IE10 and Chrome I didn't experience this problem.
This is a very nice issue to be discussed about. I also faced the same issue in my organization's work. I was clueless about it as there were no good logging about it in catalina.out or in our application logs. Http calls made from client apps to the API that was running in Tomcat_8 were getting rejected by the server with 400 bad request. Only Tomcat locahost_access logs records it by printing the 400 http status fro the request. After I increased the http header size the issue got resolved.