I have a HTTP server with runs with HTTP and HTTPS, written using Javas NIO and SSL libraries. In HTTPS mode it can communicate with or without the client certificate. However, I would like to perform renegotiation. Here the client will connect with HTTPS, browse resources and then when they hit a highly secure resource the server challenges the client for its certificate. I have been having a few problems with this and need to know what the workflow should be. Here is what I have observed with both IE 9 and Chrome.
1) When the client requests the secure resource, I respond to the HTTP request in full. I then challenge the client for their cert upon completion with
engine.setNeedClientAuth(true);
engine.beginHandshake();
The result is a TCP FIN from the client (it closes its side of the connection), and the renegotiation fails.
2) When the client requests the secure resource, I challenge for the cert before responding. In this scenario the exchange occurs, both browsers will popup a request for the cert, however as soon as it pops up the prompt a TCP FIN is sent from the client and renegotiation terminates. The client then sends another request which eventually has the certificate, at times I have to challenge twice.
So my question here is, what is supposed to happen? Is the initial browser connection supposed to remain open, or is termination like this normal?
NOTE: Another very interesting observation here is that, in scenario 2, when the browser closes the TCP connection, it then reconnects after you choose the certificate. It does not however repost the request, it just sits there and expects the server to respond? In NIO terminology its sits waiting on an OP_READ, which means there is no data on the socket input buffer. Do the browsers expect a response to the original message that it terminated the connection for??
Strange that there is absolutely no documentation or a specification for this workflow, yet for all the browsers I've tested they seem to follow this workflow.
(1) is insecure and therefore pointless to discuss further. You've already leaked the information before you even ask for the credentials.
(2) is the correct way to do this. The client shouldn't be closing the connection if it is configured to allow renegotiation. Due to an SSL security problem last year or so there was temporarily a phase where SSL renegotiation was disallowed by default. You may be running into this. In that case you should be issuing an HTTP redirect first, and closing the connection at your end to force the client to use a new connection, and the new connection should ask for a client certificate. How you arrange that in your code is up to you.
Related
Good afternoon everyone,
I'm having some issues with my HTTP server. I've made my own HTTP server (A lightweight HTTP server due some circumstances and needings) that I want to implement on a software I have. This HTTP API also is used to allow HTTPS, but my main issue comes actually with HTTP.
One issue I'm facing is retrieving HTTPS connections on the HTTP server. Using HTTPS as the server and HTTP as the connection from the client gets denied, as the Handshake fails and gives an Exception to the server. The problem using the HTTP server with an HTTPS client is that this connections keeps running, but the message is encrypted. As it's encrypted, I can't read the information and get details like the Content-Length, so the server is waiting for an end that will never come as it can't read correctly the data.
I was wondering if there's a way in Java to detect if the client is using encrypted responses to deny this connections instead of trying to read them. The main issue with this sockets is that they aren't detected as SSLSockets, they are normal sockets that can't decrypt the information in the InputStream.
Thank you in advance.
Are you aware that HTTP and HTTPS are usually served on different port numbers? So 80 is for HTTP and 443 for HTTPS. For non-privileged ports often 8000 and 8443 are used. A client that connects using TLS on a HTTP-only port is faulty, and your HTTP server should easily detect non-HTTP traffic:
If the first word received isn't one of the HTTP verbs supported by your server, such as GET, HEAD, POST, PUT, OPTIONS, etc. your server should send a 400 or 408 response (408 is request timeout, your server should only wait a reasonable amout of time for the request header) and then close the connection.
I have a Java TLS client that can send a series of requests to a server, each followed by a response from the server.
However, there are many different servers. Some are "multi-message" servers that keep a connection open after the first request, so that subsequent requests can be sent over the first connection. Others are "single-message" servers that close the connection after each message and so a new connection is required for subsequent messages. There is no a priori way for the client to know what type of server it is talking to, nor to fix the servers.
It is very desirable for single-message servers to be able to resume a session without the full handshake.
My original client code just tried to send subsequent requests down the same connection. If that failed it just opened a new connection to the server. It could thus handle both single and multi-message servers.
However, the failure when sending the second message to single-message servers seems to kill the session resumption.
My dirty work around is to notice if a message fails and then assume that it is talking to a single-message server, in which case the client then explicitly closes the socket after each response has been received. This enables subsequent connections to resume sessions.
But there has to be a better way. Testing for isInputShutdown or isConnected does not help, unsurprisingly, as there are timing issues. The connection failure for single-message server actually happens during the read of the response, after the write of the request, presumably due to buffering.
Any ideas much appreciated?
Your initial solution is correct in the case of plaintext: you will get an IOException: connection reset by peer when sending the second message, and you can just recover accordingly by reconnecting.
However in the TLS case it won't work, as you will not get IOException: connection reset by peer but a SocketException, due to a fatal TLS unexpected_message alert. RFC 2246 #7.2 states:
Alert messages with a level of fatal result
in the immediate termination of the connection. In this case, other
connections corresponding to the session may continue, but the
session identifier must be invalidated, preventing the failed session
from being used to establish new connections.
[my emphasis], because the failed session is now deemed insecure, and
unexpected_message
An inappropriate message was received. This alert is always fatal
and should never be observed in communication between proper
implementations.
Your second solution seems appropriate to me.
NB:
isInputShutdown() can never be true, as you can't call shutdownInput() on an SSLSocket.
isConnected() will never be false once the socket has been connected.
Both tell you about the state of your socket, not of the connection, so even trying them was futile. And it has nothing to do with 'timing issues'.
I am encountering an interesting issue wherein a TCP connection for a HTTP 1.1 POST request is being closed immediately following the request (ie, before the response can be sent by the server).
A few details about the test environment:
Client - Windows XP, Internet Explorer 8, Flash player 12.
Server - Java 7
Prior to the aforementioned behaviour, we have several longstanding TCP connections, each being reused for multiple HTTP requests; we open a long poll and when this poll completes, open another. We see several hours of well behaved and reused TCP connections opening polls as the previous poll closes.
Eventually -- sometimes after 12 or more hours of normal behaviour -- a poll on a long standing connection will send the HTTP POST and immediately send a TCP FIN before the server can write the response.
The client behaviour is to keep a poll open at all times, so at this point we try to open a new poll.
A new TCP connection is then opened by the client sending another HTTP POST, with the same behaviour; the request is sent, followed by a FIN from the client.
This behaviour can continue for several minutes, until the server can finally respond to kill the client. (The server detects the initial closed connection by encountering an IO Exception, the next time it can communicate with the client, the response is to tell the client to close)
Edit: We are opening connections only through the Flash client, and are not delving into low level TCP code. While Steffen Ullrich is correct, and the single sided shutdown is possible and should be dealt with, what is not clear is why a single sided shutdown is occurring at this (seemingly arbitrary) point. We are not calling close from the application to instigate this behaviour.
My questions are:
Under what circumstances would a TCP connection for a HTTP request be terminated prior to the response being received? I understand this is bad behaviour, and an incomplete HTTP transaction, so presumably something lower down is terminating the connection for an unknown reason.
Are there any diagnostics that could be used to help understand the problem? (We are currently monitoring server and client side activity with Wireshark.)
Notes:
In Wireshark, the behaviour we see is:
Longstanding TCP connection (#1) serving multiple HTTP requests.
HTTP request is made over #1.
Server ACKs the request.
Client sends FIN to close connection #1. Server responds with FIN,ACK. (The expected traffic would be the server sending the HTTP response). Around this point the server experiences an IO Exception.
Client opens connection #2 and sends HTTP request.
Behaviour continues as from 3.
Sending a request immediatly followed by a FIN is not a connection close, but shutdown of writing shutdown(socket,SHUT_WR). The client tells the server this way that it will not send any more data, but it might still receive data. It's not that uncommon.
I am trying to insert a header on all HTTPS requests that passes my proxy server.
I read that HTTPS request headers can not be tampered with.
Is there any way that I can insert a header on HTTPS requests?
Only if the proxy is the endpoint of the TLS connection with the client (i.e. the proxy decrypts the requests). Otherwise it won't actually see any HTTP headers at all, just TCP sessions containing encrypted data.
It is common for a reverse proxy local to the web server(s) to terminate TLS connections for performance - the Web server can dedicate more resources to serving applications because it doesn't need to spend CPU cycles on de/encryption.
If you do this, your server certificate must identify the proxy server, not the web server. Otherwise clients will get a warning about the identity of the server differing from the certificate identity.
If you don't control or trust the infrastructure between the proxy and your server you could re-encrypt the ongoing requests.
Unless you are willing to terminate the SSL connection on your proxy, no you can't. This is one of the main goals of SSL. The data and that includes the headers are encrypted from the client to the server.
If you decide to terminate / intercept the SSL connection on your proxy, the clients will get a warning in their browsers. If you are in charge of all the browsers of all the clients you would be able to install a custom certificate and the clients wouldn't get warned anymore.
A simple server
listen = getServer();
Logger.getAnonymousLogger().info("Listening to "+listen.toString());
SSLSocket client = (SSLSocket)listen.accept();
// adding this line fixes everything - client.write(42);
client.close();
and a simple client
SocketFactory sockMaker = SSLSocketFactory.getDefault();
Socket server = sockMaker.createSocket("localhost", 1443);
int retval = server.getInputStream().read();
assert retval == -1;
server.close();
If I don't write anything to the SSL Socket, an exception is thrown in the client side:
Exception in thread "main" javax.net.ssl.SSLException:\
Received close_notify during handshake
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:190)
I don't see why is that. Does the SSL/TLS specification requires you to write things into the socket?
See the full example.
You don't have to write anything into the socket as such, but if you close it straight away, it will generate a close_notify alert (although it's called "alert", it's part of the normal way of closing a TLS/SSL socket).
In addition, SSL/TLS sockets are designed to behave "almost" like normal TCP sockets, but there are details where they don't (and can't) because of the way SSL/TLS works.
In particular, at the beginning of an SSL/TLS connection, the SSL/TLS handshake takes place, which involves a number of read/writes from each side, before sending any application data.
The documentation for SSLSocket says:
The initial handshake on this
connection can be initiated in one of
three ways:
calling startHandshake which explicitly begins handshakes, or
any attempt to read or write application data on this socket causes an implicit handshake, or
a call to getSession tries to set up a session if there is no currently
valid session, and an implicit
handshake is done.
Essentially, the getInputStream().read() by the client in your example initiates the handshake, which causes the server to proceed with accept() and perform the handshake on its side. However, since you close it (normally, but immediately) on the server side, you don't even let any time for the handshake to complete. Hence the close_notify is sent during the handshake, which causes the exception you get. Had you tried to read or write from the server side, the handshake would have at least completed.
EDIT: Following #EJP's comment, I should clarify what I meant:
createSocket("localhost", 1443) on the client side establishes a connection and the server accepts it via accept().
getInputStream().read() on the client side makes it initiate the handshake. Thus, it sends a ClientHello TLS message to the server.
Because the server uses close() straight after accepting the socket, it sends a close_notify alert. Because the server hasn't started to read/write, it hasn't started the handshake (and thus doesn't complete it).
Note that the purpose of ServerSocket.accept(), which SSLServerSocket implements, is to create an SSLSocket, not necessarily to do anything with it. The SSLServerSocket configures it, but proceeding with the handshake is out of scope. On the one hand, it might sound as making the SSLSocket behave more transparently like a normal TCP socket; on the other hand, it would imply reading from the underlying TCP stream, so it would have side-effects.
I haven't tried, but the SSLSocket created by the SSLServerSocket might still be configurable into a client socket. After all as the RFC 2246 glossary says:
"client: The application entity that initiates a TLS connection to a server. This may or may not imply that the client initiated the underlying transport connection." This would definitely have consequences regarding the transparency and when to do the handshake from the API point of view.
(Writing an API for SSL/TLS sockets that maps to the API of normal TCP sockets is a tricky exercise, and Java doesn't do too bad a job at it. The real "fun" starts with asynchronous TLS using SSLEngine and NIO channels. It gets even better considering that either side can initiate a new handshake at any time: the implications for the level above are undefined as far as TLS is concerned, which can lead to awkward problems.)
The situation is invalid. You are trying to read data that isn't sent. This is an application protocol error. All the statements in Bruno's answer apply as well. The client is trying to do a handshake; the server is trying to close the connection. Arguably the server close could initiate a handshake if it hasn't been done already, but it doesn't.
As you've noted, another workaround would be to call startHandshake() at either end.