I am trying use Java ICAP Codec 1.0.0.GA Squid Echo example with Squid and got the following exception
ch.mimo.netty.handler.codec.icap.IcapDecodingError: Mandatory ICAP message header [Encapsulated] is missing
at ch.mimo.netty.handler.codec.icap.ReadIcapHeaderState.validateMandatoryMessageHeaders(ReadIcapHeaderState.java:104)
at ch.mimo.netty.handler.codec.icap.ReadIcapHeaderState.execute(ReadIcapHeaderState.java:54)
at ch.mimo.netty.handler.codec.icap.IcapMessageDecoder.decode(IcapMessageDecoder.java:97)
at ch.mimo.netty.handler.codec.icap.IcapMessageDecoder.decode(IcapMessageDecoder.java:37)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:470)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:443)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
Here is my Squid config (related to ICAP):
icap_enable on
icap_service service_req reqmod_precache icap://127.0.0.1:1344
adaptation_access service_req allow all
What is the problem ?
Squid's ICAP client is known not to add an Encapsulated Header to Options requests. The latest 1.0.0.GA release of the Java ICAP Codec will handle Options, and 100 Continue requests with missing Encapsulated headers.
I suggest you dump the network traffic with Wireshark and post it here.
Related
Is there a possibility to force tomcat to ignore "Expect" header or to use only HTTP 1.0?
I am sending curl request curl POST -vv 'http://127.0.0.1:8080/test' -d "#requests/test.xml" -H "SOAPAction: abc/test" -H "Content-Type: text/xml; charset=UTF-8" and curl adds by default "Expect" header which causes server waits for another request which never come.
Tech stack: Springboot 2.1.3.RELEASE, Tomcat 9.0.16
I cannot modify clients and to their requests empty header "Expect:".
I don't think there is a way to get Tomcat to respond only with an HTTP 1.0 response. That would be a protocol violation. If server gets an HTTP 1.1 request it must either respond with a HTTP 1.1 compatible response, or signal that it can't with a 505 HTTP Version Not Supported error.
Likewise, I don't think there is a way to get Tomcat to ignore a (well-formed / non-empty) Expect header. That would also be a protocol violation.
(While it is possible in theory for a server to violate the spec, I couldn't find a way to configure Tomcat to do that. Obviously you could download the Tomcat source code and modify it, but then you have the problem of maintaining your "fork".)
So what are the alternatives:
The HTTP spec says that server must ignore an Expect header in an HTTP 1.0 request. So you could add the --http1.0 option to the curl command.
The curl command uses a defaults file - ~/.curlrc - to get default overrides for various things. You could add an empty default for the Expect header into this file; see How to setup default "Expect" header for curl.
However, I think you may be worried unduly:
If you are worried that this will slow up the client (curl), don't. By default it only waits for 1 second for the 100 Continue response. (And I don't think that is what you are talking about ...)
If you are worried that this might tie up server side resources (in Tomcat), I can't see how that could be significant. The Tomcat server will timeout and close a client connection if there is no activity. This should deal with a curl command that sends the initial request but doesn't follow up.
So basically, I have made a RESTful API using ServiceTalk from Apple (Netty implementation) and Jersey and it works. Only through http though. I have seen that when I was making my React web page make a POST request through http, it would complain about CORS (which I'm still trying to fix) and that the browser (At least Brave) would not allow the request to be made because it was http and my web page was running on https using let's encrypt cert. How do I fix this issue? Do I need to add SSL with Netty? If so, how can I do that with a certificate that's going to be changing every once in a while?
I also have NGINX setup with Let's Encrypt and enabled auto-renew certificate setting from the setup wizard for NGINX + Let's Encrypt. If I can somehow make NGINX run the HTTPS request as a proxy to the netty server on http, then I think it would also be a better solution. I know this is a common practice with NodeJS Express + NGINX.
You are right, if you already have NGINX that serves your static content (html/css/js) it will be better to configure it as a proxy for a ServiceTalk backend service. That will let you keep SSL/TLS configuration in one place (NGINX config file only) and you will be able to use its auto-renew certificate feature. For an example of how you can configure NGINX as an SSL/TLS proxy for a backend service, see here: https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/
However, in this case, your connection between NGINX and ServiceTalk will not be encrypted. In some environments, it might be inappropriate according to security policies and requirements. If this is your case, you also need to configure SSL/TLS for ServiceTalk using HttpServerBuilder.secure() method that returns HttpServerSecurityConfigurator. Here is an example of a secure ServiceTalk server.
To avoid CORS, keep using NGINX as a proxy even when ServiceTalk also configured with SSL/TLS connections. If there is a requirement to avoid additional proxy on the way between a browser and backend service, target ServiceTalk directly. But NGINX gives additional features, like load balancing between multiple backend instances.
To get the best SSL performance in ServiceTalk/Netty we recommend to use OpenSSL provided instead of a built-in JDK provider. For more information, see Performance / netty-tcnative OpenSSL engine documentation section.
Note: ServiceTalk does not auto-renew SSL/TLS certificates. You will need to restart the server when certificate expires.
I'm having difficulty using Fiddler to diagnose a problem in a Java application to connect to Microsoft Graph's API. Here's my basic environment:
JDK: 1.8.0_131
Apache HttpClient: 4.4
Fiddler: 4.6
The endpoint I'm trying to reach is https://login.microsoftonline.com/common/oauth2/authorize The problem I have is that the traffic is not captured by Fiddler.
I've tried various things as suggested by Googling for this problem, but have not got it to work. In my application, I set the proxy settings as follows:
httpClientBuilder.setProxy(new HttpHost(proxyServer, proxyPort, proxyScheme));
httpClientBuilder.build();
If I use ("127.0.0.1", 8888, "https"): then the error that I see is:
Unrecognized SSL message, plaintext connection?
If I use
("127.0.0.1", 8888, "http"): then the error that I see is: I/O
exception (org.apache.http.conn.UnsupportedSchemeException) caught
when processing request to
{tls}->http://127.0.0.1:8888->https://login.microsoftonline.com:443:
http protocol is not supported
I've tried other things (for example "localhost", "localhost.", and so on as suggested, and setting: -Djsse.enableSNIExtension=false), but so far no luck. Hopefully I am missing something obvious! Thanks for any help.
It turns out that the second configuration for the proxy is the correct one to use (using "http"), but what I had forgotten was that I was setting a ConnectionSocketFactoryRegistry in my HttpClientBuilder. I had not registered a ConnectionSocketFactory for "http" and this caused the second configuration to fail. I added this to the HttpClientBuilder, and now I can see traffic being captured by Fiddler:
connectionSocketFactoryRegistryBuilder.register("http", new PlainConnectionSocketFactory());
Trying to setup the production environment, https connection gets terminated at the NGINX level. In order to accommodate this we have enabled the X-Forwarded-Proto and X-Forwarded-For HTTP Headers in the request.
Also applied code changes as indicated in this following link,
https://shuaib.me/stormpath-ssl-termination/
Still receiving the error trying to access the stormpath built-in resources like https://<..>/oauth/token
{
"error": "invalid_request",
"message": "A secure HTTPS connection is required for token requests - this is a requirement of the OAuth 2 specification."
}
Any help is much appreciated! Thanks
What version of the Java SDK are you using?. The X-Forwarded-Proto header is supported out of the box since version 1.1.0. If you are using that version (or newer) then there must be other problem; the more info you can provide the better. Thanks
I would like to run a servlet in Jetty on an HTTPS site that requires a client certificate for only part of the site (specific URLs). For example:
https://example.com/someservlet/public - no client cert required
https://example.com/someservlet/protected - client cert required
In Apache I can accomplish this by specifying SSLVerifyClient require inside a <Directory> or <Location> which forces Apache to renegotiate the SSL connection after the request is made.
I do not want to run Jetty embedded in anything else, just standalone. Is this possible? Can a Servlet cause this directly somehow? Can it be done via configuration?
As far as I know you can only specify the SSL options on a per-port basis.
Even if you could the configuration you are trying to achieve is problematic, as it needs the SSLRenegotiation which has been changed about a year ago because of a security vulnerability. The new method for performing an SSLRenogitiation is therefore only supported by newer clients and sometimes even if it is supported it does not work because of bugs.
My recommendation for an easy workaround: Configure Jetty to listen on two SSL ports:
For example on 443 without HTTPS Client auth and on 8443 with HTTPS client auth required. Then make your protected servlet only available on 8443. This is not a nice solution but 100% robust, works with Jetty and with all clients.