i am here looking for resources or sample code for proxy server which can handle both http and https requests written in java. i searched google and found lot of data on how to handle http requests but not https.
A proxy cannot handle HTTPS and still provide end to end security. It is not possible using SSL/TLS which is what HTTPS is built on.
And a proxy that doesn't provide end-to-end security only has limited utility. So I'm not surprised you can't find an existing implementation.
I'm assuming that you want a normal HTTPS proxy here, that is a proxy that will not look into the request but merely relay all the traffic to the actual HTTPS server after the user-agent has used the HTTP CONNECT method. This is how HTTP proxy servers are normally used for HTTPS requests by browsers.
I haven't tried, but you could look at Jetty and its ConnectHandler.
Related
We have a number of Jetty http(s) servers, all behind different firewalls. The http servers are at customer sites (not under our control). Opening ports in the firewalls at these sites is not an option. Right now, these servers only serve JSON documents in response to REST requests.
We have web clients that need to interact with a given http server based on URL parameter or header value.
This seems like a straightforward proxy server situation - except for the firewall.
The approach that I'm currently trying is this:
Have a centralized proxy server (also Jetty based) that listens for inbound registration requests from the remote http servers. The registration request will take the form of a Websocket connection, which will be kept alive as long at the remote HTTP server is available. On registration, the Proxy Server will capture the websocket connection and map it to a resource identifier.
The web client will connect the proxy server, and include the resource identifier in the URL or header.
The proxy server will determine the appropriate Websocket to use, then pass the request on to the HTTP server. So the request and response will travel over the Websocket. Once the response is received, it will be returned to the web client.
So this is all well and good in theory - what I'm trying to figure out is:
a) is there a better way to achieve this?
b) What's the best way to set up Jetty to do the proxying on the HTTP Server end of the pipe?
I suppose that I could use Jetty's HttpClient, but what I really want to do is just pull the HTTP bytes from the websocket and pipe them directly into the Jetty connector. It doesn't seem to make sense to parse everything out. I suppose that I could open a regular socket connection on localhost, grab the bytes from the websocket, and do it that way - but it seems silly to route through the OS like that (I'm already operating inside the HTTP Server's Jetty environment).
It sure seems like this is the sort of problem that may have already been solved... Maybe by using a custom jetty Connection that works on WebSockets instead of TCP/IP sockets?
Update: as I've been playing with this, it seems like another tricky problem is how to handle request/response behavior (and ideally support muxing over the websocket channel). One potential resource that I've found is the WAMP sub-protocol for websockets: http://wamp.ws/
In case anyone else is looking for an answer to this one - RESTEasy has a mocking framework that can be used to invoke the REST functionality without running through a full servlet container: http://docs.jboss.org/resteasy/docs/2.0.0.GA/userguide/html_single/index.html#RESTEasy_Server-side_Mock_Framework
This, combined with WAMP, appears to do what I'm looking for.
I am trying to insert a header on all HTTPS requests that passes my proxy server.
I read that HTTPS request headers can not be tampered with.
Is there any way that I can insert a header on HTTPS requests?
Only if the proxy is the endpoint of the TLS connection with the client (i.e. the proxy decrypts the requests). Otherwise it won't actually see any HTTP headers at all, just TCP sessions containing encrypted data.
It is common for a reverse proxy local to the web server(s) to terminate TLS connections for performance - the Web server can dedicate more resources to serving applications because it doesn't need to spend CPU cycles on de/encryption.
If you do this, your server certificate must identify the proxy server, not the web server. Otherwise clients will get a warning about the identity of the server differing from the certificate identity.
If you don't control or trust the infrastructure between the proxy and your server you could re-encrypt the ongoing requests.
Unless you are willing to terminate the SSL connection on your proxy, no you can't. This is one of the main goals of SSL. The data and that includes the headers are encrypted from the client to the server.
If you decide to terminate / intercept the SSL connection on your proxy, the clients will get a warning in their browsers. If you are in charge of all the browsers of all the clients you would be able to install a custom certificate and the clients wouldn't get warned anymore.
I am new to enterprise design and client server technologies. Can you get Java RMI to work over the HTTPS port 443? I found some google links to stuff about "tunneling" and using HTTP port 80. Does that mean your RMI calls are getting wrapped and then unwrapped on both ends from HTTP protocol?
HTTP tunneling is built into RMI. There is no provision for HTTPS tunneling. It's not a good solution anyway as it imposes a 10x latency.
I need something that can intercept HTTP requests, extract their information (content, destination,...), perform various analysing tasks, and finally determine if the request should be dropped or not. Legal requests must than be forwarded to the application.
Basically, same functionalities as an IDS. But mind, I am NOT looking for a packet sniffer/filter. I want something that operates on the HTTP level.
It should be implementable on linux and run on the same system as the application(s) to which the requests are headed.
As a bonus, https could be supported (unencrypted viewing of the request content)
Try mitmproxy.
mitmproxy is an SSL-capable man-in-the-middle proxy for HTTP. It provides a console interface that allows traffic flows to be inspected and edited on the fly.
mitmdump is the command-line version of mitmproxy, with the same functionality but without the user interface. Think tcpdump for HTTP.
Features
Intercept HTTP requests and responses and modify them on the fly.
Save complete HTTP conversations for later replay and analysis.
Replay the client-side of an HTTP conversations.
Replay HTTP responses of a previously recorded server.
Reverse proxy mode to forward traffic to a specified server.
Make scripted changes to HTTP traffic using Python.
SSL certificates for interception are generated on the fly.
Screenshot
Example
I setup an example Jekyll Bootstrap app which is listening on port 4000 on my localhost. To intercept it's traffic I'd do the following:
% mitmproxy --mode reverse:http://localhost:4000 -p 4001
Then connect to my mitmproxy on port 4001 from my web browser (http://localhost:4001), resulting in this in mitmproxy:
You can then select any of the GET results to see the header info associated to that GET:
Try using
Burp Proxy, sounds like what you need.
I use Wire Shark for this, if you provide all the server certs it wil even decypt HTTPS.
You should be able to use squid proxy for that (https://en.wikipedia.org/wiki/Squid_(software))
You should learn more about ICAP, then make an ICAP server of your HTTP filtering application.
I ended up using LittleProxy because it is java, fast and lightweight.
It is a originally forward proxy, so I had to adjust it for reverse proxy functionality by forwarding every request to the local host.
I did this simply by editing the HttpRequestHandler. I hardcoded the host and port address.
hostAndPort = "localhost:80";
Why not Apache HTTP Client http://hc.apache.org/httpclient-legacy/tutorial.html
This simple lib is useful.
Environment: NetBeans 7.0.1, GlassFish 3.1
I want to sniff the communication between a WCF (.NET Framework 4) client and a Metro (2.1.1) web service, and then check the messages to be sure everything is how I want it to be. The web service uses Transport Security (SSL). I already know of Fiddler, but I did not have any success using it; only HTTPS browser traffic was visible. Is there a way to set up Fiddler to capture traffic between my services? Is there any other way?
UPDATE
I tried to start client and server on different machines and then use Fiddler, but no success. I tried Wireshark to capture traffic, but did not have any success running both on localhost. If i tried them on different machines, all I could see was TCP data exchange between the services.
On localhost + Wireshark, the packet counter next to the interfaces remained the same however I was calling the service lots of times.
UPDATE2
Tried to set up the proxy for NetBeans manually in the options, and programatically, but no success:
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8888");
System.setProperty("https.proxyHost", "localhost");
System.setProperty("https.proxyPort", "8888");
Tried to start GlassFish with these JVM options, but no :( :
<jvm-options>-Dhttp.proxyHost=localhost</jvm-options>
<jvm-options>-Dhttp.proxyPort=8888</jvm-options>
<jvm-options>-Dhttps.proxyHost=localhost</jvm-options>
<jvm-options>-Dhttps.proxyPort=8888</jvm-options>
As an alternative to Fiddler, if you control the WCF client, you could enable WCF Message Logging, and it will save all the unencrypted SOAP messages to a trace log. The logging can be enabled in the app.config file, so you don't even have to rebuild the app to enable or disable the logging.
you can setup metro to dump SOAP messages, info here. personally, i use charles proxy to watch soap exchanges. you configure the java proxy using the system properties in your "update2", works very well.
Use http://portswigger.net/burp/
It has a proxy. The proxy can be used for viewing the http traffic. It can also display SSL traffic by generating self signed certificate on the fly. You need to import the generated certificate into the java key store at the jax-ws client. Ensure that you have enabled "Support invisible proxy for non-proxy aware clients"
I often use commview to monitor traffic over the local loopback adapter. One of the few tools allowing traffic capture when both your client and service are on the same computer.
You can download a trail at http://www.tamos.com/products/commview/ and see if it works for you.
May be WebScarab is what you need. There are many manuals for sniffing ssl traffic, for instance that.
One approach I've tried successfully, is to make sure SSLv2 is used (as opposed to SSLv3), and then use Wireshark as described on the SSL page on the Wikishark Wiki, but better on Citrix support page "How to decrypt SSL and TLS traffic using Wireshark ". This works by giving Wireshark the private key of the server's SSL certificate, so that it can decrypt the conversation.
To force SSLv2 in your scenario, it seems sufficient to set -Dhttps.protocols=SSLv2Hello on your server-side JVM, but I googled that together. (See, e.g., the "Why do I get a javax.net.ssl.SSLException" question on the Java 1.4.2 Troubleshooting page, and the part on https.protocols in the JavaTM Secure Socket Extension (JSSE) Reference Guide.) I haven't done this part myself, and I can't seem to find clear documentation on this point.
(P.S. In my case, when I was decrypting .NET-to-.NET SSL traffic, I thought it was the switch back to SSLv2 which made the traffic readable by Wireshark. However, this blog post suggests that I was switching at the same time from a Diffie-Hellman cipher to a non-DH one.)