Before jumping into the problem I would like to share that what is the structure of my network. I have 3 computers where 1 is acting as a server and rest of the machines are acting as clients. Server is connected to internet and sharing it with other two machines.
I have been trying to log all the HTTPS request on my server by using Jnetpcap library. I have logged all the requests from my HTTP very easily. But, unable to get grip on HTTPS requests.
My PacketHandler snippet is :
PcapPacketHandler<String> jpacketHandler = new PcapPacketHandler<String>() {
public void nextPacket(PcapPacket packet, String user) {
System.out.printf("Received packet at %s caplen=%-4d len=%-4d %s\n",
new Date(packet.getCaptureHeader().timestampInMillis()),
packet.getCaptureHeader().caplen(), // Length actually captured
packet.getCaptureHeader().wirelen(), // Original length
user // User supplied object
);
}
};
In response it returns
youtube.com.https ......Blah Blah
But, didn't return the youtube video ID that I was willing to capture.
It's the "S" in "HTTPS" that's the problem.
HTTP requests and replies are plain text. HTTPS is HTTP-over-{SSL,TLS}, and the plain text being sent in those requests and replies is almost always encrypted. You'd have to decrypt it, which is Not Easy - Wireshark has code that can sometimes do it, if you provide the right information, but, as the Wireshark Wiki entry for SSL indicates, and as a look at the Wireshark code to do the decryption will show, there's a significant amount of work involved.
Related
I would like to create a web application that is able to "ping" the client once the client has accessed certain URL (e.g. www.example.com/ping/hello) in order to get the round trip time between server and client. And by "ping" request i mean a simple request with a timestamp from server and client sends back response with its timestamp. I was hoping for this activity to be done with a single URL if possible.
The flow is something like this:
Client goes to the URL
Server sends the response to the client with its timestamp
Client then sends another response to server with new timestamp
Server finally concludes the connection with 200 OK
So far I've only been able to do the first and second steps but not sure how to ensure client to go to the same URL again without back to the first step.
My server code is something like this:
#GET
#Path("/helloping")
public Response getPingServerClient(#Context HttpServletRequest req) {
String result = Long.toString(System.currentTimeMillis());
return Response.status(200).entity(result).build();
//the code to receive the response from client containing timestamp
}
Is there a way to do that?
There are two client to server calls. You'll have to figure out a way to differentiate between these two calls.
I can think of 3 options for this purpose:
HTTP header
Query parameter in GET request
POST request with a marker to differentiate the two calls
The request/response flow will be something like this:
Client -> Server : Request
Server -> Client : Response with timestamp t1
Client -> Server : Request with timestamp t2 and the above mentioned marker
Server -> Client : Response 200
In this approach, you'll have to write custom code at both server and client side to handle the mentioned logic.
I'm not a fan of what you are proposing because you're basically forcing the client to setup up code to effectively become a server, itself. This is inconvenient for the client.
Instead, consider a ping-pong approach where the client first calls the server's ping endpoint, which returns the server's timestamp. As soon as the client obtains the server's ping response, the client is instructed to call a second pong method, which accepts the new timestamp.
It's easier and simpler to require the client to call web service methods than it is to force to client to become a pseudo server. Hence the recommendation.
Suppose I have a resource that is hosted on a websocket server, and I wish to validate the response status code 101, after the connection is upgraded, how and which libraries to use.
Im currently looking at jayway response library, when I connect to a websocket resource, initially it sends a 200 and then upgrades to 101. So code returns 200, I would like to know how this library can be used for websocket validation.
Sample code is :
String response =given().get()("https://www.ws.com:444/examples/websocket/snake.xhtml")
.getResponse().asString();
This returns a 200, but does not return the next response. I might sound a little rusty here, would appreciate if you folks have any idea how to extend it to get status 101 or if you have any other suggestions.
Also, this requirement needs to be extended to handle redirection 30x status codes as well, assume a resource is protected behind an access gateway, when a request comes for this protected resource, the gateway forwards it to identity store for verifying the user, at this time a 302 is returned, once verified, request is sent back to access gateway from where user is able to access websocket resource.
A common SO question, but no specific solid answers.
My setup:
I have a website running on Classic ASP with backed DB. Unfortunately, no SSL Certs are available
I have an Android application that will send a Google Volley to request data from the site using a bespoke but simple API
Currently:
I am still in testing, privately, so currently I just access the site as such:
On the app, the user enters a UserId and Password once.
User navigates to a Fragment which is associated with a specific ASP Page which will return some data
A Volley is sent to /mysite.com/_api/apage.asp?m=md5hashhereabcdefghijk
The server searches user records for a matching hash (built on UserID+SALT+pass). On matching record, it uses the found userid as the User's ID
apage.asp does some sql queries and returns a JSON object
app receives this JSON response and parses.
The problem:
Anyone packet sniffing, or MITM, would be able to plainly see the URLs being accessed (and server responses) and be able to replicate the query via their browser. This is what I'm trying to stop. Any SALTs or secret keys in the app would be easily seen by decompiling the APK.
Issues:
I've read all sorts of different solutions, but none of which really fit my environment. I can't use ASP session variables (RESTful being stateless), I cant use HTTPS(SSL/TLS) as there are no Certs on the Server. I can't use an App-based password as this can be decompiled and easily seen.
I appreciate that you will never get something 100% secure, but can only make people disinterested in hacking a system, or not make it worth while.
Proposed solution:
I want some feedback/thoughts on the following proposed method:
Each request will have its own handshake to authenticate the app
This will go as such:
User opens app for the first time and enters UserID/Password. This will remain with the app until it is uninstalled (or logged out), but I intend to keep the user's app logged in
User navigates in the app to a Fragment that corresponds with a specific page on the server
Volley is sent with :
UserAgent HTTP header 'some value'
generate the same authentication hash for (userid+salt+pass)
encrypt this hash with a public key
one query string /apage.asp?q=abcdefghijk.... sent to server
server decrypts using its private key
server checks this hash as I do currently.
page returns plaintext JSON values (not encrypted)
The same problem happens here whereby a MITM or sniffer could replicate the URL and get the same information back
A Second Proposed Solution:
Would it be better with every request actually starting with a whole handshake?
App sends volley to server requesting a handshake (HELO)
Server gross error check with UserAgent HTTP Header
Server logs the timestamp and IP of the request and generates a unique random code
App receives this code and builds a new hash using that unique code as a Salt
App sends second volley to /apage.asp?m=MD5(UserID+UniqueCode+Password)
Server Gross error check with originating IP, timestamp+-tolerance (30 seconds between the two requests?), UserAgent Request Header.
APage.asp uses this hash to authenticate the request, providing previous steps have successfully passed.
APage.asp returns a JSON object, as requested.
Server flags the log of originating IP/timestamp as EXPIRED, or, just remove the record.
This initial step would make it a lot harder for a sniffer or MITM to replicate the URL calls, as A) Each request would have a randomly returned code B) each code/hash combo can only be used once.
The only thing I can think of is someone decompiles the App, and sees the method of handshake, so could try to replicate this. However, the username/password hash would never match as that is the only thing they cannot get from the hash (as it is salted with the random code)
Thoughts? Could it be improved with some RSA public/private key cryptography? Could I generate my querystring apage.asp?m=abcdeghi..., Generate an MD5 Hash of that, append onto the end, then encrypt before sending?
Many thanks in advance
I have made two J2EE applications where in one servlet in ProjectX is performing a sendRedirect to another servlet of ProjectY via https protocol.
Code is something like
response.sendRedirect("https://ip:8443/ProjectY/servletY?id=123");
In ProjectY,
SerletY is having code as
PrintWriter pw = response.getWriter();
pw.print("Passed id is ID = " + request.getParameter("id"));
My Query is ,
since the data sent accross the network is ideally encrypted when using https, why am I able to see the url of the browser after redirecting to ServletY as
"https://ip:8443/ProjectY/servletY?id=123"
I have hidden the parameter using POST method , but my question is , is it actually encrypting data while sending from ProjectX(which was in http) to ProjectY (which is https call) ?
Thanks for you support.!!!
What's happening
There is no POST request involved.
The user opens ProjectX' site in the browser
It will respond with a HTTP 302 response because of your response.sendRedirect.
The user's browser will take the Location of the response and open it
Thus the user's browser establishes an TLS connection to ip:8443
After the TLS channel is opened, it will send a GET /ProjectY/servletY?id=123 HTTP/1.1
ProjectY will respond over the secure TLS channel.
Observations
If you call ProjectX in step 1 via plain HTTP, then the 302 response won't be encrypted and everybody who has access to your connection can see the id.
The user's browser will always see the id because it needs to follow the redirect in step 3.
The user will see the id in the address bar because the browser will show its new location.
When calling ProjectY, the id is protected because it is only sent via the TLS channel.
My Android App has an App Widget associated with it which is updated every 10 minutes on an Android Device. These updates send HTTP requests for data to the servers and parse the server response and updates the App as required.
As of now if you ping that URL from the browsers on your laptop or PC the server will respond and update whatever is required in the database on the server.
What I want to do is when the HTTP requests are received at the server, I want to identify if the request came from my Android App from an Android device and then respond with the data. I would like to change the code in the PHPs on the server in a way that they would display or redirect to some page if the HTTP request came from a browser or anything else except for my Android App.
Typical HTTP requests from the Apps are like http://example.com/abc.php?usera=abc&datab=xyz
I don't want to respond to this URL in the same way if it is coming from anywhere else except from the Android App. Is this possible? What would be a good way to achieve this..
Thanks for your help.
You can add a signature to the request and then check it on server-side.
Just take the query and add one secret word at the end, then make a MD5 of it that you can send as an header (or use as a user-agent). And on the server you do the same and check if the checksum is the same.
To make it a bit safer you can make a timestamp so the request only will be valid for a short time.
Make your query look like http://example.com/abc.php?usera=abc&datab=xyz×tamp=123456789 where timestamp is the current time (in unix time stamp) and add this in your app:
public static String makeCheck(String url)
{
URL u=new URL(url);
MessageDigest md = MessageDigest.getInstance("MD5");
u.getQuery();
md.update(u.getQuery().getBytes());
BigInteger bn = new BigInteger(1,md.digest("A_SECRET_WORD".getBytes()));
return bn.toString(16);
}
And when you need to add the header use something like:
request.addHeader("X-CHECKSUM", makeCheck(url) );
Then on your server you can use:
if (md5($_SERVER['QUERY_STRING']."A_SECRET_WORD")!=$_SERVER['X-CHECKSUM']) {
// Wrong checksum
}
$timediff=60;
if ( $_GET['timestamp']>(time()+$timediff) || $_GET['timestamp']<(time()-$timediff) ) {
// Bad timestamp
}
Remember to be a bit slack on the timestamp since your servers clock and the phones clock can be off sync a bit.
The typical way of doing this is using the User-Agent header in the HTTP request. if the request comes from the standard browser, it will uniquely identify both the hardware and software. For example a Nexus One running Froyo will have the following User-Agent:
Mozilla/5.0 (Linux; U; Android 2.2; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
However, if you're using HttpClient to make requests from your app, you can customise the User-Agent header that HttpClient uses as demonstrated in this answer: Android HTTP User Agent.
On the server-side you can use a regex match on the user-Agent header to determine whether a request has originated from your Android app, and send the appropriate response.
If the actual request is the same (for instance, you are not able to add a POST or GET variable to actively identify your request), you'd have to rely on other things, like user-agent.
While you can set them according to your wishes in your app (also see #mark_bakker nd #mark_allison 's answers), you should be aware that there are ways to mess with this, so don't use it for stuff you really don't want other users to see.
An android user could in theory change the user_agent string between the request leaving your app and the request leaving his/her network. So don't use it for "Android users didn't pay, so should not see this/that info" applications
The other way around, non-android users can change their user-agent too obviously, so if you have content only your paying android-users should see, they might fake the string.
In the end it might be better to just change your request if you can: you want a different reply, you should do a different request is my opinion.
When you create the HttpClient in android you can set the following
client.getParams().setParameter(CoreProtocolPNames.USER_AGENT, "MY Android device identifier");
This set the USER_AGENT for each http request send to your server. On your server you can retrieve the USER_AGENT to determine that the request came from your android device