Is there any web language that allows the client itself to create HTTP posts to external sites.
I know that JavaScript does this with XMLHttpRequest, but it does not allow cross-domain posting, unless the recipient domain wants to allow the sending domain.
I want to post data to an external site (that I don't control) and have the request be authenticated with what the client's browser already has (cookies, etc).
Is this possible? I tried cURL but it seems to make a server HTTP post, not a client HTTP post.
Edit:
A bit more insight of what I am trying to do:
I am trying to POST JSON to the website using the user's session (I said cookies but I believe they are PHP sessions, which I guess I still consider cookies).
The website does NOT check the referral (poor security #1)
I can execute javascript and html on the webpage using my personal homepage (poor security #2)
The JSON code will still work even if the content-type is form (poor security #3)
There is no security checking at all, just PHP session checking.
The form idea is wonderful and it works. The probably again is that its JSON. So having sent postdata as foo={"test":"123", "test2":"456"} the whole foo= part messes it up. Plus forms seem to turn JSON into form encoding, so its sending:
foo=%7B%22
test%22%3A+%22
123%22%2C+%22
test2%22%3A+%22
456%22%7D
when i need it to send;
{"test":"123", "test2":"456"}
So with everything known, is there a better chance of sending JSON or not?
I don't think so: You won't get hold of the user's auth cookies on the third party site from server side (because of the Single Origin Policy) and you can't make Ajax requests to the third party site.
The best you can do is probably create a <form> (maybe in an <iframe>), point it to the third party site, populate it with data, and have the user submit it (or auto-submit it). You will not be able to get hold of the request results programmatically (again because of the Single Origin Policy), but maybe it'll do - you can still show the request results to the user.
I think for obvious reasons this is not allowed. If this was allowed what would stop a malicious person from posting form data from a person's browser to any number of sites in some hidden iframe or popup window.
If this is a design of your application you need to rethink what you are trying to accomplish.
EDIT: As #Pekka was pointing out I know you can submit a form to a remote site using typical form submits. I was referring to using some client side ajax solution. Sorry for the confusion.
You should follow the way OpenID and other single-sign-on system works. How openID works is your website POSTs some token to openID service and in return gets authentication result. Refer How Does it Work? section here
Yes, you can use a special flash library that supports cross-domain calls: YUI connection manager
Added: not sure about the cookie authentication issue though...
The client cannot post to an external site directly; it's a breach of basic cross-domain security models. The exception is accessing javascript with JSONP. What you describe would require access to a user's cookies for another website, which is impossible as the browser only allows cookie access within the same domain/path.
You would need to use a server-side proxy to make cross-domain requests, but you still cannot access external cookies: http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html
Related
I promised to do something, but my managers want assurance that it can be done. What seems like a simple task is surprisingly hard to find example of. It consists of:
The browser asks a Java servlet for access to a remote site.
The servlet opens a HttpUrlConnection to that remote site.
The servlet sends a login request to that remote site.
The servlet receives a login form.
The servlet populates the login form from user session data, then
POSTs it.
The servlet receives the "welcome" from the remote site, complete
with headers and cookies.
The request of step 1 is finally filled by the content, headers, and cookies from step 6.
I'm having trouble getting code examples, leading me to think that what I want to do can't be accomplished. Can someone point me to an example?
Thanks,
Jerome.
The main problem is the domain of the session cookie.
More accurately, the question is what you're planning next after step 7.
Say your servlet is at "https://mycompany.com/loginServlet", and the remote site is a toy store "https://toys.com". Were you hoping that after step 7, the browser could be redirected to "https://toys.com" and get a proper service befitting a logged-in user? This won't work because when the browser received a session cookie from "mycompany.com" (in step7), this cookie will be attached to further requests for "mycompany.com" - but not to "toys.com", it's a security feature for excellent reason.
In short, this approach might work if both the servlet and the toys site are on the same domain. Though it makes me a bit uncomfortable in terms of security and password management, but never mind.
Obviously this approach would also work in the (unrealistic) case of letting your server proxy all further requests from the browser to "toys.com", but that's unrealistic for a typical commercial site (due to relative/absolute links, ajaxes etc.).
If it's different domains, then I second the wise comment made by "Rahul B" above: please look into SSO, and/or check with "toys.com" what login mechanisms they offer - maybe they already have something like "login with google" that's convenient enough.
what I want to do is to build a web application(proxy) that user use to request the webpage he want and
my application forward the request to the main server,
modify HTML code,
send to the client the modified one.
The question now is
How to keep my application between the client and main serevr
(for example when the user click any link inside the modified page-
ajax request - submit forms - and so on)
in another words
How to grantee that any request (after the first URL request) from the client sent to my proxy and any response come first to my proxy
The question is: Why do you need a proxy? Why do you want to build it - why not use already existing one like HAProxy ?
EDIT: sorry, I didn't read your whole post correctly. You can start with:
http://www.jtmelton.com/2007/11/27/a-simple-multi-threaded-java-http-proxy-server/
If the user is willing to, or can be forced1 to configure his clients (e.g. web browser) to use a web proxy, then your problem is already solved. Another way to do this (assuming that the user is cooperative) is to get them to install a trusted browser plugin that dynamically routes selected URLs through your proxy. But you can't do this using an untrusted webapp: the Browser sandbox won't (shouldn't) let you.
Doing it without the user's knowledge and consent requires some kind of interference at the network level. For example, a "smart" switch could recognizes TCP/IP packets on port 80 and deliberately route them to your proxy instead of the IP address that the client's browser specifies. This kind of thing is known as "deep packet inspection". It would be very difficult to implement yourself, and it requires significant compute power in your network switch if you are going to achieve high network rates through the switch.
The second problem is that making meaningful on-the-fly modifications to arbitrary HTML + Javascript responses is a really difficult problem.
The final problem is that this is only going to work with HTTP. HTTPS protects against "man in the middle" attacks ... such as this ... that monitor or interfere with the requests and responses. The best you could hope to do would be to capture the encrypted traffic between the client and the server.
1 - The normal way to force a user to do this is to implement a firewall that blocks all outgoing HTTP connections apart from those made via your proxy.
UPDATE
The problem now what should I change in the html code to enforce client to request any thing from my app --- for example for link href attribute may be www.aaaa.com?url=www.google.com but for ajax and form what I should do?
Like I said, it is a difficult task. You have to deal with the following problems:
Finding and updating absolute URLs in the HTML. (Not hard)
Finding and dealing with the base URL (if any). (Not hard)
Dealing with the URLs that you don't want to change; e.g. links to CSS, javascript (maybe), etc. (Harder ...)
Dealing with HTML that is syntactically invalid ... but not to the extent that the browser can't cope. (Hard)
Dealing with cross-site issues. (Uncertain ...)
Dealing with URLs in requests being made by javascript embedded in / called from the page. This is extremely difficult, given the myriad ways that javascript could assemble the URL.
Dealing with HTTPS. (Impossible to do securely; i.e. without the user not trusting the proxy to see private info such as passwords, credit card numbers, etc that are normally sent securely.)
and so on.
I have developed myself in the last few months about web development in java (servlets and jsp). I am developing a web server, which is mainly serving for an application. Actually it is running on google app engine. My concern is, although I am using SSL connections, sending parameters in the URL (e.g. https://www.xyz.com/server?password=1234&username=uname) may not be secure. Should I use another way or is it really secure? I don't know if this url is delivered as plaint text as whole (with the parameters)?
Any help would be appreciated!
Everything is encrypted, including the URL and its parameters. You might still avoid them because they might be stored in server-side logs and in the browser history, though.
Your problem seems to go further than Web Server and Google App Engine.
Sending a password through a web form to your server is a very common security issue. See this SO threads:
Is either GET or POST more secure than the other? (meaningly, POST will simply not display the parameter in the URL so this is not enough)
Are https URLs encrypted? (describes something similar to what you intend to do)
The complete HTTP request including the request line is encrypted inside SSL.
Example http request for the above URL which will all be contained within the SSL tunnel:
GET /server?password=1234&username=uname HTTP/1.1
Host: www.xyz.com
...
It is possible though that your application will log the requested URL, as this contains the users password this may not be OK.
Well, apart from the issues to do with logging and visibility of URLs (i.e., what happens before and after the secure communication) both GET and POST are equally secure; there is very little information that is exchanged before the encrypted channel is established, not even the first line of the HTTP protocol. But that doesn't mean you should use GET for this.
The issue is that logging in is changing the state of the server and should not be repeated without the user getting properly notified that this is happening (to prevent surprises with Javascript). The state that is being changed is of the user session information on the server, because what logging in does is associate a verified identity with that session. Because it is a (significant) change of state, the operation should not be done by GET; while you could do it by PUT technically, POST is better because of the non-idempotency assumptions associated with it (which in turn encourages browsers to pop up a warning dialog).
I'm basically trying to use Java to read some data from my school's website (homework assignments, which lessons I have and when, etc.) for personal use. However, my school requires one to be logged in to access this information.
Could anyone point me in the right direction for logging in with code and accessing this information?
Thanks,
Mike.
The Apache has a API for http client simulating.
Link: http://hc.apache.org/httpcomponents-client-ga/
You need to find out how the server handles logins. What authenticationer it used: cookies, url session id, etc.
Then you can send the server a login http form submit (as you would have done manually) and save the authenticationer.
With the authenticationer you can then access the secured sites.
I would use HtmlUnit, which is a programmatic web browser.
You tell it to load the authentication page, to fill the form with your credentials and to click on the submit button. The you click on links, just as you would do it with a real browser, but programmatically, using Java instructions.
And it even supports JavaScript, if necessary.
This problem relates to the Restlet framework and Java
When a client wants to discover the resources available on a server - they must send an HTTP request with OPTIONS as the request type. This is fine I guess for non human readable clients - i.e. in code rather than a browser.
The problem I see here is - browsers (human readable) using GET, will NOT be able to quickly discover the resources available to them and find out some extra help documentation etc - because they do not use OPTIONS as a request type.
Is there a way to make a browser send an OPTIONS/GET request so the server can fire back formatted XML to the client (as this is what happens in Restlet - i.e. the server response is to send all information back as XML), and display this in the browser?
Or have I got my thinking all wrong - i.e. the point of OPTIONS is that is meant to be used inside a client's code and not meant to be read via a browser.
Use the TunnelService (which by default is already enabled) and simply add the method=OPTIONS query parameter to your URL.
(The Restlet FAQ Q19 is a similar question.)
I think OPTIONS is not designed to be 'user-visible'.
How would you dispatch an OPTIONS request from the browser ? (note that the form element only allows GET and POST).
You could send it using XmlHttpRequest and then get back XML in your Javascript callback and render it appropriately. But I'm not convinced this is something that your user should really know about!