So I've figured out how to access a non-secure webpage in java. However, I do not know how to access a webpage that fist requires you log in on the login page to view. Could anyone help me with this?
Your question seems similar to this one which points out that not only will you need to send along credentials in your HTTP request (as post or get parameters) but also you may need to consider handling cookies.
How do I login and download a file from a https web page from Java?
Related
So, I have my script page and I want to get an output that generates on this page. It could've been done with default http client but the problem is that google requires authorization to access this page.
So how can I do it right?
I also asked about this before but now I realize that my question wasn't clear.
It is possible to post web app page to be accessible by "anyone even anonymous". So no auth is needed
We are integrating our Silverlight based Desktop Application ( which talks to Servlet based backend solution ) with Facebook login. We are able to redirect to Facebook login page using below url
https://www.facebook.com/dialog/oauth?client_id=<client id>&redirect_uri=http://localhost.example.com/fbsignin&scope=email
But the issue here is that somehow Facebook is redirecting Infinite/multiple times (with different-2 code value ) to our redirect url.
I checked on internet for same, and I think it is kind of known problem but all solutions provided only ask about updating PHP files or it SDK, but I am working on Servlet based bacend solution and not able to find any suitable solution to stop this multiple redirect.
Any Suggestion?
thanks
I think I had a similar issue.
When you get a request with the 'code' parameter, then you will want to send that back to FB. Doing so will allow you to get an 'access_token' in return.
Relevant documentation can be read at this link: Login Flow
And for clarity you would want to exchange the 'code' parameter for and 'access_token' by this GET request.
GET https://graph.facebook.com/oauth/access_token?
client_id={app-id}
&redirect_uri={redirect-uri}
&client_secret={app-secret}
&code={code-parameter}
In general, your question is similar to this question:
Get Facebook OAuth token from code parameter
I'm trying to get data from a website, but first I need to log in to the site using java. The script worked until now, but now the site installed an anti bot system. Until now the procedure was simple, I've created a HttpStreamWriter and submitted my details to the login.php page, then get the cookies and later, when I want to get data from the site, I resubmit the cookie from the login.php page, but now there is a problem: an anti bot system:
I'm not sure, but I think this is the system:
https://github.com/yuri-gushin/Roboo/blob/master/Roboo.pm
The anti bot system creates a cookie, called anti-bot and I can't access the page without that cookie, the problem is that the cookie is generated by a flash application only after the page loads so I can't get the cookie from the page?
Any ideas how to "hack" this ? Thanks!
Your need is about cookie extraction, here is how to do or on the oracle site
That is you need to connect to the site, browse the headers until Set-Cookie. Having the correct http header, you'll be able to parse it very easily.
After what, you'll have to set it back to your further request.
Edit
Flash cookie or Local Shared Object are stored in AMF. AMF wil be used store anything, the problem with your use case is that you don't know which value (or maybe class instances) have been serialized...
However you could (it will take time, at least for include all necessary libs) try with the AMFConnection to retrieve information. But I won't bet on that.
Could you contact the webmaster to have some information about that ? Or doens't this website any login api ?
I'm basically trying to use Java to read some data from my school's website (homework assignments, which lessons I have and when, etc.) for personal use. However, my school requires one to be logged in to access this information.
Could anyone point me in the right direction for logging in with code and accessing this information?
Thanks,
Mike.
The Apache has a API for http client simulating.
Link: http://hc.apache.org/httpcomponents-client-ga/
You need to find out how the server handles logins. What authenticationer it used: cookies, url session id, etc.
Then you can send the server a login http form submit (as you would have done manually) and save the authenticationer.
With the authenticationer you can then access the secured sites.
I would use HtmlUnit, which is a programmatic web browser.
You tell it to load the authentication page, to fill the form with your credentials and to click on the submit button. The you click on links, just as you would do it with a real browser, but programmatically, using Java instructions.
And it even supports JavaScript, if necessary.
Is there any web language that allows the client itself to create HTTP posts to external sites.
I know that JavaScript does this with XMLHttpRequest, but it does not allow cross-domain posting, unless the recipient domain wants to allow the sending domain.
I want to post data to an external site (that I don't control) and have the request be authenticated with what the client's browser already has (cookies, etc).
Is this possible? I tried cURL but it seems to make a server HTTP post, not a client HTTP post.
Edit:
A bit more insight of what I am trying to do:
I am trying to POST JSON to the website using the user's session (I said cookies but I believe they are PHP sessions, which I guess I still consider cookies).
The website does NOT check the referral (poor security #1)
I can execute javascript and html on the webpage using my personal homepage (poor security #2)
The JSON code will still work even if the content-type is form (poor security #3)
There is no security checking at all, just PHP session checking.
The form idea is wonderful and it works. The probably again is that its JSON. So having sent postdata as foo={"test":"123", "test2":"456"} the whole foo= part messes it up. Plus forms seem to turn JSON into form encoding, so its sending:
foo=%7B%22
test%22%3A+%22
123%22%2C+%22
test2%22%3A+%22
456%22%7D
when i need it to send;
{"test":"123", "test2":"456"}
So with everything known, is there a better chance of sending JSON or not?
I don't think so: You won't get hold of the user's auth cookies on the third party site from server side (because of the Single Origin Policy) and you can't make Ajax requests to the third party site.
The best you can do is probably create a <form> (maybe in an <iframe>), point it to the third party site, populate it with data, and have the user submit it (or auto-submit it). You will not be able to get hold of the request results programmatically (again because of the Single Origin Policy), but maybe it'll do - you can still show the request results to the user.
I think for obvious reasons this is not allowed. If this was allowed what would stop a malicious person from posting form data from a person's browser to any number of sites in some hidden iframe or popup window.
If this is a design of your application you need to rethink what you are trying to accomplish.
EDIT: As #Pekka was pointing out I know you can submit a form to a remote site using typical form submits. I was referring to using some client side ajax solution. Sorry for the confusion.
You should follow the way OpenID and other single-sign-on system works. How openID works is your website POSTs some token to openID service and in return gets authentication result. Refer How Does it Work? section here
Yes, you can use a special flash library that supports cross-domain calls: YUI connection manager
Added: not sure about the cookie authentication issue though...
The client cannot post to an external site directly; it's a breach of basic cross-domain security models. The exception is accessing javascript with JSONP. What you describe would require access to a user's cookies for another website, which is impossible as the browser only allows cookie access within the same domain/path.
You would need to use a server-side proxy to make cross-domain requests, but you still cannot access external cookies: http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html