I am writing a Java application that has a log in screen. Ideally, I would like to take the user supplied data (name, password), and submit it to an ASP form that can verify their credentials. I do not own the ASP form, I can only access the URL. I also do not want the user to be entering their credentials straight into the web form. They would enter their credentials into my program, and my program would put the data into the form and submit, and allow/deny the user based on the response.
Of course, the submit button on the ASP form is a POST request. However, constructing the URL (...login?username=name&password=pass) does not work, as the form must be submitted via the button with the text boxes filled in.
I have tried two approaches:
Using Java's URLConnection class. This does not seem to work because the form submitting is limited to the method I mentioned above, which is constructing the URL.
Using Javascript to access and edit the elements on the page. This has not worked either, because the Javascript is being run from my program, which is not a web browser, and therefore has no access to the 'document' or 'window' commonly used.
Other potential solutions I can think of:
Opening a browser to the login page but not giving it focus, running a script to fill out and submit the form, parsing the response, and then closing the browser. This would not involve the user at all, except for the input into the login page in my Java program.
Using a 3rd party Java library (suggestions? references to tutorials?).
Embedding the URL into my login screen (any help in this regard would be appreciated).
The things that cannot be changed are that my program is in Java, and that the login URL is an ASP form that hides the POST data from the URL.
Let me know if anything needs clarification. Any help is welcome.
try htmlunit, although it was designed for testing it would be ideal for this. You can use it in conjunction with Selenium webdriver
Why don't you open up your ASP form in an IFrame using javascript populate all the fields & then post it.
This should solve your problem.
Sfk is correct, i had a similar problem and manage to fill the form and submit it with Htmlunit .#sfk many thanks, you put me in the right path.
So with htmlunit
//for chrome simulation
WebClient webClient = new WebClient(BrowserVersion.CHROME_16);
//has getting an error from [http://www.google-analytics.com/ga.js with javascript on.
webClient.setJavaScriptEnabled(false);
HtmlPage page = webClient.getPage("http://yourtargetpage/Default.aspx");
//get the form by name, check page source for name
HtmlForm form = page.getFormByName("aspnetForm");
HtmlPasswordInput inputPass = form.getInputByName("your input password text field name");
HtmlTextInput userName = form.getInputByName("your input user text field name");
HtmlSubmitInput button=form.getInputByName("your target submit button");
//set username and password
userName.setText("myuser");
inputPass.setText("mypassword");
//click the submit button and get the returned page
HtmlPage page2 = button.click();
That´s it.. you got the reply page and sent the information on the fields..you can now parse the page and get the site response..
Related
I am in bit of a delicate situation here. In my organization we design stock management systems and it is a web application based on JSP pages and servlets which handles them.
I have been asked to fix a specific problem. We have a JSP page with an HTML form table where there are stock details. When user enters the details manually and submit the form, stock details updated in the database and it works fine.
Problem is this : When the user press the browser's back button, user can come to the previous page where he submitted the details. And when the user submit this, data is saved once more to the database.I need to prevent this behaviour.(Something likeclear and reload the page.)
Things I did so far : clear the browser cache.Code works fine but not the expected result.
Unfortunately I cannot share the code due to company regulations. What I need is a help to prevent this behaviour or a workaround.
Thanks in advance..
You can use a javascript function with the help of a hidden attribute to reload the web page. When the user press the back button,based on the value of the hidden attribute, page will be reloaded without loading the cached page.
Your approach of clearing cache is correct. Coupled with that, you can use this approach.
<input type="hidden" id="refreshed" value="no">
<script type="text/javascript">
onload=function(){
var e=document.getElementById("refreshed");
if(e.value=="no")e.value="yes";
else{e.value="no";location.reload();}
}
</script>
One drawback of this approach is if your clients' browsers have disabled JS, this will not work.Otherwise it should work.
When the user press the browser's back button, user can come to the
previous page where he submitted the details. And when the user submit
this, data is saved once more to the database.
According to how you described it, that is based on a doGet request. Which means every time you visit that URL, it will send the request with whatever parameters were added.
As someone already mentioned, if you switch the form to a post method and switch the Servlet to a doPost, you won't have this issue anymore.
Alternatively you can circumvent this with a javascript solution. Here are some options:
You can check if the user clicked the back button, disable form if true.
Another way is by storing a cookie which you check on page load, if it exists you can disable the form.
You can use this code also
$(document).ready(function() {
function disableBack() { window.history.forward() }
window.onload = disableBack();
window.onpageshow = function(evt) { if (evt.persisted) disableBack() }
});
You must use a Post-Redirect-Get pattern: https://en.m.wikipedia.org/wiki/Post/Redirect/Get.
Actually, every use of standard HTML forms with method="post" should be implemented with that pattern. It doesn't have any use for AJAX-posted forms, which actually could be another solution but will require more work and probably some architectural changes.
I had this same problem while building a django web app, and my solution was to not allow caching of the html that contains the form. In your request handler, do not allow the browser to cache the page. This will force the browser to get the page fresh from the document.
Which, in this case, you can just verify in your request handler if the requested form has already been submitted.
My code for reference:
from django.views.decorators.cache import never_cache
#never_cache
def GetForm(request, pk):
# Logic #
if (IsFormCompleted(pk)):
# Handle request #
Here is a solution.
give a random id in a hidden field on the form. Then on the server side, if the user resubmit, check if the random id already on the database. If so, redirect user.
I'm trying to get the textbox with u_0_1e as id, from the page wall but HtmlUnit does not find anything. The last line prints null.
Here's the code:
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(java.util.logging.Level.OFF);
WebClient client = new WebClient(BrowserVersion.CHROME);
JavaScriptEngine engine = new JavaScriptEngine(client);
client.setJavaScriptEngine(engine);
HtmlPage home = client.getPage("https://www.facebook.com/login.php");
HtmlSubmitInput login = (HtmlSubmitInput) home.getElementById("u_0_1");
HtmlTextInput name = (HtmlTextInput) home.getElementById("email");
HtmlPasswordInput pass = (HtmlPasswordInput) home.getElementById("pass");
name.setValueAttribute("myname");
pass.setValueAttribute("mypass");
HtmlPage page = login.click();
HtmlPage wall = client.getPage("https://www.facebook.com/");
System.out.println(wall.getElementById("u_0_1e"));
I have some comments about your issue.
First of all, you have disabled HtmlUnit's logging. So if you have any JavaScript issue then you are not going to see it. If you are actually getting a JavaScript error then the JavaScript code won't be fully executed. If the element you're trying to fetch was dynamically fetched from the server (probably using AJAX) then the JavaScript errors, if any, might result in that element not being fetched.
If you are webscraping, which is clearly the case, then you don't have any control over the JS so you can only accept it as not working or disable JS and manually processing the AJAX requests.
Of course, you will see the page perfectly working on a real browser but take into consideration that the JavaScript engine HtmlUnit uses is different from the real browsers.
Secondly, the two lines containing the word engine are absolutely unneeded.
Thirdly, as I mentioned in a previous question of yours, this will be more suitable to be handled by means of the Facebook API.
Finally, you might find this other answer useful:
JavaScript not being properly executed in HtmlUnit
I want to login into a website. I need to identify the url to login. The view source shows post method as follows.
<form id="signIn" onsubmit="return false;" action="/f1/logon" method="post">
I see that there is no javascript that validates the url.
When i use the below url directly on a browser,
https://www.abc.com/f1/logon
I get a blank page. When I use below,
https://www.abc.com/logon
I dont see the repsonse of the loggedin page. It shows the response of the signin page though. How do i identify the correct url to autologin. downloaded soem toold like fiddler but no help. any inputs?
Invoking a URL in a browser sends a GET request. This form, however, is configured to send a POST request (look at the method attribute), so it makes sense that you are not seeing anything in your browser.
It's strange that the onsubmit attribute returns false. This prevents the form from being submitted at all. Take a look at the "login" button. There may be some Javascript code there which does the form submission.
Fidller is a good tool, you can start capture job and then you perform a login. Find which request contain your login and password.
Here is my situation: the user selects a section (for example from a dropdown) such as "Section1," "Section2" or "Section3." Then he clicks the OK button (or some link).
What I need to happen: after he clicks on that button/link, he will be redirected to the selected section, e.g. www.homepage.com/docs#section2.
So far, I have not been able to process the form from Link's onClick method, nor have I been able to call some clickLink on Link from the Button method onSubmit().
I would prefer not to use AJAX or JavaScript. How can I do this?
That's because a Link doesn't submit the form. It just acts as a link to somewhere. To access your formdata you'll need to submit the form first. Try using a SubmitLink instead of a Link and call
getRequestCycle().setRequestTarget
(new RedirectRequestTarget("www.homepage.com/docs#section2"));
from the onSubmit function of the SubmitLink.
Judging from the Javadoc this should work but I can't test it right now.
A RequestTarget that will send a redirect url to the browser. Use this if you
want to direct the browser to some external URL, like Google etc, immediately.
Or if you want to redirect to a Wicket page. If you want to redirect with a
delay the RedirectPage will do a meta tag redirect with a delay.
Did you try Link.setAnchor(Component)?
I want to fill a text field of a HTTP form through java and then want to click on the submit button through java so as to get the page source of the document returned after submitting the form.
I can do this by sending HTTP request directly but I don't to this in this way.
I usually do it using HtmlUnit. They have an example on their page :
#Test
public void submittingForm() throws Exception {
final WebClient webClient = new WebClient();
// Get the first page
final HtmlPage page1 = webClient.getPage("http://some_url");
// Get the form that we are dealing with and within that form,
// find the submit button and the field that we want to change.
final HtmlForm form = page1.getFormByName("myform");
final HtmlSubmitInput button = form.getInputByName("submitbutton");
final HtmlTextInput textField = form.getInputByName("userid");
// Change the value of the text field
textField.setValueAttribute("root");
// Now submit the form by clicking the button and get back the second page.
final HtmlPage page2 = button.click();
}
And you can read more here.
If you don't want to talk HTTP directly (why?), then take a look at Watij.
It allows you to invoke a browser (IE) as a COM control within your Java process, navigate through page elements by using their document ids etc., fill in forms and press buttons. Because it's running a browser, Javascript will run as normal (like if you were doing this manually).
You would probably need to write a Java Applet, as the only other way than sending a direct request would be to have it interface with the browser.
Of course, for this to work, you would have to embed the applet in the page. If you don't control the page, this can't be done. If you do control the page, you might as well be using Javascript, instead of trying to get a Java Applet to do it, which would be much more cumbersome and difficult.
Just to clarify, what is the problem you are having creating an HTTP Request and why do you want to use a different method?