I want to login to a website (http://www.orkut.com) through
com.gargoylesoftware.htmlunit.WebClient
But when I click on the "Submit" button, it doesn't take me to the expected page that should come after login. Instead it returns the same login page again. In clear sense, there is some problem in login. When I try the same code with sites that doen't have javascript, it works fine so I think I am not able to handle scripts.
I am trying using the follwoing code:
public static void main(String[] args) {
final WebClient webClient = new WebClient();
try {
HtmlPage loginPage = webClient.getPage(new URL("https://www.google.com/accounts/ServiceLogin?service=orkut&hl=en-US&rm=false&continue=http%3A%2F%2Fwww.orkut.com%2FRedirLogin%3Fmsg%3D0%26page%3Dhttp%253A%252F%252Fwww.orkut.co.in%252FHome.aspx&cd=IN&passive=true&skipvpage=true&sendvemail=false"));
System.out.println(loginPage.getTextContent());
List<HtmlForm> forms = loginPage.getForms();
HtmlForm loginForm = forms.get(0);
HtmlInput username = loginForm.getInputByName("Email");
HtmlInput password = loginForm.getInputByName("Passwd");
HtmlInput submit = loginForm.getInputByName("signIn");
username.setNodeValue("username");
password.setNodeValue("password");
HtmlPage homePage = submit.click();
Thread.sleep(10 * 1000);
System.out.println(homePage.getTextContent());
}catch(Exception ex) {
ex.printStackTrace();
}
}
When we do click on the "submit" button, in actual it calls first this function
onsubmit="return(gaia_onLoginSubmit());"
specified as the attribute of the form below
<form id="gaia_loginform" action="https://www.google.com/accounts/ServiceLoginAuth?service=orkut" method="post"
onsubmit="return(gaia_onLoginSubmit());">
Can anyone help me in this.
NOTE: I WILL PAY FOR THE SOLUTION
According to their site the JavaScript support is provided by Mozilla Rhino, so maybe all you need is to add it to your classpath (and perhaps fiddle with some configurations).
Also, HtmlUnit has professional support
Related
I have written a simple program which should login via a form on a website.
Unfortunately, the form in the html has no name or id.
I use the latest version of HtmlUnit and Java 11.
I tried to find the form with the .getForms () method, but without success.
Html Snippet from Website i try to login
Here is my code to find the form:
//Get the form
HtmlForm form = LoginPage.getFormByName("I tried several options here");
//Get the Submit button
final HtmlButton loginButton = form.getButtonByName("Anmelden");
//Get the text fields for password and username
final HtmlTextInput username = form.getInputByName("text");
final HtmlTextInput password = form.getInputByName("password");
Whatever I tried, I didn't find any form.
This is my connection class if it helps:
public HtmlPage CslPlasmaConnection(){
//Create Webclient to connect to CslPlasma
WebClient CslPlasmaConnection = new WebClient(BrowserVersion.BEST_SUPPORTED);
//helper variable ini with null
HtmlPage CslPlasmaLoginPage = null;
//Get the content from CslPlasma
try {
CslPlasmaLoginPage = CslPlasmaConnection.getPage(URL);
} catch (IOException e) {
e.printStackTrace();
}
//Return CslPlasma Login Page
return CslPlasmaLoginPage;
}
Without knowing the page i can only guess...
Have a look at this answer https://stackoverflow.com/a/54188201/4804091
And try to use the latest page (maybe there is some js that creates the form).
webClient.getPage(url);
webClient.waitForBackgroundJavaScript(10000);
HtmlPage page = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();
If you're sure this is the only form on the page or you know which form number it is, you can use page.getForms() to get all forms of the page and get yours from the resulting list.
Like so:
HtmlForm form = LoginPage.getForms().get(0); // if it's the only form, its index is 0
i built java application on desktop that allows sharing video to youtube to specific google acount.
I used the suggested code from example for uploading a video from:
https://developers.google.com/youtube/v3/code_samples/java#upload_a_video
with the json token of the user.
But when i'm trying to upload it opens a web page of google and askes me to log-in and approve uploading videos with my user.
Is there a way to skip this level and do it from the java code?
I don't want to make the user log-in in a web page to his google acount and only use the GUI i made with java.
You can copy class
AuthorizationCodeInstalledApp
and override the method
browse(String url)
You need to realize, go to a url programmatically, like this(I use htmlunit):
public void browse(String url) throws IOException {
WebClient webClient = initWebClient();
HtmlPage htmlPage = webClient.getPage(url);
//first you need login with your email and password
final HtmlTextInput login = (HtmlTextInput) htmlPage.getByXPath("//input[#type='email']").get(0);
final HtmlPasswordInput pass = (HtmlPasswordInput) htmlPage.getByXPath("//input[#type='password']").get(0);
HtmlSubmitInput button = (HtmlSubmitInput) htmlPage.getByXPath("//input[#type='submit']").get(0);
//set input login and passwd
login.setText(this.login);
pass.setText(this.passwd);
//press submit button
button.click();
//next need select account
htmlPage = webClient.getPage(url);
DomNodeList<HtmlElement> list = htmlPage.getElementById("account-list").getElementsByTagName("li");
String account = list.get(0).getElementsByTagName("a").get(0).getAttribute("href");
System.out.println(account);
htmlPage = webClient.getPage(account);
//and click submit button for approve
System.out.println("Wait 10sec.");
webClient.waitForBackgroundJavaScript(10000);
HtmlButton submitInput = (HtmlButton) htmlPage.getElementById("submit_approve_access");
submitInput.click();
}
that's fine works for me.
There are some html parsing libraries available,
but what do you do if you need to authenticate, and carry the cookies with each request?
And generally what if you need to press some button in order to get the content that you want to parse,
for example that button calculates something, or gets some data trough websocets etc...
Is there a technology to simulate behaviour in browser (so that all js is actually working) and parse from there...?
UPDATE
Maybe for this purpose i need to embed chromium and use traditional parsers? Tough i dont understand how do i trigger a click...
HtmlUnit: http://htmlunit.sourceforge.net/
public static void main(String... args) throws Exception {
final WebClient webClient = new WebClient();
final HtmlPage page1 = webClient.getPage("http://some_url");
final HtmlForm form = page1.getFormByName("myform");
final HtmlSubmitInput button = form.getInputByName("submitbutton");
final HtmlTextInput textField = form.getInputByName("userid");
textField.setValueAttribute("root");
final HtmlPage page2 = button.click();
webClient.closeAllWindows();
}
I'm trying to send a message on www.meetme.com but can't figure out how to do it. I can type in the message in the comment area but clicking the Send button doesn't do anything. What am I doing wrong? When I login and press the Login button the page does change and everything is fine. Anyone have any ideas or clues?
HtmlPage htmlPage = null;
HtmlElement htmlElement;
WebClient webClient = null;
HtmlButton htmlButton;
HtmlForm htmlForm;
try{
// Create and initialize WebClient object
webClient = new WebClient(BrowserVersion.FIREFOX_17 );
webClient.setCssEnabled(false);
webClient.setJavaScriptEnabled(false);
webClient.setThrowExceptionOnFailingStatusCode(false);
webClient.setThrowExceptionOnScriptError(false);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setUseInsecureSSL(true);
webClient.getCookieManager().setCookiesEnabled(true);
/*webClient.setRefreshHandler(new RefreshHandler() {
public void handleRefresh(Page page, URL url, int arg) throws IOException {
System.out.println("handleRefresh");
}
});*/
htmlPage = webClient.getPage("http://www.meetme.com");
htmlForm = htmlPage.getFirstByXPath("//form[#action='https://ssl.meetme.com/login']");
htmlForm.getInputByName("username").setValueAttribute("blah#gmail.com");
htmlForm.getInputByName("password").setValueAttribute("blah");
//Signing in
htmlButton = htmlForm.getElementById("login_form_submit");
htmlPage = (HtmlPage) htmlButton.click();
htmlPage = webClient.getPage("http://www.meetme.com/member/1234567890");
System.out.println("BEFORE CLICK");
System.out.println(htmlPage.asText());
//type message in text area
HtmlTextArea commentArea = (HtmlTextArea)htmlPage.getFirstByXPath("//textarea[#id='profileQMBody']");
commentArea.setText("Testing");
htmlButton = (HtmlButton) htmlPage.getHtmlElementById("profileQMSend");
htmlPage = (HtmlPage)htmlButton.click();
webClient.waitForBackgroundJavaScript(7000);
//The print is exactly the same as the BEFORE CLICK print
System.out.println("AFTER CLICK");
System.out.println(htmlPage.asText());
}catch(ElementNotFoundException e){
e.printStackTrace();
}catch(Exception e){
e.printStackTrace();
}
Without knowing much about the webpage you're accessing, you just can't perform an AJAX request with JavaScript disabled. If changing that doesn't result in success, then you will have to keep debugging, but make sure JavaScript is enabled.
Additionally, make sure you're using HtmlUnit 1.12 and update all the deprecated methods in your code.
BTW, I'd also recommend to turn may JavaScript warnings off. Check this answer to see how you can do that.
I have two files, a list of usernames and a list of passwords. I need to write a program to check each user name with the list of passwords. Then I need to go to a website and see if it logs in. I am not very sure how to go about the comparing and how to simulate the program to log in the website enter the information. Could you please help me out with this? It's a homework problem.
Regardless of the language you choose to implement this in, the basic idea is to simulate log-ins programatically. This can be done by logging in manually and looking at the HTTP headers, then sending "forged" headers programatically, changing the user/password fields.
Most log-ins will use POST and making a POST is not entirely straightforward. If you are allowed to use external libraries, you can try cURL. Simply set the appropriate headers and look at the response to check if your attempt was successful or not. If not, try again with a new combination.
In pseudo code:
bool simulate_login(user, password) :
request = new request(url)
request.set_method('POST')
request.set_header('name', user)
request.set_header('pass', password)
response = request.fetch_reponse()
return response.contains("Login successful")
success = []
foreach user:
foreach password:
if (simulate_login(user, password)):
success.append((user, password))
break
If you would like to use java you can try with HtmlUnit (see: http://htmlunit.sourceforge.net/gettingStarted.html) or if you are allowed Groovy you can go with http://www.gebish.org/
Here is the example from getting started guide that is relevant to your case:
public void login() throws Exception {
final WebClient webClient = new WebClient();
// Get the first page
final HtmlPage page1 = webClient.getPage("http://some_url");
// Get the form that we are dealing with and within that form,
// find the submit button and the field that we want to change.
final HtmlForm form = page1.getFormByName("myform");
final HtmlSubmitInput button = form.getInputByName("submitbutton");
final HtmlTextInput textField = form.getInputByName("userid");
// Change the value of the text field
textField.setValueAttribute("username");
// Do similar for password and that's all
// Now submit the form by clicking the button and get back the second page.
final HtmlPage page2 = button.click();
webClient.closeAllWindows();
}
If you would like to use java you can try with HtmlUnit (see: http://htmlunit.sourceforge.net/gettingStarted.html) or if you are allowed Groovy you can go with http://www.gebish.org/
Here is the example from getting started guide that is relevant to your case:
public void login() throws Exception {
final WebClient webClient = new WebClient();
// Get the first page
final HtmlPage page1 = webClient.getPage("http://some_url");
// Get the form that we are dealing with and within that form,
// find the submit button and the field that we want to change.
final HtmlForm form = page1.getFormByName("myform");
final HtmlSubmitInput button = form.getInputByName("submitbutton");
final HtmlTextInput textField = form.getInputByName("userid");
// Change the value of the text field
textField.setValueAttribute("username");
// Do similar for password and that's all
// Now submit the form by clicking the button and get back the second page.
final HtmlPage page2 = button.click();
List item