I want to simulate opening a web page in java, I know I can do this to actually open the page in my browser on my computer,
String htmlFilePath = "path/to/html/file.html"; // path to your new file
File htmlFile = new File(htmlFilePath);
// open the default web browser for the HTML page
Desktop.getDesktop().browse(htmlFile.toURI());
// if a web browser is the default HTML handler, this might work too
Desktop.getDesktop().open(htmlFile);
But is there a way to simulate it so I don't actually see it open on my computer, but it still evaluates like someone did open the web page.
Or if that is not possible what would be the easiest way to physically open it on my computer and then have a way of getting a callback so that I know when the page has been loaded?
Thanks
There are several ways to emulate an HTTP client (such as a web browser):
Jersey (Java) - https://jersey.java.net/documentation/latest/client.html
Apache HTTPClient (Java) - https://hc.apache.org/
JMeter (Java) - Use JMeter to record an HTTP request and replay it as a test - https://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.pdf
Selenium (browser plugin) - http://www.seleniumhq.org/
CURL (command line tool) - http://curl.haxx.se/
I do recommend Jersey in your case. It is a tool especially designed for REST. So it may even help server-side development.
I know you specifically asked for a Java solution, but the last two options are really popular.
I have a sample program here that uses the Selenium library.
Launch Firefox and Wait until it is Closed
The program has a code that launches a browser and opens a website. It can detect if the browser has done loading the website. It can also detect if the browser was closed.
Related
I'm developing a web app (in Spring Boot) that would need to simplify the user interaction with other websites (third party, not developed by me). The idea is that my app pre-fill a form so the user won't have to do it.
As I began to read about this I stumble upon HtmlUnit and then Selenium... I tried Selenium and I was very happy while I was developing because I manage to open a browser window and fill it as I want it.
Then I thought it would be a good idea to test it from another computer in the same LAN. So open up a browser, navigate to my ip:port and when I hit the button that should perform the automation described, the browser open up on my "server" machine where the app is running and not on the client.
It was very frustrating. Keep reading and I found something about Selenium Hub and Grid. I've read several articles about it but it confuses me because they are all oriented at performing tests and I don't even understand where should I start or if it even possible.
What I want is to have my web-app installed in a Ubuntu Server and any client could access it and when hitting a button open a web page in a specific URL and pre-fill the form data.
Can you guide me? I'm in the correct path with Selenium Grid and Hub or there's any other technology I should be using?
I am actually involed in a pet project that involves scraping data from websites. The application I am writing is entirely in java. And this application runs for several hours scraping data from web pages.
Because of this, it happened that many times my IP has been blocked in several websites. That is the reason I am trying to access the websites through Tor networks.
I have used the code from this Stackoverlow link for running the Tor service from Orchid.
So after running the Tor service, I am using phantomjs to scrape websites.. So
I am running phantomjs as phantomjs --proxy-type=socks5 --proxy=127.0.0.1:9150 script.js (Since tor service running on 9150 port. PhantomJS v2.1)
script.js contains
var page = new WebPage()
var fs = require('fs');
page.open("WEBSITE_ADDRESS", function() {
page.evaluate(function() {
});
});
page.onLoadFinished = function() {
fs.write('FILE_LOCATION', page.content, 'w');
phantom.exit();
};
Now here is the problem. When I try to run phantomjs, it quickly returns back returning an empty HTML file. But when I do the same using Tor.exe (i.e starting the tor.exe file and then trying to run PhantomJS using the same command specified above), It works perfectly. it works for both https and http. But while running Orchid, both https and http are not working.
One more thing is that, when i try to connect to some website from java class (using HttpURLConnection class) where the method for starting tor service is invoked, I am able to access both http and https websites (Tor service is working since I am getting a new ip address each time I visit whatismyip.com)
The reason i am not going with java based web scraping libraries like jsoup is that the websites i am trying to scrape data from use javascript excessively. Due to this, i always end up getting an imcomplete page which is not the case with phantomJs. And also i don't want to continue using tor.exe for running the tor service as it makes the project heavy. And also i cannot control tor.exe completely from java
Please help me with this.
After so much struggle on setting up PhantomJS to work with SOCKS(Tor), I finally decided to give on that exe. It is a known issue with Phantomjs
Now instead I am using JBrowserDriver for web scraping. That works a charm with Orchid Tor service. Now that everything in java, I am able to control everything .
And one more point, JBrowserDriver employs multiple threads while downloading a page, and seemingly faster than PhantomJS.
Thanks all for your efforts.
My question is just as the title indicates.
I have a JavaFX app, and need to open a browser window. After opening it, I want to be able to communicate from the browser back to the app that opened it.
For example, if I opened the default browser window like this:
URI u = new URI(url);
java.awt.Desktop.getDesktop().browse(u);
Three options:
Through Javascript
Using javascript, and the window name you could access any window.
E.g. See: https://stackoverflow.com/a/16525481/1688441
Through Ajax Calls and a Server/Database
As another user answered, communication could be done with an intermediate server.
Third party library
The only other thing I can think of is using a 3rd party library to get the window within Operating System, though not much more I can suggest.
You can communicate back and forth with a WebView component. See http://docs.oracle.com/javafx/2/webview/jfxpub-webview.htm
If you are launching a page in the hosts browser, you'll need to develop a server based method to communicate.
I'm searching for a way to interact with different android browsers.
My intention is to get the URL of the website that is opened by the browser.
My app should run in background and give an alert when a specific url is opened, for now this is for testing and to learn more about android.
Is there any way to tell the browser to pass the url-string to my app everytime a new url is entered?
I've already got the idea to implement a local proxyserver wich checks all http-requests for a specific url, but is there a better way?
aditionally this should work with every browser not just the standard one but also eg. firefox or dolphin.
I already searched the last few days but couldn't find an answer....
Yes, you can do this! Check out http://seleniumhq.org/ for how to automate a web browser.
I am preparing a program in java and its purpose is to make the HttpWebRequest for a url which can open in any browser (i am planning to use watij for opening the page in different browser).Now my program should take the screen shot of the presently opened url once the page has loaded successfully.I need to run a piece of code after the browser has loaded successfully in the browser.I dont want to use the javascript document.ready function for identifying if the page is loaded successfully or not.I should be able to know the page load complete status in the server side (s0 that i can execute that piece of code for taking the screenshot).How can i do it in java.Or is there any other way to do achieve that if its not possible in java.May be create some plugins.
An early reply is highly valued,
Regards,Sagar.
you can do it using applet, [just taking snapshot i meant that you want.]