This question already has answers here:
Download files in Java, Selenium using ChromeDriver and headless mode
(2 answers)
Closed 4 years ago.
Below code I have try to download PDF from chrome browser using selenium.
public static void main(String args[]) throws InterruptedException, AWTException, IOException, DocumentException {
System.setProperty("webdriver.chrome.driver", "/home/sejalj/OtherProj/webDrivers/chromedriver_64");
ChromeOptions ChromeOptions = new ChromeOptions();
ChromeOptions.addArguments("--headless", "window-size=1024,768", "--no-sandbox");
WebDriver driver = new ChromeDriver(ChromeOptions);
String baseUrl = "http://url.com/";
driver.get(baseUrl);
driver.findElement(By.name("wl_user_name")).sendKeys("uname");
driver.findElement(By.name("wl_user_password")).sendKeys("password");
driver.findElement(By.cssSelector("input[value=Login]")).click();
String pdfUrl = "https://www.pdfurl.com/displayImageDocs.php?
f=MjAxODA3MjQxMDUwNzA4Ni5QREY=&p=aW1hZ2UuaW1hZ2ViYW5rLmJsdsaddhbmsxLjIwMTgwNy4yMDE4MDcyNA==&a=MTAwMjM0&POL_NUM=AAS06036999";";
// Opens pdf of specific URL
driver.get(pdfUrl);
Actions a = new Actions(driver);
// To press CTRL+S
a.keyUp(Keys.CONTROL).sendKeys("s").build().perform();
Robot robot = new java.awt.Robot();
robot.keyPress(KeyEvent.VK_BACK_SPACE);
robot.keyRelease(KeyEvent.VK_BACK_SPACE);
int keyInput[] = {KeyEvent.VK_A, KeyEvent.VK_A, KeyEvent.VK_A, KeyEvent.VK_A, KeyEvent.VK_UNDERSCORE,
KeyEvent.VK_P, KeyEvent.VK_O, KeyEvent.VK_L, KeyEvent.VK_I, KeyEvent.VK_C, KeyEvent.VK_Y
};
for (int i = 0; i < keyInput.length; i++) {
robot.keyPress(keyInput[i]);
robot.keyRelease(keyInput[i]);
}
// To press ENTER
robot.keyPress(KeyEvent.VK_ENTER);
Thread.sleep(1000);
robot.keyRelease(KeyEvent.VK_ENTER);
System.out.println("Preparing Policies document...Please wait");
System.out.println("Document prepared..");
}
Above code working fine while chrome run without --headless mode.
But in --headless mode above code not working.
Robot class does not support in headless mode.
Please guide.
Solution that has been mark as working can be found here:
Download files in Java, Selenium using ChromeDriver and headless mode
Try this one and remove code after driver.get(pdfUrl);
this may help you using HashMap of string with objects,
<-- language: lang-java -->
HashMap<String, Object> prefs = new HashMap<>();
prefs.put("plugins.always_open_pdf_externally", true);
ChromeOptions.setExperimentalOption("prefs", prefs);
I got the solution after referring the below link:
Download files in Java, Selenium using ChromeDriver and headless mode
Here is the working code to download PDF file in headless mode.
public static void main(String args[]) throws InterruptedException, AWTException, IOException, DocumentException {
System.setProperty("webdriver.chrome.driver", "/home/OtherProj/webDrivers/chromedriver_64");
String downloadPath = "/home/Downloads/AAAA/";
File file = new File(downloadPath);
if(!file.exists())
file.mkdirs();
ChromeOptions chromeOptions = new ChromeOptions();
HashMap<String, Object> prefs = new HashMap<>();
prefs.put("plugins.always_open_pdf_externally", true);
chromeOptions.addArguments("--test-type");
chromeOptions.addArguments("--disable-extensions");
chromeOptions.setExperimentalOption("prefs", prefs);
chromeOptions.setHeadless(true);
String pdfUrl = "https://www.dummyurl.com/prod/displayImageDocs.php?f=MjAxODsafdfgsdjhgsjkA3MjQxMDUwNzA4Ni5QREY=&p=aW1hZ2UuaW1hZ2ViYW5rLmJhbmsxLjIwMTgwNy4yMDE4MDcyNA==&a=MTAwMsgsdjM0&POL_NUM=AAS06036999";
ChromeDriverService driverService = ChromeDriverService.createDefaultService();
WebDriver driver = new ChromeDriver(driverService, chromeOptions);
// Saves the file on the given path
PDFDemo.downloadFile(downloadPath, driverService, driver, pdfUrl);
logger.debug("File Downloded Start");
long fileSizeOne;
long fileSizeTwo;
do {
File downlodedFolder = new File(downloadPath);
File downlodedFile = downlodedFolder.listFiles()[0];
fileSizeOne = downlodedFile.length();
logger.debug("fileSizeOne " + fileSizeOne);
Thread.sleep(1500);
fileSizeTwo = downlodedFile.length();
logger.debug("fileSizeTwo " + fileSizeTwo);
} while (fileSizeTwo != fileSizeOne);
logger.debug("File Downloded Completed");
System.out.println("Document Downloaded..");
}
private static void downloadFile(String downloadPath, ChromeDriverService driverService, WebDriver driver, String pdfUrl) throws ClientProtocolException, IOException, InterruptedException {
Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");
Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadPath);
commandParams.put("params", params);
HttpClient httpClient = HttpClientBuilder.create().build();
ObjectMapper objectMapper = new ObjectMapper();
String command = objectMapper.writeValueAsString(commandParams);
String u = driverService.getUrl().toString() + "/session/" + ((RemoteWebDriver) driver).getSessionId() + "/chromium/send_command";
HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/pdf");
request.setEntity(new StringEntity(command));
httpClient.execute(request);
// Opens pdf of specific URL
driver.get(pdfUrl);
}
Related
public class Test {
public static void main(String[] args) throws Exception{
String driverGoogleChromePath = "D:\\Project Doc\\Jar\\selenium\\chromedriver_win32\\chromedriver.exe";
String refererurl = "http://www.iteye.com/";
String hostAndPort = "112.91.159.66:3128";
int port = 3128;
//String host = "112.91.159.66";
System.setProperty("webdriver.chrome.driver",driverGoogleChromePath);
//start the proxy
BrowserMobProxy proxy = new BrowserMobProxyServer();
Map<String, String> headers = new HashMap<String, String>();
headers.put("Referer",refererurl);
proxy.addHeaders(headers);
proxy.start(port);
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
seleniumProxy.setHttpProxy(hostAndPort)
.setSslProxy(hostAndPort)
.setFtpProxy(hostAndPort);
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
ChromeOptions option = new ChromeOptions().merge(capabilities);
WebDriver driver = new ChromeDriver(option);
Thread.sleep(3000);
driver.get("http://www.java1234.com/");
proxy.stop();
//driver.close();
}
This is my java code, I have set the referer to use the .addHeaders, but when I run it, the browser didn't show the referer, why, how to do show this ?
selenium: selenium-server-standalone-3.141.5.jar
browser: browsermob-dist-2.1.4.jar
enter image description here
in my test I'm trying to download file from server, verify (if its downloaded) and delete it. But everytime I run my test with remote server (Selenium Grid, Chrome version=59.0.3071.115), I receive NullPointer during verification.
If I try run the test locally it passes without any problems.
My remote driver class
private WebDriver getRemoteDriver(Browser browser) throws MalformedURLException {
DesiredCapabilities desiredCapabilities = getCapabilities(browser);
desiredCapabilities.setCapability("platform", "LINUX");
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.BROWSER, Level.ALL);
desiredCapabilities.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);
String downloadFilepath = System.getProperty("user.dir") + File.separator + "downloads";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
chromePrefs.put("download.directory_upgrade", true);
chromePrefs.put("download.extensions_to_open", "");
chromePrefs.put("download.prompt_for_download", false);
chromePrefs.put("plugins.always_open_pdf_externally", false);
ChromeOptions options = new ChromeOptions();
HashMap<String, Object> chromeOptionsMap = new HashMap<String, Object>();
options.setExperimentalOption("prefs", chromePrefs);
desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, chromeOptionsMap);
desiredCapabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, options);
final RemoteWebDriver remoteWebDriver = new RemoteWebDriver(new URL(remoteUrl), desiredCapabilities);
remoteWebDriver.setFileDetector(new LocalFileDetector());
return new Augmenter().augment(remoteWebDriver);
}
And part of the test looks like this: Click button to download file, wait for new file and verify it, delete file.
driver.findElement(By.xpath(".//*[#id='browseOrders:browseView:browseViewPdfExpTblLnk']/span[1]"))
.click();
File file = navigationUtil.WaitForNewFile(download_folder, ".pdf", 60);
String fname = file.getName();
logger.info(fname);
boolean success = file.exists();
Assert.assertTrue(success);
file.delete();
Im constantly getting java.lang.NullPointerException in line String fname = file.getName(); So I assume that the file is never downloaded.
Why there is a problem on remote server to find and click some elements. In this case there is menu that shows up after clicking "Gear button" and download button, but remote machine can't find the "Gear button" even when I try different ways to find it.
I just write a test which is supposed to download pdf files via webApp(Yep, I know, I should not do it on selenium, but You know, Orders.)
What do I need?
For diffrent scenarios I have to download difrent pdf, rename it and place to custom catalog. So, I have to handle with system modal window.
Everything works great, so test is run on remote host, and when I click to download the file I handle the system modal window(I used robotil package, it is extended robot class which allow us use robot class on remote host) so I use robotil class to type path to file, and file name on system modal and then click "Enter" to confirm and save the file. This is everything I need and it works, so where is the problem? here: SOMEONE should be logged to remote host, if Im logged via rdp and looks at screen(and doing my stuff on my host) then everything is great, but for the case when no one is logged, it looks like that during tests webbrowswer do not have a FOCUS, so everytime robotil class do some action this action is not focused on webbroswer(as it should).
test class:
#Test
public void compareDeposits() throws Exception {
HomePage homePage = new HomePage(driver);
PageFactory.initElements(driver, homePage);
PrintDepositsPage printDepositsPage = (PrintDepositsPage) homePage.openViaUrl(Data.baseUrl).openViewViaTopMenu(
ETopMenuItem.PrintDeposits);
((PrintDepositsPage) printDepositsPage).goToPrintedDepositsTab();
printDepositsPage.getPrintedDepositsDateRangeFromInput().click();
printDepositsPage.getPrintedDepositsDateRangeFromInput().clear();
printDepositsPage.getPrintedDepositsGoButton().click();
printDepositsPage.getFirstRecordOnPrintedDepositsTab().click();
handler.getRobot().mouseClick(371, 274, InputEvent.BUTTON1_MASK);// get focus
printDepositsPage.getPrintButtonEnabled().click();
handler.downloadFile("DepositTest");
handler object declaration:
class SystemModalWindowHandler {
private RemoteWebDriver driver;
private Date date = new Date();
private DateFormat dateFormat = new SimpleDateFormat("yyy/mm/dd");
private String extendedTestName = dateFormat.format(date).replace("/", ".") + ".pdf";
private Robotil robotil = new Robotil("xxxxx", 6667);
public Robotil getRobot(){
return robotil;
}
public void downloadFile(String testFileName) throws AWTException, InterruptedException {
boolean continueBool = true;
while (continueBool) {
String pathToTestFile = new String("C:\\DiffPdfData\\" + testFileName + "\\"
+ extendedTestName);
Thread.sleep(3000);
for (int i = 0; i < pathToTestFile.length(); i++) {
System.out.println(KeyStroke.getKeyStroke(pathToTestFile.charAt(i)) + " = "
+ (int) pathToTestFile.charAt(i));
if ((int) pathToTestFile.charAt(i) == 58) {
robotil.pressKey(KeyEvent.VK_SHIFT);
robotil.pressAndReleaseKey(KeyEvent.VK_SEMICOLON);
robotil.releaseKey(KeyEvent.VK_SHIFT);
}
else {
robotil.pressAndReleaseKey(KeyEvent.getExtendedKeyCodeForChar((int) pathToTestFile.charAt(i)));
}
}
robotil.pressAndReleaseKey(KeyEvent.VK_ENTER);
continueBool = false;
}
is there any way to get focus on webbrowser when no one is logged in?.
I believe that using the mentioned strategy you won't be able to accomplish it without logged-in user. So I suggest you to use a simpler solution.
You can configure Firefox the directly download the files - File types and download actions
If you don't want to hardcode the setting for your browser, you can setup a specific FF profile only for your tests, where you can configure where you want the files to be downloaded.
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.setPreference("browser.download.folderList",2);
firefoxProfile.setPreference("browser.download.manager.showWhenStarting",false);
firefoxProfile.setPreference("browser.download.dir","c:\\downloads");
firefoxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk","text/csv");
WebDriver driver = new FirefoxDriver(firefoxProfile);
Chrome Driver:
String downloadFilepath = "/path/to/download";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new ChromeDriver(cap);
I'm a rookie test developer using selenium 2.45 and I'm trying to configure my FirefoxDriver to use my company's proxy settings. I am failing to do so :)
I am following the instruction from here to create a profile on-the-fly:
Using a Proxy for FF
My code looks like this:
public static WebDriver driver;
String usedProxy = "http://myproxy:8080";
org.openqa.selenium.Proxy proxy = new org.openqa.selenium.Proxy();
proxy.setHttpProxy(usedProxy).setFtpProxy(usedProxy).setSslProxy(usedProxy);
DesiredCapabilities cap = new DesiredCapabilities();
cap.setCapability(CapabilityType.PROXY, proxy);
driver = new FirefoxDriver(cap);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://TestWebsite.com");
I am not receiving any kind of errors but the connection is not working for this browser. When checking Options>Advanced>Network>Connection Settings from Firefox menu, proxy is set to manual but text inputs only contain "http://"
PS: I have a feeling this is relevant but am not sure: TestWebsite.com will load via https only (it's a shopping cart)
If your goal is to test your functionality on different IP addresses, you can use the Tor Browser.
public IWebDriver Driver { get; set; }
public Process TorProcess { get; set; }
public WebDriverWait Wait { get; set; }
[TestInitialize]
public void SetupTest()
{
String torBinaryPath = #"C:\Users\aangelov\Desktop\Tor Browser\Browser\firefox.exe";
this.TorProcess = new Process();
this.TorProcess.StartInfo.FileName = torBinaryPath;
this.TorProcess.StartInfo.Arguments = "-n";
this.TorProcess.StartInfo.WindowStyle = ProcessWindowStyle.Maximized;
this.TorProcess.Start();
FirefoxProfile profile = new FirefoxProfile();
profile.SetPreference("network.proxy.type", 1);
profile.SetPreference("network.proxy.socks", "127.0.0.1");
profile.SetPreference("network.proxy.socks_port", 9150);
this.Driver = new FirefoxDriver(profile);
this.Wait = new WebDriverWait(this.Driver, TimeSpan.FromSeconds(60));
}
[TestCleanup]
public void TeardownTest()
{
this.Driver.Quit();
this.TorProcess.Kill();
}
Here is the code to refresh the Tor identity.
public void RefreshTorIdentity()
{
Socket server = null;
try
{
IPEndPoint ip = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9151);
server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
server.Connect(ip);
server.Send(Encoding.ASCII.GetBytes("AUTHENTICATE \"johnsmith\"" + Environment.NewLine));
byte[] data = new byte[1024];
int receivedDataLength = server.Receive(data);
string stringData = Encoding.ASCII.GetString(data, 0, receivedDataLength);
server.Send(Encoding.ASCII.GetBytes("SIGNAL NEWNYM" + Environment.NewLine));
data = new byte[1024];
receivedDataLength = server.Receive(data);
stringData = Encoding.ASCII.GetString(data, 0, receivedDataLength);
if (!stringData.Contains("250"))
{
Console.WriteLine("Unable to signal new user to server.");
server.Shutdown(SocketShutdown.Both);
server.Close();
}
}
finally
{
server.Close();
}
}
You can find more detailed information here: http://automatetheplanet.com/using-selenium-webdriver-tor-c-code/
The code examples are in C# but the code should be identical in Java.
Try firefox profile as well. Note this is C# code and converting to Java should be fairly simply
string usedProxy = "http://myproxy:8080";
Proxy proxy = new OpenQA.Selenium.Proxy();
proxy.HttpProxy = usedProxy;
proxy.FtpProxy = usedProxy;
proxy.SslProxy = usedProxy;
FirefoxProfile profile = new FirefoxProfile();
profile.SetProxyPreferences(proxy);
This code works for me:
FirefoxOptions options = new FirefoxOptions();
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.http", <proxy_url>);
profile.setPreference("network.proxy.http_port", 8080);
profile.setPreference("network.proxy.ssl", <proxy_url>);
profile.setPreference("network.proxy.ssl_port", 8080);
options.setProfile(profile);
options.setAcceptInsecureCerts(true)
options.setCapability("disable-restore-session-state", true);
options.setCapability("marionette", true);
WebDriver driver = new FirefoxDriver(options);
I followed answers from this link: Webdriver and proxy server for firefox
So I am looking to do some simple data collection on a few .onion sites. I am going about this by using selenium webdriver to call Tor as part of the Firefox webdriver. However, I can't seem to figure out how to get firefox to successfully go to .onion sites. Here is the code.
public static void main(String[] args) throws InterruptedException, IOException {
File torProfileDir = new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default");
FirefoxBinary binary = new FirefoxBinary(new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\firefox.exe"));
FirefoxProfile torProfile = new FirefoxProfile(torProfileDir);
torProfile.setPreference("webdriver.load.strategy", "unstable");
try {
binary.startProfile(torProfile, torProfileDir, "");
} catch (IOException e) {
e.printStackTrace();
}
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.socks", "127.0.0.1");
profile.setPreference("network.proxy.socks_port", 9150);
FirefoxDriver driver = new FirefoxDriver(profile);
int firstCheck = "0";
while (firstCheck == 0) {
driver.navigate().to("onion site here");
......
The problem I am having is that I end up with a firefox browser that cannot connect to .onion sites. If I change FirefoxDriver driver = new FirefoxDriver(profile); to FirefoxDriver driver = new FirefoxDriver(binary, profile); then I am left with a blank Tor window that I can't seem to control with the webdriver.
Anyone have any ideas on how to fix this? any help would be appreciated!
Figured it out. Needed to add a lot of permissions to the new profile. Here is the fixed code for those that are interested. It allows you to browse the dark web with Firefox as if you were using Tor. The commands for controlling the webdriver don't change.
File torProfileDir = new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default");
FirefoxBinary binary = new FirefoxBinary(new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\firefox.exe"));
FirefoxProfile torProfile = new FirefoxProfile(torProfileDir);
torProfile.setPreference("webdriver.load.strategy", "unstable");
try {
binary.startProfile(torProfile, torProfileDir, "");
} catch (IOException e) {
e.printStackTrace();
}
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.socks", "127.0.0.1");
profile.setPreference("network.proxy.socks_port", 9150);
profile.setPreference("network.proxy.socks_version", 5);
profile.setPreference("places.history.enabled", false);
profile.setPreference("privacy.clearOnShutdown.offlineApps", true);
profile.setPreference("privacy.clearOnShutdown.passwords", true);
profile.setPreference("privacy.clearOnShutdown.siteSettings", true);
profile.setPreference("privacy.sanitize.sanitizeOnShutdown", true);
profile.setPreference("signon.rememberSignons", false);
profile.setPreference("network.cookie.lifetimePolicy", 2);
profile.setPreference("network.dns.disablePrefetch", true);
profile.setPreference("network.http.sendRefererHeader", 0);
profile.setPreference("network.proxy.socks_remote_dns", true);
FirefoxDriver driver = new FirefoxDriver(profile);
String firstCheck = "";
while (firstCheck == 0) {
driver.get("http://kbhpodhnfxl3clb4.onion/");
.........................