I am using Browsermob proxy to catch HAR content of loaded web page thru chrome webdriver.
<dependency>
<groupId>net.lightbody.bmp</groupId>
<artifactId>browsermob-core-littleproxy</artifactId>
<version>2.1.0-beta-3</version>
</dependency>
And here is how I initialize proxy:
ChromeOptions options = new ChromeOptions();
for (String extension : DriverProperties.CHROME_EXTENSIONS.getStringArray()) {
options.addExtensions(new File(extension));
}
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.ForSeleniumServer.ENSURING_CLEAN_SESSION, true);
capabilities.setCapability(CapabilityType.SUPPORTS_APPLICATION_CACHE, false);
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
proxy = new BrowserMobProxyServer();
proxy.start(0);
proxy.setHarCaptureTypes(CaptureType.RESPONSE_CONTENT, CaptureType.RESPONSE_BINARY_CONTENT);
// get the Selenium proxy object
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
proxy.newHar();
ChromeDriver driver = new ChromeDriver(capabilities);
But when I try to get url via https (for exapmle https://screen.yahoo.com) I see "This web page has a redirect loop" in the web browser. Is there any solution for this?
Related
I am using the code mentioned below but it is not working Chrome Version 79:
DesiredCapabilities capability = DesiredCapabilities.chrome();
capability.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
//capability.setCapability("pageLoadStrategy", "none");
System.setProperty("webdriver.chrome.driver","driver//chromedriver.exe");
driver = new ChromeDriver(capability);
//driver.manage().deleteAllCookies();
driver.manage().window().maximize();
//Runtime.getRuntime().exec("AutoIT_Exe//AutoIT_Login.exe");
driver.get(url);
You can use switches to ignore such errors
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--ignore-certificate-errors"));
driver = new ChromeDriver(capabilities);
I use ChromeDriver 2.33 with WebDriver 3.6.0 and try to set default directory for file download.
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("download.default_directory", Vars.DOWNLOAD_FOLDER_ROOT);
DesiredCapabilities caps = DesiredCapabilities.chrome();
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized");
options.setExperimentalOption("prefs", prefs);
caps.setCapability(ChromeOptions.CAPABILITY, options);
driver = new ChromeDriver(caps);
I found this in docs:
Use ChromeDriver(ChromeOptions) instead. Creates a new ChromeDriver
instance. The capabilities will be passed to the chromedriver service.
I hope you wanted to ask about the workaround to avoid the deprecation.
The old method of just building with Capabilities is deprecated. Now, it takes a ChromeDriverService & Capabilities as parameters. So, just a build a ChromeDriverService and pass the same along with your Capabilities to remove the deprecation warning.
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File("/usr/local/chromedriver"))
.usingAnyFreePort()
.build();
ChromeDriver driver = new ChromeDriver(service, capabilities);
EDIT:
Since ChromeDriver(service, capabilities) is deprecated now as well, you can use,
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File("/usr/local/chromedriver"))
.usingAnyFreePort()
.build();
ChromeOptions options = new ChromeOptions();
options.merge(capabilities);
ChromeDriver driver = new ChromeDriver(service, options);
However, You can completely skip DesiredCapabilities and use only ChromeOptions with setCapability method like,
ChromeOptions options = new ChromeOptions();
options.setCapability("capability_name", "capability_value");
driver = new ChromeDriver(options);
The new way to use chrome capabilities is like this :
ChromeOptions options = new ChromeOptions();
// Proxy proxy = new Proxy();
// proxy.setHttpProxy("myhttpproxy:3337");
// options.setCapability("proxy", proxy);
// options.addArguments("--headless");
// options.addArguments("--disable-gpu");
// options.setAcceptInsecureCerts(true);
// options.addArguments("--allow-insecure-localhost");
// options.addArguments("--lang=fr-CA");
options.addArguments("--start-maximized");
driver = new ChromeDriver(options);
You can get more options by looking at this site : https://sites.google.com/a/chromium.org/chromedriver/capabilities
I need to get headless chrome to ignore certificate errors. The option is ignored when running in headless mode, and the driver returns empty html body tags when navigating to an https resource.
<html xmlns="http://www.w3.org/1999/xhtml"><head></head><body></body></html>
This is how I am configuring my chrome driver.
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--headless", "--disable-gpu", "--window-size=1920,1200","--ignore-certificate-errors");
DesiredCapabilities cap=DesiredCapabilities.chrome();
cap.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
chromeHeadlessDriver = new ChromeDriver(cap);
This thread confirms that --ignore-certificate-errors is ignored in headless mode.
They mention about devtool protocol.
Is it something I can invoke from java? Are there any other alternatives?
There is an excellent article on medium.com by sahajamit
and i have tested the below code, it works perfectly fine with self-signed certificate https://badssl.com/
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("useAutomationExtension", false);
options.addArguments("--headless", "--window-size=1920,1200","--ignore-certificate-errors");
DesiredCapabilities crcapabilities = DesiredCapabilities.chrome();
crcapabilities.setCapability(ChromeOptions.CAPABILITY, options);
crcapabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
crcapabilities.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
System.setProperty(ChromeDriverService.CHROME_DRIVER_LOG_PROPERTY, "C:\\temp\\chrome\\chromedriver.log");
System.setProperty(ChromeDriverService.CHROME_DRIVER_EXE_PROPERTY, "C:\\temp\\chrome\\chromedriver.exe");
ChromeDriverService service = null;
try {
service = new ChromeDriverService.Builder()
.usingAnyFreePort()
.withVerbose(true)
.build();
service.start();
} catch (IOException e) {
e.printStackTrace();
}
RemoteWebDriver driver = new RemoteWebDriver(service.getUrl(),crcapabilities);
driver.get("https://self-signed.badssl.com/");
System.out.println(driver.getPageSource());
driver.quit();
software/framework versions
Google Chrome Version 64.0.3282.186
Google Chrome Driver Version 64.0.3282.186
Selenium version 3.11.0
This works for me on ChromeDriver 80.
ChromeOptions option = new ChromeOptions();
option.AddArgument("--headless");
option.AcceptInsecureCertificates = true;
driver = new ChromeDriver(option);
#amila-kumara is working but, usage of DesiredCapabilities.chrome() gives warning to use ChromeOptions. Please see the updated answer.
Set the chrome option values
System.setProperty("webdriver.chrome.driver", Config.NDAC_WEBDRIVER_PATH);
ChromeOptions options = new ChromeOptions();
options.addArguments("--window-size=1920,1200");
options.setAcceptInsecureCerts(true);
options.setHeadless(true);
Start the service
ChromeDriverService service = null;
try {
service = new ChromeDriverService.Builder()
.usingAnyFreePort()
.withVerbose(true)
.build();
service.start();
remoteWebdriverUrl = service.getUrl();
System.out.println("Starting the url " + remoteWebdriverUrl);
} catch (IOException e) {
e.printStackTrace();
}
Note: I was facing the issue while closing the driver(with RemoteWebDriver), chromedriver.exe process won't close even when you use driver.quit(). To fix the issue use ChromeDriver instead of RemoteWebDriver
RemoteWebDriver driver = new ChromeDriver(service, options);
To properly close the driver, use
driver.close();
driver.quit();
I am unable to capture request and response using browsermob(selenium+PhantomJS browser)
please refer the sample code
server = new BrowserMobProxyServer();
server.start(0);
server.newHar("contracts");
Capabilities:
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setJavascriptEnabled(true); capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS,
new String[] {"--web-security=false", "--ssl-protocol=any",
"--ignore-ssl-errors=yes"});
capabilities.setCapability("takeScreenshot", true);
URL hubUrl = new URL("http://152.188.0.42:5555/wd/hub");
server.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
launching the application:
driver = new RemoteWebDriver(hubUrl,capabilities);
driver.get("http://www.google.com");
Creating the har file:
Har har = server.getHar();
FileOutputStream fos = new FileOutputStream("runnowNew.har");
har.writeTo(fos);
Har which is generated by using the above code:
{
"log":
{
"version":"1.2",
"creator":{"name":"BrowserMob Proxy","version":"2.1.2","comment":""},
"pages":[{"id":"contracts","startedDateTime":"2016-10-05T12:56:33.460+05:30","title":"contracts","pageTimings":{"comment":""},"comment":""}],
"entries":[],
"comment":""
}
}
I think the problem is that you don't use BrowserMob Proxy as a proxy for the Selenium traffic.
You need to set the proxy of Selenium (ip & port) to the one configured in BrowserMob Proxy.
IP is probably 127.0.0.1 since you work locally, and you can use server.getPort() to get the port that BrowserMob Proxy is listening to.
public class Test_One
{
public static void main(String[] args) throws Exception {
ProxyServer server = new ProxyServer(8105);
server.start();
server.setCaptureHeaders(true);
server.setCaptureContent(true);
server.newHar("test");
DesiredCapabilities capabilities = new DesiredCapabilities();
Proxy proxy = server.seleniumProxy();
FirefoxProfile profile = new FirefoxProfile();
profile.setAcceptUntrustedCertificates(true);
profile.setAssumeUntrustedCertificateIssuer(true);
profile.setPreference("network.proxy.http", "localhost");
profile.setPreference("network.proxy.http_port", 8105);
profile.setPreference("network.proxy.ssl", "localhost");
profile.setPreference("network.proxy.ssl_port", 8105);
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.no_proxies_on", "");
//profile.setProxyPreferences(proxy);
profile.setPreference(key, value)
capabilities.setCapability(FirefoxDriver.PROFILE,profile);
capabilities.setCapability(CapabilityType.PROXY, proxy);
WebDriver driver = new FirefoxDriver(capabilities);
driver.get("http://www.google.com");
Har har1 = server.getHar();
}
}
I'm new to selenium and broswermob. This is my code. Whe I try to execute this and getting error The method setProxyPreferences(Proxy) is undefined for the type FirefoxProfile. How to solve this?
You (generally) don't need to configure the FirefoxProfile settings manually. The Using With Selenium section of the readme file gives an example of how to use the proxy with Selenium:
// start the proxy
ProxyServer server = new ProxyServer(4444);
server.start();
// get the Selenium proxy object
Proxy proxy = server.seleniumProxy();
// configure it as a desired capability
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
// start the browser up
WebDriver driver = new FirefoxDriver(capabilities);
// create a new HAR with the label "yahoo.com"
server.newHar("yahoo.com");
// open yahoo.com
driver.get("http://yahoo.com");
// get the HAR data
Har har = server.getHar();
If that doesn't work, it may be a problem with your specific version of Firefox and/or Selenium and/or BrowserMob Proxy. What versions are you using? A stack trace with the exact error message would also help.
Try removing profile from capabilities and passing it directly to FirefoxDriver:
driver = new FirefoxDriver(new FirefoxBinary(),profile,capabilities);