How to get .har file or network request using selenium4 - java

As we know One of the features added in the new version of Selenium (4.0.0-alpha-2) is a very nice Interface for Chrome DevTools API in Java.DevTools API offers great capabilities for controlling the Browser and the Web Traffic
As per documentation using the latest version of selenium we can capture the network request from the session.
Before I used browsermob for getting the network request but unfortunately they didn't update it a couple of years.
I am looking for someone who used this selenium4 dev tools API for getting all the internal request.
Can anyone suggest me how can I start to get all the requests? Thanks, advance

You can find #adiohana's example in the selenium-chrome-devtools-examples repo on gitHub.
I think youed find this test example helpful:
public class ChromeDevToolsTest {
private static ChromeDriver chromeDriver;
private static DevTools chromeDevTools;
#BeforeClass
public static void initDriverAndDevTools() {
chromeDriver = new ChromeDriver();
// dev-tools handler
chromeDevTools = chromeDriver.getDevTools();
chromeDevTools.createSession();
}
#Test
public void interceptRequestAndContinue() {
//enable Network
chromeDevTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
//add listener to intercept request and continue
chromeDevTools.addListener(Network.requestIntercepted(),
requestIntercepted -> chromeDevTools.send(
Network.continueInterceptedRequest(requestIntercepted.getInterceptionId(),
Optional.empty(),
Optional.empty(),
Optional.empty(), Optional.empty(),
Optional.empty(),
Optional.empty(), Optional.empty())));
//set request interception only for css requests
RequestPattern requestPattern = new RequestPattern("*.css", ResourceType.Stylesheet, InterceptionStage.HeadersReceived);
chromeDevTools.send(Network.setRequestInterception(ImmutableList.of(requestPattern)));
chromeDriver.get("https://apache.org");
}
You need to add the following imports:
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.Command;
import org.openqa.selenium.devtools.Console;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.network.Network;
import org.openqa.selenium.devtools.network.model.BlockedReason;
import org.openqa.selenium.devtools.network.model.InterceptionStage;
import org.openqa.selenium.devtools.network.model.RequestPattern;
import org.openqa.selenium.devtools.network.model.ResourceType;
import org.openqa.selenium.devtools.security.Security;
import java.util.Optional;

Related

Unable to navigate to website using Java Selenium ChromeDriver with ChromeOptions

I am trying to use Java Selenium's ChromeDriver to log in to my own google profile and navigate to a website. However, what I found out is that if I log in to my profile, I will not be able to navigate to any website. On the other hand, if I did not log in to my profile, I can navigate to the website.
I attach the code below for your reference, the code immediately below is able to log in to profile but unable to navigate to bing website.
package Package1;
import java.util.concurrent.TimeUnit;
import java.io.File;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriver.Options;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeDriverService;
import org.openqa.selenium.chrome.ChromeOptions;
public class MyClass {
public static void main(String[] args) {
// System Property for Chrome Driver
System.setProperty("webdriver.chrome.driver", "C:\\chromedriver.exe");
String userProfile = "C:\\Users\\XXXXX\\AppData\\Local\\Google\\Chrome\\User Data";
ChromeOptions options = new ChromeOptions();
options.addArguments("--user-data-dir=" + userProfile);
options.addArguments("--start-maximized");
WebDriver driver = new ChromeDriver(options);
driver.get("https://bing.com");
}
}
The code below is able to navigate to bing website, but for my usage I need it to have a profile.
package Package1;
import java.io.File;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriver.Options;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeDriverService;
import org.openqa.selenium.chrome.ChromeOptions;
public class MyClass {
public static void main(String[] args) {
// System Property for Chrome Driver
System.setProperty("webdriver.chrome.driver", "C:\\chromedriver.exe")
WebDriver driver = new ChromeDriver();
driver.get("https://bing.com");
}
}
I know I have imported some libraries that I did not use, that should not be the problem here. How should I edit my code to make it work?
Okay solved. If I have existing chrome browser opened then the code will not be able to log in to my profile and navigate to website. But if there are no instances of chrome browser opened prior to running the code. It will work.
Will appreciate it if anyone knows why this is the way?

Stringing together multiple page object files in Eclipse/Cucumber/Java

I am having trouble stringing together multiple Page Object Files and step definitions.
I have imported the following, which is super heavily edited due to it being work code:
package <the package my step definition file is in>
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import <the other project my team created>.browsercapabilities.Browser;
import <the other project my team created>.Browsers;
import <a package a different step definition file is
in>.addNewContact.AddNewContactStepDefs;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import <the other project my team created>.webcomponents.WebButton;
import <my page objects package>.authenticationpages.CommandLoginPage;
import <my page object package>ThirdPageObjectFile;
import <my page object package>.SecondPageObjectFile;
import <my page object package>.FirstPageObjectFile;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.Select;
import org.openqa.selenium.support.ui.WebDriverWait;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.When;
public class ThirdPageStepDefs extends BrowserBase {
private static final Logger LOG =
LoggerFactory.getLogger(ThirdPageStepDefs.class);
private WebDriver driver = mainBrowser().getDriver();
private Browser browser = Browsers.getBrowser();
private WebDriverWait wait = new WebDriverWait(driver,10);
LoginPage login = new LoginPage (browser);
SecondPageObjectFile contact = new SecondPageObjectFile (browser);
FirstPageObjectFile searchContact = new FirstPageObjectFile (browser);
ThirdPageObjectFile deal = new ThirdPageObjectFile (browser);
#Then("^I click the contacts icon$")
public void i_click_the_contacts_icon() throws Throwable {
contact.get_contact_link().clickLink();
}
I was getting an initialization error when I ran the runner code saying that the runner could not instantiate an instance of my ThirdStepDefFile, i.e. "Failed to Instantiate Class "
My runner is working fine:
package <Utility package>.utilities;
import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import cucumber.api.testng.AbstractTestNGCucumberTests;
#RunWith(Cucumber.class)
#CucumberOptions(
features="src\\test\\resources\\<my feature folder>",
plugin={"pretty", "html:target/cucumber-html-report",
"json:target/cucumber.json"},
glue= "<My step def folder>",
tags = "#<my tags>"
)
public class ComUITestRunner extends AbstractTestNGCucumberTests {
}
I have many more lines in scenarios that I've commented out and since I was getting the initialization error, I edited to troubleshoot as following:
-Comment all methods out (green)
-Leave all methods in (red)
-Leave one method in (red)
When I have even one method that isn't commented out, then I get an error.
My project is a series of screens that a user has to get to in order to log in. The login screen, first screen, second screen, and third screen all have their own page objects. Eventually I want to abstract those screens out to a main project that all sub projects share as they'll be pre-conditions to everything else. Stringing these screens together seems to fail and I'm considering refactoring the whole thing.
Am also new to Java/Eclipse, so it's possible that I am not seeing the stack trace correctly. For instance, the bottom of my junit screen says "62 more" lines detailing the failure that I'm not able to see. I tried debugging to no avail and I clicked on the outline on the left hand side of the junit output and when I hovered over it, it says Test Class Not Found

Glassfish & MongoDB connection error : NoClassDefFoundError

I am running a Glassfish server that is trying to connect to MongoDB. At first I created seperate projects for the server and MongoDB. So now I am trying to merge those projects but it appears anything I try to do it results in a faliure.
The current error I am getting is:
2018-07-05T19:54:36.249+0200|Severe: java.lang.NoClassDefFoundError: org/bson/conversions/Bson
I am well aware that the error happens in runtime and that the possible cause is my classpath.
Currently I copied all of my code from one project to another, added Maven dependencies and the following happens:
if I create a separate .java file for my MongoDB and run it in the same folder that the Glassfish server is, it works perfectly fine.
if I run the server and try to call methods from the other class (a little bit modified) the upper error appears
Simplified code example withouth error:
import org.bson.Document;
import org.bson.conversions.Bson;
import com.mongodb.BasicDBObject;
import com.mongodb.MongoClient;
import com.mongodb.client.FindIterable;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import com.mongodb.client.MongoIterable;
import com.mongodb.client.model.Filters;
import com.mongodb.client.model.Updates;
public class MyClass{
public static void main(String[]args){
String ip = "127.0.0.1";
int port = 27017;
MongoClient mongoClient = new MongoClient(ip,port);
/* Remaining code */
}
}
With error:
import org.bson.Document;
import org.bson.conversions.Bson;
import com.mongodb.BasicDBObject;
import com.mongodb.MongoClient;
import com.mongodb.client.FindIterable;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import com.mongodb.client.MongoIterable;
import com.mongodb.client.model.Filters;
import com.mongodb.client.model.Updates;
public class MyClass{
private MongoClient mongoClient;
public MyClass(String ip, int port){
mongoClient = new MongoClient(ip, port); // Error called here
}
/* Remaining code */
}
Called from the server.java file:
MyClass mc = new MyClass("127.0.0.1",27017);
I also tried to download all of the bson jar files separately and add them to the project but that had no effect...
The working solution for me was to delete the whole project and create it once more. Apparently there was a problem with Eclipse or I made a mistake before and forgot about it.

Apache Spark with elasticsearch V5.X

I am a novice in Java and I am looking for some examples of connector between elasticsearch V5.X and Spark in order to see some use cases.
At the moment here is my code :
package Spark;
import org.apache.hadoop.conf.Configuration;
import org.apache.log4j.Level;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.junit.Test;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import scala.collection.immutable.Map;
import twitter4j.Status;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.SparkConf;
import org.elasticsearch.spark.rdd.api.java.JavaEsSpark;
public class EsSpark {
public EsSpark(){
SparkConf conf = new SparkConf().setAppName("MyApp1").setMaster("localhost");
conf.set("es.index.auto.create", "true");
JavaSparkContext jsc = new JavaSparkContext(conf);
Map<String, ?> numbers = (Map<String, ?>) ImmutableMap.of("one", 1, "two", 2);
Map<String, ?> airports = (Map<String, ?>) ImmutableMap.of("OTP", "Otopeni", "SFO", "San Fran");
JavaRDD<Map<String, ?>> javaRDD = jsc.parallelize(ImmutableList.of(numbers, airports));
JavaEsSpark.saveToEs(javaRDD, "spark/docs");
}
}
Thanks.
Except if you are working with a local instance of Elasticsearch, there are some important settings to be provided, notably es.nodes.
You can do it using
conf.set("es.nodes", "eshost:9200");
You can even specify multiple instances, prefer master nodes, but not all nodes are required.
Please refer to official documentation.
People at discussion forum at elastic often publish some code you can use as example.
Ensure to provide several documents as the EsSpark or EsSparkStreaming objects. Do not send 1 document each time, prefer multiple documents.
EsSpark or EsSparkStreaming connect to the nodes you provide, they check for the cluster topology (number of nodes, types of nodes) and they will send data directly to data nodes and to the correct shard (avoiding hops).
It is possible to prevent to push data directly to the data nodes (using the settings specified in this section of the documentation), but you will introduce bottlenecks.

APPLICATION_JSON cannot be resolved or is not a field

I have a problem producing JSON in my application.
I'm trying a tutorial about Consuming Java Restful Web Service with AngularJS.
I've created a dynamic web project as my RESTful server and then added the following libraries:
asm-3.3.1.jar
jackson-core-asl-1.9.9.jar
jackson-jaxrs-1.9.9.jar
jackson-mapper-asl-1.9.9.jar
jackson-xc-1.9.9.jar
jersey-bundle-1.8.jar
jersey-bundle-1.9.jar
jersey-client-1.9.jar
jersey-core-1.9.jar
jersey-json-1.9.jar
jersey-media-json-jettison-2.4.jar
jersey-server-1.9.1.jar
jersey-servlet-1.12.jar
jettison-1.3.3.jar
jsr311-api-1.1.1.jar
Here is the service :
package ws;
import java.awt.PageAttributes.MediaType;
import java.util.*;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import entities.Prospect;
#Path("prospect")
public class ProspectRestful {
#GET
#Path("findall")
#Produces(MediaType.APPLICATION_JSON)
public List<Prospect> findAll(){
List<Prospect> result = new ArrayList<Prospect>();
result.add(new Prospect(35, "Name 35", "last35"));
result.add(new Prospect(36, "Name 36", "last36"));
return result;
}
}
But I'm getting this error:
APPLICATION_JSON cannot be resolved or is not a field.
I've seen in another question that jersey-media-json-jettison-2.4.jar should be there so I added it but no sign.
I'm sure I'm not using a Maven project, in the same tutorial they didn't use Maven either.
Remove this
import java.awt.PageAttributes.MediaType;
and add this
import javax.ws.rs.core.MediaType;
In my case it is not working, so if anyone facing the error then try...
import org.springframework.http.MediaType;
It worked for me...

Categories