I am a beginner at automation testing.
I am trying to execute tests that were written in java using Selenium WebDriver, Cucumber with Gherkin annotations in JMeter.
When I run my tests without JMeter, I just use the opportunity to run them as Junit test using the runner class (in Eclipse).
To run my tests in JMeter, I created a jar file, uploaded it to Junit sub-folder, all dependencies-jars were uploaded to lib sub-folder.
I tried to create a JUnit request in JMeter, but it didn't help as when we use Cucumber, we don't mark methods with #Test. I have just an empty constructor for my Runner class (as you can see from the picture above).
I tried to add my jar to Test Plan and create an instance of my class in BeanShell, but it is not executed as JUnit tests.
My question is how to run java test scripts that were written based on Cucumber with Gherkin (based on Selenium WebDriver)?
Maybe I have to use other Samplers...
Any examples are greatly appreciated.
I believe you need to run another class, in particular io.cucumber.core.cli.Main, something like:
io.cucumber.core.cli.Main.main(new String[]{
"--glue",
"the package which contains the glue classes",
"/your/feature/file"});
Check out Running Cucumber -> From the command line for all the available arguments explained. By the way, you can use OS Process Sampler for this, it will be way easier.
Depending on what you're trying to achieve it might be faster and easier to convert your Selenium tests into "native" JMeter ones as if you're going to use your Selenium tests for creating the load it will require immense hardware resources and you won't get performance metrics on the HTTP protocol level.
Related
I'm trying to make the integration between JUnit, Jenkins and TM4J (Jira) using this tutorial: https://support.smartbear.com/tm4j-cloud/docs/api-and-test-automation/junit-integration.html.
The problem is that the tm4j_result.json file is NOT generated, although the result from the Cucumber integration is being generated successfully.
Is it possible to generate BOTH Cucumber and JUnit reports?
This is the Jenkins reference, which contains the exact same file pattern example and the bitbucket code in the reference shows only
adaptivist library and surefire listener
To answer your question : "Is it possible to generate BOTH Cucumber and JUnit reports?"
I would answer there is no asking this, since from your automated tests project, you will "either" work with BDD, hence use the cucumber file, or, work with non-BDD; which will in this case resort to the Junit file.
Should you have both, a solution using both will be needed at a pipeline level and a double test execution command will be deemed necessary to keep things reasonably simple.
There is no mix of BDD and non BDD all in the same test execution command, to put it simply !
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
I am trying to get java source code coverage after testing with a GUI based functional test using selenium.
what all source (actual application) code( NOT selenium test code) was executed or covered after running the selenium functional tests in java.
Is this can be achieved using Emma or jacoco or any other?
Anyone who has done this could please give me an idea to start this ?
Our Java Test Coverage tool can do this.
The process is straightfoward:
Apply the Test Coverage tool instrumenter to your source code and compile.
Run the application, exercising it any way you desire (including running selenium, or hand interaction, or unit tests)
Cause the application to exit from main.
The instrumenter's code inserted in main will write coverage data
Display/explore the test coverage with the tools GUI interface
You may have a long-running (server) application that doesn't naturally exit; in that case you can't get the test coverage dumped by exiting main because you didn't exit. In this case, you add a call to the tool's DumpTestCoverage() method, at some place in your application that you can cause to be executed by some interaction with the application. Then when you want a snapshot, simply exercise that feature.
I always use Jacoco plugin with Gradle. It cool in a way that it generates report and makes everything simple. You can configure it in build.gradle file.
I am really new to Fit/Fitnesse and, in general, to test automation.
I am trying to use them from Eclipse.
I have several question about it:
is there a way to obtain the html tables that Fitnesse pass to Fit?
once I write several tests with Fitnesse, is there a way to call them several times from Java without clicking on the Test button of the wiki?
About passing objects from one table to another in a flow. I read about symbols but it seems that, in java, they works only with ColumnFixturewhile I would like to use DoFixture. how to do this?
Finally,is if there is any plugin for eclipse you suggest to use with Fit/Fitnesse?
Regarding you question 2: I would recommend using the JUnit integration (#RunWith(FitNesseRunner.class) to run the test page (or a suite) as a unit test from Eclipse. This also gives you the ability to debug inside your fixture code.
It takes a bit of configuration to get it running 'just right'. In my pre-packaged FitNesse I provide a unit test FixtureDebug where you only have to enter the test name (and you can also use that to run your tests on a build/continuous integration server).
What are the possible ways to report or capture the result?
Or should we keep on staring at the screen when the automation script runs?
Thanks in advance!
You can get the test report using testng.
First install testng by following this link
After installation of testng, write some test cases in a class using #Test
annotation. Then click on run and run as a testng test. Then refresh your project and in test-output folder, you will find the testng report.
If you google, you will find a lots of links about this.
You can go through this link
Selenium (like other browser automation frameworks) is most commonly used to write automated acceptance tests within a test framework such as Cucumber-JVM. I mention Cucumber-JVM since it's designed for acceptance testing and is the most common such framework used with Java applications, but you could use JUnit or TestNG or another test framework if you wanted.
In any such framework, within each test, Selenium or another browser driver simulates user actions in the UI and statements written in the test framework's API/DSL assert that the expected result occurred. The test framework runs all of the tests and reports any which failed.
It is still often useful to watch the screen while debugging a failing test, however.
you can take screenshots of the screens, then you can create PDF's/doc's if you need. to capture screenshot you the command
screenShotImage = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
use screenShotImage.getAbsolutePath() to get file path.
Test results are always linked to project needs, i have seen scenarios where test results should be in PDF.
Here are some suggestions on test results
Extent reports - This is the java library contains graphs as well as
detailed steps. This even allows screenshots
If you are using TestNG or Junit, it has default reporting. However if you are looking for more advanced graphical report, use
RealTimeReport
if you link the results in documentation, create a custom report code