We Keep Coding

Correct way to re-run the failed tests with Selenium-Cucumber in kotlin jvm

I am trying to re-run the failed functional tests for our application as I have seen these tests to be flaky.
Our setup is as follows
Platform / Stack - Kotlin 1.8.0 (JVM)
Webdriver - Selenium java:4.7.2
TestRunner - Cucumber Java8 (4.3.0) with Cucumber jUnit (4.3.0)
I have a TestSuite.kt which looks like below
import com.et.functionaltests.steps.WebDriverFactory
import cucumber.api.CucumberOptions
import cucumber.api.junit.Cucumber
import org.junit.AfterClass
import org.junit.runner.RunWith
#RunWith(Cucumber::class)
#CucumberOptions(plugin = ["html:report", "pretty", "rerun:target/rerun.txt"])
object TestSuite {
#JvmStatic
#AfterClass
fun teardown() {
WebDriverFactory.close()
}
}
As you can see I tried adding the "rerun:target/rerun.txt" in the plugins list after reading multiple solutions on the stack overflow I have also added another dependency
testImplementation("org.apache.maven.plugins:maven-surefire-plugin:2.22.2") following this article
I have also created a another class called FailedRunner.kt whose content is as below
import com.et.functionaltests.steps.WebDriverFactory
import cucumber.api.CucumberOptions
import cucumber.api.junit.Cucumber
import org.junit.AfterClass
import org.junit.runner.RunWith
#RunWith(Cucumber::class)
#CucumberOptions(features = ["#target/rerun.txt"])
object FailedRunner {
#JvmStatic
#AfterClass
fun teardown() {
WebDriverFactory.close()
}
}
So after all this when I run the test using the gradle command ENV=dev gradle clean test it runs FailedRunner first and then the TestSuit. I Would love to hear and learn more about fixing this and also proper way of doing this. Thanks in advance.

JUnit5 Test Suite doesn't test all classes from a package

I've got a Suite class as follows:
package test.suite;
import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.SuiteDisplayName;
import org.junit.runner.RunWith;
#RunWith(JUnitPlatform.class)
#SuiteDisplayName("JUnit Platform Suite Demo")
#SelectPackages("test")
public class Suite {
}
And two identical test classes (they differ only by their name at the moment) in the same package called test:
package test;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
public class testClass1 {
#Test
void mainA() {
assertTrue(42==42);
}
When I hit run on 'Suite', only testClass1 will be tested and testClass2 is left out. What am I missing?
The Suite should test classes even recursively from other subpackages in the given package if I'm not mistaken.
--EDIT:
Initially I created two test classes for the same class, and the second test class gets ignored. If I add a third test class in the package, but this time for a separate class, that will get tested as expected.

I was experiencing same issue. I had several files and in my case when i do run as suite some of them would be included and other would not. It was pretty frustrating as I can run them fine individually.
It turned out the one that were not included were those whose name did not end with Test. For example, ClassA wouldn't be added but once i renamed to ClassATest it got added when running as suite package feature.

Can't add the test classes in JUnit suite

I use Eclipse Oxygen and JUnit 5. I have a class with a method and 3 test cases for the method that work fine. When I try to create a JUnit test suite to group all the cases by new/other/Java/JUnit/Junit test case in the window for test suite nothing appears in "Test classes to include in suite" despite that everything is in one packet and even set to public. I create the test suite and manually type in the classes I want to include in the test suit.
package testing;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({oddNumberOfLettersPalindromeTest.class,
evenNumberOfLettersPalindromeTest.class, notAPalindromeTest.class})
public class AllTests {
}
When I run the AllTest suite I get only 1/1 runs for the same AllTest suite class. All my JUnit tests have #Test as well

Correct me if I wrong but do you want to add new classes every time foreach test? In that case you can use the #Before tag in JUnit. What it does is it generates a new class every time a #Test is hit.
private TestClass test;
#Before
public void setUp()
{
test = new TestClass();
//You can declare other classes right here
}

did you tried to follow this example ?
JUnit - Suite Test - Tutorials Point

It has to do with versions of JUnit or something, on my laptop it was JUnit 5 and eclipse oxygen, now I am on JUnit4 and eclipse Mars and everything is working fine. I just wanted to group few testcases in one suite

It looks like your test suite is not importing the test cases you want to trigger. Also, apparently your test cases are not following the class naming pattern (starting with lower case and so on).
package testing;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import OddNumberOfLettersPalindromeTest;
import EvenNumberOfLettersPalindromeTest;
import NotAPalindromeTest;
#RunWith(Suite.class)
#SuiteClasses({OddNumberOfLettersPalindromeTest.class,
EvenNumberOfLettersPalindromeTest.class, NotAPalindromeTest.class})
public class AllTests {
}
I believe by doing this it has no reason to work.

Try changing the Junit version to Junit4 in build path -> libraries.

Why do I get an error "package org.mockito.runners does not exist"?

I have pluged up require dependency
testCompile 'org.mockito:mockito-core:1.10.19'
Then I put my test code to /src/test/java/ directory
then I have tried launch such test
import org.junit.Test;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
public class PresenterActivityAcceptNotAcceptTest {
#Test
public void emailValidator_CorrectEmailSimple_ReturnsTrue() {
boolean dd = true;
assertThat(dd, is(true));
}
it works properly, but if I add anything witch associated with Mock lib
for example #RunWith
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.runners.MockitoJUnitRunner;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
#RunWith(MockitoJUnitRunner.class)
public class PresenterActivityAcceptNotAcceptTest {
#Test
public void emailValidator_CorrectEmailSimple_ReturnsTrue() {
boolean dd = true;
assertThat(dd, is(true));
}
I got such error
Error:Execution failed for task ':Application:compileDebugJavaWithJavac'.
> Compilation failed; see the compiler error output for details.
Error:(10, 10) error: cannot find symbol class MockitoJUnitRunner
Error:(5, 27) error: package org.mockito.runners does not exist
/home/aleksey/Downloads/NTZ/FittingRoom/Application/src/test/java/com/fittingroom/newtimezone/presenters/PresenterActivityAcceptNotAcceptTest.java
What am I doing wrong?
If I forger about something feel free to ask
Thanks in advance!

It looks like Gradle is not doing it's job.
Manually adding the jars may fixed the problem.
How to Download and Install jar go here .
and for download mockito use this link
https://mvnrepository.com/artifact/org.mockito/mockito-core/1.10.19

As stated in Baeldung's article, starting from Mockito version 2.2.20, the package for MockitoJUnitRunner has changed. So change :
import org.mockito.runners.MockitoJUnitRunner;
To :
import org.mockito.junit.MockitoJUnitRunner;
As usual, you have to import the mockito-core library in your build.gradle :
dependencies {
testImplementation 'org.mockito:mockito-core:4.2.0'
}
As a side note, you will also need to change import for Matchers if you use them. For instance :
From :
import static org.mockito.Matchers.any;
To :
import static org.mockito.Mockito.any;

Got the same issue. Attempted to implement the developer.android documentation example.
Fixed by changed org.mockito version to the recent at the time in build.gradle :
dependencies {
testImplementation 'org.mockito:mockito-core:2.28.2'
}

Open File > Project Structure...
and then add it manually as library dependency:

I also got this exception when I tried to write instrumentalTests in src/androidTest/java/ but by default mockito's scope in dependencies is set as Unit Test implementation (you can see that in File => Project Structure => Modules => Dependencies). I've just changed this parameter to Test implementation and it works!

does your project look like this?:

Is it possible to run JUnit tests from multiple packages in Eclipse?

Is it possible to run JUnit tests for multiple packages at the same time without manually creating test suites.
For example if I have the hierarchy:
code.branchone
code.branchone.aaa
code.branchone.bbb
code.branchtwo
code.branchtwo.aaa
code.branchtwo.bbb
Is it possible to:
Run all tests in code.branchone and in descendent packages
Run all tests in say code.branchone.aaa and code.branchtwo.bbb
The problem I see with manually creating test suites is that when new tests come along you may forget to add them.

Yes, it is possible. The easiest way for me at least is to add a test suite class. It can look like this:
package tests;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import tests.message.ATest;
import tests.validator.BTest;
import tests.validator.CTest;
import tests.validator.DTest;
#RunWith(Suite.class)
#SuiteClasses({ ATest.class,
BTest.class,
CTest.class,
DTest.class })
public class AllTests {
}
This will allow you to test any class that you import no matter what package it is in. To run this in eclipse you just right click the AllTests class and run it as JUnit test. It will then run all the tests you define in #SuiteClasses.
This will work with linked sources as well, I use it all the time.

An other way:
Click on the black triangle denoted by red rectangle in the picture below (in your Eclipse, not here :).)
Then open run configurations, create a new configuration and then set "Run all tests..." as exemplified in the image below.

Maybe not exactly what the original question was, but you can easily run all tests of a whole Project, by simply right-clicking the project -> Run As JUnitTest. Don't worry where the annotated classes reside, this will be scanned.
This does not work if applied to the test-src-folder or a package with subpackes. Quite a shame actually -.-

I'm sure u can tweak this a bit. Make a Collection of the CLASSES_DIR property and loop over it in the findClasses method. (junit4)
http://burtbeckwith.com/blog/?p=52

I beleieve that you can add all your test packages to a single directory. If you right click on this directory, then you should find the "run as -> JUnit test" option available. This will run all tests contained in the directory and will catch anything you've added. Any new tests get put in there with the rest of them and whatever package name you have it doesn't matter. Hope that helps

Sure, right-click on the packages you want, and select Run As... JUnit Test

In Eclipse, on your debug/run configurations you have the following options:
Run a single test
Run all tests in the selected project, package or source folder
I think the second option is your friend in this case.

If you are using JUnit 5 you can cherry pick which tests you want to run:
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;
import tests.Test1;
import tests.Test2;
import tests.Test3;
#Suite
#SelectClasses({
Test1.class,
Test2.class,
Test3.class
})
public class AllMyTests {
}
If you want to select which packages to run you can:
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;
#Suite
#SelectPackages({
"com.package1",
"com.package2",
"com.package3"
})
public class AllMyTests {
}
If you need to be able to exclude certain sub-packages you can do:
import org.junit.platform.suite.api.ExcludePackages;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;
#Suite
#SelectPackages({
"com.package1",
"com.package2",
"com.package3"
})
#ExcludePackages({
"com.package1.subpackage",
"com.package2.othersubpackage"
})
public class AllMyTests {
}
Other useful annotations:
To match a particular pattern:
#IncludeClassNamePatterns({"^.*ATests?$"})
To exclude a particular pattern:
#ExcludeClassNamePatterns({"^.*ATests?$"})
Examples were based on: https://howtodoinjava.com/junit5/junit5-test-suites-examples/