I am currently working on a Maven Project, using JUnit for defining tests and Jenkins for CI and am looking into how I can group my tests.
Say I had a test class with 20 tests, but I don't want to run all 20 tests, I want to be able to configure which tests to run. For Example, in another standalone project using TestNG and Selenium you can create a test method with the following annotation:
#Test (groups = { "AllTest" })
public void myTestMethod()
{
.. do something
.. assert something
}
... and then I am able to call which group to run based on an XML configuration.
Is it possible to define such type of groupings using Jenkins? I have researched into this and came across the plugin "Tests Selector Plugin" however can't understand how to get started once I've installed the plugin. There is a Wiki Page for it but I can't understand what to do after installing.
I have copy pasted the example property file, and didn't really understand what I needed to manipulate in it. When building, I simply get that the property file cannot be found or Jenkins doesn't have permission; can't find a way around this either :(
It's possible via maven + maven-surefire-plugin
http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html
You can run a single test, set of tests or tests by regexp.
Related
I'm migrating to Junit5 from 4. Can run all tests fine, single tests fine (both via the green play buttons). When I try to run tests tagged with #Tag (#Tag("foo")), it tells me that no tests are found. In the run configuration, I'm putting foo in the "Tag expressions" box, using Test kind: Tags. I've used Categories with Junit4 in the past, and Tags seem like a 1-to-1 replacement, right? I do still have some Category tags in my codebase, as I'm switching over. Several tests tagged with #Tag as well. Whether the #Category tag is present or not makes no difference in the behavior.
IntelliJ result:
Running categories [interface org.junit.Test]
Process finished with exit code 0
Use ˋorg.junit.jupiter.api.Testˋ instead of ˋorg.junit.Testˋ to run JUnit5 tests. Only those can handle the ˋTagˋ annotation.
I am running some tests on an Android phone using gradle. However I would like to be able to select which tests to run. For the tests I am using jUnit4 and categories.
This is how the tests are build and executed from jenkins:
call gradle assembleDebug assembleDebugAndroidTest
call gradle connectedDebugAndroidTest
This is how a test looks like:
#Category(IncludeTest.class)
#Test
public void test_Test_06() throws Exception {
TestData.setUpTest("test_Test_06");
Log.d("Test: Test 6 included");
}
#Category(ExcludeTest.class)
#Test
public void test_Test_07() throws Exception {
TestData.setUpTest("test_Test_07");
Log.d("Test: Test 7 excluded");
}
In my gradle.build I have tried the following without success:
test {
useJUnit {
includeCategories 'com.abc.def.IncludeTest'
excludeCategories 'com.abc.def.ExcludeTest'
}
}
My structure is as follows:
/someFolder/gradle.build
/someFolder/app/src/android/java/
In java i have a package named com.abc and in that package there is another package, def where my IncludeTest and ExcludeTest interfaces are.
I have tried different paths to Include/ExludeTest in gradle.build but it just does not work, all test all always executed.
I have also tried putting the includeCategories/excludeCategories in a task and made sure the task was actually started. But still all test were executed. Just seems like includeCategories/excludeCategories does not do anything.
Is there anything basic I am doing wrong? Are there any other ways of selecting categories?
After some more research I found out that the includeCategories/excludeCategories does not work with Android.
I found a different solution when using AndroidJUnitRunner. In Gradle it is possible to filter tests on annotations. To include annotations:
call gradle connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.annotation=com.abc.def.IncludeTest
There is also the possibility to exclude annotations using notAnnotation instead.
And of course there is syntax if you want to use multiple annotations with OR/AND combinations.
The following is also possible if you are not using connectedDebugAndroidTest:
adb shell am instrument -w -e annotation com.android.foo.MyAnnotation com.android.foo/android.support.test.runner.AndroidJUnitRunner
Some documentation: https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner.html
How does TestNG work when it is looking for its test classes?
I created a separate project from my TestNG project using JavaFX to create a GUI, when the GUI is run it enables you to select an XML file and then the path to the testNG XML is saved in a variable.
When I run the following code:
String xmlFileName = selectedFile.getAbsolutePath();
TestNG testng = new TestNG();
List<String> testFilesList = new ArrayList<String>();
testFilesList.add(xmlFileName); //test suite resides in the working directory's root folder
testng.setTestSuites(testFilesList); //you can addd multiple suites either here by adding multiple files or include all suites needed in the testng.xml file
testng.setUseDefaultListeners(false);
testng.addListener(htmlRep);
testng.run();
I get an error saying that it cannot find my test classes inside the XML.
[TestNG] [ERROR]
Cannot find class in classpath: com.emc.qe.u360.tests.LogoutTests
I then decided to make a java class within my TestNG project, and copy the code over and it still gives the same error message.
Where is TestNG looking for the test classes when the above code is run?
What difference is the code above from manually running the XML from eclipse itself? The code seems to be working, as from what I can see if the code for running the XML was broken or not working, it wouldn't be able to give me that error as it wouldn't be able to determine what is specified in the XML.
Any help would be appreciated, thank you.
EDIT: Just for background information, currently I am using Jenkins to run the tests, what I'm trying to do with the GUI is to create an alternative solution to running the tests outside of the Project/Framework, that can be used universally, a user can just select their XML and then the tests will be executed.
It's configurable if you use the Maven Surefire or the Gradle TestNG runner, but by default I think TestNG looks in src/test/java for your class names. Additionally, I think if your running TestNG from code, as a standalone run, you might be able to adjust the path-base where it looks for classes.
So I edited the name of a JUnit test and now it wont work. Instead I get Unrooted Tests: initializationError.
This is a simple test. Infact it is a test for JUnit tests as I am just starting to use it.
#Test
public void testRun()
String s = null;
assertNull(s);
}
and all i did was change it to testRun2(). Also when I run the file not the individual test, it still runs the old testRun(), not testRun2().
My project has Maven not sure if that is a factor. And I have updated the project
So it turned out that I needed to rebuild using Maven to update the classes. Now it works fine and I can add/modify test cases.
In my case, i changed the method name and it didn't update it automatically, so the above solution of Project> Clean worked well for me.
Another way this error would occur is forgetting the Test annotation. Encountered when right click method name in Eclipse and Run As -> Junit Test.
I have a java project in eclipse, when I press the project right click -> run as junit some tests do not run. I attached a picture, see YamiMailSenderTest for example.
When I try to run the tests directly they are running.
I am using eclipse 3.7.2.
and expanded view:
Any idea?
Ran into the same problem, my error was that I wrote: public void myMethodName(){ //assertions }
instead of: public void testMyMethodName() { //assertions }
the test before the MyMethodName is important.
It's a bit late, but in case anyone finds this via a search engine:
If a Test is run multiple times the results provided by JUnit are indistinguishable for those Tests and thus the results are only displayed for one run. See also the following Eclipse bug report: https://bugs.eclipse.org/bugs/show_bug.cgi?id=172256
Check if you are excluding tests from run by attributes and check under Run > Run Configurations if your JUnit configuration are excluding any tests.
In jUnit 4, a test case needs to have #Test annotation. The test case can be set to ignore with #Ignore annotation. The whole test class can also be set to ignore by placing the #Ignore annotation right above the class declaration. Note: In jUnit 4 , there is no need to extend Testcase class as in jUnit 3. Everything is in annotation.
I have no idea about jUnit 3 since I use only 4.
I had a similar problem. For some reason, the "Run As -> jUnit Test" was always skiping the first test package. I was on an older version of Eclipse and SpringSource.
I moved back to Juno - Version: 4.2.1 and all my test run when I perform: "Run As -> jUnit Test. "
I had the same problem. Eclipse would only recognize and run 5 out of my 9 tests. After much troubleshooting I found this trick to convince Eclipse to recognize the remaining tests: just open each file, hit space and then backspace to mark it as changed, and save it. Then, Eclipse will recognize it as a test.