I want to develop a stand-alone test-solution delivered as a jar that can be used in a CI/CD environment without being recompiled all the time. Therefore I packed a fat-jar from a multi-maven-module containing a few libaries, a Spring Boot application and a submodule called test-runner.
Executing the fat-jar from within GitLab CI/CD works, but I think that was only the first half of it. I want to produce a JUnit XML report to output the test-results. What I understood from my research is that I would have to implement my own reporter. Is there a more complete example out there?
The test runner
public class Runner {
SummaryGeneratingListener listener = new SummaryGeneratingListener();
public void runOne() {
LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
.selectors(selectClass(MyTest.class)).build();
Launcher launcher = LauncherFactory.create();
TestPlan testPlan = launcher.discover(request);
launcher.registerTestExecutionListeners(listener);
launcher.execute(testPlan);
}
public static void resultReport(Result result) {
System.out.println("Finished. Result: Failures: " + result.getFailureCount() + ". Ignored: "
+ result.getIgnoreCount() + ". Tests run: " + result.getRunCount() + ". Time: "
+ result.getRunTime() + "ms.");
}
public static void main(String[] args) {
Runner runner = new Runner();
runner.runOne();
TestExecutionSummary summary = runner.listener.getSummary();
summary.printTo(new PrintWriter(System.out));
}
}
Background:
My test-solution is generic and uses a configuration file to parameterize the tests. All tests run in parallel versus a system-under-test. So before this attempt all gitlab-jobs called mvn test to execute the tests and generate the reports, but it recompiled everything every run. I thought about speeding things up.
To generate XML reports, you can use the LegacyXmlReportGeneratingListener with a path to save your reports to as first argument:
LegacyXmlReportGeneratingListener xmlListener = new LegacyXmlReportGeneratingListener(Paths.get("reports"), new PrintWriter(System.out));
In your runOne() method, you need to register your listener accordingly:
public void runOne() {
LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
.selectors(selectClass(MyTest.class)).build();
Launcher launcher = LauncherFactory.create();
TestPlan testPlan = launcher.discover(request);
launcher.registerTestExecutionListeners(listener);
launcher.registerTestExecutionListeners(xmlListener);
launcher.execute(testPlan);
}
This will generate one XML file per test root in the folder you passed to the listener during initialization.
More information can be found in the JavaDoc
You can use Console launcher to geneate Junit5 xml reports
java -jar junit-platform-console-standalone-1.6.2.jar #junitArgs.txt --reports-dir=reports
junitArgs.txt file has following info:
-classpath fat jar path
--scan-classpath
Related
I have an automation test that I use Cucumber, Junit and I can run with Java Application.
Runner Class:
#RunWith(Cucumber.class)
#CucumberOptions(strict = false, features = "src/main/java/FaixaVS_NaoCadastrado/FaixaVS_NaoCadastrado/FaixaVS_NaoCadastrado.feature", glue = { "StepDefinition" }, format = { "pretty",
"json:C:/Automação Receba Em Casa/Evidências/FVS_NaoCadastrado/Relatório/cucumber.json" }, tags = { "~#ignore" })
public class Runner {
}
Jar Class:
public class Jar {
public static void main(String[] args) {
Result result = JUnitCore.runClasses(Runner.class);
for (Failure failure : result.getFailures()) {
System.out.println(failure.toString());
}
}
}
I can run it inside Eclipse as Java Application, but when I Export it like a Runnable jar and run it I receive the follow massage from CMD:
C:\Users\c.guiao.de.oliveira>java -jar
C:\Users\c.guiao.de.oliveira\Desktop\FVS_NaoCadastrado.jar
initializationError(FaixaVS_NaoCadastrado.FaixaVS_NaoCadastrado.Runner):
Expected a file URL:rsrc:cucumber-java-1.2.3.jar
Can you help me?
Few options:
Verify that the path you are using to get resource is correct. I
encounter tests that IntelliJ (that is not case sensitive when it
comes to resource path) executed successfully and maven clean
install failed.
Verify that cucumber dependencies are part of the jar. It can work in IntelliJ if you mistakenly added it to the classpath instead of adding it to the pom.xml file.
I have created a test automation framework using maven and cucumber.
1) I want to create a jar file which includes everything (all project files)
2) Then I want to run a test from the command line using above created jar like using the command
(mvn clean test -Dcucumber.options='--tags #all')
I don't want to use the main method or anything.
java -Dcucumber.options="--tags #all" -jar your-test-jar.jar
Try this. Although I am not sure why you don't want to use the main method. If you don't use the main method it will just become too complicated.
Update:
Write a main method and run Cucumber main method from it. The arguments are what you would pass in as your Cucumber command line arguments.
public static void main(String[] args) throws Throwable {
String[] arguments = {"a", "b"};
cucumber.api.cli.Main.main(arguments);
}
If I have understood your question clearly, this might do your work.
This should help you run Cucumber from your executable.
The below code worked for me to execute the cucumber tests from runnable jar with test frame work as TestNG.
Executing jar: java -jar ProductsAutomation-0.0.1-SNAPSHOT-jar-with-dependencies.jar
import io.cucumber.core.cli.Main;
public static void main(String args[]) throws Throwable {
try {
Main.main(new String[] {
"-g","com.sadakar.cucumber.common",
"-g","com.sadakar.cucumber.runner",
"classpath:features",
"-t","#SmokeTest",
"-p", "pretty",
"-p", "json:target/cucumber-reports/cucumber.json",
"-p", "html:target/cucumber-reports/cucumberreport.html",
"-m"
}
);
} catch (Exception e) {
e.printStackTrace();
System.out.println("Main method exception : " + e);
}
}
Background:
I am currently working on a project in eclipse that programatically executes JUnit tests that are pushed to a server.
So far everything works but I would like to know the results of the tests (specifically any failures) so I can push them out to an email. Right now the tests just output to the console but that doesn't seem to give me much output to actually use.
Right now I use the Runtime class to call the tests but that doesn't seem to have the functionality I need for getting results.
I have looked into the JUnitCore class but can't call any tests outside of the current java project.
So my main question would be how can I use JUnitCore to run junit tests in a specific JAR file? Or is there an easier way to approach this problem using a different class?
This is the only thing I've been able to get to work:
RunTests()
{
junitCore = new JUnitCore();
junitCore.run(AllTests.class);
}
But I would like to do something along the lines of this:
RunTests()
{
junitCore = new JUnitCore();
junitCore.run("C:\\$batch\\test\\hil research\\201507071307\\CommsTestRunner\\plugins\\TestSuite\\US35644.class");
}
I would appreciate any suggestions to this problem I am having. I'm an EE and was just introduced to java last month so this has been quite the challenge for me.
JUnitCore expects to read loaded classes, not class files in a JAR. Your real question is likely how to load the JAR (or directory of .class files) so it can be run.
Poke around with URLClassLoader; once you've amended the classpath appropriately, you can get a Class out of findClass and pass it into the JUnitCore methods you've found.
Since the tests might have classes that are also used by your server (but not necessarily at the same version) I would suggest not having your server directly run the tests. Instead, you can have your server start a new JVM that runs the tests. This is how IDEs like Eclipse run tests. You simply need to write a main class that has JUnit run the tests, and serializes the results on disk.
Your main class would look something like this:
public class MyRunner {
public static void main(String... args) throws IOException {
String path = System.getProperty("resultPath");
if (path == null) {
throw new NullPointerException("must specify resultPath property");
}
// Possibly install a security manager to prevent calls to System.exit()
Result result = new JUnitCore().runMain(new RealSystem(), args);
try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(path)) {
out.writeObject(result);
}
System.exit(result.wasSuccessful() ? 0 : 1);
}
}
Then your server simply needs to construct a java command line with the jars that include the tests, the JUnit jar file, and a jar that contains MyRunner.
I'm using maven (plugin version-1.7) and Aspectj-1.8.3.
My scenario is as follow: I have an installer jar that I want to test.
The installer is using another jar, my-common.jar library that wraps Apache's utility commons-exec-1.3 and use it to execute commands, the method that I wrote looks like this:
public static int execCommand(String command) throws ExecuteException, IOException, InterruptedException {
logger.debug("About to execute command: ", command);
CommandLine commandLine = CommandLine.parse(command);
DefaultExecutor executor = new DefaultExecutor();
executor.setProcessDestroyer(new ShutdownHookProcessDestroyer());
ExecuteWatchdog watchdog = new ExecuteWatchdog(ExecuteWatchdog.INFINITE_TIMEOUT);
executor.setWatchdog(watchdog);
DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();
executor.execute(commandLine, resultHandler);
return 0;
}
The problem is that since my test executes another jar,I mean, the installer, and the installer jar executes another one(lets name it app.jar) and than, the installer terminates and the app.jar keeps running (first, the installer is doing the installment and preparing the environment, and than, he executes the app.jar), the app jar is not being terminated when the testing suite is done (this was my intention and how it suppose to be in production environment).
The global target is to kill all of those processes being created under the integration tests suite.
My solution:
Since the process id's are exposed only to the java.lang.UNIXProcess I thought to collect all of the processes ids and then terminate them manually at the end of the tests suite.
I thought to put an Aspect like this:
static Collection<Integer> PidsToKill = new LinkedList<Integer>();
#After("!cflow(within(IntegrationTestsAspects)) && call(* java.lang.UNIXProcess.destroyProcess(int))&&args(pid)")
public void foo3(int pid) {
PidsToKill.add(new Integer(pid));
}
That was my idea to solve the problem without redesign some parts of my code.
So, to some it up, I'm looking for a way to make sure that all of the sub-processes being created under the integration tests suite are terminated.
Any solution is welcomed.
I have a .Jar file that will just load data into database when it's run. I have scheduled to run this job via Jenkins. When I execute the job in Jenkins it runs the .JAR successfully. However say if there is a null pointer exception in the job and it did not complete successfully. Even then Jenkins says that job has "Passed". How do I fail the job if there is an issue during the job execution?
#Corey's solution is good. And if you don't want to write a JUnit test and support it in Jenkins, you can just do what he alluded to earlier: catch the null-pointer exception (really, just have a top-level catch in your app), and call the API to exit with a return code:
try {
myCode.call();
catch (Exception e) {
System.out.println("An exception was caught at the top level:" + e);
System.exit(-1);
}
Last time I had this problem, I decided to take a different tack and changed the program call into a junit test. Jenkins was quite happy then.
Steps I took:
1. create an empty (maven) project
2. added a single java class SmokeTest.java
3. Added test that called the method I was testing via a script
4. Create a (maven) Jenkins job to run the project
Contents of my test:
public class SmokeTest
{
private static final String OK = "OK"; //$NON-NLS-1$
#Test
public void test()
{
// Create a new instance of the Firefox driver
final WebDriver driver = new HtmlUnitDriver();
final String url = PropertyManager.getInstance().getString(PropertyManager.SMOKE_TEST_URL_BASE) + "smoke/smoketest"; //$NON-NLS-1$
AuditLog.registerEvent("Smoke test url is: " + url, this.getClass(), AuditLog.INFO); //$NON-NLS-1$
driver.get(url);
// Find the text element by its id
final WebElement databaseElement = driver.findElement(By.id("database")); //$NON-NLS-1$
final String databaseResult = databaseElement.getText();
Assert.assertEquals(SmokeTest.OK, databaseResult);
//Close the browser
driver.quit();
}
}
The most important part here is the "Assert.assertEquals" line. The result of this is pickup by jUnit and therefore jenkins
Jenkins jobs fails if the exit code is anything but zero.
System.exit(1);
Should work (or fail, to be more precise :-)