Is it possible to perform some high-level operation at Unit testing end?
Like grabbing all test results and logs and sending them by email to developer?
Would like solution either in JUnit or with Gradle.
A pure JUnit approach could be to implement your own RunListener to be able to grab the information of each of your tests.
To invoke your listener, you need to run your tests through JUnitCore.
public void main(String... args) {
JUnitCore core= new JUnitCore();
core.addListener(new MyRunListener());
core.run(MyTestClass.class);
}
A pure Gradle approach could be to implement your own TestListener to be able to grab the information of each of your tests and send the final report.
In build.gradle you will need to add gradle.addListener(new my.package.MyTestListener()). Your listener must be in the build source directory also known as buildSrc.
test.finalizedBy(someHighLevelOperationTask)
Example: Here is a task that you can call gradle emailTestResults that would depend on 'zipTestResults' which depends on test. I have not done the email part of it, there's a question that covers that: Sending email using Gradle
So this just spits out a println in place of the email task:
apply plugin: 'java'
task zipTestResults(dependsOn: 'test', type: Zip){
from 'build/reports/tests'
baseName 'TestReport'
destinationDir file('build/reports')
}
task emailTestResults(dependsOn: 'zipTestResults') << {
println 'Emailing...' + file('build/reports/TestReport.zip')
}
repositories{
jcenter()
}
dependencies{
testCompile 'junit:junit:4.12'
}
But that won't work if the test fails, so, there maybe a better way, but you can add this:
test.ignoreFailures = true
Then if you then always want to email the test results add:
test.finalizedBy(emailTestResults)
Then a gradle test would finish by emailing the results.
Related
I am working with Gradle 7.1, and I am trying to write some of the tasks resuts into a file.
Specifically, I would like to write the output of dependencies task into a file after each jar task execution.
Looking for some solutions, I understand that at first I need to have jar.finalizedBy(dependencies) in order fot it to run.
However, I can't find how to redirect the dependencies's specific output into a file. All the solutions that I have found discuss Exec tasks, which dependencies isn't.
I am looking for somehing like dependencies.doFirst(///REDIRECT HERE).
You can make dependencies task write to file by attaching a StandardOutputListener:
tasks.named('dependencies').configure {
it.logging.addStandardOutputListener(new StandardOutputListener() {
#Override
void onOutput(CharSequence charSequence) {
project.file("$buildDir/dependencies_task_output.txt") << charSequence
}
})
}
This can also be done with any other Gradle task.
I'd like to publish a library with two different API versions where both use the same core code underneath. I tried shading/shadowing but have struggles getting the visibility right (I'd like to hide the core code from the API user). So I wanted to achieve my goals by having different source sets and configurations:
sourceSets {
// the `main` source set acts as the common code base for `api` and `api2`
api {
java {
srcDir 'src/api/java'
// Includes classes from `main`:
compileClasspath += sourceSets.main.output
runtimeClasspath += sourceSets.main.output
}
}
api2 {
java {
srcDir 'src/api2/java'
// Includes classes from `main`:
compileClasspath += sourceSets.main.output
runtimeClasspath += sourceSets.main.output
}
}
}
configurations {
common {
canBeResolved = true
canBeConsumed = false
}
// These art the configurations used both for being consumed with `project(...)` or published:
exposedApi {
canBeResolved = true
canBeConsumed = true
extendsFrom common
}
exposedApi2 {
canBeResolved = true
canBeConsumed = true
extendsFrom common
}
}
task apiJar(type: Jar) {
group = 'build'
from configurations.exposedApi
baseName = 'api'
}
task api2Jar(type: Jar) {
group = 'build'
from configurations.exposedApi2
baseName = 'api2'
}
publishing {
publications {
api(MavenPublication) {
artifact apiJar
artifactId 'mylib-api'
}
api2(MavenPublication) {
artifact api2Jar
artifactId 'mylib-api2'
}
}
}
dependencies {
common sourceSets.main.output
exposedApi sourceSets.api.output
exposedApi2 sourceSets.api2.output
}
If I want to use one of these APIs I can easily use project(path: ':mylib', configuration: 'exposedApi2') or use one of the published Maven artifacts and it works nicely.
But as soon as I change classes in the main source set to internal in order to achieve proper encapsulation of the main code, the API code won't compile anymore:
Cannot access 'SomeClassInMain': it is internal in '' (<-- yes, it really shows nothing in the '')
I also tried to merge the source set into one, so there is technically not really a main source set anymore:
sourceSets {
api {
java {
srcDirs('src/api/java', 'src/main/java')
}
}
api2 {
java {
srcDirs('src/api2/java', 'src/main/java')
}
}
}
That now works all as intended, no compilation errors, calls from the API to main work as expected and the classes in main even have internal visibility. But unfortunately IntelliJ seems to not pick up the fact that classes in main are really part of the same source set. I get an error (Unresolved reference: SomeClassInMain) in the IDE every time I mention a class from the main sources and of course no auto-completion would work, too, making the solution somehow not really practical in the end.
So just to sum up the goal:
it's important that the main sources are accessible to the API
but not to the user using the API (or the Maven publication) – the only thing the user should be facing is the API
If possible, I'd like to not put the API and main code in separate modules and publish them separately for encapsulation reasons
I tried a shading/shadowing (fat/uber JAR) approach but I haven't managed to reduce the visibility to internal in the main sources
I'm new to the topic of these complicated kinds of build configurations so maybe I simply have chosen the wrong approach. Maybe there's a better one which I haven't yet managed to find?
Many, many thanks in advance!
So i have a gradle test task which runs all my tests. How can I set up gradle to run this task 100 times? It works and runs all my tests, I just need an option to choose how many times to run this.
The task in build.gradle:
test {
// enable JUnit Platform (a.k.a. JUnit 5) support
useJUnitPlatform()
// set a system property for the test JVM(s)
systemProperty 'some.prop', 'value'
// explicitly include or exclude tests
include 'com/company/calculator/**'
// show standard out and standard error of the test JVM(s) on the console
testLogging.showStandardStreams = true
// set heap size for the test JVM(s)
minHeapSize = "128m"
maxHeapSize = "512m"
// set JVM arguments for the test JVM(s)
jvmArgs '-XX:MaxPermSize=256m'
// listen to events in the test execution lifecycle
beforeTest { descriptor ->
logger.lifecycle("Running test: " + descriptor)
}
// Fail the 'test' task on the first test failure
failFast = true
// listen to standard out and standard error of the test JVM(s)
onOutput { descriptor, event ->
logger.lifecycle("Test: " + descriptor + " produced standard out/err: " + event.message )
}
The use case is that i want to test performance of different assertions and mocking libraries (i have multiple branches with tests written using different libraries), to do that i need to run test suite multiple times.
To test performance i need to measure the time it takes to run these tests for example 100 times (maybe 1000), on each libraries set.
One option might be this --rerun-tasks flag.
gradle test --rerun-tasks
From the Gradle user guide.
Another option, from a similar question, is creating a subclass of the Test class that returns a task with multiple copies of all tests, code here: https://stackoverflow.com/a/41650455/1686615 .
There really are many ways to do this at different levels, with Gradle code as in that link, or perhaps in .gradle files, with a parameter passed into the test code, or on the command line. Maybe indicate more about your use case or if there is a particular level at which you'd like to make the change.
I'm trying to exclude a 'quarantine' folder that I set up for Selenium tests that need to be updated and I do not wish to have run. I know that one solution is to set up and assign test groups for the tests in these classes but given the sheer size and volume of tests that will be in here, I'd rather do it using an Ant-style filter.
Here is a snippet of my build.gradle file:
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'eclipse'
repositories {
mavenCentral()
}
dependencies {
compile "org.seleniumhq.selenium:selenium-java:2.35.0"
compile "org.testng:testng:5.14.10"
testCompile('org.uncommons:reportng:1.1.2') {
exclude group: 'org.testng'
}
testCompile "junit:junit:4.8.2"
compile "com.jayway.restassured:rest-assured:1.8.1"
}
//initialize thread count variable for parallel testing and default to 1
def threadCount = System.getProperty("MAXTHREADS", "1")
tasks.withType(Test) {
maxParallelForks = 1
forkEvery = 1000
ignoreFailures = false
// Pass all system properties to the tests
systemProperties = System.getProperties()
// Makes the standard streams (err and out) visible at console when running tests
testLogging.showStandardStreams = true
exclude '**/tasks/'
exclude '**/disabled/'
classpath += configurations.testCompile
}
task firefox(type: Test) {
maxParallelForks = Integer.valueOf(threadCount) //default is 1 if not specified
testLogging.events "started"
testLogging {
events "started", "passed", "skipped", "failed", "standardOut", "standardError"
exceptionFormat "full" // default is "short"
}
useTestNG() {
excludeGroups 'chrome'
useDefaultListeners = false
listeners << 'org.uncommons.reportng.HTMLReporter'
listeners << 'org.uncommons.reportng.JUnitXMLReporter'
listeners << 'com.xmatters.testng.Listener'
}
testResultsDir = file("${buildDir}/test-results/firefox")
testReportDir = file("${reporting.baseDir}/firefox")
systemProperties.BROWSER = System.getProperty('BROWSER', 'firefox')
exclude '**/selenium/'
exclude '**/setupscripts/'
}
task chrome(type: Test) {
maxParallelForks = Integer.valueOf(threadCount) //default is 1 if not specified
testLogging.events "started"
useTestNG() {
useDefaultListeners = false;
listeners << 'org.uncommons.reportng.HTMLReporter'
listeners << 'org.uncommons.reportng.JUnitXMLReporter'
listeners << 'com.xmatters.testng.Listener'
}
testResultsDir = file("${buildDir}/test-results/chrome")
testReportDir = file("${reporting.baseDir}/chrome")
systemProperties.BROWSER = System.getProperty('BROWSER', 'chrome')
exclude '**/selenium/'
exclude '**/setupscripts/'
}
On line 34 you can see exclude '**/disabled/' that I added. This folder is a couple levels up from the root folder. The preceding like with exclude '**/tasks/' was already in the build file and seems to work fine with a similar directory structure.
When I run the build, tests in the /disabled/ folder are still getting run. Is there something I'm doing wrong here? I'm assuming that with that syntax, a directory named 'exclude' a couple levels up would be ignored by scanForTestClasses which is true by default. Any idea what is up here?
One other thing I've noticed in Gradle test report is that the package name listed in the report is default-packagefor the excluded tests that are not 'excluding' whereas the other tests that are meant to be run are listing the correct package names. The package names in the Java files match their folder structure correctly so I'm not sure why this is being reported this way. I've checked for duplicates, typos, etc, and am not getting anywhere.
If anyone could shed some light on this that would be great as having these incomplete / broken test classes running is causing failures that should be ignored until these tests are updated.
These test are being run using the Gradle wrapper generated bash script on our test CI (Jenkins) box running on Linux.
Looks like the exclude pattern is applied to the relative path of the files (i.e. relative to your root folder), which explains why it works for folders under your root folder.
Using an excludeSpec (see Gradle Test task DSL) should work fine:
exclude { it.file.canonicalPath.contains('/disabled/')}
Of course, pay attention to / vs \ according to your OS.
This is probably more a Gradle question than a Caliper question, but I am still rather new to Gradle. I am interested in providing a task in my build that can run some benchmarks using Caliper. I have already added Caliper to my testCompile dependencies, and that works and pulls everything down. I would like to know how to provide a task that will actually run the benchmarks.
Btw, I already know about caliper-ci. I do have a Jenkins build, but it's on a cloud service that doesn't yet allow me to configure usage of caliper-ci, and besides, I want to be able to run locally before committing changes to the cloud.
It turns out it was a simple case of using JavaExec (I was new to that anyway):
task runBenchmark(type: JavaExec, dependsOn: test) {
def vmVer = System.getProperty('java.version')
def osName = System.getProperty('os.name').replaceAll('\\s','')
def osArch = System.getProperty('os.arch')
def fnameBase = "ver${version}_${osName}-${osArch}_jvm${vmVer}"
def benchMarksDir = "${project.buildDir}/benchmarks"
ant.mkdir(dir: benchMarksDir)
def outStream = new FileOutputStream("${benchMarksDir}/${fnameBase}-out.txt")
standardOutput = outStream
main = 'org.funcito.benchmarks.MyBenchmark'
classpath = sourceSets.test.runtimeClasspath
args = ['--saveResults', "${benchMarksDir}/${fnameBase}.json", '-Jmode=-server,-client']
}