We Keep Coding

How to extract pipeline dsl in the pipeline plugin with the Java?

I am developing a Jenkins pipeline plugin for CNB(buildpacks). I need to get the variables ​​in the pipeline script with Java but I still can't succeed.
This is the my pipeline script.
buildpacks {
builder = "some/builder"
}
And I can access these variables(like builder variable) ​​with Groovy language in the buildpacks.groovy
package dsl
// The call(body) method in any file in workflowLibs.git/vars is exposed as a
// method with the same name as the file.
def call(body) {
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()
try {
echo "${config.builder}"
} catch (Exception rethrow) {
throw rethrow
}
}
But as i said i need to get these variables in Java.
Below is my class that I inherited from the GlobalVariable class.
public abstract class PipelineDSLGlobal extends GlobalVariable {
public abstract String getFunctionName();
#Override
public String getName() {
return getFunctionName();
}
#Override
public Object getValue(CpsScript script) throws Exception {
Binding binding = script.getBinding();
CpsThread c = CpsThread.current();
if (c == null)
throw new IllegalStateException("Expected to be called from CpsThread");
ClassLoader cl = getClass().getClassLoader();
String scriptPath = "dsl/" + getFunctionName() + ".groovy";
Reader r = new InputStreamReader(cl.getResourceAsStream(scriptPath), "UTF-8");
GroovyCodeSource gsc = new GroovyCodeSource(r, getFunctionName() + ".groovy", cl.getResource(scriptPath).getFile());
gsc.setCachable(true);
System.out.println(gsc.toString());
Object pipelineDSL = c.getExecution()
.getShell()
.getClassLoader()
.parseClass(gsc)
.getDeclaredConstructor()
.newInstance();
binding.setVariable(getName(), pipelineDSL);
r.close();
System.out.println("test");
return pipelineDSL;
}
}
And below is my class that i created for my buildpacksdsl.
package io.jenkins.plugins.buildpacks;
import hudson.Extension;
import io.jenkins.plugins.pipelinedsl.PipelineDSLGlobal;
import org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.ProxyWhitelist;
import org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist;
import java.io.IOException;
#Extension
public class BuildpacksDSL extends PipelineDSLGlobal {
#Override
public String getFunctionName() {
return "buildpacks";
}
#Extension
public static class MiscWhitelist extends ProxyWhitelist {
public MiscWhitelist() throws IOException {
super(new StaticWhitelist(
"method java.util.Map$Entry getKey",
"method java.util.Map$Entry getValue"
));
}
}
}
If you want to see the structure in more detail, you can take a look at the repository.
Can someone help me ? Thanks.

We found a little solution.
We created an instance of a class using compatibility between Groovy and Java.
And since we can already get values ​​with Groovy, we can pass parameters directly in the constructor method.
There is probably a more efficient method. But now it's working.
// Buildpacks.groovy
...
import io.jenkins.plugins.buildpacks.pipeline.BuildpacksDSL.BuildpacksPipelineDSL
class Buildpacks implements Serializable {
// first executed method is similar to main method in java
public void call(final Closure body) {
// the config array is the array that holds the variables.
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()
// creating a new instance, when we give the 'config' array in the constructor, the variables is transferred.
BuildpacksPipelineDSL pipeline = new BuildpacksPipelineDSL(config)
pipeline.build()
}
}
...
// BuildpacksDSL.java
...
public static class BuildpacksPipelineDSL {
public BuildpacksPipelineDSL() {
}
/**
* This constructor takes dsl parameters, logger and envs from Jenkins and
* extracts them to local variables.
*
* #param c
* #throws Exception
*/
public BuildpacksPipelineDSL(LinkedHashMap<String, Object> c)
throws Exception {
// codes...
}
}
...

Use reflection to create classes at runtime

I have to create a list of objects, which are configured according to the name of some classes received as input.
For each object I have to call a method, which add an operation that is created dynamically.
However I don't know exactly ho to resolve the problem.
Please see an example below.
String className; // this is an input parameter
final Class<?> classType = Class.forName(className);
// here I would like to use classType instead of "?" but it gives me an error.
Task<?> task = TaskFactory.createTask((String)classType.getField("_TYPE").get(null)));
tasks.put(task, null);
task.addOperation(new Operation<classType>() { // this gives an error
#Override
public void onNewInput(classType input) { // this gives an error
System.out.println(input)
}
});

As you can see from the comments, the surrounding infrastructure and the intention are not entirely clear. However, you can achieve a certain degree of type-safety with a "helper" method that captures the type of the given Task, and allows you to work with this type internally:
public class RuntimeType
{
public static void main(String[] args) throws Exception
{
String className = "";
final Class<?> classType = Class.forName(className);
Task<?> task = TaskFactory.createTask((String)classType.getField("_TYPE").get(null));
addOperation(task);
}
private static <T> void addOperation(Task<T> task)
{
task.addOperation(new Operation<T>()
{
#Override
public void onNewInput(T input)
{
System.out.println(input);
}
});
}
}
class TaskFactory
{
public static Task<?> createTask(String string)
{
return null;
}
}
class Task<T>
{
public void addOperation(Operation<T> operation)
{
}
}
interface Operation<T>
{
void onNewInput(T input);
}

Junit and java classes.

The below is a simple java class file that checks if the file provided by the user is under the home directory or not. It throws an exception when the file is not under the home directory.
public class A {
public static void main(String args[]) {
if (new A().processArgs(args[0]) {
throw Exception("Not under home directory");
}
}
// A simple method to check if the file is at home directory
private boolean processArgs(String s) {
File f = new File(s);
String userHome = System.getProperty("user.home");
if (s.startsWith(userHome) && f.exists() && additionalLogic())
return true;
else
return false;
}
// Additional business Logic
private boolean additionalBusinessLogic() {
// Do wonderful things.
}
}
I want to write a simple Junit test case for testing the java class. Primary concern to test is the additional Business logic method. Is there a way I can bypass the check where directory must be under user home directory.
I am not comfortable in adding logic in my main class to make it aware of the Junit classes. Is there a better way to do this?

While there's nothing wrong with fab's solution, I decided to write another:
public class Main {
public static void main(String args[]) {
// TODO: Should check args length
Validator validator = new Validator();
validator.validateArgs(args[0]);
}
}
public interface Configuration {
public String getHomeDirectory();
}
public class DefaultConfiguration implements Configuration {
public String getHomeDirectory() {
String home = System.getProperty("user.home");
if (home == null) {
throw new RuntimeException("User home directory is not set!");
}
return home;
}
}
public class Validator {
private Configuration configuration;
public Validator() {
this(new DefaultConfiguration());
}
public Validator(Configuration configuration) {
this.configuration = configuration;
}
// A simple method to check if the file is at home directory
public void validateArgs(String s) {
File f = new File(s);
if (!s.startsWith(configuration.getHomeDirectory()) || !f.exists() || !additionalBusinessLogic())
throw new RuntimeException("Not under home directory!");
}
// Additional business Logic
private boolean additionalBusinessLogic() {
// TODO...
return true;
}
}
public class ValidatorTest {
#Test
public void validateValidArgsTest() {
final String homeDirectory = ".."; // TODO
String existingFile = homeDirectory + ".."; // TODO
new Validator(new Configuration() {
public String getHomeDirectory() {
return homeDirectory;
}
}).validateArgs(existingFile);
}
#Test(expected = RuntimeException.class)
public void validateInvalidArgsTest() {
String existingFile = ".."; // TODO
new Validator(new Configuration() {
public String getHomeDirectory() {
return "-INVALID PATH-";
}
}).validateArgs(existingFile);
}
}

You don't need to make the class aware of the test to make it more testable. You just need to decouple the additional logic from the i/o stuff, which will also result in a better design:
public class A {
private WonderfulThingsDoer wonderfulService;
public void main(String args[]) {
wonderfulService = new WonderfulThingsDoer();
if (processArgs(args[0]) {
throw Exception("Not under home directory");
}
}
// A simple method to check if the file is at home directory
private boolean processArgs(String s) {
File f = new File(s);
String userHome = System.getProperty("user.home");
if (s.startsWith(userHome) && f.exists() && additionalBusinessLogic())
return true;
else
return false;
}
// Additional business Logic
private boolean additionalBusinessLogic() {
return wonderfulService.doWonderfulThings();
}
}
public class WonderfulThingsDoer {
public boolean doWonderfulThings() {
// Do wonderful things.
return true;
}
}
Voilá, extracted a testable unit.

Simply don't hard code the "user.home"
Create a field home, that you change in the unit code, to point to the test directory:
public class A {
private static String homeDir;
protected static void setHomeDir(String home) {
this.homeDir = home;
}
public static void main(String args[]) {
if (homeDir == null) {
homeDir = System.getProperty("user.home");
}
A a = new A();
if (a.processArgs(args[0]) {
throw new InvalidArgumentException("Not under home directory");
}
}
// A simple method to check if the file is at home directory
protected boolean processArgs(String s) {
File f = new File(s);
if (s.startsWith(A.homeDir) && f.exists() && additionalLogic())
return true;
else
return false;
}
// Additional business Logic
private boolean additionalBusinessLogic() {
// Do wonderful things.
}
}
Now in the Unit Test, set the homeDir to your test directory
public void testMainHomeExisting() {
A a = new A;
String home = "./testdata/";
A.setHomeDir(home);
String[] args = new String[]{home}; // hope this compiles otherwise fix it
// no assert needed here, if test fails, an Exception is thrown
A.main(args);
}
Now a test case for home not existing
public void testMainHomeNotExisting() {
A a = new A;
String home = "./notExistingFooBarFooFoo/";
A.setHomeDir(home);
String[] args = new String[]{home}; // hope this compiles otherwise fix it
// no assert needed here, if test fails, an Exception is thrown
try {
A.main(args);
// if code works the next line should not be reached:
fail("Expected IllegalArgumentException");
} catch (IllegalArgumentException ex) {
// as expected got IllegalArgumentException
}
}

Just make a Test for the core business method additionalBusinessLogic only. You don't need to call main.

I see no reason to call main.
When you're writing a unit test, you want them to be modular enough to call without relying too much on external methods - and what you can't call you can mock, using something like EasyMock, PowerMock or Mockito.

I would change the method you want to test...
Remove the access modifier
Pass in the File variable you need to be able to do your logic
boolean additionalBusinessLogic(File f)
This will allow a test class in the same package to invoke the method. If you leave it private, no other classes will be able to call it.
Once you can call the method you want to test, the test class is easy...
public class MyClassTest {
#Test
public void additionalBusinessLogic_shouldFoo_whenSomeCondition() {
// setup
A a = new A();
File mockFile = mock(File.class);
// other setup stuff
// execute
boolean result = a.additionalBusinessLogic(mockFile);
// assert
// verify whatever you need to
}
}
For a good mocking framework, I would suggest Mockito.

How to test file sizes with mockito in java?

I have method for which I need to create a JUnit test:
public class MyClass {
private String file1;
private String file2;
public void myMethodSpaceCheck(){
if (new File(file1).size() > new File(file2).size() {
throw new Exception .....
}
}
}
Is it possible to use Mockito to create that JUnit test?

When dealing with files in Java, my preferred option is to go with Apache VFS, as I can then treat them as any other POJO. Obviously, that's a lot of work when you are already stuck with the File API.
Another option is to forget Mockito entirely and write those files on the system. I usually avoid that, as it sometimes make it harder to have tests run in parallel on some systems.
For this specific situation, my solution is generally to provide a special class, say FileBuilder, that can instantiate new Files:
public class FileBuilder {
public java.io.File newFile(String pathname) {
return new java.io.File(pathname);
}
}
I then mock this class before passing it to MyClass, and instrument it as appropriate:
#Test(expected = Exception.class)
public void should_fail_when_file1_is_bigger_than_file2() {
FileBuilder mockFile1 = file(2L);
FileBuilder mockFile2 = file(1L);
FileBuilder mockFileBuilder = mock(FileBuilder.class);
when(mockFileBuilder.newFile("file1").thenReturn(mockFile1);
when(mockFileBuilder.newFile("file2").thenReturn(mockFile2);
new MyClass(mockFileBuilder).myMethodSpaceCheck();
}
private static File file(long length) {
File mockFile = mock(File.class);
when(mockFile.length()).thenReturn(length);
return mockFile;
}
(your example mentions File.size(); I assumed you meant File.length())
The actual implementation of MyClass would look like this:
public class MyClass {
private String file1;
private String file2;
private final FileBuilder fileBuilder;
public MyClass() {
this(new FileBuilder());
}
#VisibleForTesting
MyClass(FileBuilder fileBuilder) {
this.fileBuilder = fileBuilder;
}
public void myMethodSpaceCheck() //...
}

Using different classloaders for different JUnit tests?

I have a Singleton/Factory object that I'd like to write a JUnit test for. The Factory method decides which implementing class to instantiate based upon a classname in a properties file on the classpath. If no properties file is found, or the properties file does not contain the classname key, then the class will instantiate a default implementing class.
Since the factory keeps a static instance of the Singleton to use once it has been instantiated, to be able to test the "failover" logic in the Factory method I would need to run each test method in a different classloader.
Is there any way with JUnit (or with another unit testing package) to do this?
edit: here is some of the Factory code that is in use:
private static MyClass myClassImpl = instantiateMyClass();
private static MyClass instantiateMyClass() {
MyClass newMyClass = null;
String className = null;
try {
Properties props = getProperties();
className = props.getProperty(PROPERTY_CLASSNAME_KEY);
if (className == null) {
log.warn("instantiateMyClass: Property [" + PROPERTY_CLASSNAME_KEY
+ "] not found in properties, using default MyClass class [" + DEFAULT_CLASSNAME + "]");
className = DEFAULT_CLASSNAME;
}
Class MyClassClass = Class.forName(className);
Object MyClassObj = MyClassClass.newInstance();
if (MyClassObj instanceof MyClass) {
newMyClass = (MyClass) MyClassObj;
}
}
catch (...) {
...
}
return newMyClass;
}
private static Properties getProperties() throws IOException {
Properties props = new Properties();
InputStream stream = Thread.currentThread().getContextClassLoader().getResourceAsStream(PROPERTIES_FILENAME);
if (stream != null) {
props.load(stream);
}
else {
log.error("getProperties: could not load properties file [" + PROPERTIES_FILENAME + "] from classpath, file not found");
}
return props;
}

This question might be old but since this was the nearest answer I found when I had this problem I though I'd describe my solution.
Using JUnit 4
Split your tests up so that there is one test method per class (this solution only changes classloaders between classes, not between methods as the parent runner gathers all the methods once per class)
Add the #RunWith(SeparateClassloaderTestRunner.class) annotation to your test classes.
Create the SeparateClassloaderTestRunner to look like this:
public class SeparateClassloaderTestRunner extends BlockJUnit4ClassRunner {
public SeparateClassloaderTestRunner(Class<?> clazz) throws InitializationError {
super(getFromTestClassloader(clazz));
}
private static Class<?> getFromTestClassloader(Class<?> clazz) throws InitializationError {
try {
ClassLoader testClassLoader = new TestClassLoader();
return Class.forName(clazz.getName(), true, testClassLoader);
} catch (ClassNotFoundException e) {
throw new InitializationError(e);
}
}
public static class TestClassLoader extends URLClassLoader {
public TestClassLoader() {
super(((URLClassLoader)getSystemClassLoader()).getURLs());
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
if (name.startsWith("org.mypackages.")) {
return super.findClass(name);
}
return super.loadClass(name);
}
}
}
Note I had to do this to test code running in a legacy framework which I couldn't change. Given the choice I'd reduce the use of statics and/or put test hooks in to allow the system to be reset. It may not be pretty but it allows me to test an awful lot of code that would be difficult otherwise.
Also this solution breaks anything else that relies on classloading tricks such as Mockito.

When I run into these sort of situations I prefer to use what is a bit of a hack. I might instead expose a protected method such as reinitialize(), then invoke this from the test to effectively set the factory back to its initial state. This method only exists for the test cases, and I document it as such.
It is a bit of a hack, but it's a lot easier than other options and you won't need a 3rd party lib to do it (though if you prefer a cleaner solution, there probably are some kind of 3rd party tools out there you could use).

You can use Reflection to set myClassImpl by calling instantiateMyClass() again. Take a look at this answer to see example patterns for playing around with private methods and variables.

If executing Junit via the Ant task you can set fork=true to execute every class of tests in it's own JVM. Also put each test method in its own class and they will each load and initialise their own version of MyClass. It's extreme but very effective.

Below you can find a sample that does not need a separate JUnit test runner and works also with classloading tricks such as Mockito.
package com.mycompany.app;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import java.net.URLClassLoader;
import org.junit.Test;
public class ApplicationInSeparateClassLoaderTest {
#Test
public void testApplicationInSeparateClassLoader1() throws Exception {
testApplicationInSeparateClassLoader();
}
#Test
public void testApplicationInSeparateClassLoader2() throws Exception {
testApplicationInSeparateClassLoader();
}
private void testApplicationInSeparateClassLoader() throws Exception {
//run application code in separate class loader in order to isolate static state between test runs
Runnable runnable = mock(Runnable.class);
//set up your mock object expectations here, if needed
InterfaceToApplicationDependentCode tester = makeCodeToRunInSeparateClassLoader(
"com.mycompany.app", InterfaceToApplicationDependentCode.class, CodeToRunInApplicationClassLoader.class);
//if you want to try the code without class loader isolation, comment out above line and comment in the line below
//CodeToRunInApplicationClassLoader tester = new CodeToRunInApplicationClassLoaderImpl();
tester.testTheCode(runnable);
verify(runnable).run();
assertEquals("should be one invocation!", 1, tester.getNumOfInvocations());
}
/**
* Create a new class loader for loading application-dependent code and return an instance of that.
*/
#SuppressWarnings("unchecked")
private <I, T> I makeCodeToRunInSeparateClassLoader(
String packageName, Class<I> testCodeInterfaceClass, Class<T> testCodeImplClass) throws Exception {
TestApplicationClassLoader cl = new TestApplicationClassLoader(
packageName, getClass(), testCodeInterfaceClass);
Class<?> testerClass = cl.loadClass(testCodeImplClass.getName());
return (I) testerClass.newInstance();
}
/**
* Bridge interface, implemented by code that should be run in application class loader.
* This interface is loaded by the same class loader as the unit test class, so
* we can call the application-dependent code without need for reflection.
*/
public static interface InterfaceToApplicationDependentCode {
void testTheCode(Runnable run);
int getNumOfInvocations();
}
/**
* Test-specific code to call application-dependent code. This class is loaded by
* the same class loader as the application code.
*/
public static class CodeToRunInApplicationClassLoader implements InterfaceToApplicationDependentCode {
private static int numOfInvocations = 0;
#Override
public void testTheCode(Runnable runnable) {
numOfInvocations++;
runnable.run();
}
#Override
public int getNumOfInvocations() {
return numOfInvocations;
}
}
/**
* Loads application classes in separate class loader from test classes.
*/
private static class TestApplicationClassLoader extends URLClassLoader {
private final String appPackage;
private final String mainTestClassName;
private final String[] testSupportClassNames;
public TestApplicationClassLoader(String appPackage, Class<?> mainTestClass, Class<?>... testSupportClasses) {
super(((URLClassLoader) getSystemClassLoader()).getURLs());
this.appPackage = appPackage;
this.mainTestClassName = mainTestClass.getName();
this.testSupportClassNames = convertClassesToStrings(testSupportClasses);
}
private String[] convertClassesToStrings(Class<?>[] classes) {
String[] results = new String[classes.length];
for (int i = 0; i < classes.length; i++) {
results[i] = classes[i].getName();
}
return results;
}
#Override
public Class<?> loadClass(String className) throws ClassNotFoundException {
if (isApplicationClass(className)) {
//look for class only in local class loader
return super.findClass(className);
}
//look for class in parent class loader first and only then in local class loader
return super.loadClass(className);
}
private boolean isApplicationClass(String className) {
if (mainTestClassName.equals(className)) {
return false;
}
for (int i = 0; i < testSupportClassNames.length; i++) {
if (testSupportClassNames[i].equals(className)) {
return false;
}
}
return className.startsWith(appPackage);
}
}
}