We Keep Coding

Strange issue with using Java reflection across multiple projects

I have created a Java project that implements a State Machine. The idea behind it is that the SM itself runs in a background thread as a simple loop over an array of objects representing different actions.
The actions in question are from another object called an ActionPack that contains the Action objects.
This works well using Java reflection to look into the instantiated ActionPack at run time and import them into the state machine where they are pushed onto the array. The state machine then iterates over the array when told to run and calls the execute method of each action.
As I said, this works well and all unit tests are green ... but only when I run the them from within IntelliJ. If I try running them via "mvn test", the Reflection code fails to find the classes within the action pack.
This is the failing reflection method:
/**
* Use introspection to read in all of the classes in a given action pack
* Filter out the ones that are actions and return them in an array
*/
public ArrayList getActionsFromActionPack(
DataAccessLayer dataAccessLayer,
String runRoot
) {
String packageName = this.getClass().getPackage().getName();
System.out.println("PACKAGE NAME = " + packageName);
List<ClassLoader> classLoadersList = new LinkedList<>();
classLoadersList.add(ClasspathHelper.contextClassLoader());
classLoadersList.add(ClasspathHelper.staticClassLoader());
Reflections reflections = new Reflections(new ConfigurationBuilder()
.setScanners(new SubTypesScanner(false /* don't exclude Object.class */), new ResourcesScanner())
.setUrls(ClasspathHelper.forClassLoader(classLoadersList.toArray(new ClassLoader[0])))
.filterInputsBy(new FilterBuilder().include(FilterBuilder.prefix(packageName))));
Set<Class<? extends Action>> classes = reflections.getSubTypesOf(Action.class);
System.out.println("Number of classes = " + classes.size());
ArrayList<IAction> actions = new ArrayList<>();
for (Class a : classes) {
try {
Action action = (Action) Class.forName(a.getName()).newInstance();
// Give the action access to the DAL and path to control directory
action.setDataAccessLayer(dataAccessLayer);
action.setRunRoot(runRoot);
actions.add(action);
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.err.println(e.getClass().getName() + ": " + e.getMessage());
System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace();
System.err.println(e.getClass().getName() + ": " + e.getMessage());
System.exit(1);
} catch (InstantiationException e) {
e.printStackTrace();
System.err.println(e.getClass().getName() + ": " + e.getMessage());
System.exit(1);
}
}
return actions;
}
With those debug print statements in there, when I run the Unit Tests from within IntelliJ I see this output:
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 7
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 7
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 7
PACKAGE NAME = com.github.museadmin.infinite_state_machine.test.classes
Number of classes = 1
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 7
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 7
Process finished with exit code 0
But when I run via mvn test:
PACKAGE NAME = com.github.museadmin.infinite_state_machine.core.action_pack
Number of classes = 0
Exception in thread "UnitTestThread" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at com.github.museadmin.infinite_state_machine.core.InfiniteStateMachine.run(InfiniteStateMachine.java:36)
at java.lang.Thread.run(Thread.java:745)
So it looks like the reflective method correctly identifies the package that we're in, but then fails to find the classes within it if called via maven.
The failure occurs in the final line of setup() of the unit test when I call importActionPack
#Before
public void setup() {
ClassLoader loader = Thread.currentThread().getContextClassLoader();
URL is = loader.getResource(PROPERTIES);
ismCoreActionPack = new ISMCoreActionPack();
infiniteStateMachine = new InfiniteStateMachine(is.getPath());
infiniteStateMachine.importActionPack(ismCoreActionPack);
}
This code is spread across three projects:
infinite-state-machine (The state machine itself)
infinite-state-machine-common (Where the action pack and action types are defined)
infinite-state-machine-core (The action pack I'm failing to import)
The Action Pack has an interface in infinite-state-machine-common:
package com.github.museadmin.infinite_state_machine.common.action;
import com.github.museadmin.infinite_state_machine.common.dal.DataAccessLayer;
import org.json.JSONObject;
import java.util.ArrayList;
public interface IActionPack {
JSONObject getJsonObjectFromResourceFile(String fileName);
ArrayList getActionsFromActionPack(
DataAccessLayer dataAccessLayer,
String runRoot
);
}
The actual implementation of the action pack is also in infinite-state-machine-common:
public class ActionPack implements IActionPack
And that is where the reflections method public ArrayList getActionsFromActionPack() is found.
The Action class and its interface are also in infinite-state-machine-common:
package com.github.museadmin.infinite_state_machine.common.action;
import com.github.museadmin.infinite_state_machine.common.dal.DataAccessLayer;
import org.json.JSONObject;
import java.util.ArrayList;
public interface IAction {
void execute();
Boolean active();
boolean active(String action);
void activate(String actionName);
Boolean afterActionsComplete();
Boolean beforeActionsComplete();
void clearPayload(String actionName);
String createRunDirectory(String directory);
void deactivate();
void deactivate(String actionFlag);
JSONObject getJsonObjectFromFile(String fileName);
ArrayList<JSONObject> getUnprocessedMessages();
void insertMessage(JSONObject message);
void insertProperty(String property, String value);
void markMessageProcessed(Integer id);
void setDataAccessLayer(DataAccessLayer dataAccessLayer);
void setRunRoot(String runRoot);
String queryProperty(String property);
String queryRunPhase();
void setState(String stateName);
void updatePayload(String actionName, String payload);
void updateProperty(String property, String value);
void updateRunPhase(String runPhase);
void unsetState(String stateName);
}
The Action parent class itself is too big to post here but is defined in the same package:
public abstract class Action implements IAction {
}
If it helps, this is a typical action from the core package showing the execute method:
package com.github.museadmin.infinite_state_machine.core.action_pack;
import com.github.museadmin.infinite_state_machine.common.action.Action;
/**
* Action that verifies the state of the ISM and confirms
* we have booted up correctly and can now run
*/
public class ActionConfirmReadyToRun extends Action {
/**
* Confirm we've got through the bootstrapping process ok.
* and all BEFORE actions completed
*/
public void execute() {
if (active()) {
if (beforeActionsComplete()) {
setState("READY_TO_RUN");
updateRunPhase("RUNNING");
deactivate();
}
}
}
}
SUMMARY
I don't know if this is a problem with the way I am using the Reflections library as I would expect it to fail when I ran the tests from IntelliJ...
So perhaps this is an issue with Maven?
I have built and locally installed all of the projects, so they are definitely available and built at this point.
FYI - At the moment I have put the Unit tests in their own separate project to prevent this issue from breaking the build as I push the projects to Nexus.
Any help or guidance on debugging this issue would be much appreciated.
Brad

JVM Security Manager File permissions - custom policy

I've found a somehow unexpected behaviour using JVM Security Manager custom policies.
repo: https://github.com/pedrorijo91/jvm-sec-manager
in branch master, go into the /code folder:
custom policy file grants File read permission for file ../allow/allow.txt
no permission for the file ../deny/deny.txt
the code in the HelloWorld.java tries to read both files
There's a run.sh script to run the command
Now everything works as expected: the allowed file reads, but the other throws a security exception: java.security.AccessControlException: access denied ("java.io.FilePermission" "../deny/deny.txt" "read")
But if I move both files (../allow/allow.txt and ../deny/deny.txt) to the code folder (changing the custom policy and the java code to use those files), I get no exception. (branch 'unexpected')
Is the current directory a special case or something else is happening?

Brief explanation
This behaviour is documented in a number of places:
FilePermission's overview;
the Permissions in the JDK document; and
the URLClassLoader#getPermissions(CodeSource) method.
The latter two reiterate the closing note of the first one, which states that:
Code can always read a file from the same directory it's in (or a subdirectory of that directory); it does not need explicit permission to do so.
In other words, if
(HelloWorld.class.getProtectionDomain().getCodeSource().implies(
new CodeSource(new URL("file:" + codeDir),
(Certificate[]) null)) == true)
then HelloWorld will by default be granted read access to the denoted directory and its descendants. Particularly for the code directory itself this should make some intuitive sense, as otherwise the class would be unable to even access public-access classes within its very package.
The full story
It is basically up to the ClassLoader: If it statically assigned any Permissions to the ProtectionDomain to which it mapped the class--which applies to both java.net.URLClassLoader and sun.misc.Launcher$AppClassLoader (the OpenJDK-specific default system class loader)--these permissions will always be accorded to the domain, regardless of the Policy in effect.
Workarounds
The typical "quick-n'-dirty" workaround to anything authorization-related is to extend SecurityManager and override the methods irking you; i.e. in this case the checkRead group of methods.
For a more thorough solution that doesn't reduce the flexibility of AccessController and friends, on the other hand, you would have to write a class loader that at the very least overrides URLClassLoader#getPermissions(CodeSource) and/or restricts loaded classes' domains' CodeSources down to the file level (code sources of domains assigned by default by URLClassLoader and AppClassLoader imply (recursively) the .class file's classpath entry (JAR or directory)). For further granularity, your loader might as well assign instances of your own domain subclass, and/or domains encapsulating code sources of your own subclass, overriding respectively ProtectionDomain#implies(Permission) and/or CodeSource#implies(CodeSource); the former could for example be made to support "negative permission" semantics, and the latter could base code source implication on arbitrary logic, potentially decoupled from physical code location (think e.g. "trust levels").
Clarification as per the comments
To prove that under a different class loader these permissions actually matter, consider the following example: There are two classes, A and B; A has the main method, which simply calls a method on B. Additionally, the application is launched using a different system class loader, which a) assigns domains on a per-class basis (rather than on a per-classpath-entry basis, as is the default) to classes it loads, without b) assigning any permissions to these domains.
Loader:
package com.example.q45897574;
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.security.AccessController;
import java.security.CodeSource;
import java.security.PermissionCollection;
import java.security.Permissions;
import java.security.PrivilegedAction;
import java.security.ProtectionDomain;
import java.security.cert.Certificate;
import java.util.LinkedHashSet;
import java.util.Set;
import java.util.regex.Pattern;
public class RestrictiveClassLoader extends URLClassLoader {
private static final Pattern COMMON_SYSTEM_RESOURCE_NAMES = Pattern
.compile("(((net\\.)?java)|(java(x)?)|(sun|oracle))\\.[a-zA-Z0-9\\.\\-_\\$\\.]+");
private static final String OWN_CLASS_NAME = RestrictiveClassLoader.class.getName();
private static final URL[] EMPTY_URL_ARRAY = new URL[0], CLASSPATH_ENTRY_URLS;
private static final PermissionCollection NO_PERMS = new Permissions();
static {
String[] classpathEntries = AccessController.doPrivileged(new PrivilegedAction<String>() {
#Override
public String run() {
return System.getProperty("java.class.path");
}
}).split(File.pathSeparator);
Set<URL> classpathEntryUrls = new LinkedHashSet<>(classpathEntries.length, 1);
for (String classpathEntry : classpathEntries) {
try {
URL classpathEntryUrl;
if (classpathEntry.endsWith(".jar")) {
classpathEntryUrl = new URL("file:jar:".concat(classpathEntry));
}
else {
if (!classpathEntry.endsWith("/")) {
classpathEntry = classpathEntry.concat("/");
}
classpathEntryUrl = new URL("file:".concat(classpathEntry));
}
classpathEntryUrls.add(classpathEntryUrl);
}
catch (MalformedURLException mue) {
}
}
CLASSPATH_ENTRY_URLS = classpathEntryUrls.toArray(EMPTY_URL_ARRAY);
}
private static byte[] readClassData(URL classResource) throws IOException {
try (InputStream in = new BufferedInputStream(classResource.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream()) {
while (in.available() > 0) {
out.write(in.read());
}
return out.toByteArray();
}
}
public RestrictiveClassLoader(ClassLoader parent) {
super(EMPTY_URL_ARRAY, parent);
for (URL classpathEntryUrl : CLASSPATH_ENTRY_URLS) {
addURL(classpathEntryUrl);
}
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
if (name == null) {
throw new ClassNotFoundException("< null >", new NullPointerException("name argument must not be null."));
}
if (OWN_CLASS_NAME.equals(name)) {
return RestrictiveClassLoader.class;
}
if (COMMON_SYSTEM_RESOURCE_NAMES.matcher(name).matches()) {
return getParent().loadClass(name);
}
Class<?> ret = findLoadedClass(name);
if (ret != null) {
return ret;
}
return findClass(name);
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
String modifiedClassName = name.replace(".", "/").concat(".class");
URL classResource = findResource(modifiedClassName);
if (classResource == null) {
throw new ClassNotFoundException(name);
}
byte[] classData;
try {
classData = readClassData(classResource);
}
catch (IOException ioe) {
throw new ClassNotFoundException(name, ioe);
}
return defineClass(name, classData, 0, classData.length, constructClassDomain(classResource));
}
#Override
protected PermissionCollection getPermissions(CodeSource codesource) {
return NO_PERMS;
}
private ProtectionDomain constructClassDomain(URL codeSourceLocation) {
CodeSource cs = new CodeSource(codeSourceLocation, (Certificate[]) null);
return new ProtectionDomain(cs, getPermissions(cs), this, null);
}
}
A:
package com.example.q45897574;
public class A {
public static void main(String... args) {
/*
* Note:
* > Can't we set the security manager via launch argument?
* No, it has to be set here, or bootstrapping will fail.
* > Why?
* Because our class loader's domain is unprivileged.
* > Can't it be privileged?
* Yes, but then everything under the same classpath entry becomes
* privileged too, because our loader's domain's code source--which
* _its own_ loader creates, thus escaping our control--implies _the
* entire_ classpath entry. There are various workarounds, which
* however fall outside of this example's scope.
*/
System.setSecurityManager(new SecurityManager());
B.b();
}
}
B:
package com.example.q45897574;
public class B {
public static void b() {
System.out.println("success!");
}
}
Unprivileged test:
Make sure nothing is granted at the policy level; then run (assuming a Linux-based OS--modify classpath as appropriate):
java -cp "/home/your_user/classpath/" \
-Djava.system.class.loader=com.example.q45897574.RestrictiveClassLoader \
-Djava.security.debug=access=failure com.example.q45897574.A
You should get a NoClassDefFoundError, along with a failed FilePermission for com.example.q45897574.A.
Privileged test:
Now grant the necessary permission to A (again make sure to correct both the codeBase (code source URL) and permission target name):
grant codeBase "file:/home/your_user/classpath/com/example/q45897574/A.class" {
permission java.io.FilePermission "/home/your_user/classpath/com/example/q45897574/B.class", "read";
};
...and re-run. This time execution should complete successfully.

Using java securityManager blocks me from reading files

In my java code I call another 3rd party java class.
I want to catch that latter System.exit() exit code
So I use security-manager as suggested in this post
The problem is that I cannot read files now, I get permissions error
as seen in that post.
How can I catch the exit code and still read files?
Published class MyClass {
class MySecurityManager extends SecurityManager {
#Override
public void checkExit(int status) {
throw new SecurityException();
}
}
public void foo() {
MySecurityManager secManager = new MySecurityManager();
System.setSecurityManager(secManager);
try {
ConfigValidator.main(new String[]{"-dirs", SdkServiceConfig.s.PROPERTIES_FILE_PATH});
new FileInputStream(new File("/Users/eladb/WorkspaceQa/sdk-service/src/main/resources/convert_conditions.sh"));
} catch (SecurityException e) {
//Do something if the external code used System.exit()
String a = "1";
}
catch (Exception e) {
logger.error("failed converting properties file to proto", e);
}
}
}

You have two separate problems: Your trusted code cannot read the file, while the untrusted third-party library can still call System#exit unhindered. The former can be easily circumvented by granting further privileges to the trusted code; the latter is a tad trickier to address.
A bit of background
Privilege assignment
Code (the ProtectionDomains encapsulated by a thread's AccessControlContext) generally gets assigned Permissions in two ways: Statically, by the ClassLoader, upon class definition, and/or dynamically, by the Policy in effect. Other, less frequently encountered alternatives, exist as well: DomainCombiners, for instance, can modify AccessControlContexts' domains (and therefore the effective permissions of their respective code that is subject to authorization) on the fly, and custom domain implementations may use their own logic for permission implication, possibly disregarding or altering the semantics of the policy. By default the permission set of a domain is the union of its static and dynamic permissions. As for how exactly classes are mapped to domains, it is, for the most part, up to the loader's implementation. By default, all classes, JAR'ed or otherwise, residing beneath the same class path entry, are grouped under the same domain. More restrictive class loaders may choose to e.g. allocate a domain per class, which could be used to prevent communication even between classes within the same package.
Privilege evaluation
Under the default SecurityManager, for a privileged operation (an invocation of any method having a SecurityManager#checkXXX within its body) to succeed, every domain (of every class of every method) of the effective AccessControlContext must have been assigned, as explained above, the permission being checked. Recall however that the context need not necessarily represent "the truth" (the actual call stack)—system code gets optimized away early on, while AccessController#doPrivileged calls, along with the DomainCombiner potentially coupled to the AccessControlContext can modify the context's domains, and the authorization algorithm in its entirety, consequently.
Problem and workarounds
The issue with System#exit is that the corresponding permission (RuntimePermission("exitVM.*")) is one amongst few that are statically assigned by the default application class loader (sun.misc.Launcher$AppClassLoader) to all domains associated with classes loaded from the class path.
A number of alternatives come to mind:
Installing a custom SecurityManager which denies the particular right based on, e.g., the class attempting to terminate the JVM process.
Loading the third-party library from a "remote" location (a directory outside of the class path), so that it gets treated as "untrusted" code by its class loader.
Authoring and installing a different application class loader, which does not assign the extraneous permission.
Plugging a custom domain combiner into the access control context, which replaces, at the time of an authorization decision, all third-party domains with equivalent ones that do not have the offending permission.
I should, for the sake of completeness, note that at the Policy level, unfortunately, nothing can be done to negate statically assigned permissions.
The first option is overall the most convenient one, but I will not explore it further because:
The default SecurityManager is quite flexible, thanks to the handful of components (AccessController et al.) it interacts with. The background section in the beginning aimed to serve as a reminder of that flexibility, which "quick-n'-dirty" method overrides tend to cripple.
Careless modifications of the default implementation might cause (system) code to misbehave in curious ways.
Frankly, because it's boring―it's the one-size-fits-all solution perpetually advocated, while the fact that the default manager was standardized in 1.2 for a reason has long been forgotten.
The second alternative is easy to implement but impractical, complicating either development or the build process. Assuming you are not planning to invoke the library solely reflectively, or aided by interfaces present on the class path, it would have to be present initially, during development, and relocated before execution.
The third is, at least in the context of a standalone Java SE application, fairly straightforward and should not pose too much of a burden on performance. It is the approach I will favour herein.
The last option is the most novel and least convenient. It is hard to securely implement, has the greatest potential for performance degradation, and burdens client code with ensuring presence of the combiner prior to every delegation to untrusted code.
Proposed solution
The custom ClassLoader
The following is to be used as the replacement of the default application loader, or alternatively as the context class loader, or the loader used to load at least the untrusted classes. There is nothing novel to this implementation—all it does is prevent delegation to the default application class loader when the class in question is assumed to not be a system one. URLClassLoader#findClass, in turn, does not assign RuntimePermission("exitVM.*") to the domains of the classes it defines.
package com.example.trusted;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.regex.Pattern;
public class ClasspathClassLoader extends URLClassLoader {
private static final Pattern SYSTEM_CLASS_PREFIX = Pattern.compile("((java(x)?|sun|oracle)\\.).*");
public ClasspathClassLoader(ClassLoader parent) {
super(new URL[0], parent);
String[] classpath = System.getProperty("java.class.path").split(File.pathSeparator);
for (String classpathEntry : classpath) {
try {
if (!classpathEntry.endsWith(".jar") && !classpathEntry.endsWith("/")) {
// URLClassLoader assumes paths without a trailing '/' to be JARs by default
classpathEntry += "/";
}
addURL(new URL("file:" + classpathEntry));
}
catch (MalformedURLException mue) {
System.err.println(MessageFormat.format("Erroneous class path entry [{0}] skipped.", classpathEntry));
}
}
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
Class<?> ret;
synchronized (getClassLoadingLock(name)) {
ret = findLoadedClass(name);
if (ret != null) {
return ret;
}
if (SYSTEM_CLASS_PREFIX.matcher(name).matches()) {
return super.loadClass(name, resolve);
}
ret = findClass(name);
if (resolve) {
resolveClass(ret);
}
}
return ret;
}
}
If you also wish to fine-tune the domains assigned to loaded classes, you will additionally have to override findClass. The following variant of the loader is a very crude attempt at doing so. constructClassDomain therein merely creates one domain per class path entry (which is more or less the default), but can be modified to do something different.
package com.example.trusted;
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.ByteBuffer;
import java.security.AccessController;
import java.security.CodeSource;
import java.security.PrivilegedAction;
import java.security.ProtectionDomain;
import java.security.cert.Certificate;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;
public final class ClasspathClassLoader extends URLClassLoader {
private static final Pattern SYSTEM_CLASS_PREFIX = Pattern.compile("((java(x)?|sun|oracle)\\.).*");
private static final List<WeakReference<ProtectionDomain>> DOMAIN_CACHE = new ArrayList<>();
// constructor, loadClass same as above
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
URL classOrigin = getClassResource(name);
if (classOrigin == null) {
return super.findClass(name);
}
URL classCodeSourceOrigin = getClassCodeSourceResource(classOrigin);
if (classCodeSourceOrigin == null) {
return super.findClass(name);
}
return defineClass(name, readClassData(classOrigin), constructClassDomain(classCodeSourceOrigin));
}
private URL getClassResource(String name) {
return AccessController.doPrivileged((PrivilegedAction<URL>) () -> getResource(name.replace(".", "/") + ".class"));
}
private URL getClassCodeSourceResource(URL classResource) {
for (URL classpathEntry : getURLs()) {
if (classResource.getPath().startsWith(classpathEntry.getPath())) {
return classpathEntry;
}
}
return null;
}
private ByteBuffer readClassData(URL classResource) {
try {
BufferedInputStream in = new BufferedInputStream(classResource.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
int i;
while ((i = in.read()) != -1) {
out.write(i);
}
return ByteBuffer.wrap(out.toByteArray());
}
catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
private ProtectionDomain constructClassDomain(URL classCodeSourceResource) {
ProtectionDomain ret = getCachedDomain(classCodeSourceResource);
if (ret == null) {
CodeSource cs = new CodeSource(classCodeSourceResource, (Certificate[]) null);
DOMAIN_CACHE.add(new WeakReference<>(ret = new ProtectionDomain(cs, getPermissions(cs), this, null)));
}
return ret;
}
private ProtectionDomain getCachedDomain(URL classCodeSourceResource) {
for (WeakReference<ProtectionDomain> domainRef : DOMAIN_CACHE) {
ProtectionDomain domain = domainRef.get();
if (domain == null) {
DOMAIN_CACHE.remove(domainRef);
}
else if (domain.getCodeSource().implies(new CodeSource(classCodeSourceResource, (Certificate[]) null))) {
return domain;
}
}
return null;
}
}
The "unsafe" code
package com.example.untrusted;
public class Test {
public static void testExitVm() {
System.out.println("May I...?!");
System.exit(-1);
}
}
The entry point
package com.example.trusted;
import java.security.AccessControlException;
import java.security.Permission;
import com.example.untrusted.Test;
public class Main {
private static final Permission EXIT_VM_PERM = new RuntimePermission("exitVM.*");
public static void main(String... args) {
System.setSecurityManager(new SecurityManager());
try {
Test.testExitVm();
}
catch (AccessControlException ace) {
Permission deniedPerm = ace.getPermission();
if (EXIT_VM_PERM.implies(deniedPerm)) {
ace.printStackTrace();
handleUnauthorizedVmExitAttempt(Integer.parseInt(deniedPerm.getName().replace("exitVM.", "")));
}
}
}
private static void handleUnauthorizedVmExitAttempt(int exitCode) {
System.out.println("here let me do it for you");
System.exit(exitCode);
}
}
Testing
Packaging
Place the loader and the main class in one JAR (let's call it trusted.jar) and the demo untrusted class in another (untrusted.jar).
Assigning privileges
The default Policy (sun.security.provider.PolicyFile) is backed by the file at <JRE>/lib/security/java.policy, as well as any of the files referenced by the policy.url.n properties in <JRE>/lib/security/java.security. Modify the former (the latter should hopefully be empty by default) as follows:
// Standard extensions get all permissions by default
grant codeBase "file:${{java.ext.dirs}}/*" {
permission java.security.AllPermission;
};
// no default permissions
grant {};
// trusted code
grant codeBase "file:///path/to/trusted.jar" {
permission java.security.AllPermission;
};
// third-party code
grant codeBase "file:///path/to/untrusted.jar" {
permission java.lang.RuntimePermission "exitVM.-1", "";
};
Note that it is virtually impossible to get components extending the security infrastructure (custom class loaders, policy providers, etc.) to work properly without granting them AllPermission.
Running
Run:
java -classpath "/path/to/trusted.jar:/path/to/untrusted.jar" -Djava.system.class.loader=com.example.trusted.ClasspathClassLoader com.example.trusted.Main
The privileged operation should succeed.
Next comment out the RuntimePermission under untrusted.jar, within the policy file, and re-run. The privileged operation should fail.
As a closing note, when debugging AccessControlExceptions, running with -Djava.security.debug=access=domain,access=failure,policy can help track down offending domains and policy configuration issues.

How to get the exception that was thrown when a Cucumber test failed in Java?

I can perform actions on test failure by using:
#After
public void afterTest(Scenario scenario) {
if (scenario.isFailed()) {
/*Do stuff*/
}
}
However some of the actions I need to perform depend on the Exception that was thrown and in what context it was thrown. Is there a way to get the Throwable that caused the test to fail? For example in JUnit I would do this by extending TestWatcher and add a rule to my tests:
#Override
protected void failed(Throwable e, Description description) {
/*Do stuff with e*/
}
However the cucumber-junit iplementation does not allow the use of rules, so this solution would not work with Cucumber.
I don't think I need to explain why accessing a thrown exception on test failure would be useful, however I will still provide an Example:
My test environment is not always stable, so my tests might fail unexpectedly at any moment (there's no specific place I can try to catch the exception since it could occur at any time). When this happens I need the test to reschedule for another attempt, and log the incident so that we can get some good statistical data on the environment instability (when, how frequent, how long etc.)

The problem with the work around suggested by Frank Escobar:
By using reflection to reach into a frameworks internals you're depending on implementation details. This is a bad practice, when ever the framework changes its implementation your code may break as you will observe in Cucumber v5.0.0.
Hooks in Cucumber are designed to manipulate the test execution context before and after a scenario. They're not made to report on the test execution itself. Reporting is cross cutting concern and best managed by using the plugin system.
For example:
package com.example;
import io.cucumber.plugin.ConcurrentEventListener;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.Result;
import io.cucumber.plugin.event.Status;
import io.cucumber.plugin.event.TestCase;
import io.cucumber.plugin.event.TestCaseFinished;
public class MyTestListener implements ConcurrentEventListener {
#Override
public void setEventPublisher(EventPublisher publisher) {
publisher.registerHandlerFor(TestCaseFinished.class, this::handleTestCaseFinished);
}
private void handleTestCaseFinished(TestCaseFinished event) {
TestCase testCase = event.getTestCase();
Result result = event.getResult();
Status status = result.getStatus();
Throwable error = result.getError();
String scenarioName = testCase.getName();
String id = "" + testCase.getUri() + testCase.getLine();
System.out.println("Testcase " + id + " - " + status.name());
}
}
When using JUnit 4 and TestNG you can activate this plugin using:
#CucumberOptions(plugin="com.example.MyTestListener")
With JUnit 5 you add it to junit-platform.properties:
cucumber.plugin=com.example.MyTestListener
Or if you are using the CLI
--plugin com.example.MyTestListener

I've implemented this method using reflections. You can't access directly to steps errors (stack trace). I've created this static method which allows you to access to "stepResults" attribute and then you can iterate and get the error and do whatever you want.
import cucumber.runtime.ScenarioImpl;
import gherkin.formatter.model.Result;
import org.apache.commons.lang3.reflect.FieldUtils;
import java.lang.reflect.Field;
import java.util.ArrayList;
#After
public void afterScenario(Scenario scenario) {
if (scenario.isFailed())
logError(scenario);
}
private static void logError(Scenario scenario) {
Field field = FieldUtils.getField(((ScenarioImpl) scenario).getClass(), "stepResults", true);
field.setAccessible(true);
try {
ArrayList<Result> results = (ArrayList<Result>) field.get(scenario);
for (Result result : results) {
if (result.getError() != null)
LOGGER.error("Error Scenario: {}", scenario.getId(), result.getError());
}
} catch (Exception e) {
LOGGER.error("Error while logging error", e);
}
}

You can to this by writing your own custom implementation of Formatter & Reporter interface. The empty implementation of Formatter is the NullFormatter.java which you can extend. You will need to provide implementations for the Reporter interface.
The methods which would be of interest will be the result() of the Reporter interface and possibly the done() method of Formatter. The result() has the Result object which has the exceptions.
You can look at RerunFormatter.java for clarity.
Github Formatter source
public void result(Result result) {
//Code to create logs or store to a database etc...
result.getError();
result.getErrorMessage();
}
You will need to add this class(com.myimpl.CustomFormRep) to the plugin option.
plugin={"pretty", "html:report", "json:reports.json","rerun:target/rerun.txt",com.myimpl.CustomFormRep}
More details on custom formatters.
You can use the rerun plugin to get a list of failed scenarios to run again. Not sure about scheduling a run of failed tests, code to create a batch job or schedule one on your CI tool.

This is the workaround for cucumber-java version 4.8.0 using reflection.
import cucumber.api.Result;
import io.cucumber.core.api.Scenario;
import io.cucumber.core.logging.Logger;
import io.cucumber.core.logging.LoggerFactory;
import io.cucumber.java.After;
import org.apache.commons.lang3.ClassUtils;
import org.apache.commons.lang3.reflect.FieldUtils;
import java.io.IOException;
import java.lang.reflect.Field;
import java.net.URL;
import java.util.ArrayList;
#After
public void afterScenario(Scenario scenario) throws IOException {
if(!scenario.getStatus().isOk(true)){
logError(scenario);
}
}
private static void logError(Scenario scenario) {
try {
Class clasz = ClassUtils.getClass("cucumber.runtime.java.JavaHookDefinition$ScenarioAdaptor");
Field fieldScenario = FieldUtils.getField(clasz, "scenario", true);
fieldScenario.setAccessible(true);
Object objectScenario = fieldScenario.get(scenario);
Field fieldStepResults = objectScenario.getClass().getDeclaredField("stepResults");
fieldStepResults.setAccessible(true);
ArrayList<Result> results = (ArrayList<Result>) fieldStepResults.get(objectScenario);
for (Result result : results) {
if (result.getError() != null) {
LOGGER.error(String.format("Error Scenario: %s", scenario.getId()), result.getError());
}
}
} catch (Exception e) {
LOGGER.error("Error while logging error", e);
}
}

For cucumber-js https://www.npmjs.com/package/cucumber/v/6.0.3
import { After } from 'cucumber'
After(async function(scenario: any) {
const exception = scenario.result.exception
if (exception) {
this.logger.log({ level: 'error', message: '-----------StackTrace-----------' })
this.logger.log({ level: 'error', message: exception.stack })
this.logger.log({ level: 'error', message: '-----------End-StackTrace-----------' })
}
})

After a lot of experimentation I now removed the Before/After-Annotations and rely on Cucumber-Events instead. They contain the TestCase (which is what the Scenario-class wraps) and a Result where you can call getError(); to get the Throwable.
Here is a simple example to get it working
import io.cucumber.plugin.EventListener;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.Result;
import io.cucumber.plugin.event.Status;
import io.cucumber.plugin.event.TestCase;
import io.cucumber.plugin.event.TestCaseFinished;
import io.cucumber.plugin.event.TestCaseStarted;
import org.openqa.selenium.WebDriver;
public class TestCaseListener implements EventListener {
#Override
public void setEventPublisher(final EventPublisher publisher) {
publisher.registerHandlerFor(TestCaseStarted.class, this::onTestCaseStarted);
publisher.registerHandlerFor(TestCaseFinished.class, this::onTestCaseFinished);
}
public void onTestCaseStarted(TestCaseStarted event) {
TestCase testCase = event.getTestCase();
System.out.println("Starting " + testCase.getName());
// Other stuff you did in your #Before-Method.
// ...
}
private void onTestCaseFinished(final TestCaseFinished event) {
TestCase testCase = event.getTestCase();
System.out.println("Finished " + testCase.getName());
Result result = event.getResult();
if (result.getStatus() == Status.FAILED) {
final Throwable error = result.getError();
error.printStackTrace();
}
// Other stuff you did in your #After-Method.
// ...
}
}
All that's left to do is to register this class as a Cucumber-Plugin.
I did this by modifying my #CucumberOptions-annotation:
#CucumberOptions(plugin = {"com.example.TestCaseListener"})
I find this much cleaner than all of this reflection-madness, however it requires a lot more code-changes.
Edit
I don't know why, but this caused a lot of tests to randomly fail in a multithreaded environment.
I tried to figure it out, but now also use the ugly reflections mentioned in this thread:
public class SeleniumUtils {
private static final Logger log = LoggerFactory.getLogger(SeleniumUtils.class);
private static final Field field = FieldUtils.getField(Scenario.class, "delegate", true);
private static Method getError;
public static Throwable getError(Scenario scenario) {
try {
final TestCaseState testCase = (TestCaseState) field.get(scenario);
if (getError == null) {
getError = MethodUtils.getMatchingMethod(testCase.getClass(), "getError");
getError.setAccessible(true);
}
return (Throwable) getError.invoke(testCase);
} catch (Exception e) {
log.warn("error receiving exception", e);
}
return null;
}
}

If you just want to massage the result being sent to the report then you can extend the CucumberJSONFormatter and override the result method like this:
public class CustomReporter extends CucumberJSONFormatter {
CustomReporter(Appendable out) {
super(out);
}
/**
* Truncate the error in the output to the testresult.json file.
* #param result the error result
*/
#Override
void result(Result result) {
String errorMessage = null;
if (result.error) {
errorMessage = "Error: " + truncateError(result.error);
}
Result myResult = new Result(result.status, result.duration, errorMessage);
// Log the truncated error to the JSON report
super.result(myResult);
}
}
Then set the plugin option to:
plugin = ["com.myimpl.CustomReporter:build/reports/json/testresult.json"]

Overriding the classloader to get every loaded class bytes and name

Yes , so what im trying to do is getting the class bytes of every loaded class loaded by the jvm during run time . Instrumentation wont work for this case because the program im trying to load has encrypted his classes files and load it with its own class loader.
Here's my attempt : https://gist.github.com/MalikDz/944cae9c168fa05fbd0a
here the output (error) : https://gist.github.com/MalikDz/fdf20df16b951d41cb78
Thanks a lot !

You can use a Java Agent to do this trick:
The Agent is very straightforward: It registers a class transformer, which can get access to the byte-code:
import java.lang.instrument.Instrumentation;
import java.lang.instrument.ClassFileTransformer;
public class ClassDumpAgent
{
/**
* This method is called before the application’s main-method is called, when
* this agent is specified to the Java VM.
**/
public static void premain(String agentArgs, Instrumentation inst)
{
ClassFileTransformer trans = new ClassDumpTransformer();
inst.addTransformer(trans);
}
}
The ClassFileTransformer that is used simply dumps the byte-array with byte-code to the file system:
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;
public class ClassDumpTransformer implements ClassFileTransformer
{
private File rootFolder = new File("C:\\temp\\dump");
public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined,
ProtectionDomain protectionDomain, byte[] classfileBuffer)
throws IllegalClassFormatException
{
File dumpFile = new File(rootFolder,className+".class");
dumpFile.getParentFile().mkdirs();
try {
FileOutputStream out = new FileOutputStream(dumpFile);
try {
out.write(classfileBuffer);
} finally {
out.close();
}
} catch (IOException e) {
throw new IllegalClassFormatException(e.getMessage());
}
return classfileBuffer;
}
}
To package this class dumping agent, you need to JAR the two classes and include a MANIFEST.MF for this agent:
Manifest-Version: 1.0
PreMain-Class: ClassDumpAgent
To run the application with this agent, use a command-line like this:
java -javaagent:cldumpagent.jar MyApplication
Some remarks about the solution:
The classes are dumped in a hardcoded folder (C:\TEMP\DUMP), you
might want to change this.
The transformer will dump all classes, including the JDK Runtime. You might want to filter which packages are dumped.
Be careful with reverse-engineering, in some countries this might be considered illegal.
Barry