I've found a somehow unexpected behaviour using JVM Security Manager custom policies.
repo: https://github.com/pedrorijo91/jvm-sec-manager
in branch master, go into the /code folder:
custom policy file grants File read permission for file ../allow/allow.txt
no permission for the file ../deny/deny.txt
the code in the HelloWorld.java tries to read both files
There's a run.sh script to run the command
Now everything works as expected: the allowed file reads, but the other throws a security exception: java.security.AccessControlException: access denied ("java.io.FilePermission" "../deny/deny.txt" "read")
But if I move both files (../allow/allow.txt and ../deny/deny.txt) to the code folder (changing the custom policy and the java code to use those files), I get no exception. (branch 'unexpected')
Is the current directory a special case or something else is happening?
Brief explanation
This behaviour is documented in a number of places:
FilePermission's overview;
the Permissions in the JDK document; and
the URLClassLoader#getPermissions(CodeSource) method.
The latter two reiterate the closing note of the first one, which states that:
Code can always read a file from the same directory it's in (or a subdirectory of that directory); it does not need explicit permission to do so.
In other words, if
(HelloWorld.class.getProtectionDomain().getCodeSource().implies(
new CodeSource(new URL("file:" + codeDir),
(Certificate[]) null)) == true)
then HelloWorld will by default be granted read access to the denoted directory and its descendants. Particularly for the code directory itself this should make some intuitive sense, as otherwise the class would be unable to even access public-access classes within its very package.
The full story
It is basically up to the ClassLoader: If it statically assigned any Permissions to the ProtectionDomain to which it mapped the class--which applies to both java.net.URLClassLoader and sun.misc.Launcher$AppClassLoader (the OpenJDK-specific default system class loader)--these permissions will always be accorded to the domain, regardless of the Policy in effect.
Workarounds
The typical "quick-n'-dirty" workaround to anything authorization-related is to extend SecurityManager and override the methods irking you; i.e. in this case the checkRead group of methods.
For a more thorough solution that doesn't reduce the flexibility of AccessController and friends, on the other hand, you would have to write a class loader that at the very least overrides URLClassLoader#getPermissions(CodeSource) and/or restricts loaded classes' domains' CodeSources down to the file level (code sources of domains assigned by default by URLClassLoader and AppClassLoader imply (recursively) the .class file's classpath entry (JAR or directory)). For further granularity, your loader might as well assign instances of your own domain subclass, and/or domains encapsulating code sources of your own subclass, overriding respectively ProtectionDomain#implies(Permission) and/or CodeSource#implies(CodeSource); the former could for example be made to support "negative permission" semantics, and the latter could base code source implication on arbitrary logic, potentially decoupled from physical code location (think e.g. "trust levels").
Clarification as per the comments
To prove that under a different class loader these permissions actually matter, consider the following example: There are two classes, A and B; A has the main method, which simply calls a method on B. Additionally, the application is launched using a different system class loader, which a) assigns domains on a per-class basis (rather than on a per-classpath-entry basis, as is the default) to classes it loads, without b) assigning any permissions to these domains.
Loader:
package com.example.q45897574;
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.security.AccessController;
import java.security.CodeSource;
import java.security.PermissionCollection;
import java.security.Permissions;
import java.security.PrivilegedAction;
import java.security.ProtectionDomain;
import java.security.cert.Certificate;
import java.util.LinkedHashSet;
import java.util.Set;
import java.util.regex.Pattern;
public class RestrictiveClassLoader extends URLClassLoader {
private static final Pattern COMMON_SYSTEM_RESOURCE_NAMES = Pattern
.compile("(((net\\.)?java)|(java(x)?)|(sun|oracle))\\.[a-zA-Z0-9\\.\\-_\\$\\.]+");
private static final String OWN_CLASS_NAME = RestrictiveClassLoader.class.getName();
private static final URL[] EMPTY_URL_ARRAY = new URL[0], CLASSPATH_ENTRY_URLS;
private static final PermissionCollection NO_PERMS = new Permissions();
static {
String[] classpathEntries = AccessController.doPrivileged(new PrivilegedAction<String>() {
#Override
public String run() {
return System.getProperty("java.class.path");
}
}).split(File.pathSeparator);
Set<URL> classpathEntryUrls = new LinkedHashSet<>(classpathEntries.length, 1);
for (String classpathEntry : classpathEntries) {
try {
URL classpathEntryUrl;
if (classpathEntry.endsWith(".jar")) {
classpathEntryUrl = new URL("file:jar:".concat(classpathEntry));
}
else {
if (!classpathEntry.endsWith("/")) {
classpathEntry = classpathEntry.concat("/");
}
classpathEntryUrl = new URL("file:".concat(classpathEntry));
}
classpathEntryUrls.add(classpathEntryUrl);
}
catch (MalformedURLException mue) {
}
}
CLASSPATH_ENTRY_URLS = classpathEntryUrls.toArray(EMPTY_URL_ARRAY);
}
private static byte[] readClassData(URL classResource) throws IOException {
try (InputStream in = new BufferedInputStream(classResource.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream()) {
while (in.available() > 0) {
out.write(in.read());
}
return out.toByteArray();
}
}
public RestrictiveClassLoader(ClassLoader parent) {
super(EMPTY_URL_ARRAY, parent);
for (URL classpathEntryUrl : CLASSPATH_ENTRY_URLS) {
addURL(classpathEntryUrl);
}
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
if (name == null) {
throw new ClassNotFoundException("< null >", new NullPointerException("name argument must not be null."));
}
if (OWN_CLASS_NAME.equals(name)) {
return RestrictiveClassLoader.class;
}
if (COMMON_SYSTEM_RESOURCE_NAMES.matcher(name).matches()) {
return getParent().loadClass(name);
}
Class<?> ret = findLoadedClass(name);
if (ret != null) {
return ret;
}
return findClass(name);
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
String modifiedClassName = name.replace(".", "/").concat(".class");
URL classResource = findResource(modifiedClassName);
if (classResource == null) {
throw new ClassNotFoundException(name);
}
byte[] classData;
try {
classData = readClassData(classResource);
}
catch (IOException ioe) {
throw new ClassNotFoundException(name, ioe);
}
return defineClass(name, classData, 0, classData.length, constructClassDomain(classResource));
}
#Override
protected PermissionCollection getPermissions(CodeSource codesource) {
return NO_PERMS;
}
private ProtectionDomain constructClassDomain(URL codeSourceLocation) {
CodeSource cs = new CodeSource(codeSourceLocation, (Certificate[]) null);
return new ProtectionDomain(cs, getPermissions(cs), this, null);
}
}
A:
package com.example.q45897574;
public class A {
public static void main(String... args) {
/*
* Note:
* > Can't we set the security manager via launch argument?
* No, it has to be set here, or bootstrapping will fail.
* > Why?
* Because our class loader's domain is unprivileged.
* > Can't it be privileged?
* Yes, but then everything under the same classpath entry becomes
* privileged too, because our loader's domain's code source--which
* _its own_ loader creates, thus escaping our control--implies _the
* entire_ classpath entry. There are various workarounds, which
* however fall outside of this example's scope.
*/
System.setSecurityManager(new SecurityManager());
B.b();
}
}
B:
package com.example.q45897574;
public class B {
public static void b() {
System.out.println("success!");
}
}
Unprivileged test:
Make sure nothing is granted at the policy level; then run (assuming a Linux-based OS--modify classpath as appropriate):
java -cp "/home/your_user/classpath/" \
-Djava.system.class.loader=com.example.q45897574.RestrictiveClassLoader \
-Djava.security.debug=access=failure com.example.q45897574.A
You should get a NoClassDefFoundError, along with a failed FilePermission for com.example.q45897574.A.
Privileged test:
Now grant the necessary permission to A (again make sure to correct both the codeBase (code source URL) and permission target name):
grant codeBase "file:/home/your_user/classpath/com/example/q45897574/A.class" {
permission java.io.FilePermission "/home/your_user/classpath/com/example/q45897574/B.class", "read";
};
...and re-run. This time execution should complete successfully.
Related
I am developing a client-server application using Java Sockets where at some point the client has to download some classes from the server (through the already bound Socket) and use them. I am able to read the class file bytes and send them to the client. So then, the client has to use a ClassLoader to load the classes from memory. So, in my opinion, the problem is not really related to Sockets, but it is about custom class loading (you will see why I say that, later in this post).
My setup is as follows: the client project with a single package named client with a single class in it, named LiveClientTest and then the server project with a single package named server with 3 classes in it: ClientMain, LiveServerTest (the entry point) and MyStrings. In short, all server-side code is under the package server and all client-side under the client. Each of the two packages is in a different project also.
The problem occurs when the custom ClassLoader of the client (named LiveClientTest.MemoryClassLoader) tries to load a non-static nested class (named MyStrings.NestedNonStaticSubclass) which refers to its enclosing class (named MyStrings) before constructing the object (of type MyStrings.NestedNonStaticSubclass). Although the classes compile fine, the error appears at runtime, while loading the class MyStrings.NestedNonStaticSubclass with the custom ClassLoader. I know it sounds weird, but you can see what I mean if you take a look at the code.
Server side code:
LiveServerTest class (entry point):
package server;
import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;
import javax.swing.JOptionPane;
public class LiveServerTest {
//Convert class binary name representation to file path:
public static String classNameToResourcePath(final String className) {
return className.replace('.', File.separatorChar) + ".class";
}
//Get directory location of the given class (packages not included):
public static Path getDirectoyPath(final Class clazz) throws URISyntaxException {
return new File(clazz.getProtectionDomain().getCodeSource().getLocation().toURI()).toPath();
}
//Get absolute file location of the given class:
public static Path getClassFilePath(final Class c) throws URISyntaxException {
return getDirectoyPath(c).resolve(classNameToResourcePath(c.getName()));
}
public static void main(final String[] args) {
ServerSocket srv = null;
try {
srv = new ServerSocket(Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host port number:"))));
System.out.println("Waiting for client connection...");
try (final Socket sck = srv.accept();
final OutputStream os = sck.getOutputStream()) {
srv.close();
srv = null;
//These are the classes we need the client to load:
final Class[] clientClasses = new Class[] {
ClientMain.class,
MyStrings.class,
MyStrings.NestedStatic.class,
MyStrings.NestedNonStaticSubclass.class
};
System.out.println("Sending all client classes' bytes to client...");
final DataOutputStream dos = new DataOutputStream(os);
dos.writeInt(clientClasses.length);
for (final Class clazz: clientClasses) {
final byte[] contents = Files.readAllBytes(getClassFilePath(clazz));
dos.writeUTF(clazz.getName());
dos.writeInt(contents.length);
dos.write(contents);
}
System.out.println("Main application now starts...");
//Here would go the server side code for the client-server application.
}
}
catch (final IOException | URISyntaxException | RuntimeException x) {
JOptionPane.showMessageDialog(null, x.toString());
}
finally {
System.out.println("Done.");
try { if (srv != null) srv.close(); } catch (final IOException iox) {}
}
}
}
ClientMain class:
package server;
import java.io.DataInputStream;
import java.net.Socket;
public class ClientMain {
//This method is called by the client to start the application:
public static void main(final Socket sck,
final DataInputStream dis,
final ClassLoader loader,
final String[] args) {
System.out.println("Running...");
//Here would go the client side code for the client-server application.
//Just test that all the classes are loaded successfully:
System.out.println(new MyStrings("A", "B", "C").new NestedNonStaticSubclass().getFunction().apply(2)); //Should print "C".
}
}
MyStrings class:
package server;
import java.util.function.IntFunction;
public class MyStrings {
public static class NestedStatic {
private final IntFunction<String> f;
public NestedStatic(final IntFunction<String> f) {
this.f = f;
}
public IntFunction<String> getFunction() {
return f;
}
}
//This class produces the error when loaded:
public class NestedNonStaticSubclass extends NestedStatic {
public NestedNonStaticSubclass() {
super(i -> getString(i)); //Here we refer to MyStrings object before constructing the NestedNonStaticSubclass object.
}
}
private final String[] array;
public MyStrings(final String... array) {
this.array = array.clone();
}
public String getString(final int i) {
return array[i];
}
}
Client side code:
package client;
import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.net.Socket;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Objects;
import javax.swing.JOptionPane;
public class LiveClientTest {
public static class MemoryClassLoader extends ClassLoader {
//The key is the name of the class, and value is the compiled class file bytes:
private final HashMap<String, byte[]> classByteCode = new HashMap<>();
#Override
public /*synchronized*/ Class findClass(final String name) throws ClassNotFoundException {
try {
return super.findClass(name);
}
catch (final ClassNotFoundException cnfx) {
if (!classByteCode.containsKey(name))
throw new ClassNotFoundException(name);
final byte[] byteCode = classByteCode.get(name);
return defineClass(name, byteCode, 0, byteCode.length);
}
}
//Try to load all classes that are downloaded (with readClass):
ArrayList<String> loadClasses() {
final ArrayList<String> classNames = new ArrayList<>(classByteCode.keySet());
int oldSize;
do {
oldSize = classNames.size();
final Iterator<String> classNamesIter = classNames.iterator();
while (classNamesIter.hasNext()) {
final String className = classNamesIter.next();
try {
loadClass(className);
classNamesIter.remove();
}
catch (final ClassNotFoundException x) {
}
}
}
while (classNames.size() < oldSize); //If we reached a point where there are some classes that can not be loaded, then quit...
return classNames; //Return the classes that failed to be loaded (if any) (should be empty).
}
//Read class bytes from supplied DataInputStream:
void readClass(final DataInputStream dis) throws IOException {
final String name = dis.readUTF();
final byte[] contents = new byte[dis.readInt()];
int i = 0, n = dis.read(contents);
//Make sure all 'contents.length' multitude of bytes are read:
while (n >= 0 && (i + n) < contents.length) {
i += n;
n = dis.read(contents, i, contents.length - i);
}
if (/*n < 0 ||*/ (i + n) != contents.length)
throw new IOException("Unfinished class input (" + name + " - " + contents.length + ").");
classByteCode.put(name, contents);
}
}
public static void main(final String[] args) {
try {
final String host = Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host name or address:"));
final int port = Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host port number:")));
try (final Socket sck = new Socket(host, port);
final InputStream is = sck.getInputStream()) {
final DataInputStream dis = new DataInputStream(is);
final MemoryClassLoader loader = new MemoryClassLoader();
//Download all classes and put them into the class loader:
System.out.println("Downloading...");
for (int i = dis.readInt(); i > 0; --i)
loader.readClass(dis);
//Load all downloaded classes from the class loader:
System.out.println("Loading...");
System.out.println("Failed to load: " + loader.loadClasses() + '.');
//Call main method in main downloaded class:
loader
.loadClass("server.ClientMain") //package is from server side.
.getDeclaredMethod("main", Socket.class, DataInputStream.class, ClassLoader.class, String[].class)
.invoke(null, sck, dis, loader, args);
}
}
catch (final IOException | ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InvocationTargetException | RuntimeException x) {
x.printStackTrace();
JOptionPane.showMessageDialog(null, x);
}
}
}
Client side output:
Downloading...
Loading...
Exception in thread "main" java.lang.ClassFormatError: Illegal field name "server.MyStrings$this" in class server/MyStrings$NestedNonStaticSubclass
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
at client.LiveClientTest$MemoryClassLoader.findClass(LiveClientTest.java:30)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at client.LiveClientTest$MemoryClassLoader.loadClasses(LiveClientTest.java:44)
at client.LiveClientTest.main(LiveClientTest.java:89)
So my question is:
Why does the code fail with a ClassFormatError, what does that mean, and how to avoid it in this particular scenario?
My question is not:
What alternatives exist? (such as using a URLClassLoader, or alternative ways of class loading from memory other than a custom ClassLoader, etc...)
How to reproduce:
I am using JRE 1.8.0_251 (and I would like this to work for 1.8), so I think you must put the source files in different projects (one for the client and one for the server) in order to make sure that the client does not already have direct visibility of the server's classes while class-loading them.
Run the server's main class server.LiveServerTest and input a port number for the server in the dialog that pops up. Then, run the client's main class client.LiveClientTest and enter localhost for the host (first dialog that pops up) and then the port number of the server (second dialog that pops up).
The stack trace will be in your CLI (through System.err) and not in a GUI.
The code will not work if you build the projects into jar files, but it is written to work for plain class files into directories, for simplicity. For example, in NetBeans, don't build into jar files, but rather click Run File for each entry point.
The built classes should have their file's .class extention in lower case.
Warning:
If you put the source files in different packages but in the same project, or even worse in the same package, then the class loading might succeed without errors because:
I am using the default constructor of the ClassLoader class in my LiveClientTest.MemoryClassLoader class, which means the parent class loader is the system class loader.
The LiveClientTest.MemoryClassLoader.findClass method first searches the parent ClassLoader and then, if that fails, it searches the downloaded classes. To my knowledge, this is the suggested way of implementing this, mainly because the ClassLoader class (which is the parent class of my LiveClientTest.MemoryClassLoader class) caches already defined classes.
References:
How to load JAR files dynamically at Runtime?
In which scenarios the remote class loading are needed?
Does the Java ClassLoader load inner classes?
Java - Get a list of all Classes loaded in the JVM
Java: How to load a class (and its inner classes) that is already on the class path?
Create new ClassLoader to reload Class
How to use classloader to load class file from server to client
Custom Java classloader and inner classes
ClassFormatError in java 8?
JVM Invalid Nested Class Name?
https://bugs.openjdk.java.net/browse/JDK-8145051
https://www.programming-books.io/essential/java/implementing-a-custom-classloader-0f0fe95cf7224c668e631a671eef3b94
https://www.baeldung.com/java-classloaders
https://www.infoworld.com/article/2077260/learn-java-the-basics-of-java-class-loaders.html
https://www.oracle.com/technical-resources/articles/javase/classloaders.html
https://www.ibm.com/developerworks/library/j-onejar/index.html
I am new to class loading, so please don't take my words for granted.
Huge post, because of the divided code of the MRE. Sorry. I tried to make the code as minimal as possible.
In my java code I call another 3rd party java class.
I want to catch that latter System.exit() exit code
So I use security-manager as suggested in this post
The problem is that I cannot read files now, I get permissions error
as seen in that post.
How can I catch the exit code and still read files?
Published class MyClass {
class MySecurityManager extends SecurityManager {
#Override
public void checkExit(int status) {
throw new SecurityException();
}
}
public void foo() {
MySecurityManager secManager = new MySecurityManager();
System.setSecurityManager(secManager);
try {
ConfigValidator.main(new String[]{"-dirs", SdkServiceConfig.s.PROPERTIES_FILE_PATH});
new FileInputStream(new File("/Users/eladb/WorkspaceQa/sdk-service/src/main/resources/convert_conditions.sh"));
} catch (SecurityException e) {
//Do something if the external code used System.exit()
String a = "1";
}
catch (Exception e) {
logger.error("failed converting properties file to proto", e);
}
}
}
You have two separate problems: Your trusted code cannot read the file, while the untrusted third-party library can still call System#exit unhindered. The former can be easily circumvented by granting further privileges to the trusted code; the latter is a tad trickier to address.
A bit of background
Privilege assignment
Code (the ProtectionDomains encapsulated by a thread's AccessControlContext) generally gets assigned Permissions in two ways: Statically, by the ClassLoader, upon class definition, and/or dynamically, by the Policy in effect. Other, less frequently encountered alternatives, exist as well: DomainCombiners, for instance, can modify AccessControlContexts' domains (and therefore the effective permissions of their respective code that is subject to authorization) on the fly, and custom domain implementations may use their own logic for permission implication, possibly disregarding or altering the semantics of the policy. By default the permission set of a domain is the union of its static and dynamic permissions. As for how exactly classes are mapped to domains, it is, for the most part, up to the loader's implementation. By default, all classes, JAR'ed or otherwise, residing beneath the same class path entry, are grouped under the same domain. More restrictive class loaders may choose to e.g. allocate a domain per class, which could be used to prevent communication even between classes within the same package.
Privilege evaluation
Under the default SecurityManager, for a privileged operation (an invocation of any method having a SecurityManager#checkXXX within its body) to succeed, every domain (of every class of every method) of the effective AccessControlContext must have been assigned, as explained above, the permission being checked. Recall however that the context need not necessarily represent "the truth" (the actual call stack)—system code gets optimized away early on, while AccessController#doPrivileged calls, along with the DomainCombiner potentially coupled to the AccessControlContext can modify the context's domains, and the authorization algorithm in its entirety, consequently.
Problem and workarounds
The issue with System#exit is that the corresponding permission (RuntimePermission("exitVM.*")) is one amongst few that are statically assigned by the default application class loader (sun.misc.Launcher$AppClassLoader) to all domains associated with classes loaded from the class path.
A number of alternatives come to mind:
Installing a custom SecurityManager which denies the particular right based on, e.g., the class attempting to terminate the JVM process.
Loading the third-party library from a "remote" location (a directory outside of the class path), so that it gets treated as "untrusted" code by its class loader.
Authoring and installing a different application class loader, which does not assign the extraneous permission.
Plugging a custom domain combiner into the access control context, which replaces, at the time of an authorization decision, all third-party domains with equivalent ones that do not have the offending permission.
I should, for the sake of completeness, note that at the Policy level, unfortunately, nothing can be done to negate statically assigned permissions.
The first option is overall the most convenient one, but I will not explore it further because:
The default SecurityManager is quite flexible, thanks to the handful of components (AccessController et al.) it interacts with. The background section in the beginning aimed to serve as a reminder of that flexibility, which "quick-n'-dirty" method overrides tend to cripple.
Careless modifications of the default implementation might cause (system) code to misbehave in curious ways.
Frankly, because it's boring―it's the one-size-fits-all solution perpetually advocated, while the fact that the default manager was standardized in 1.2 for a reason has long been forgotten.
The second alternative is easy to implement but impractical, complicating either development or the build process. Assuming you are not planning to invoke the library solely reflectively, or aided by interfaces present on the class path, it would have to be present initially, during development, and relocated before execution.
The third is, at least in the context of a standalone Java SE application, fairly straightforward and should not pose too much of a burden on performance. It is the approach I will favour herein.
The last option is the most novel and least convenient. It is hard to securely implement, has the greatest potential for performance degradation, and burdens client code with ensuring presence of the combiner prior to every delegation to untrusted code.
Proposed solution
The custom ClassLoader
The following is to be used as the replacement of the default application loader, or alternatively as the context class loader, or the loader used to load at least the untrusted classes. There is nothing novel to this implementation—all it does is prevent delegation to the default application class loader when the class in question is assumed to not be a system one. URLClassLoader#findClass, in turn, does not assign RuntimePermission("exitVM.*") to the domains of the classes it defines.
package com.example.trusted;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.regex.Pattern;
public class ClasspathClassLoader extends URLClassLoader {
private static final Pattern SYSTEM_CLASS_PREFIX = Pattern.compile("((java(x)?|sun|oracle)\\.).*");
public ClasspathClassLoader(ClassLoader parent) {
super(new URL[0], parent);
String[] classpath = System.getProperty("java.class.path").split(File.pathSeparator);
for (String classpathEntry : classpath) {
try {
if (!classpathEntry.endsWith(".jar") && !classpathEntry.endsWith("/")) {
// URLClassLoader assumes paths without a trailing '/' to be JARs by default
classpathEntry += "/";
}
addURL(new URL("file:" + classpathEntry));
}
catch (MalformedURLException mue) {
System.err.println(MessageFormat.format("Erroneous class path entry [{0}] skipped.", classpathEntry));
}
}
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
Class<?> ret;
synchronized (getClassLoadingLock(name)) {
ret = findLoadedClass(name);
if (ret != null) {
return ret;
}
if (SYSTEM_CLASS_PREFIX.matcher(name).matches()) {
return super.loadClass(name, resolve);
}
ret = findClass(name);
if (resolve) {
resolveClass(ret);
}
}
return ret;
}
}
If you also wish to fine-tune the domains assigned to loaded classes, you will additionally have to override findClass. The following variant of the loader is a very crude attempt at doing so. constructClassDomain therein merely creates one domain per class path entry (which is more or less the default), but can be modified to do something different.
package com.example.trusted;
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.ByteBuffer;
import java.security.AccessController;
import java.security.CodeSource;
import java.security.PrivilegedAction;
import java.security.ProtectionDomain;
import java.security.cert.Certificate;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;
public final class ClasspathClassLoader extends URLClassLoader {
private static final Pattern SYSTEM_CLASS_PREFIX = Pattern.compile("((java(x)?|sun|oracle)\\.).*");
private static final List<WeakReference<ProtectionDomain>> DOMAIN_CACHE = new ArrayList<>();
// constructor, loadClass same as above
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
URL classOrigin = getClassResource(name);
if (classOrigin == null) {
return super.findClass(name);
}
URL classCodeSourceOrigin = getClassCodeSourceResource(classOrigin);
if (classCodeSourceOrigin == null) {
return super.findClass(name);
}
return defineClass(name, readClassData(classOrigin), constructClassDomain(classCodeSourceOrigin));
}
private URL getClassResource(String name) {
return AccessController.doPrivileged((PrivilegedAction<URL>) () -> getResource(name.replace(".", "/") + ".class"));
}
private URL getClassCodeSourceResource(URL classResource) {
for (URL classpathEntry : getURLs()) {
if (classResource.getPath().startsWith(classpathEntry.getPath())) {
return classpathEntry;
}
}
return null;
}
private ByteBuffer readClassData(URL classResource) {
try {
BufferedInputStream in = new BufferedInputStream(classResource.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
int i;
while ((i = in.read()) != -1) {
out.write(i);
}
return ByteBuffer.wrap(out.toByteArray());
}
catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
private ProtectionDomain constructClassDomain(URL classCodeSourceResource) {
ProtectionDomain ret = getCachedDomain(classCodeSourceResource);
if (ret == null) {
CodeSource cs = new CodeSource(classCodeSourceResource, (Certificate[]) null);
DOMAIN_CACHE.add(new WeakReference<>(ret = new ProtectionDomain(cs, getPermissions(cs), this, null)));
}
return ret;
}
private ProtectionDomain getCachedDomain(URL classCodeSourceResource) {
for (WeakReference<ProtectionDomain> domainRef : DOMAIN_CACHE) {
ProtectionDomain domain = domainRef.get();
if (domain == null) {
DOMAIN_CACHE.remove(domainRef);
}
else if (domain.getCodeSource().implies(new CodeSource(classCodeSourceResource, (Certificate[]) null))) {
return domain;
}
}
return null;
}
}
The "unsafe" code
package com.example.untrusted;
public class Test {
public static void testExitVm() {
System.out.println("May I...?!");
System.exit(-1);
}
}
The entry point
package com.example.trusted;
import java.security.AccessControlException;
import java.security.Permission;
import com.example.untrusted.Test;
public class Main {
private static final Permission EXIT_VM_PERM = new RuntimePermission("exitVM.*");
public static void main(String... args) {
System.setSecurityManager(new SecurityManager());
try {
Test.testExitVm();
}
catch (AccessControlException ace) {
Permission deniedPerm = ace.getPermission();
if (EXIT_VM_PERM.implies(deniedPerm)) {
ace.printStackTrace();
handleUnauthorizedVmExitAttempt(Integer.parseInt(deniedPerm.getName().replace("exitVM.", "")));
}
}
}
private static void handleUnauthorizedVmExitAttempt(int exitCode) {
System.out.println("here let me do it for you");
System.exit(exitCode);
}
}
Testing
Packaging
Place the loader and the main class in one JAR (let's call it trusted.jar) and the demo untrusted class in another (untrusted.jar).
Assigning privileges
The default Policy (sun.security.provider.PolicyFile) is backed by the file at <JRE>/lib/security/java.policy, as well as any of the files referenced by the policy.url.n properties in <JRE>/lib/security/java.security. Modify the former (the latter should hopefully be empty by default) as follows:
// Standard extensions get all permissions by default
grant codeBase "file:${{java.ext.dirs}}/*" {
permission java.security.AllPermission;
};
// no default permissions
grant {};
// trusted code
grant codeBase "file:///path/to/trusted.jar" {
permission java.security.AllPermission;
};
// third-party code
grant codeBase "file:///path/to/untrusted.jar" {
permission java.lang.RuntimePermission "exitVM.-1", "";
};
Note that it is virtually impossible to get components extending the security infrastructure (custom class loaders, policy providers, etc.) to work properly without granting them AllPermission.
Running
Run:
java -classpath "/path/to/trusted.jar:/path/to/untrusted.jar" -Djava.system.class.loader=com.example.trusted.ClasspathClassLoader com.example.trusted.Main
The privileged operation should succeed.
Next comment out the RuntimePermission under untrusted.jar, within the policy file, and re-run. The privileged operation should fail.
As a closing note, when debugging AccessControlExceptions, running with -Djava.security.debug=access=domain,access=failure,policy can help track down offending domains and policy configuration issues.
I need to read and modify the 'Compressed' attribute of a file on an NTFS partition from Java. I imagined something in the java.nio.file.attribute package would do it -- hell it's a complex enough package, but I can't find this attribute.
The DosFileAttributes class has getters for the classic hidden/system/readonly/archive attributes only.
I tried Files.readAttributes which allows dynamically retrieving all attributes from a particular "attribute view". Under "dos:*" there was only the same attributes that are already available from the public methods of the DosFileAttributes class. I tried "ntfs:*" and "windows:*" but they weren't accepted as valid view names.
I also tried the UserDefinedFileAttributeView, but it gave me an empty list on any file I tried.
I wondered about shelling out to the attrib command (accepting the limitation that it wouldn't work for NTFS partitions mounted under Linux or other OSes) but that doesn't seem to support the attribute either. Help?
Since this does seem to be missing from the standard Java API, I took a look at doing it myself with JNA. It's my first experience with JNA. It's not as pretty as I'd like and JNA's code seems to be sorely lacking in generics, but it's a hundred times better than mucking about with JNI and trying to set up awful cross-compilers for the different platforms needed (minimum x86 & x64 even if you target only one OS). It's the annoying compilation process that drove me from C++ to Java in the first place, and I hope never to have to return to it.
Anyway, this seems to work. Hopefully it will be useful to someone else too. It provides four public methods:
isAvailable() -- whether or not calling the other methods should work (i.e., we are on Windows and the JNA native lib loaded okay)
isCompressed(File)
setCompressed(File, boolean)
volumeSupportsFileCompression(File) -- asks Windows if the partition where a file is located supports [individual] file compression. E.g., it's true on NTFS, and false on FAT (USB sticks and so on).
Compression in the Windows API is done through a dedicated I/O control operation, and is not merely a "SetAttributes" call. If it were simpler (isomorphic with other file attributes) I'd have put the encryption attribute in there as well for completeness' sake, but anyway.
import java.io.File;
import java.io.IOException;
import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.ptr.IntByReference;
import com.sun.jna.ptr.ShortByReference;
import com.sun.jna.win32.StdCallLibrary;
import com.sun.jna.win32.W32APIOptions;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.WinNT.HANDLE;
public class WindowsFileOps {
private WindowsFileOps() {}
private static interface Kernel32Extra extends StdCallLibrary {
int COMPRESSION_FORMAT_NONE = 0x00000000;
int COMPRESSION_FORMAT_DEFAULT = 0x00000001;
int FSCTL_SET_COMPRESSION = 0x0009C040;
Kernel32Extra INSTANCE = (Kernel32Extra)Native.loadLibrary("kernel32",
Kernel32Extra.class, W32APIOptions.UNICODE_OPTIONS);
boolean GetVolumeInformation(
String lpRootPathName,
Pointer lpVolumeNameBuffer,
int nVolumeNameSize,
IntByReference lpVolumeSerialNumber,
IntByReference lpMaximumComponentLength,
IntByReference lpFileSystemFlags,
Pointer lpFileSystemNameBuffer,
int nFileSystemNameSize
);
}
private static Boolean isAvailable;
public static boolean isAvailable() {
if (isAvailable == null) {
try {
isAvailable = Kernel32.INSTANCE != null && Kernel32Extra.INSTANCE != null;
} catch (Throwable t) {
isAvailable = false;
}
}
return isAvailable;
}
private static String pathString(File file) {
// "\\?\" is a Windows API thing that enables paths longer than 260 chars
return "\\\\?\\" + file.getAbsolutePath();
}
private static int getAttributes(File file) throws IOException {
int attrib = Kernel32.INSTANCE.GetFileAttributes(pathString(file));
if (attrib == Kernel32.INVALID_FILE_ATTRIBUTES) {
throw new IOException("Unable to read file attributes of " + file);
}
return attrib;
}
public static boolean isCompressed(File file) throws IOException {
return (getAttributes(file) & Kernel32.FILE_ATTRIBUTE_COMPRESSED) != 0;
}
public static void setCompressed(File file, boolean compressed) throws IOException {
HANDLE hFile = Kernel32.INSTANCE.CreateFile(
pathString(file),
Kernel32.GENERIC_READ | Kernel32.GENERIC_WRITE,
Kernel32.FILE_SHARE_READ,
null,
Kernel32.OPEN_EXISTING,
0,
null);
try {
if (!Kernel32.INSTANCE.DeviceIoControl(
hFile,
Kernel32Extra.FSCTL_SET_COMPRESSION,
new ShortByReference((short)(
compressed
? Kernel32Extra.COMPRESSION_FORMAT_DEFAULT
: Kernel32Extra.COMPRESSION_FORMAT_NONE
)).getPointer(),
2,
null, 0,
new IntByReference(),
null
)) throw new IOException("Unable to alter compression attribute of " + file);
} finally {
Kernel32.INSTANCE.CloseHandle(hFile);
}
}
public static boolean volumeSupportsFileCompression(File file) throws IOException {
IntByReference flags = new IntByReference();
if (!Kernel32Extra.INSTANCE.GetVolumeInformation(
pathString(file.getAbsoluteFile().toPath().getRoot().toFile()),
null, 0,
null,
null,
flags,
null, 0
)) throw new IOException("GetVolumeInformation failure");
return (flags.getValue() & Kernel32.FILE_FILE_COMPRESSION) != 0;
}
}
Instead of DosFileAttributes try use BasicFileAttributes and check isOther() and isRegularFile() flags. That may tell you if file is compressed. If not you would have to create your own FileSystemProvider impl for NTFS or write small JNI code that will use WinAPI to read that flag for you.
I m trying to externalize the configuration of aop.xml so I removed the aop.xml from META-INF and made it available in the server for manual configuration by sys admins.
When I try to use an external aop.xml using
-Dorg.aspectj.weaver.loadtime.configuration="file:D:\Workspace\tomcat7\shared\lib\aop.xml"
I get java.lang.RuntimeException: Cannot register non aspect: aspectclass.... mainly because the aj casses are not loaded by AppClassLoader yet at that time. And the next time it tries to register the aspects from the WebAppClassLoader ( after all the classes are loaded), it works fine, but i still get the exceptions logged from the 1st attempt to register it.
The exception is caught and logged at ClassLoaderWeavingAdaptor.java line 307.
when the following line is called:
success = registerAspects(weaver, loader, definitions);
the exception is caught and logged.
try {
registerOptions(weaver, loader, definitions);
registerAspectExclude(weaver, loader, definitions);
registerAspectInclude(weaver, loader, definitions);
success = registerAspects(weaver, loader, definitions);
registerIncludeExclude(weaver, loader, definitions);
registerDump(weaver, loader, definitions);
} catch (Exception ex) {
trace.error("register definition failed", ex);
success = false;
warn("register definition failed", (ex instanceof AbortException) ? null : ex);
}
the exception is thrown excactly in the following line in BcelWeaver.java
if (type.isAspect()) {
......
} else {
// FIXME AV - better warning upon no such aspect from aop.xml
RuntimeException ex = new RuntimeException("Cannot register non aspect: " + type.getName() + " , " + aspectName);
if (trace.isTraceEnabled()) {
trace.exit("addLibraryAspect", ex);
}
throw ex;
}
How can I prevent the classLoader from logging the error to the console, when the aspects are not loaded yet. I was thinking of commenting the line that logs the exception from the source file and rebuilding the aspectjweaver jar file, but was looking for a better solution without modifying the aspectj source.
I am not sure that there is an easy way out of your problem. As I said I haven't worked with AspectJ before but I believe this is a mis-behaviour of the weaver.
Problem description: During boot the agent tries to apply weaving other not only to the WebAppClassLoader but to the whole classloader chain (once per classloader) i.e. to: sun.misc.Launcher$AppClassLoader, sun.misc.Launcher$ExtClassLoader, org.apache.catalina.loader.StandardClassLoader (the tomcat's classloader). When you use the META-INF/aop.xml approach it disables weaving for the above classloaders because "a configuration file is not available" (if you enable verbose mode you can see those messages in console). When you use the file configuration approach, a configuration is available for all the classloaders in the chain. Since it does find a configuration file, the agent parses the definitions, it does not find the aspects' class and shows the error.
The weird thing is that, as described in the configuration documentation if you use the WeavingURLClassLoader approach for load time weaving, "... it also allows the user to explicitly restrict by class loader which classes can be woven". So this is actually a feature (!) that the classloader approach can have but the agent approach doesn't. (Unfortunately I was not able to use this approach)
The good (and the bad) news: The good news is that you can easily create your own agent that will ignore the weaving for the aforementioned classloaders. The bad news is that restricting weaving per classloader is not enough because if you have other applications in the same server, Tomcat would still use the WebAppClassLoader to load them so you would still get error messages for those applications. (Perhaps you could extend the classes below to filter packages/classes as well, in that case).
Below you can find two class for the modified agent. To use them you would need to do the following:
Un-jar the aspectjweaver.jar to a folder
Under org/aspectj/weaver/loadtime create a new folder filter to match the package name and put there the two new classes after you compile them.
Edit the META-INF/MANIFEST.MF file and change the line
Premain-Class: org.aspectj.weaver.loadtime.Agent to
Premain-Class: org.aspectj.weaver.loadtime.filter.FilterAgent
Re-jar and you have your new agent ready.
When starting the JVM you can now pass a new system property with a comma separated list of the classloaders you would like to ignore i.e. -Dorg.aspectj.weaver.loadtime.filter=sun.misc.Launcher$AppClassLoader,sun.misc.Launcher$ExtClassLoader,org.apache.catalina.loader.StandardClassLoader ( I have set CATALINA_OPTS to do that).
The classes are a modified copy of the original agent's classes Agent and ClassPreProcessorAgentAdapter. The only code I have added is the part that parses the above system property if it exists and to ignore calls for the classloaders we are not interested in.
Use at your own risk :) I hope that helps
package org.aspectj.weaver.loadtime.filter;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.Instrumentation;
public class FilterAgent {
private static Instrumentation s_instrumentation;
// Use our own version of ClassFileTransformer that would filter out selected classloaders
private static ClassFileTransformer s_transformer = new ClassPreprocessorFilteredAdapter();
/**
* JSR-163 preMain Agent entry method
*
* #param options
* #param instrumentation
*/
public static void premain(String options, Instrumentation instrumentation) {
/* Handle duplicate agents */
if (s_instrumentation != null) {
return;
}
s_instrumentation = instrumentation;
s_instrumentation.addTransformer(s_transformer);
}
public static Instrumentation getInstrumentation() {
if (s_instrumentation == null) {
throw new UnsupportedOperationException("Java 5 was not started with preMain -javaagent for AspectJ");
}
return s_instrumentation;
}
}
//-----------------------------------------------------------------------------------
package org.aspectj.weaver.loadtime.filter;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;
import java.util.HashMap;
import java.util.Map;
import org.aspectj.weaver.loadtime.Aj;
import org.aspectj.weaver.loadtime.ClassPreProcessor;
public class ClassPreprocessorFilteredAdapter implements ClassFileTransformer {
/**
* Concrete preprocessor.
*/
private static ClassPreProcessor s_preProcessor;
private static Map<String, String> ignoredClassloaderNames = new HashMap<String, String>();
static {
try {
s_preProcessor = new Aj();
s_preProcessor.initialize();
String ignoredLoaders = System.getProperty("org.aspectj.weaver.loadtime.filter", "");
if (ignoredLoaders.length() > 0) {
String[] loaders = ignoredLoaders.split(",");
for (String s : loaders) {
s = s.trim();
ignoredClassloaderNames.put(s, s);
System.out.println("---> Will filtered out classloader: " + s);
}
}
} catch (Exception e) {
throw new ExceptionInInitializerError("could not initialize JSR163 preprocessor due to: " + e.toString());
}
}
/**
* Invokes the weaver to modify some set of input bytes.
*
* #param loader the defining class loader
* #param className the name of class being loaded
* #param classBeingRedefined is set when hotswap is being attempted
* #param protectionDomain the protection domain for the class being loaded
* #param bytes the incoming bytes (before weaving)
* #return the woven bytes
*/
#Override
public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] bytes) throws IllegalClassFormatException {
if (classBeingRedefined != null) {
System.err.println("INFO: (Enh120375): AspectJ attempting reweave of '" + className + "'");
}
String loaderName = loader.getClass().getName();
if (shouldIgnoreClassLoader(loaderName)) {
return bytes;
}
return s_preProcessor.preProcess(className, bytes, loader, protectionDomain);
}
private boolean shouldIgnoreClassLoader(String loaderName) {
boolean result = false;
String ignoredLoader = ignoredClassloaderNames.get(loaderName);
if (ignoredLoader != null) {
result = true; // if the loader name exists in the map we will ignore weaving
}
return result;
}
}
If you need the feature to exclude classloaders from weaving with the agent approach, there is a developer build available providing a new command line switch -Daj.weaving.loadersToSkip to do that. The topic is being discussed on a thread of the AspectJ users mailing list. The feature will probably make it into AspectJ 1.7.4, but is not available in 1.7.3 yet.
Update:
The feature did make it into AspectJ 1.7.4 even though it is not explicitly mentioned in the release notes, but listed under resolved issues for that release.
What I ended up doing is changing the LOG Level for the error message from ERROR to DEBUG, as I don't see this as an ERROR ( at least in my case ). this case I can still see the error when I enable the DEBUG level. so I modified the source file below and rebuild my aspectjweaver-1.7.1.jar
try {
registerOptions(weaver, loader, definitions);
registerAspectExclude(weaver, loader, definitions);
registerAspectInclude(weaver, loader, definitions);
success = registerAspects(weaver, loader, definitions);
registerIncludeExclude(weaver, loader, definitions);
registerDump(weaver, loader, definitions);
} catch (Exception ex) {
//(CHANGE 1) trace.error("register definition failed", ex);
trace.debug( "register definition failed" + ex.getMessage());
success = false;
// (CHANGE 2) warn("register definition failed", (ex instanceof AbortException) ? null : ex);
debug("register definition failed" + ((ex instanceof AbortException) ? null : ex));
}
Our java application relies on some resources which are available on a network share. This network share is located on the classpath, and the resources are read at runtime using MyClass.class.getResourceAsStream("/myfile.jpg").
java -Djava.class.path=\\myserver\myshare:C:\myjar.jar MainClass
When the share is available at startup, everything runs smoothly. Image and properties files which are located in the share can be read using getResourceAsStream(). However, if the share is not online when the application starts, even if the share comes online before any resources are read, they cannot be read using getResourceAsStream().
Doing some digging using eclispse + decompiler, I noticed one difference. The default classloader inherits from URLClassLoader, and its ucp member (URLClassPath) contains a list of URLClassPath.Loader instances. In the first scenario, it contains a URLClassPath.FileLoader and a URLClassPath.JarLoader. In the second scenario, it only contains a jar loader.
It's like java determines that the classpath entry is invalid and completely discards it.
Why is this? How can I avoid it?
Update
I am unable to change the mechanism by which we are loading resources because of a few reasons:
There are far too many areas which currently load files this way for me change at the moment
There are situations where by the resource is actually being loaded by a third party component
I have no problem creating a custom class loader, I just need some guidance on how to do it.
I tried with this, but was unable to get expected results:
import java.net.URL;
import java.net.URLClassLoader;
public class MyUrlClassLoader extends URLClassLoader {
public MyUrlClassLoader(ClassLoader parent) {
super(new URL[0], parent);
System.out.println("MyUrlClassLoader ctor");
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
System.out.println("url find class " + name);
return super.findClass(name);
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
System.out.println("url load class " + name);
return super.loadClass(name);
}
#Override
public URL getResource(String name) {
System.out.println("url get resource " + name);
return super.getResource(name);
}
}
import java.net.URL;
public class ClassLoaderMain {
public static void main(String[] args) throws ClassNotFoundException {
URL url = ClassLoaderMain.class.getResource("/myfile.txt");
System.out.print("Loaded? ");
System.out.println(url != null);
System.out.println(ClassLoaderMain.class.getClassLoader().toString());
System.out.println(MyUrlClassLoader.class.getClassLoader().toString());
System.out.println(FakeClass.class.getClassLoader().toString());
}
}
When I run java -cp . -Djava.system.class.loader=MyUrlClassLoader ClassLoaderMain
This outputs:
MyUrlClassLoader ctor
url load class java.lang.System
url load class java.nio.charset.Charset
url load class java.lang.String
url load class ClassLoaderMain
Loaded? true
sun.misc.Launcher$AppClassLoader#923e30
sun.misc.Launcher$AppClassLoader#923e30
sun.misc.Launcher$AppClassLoader#923e30
So my class loader is being created, and load class is being called, but it doesn't appear to be the class loader for the classes it is loading?
I ended up resolving this by creating my own ClassLoader, deriving from URLClassLoader.
import java.io.File;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.List;
public class CustomClassLoader extends URLClassLoader {
public CustomClassLoader(ClassLoader parent) {
// System classloader will filter inaccessible URLs.
// Force null parent to avoid using system classloader.
super(createURLReferences(), null);
}
/**
* Build an array of URLs based on the java.class.path property.
* #return An array of urls to search for classes.
*/
private static URL[] createURLReferences() {
String classpath = System.getProperty("java.class.path");
String[] classpathEntries = classpath.split(System.getProperty("path.separator"));
List<URL> urls = new ArrayList<URL>();
for (String classpathEntry : classpathEntries) {
File classpathFile = new File(classpathEntry);
URI uri = classpathFile.toURI();
try {
URL url = uri.toURL();
urls.add(url);
} catch (MalformedURLException e) {
System.out.println("Ignoring classpath entry: " + classpathEntry);
}
}
return urls.toArray(new URL[urls.size()]);
}
}