We Keep Coding

sbt issues with my ClassBuilder implementation (Scala)

What is Intellij doing for me that sbt isn't?
I am working on a Scala project for which I'd like to support user-defined plugin classes. I've implemented a subclass of ClassLoader:
class PluginLoader(directory: String) extends ClassLoader {
override def findClass(className: String): Class[_] = {
val bytes = loadClassData(className)
defineClass(className,bytes,0,bytes.length)
}
def loadClassData(className: String): Array[Byte] = {
val fileName = className.replace('.',File.separatorChar) + ".class"
val file = new File(directory,fileName)
val bytes = new Array[Byte](file.length.toInt)
val inputStream = new DataInputStream(new FileInputStream(file))
inputStream.readFully(bytes)
inputStream.close()
bytes
}
}
and I call this with user-defined directory and name of a pre-compiled scala class:
new PluginLoader(directory).loadClass(className).newInstance()
findClass appears to succeed at finding the class the user provides. However, it is then called again looking for scala.runtime.java8.JFunction2$mcIII$sp.class. It looks in the user-provided directory, so it cannot find the loaded class.
Here's the error I get: (run-main-0) java.io.FileNotFoundException: ./scala/runtime/java8/JFunction2$mcIII$sp.class (No such file or directory)
I have successfully run this application in Intellij, which makes me think I am missing some configuration in sbt, and Intellij is catching my mistake for me.
Do I need to do additional configuration? Should I be loading in an entire jar file instead of just passing a class file?

When you compile a class - not only Scala class, Java class as well - then it not necessarily mean that you will get a single .class file.
Internal classes, lambdas/closures, etc will result with additional bytecode, that is required to run, and which doesn't live in the same file as your "original" class.
As far as I can tell here Java created some class because it is the body of some lambda, and you did not loaded it.
Bottom line is: you'd better load everything. IntelliJ might have simply passed a whole classpath to JVM, so your classloader might not have even used your extensions (though you'd have to attach debugger in order to be sure).

How to extract the file jre-9/lib/modules?

In JRE-9/lib directory (at least on Windows), there is a new file called modules whose size is about 107 MB. Is it possible to extract that file or maybe list java modules within it?
I can see that a new tool called jmod is available at jdk-9/bin/jmod.exe, but that is for reading .jmod files which is located at jdk-9/jmods and it cannot read the file modules.

The modules file is a container file. It's internal to the JDK and the format is not documented (it may change at any time). For troubleshooting purposes, the jimage tool in the bin directory can be used to list or extract the contents.

The runtime resources are handled in a backwards compatible way. E.g. when you did
URL url = Object.class.getResource("Object.class");
System.out.println(url);
in the past, you usually got something like
jar:file:/path-to-jre/lib/rt.jar!/java/lang/Object.class
Running the same under Java 9 will give you
jrt:/java.base/java/lang/Object.class
instead. In either case, you can open a FileSystem on it to inspect the other available resources (since Java 7). While the ZipFileSystem had to be created via FileSystems.newFileSystem first, Java 9’s file system is even already open for use:
private static void readMyOwnJRE() throws IOException {
try {
Path p = Paths.get(URI.create("jrt:/")).resolve("/modules");
System.out.println("My own JRE's modules:");
Files.list(p).forEach(m -> System.out.println(m.getFileName()));
System.out.println();
} catch(FileSystemNotFoundException ex) {
System.out.println("Could not read my modules (perhaps not Java 9?).");
}
}
If you are running under a different JRE than the one you want to inspect, you have to load the appropriate file system implementation manually first, but this opens the possibility to inspect a Java 9 installation even from a Java 8 JRE:
public static void readOtherJRE(Path pathToJRE) throws IOException {
Path p = pathToJRE.resolve("lib").resolve("jrt-fs.jar");
if(Files.exists(p)) {
try(URLClassLoader loader = new URLClassLoader(new URL[]{ p.toUri().toURL() });
FileSystem fs = FileSystems.newFileSystem(URI.create("jrt:/"),
Collections.emptyMap(),
loader)) {
System.out.println("Modules of "+pathToJRE);
Files.list(fs.getPath("/modules")).forEach(System.out::println);
System.out.println();
}
}
}
Once you have a file system (or a Path into it), you can use all standard functions of, e.g. Files to inspect or extract/copy the data, though the right term would be “to store an equivalent class file” in a different file system, as the runtime image’s representation doesn’t have to be a class file at all.

The modules file is meant to be a single file (not meant to be extracted anywhere) which contains a binary representation of all of the modules present in the JDK in an undocument format which is subject to change. You can list the modules it contains with java --list-modules.
Initially the modules file will contain every module and basically double the side of the JDK on its own, but once you "minify" your JDK using the jlink utility, the modules file will become smaller (assuming your program contains a subset of the modules provided with the JDK). For more info on jlink see here: http://openjdk.java.net/jeps/282

you can use
$ jimage list $JAVA_HOME/lib/modules
to list all the system packages' name,like "java/io/EOFException.class",etc.
also,you can use
$ jimage extract --dir=<directory> $JAVA_HOME/lib/modules
to extract all the .class files into the specified directory.
reference:https://adoptopenjdk.gitbooks.io/adoptopenjdk-getting-started-kit/en/intermediate-steps/openjdk9-jimage.html

You can list modules via java --list-modules

You can use libjimage to read this file.

Extract resource folder from running jar in Java 7

My resources folder inside my jar includes a directory with several binary files. I am attempting to use this code to extract them:
try(InputStream is = ExternalHTMLThumbnail.class.getResourceAsStream("/wkhtmltoimage")) {
Files.copy(is, Paths.get("/home/dan/wkhtmltoimage");
}
This is throwing the error
java.nio.file.NoSuchFileException: /home/dan/wkhtmltoimage
Which comes from
if (errno() == UnixConstants.ENOENT)
return new NoSuchFileException(file, other, null);
in UnixException.java. Even though in Files.java the correct options are passed:
ostream = newOutputStream(target, StandardOpenOption.CREATE_NEW,
StandardOpenOption.WRITE);
from Files.copy. Of course there's not! That's why I'm trying to make it. I don't yet understand Path and Files enough to do this right. What's the best way to extract the directory and all its contents?
Confused because the docs for Files.copy claims
By default, the copy fails if the target file already exists or is a symbolic link
(Apparently it fails if the target file doesn't exist as well?)
And lists the possible exceptions, and NoSuchFileException is not one of them.

If you're using Guava:
URL url = Resources.getResource(ExternalHTMLThumbnail.class, "wkhtmltoimage");
byte[] bytes = Resources.toByteArray(url);
Files.write(bytes, new File("/my/path/myFile"));
You could of course just chain that all into one line; I declared the variables to make it more readable.

The file that does not exist may actually be the directory you're trying to create the file in.
/home/dan/wkhtmltoimage
Does /home/dan exist? Probably not if you're on a Mac.

Loading Resources from the ClassPath in Java ME

In Java I can have multiple instances of the some resource in the class path, and can access them thus:
ClassLoader loader = Thread.currentThread().getContextClassLoader();
Enumeration<URL> resources = loader.getResources("META-INF/services/myService.properties");
while(resources.hasMoreElements()){
URL resource = resources.nextElement();
// process each resource here
}
This allows use of service provider interfaces, where multiple implementations can be made available at runtime by dropping additional jars into the classpath.
Is there any equivalent to this in Java ME?
[UPDATE 2011-April-11]
If I could somehow get a list of jar files in the classpath, I could probably just pass the URLs to the Connector like this:
String [] jarFiles = ... // somehow I got this
for(int i = jarFiles.length-1; i >= 0; i--){
String url = "jar:file:"+jarFiles[i]+"!/META-INF/services/myService.properties";
InputStream in = Connector.openInputStream(url);
//process each resource here
}
Any idea how I might get all jar files in the classpath in J2ME?

After some research, it looks like the application must be contained within a single Jar file in a J2ME environment, so concepts like the classpath simply do not apply, meaning that this is not possible. All dependencies must be available at compile time.

Can a loaded JAR be deleted by the Java-Process?

Hello I have following Problem:
Within an uninstall-process I load a JAR (jdbc-driver).
URL pDriverJar = jarToDelete.toURI().toURL();
URL[] lURLList = new URL[]{pDriverJar};
URLClassLoader lLoader = new URLClassLoader(lURLList, Thread.currentThread().getContextClassLoader());
Thread.currentThread().setContextClassLoader(lLoader);
Class<?> aClass = Class.forName("jdbc.Driver"); // was Oracle: oracle.jdbc.OracleDriver but should not be important
if(jarToDelete.delete()){
System.out.println("deleted");
}else {
jarToDelete.deleteOnExit();
}
After terminiation of the JVM, the jar is still existant.
As a workarround, I've created a tempfile, and copied the Jar to that tempfile. But now the Tempfile will not be deleted.
I've read, that if the ClassLoad is GC, the loaded jars can be removed.
Does anyone have an Idea, how to delete this File?

It depends on the operating system. Windows will not let you delete files that are in-use, but Linux will.
One solution would be to start a second process that waits for your JVM to die and then deletes the file, as even if you clear all references to classloaders using it, there is no guarantee that they will release the file. There is no way to force garbage collection (or even finalization) of an object.
Another solution would be to write the classloader that loads the Jar. That way, when you want to get rid of it, you can be certain that the Jar is closed. If the only object that opened it was your classloader, then you can be certain it is free and should be deletable.

This issue was fixed in Java 7; use the close() method in the ClassLoader class. For older versions, there are several options:
Write a custom classloader, e.g. like this
Use reflection and close all JarFile instances; they reside in sun.misc.URLClassPath

As I wanted to kill off a jar-file on the classpath that I needed for java11 but led to problems in java 1.8 or below; A solution that I used, without having to start another process to kill the jar file, was to just make the file 0 bytes by
FileOutputStream out = new FileOutputStream(f);
out.write(new byte[0]);
out.flush(); out.close();
And next time the application starts this code then manages to delete it, even though the 0-byte-file is still in the manifest:
if (f.exists()){
if (f.delete() == false) f.deleteOnExit();
}