We Keep Coding

JDK-18 foreign functions: How to work with multiple functions that have the same name, but come from different native libraries?

Let's say I loaded a native library gg3 that has a function get_gg(), and would like to load another native library gg4 that also has a get_gg() function - how can I look one up in a "library aware" manner (for example, use the get_gg() from gg4 on certain occasions, and otherwise from gg3)?
I tried CLinker.systemCLinker().lookup() but that seems to just fetch the first symbol that is found in the loaded native libraries.
After searching in the internal classes, I found SystemLookup.libLookup() which seemed promising but isn't public, so I implemented a similar method to lookup a loaded native lib:
private static SymbolLookup libLookup(Function<NativeLibraries, NativeLibrary> loader) {
NativeLibrary lib = loader.apply(NativeLibraries.rawNativeLibraries(SystemLookup.class, false));
return name -> {
Objects.requireNonNull(name);
try {
long addr = lib.lookup(name);
return addr == 0 ?
Optional.empty() :
Optional.of(NativeSymbol.ofAddress(name, MemoryAddress.ofLong(addr), ResourceScope.globalScope()));
} catch (NoSuchMethodException e) {
return Optional.empty();
}
};
}
However, this apparently is prohibited, because I cannot load the library from another class loader (and I don't know what class loader was used originally):
java.lang.UnsatisfiedLinkError: Native Library already loaded in another classloader
So how can I either find & use the appropriate class loader, or manage to find the library without re-loading it, so I can look for a symbol specifically from it?

FWIW, the API needed to do this was made public in 19
Before that, the advice was to manually implement this by binding dlopen, dlclose, and dlsym (or the Windows equivalents) to load the library and look up symbols.
e.g. on my Windows machine:
public class Main {
static final MethodHandle MH_LoadLibraryA;
static final MethodHandle MH_FreeLibrary;
static final MethodHandle MH_GetProcAddress;
static {
System.loadLibrary("Kernel32");
SymbolLookup lookup = SymbolLookup.loaderLookup();
CLinker linker = CLinker.systemCLinker();
MH_LoadLibraryA = linker.downcallHandle(
lookup.lookup("LoadLibraryA").orElseThrow(),
FunctionDescriptor.of(ADDRESS, ADDRESS));
MH_FreeLibrary = linker.downcallHandle(
lookup.lookup("FreeLibrary").orElseThrow(),
FunctionDescriptor.of(JAVA_INT, ADDRESS));
MH_GetProcAddress = linker.downcallHandle(
lookup.lookup("GetProcAddress").orElseThrow(),
FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS));
}
static MemoryAddress LoadLibraryA(String name) {
try (ResourceScope scope = ResourceScope.newConfinedScope()) {
SegmentAllocator allocator = SegmentAllocator.nativeAllocator(scope);
return (MemoryAddress) MH_LoadLibraryA.invokeExact((Addressable) allocator.allocateUtf8String(name));
} catch (Throwable t) {
throw new RuntimeException(t);
}
}
static boolean FreeLibrary(MemoryAddress handle) {
try {
return ((int) MH_FreeLibrary.invokeExact((Addressable) handle)) != 0;
} catch (Throwable t) {
throw new RuntimeException(t);
}
}
static MemoryAddress GetProcAddress(MemoryAddress handle, String name) {
try (ResourceScope scope = ResourceScope.newConfinedScope()) {
SegmentAllocator allocator = SegmentAllocator.nativeAllocator(scope);
return (MemoryAddress) MH_GetProcAddress.invokeExact((Addressable) handle, (Addressable) allocator.allocateUtf8String(name));
} catch (Throwable t) {
throw new RuntimeException(t);
}
}
static SymbolLookup libraryLookup(String libraryName, ResourceScope scope) {
MemoryAddress handle = LoadLibraryA(libraryName);
if (handle == MemoryAddress.NULL) {
throw new IllegalArgumentException("Cannot find library: " + libraryName);
}
scope.addCloseAction(() -> FreeLibrary(handle));
return name -> {
var addr = GetProcAddress(handle, name);
return addr == MemoryAddress.NULL ?
Optional.empty() : Optional.of(NativeSymbol.ofAddress(name, addr, scope));
};
}
public static void main(String[] args) throws Throwable {
try (ResourceScope scope = ResourceScope.newConfinedScope()) {
SymbolLookup lookup = libraryLookup("gg", scope);
MethodHandle handle = CLinker.systemCLinker().downcallHandle(
lookup.lookup("get_gg").orElseThrow(),
FunctionDescriptor.of(JAVA_INT));
int x = (int) handle.invoke();
System.out.println(x);
}
}
}

User URLClassLoader to load jar file "on the fly"

Ok, basically, I try to use the method described here JarFileLoader to load a jar containing a class that will be used the same as if it was on the classpath (the class name will be dynamic so that we can just add any jar with any class and the program will load it through parsing a text file, in the main line).
Problem is that when I debug and check the URLClassLoader object
protected Class<?> findClass(final String name)
Line :
Resource res = ucp.getResource(path, false);
the getResource() does not find the class name in parameter.
Does someone already try loading a jar file this way ?
Thanks.
Loader :
public class JarFileLoader extends URLClassLoader {
public JarFileLoader() {
super(new URL[] {});
}
public JarFileLoader withFile(String jarFile) {
return withFile(new File(jarFile));
}
public JarFileLoader withFile(File jarFile) {
try {
if (jarFile.exists())
addURL(new URL("file://" + jarFile.getAbsolutePath() + "!/"));
} catch (MalformedURLException e) {
throw new IllegalArgumentException(e);
}
return this;
}
public JarFileLoader withLibDir(String path) {
Stream.of(new File(path).listFiles(f -> f.getName().endsWith(".jar"))).forEach(this::withFile);
return this;
}
}
Main :
public static void main(String[] args) {
new Initializer();
JarFileLoader cl = new JarFileLoader();
cl = cl.withFile(new File("libs/dpr-common.jar"));
try {
cl.loadClass("com.*****.atm.dpr.common.util.DPRConfigurationLoader");
System.out.println("Success!");
} catch (ClassNotFoundException e) {
System.out.println("Failed.");
e.printStackTrace();
} finally {
try {
cl.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Here the test class I used. When I debug URLClassLoader I can see in the third loop the path of the jar file(loop on the classpath and the URL you add here), but still does not find ressource (and cannot debug the class URLClassPath so do not know what getRessource does exactly).

Ok I take the answer from this question : How to load all the jars from a directory dynamically?
And changing the URL part at the beginning with the way it is done in the long part it works.
So an example could be :
String path = "libs/dpr-common.jar";
if (new File(path).exists()) {
URL myJarFile = new File(path).toURI().toURL();
URL[] urls = { myJarFile };
URLClassLoader child = new URLClassLoader(urls);
Class DPRConfLoad = Class.forName("com.thales.atm.dpr.common.util.DPRConfigurationLoader", true, child);
Method method = DPRConfLoad.getDeclaredMethod("getInstance");
final Object dprConf = method.invoke(DPRConfLoad);
}
All my time wasted in search while it was the example which was wrong... Still does not understand why they use a stupid URL like "jar:file..." etc.
Thanks everyone.

Issue with loading class from jar file represented as byte array

I'm trying to create an instance of a class from jar file loaded on a byte array.
I'm receiving two args:
1. byte[] which represents jar file with required class
2. Qualified class name
When I'm testing it locally it works as expected, but when I upload exactly the same jar file with the same qualified class name remotely (using web application implemented with Spring MVC for back and AngularJS for front end deployed in Tomcat server) It can't find the required class:
java.lang.ClassNotFoundException
When I was debugging, it turned out, that classloader is properly invoked but no one class is loaded from jar.
I would be grateful if anyone can tell what can be the reason of that difference or how can I implement this functionality in other ways.
A method which loads class and returns an instance of it:
public static <T> T getInstanceOfLoadedClass(byte[] jarFileBytes, String qualifiedClassName) throws ClassFromJarInstantiationException {
LOGGER.info("Getting instance of class from loaded jar file. Class name: " + qualifiedClassName);
try {
return (T) Class.forName(qualifiedClassName, true, new ByteClassLoader(jarFileBytes)).newInstance();
} catch (InstantiationException | IllegalAccessException | IOException | ClassNotFoundException | NoSuchFieldException e) {
LOGGER.error("Exception was thrown while reading jar file for " + qualifiedClassName + "class.", e);
throw new ClassFromJarInstantiationException(e);
}
}
Custom ByteClassLoader:
public class ByteClassLoader extends ClassLoader {
private static final Logger LOGGER = Logger.getLogger(ByteClassLoader.class);
private final byte[] jarBytes;
private final Set<String> names;
public ByteClassLoader(byte[] jarBytes) throws IOException {
this.jarBytes = jarBytes;
this.names = loadNames(jarBytes);
}
private Set<String> loadNames(byte[] jarBytes) throws IOException {
Set<String> set = new HashSet<>();
try (ZipInputStream jis = new ZipInputStream(new ByteArrayInputStream(jarBytes))) {
ZipEntry entry;
while ((entry = jis.getNextEntry()) != null) {
set.add(entry.getName());
}
}
return Collections.unmodifiableSet(set);
}
#Override
public InputStream getResourceAsStream(String resourceName) {
if (!names.contains(resourceName)) {
return null;
}
boolean found = false;
ZipInputStream zipInputStream = null;
try {
zipInputStream = new ZipInputStream(new ByteArrayInputStream(jarBytes));
ZipEntry entry;
while ((entry = zipInputStream.getNextEntry()) != null) {
if (entry.getName().equals(resourceName)) {
found = true;
return zipInputStream;
}
}
} catch (IOException e) {
LOGGER.error("ByteClassLoader threw exception while reading jar byte stream for resource: "+resourceName, e);
e.printStackTrace();
} finally {
if (zipInputStream != null && !found) {
try {
zipInputStream.close();
} catch (IOException e) {
LOGGER.error("ByteClassLoader threw exception while closing jar byte stream for resource: "+resourceName, e);
e.printStackTrace();
}
}
}
return null;
} }

The problem was that the class required to be loaded was in a range of classloader while it was tested.
Hope it helps someone in solving this problem because it is really easy to miss.

ClassNotFoundException when deserializing a binary class file's contents

I don't know much about Java. I'm trying to read a file containing an int and various instances of a class called "Automobile". When I deserialize it, though, the program throws a ClassNotFoundException and I can't seem to understand why.
Here's the code:
try {
FileInputStream fin = new FileInputStream(inputFile);
ObjectInputStream input = new ObjectInputStream(fin);
conto = input.readInt();
Automobile[] macchine = new Automobile[conto];
for(int i = 0; i < conto; i++) {
macchine[i] = (Automobile)input.readObject();
}
String targa;
System.out.print("\nInserire le cifre di una targa per rintracciare l'automobile: ");
targa = sc1.nextLine();
for(int i = 0; i < conto; i++) {
if(macchine[i].getTarga().equals(targa))
System.out.println(macchine[i]);
}
} catch(IOException e) {
System.out.println("Errore nella lettura del file "+inputFile);
} catch(java.lang.ClassNotFoundException e) {
System.out.println("Class not found");
}
Thanks in advance.
EDIT: here's the stacktrace
java.lang.ClassNotFoundException: es4.Automobile
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:604)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1575)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at es4p2.Main.main(Main.java:35)

When you deserialize a serialized object tree, the classes of all the objects have to be on the classpath. In this context, a ClassNotFoundException most likely means that one of the classes required is not on the classpath. You have to address this for deserialization to work.
In this case, the es4.Automobile is missing.
Could the problem be caused by a custom exception I made which is fired by Automobile?
The only other possibilities I can think of are:
es4.Automobile has a direct or indirect dependency on some other class that is missing
the static initialization of es4.Automobile or a dependent class has thrown an exception that has not been caught internally to the class.
But both of those should (I think) have resulted in a different stack trace.
I just noticed the package name is es4p2, not es4. Why does it say es4? Could it be because the program which saves the file uses another package name?
I've no idea why they are different. You'd need to talk to whoever wrote the code / produced the serialized objects. However, this is most likely the cause of your problem. A class with a different package name is a different class. Period.
You should always output (or better, log) the stacktrace when an unexpected exception is caught. That will tell you (and us) more about what has gone wrong, and in this case the name of the class that is missing.

This is and old question but this may help someone else. I faced the same issue and the problem was that I was not using the current thread class loader. You will find below the serializer class that I used in a grails project, should be quite straightforward use this in java
Hope this helps
public final class Serializer<T> {
/**
* Converts an Object to a byte array.
*
* #param object, the Object to serialize.
* #return, the byte array that stores the serialized object.
*/
public static byte[] serialize(T object) {
ByteArrayOutputStream bos = new ByteArrayOutputStream()
ObjectOutput out = null
try {
out = new ObjectOutputStream(bos)
out.writeObject(object)
byte[] byteArray = bos.toByteArray()
return byteArray
} catch (IOException e) {
e.printStackTrace()
return null
} finally {
try {
if (out != null)
out.close()
} catch (IOException ex) {
ex.printStackTrace()
return null
}
try {
bos.close()
} catch (IOException ex) {
ex.printStackTrace()
return null
}
}
}
/**
* Converts a byte array to an Object.
*
* #param byteArray, a byte array that represents a serialized Object.
* #return, an instance of the Object class.
*/
public static Object deserialize(byte[] byteArray) {
ByteArrayInputStream bis = new ByteArrayInputStream(byteArray)
ObjectInput input = null
try {
input = new ObjectInputStream(bis){
#Override protected Class<?> resolveClass(final ObjectStreamClass desc) throws IOException, ClassNotFoundException {
ClassLoader cl = Thread.currentThread().getContextClassLoader();
if (cl == null) return super.resolveClass(desc);
return Class.forName(desc.getName(), false, cl);
}
};
Object o = input.readObject()
return o
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace()
return null
} finally {
try {
bis.close()
} catch (IOException ex) {
}
try {
if (input != null)
input.close()
} catch (IOException ex) {
ex.printStackTrace()
return null
}
}
}

This generally happens if your class Automobile is not in the runtime classpath.

I fixed it in an easier way than the other answers -- my problem occurred when using the class in multiple projects.
If you have multiple projects, make sure that the specific class you're deserializing is in the exact same path! That means, the same package names etc inside that project. Otherwise it won't find it and cause the ClassNotFoundException to be thrown.
So if it's in
/myPackage/otherPackage/Test.java
Then make sure, that path is exactly the same in your other project.

Ive had a similar problem with a ObjectInputStream reading serialized Objects. The classes for those Objects i added at runtime with a URLClassloader. The problem was that the ObjectInputStream did not use the Thread ClassLoader which i set with
Thread.currentThread().setContextClassLoader(cl);
but instead the AppClassLoader, which you cannot customize with java 9. So i made my own ObjectInputStream as a subtype of the original one and overidden the resolveClass Method:
#Override
protected Class<?> resolveClass(ObjectStreamClass desc) throws IOException, ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, Thread.currentThread()
.getContextClassLoader());
} catch (ClassNotFoundException ex) {
Class<?> cl = primClasses.get(name);
if (cl != null) {
return cl;
} else {
throw ex;
}
}
}

Does your Automobile class have a field like this?
private static final long serialVersionUID = 140605814607823206L; // some unique number
If not, define one. Let us know if that fixes it.

You can access the class name from the message of the ClassNotFound exception - it's horrible to depend on this in the code - but it should give you some idea. I wish there were some better way of getting information about serialised objects without having to have the class available.

GWT Server-side reflection

I am trying to use Reflection on the server side only of a GWT app. I have a basic example working in a non-GWT example which can be seen below.
package com.xyz.reflection;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
public class EntryPoint {
/**
* #param args
*/
public static void main(String[] args) {
ClassLoader dynClassLoader = ClassLoader.getSystemClassLoader();
Class<?> dynClass = null;
try {
dynClass = dynClassLoader.loadClass("com.xyz.reflection.RunMe");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
Object dynInstance = null;
try {
dynInstance = dynClass.newInstance();
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
Method dynMethod = null;
try {
try {
dynMethod = dynInstance.getClass().getMethod("returnSid",
new Class[] { PassMe.class });
} catch (SecurityException e) {
e.printStackTrace();
} catch (NoSuchMethodException e) {
e.printStackTrace();
}
String returnValue = (String) dynMethod.invoke(dynInstance,
new Object[] { new PassMe() });
System.out.println("Return Value: " + returnValue.toString());
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
}
using the aptly named:
package com.xyz.reflection;
public class PassMe {
private String sid = "DEFAULT_SID";
public PassMe() {
this.sid = "INITIATED_SID";
}
public String getSid() {
return sid;
}
public void setSid(String sid) {
this.sid = sid;
}
}
and:
package com.xyz.reflection;
public class RunMe {
public String returnSid(PassMe s) {
return s.getSid();
}
}
This runs fine. When I try running this from a GWT server side class it doesn't work, and instead returns
java.lang.NoSuchMethodException: com.xyz.reflection.RunMe.returnSid(com.xyz.reflection.PassMe)
If I change the parameter to a String (instead of the 'PassMe' class) it works fine. Why does it not like passing my 'PassMe' class? I thought it might be an issue with serialization despite being 100% server code, but I haven't had any luck with this either.
Thanks in advance for any help anyone can give me with this.

In addition to finrod's solution, you can also change your class loader to something like:
ClassLoader dynClassLoader = PassMe.class.getClassLoader();
And you can use the PassMe.class style lookup again. The class used to find a loader doesn't seem to matter though. Just not the system loader.
Weird stuff. I wouldn't doubt if GWT is doing something weird with the class loader though.
Edit: Yep. GWT sets the system class loader to com.google.appengine.tools.development.IsolatedAppClassLoader in dev mode.

I think this could be related to Class loading - but it is just a hunch as I cannot experiment with it in context similar to yours.
Here are some suggestions to try:
You use:
ClassLoader dynClassLoader = ClassLoader.getSystemClassLoader();
dynClass = dynClassLoader.loadClass("com.xyz.reflection.RunMe");
To load the RunMe Class.
However to load the PassMe class you use:
PassMe.class
Try to load the PassMe Class through the dynClassLoader and use that instance in the getMethod() instead of the PassMe.class.
I wonder, do you need to use the dynClassLoader?

This is pretty much a shot in the dark, but does it help if you replace
dynMethod = dynInstance.getClass().getMethod("returnSid",
new Class[] { PassMe.class });
with
dynMethod = dynInstance.getClass().getMethod("returnSid", PassMe.class );
?
It doesn't make a difference outside of a GWT server, but it may exercise the container's VM differently.

Did you put your reflection file in the server side package?
For example:
org.myproject.client - your gwt client package (put here your java files that are able to be complied into java script NO REFLECTION)
org.myproject.server - put here any java files including reflection
org.myproject.shared - put here java classes that are able to be compiled into java script