UnsatisfiedLinkError in non-updated bundle when updating a client bundle - java

I have a bundle ("com.elsten.bliss.platform") that uses another bundle ("net.contentobjects.jnotify"). net.contentobjects.jnotify loads native libraries to do its job, which are contributed by fragments. The native code is loaded in a static class initializer inside a class within the net.contentobjects.jnotify bundle:
static
{
System.loadLibrary("jnotify");
int res = nativeInit();
if (res != 0)
{
throw new RuntimeException("Error initializing fshook_inotify library. linux error code #" + res + ", man errno for more info");
}
init();
}
I can start and stop com.elsten.bliss.platform and it appears to work ok. The problem comes when updating com.elsten.bliss.platform. When I update, I get:
2013-04-04 11:58:20,356 [ERROR] Couldn't initialise JNotify: java.lang.UnsatisfiedLinkError: Native Library /home/gravelld/eclipse-workspaces/bliss/net.contentobjects.jnotify.linux.amd64/lib/libjnotify.so already loaded in another classloader (JnotifyFileSystemObserver.java:53, thread platformExecutor)
java.lang.UnsatisfiedLinkError: Native Library /home/gravelld/eclipse-workspaces/bliss/net.contentobjects.jnotify.linux.amd64/lib/libjnotify.so already loaded in another classloader
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1715)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1675)
at java.lang.Runtime.loadLibrary0(Runtime.java:840)
at java.lang.System.loadLibrary(System.java:1047)
at net.contentobjects.jnotify.linux.JNotify_linux.<clinit>(JNotify_linux.java:48)
at net.contentobjects.jnotify.linux.JNotifyAdapterLinux.<init>(JNotifyAdapterLinux.java:76)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at java.lang.Class.newInstance0(Class.java:374)
at java.lang.Class.newInstance(Class.java:327)
at net.contentobjects.jnotify.JNotify.<clinit>(JNotify.java:75)
at com.elsten.bliss.platform.storage.file.JnotifyFileSystemObserver.startWatching(JnotifyFileSystemObserver.java:43)
at com.elsten.bliss.platform.storage.file.NotifyFilesAtStartFileSystemObserver.start(NotifyFilesAtStartFileSystemObserver.java:117)
at com.elsten.bliss.platform2.PlumbedStorageSubsystem.start(PlumbedStorageSubsystem.java:69)
at com.elsten.bliss.client.impl.ConfigurationClientImpl$3.doRun(ConfigurationClientImpl.java:337)
at com.elsten.util.CatchThrowableRunnable.run(CatchThrowableRunnable.java:23)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Ok, so I know you can only have any one native library loaded by one classloader. But in this case, why is there more than one class loader? The net.contentobjects.jnotify is not updated, so I thought the existing classes should just continue on and be used by JnotifyFileSystemObserver.startWatching - it appears this method is reloading the class.
How do I avoid this?

Interestingly, that code if called twice should not raise an exception; the javadoc for java.lang.Runtime#loadLibrary() states that
If this method is called more than once with the same library name, the second and subsequent calls are ignored.
So, there might be more to it. Have you tried setting the singleton property in the Bundle-SymbolicName entry of the MANIFEST.MF?

Related

When load some class in checkPermission method why SecurityManager emit recursive update exception?

I'm upgrading jdk 8 to 11.
I load some class in checkPermission method then security manager emit recursive update exception. but use jdk1.8.0_202 everything works fine.
What causes this problem?
My environment.
OS: macOS 10.15.6
JDK(Oracle): 11.0.8
IDE: Intellij 2019 3
Main
public class Main {
public static void main(String[] args) {
System.out.println("Hello world");
}
}
SecurityManager
package sm;
import java.security.Permission;
public class MySecurityManager extends SecurityManager {
#Override
public void checkPermission(Permission permission) {
// Problem occurs when load ServicePermission.class
if (permission instanceof javax.security.auth.kerberos.ServicePermission) {
// throw new SecurityException("javax.security.auth.kerberos.ServicePermission is not allowed.");
}
}
#Override
public void checkPermission(Permission permission, Object context) {
this.checkPermission(permission);
}
}
Run with -Djava.security.manager=sm.MySecurityManager
Console logs
Error occurred during initialization of VM
java.lang.BootstrapMethodError: bootstrap method initialization exception
at java.lang.invoke.BootstrapMethodInvoker.invoke(java.base#11.0.8/BootstrapMethodInvoker.java:194)
at java.lang.invoke.CallSite.makeSite(java.base#11.0.8/CallSite.java:307)
at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(java.base#11.0.8/MethodHandleNatives.java:258)
at java.lang.invoke.MethodHandleNatives.linkCallSite(java.base#11.0.8/MethodHandleNatives.java:248)
at sun.net.www.protocol.jrt.JavaRuntimeURLConnection.<clinit>(java.base#11.0.8/JavaRuntimeURLConnection.java:55)
at sun.net.www.protocol.jrt.Handler.openConnection(java.base#11.0.8/Handler.java:42)
at java.net.URL.openConnection(java.base#11.0.8/URL.java:1074)
at jdk.internal.module.SystemModuleFinders$SystemModuleReader.checkPermissionToConnect(java.base#11.0.8/SystemModuleFinders.java:405)
at jdk.internal.module.SystemModuleFinders$SystemModuleReader.<init>(java.base#11.0.8/SystemModuleFinders.java:414)
at jdk.internal.module.SystemModuleFinders$2.get(java.base#11.0.8/SystemModuleFinders.java:315)
at jdk.internal.module.SystemModuleFinders$2.get(java.base#11.0.8/SystemModuleFinders.java:312)
at jdk.internal.module.ModuleReferenceImpl.open(java.base#11.0.8/ModuleReferenceImpl.java:93)
at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base#11.0.8/BuiltinClassLoader.java:961)
at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base#11.0.8/BuiltinClassLoader.java:958)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base#11.0.8/ConcurrentHashMap.java:1705)
at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base#11.0.8/BuiltinClassLoader.java:969)
at jdk.internal.loader.BuiltinClassLoader.defineClass(java.base#11.0.8/BuiltinClassLoader.java:731)
at jdk.internal.loader.BuiltinClassLoader.lambda$findClassInModuleOrNull$2(java.base#11.0.8/BuiltinClassLoader.java:682)
at java.security.AccessController.doPrivileged(java.base#11.0.8/Native Method)
at jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(java.base#11.0.8/BuiltinClassLoader.java:683)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:605)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:640)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:609)
at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base#11.0.8/BuiltinClassLoader.java:579)
at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base#11.0.8/ClassLoaders.java:178)
at java.lang.ClassLoader.loadClass(java.base#11.0.8/ClassLoader.java:521)
at sm.MySecurityManager.checkPermission(MySecurityManager.java:11)
at java.lang.SecurityManager.checkPropertyAccess(java.base#11.0.8/SecurityManager.java:1066)
at java.lang.System.getProperty(java.base#11.0.8/System.java:814)
at java.lang.ClassLoader.initSystemClassLoader(java.base#11.0.8/ClassLoader.java:1971)
at java.lang.System.initPhase3(java.base#11.0.8/System.java:2070)
Caused by: java.lang.IllegalStateException: Recursive update
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base#11.0.8/ConcurrentHashMap.java:1760)
at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base#11.0.8/BuiltinClassLoader.java:969)
at jdk.internal.loader.BuiltinClassLoader.defineClass(java.base#11.0.8/BuiltinClassLoader.java:731)
at jdk.internal.loader.BuiltinClassLoader.lambda$findClassInModuleOrNull$2(java.base#11.0.8/BuiltinClassLoader.java:682)
at java.security.AccessController.doPrivileged(java.base#11.0.8/Native Method)
at jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(java.base#11.0.8/BuiltinClassLoader.java:683)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:605)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:640)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:609)
at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base#11.0.8/BuiltinClassLoader.java:579)
at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base#11.0.8/ClassLoaders.java:178)
at java.lang.ClassLoader.loadClass(java.base#11.0.8/ClassLoader.java:521)
at sm.MySecurityManager.checkPermission(MySecurityManager.java:11)
at java.lang.reflect.AccessibleObject.checkPermission(java.base#11.0.8/AccessibleObject.java:83)
at java.lang.reflect.Constructor.setAccessible(java.base#11.0.8/Constructor.java:180)
at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base#11.0.8/InnerClassLambdaMetafactory.java:206)
at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base#11.0.8/InnerClassLambdaMetafactory.java:199)
at java.security.AccessController.doPrivileged(java.base#11.0.8/Native Method)
at java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(java.base#11.0.8/InnerClassLambdaMetafactory.java:198)
at java.lang.invoke.LambdaMetafactory.metafactory(java.base#11.0.8/LambdaMetafactory.java:329)
at java.lang.invoke.BootstrapMethodInvoker.invoke(java.base#11.0.8/BootstrapMethodInvoker.java:127)
at java.lang.invoke.CallSite.makeSite(java.base#11.0.8/CallSite.java:307)
at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(java.base#11.0.8/MethodHandleNatives.java:258)
at java.lang.invoke.MethodHandleNatives.linkCallSite(java.base#11.0.8/MethodHandleNatives.java:248)
at sun.net.www.protocol.jrt.JavaRuntimeURLConnection.<clinit>(java.base#11.0.8/JavaRuntimeURLConnection.java:55)
at sun.net.www.protocol.jrt.Handler.openConnection(java.base#11.0.8/Handler.java:42)
at java.net.URL.openConnection(java.base#11.0.8/URL.java:1074)
at jdk.internal.module.SystemModuleFinders$SystemModuleReader.checkPermissionToConnect(java.base#11.0.8/SystemModuleFinders.java:405)
at jdk.internal.module.SystemModuleFinders$SystemModuleReader.<init>(java.base#11.0.8/SystemModuleFinders.java:414)
at jdk.internal.module.SystemModuleFinders$2.get(java.base#11.0.8/SystemModuleFinders.java:315)
at jdk.internal.module.SystemModuleFinders$2.get(java.base#11.0.8/SystemModuleFinders.java:312)
at jdk.internal.module.ModuleReferenceImpl.open(java.base#11.0.8/ModuleReferenceImpl.java:93)
at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base#11.0.8/BuiltinClassLoader.java:961)
at jdk.internal.loader.BuiltinClassLoader$5.apply(java.base#11.0.8/BuiltinClassLoader.java:958)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base#11.0.8/ConcurrentHashMap.java:1705)
at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base#11.0.8/BuiltinClassLoader.java:969)
at jdk.internal.loader.BuiltinClassLoader.defineClass(java.base#11.0.8/BuiltinClassLoader.java:731)
at jdk.internal.loader.BuiltinClassLoader.lambda$findClassInModuleOrNull$2(java.base#11.0.8/BuiltinClassLoader.java:682)
at java.security.AccessController.doPrivileged(java.base#11.0.8/Native Method)
at jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(java.base#11.0.8/BuiltinClassLoader.java:683)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:605)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:640)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base#11.0.8/BuiltinClassLoader.java:609)
at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base#11.0.8/BuiltinClassLoader.java:579)
at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base#11.0.8/ClassLoaders.java:178)
at java.lang.ClassLoader.loadClass(java.base#11.0.8/ClassLoader.java:521)
at sm.MySecurityManager.checkPermission(MySecurityManager.java:11)
at java.lang.SecurityManager.checkPropertyAccess(java.base#11.0.8/SecurityManager.java:1066)
at java.lang.System.getProperty(java.base#11.0.8/System.java:814)
at java.lang.ClassLoader.initSystemClassLoader(java.base#11.0.8/ClassLoader.java:1971)
at java.lang.System.initPhase3(java.base#11.0.8/System.java:2070)
Process finished with exit code 1
The stack trace indicates that the issue is connected with the module loading rather than class loading, which explains why you don’t have the problem in JDK 8 that doesn’t have modules.
When you read the stack trace starting at the bottom, i.e.
at java.lang.System.initPhase3(java.base#11.0.8/System.java:2070)
you will encounter the stack frames
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base#11.0.8/ConcurrentHashMap.java:1705)
at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base#11.0.8/BuiltinClassLoader.java:969)
indicating an attempt to load a module. This will eventually end up at a privileged action that needs a check, so you’ll find the line
at sm.MySecurityManager.checkPermission(MySecurityManager.java:11)
which triggers the loading of javax.security.auth.kerberos.ServicePermission which is in the module java.security.jgss which apparently has not been loaded before.
So the loadClass call ends up again at
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(java.base#11.0.8/ConcurrentHashMap.java:1760)
at jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(java.base#11.0.8/BuiltinClassLoader.java:969)
which triggers the “java.lang.IllegalStateException: Recursive update”, as calling computeIfAbsent is not allowed from another computeIfAbsent call on the same ConcurrentHashMap. Since ignoring this constraint can lead to corrupted maps, a check has been added in Java 9 to reject such attempts. See this Q&A.
Generally, triggering class loading from a security manager that might get checked again during the class loading can be problematic. I suggest resorting to the documented toString() output for comparison. After all, that’s what the policy file based security implementation does as well.
Since ServicePermission is final, a cheaper permission.getClass().getName().equals( "javax.security.auth.kerberos.ServicePermission") would do as well. Both approaches avoid loading the permission if it has not been used before. As indicated by the issue, this may even save the loading of an entire module.

Flink can not find groovy class during checkpoint

i have a problem in flink. my real-time compute engine use groovy script to expend compute type(like:sum、average、count adn so on). we define a standard compute interface(AbstractCompute),and if i want expand a compute type in this framework i just need impl AbstractCompute.and then store groovy script in the db. then application can read script by task and load into jvm by GroovyClassLoader.
This process does not use Flink again, depending on the work is very good。The reason is that Flink uses another ClassLoader (FlinkUserCodeClassLoaders$ChildFirstClassLoader) to load the object instantiated by the groovy script at checkpoint instead of using GroovyClassLoader.
Code
// Init Groovy ClassLoader
CompilerConfiguration classLoaderConfig = new CompilerConfiguration();
classLoaderConfig.setSourceEncoding("UTF-8");
CLASS_LOADER = new GroovyClassLoader(Thread.currentThread().getContextClassLoader(), classLoaderConfig);
......
......
// parse script and new instance and put into cache
Class clazz = CLASS_LOADER.parseClass(computeType.getScript());
AbstractComputable computableObject = (AbstractComputable) clazz.newInstance();
removeComputeType(computeType);
// 自定义计算方式对象存入缓存
IndicatorCache.COMPUTABLE_OBJECT_CACHE.put(computeType.getId().intValue(), computableObject);
......
......
AbstractComputable computable = IndicatorCache.COMPUTABLE_OBJECT_CACHE.get(indicator.getComputeType());
if (computable == null) {
if (log.isDebugEnabled()) {
log.debug("without computeType:{} in cache", indicator.getComputeType());
}
return false;
}
indicator.setComputableObject(computable);
Exception stack:
com.esotericsoftware.kryo.KryoException: Unable to find class: com.xxx.xxx.common.computable.CurValueCompute
Serialization trace:
computableObject (com.xxx.xxx.common.pojo.property.IndicatorProperty)
normalIndicatorList (com.xxx.xxx.common.pojo.property.ComputeTuple)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:657)
at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:231)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:577)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:579)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:40)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:579)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
at org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollectWithTimestamp(StreamSourceContexts.java:310)
at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collectWithTimestamp(StreamSourceContexts.java:409)
at org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher.emitRecordWithTimestamp(AbstractFetcher.java:398)
at org.apache.flink.streaming.connectors.kafka.internal.Kafka010Fetcher.emitRecord(Kafka010Fetcher.java:89)
at org.apache.flink.streaming.connectors.kafka.internal.Kafka09Fetcher.runFetchLoop(Kafka09Fetcher.java:154)
at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:738)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.xxx.xxx.common.computable.CurValueCompute
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:129)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)
... 41 common frames omitted
How to use Groovy dynamic language correctly in Flink?
Flink needs to know the types it is processing. Otherwise it is not possible to serialize and deserialize the instances. Therefore, the class definitions need to be contained in the user code jar which you submit to the Flink cluster.
If you want to support dynamically loaded classes, then you should serialize these instances into a generic format (e.g. AbstractComputeContainer) which you completely resolve in the user code function where you have the GroovyClassLoader.

ClassLoader.getResources() returns empty enumeration

I have been using the getResources() function of the ClassLoader class to load classes in a certain Plugin I am coding in Minecraft.
The package the main class is found in is: "rr.aesir". I tried every possibility with this package name with slashes before, after, dots, inbetween and everything returned an empty Enumeration. (Spigot version: 1.12.1). tried to call it from other classes aswell.
ClassLoader loader = this.getClass().getClassLoader();
Enumeration<URL> urls = this.loader.getResources("/rr/aesir");
String path = urls.nextElement().getPath();
This code is placed in my main class that extends JavaPlugin. I am getting an error saying NoSuchElementException. I have not called nextElement() anywhere in the class so it's not that I skipped any existing elements.
I am out of options, any help is accepted.
Complete Stack Trace:
[03:07:43 INFO]: [Aesir] Enabling Aesir v1.0
[03:07:43 INFO]: [Aesir] Aesir has been enabled.
[03:07:43 ERROR]: Error occurred while enabling Aesir v1.0 (Is it up to date?)
java.util.NoSuchElementException: null
at sun.misc.CompoundEnumeration.nextElement(Unknown Source) ~[?:1.8.0_151]
at rr.aesir.Aesir.load(Aesir.java:72) ~[?:?]
at rr.aesir.Aesir.onEnable(Aesir.java:47) ~[?:?]
at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264) ~[spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:337) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:402) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.craftbukkit.v1_12_R1.CraftServer.enablePlugin(CraftServer.java:384) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.craftbukkit.v1_12_R1.CraftServer.enablePlugins(CraftServer.java:333) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.craftbukkit.v1_12_R1.CraftServer.reload(CraftServer.java:755) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.Bukkit.reload(Bukkit.java:525) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.command.defaults.ReloadCommand.execute(ReloadCommand.java:25) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.command.SimpleCommandMap.dispatch(SimpleCommandMap.java:141) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.craftbukkit.v1_12_R1.CraftServer.dispatchCommand(CraftServer.java:651) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at org.bukkit.craftbukkit.v1_12_R1.CraftServer.dispatchServerCommand(CraftServer.java:637) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at net.minecraft.server.v1_12_R1.DedicatedServer.aP(DedicatedServer.java:444) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at net.minecraft.server.v1_12_R1.DedicatedServer.D(DedicatedServer.java:407) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at net.minecraft.server.v1_12_R1.MinecraftServer.C(MinecraftServer.java:679) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at net.minecraft.server.v1_12_R1.MinecraftServer.run(MinecraftServer.java:577) [spigot-1.12.1.jar:git-Spigot-da42974-8f47214]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_151]
Picture of class structure:
Class structure inside project.
Remove the leading /.
To make code compile, also remove this. before loader.
ClassLoader loader = this.getClass().getClassLoader();
Enumeration<URL> urls = loader.getResources("rr/aesir");
String path = urls.nextElement().getPath();
Note that getPath() may not return the string you think it does.
url.toString() returns something like file:/C:/foo/bar/rr/aesir
url.getPath() returns /C:/foo/bar/rr/aesir
If you want an actual path, call Paths.get(url.toURI()).toString()
It returns C:\foo\bar\rr\aesir
If you are pre-Java 7, call new File(url.toURI()).toString()
That of course won't work if your code is running from a jar file.
I think that you're a bit misunderstanding what getResources exactly does, assuming that you want to list all the resources within the rr/aesir directory.
According to the javadoc:
Finds all the resources with the given name. A resource is some data (images, audio, text, etc) that can be accessed by class code in a way that is independent of the location of the code.
There can be multiple resources with a specific name, that's why it returns a enumeration. And does not list resources in the directory.
The solution in the following question is what you can use to list the resources:
How do I list the files inside a JAR file?

Loading a jar at runtime causes a NoClassDefFoundError/ClassNotFoundException

Summary: Loading a jar from a running Java program causes a NoClassDefFoundError caused by a ClassNotFoundException caused by inter-class dependencies (e.g. import statements). How can I get around it?
The problem in more detail:
I am attempting to programmatically load a jar file -- let's call it "Server" -- into the Java Virtual Machine through my own Java program -- let's call it "ServerAPI" -- and use extension and some other tricks to modify the behavior of and interact with Server. ServerAPI depends on Server, but if Server is not present, ServerAPI still has to be able to run and download Server from a website.
To avoid errors caused by ServerAPI loading without satisfying its dependencies from Server, I have made a launcher -- let's call it "Launcher" -- that is intended to download Server and set up ServerAPI as necessary, then load Server and ServerAPI, then run ServerAPI.
However, when I attempt to load jars from Launcher, I get errors caused because the ClassLoaders are unable to resolve the other classes in the file that the class it's loading depends on. In short, if I try to load Class A, it will throw an error if A imports B because I haven't loaded B yet. However, if B also imports A, I'm stuck because I can't figure out how to load two classes at once or how to load a class without the JVM running its validation.
Why all the restrictions have led me to this problem:
I am attempting to modify and add to the behavior of Server, but for complicated legal reasons, I cannot modify the program directly, so I have created ServerAPI that depends on and can tweak the behavior of Server from the outside.
However, for more complicated legal reasons, Server and ServerAPI cannot simply be downloaded together. Launcher (see above) has to be downloaded with ServerAPI, then Launcher needs to download Server. Finally, ServerAPI can be run using Server as a dependency. That's why this problem is so complex.
This problem will also apply to a later part of the project, which will involve a plugin-based API interface that needs to be able to load and unload plugins from jar files while running.
Research I have already done on this problem:
I have read through and failed to be helped by:
this question, which only addresses the issue of a single method and does not address inter-class dependency errors;
this question, which will not work because I cannot shut down and restart the program every time a jar is loaded or unloaded (mainly for the plugin part I briefly mentioned);
this question, which only works for situations where the dependencies are present when the program starts;
this question, which has the same problem as #2;
this question, which has the same problem as #3;
this article, from which I learned about the hidden loadClass(String, boolean) method, but trying with true and false values did not help;
this question, which has the same problem as #1;
and more. Nothing has worked.
//EDIT:
Attempts I have made so far:
I have tried using URLClassLoaders to load the jar using the JarEntries from the JarFile similar to this question. I tried this both by using and calling a URLClassLoader's loadClass(String) method and by making a class that extends URLClassLoader so that I could utilize loadClass(String, boolean resolve) to try to force the ClassLoader to resolve all the classes it loads. Both ways, I got this same error:
I couldn't find the class in the JarEntry!
entry name="org/apache/logging/log4j/core/appender/db/jpa/converter/ContextMapAttributeConverter.class"
class name="org.apache.logging.log4j.core.appender.db.jpa.converter.ContextMapAttributeConverter"
java.lang.NoClassDefFoundError: javax/persistence/AttributeConverter
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at Corundum.launcher.CorundumClassLoader.load(CorundumClassLoader.java:52)
at Corundum.launcher.CorundumLauncher.main(CorundumLauncher.java:47)
Caused by: java.lang.ClassNotFoundException: javax.persistence.AttributeConverter
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more
//END EDIT
//EDIT 2:
Here is a sample of the code that I used to load a class while trying to resolve it. This was inside a class that I made that extends URLClassLoader. On the line beginning with Class<?> clazz = loadClass(, I have tried using true and false as the boolean argument; both attempts resulted in the same error above.
public boolean load(ClassLoadAction class_action, FinishLoadAction end_action) {
// establish the jar associated with this ClassLoader as a JarFile
JarFile jar;
try {
jar = new JarFile(jar_path);
} catch (IOException exception) {
System.out.println("There was a problem loading the " + jar_path + "!");
exception.printStackTrace();
return false;
}
// load each class in the JarFile through its JarEntries
Enumeration<JarEntry> entries = jar.entries();
if (entries.hasMoreElements())
for (JarEntry entry = entries.nextElement(); entries.hasMoreElements(); entry = entries.nextElement())
if (!entry.isDirectory() && entry.getName().endsWith(".class"))
try {
/* this "true" in the line below is the whole reason this class is necessary; it makes the URLClassLoader this class extends "resolve" the class,
* meaning it also loads all the classes this class refers to */
Class<?> clazz = loadClass(entry.getName().substring(0, entry.getName().length() - 6).replaceAll("/", "."), true);
class_action.onClassLoad(this, jar, clazz, end_action);
} catch (ClassNotFoundException | NoClassDefFoundError exception) {
try {
close();
} catch (IOException exception2) {
System.out.println("There was a problem closing the URLClassLoader after the following " + exception2.getClass().getSimpleName() + "!");
exception.printStackTrace();
}
try {
jar.close();
} catch (IOException exception2) {
System.out.println("There was a problem closing the JarFile after the following ClassNotFoundException!");
exception.printStackTrace();
}
System.out.println("I couldn't find the class in the JarEntry!\nentry name=\"" + entry.getName() + "\"\nclass name=\""
+ entry.getName().substring(0, entry.getName().length() - 6).replaceAll("/", ".") + "\"");
exception.printStackTrace();
return false;
}
// once all the classes are loaded, close the ClassLoader and run the plugin's main class(es) load() method(s)
try {
jar.close();
} catch (IOException exception) {
System.out.println("I couldn't close the URLClassLoader used to load this jar file!\njar file=\"" + jar.getName() + "\"");
exception.printStackTrace();
return false;
}
end_action.onFinishLoad(this, null, class_action);
System.out.println("loaded " + jar_path);
// TODO TEST
try {
close();
} catch (IOException exception) {
System.out.println("I couldn't close the URLClassLoader used to load this jar file!\njar file=\"" + jar_path + "\"");
exception.printStackTrace();
return false;
}
return true;
}
//END EDIT 2
I realize that there must be a simple solution to this, but for the life of me I cannot seem to find it. Any help would make me eternally grateful. Thank you.
Embarassingly, I found that the answer was that the error message was telling the truth. javax.persistence.AttributeConverter, the class that the loader was claiming was not present, was not in the jar.
I fixed the issue by loading only the main class and the ClassLoader all references classes, essentially loading all the classes in the jar that are used in the program, which is all I need.
Now, I could have sworn that I checked for this before and found that class; I figure I must have actually checked the Apache open source repository for the class rather than the actual Server when I checked that. I can't remember.
In any case, AttributeConverter is missing. I don't know how or why they managed to compile a jar with missing dependencies, but I guess their main processes never use that part of the code, so it never threw errors.
I'm sorry to have wasted everyone's time...including my own. I have been stuck on this problem for a while now.
Moral of this story:
If you're trying to load an executable jar, don't bother loading all the classes in a jar unless you actually have to. Just load the main class; that will load everything the program needs to run.
//EDIT:
I have now started having the same error, but it does not appear until I attempt to call a method from a loaded class. The question is apparently still open. Please downvote and disregard this answer.

java.lang.NoClassDefFoundError: Could not initialize class java.net.ProxySelector

I'm running into a problem with a java app constantly throwing:
java.lang.NoClassDefFoundError: Could not initialize class java.net.ProxySelector.
I am running Suse Linux 10.3 and running java 1.6.0. My CLASSPATH is set to
/usr/lib/jvm/jre-1.6.0-openjdk/lib.
No other users seem to be having this error so I'm assuming its my setup. For those wondering the app is yamj (http://code.google.com/p/moviejukebox/)
Any ideas as to what maybe missing or what I maybe doing wrong?
Edit the full trace of the error is as follows:
java.lang.NoClassDefFoundError: Could not initialize class java.net.ProxySelector
at sun.net.www.protocol.http.HttpURLConnection$5.run(HttpURLConnection.java:736)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:732)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:672)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:997)
at com.moviejukebox.thetvdb.tools.XMLHelper.getEventReader(XMLHelper.java:19)
at com.moviejukebox.thetvdb.model.Mirrors.(Mirrors.java:30)
at com.moviejukebox.thetvdb.TheTVDB.(TheTVDB.java:37)
at com.moviejukebox.plugin.TheTvDBPlugin.(TheTvDBPlugin.java:57)
at sun.reflect.GeneratedConstructorAccessor2.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at java.lang.Class.newInstance0(Class.java:372)
at java.lang.Class.newInstance(Class.java:325)
at com.moviejukebox.plugin.DatabasePluginController.getMovieDatabasePlugin(DatabasePluginController.java:96)
at com.moviejukebox.plugin.DatabasePluginController.access$000(DatabasePluginController.java:30)
at com.moviejukebox.plugin.DatabasePluginController$1.initialValue(DatabasePluginController.java:44)
at com.moviejukebox.plugin.DatabasePluginController$1.initialValue(DatabasePluginController.java:39)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:160)
at java.lang.ThreadLocal.get(ThreadLocal.java:150)
at com.moviejukebox.plugin.DatabasePluginController.scan(DatabasePluginController.java:70)
at com.moviejukebox.MovieJukebox.updateMovieData(MovieJukebox.java:1051)
at com.moviejukebox.MovieJukebox.access$100(MovieJukebox.java:80)
at com.moviejukebox.MovieJukebox$4.call(MovieJukebox.java:613)
at com.moviejukebox.MovieJukebox$4.call(MovieJukebox.java:600)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java
ProxySelector is an abstract class. Are you trying to instantiate it directly?
My CLASSPATH is set to
/usr/lib/jvm/jre-1.6.0-openjdk/lib.
don't think that should be in your CLASSPATH
try clearing the CLASSPATH and running it
Firstly, you shouldn't have to put "/usr/lib/jvm/jre-1.6.0-openjdk/lib" on your classpath. The "java" command should put all of the standard J2SE libraries on the bootclasspath without you doing anything.
Second, it would help if you gave us the full stacktrace, not just the exception message. I suspect that the real problem is that java.net.ProxySelector (or something it depends on) is failing during static initialization. But only a stacktrace would confirm that.
Since it's second result in google search on this error, I want to post this piece of code I found at some forum that helped me with the same exception. Cannot explain in details - it was just a quick test project for me, so I didn't have time for deeper investigation.
static {
try {
Class c = Class.forName("sun.net.spi.DefaultProxySelector");
if (c != null && ProxySelector.class.isAssignableFrom(c)) {
theProxySelector = (ProxySelector) c.newInstance();
}
} catch (Exception e) {
theProxySelector = null;
}
}

Categories