We Keep Coding

ClassNotFoundException and NoClassDefFoundError in Flink app using Cassandra Driver

I am developing a Flink application and it will use Cassandra Driver to interact with Cassandra DB. The Cassandra Driver is implemented in Singleton fashion and multiple Flink process functions will interact with it to get data from Cassandra. I also add a future callback to each Session.executeAsync's ResultSetFuture. The app is run on Kubernetes through Docker containers.
The environment is:
Flink version is 1.10.0 and using shaded netty, hadoop, guava and jackson.
Using cassandra-driver-mapping: 3.9.0 and shaded cassandra-driver-core: 3.9.0.
All dependencies are packaged in a single jar using Bazel. Before starting the Flink app, I check all the required classes are in the jar and are correct and complete. And I use the shaded dependency in order to avoid class loading conflict in JVM. But, when I start and run the Flink app. I keep seeing the following ClassNotFoundException in the Taskmanager logs.
java.lang.NoClassDefFoundError: com/datastax/driver/core/SessionManager$State
at com.datastax.driver.core.SessionManager.getState(SessionManager.java:211)
at io.uhana.cassandra.CassandraDriver.sessionNeedsReconnect(CassandraDriver.java:508)
at io.uhana.cassandra.CassandraDriver.access$000(CassandraDriver.java:61)
at io.uhana.cassandra.CassandraDriver$1.onFailure(CassandraDriver.java:518)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1387)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1015)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:868)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:713)
at com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:230)
at com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:235)
at com.datastax.driver.core.RequestHandler.access$2600(RequestHandler.java:61)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:1011)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:647)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180)
at com.datastax.shaded.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.datastax.shaded.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.datastax.shaded.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.datastax.shaded.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at com.datastax.shaded.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.datastax.shaded.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.datastax.shaded.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.datastax.shaded.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921)
at com.datastax.shaded.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:135)
at com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
at com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:546)
at com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:500)
at com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
at com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at com.datastax.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassNotFoundException: com.datastax.driver.core.SessionManager$State
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:69)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 49 more
and
ConstantReconnectionPolicy$ConstantSchedule' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:
java.lang.NoClassDefFoundError: com/datastax/shaded/netty/handler/timeout/IdleState
at com.datastax.shaded.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:493)
at com.datastax.shaded.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466)
at com.datastax.shaded.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at com.datastax.shaded.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
at com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
at com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
at com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at com.datastax.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassNotFoundException: com.datastax.shaded.netty.handler.timeout.IdleState
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:69)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 9 more
I also notice that these issues are easier to reproduce when giving more resources and parallelism to the Flink app and the process functions. And the issues are most likely happen in the future callback.
Any help appreciated!

java.lang.NoClassDefFoundError: org/yaml/snakeyaml/constructor/BaseConstructor

When I am trying to run flutter app in simulator i get error:
java.lang.NoClassDefFoundError: org/yaml/snakeyaml/constructor/BaseConstructor
at org.testng.internal.YamlParser.parse(YamlParser.java:16)
at org.testng.internal.YamlParser.parse(YamlParser.java:10)
at org.testng.xml.Parser.parse(Parser.java:168)
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:311)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:46)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
Caused by: java.lang.ClassNotFoundException: org.yaml.snakeyaml.constructor.BaseConstructor
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 6 more
Process finished with exit code -1

You are missing the JAR file for snakeyaml from the classpath. However, judging from the stacktrace, you are not running a flutter app at this point. It looks like that the exception is occurring in the testng launcher.
So I suspect that the problem is a version conflict between the versions of snakeyaml required by the test framework and something you are testing. Here is an example Q&A from a different context:
java.lang.NoClassDefFoundError: org/yaml
The work-around in that case was to exclude nestng's transitive snakeyaml dependency; see above link for the details. (You will most likely need to map from the Maven POM syntax to the equivalent Gradle syntax.)

Why Java doesn't recognize net.sf.jasperreports.engine.util.JRLoader class?

Good afternoon.
I am working on JDK 1.6.
I am trying to integrate JasperReports to the Java Web Application of company where I work. First, I created an simple Java Application for to test the reports generation, and I used the next libraries (jars):
Commons collections 3.2.1
Commons beanutils 1.9.0
Commons digester 2.1
Commons logging 1.2
Groovy all 1.8.0
Joda time 2.0
JasperReports 6.0.0
With these I was able to create a report but, when I transfered this libraries to the Java Web Application,happened the next exception:
javax.faces.FacesException: #{bean.functionCalled}: java.lang.reflect.InvocationTargetException
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:118)
at org.primefaces.application.CleanupActionListener.processAction(CleanupActionListener.java:42)
...
Caused by: javax.faces.el.EvaluationException: java.lang.reflect.InvocationTargetException
at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:102)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
... 51 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
... 52 more
Caused by: java.lang.NoClassDefFoundError: net/sf/jasperreports/engine/util/JRLoader
... 85 more
Caused by: java.lang.ClassNotFoundException: net.sf.jasperreports.engine.util.JRLoader
at java.net.URLClassLoader$1.run(Unknown Source)
...
... 86 more
I found out that the cause of the problem is the commons loggin version, or the libraries combination.
By default, this Java Web Application has the next libraries:
Commons collections 3.2
Commons beanutils 1.6
Commons digester 1.8.1
Commons logging 1.1.1
Groovy all 1.5.4
Joda time 1.6
I'm making differents versions combinations but not working, the problem persist.
What you recommend me?
Thank you so much.

This is a missing jars in the class path issue.
I had this issue and found that with by adding the missing jars to the class path it resolved the issue.
From the below article java.dzone.com article here I found that the minimal jars necessary are...Now The versions numbers have changed but with these libraries only I managed to get it working.
jasperreports-3.5.3.jar and above version
commons-digester-2.1.jar
commons-collections-3.2.jar (commons-collections.jar)
commons-logging-1.2.jar
commons-beanutils.jar
iText-5.0.jar (used infor PDF exporting)

Cannot run the code due to java.lang.NoClassDefFoundError

I am investigating the jsprit library. For this I just created a new project in Eclipse and copied-pasted the demo example class. Then I added all jars to the path, including log4j-1.2.17.jar. Nevrtheless I cannot execute the demo code due to the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
at jsprit.core.problem.vehicle.VehicleImpl$Builder.<clinit>(VehicleImpl.java:108)
at com.test.jsprit.main(jsprit.java:49)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.LogManager
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
What does it actually mean and how to solve this issue? Does it mean that some other *.jar file is missed?
UPDATE:
The error occurs at the following line:
Builder vehicleBuilder = VehicleImpl.Builder.newInstance("vehicle");

You are using the wrong version of the log4j lib. You need the 2.x version.
See:
LogManager (Version 1.x)
LogManager (Version 2.x)

log4j-1.2.17.jar implements log4j version 1 and its pretty old. What you need is implementation of log4j 2 - add this as the dependency: link

How to connect Java with Titan with embedded Cassandra

I installed titan-cassandra-0.4.4, and I can run cassandra and gremlin perfectly (The local node).
I want to test an algorithm in Java, so I need to connect Java to titan.
My pom is:
<dependency>
<groupId>com.thinkaurelius.titan</groupId>
<artifactId>titan-core</artifactId>
<version>0.4.4</version>
</dependency>
My codes are:
BaseConfiguration baseConfiguration = new BaseConfiguration();
baseConfiguration.setProperty("storage.backend", "cassandra");
baseConfiguration.setProperty("storage.hostname", "127.0.0.1");
baseConfiguration.setProperty("storage.tablename","test");
TitanGraph titanGraph = TitanFactory.open(baseConfiguration);
In the last line, the error is:
Exception in thread "main" java.lang.NoClassDefFoundError: com/netflix/astyanax/connectionpool/exceptions/ConnectionException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:202)
at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:97)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:398)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:78)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:60)
at Connecting.main(Connecting.java:20)
Caused by: java.lang.ClassNotFoundException: com.netflix.astyanax.connectionpool.exceptions.ConnectionException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
I first start the cassandra, then titan gremlin.sh, and run the java code at last. I do not know if I was missing something?
Thanks in advance!

You should include a dependency for titan-casssandra.
<dependency>
<groupId>com.thinkaurelius.titan</groupId>
<artifactId>titan-casssandra</artifactId>
<version>0.4.4</version>
</dependency>
Also note if you're just getting started with Titan, version 0.4.4 is far out of date. Titan 1.0 was released in September 2015. You can find more information at http://titandb.io