We Keep Coding

java.lang.NoClassDefFoundError: Failed to link org/pentaho/di/core/attributes/metastore/EmbeddedMetaStore

I'm building a batch with Pentaho Kettle to process some files. This batch is started by a servlet. However, when I try to start it, I get the following warn which actually block the process:
WARN [org.jboss.modules] (Batch Thread - 2) Failed to define class org.pentaho.di.core.attributes.metastore.EmbeddedMetaStore in Module "deployment.my-program-1.0.0.0-SNAPSHOT.war:main" from Service Module Loader: java.lang.NoClassDefFoundError: Failed to link org/pentaho/di/core/attributes/metastore/EmbeddedMetaStore (Module "deployment.my-program-1.0.0.0-SNAPSHOT.war:main" from Service Module Loader): org/pentaho/metastore/api/BaseMetaStore
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:446)
at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:274)
at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:78)
at org.jboss.modules.Module.loadModuleClass(Module.java:605)
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:363)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:351)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:93)
at org.pentaho.di.base.AbstractMeta.<init>(AbstractMeta.java:162)
at org.pentaho.di.job.JobMeta.<init>(JobMeta.java:201)
at org.pentaho.di.job.JobMeta.<init>(JobMeta.java:755)
However, when I look in my lib folder, the jar containing this class (kettle-engine-8.2.0.0-342.jar) is there and the class is also in it.
I'm using a JBoss EAP 7.0 with Java 1.8.0.161 on my computer for development. I start it with my IDE (Eclipse Photon). Here are my dependencies in the pom.xml:
<dependencies>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>7.0</version>
</dependency>
<!-- vf2 debug -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-vfs2</artifactId>
<version>2.2</version>
</dependency>
<!-- maven-war-plugin-->
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
</dependency>
<!-- pentaho kettle -->
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
<version>8.2.0.0-342</version>
</dependency>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
<version>8.2.0.0-342</version>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
<version>2.5.16</version>
</dependency>
<dependency>
<groupId>org.mozilla</groupId>
<artifactId>rhino</artifactId>
<version>1.7R3</version>
</dependency>
<!-- this correct an error -->
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.6</version>
</dependency>
<!-- netty version override to correct netty 3.7 bug / FYI, pentaho don't use netty anyway-->
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
<version>3.9.0.Final</version>
</dependency>
</dependencies>
Important : this issue does not occur when using previous version of kettle-core and kettle-engine (tested with 7.0).
It's an embedded call so I do not directly call this class in my code.
I do not have multiple versions of this jar in my classpath.

Another person in my company found out so i'll just answer here to close the topic.
I didn't show it here but in my dependencies I had a dependency to some custom framework from work. This framework actually included pentaho, but the 7.0 version. Then, #suresh was right when he talk about multiple version jar.
So the answer is :
Check if any dependency actually contains duplicate dependency you're calling somewhere else. To avoid this issue, use the excluding syntax in the dependency calling the wrong versions :
<exclusions>
<exclusion>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
</exclusion>
<exclusion>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
</exclusion>
</exclusions>

mssql-jdbc version 6.2.2.jre8 dependency results in version 6.1.0.jre7 in war

I have the following dependency in my Maven project
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.2.2.jre8</version>
<!-- Exclude unused Azure dependencies -->
<exclusions>
<exclusion>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-keyvault</artifactId>
</exclusion>
<exclusion>
<groupId>com.microsoft.azure</groupId>
<artifactId>adal4j</artifactId>
</exclusion>
</exclusions>
</dependency>
I am using the newest non-preview mssql-jdbc release in Maven here
https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc
However when I run mvn clean package my resulting war file has mssql-jdbc-6.1.0.jre7.jar in WEB-INF/lib instead of the expected mssql-jdbc-6.2.2.jre8.jar. I have tried clearing my local .m2 repository and repackaging but that did not help. It is worth noting that 6.1.0.jre7 is actually the oldest version out there, so my guess is that for some reason it cannot find the newest one and is reverting to the oldest? I'm stuck.

I encountered same problem, and I am using Spring Boot 1.5.x.
The root cause is: Spring Boot(1.5.x) specified the version of "6.1.0.jre7".
Solution is: specify your version in the main module which contains Spring boot Application.
Previous part of my dependencies:
Sub module 1:
<groupId>com.my.company</groupId>
<artifactId>sub-module1</artifactId>
<version>1.0</version>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>7.1.3.jre8-preview</version>
</dependency>
Main module, which contains Spring boot Application:
<dependency>
<groupId>com.my.company</groupId>
<artifactId>sub-module1</artifactId>
<version>1.0</version>
</dependency>
And I found Main module boot.jar included "mssql-jdbc-6.1.0.jre7.jar".
Finally I found the version is specified in "spring-boot-dependencies" which is the parent of "spring-boot-parent".
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>1.5.16.RELEASE</version>
<mssql-jdbc.version>6.1.0.jre7</mssql-jdbc.version>
Solution:
Specify the version in main module:
<dependency>
<groupId>com.my.company</groupId>
<artifactId>sub-module1</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>7.1.3.jre8-preview</version>
</dependency>
Checked the result by mvn denpendency:tree, it's ok

java.lang.NoClassDefFoundError: io/netty/handler/ssl/SslContextBuilder

I am trying to implement relayrides/pushy, but am getting the following runtime error:
Jun 28, 2017 2:06:58 PM com.turo.pushy.apns.SslUtil getSslProvider
INFO: Native SSL provider not available; will use JDK SSL provider.
Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/handler/ssl/SslContextBuilder
at com.turo.pushy.apns.ApnsClientBuilder.build(ApnsClientBuilder.java:396)
at com.jobs.spring.service.NotificationServiceImpl.sendIOSPushNotification(NotificationServiceImpl.java:122)
Caused by: java.lang.ClassNotFoundException: io.netty.handler.ssl.SslContextBuilder
pom.xml
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<!-- Push Notifications -->
<dependency>
<groupId>com.turo</groupId>
<artifactId>pushy</artifactId>
<version>0.10</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
<version>2.0.5.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
<version>4.0.27.Final</version>
</dependency>
<dependency>
<groupId>com.ning</groupId>
<artifactId>async-http-client</artifactId>
<version>1.9.40</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.1</version>
</dependency>
java
final ApnsClient apnsClient = new ApnsClientBuilder()
.setClientCredentials(new File(PATH_TO_P12_CERT), CERT_PASSWORD )
.build();
I am guessing my mvn dependencies are incorrect. Any help appreciated.
UPDATE
I updated my dependencies to:
<dependency>
<groupId>com.turo</groupId>
<artifactId>pushy</artifactId>
<version>0.10</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.11.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
<version>2.0.1.Final</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.6</version>
</dependency>
But now get:
Jun 28, 2017 2:40:18 PM com.turo.pushy.apns.SslUtil getSslProvider
INFO: Native SSL provider not available; will use JDK SSL provider.
Exception in thread "main" java.lang.NoSuchMethodError: io.netty.bootstrap.Bootstrap.config()Lio/netty/bootstrap/BootstrapConfig;
at com.turo.pushy.apns.ApnsClient.<init>(ApnsClient.java:172)
at com.turo.pushy.apns.ApnsClientBuilder.build(ApnsClientBuilder.java:420)
at com.jobs.spring.service.NotificationServiceImpl.sendIOSPushNotification(NotificationServiceImpl.java:121)

seems like you mix netty 4.1 and 4.0. You need to only use 4.1 if you want to use pushy.

Did you clean your classpath in between I created a test project containing only the dependency
<dependency>
<groupId>com.turo</groupId>
<artifactId>pushy</artifactId>
<version>0.10</version>
</dependency>
and a simple main class just creating the client with the snipped you are using above and it works out fine
To me it looks like there might be two versions of BootstrapConfig on the classpath. Try to remove all dependencies besides pushy and clean/refreshing the maven dependencies.

Hadoop 2.6.x and Amazon AWS SDK Library conflicts http-core conflict

My job writes each record to DynamoDB in the Hadoop's map.
I cannot make it run with Hadoop 2.6 which has httpclient-4.2.5.jar and httpcore-4.2.5.jar.
AWS which I am using was built using httpclient-4.5.2.jar and httpcore-4.4.4.jar.
When I am using classpath to include the new jar files, it gives the following exception.
java.lang.Exception: java.lang.NoSuchFieldError: INSTANCE
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NoSuchFieldError: INSTANCE
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:144)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:87)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:49)
To me, it looks like Hadoop was built using the old libraries and something has changed in the syntax.
What's the reasonable solution than recompiling older sources of AWS?

As an update, I had to switch to Maven and play around with the versions a bit.
<!-- http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0</version>
</dependency>
<!-- http://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.4</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.netpreserve.commons</groupId>
<artifactId>webarchive-commons</artifactId>
<version>1.1.4</version>
</dependency>
Finally, it works

Maven cannot load dependency from another dependency in repository

I am new to Maven, I try to load tranquility using the following in pom.xml:
<dependency>
<groupId>io.druid</groupId>
<artifactId>tranquility-core_2.11</artifactId>
<version>0.7.0</version>
</dependency>
Then I get an error: Failure to find com.fasterxml.jackson-module-scala_2.11:jar:2.4.6.
I tried to search in search.maven.org, and find out that, inside the io.druid tranquility-core module, the dependency is:
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.4.6</version>
</dependency>
However, when I try to search Jackson-module-scala_2.11 in the central repository, there's no 2.4.6 version there, there's only 2.4.5 and 2.5.0. See the following link: http://search.maven.org/#search|gav|1|g%3A%22com.fasterxml.jackson.module%22%20AND%20a%3A%22jackson-module-scala_2.11%22
So is there a way for me to build it successfully even the module in central repository (in this case the tranquility module) makes mistake in referencing another module?
Thanks.

you can exclude the dependency in the io.druid tranquility, and dependency it by yourself with the exist version like this:
<dependency>
<groupId>io.druid</groupId>
<artifactId>tranquility-core_2.11</artifactId>
<version>0.7.0</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.5.0</version>
</dependency>