I have a java spark process which I call using -
spark-submit --class MyClass target/MyJar.jar
The last part of this process is to write it locally and then copy it to s3 as it need to have specific name (can also be written to s3 and mv'ed there, for the sake of the question the error stays the same).
The code compiles and runs but when it get to the code snippet below I get the following error -
java.lang.NoClassDefFoundError: com/amazonaws/services/s3/AmazonS3 at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:348) at
org.apache.spark.util.Utils$.classForName(Utils.scala:229) at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:700)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused
by: java.lang.ClassNotFoundException:
com.amazonaws.services.s3.AmazonS3 at
java.net.URLClassLoader.findClass(URLClassLoader.java:381) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
public static void saveToS3(Dataset<Row> df, String outputBucket, String outputPath) throws IOException {
String tmpFile = "temp" + Long.toString(System.nanoTime());
df.coalesce(1).write().option("header", true).csv(tmpFile);
File directory = new File(tmpFile);
AmazonS3 s3client = new AmazonS3Client();
for (File file : directory.listFiles()) {
if (file.getName().startsWith("part_") || file.getName().endsWith("csv")) {
s3client.putObject(new PutObjectRequest(outputBucket, getS3path(outputPath), file));
}
file.delete();
}
directory.delete();
}
I use the following dependencies which might be relevant
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.7.4</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>2.7.2</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api-2.5</artifactId>
</exclusion>
</exclusions>
</dependency>
Any idea?
Dependency tree -
[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile
[INFO] | | \- com.esotericsoftware:kryo-shaded:jar:3.0.3:compile
[INFO] | | +- com.esotericsoftware:minlog:jar:1.3.0:compile
[INFO] | | \- org.objenesis:objenesis:jar:2.1:compile
[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile
[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile
[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0:compile
[INFO] | | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0:compile
[INFO] | | | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.2.0:compile
[INFO] | | | | | \- com.google.inject:guice:jar:3.0:compile
[INFO] | | | | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | | | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0:compile
[INFO] | | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0:compile
[INFO] | | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0:compile
[INFO] | | \- org.apache.hadoop:hadoop-annotations:jar:2.2.0:compile
[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile
[INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
[INFO] | | \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile
[INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:compile
[INFO] | | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- com.google.guava:guava:jar:14.0.1:compile
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.4.1:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.16:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile
[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile
[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile
[INFO] | +- commons-net:commons-net:jar:2.2:compile
[INFO] | +- org.scala-lang:scala-library:jar:2.11.8:compile
[INFO] | +- org.json4s:json4s-jackson_2.11:jar:3.2.11:compile
[INFO] | | \- org.json4s:json4s-core_2.11:jar:3.2.11:compile
[INFO] | | +- org.json4s:json4s-ast_2.11:jar:3.2.11:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | | \- org.scala-lang:scalap:jar:2.11.0:compile
[INFO] | | \- org.scala-lang:scala-compiler:jar:2.11.0:compile
[INFO] | | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.1:compile
[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile
[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile
[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile
[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile
[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile
[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile
[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile
[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile
[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile
[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile
[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile
[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile
[INFO] | +- io.netty:netty-all:jar:4.0.42.Final:compile
[INFO] | +- io.netty:netty:jar:3.8.0.Final:compile
[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile
[INFO] | | +- org.scala-lang:scala-reflect:jar:2.11.7:compile
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile
[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile
[INFO] | +- oro:oro:jar:2.0.8:compile
[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile
[INFO] | +- net.sf.py4j:py4j:jar:0.10.4:compile
[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile
[INFO] | | \- org.scalatest:scalatest_2.11:jar:2.2.6:compile
[INFO] | | \- org.scala-lang.modules:scala-xml_2.11:jar:1.0.2:compile
[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile
[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile
[INFO] +- org.apache.hadoop:hadoop-aws:jar:2.7.2:compile
[INFO] | \- org.apache.hadoop:hadoop-common:jar:2.7.2:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.7.2:compile
[INFO] | | \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile
[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.spark:spark-sql_2.10:jar:2.1.0:compile
[INFO] | +- com.univocity:univocity-parsers:jar:2.2.1:compile
[INFO] | +- org.apache.spark:spark-sketch_2.10:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-core_2.10:jar:2.1.0:compile
[INFO] | | +- com.twitter:chill_2.10:jar:0.8.0:compile
[INFO] | | +- org.apache.spark:spark-launcher_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-network-common_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-network-shuffle_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-unsafe_2.10:jar:2.1.0:compile
[INFO] | | +- org.json4s:json4s-jackson_2.10:jar:3.2.11:compile
[INFO] | | | \- org.json4s:json4s-core_2.10:jar:3.2.11:compile
[INFO] | | | \- org.json4s:json4s-ast_2.10:jar:3.2.11:compile
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.6.5:compile
[INFO] | +- org.apache.spark:spark-catalyst_2.10:jar:2.1.0:compile
[INFO] | | +- org.codehaus.janino:janino:jar:3.0.0:compile
[INFO] | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile
[INFO] | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile
[INFO] | +- org.apache.spark:spark-tags_2.10:jar:2.1.0:compile
[INFO] | | \- org.scalatest:scalatest_2.10:jar:2.2.6:compile
[INFO] | +- org.apache.parquet:parquet-column:jar:1.8.1:compile
[INFO] | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile
[INFO] | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile
[INFO] | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile
[INFO] | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile
[INFO] | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile
[INFO] +- com.amazonaws:aws-java-sdk:jar:1.7.4:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2:compile
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.3:compile
[INFO] | \- joda-time:joda-time:jar:2.9.9:compile (version selected from constraint [2.2,))
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.7.3:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.3:compile
[INFO] \- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.3:compile
You appear to be using a really old version of AWS SDK (1.7.4) from 2014. Your apache spark is 2.1, which is from 2016.
Commit history for the AmazonS3Client only goes back to 1.9, so it very well may not exist in your AWS SDK.
The reason your code compiles is that your code itself isn't the problem, but one of your dependencies is trying to use a more recent version of S3 client that your old SDK doesn't have. In other words, this isn't a code problem, it's a dependency management problem. If you're not directly using the SDK at all, it's safe to use a more appropriate AWS SDK 1.11.X, if you are, you'll have to coax your code to that point.
Typically, the spark-core module would point out its dependency to a version of aws, but I assume it was specifically excluded from its own so you could provide your own version of the SDK for your use without any sort of conflict, with the caveat that if you use too old of a dependency, it just breaks.
After massaging my pom file I was able to get to a stable with the following pom file and packaging with -
mvn clean compile assembly:single;
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.group</groupId>
<artifactId>arifact</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<jdk.version>1.8</jdk.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>2.8.2</version>
</dependency>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.11.229</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.group.artifact.MainClass</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
</project>
Related
My code seems to be breaking on a transitive dependency issue that I cannot find. I am trying to migrate from JBoss EAP 7.3 with RedHat's PAM 7.9 product (which is JBPM 7.48) to JBoss EAP 7.4 with PAM 7.12 (ie JBPM 7.59) and am experiencing the issue in a servlet of mine that calls the kie-server via a kie-client API. I have read about and encountered RESTEasy issues before with JBoss, so based upon my error have removed the tags to hopefully avoid any inconsistencies - but still have an odd error.
This is the stack trace that I get which has been edited for brevity:
5:37:57,975 ERROR [io.undertow.request] (default task-4) UT005023: Exception handling request to /PamHelper/pam/helper/pamQuery: org.jboss.resteasy.spi.UnhandledException: java.lang.NoSuchFieldError: _typeCache
at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:82)
at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:346)
at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:193)
...
at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
...
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
...
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchFieldError: _typeCache
at org.kie.server.api.marshalling.json.FallbackableTypeFactory.withFallbackClassLoader(FallbackableTypeFactory.java:62)
at org.kie.server.api.marshalling.json.JSONMarshaller.configureMarshaller(JSONMarshaller.java:284)
at org.kie.server.api.marshalling.json.JSONMarshaller.<init>(JSONMarshaller.java:180)
at org.kie.server.api.marshalling.BaseMarshallerBuilder.build(BaseMarshallerBuilder.java:45)
at org.kie.server.api.marshalling.MarshallerFactory.getMarshaller(MarshallerFactory.java:50)
at org.kie.server.client.impl.AbstractKieServicesClientImpl.<init>(AbstractKieServicesClientImpl.java:84)
at org.kie.server.client.impl.KieServicesClientImpl.<init>(KieServicesClientImpl.java:74)
at org.kie.server.client.KieServicesFactory.newKieServicesClient(KieServicesFactory.java:101)
This is my dependency tree. I have guessing there is a conflict either in a RestEasy dependency or (from the error message) some odd problem initializing marshalling; there is no typeCache field either in my code or the JBPM source code.
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) # PamServer ---
[INFO] com.gok.ams:PamServer:war:1.0-SNAPSHOT
[INFO] +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.3:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.3:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-core:jar:2.12.3:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.3:compile
[INFO] +- jakarta.persistence:jakarta.persistence-api:jar:2.2.3:compile
[INFO] +- com.gok.ams:ams-pam-util:jar:0.0.12-SNAPSHOT:compile
[INFO] | +- org.hibernate:hibernate-core:jar:5.5.8.Final-redhat-00001:compile
[INFO] | | +- javax.persistence:javax.persistence-api:jar:2.2:compile
[INFO] | | +- org.javassist:javassist:jar:3.27.0.GA-redhat-00001:compile
[INFO] | | +- net.bytebuddy:byte-buddy:jar:1.11.12.redhat-00002:compile
[INFO] | | +- antlr:antlr:jar:2.7.7.redhat-7:compile
[INFO] | | +- org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:jar:1.1.1.Final:compile
[INFO] | | +- org.jboss:jandex:jar:2.2.3.Final-redhat-00001:compile
[INFO] | | +- com.fasterxml:classmate:jar:1.5.1.redhat-00001:compile
[INFO] | | +- javax.activation:javax.activation-api:jar:1.2.0:compile
[INFO] | | +- org.hibernate.common:hibernate-commons-annotations:jar:5.1.2.Final-redhat-00003:compile
[INFO] | | +- javax.xml.bind:jaxb-api:jar:2.3.1.redhat-00001:compile
[INFO] | | \- org.glassfish.jaxb:jaxb-runtime:jar:2.3.1.redhat-00002:compile
[INFO] | | +- org.glassfish.jaxb:txw2:jar:2.3.1.redhat-00002:compile
[INFO] | | +- com.sun.istack:istack-commons-runtime:jar:3.0.7.redhat-00001:compile
[INFO] | | +- org.jvnet.staxex:stax-ex:jar:1.8:compile
[INFO] | | \- com.sun.xml.fastinfoset:FastInfoset:jar:1.2.15:compile
[INFO] | \- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] +- com.gok.ams:ams-pam-workitemhandlers:jar:2.0.6.41-SNAPSHOT:compile
[INFO] | \- com.gok.ams:amsObjectModel:jar:0.0.65-SNAPSHOT:compile
[INFO] +- org.jbpm:jbpm-workitems-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- org.drools:drools-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.kie.soup:kie-soup-xstream:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-core-reflective:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.kie:kie-memory-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | \- org.drools:drools-core-dynamic:jar:7.59.0.Final-redhat-00006:runtime
[INFO] | +- org.jbpm:jbpm-flow:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-mvel:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-serialization-protobuf:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.kie.soup:kie-soup-commons:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.kie.soup:kie-soup-project-datamodel-commons:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.kie.soup:kie-soup-project-datamodel-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.kie:kie-dmn-feel:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.antlr:antlr4-runtime:jar:4.8.0.redhat-00002:compile
[INFO] | | | +- com.github.javaparser:javaparser-core:jar:3.13.10:compile
[INFO] | | | +- org.drools:drools-mvel-parser:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- ch.obermuhlner:big-math:jar:2.0.1:compile
[INFO] | | +- org.kie:kie-dmn-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-dmn-backend:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.drools:drools-ruleunit:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.drools:drools-alphanetwork-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.jpmml:pmml-model:jar:1.5.1:compile
[INFO] | | | \- org.jpmml:pmml-agent:jar:1.5.1:compile
[INFO] | | +- com.google.protobuf:protobuf-java:jar:3.6.1:compile
[INFO] | | \- org.mvel:mvel2:jar:2.4.12.Final-redhat-00001:compile
[INFO] | +- org.jbpm:jbpm-bpmn2:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-ecj:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | \- org.jbpm:jbpm-flow-builder:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- org.kie:kie-internal:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- org.kie:kie-dmn-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | \- org.kie:kie-dmn-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- commons-io:commons-io:jar:2.7.0.redhat-00003:compile
[INFO] | +- org.apache.commons:commons-exec:jar:1.3:compile
[INFO] | \- org.antlr:ST4:jar:4.0.7:compile
[INFO] | \- org.antlr:antlr-runtime:jar:3.5:compile
[INFO] | \- org.antlr:stringtemplate:jar:3.2.1:compile
[INFO] +- org.jbpm:jbpm-services-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] +- org.kie:kie-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | \- org.kie.soup:kie-soup-maven-support:jar:7.59.0.Final-redhat-00006:compile
[INFO] +- org.kie.server:kie-server-client:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- org.optaplanner:optaplanner-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-canonical-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.drools:drools-model-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.drools:drools-mvel-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.apache.commons:commons-math3:jar:3.4.1.redhat-3:compile
[INFO] | | +- org.reflections:reflections:jar:0.9.11.redhat-00004:compile
[INFO] | | | \- com.google.guava:guava:jar:25.0.0.redhat-1:compile
[INFO] | | | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | | | +- org.checkerframework:checker-compat-qual:jar:2.0.0:compile
[INFO] | | | +- com.google.errorprone:error_prone_annotations:jar:2.1.3:compile
[INFO] | | | +- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] | | | \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.14:compile
[INFO] | | \- com.thoughtworks.xstream:xstream:jar:1.4.18.redhat-00001:compile
[INFO] | | \- io.github.x-stream:mxparser:jar:1.2.2:compile
[INFO] | | \- xmlpull:xmlpull:jar:1.1.3.1:compile
[INFO] | +- org.kie.server:kie-server-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.optaplanner:optaplanner-persistence-xstream:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.optaplanner:optaplanner-persistence-common:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- org.optaplanner:optaplanner-persistence-jaxb:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.12.1.redhat-00002:compile
[INFO] | | +- org.kie:kie-pmml-dependencies:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-commons:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | | \- org.apache.commons:commons-text:jar:1.9:compile
[INFO] | | | +- org.kie:kie-pmml-compiler-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-compiler-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-compiler-commons:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-evaluator-api:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-evaluator-core:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-evaluator-assembler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-kie-internal:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-common:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-regression-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-regression-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-regression-evaluator:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-tree-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-tree-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-tree-evaluator:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | | \- org.kie:kie-pmml-compiler-api:jar:tests:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-scorecard-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-scorecard-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-drools-scorecard-evaluator:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-mining-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-mining-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-mining-evaluator:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-clustering-model:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | +- org.kie:kie-pmml-models-clustering-compiler:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | | \- org.kie:kie-pmml-models-clustering-evaluator:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | \- org.drools:kie-pmml:jar:7.59.0.Final-redhat-00006:compile
[INFO] | +- jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.6.redhat-00001:compile
[INFO] | +- org.kie.server:kie-server-common:jar:7.59.0.Final-redhat-00006:compile
[INFO] | | \- jakarta.servlet:jakarta.servlet-api:jar:4.0.3.redhat-00002:compile
[INFO] | +- jakarta.jms:jakarta.jms-api:jar:2.0.3.redhat-00001:compile
[INFO] | +- com.sun.xml.bind:jaxb-core:jar:2.3.0.redhat-1:compile
[INFO] | +- com.sun.xml.bind:jaxb-impl:jar:2.3.0.redhat-1:compile
[INFO] | \- org.slf4j:jcl-over-slf4j:jar:1.7.30.redhat-00001:runtime
[INFO] +- org.slf4j:slf4j-api:jar:1.7.26:compile
[INFO] +- org.jboss.resteasy:resteasy-jaxrs:jar:3.0.14.Final:compile
[INFO] | +- org.jboss.spec.javax.ws.rs:jboss-jaxrs-api_2.0_spec:jar:1.0.0.Final:compile
[INFO] | +- org.jboss.spec.javax.annotation:jboss-annotations-api_1.2_spec:jar:1.0.0.Final:compile
[INFO] | +- javax.activation:activation:jar:1.1.1:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.3.6:compile
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.3.3:compile
[INFO] | | \- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | +- net.jcip:jcip-annotations:jar:1.0:compile
[INFO] | \- org.jboss.logging:jboss-logging:jar:3.1.4.GA:compile
[INFO] +- org.jboss.resteasy:resteasy-servlet-initializer:jar:6.0.0.Final:compile
[INFO] | +- org.jboss.resteasy:resteasy-core:jar:6.0.0.Final:compile
[INFO] | | +- jakarta.annotation:jakarta.annotation-api:jar:2.0.0:compile
[INFO] | | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:3.0.1:compile
[INFO] | | +- jakarta.activation:jakarta.activation-api:jar:2.0.0:compile
[INFO] | | +- jakarta.validation:jakarta.validation-api:jar:3.0.0:compile
[INFO] | | \- com.ibm.async:asyncutil:jar:0.1.0:compile
[INFO] | \- org.jboss.spec.javax.ws.rs:jboss-jaxrs-api_3.0_spec:jar:1.0.0.Final:compile
[INFO] +- org.jboss.resteasy:resteasy-client:jar:6.0.0.Final:compile
[INFO] | +- org.jboss.resteasy:resteasy-client-api:jar:6.0.0.Final:compile
[INFO] | +- org.jboss.resteasy:resteasy-core-spi:jar:6.0.0.Final:compile
[INFO] | +- commons-codec:commons-codec:jar:1.15:compile
[INFO] | \- org.reactivestreams:reactive-streams:jar:1.0.3:compile
[INFO] +- junit:junit:jar:4.13.2:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] \- org.mockito:mockito-all:jar:1.10.16:test
And this is the dependency part of my pom.xml not related to testing.
<properties>
<version.org.kie>7.59.0.Final-redhat-00006</version.org.kie>
<!--version.org.kie>7.48.0.Final-redhat-00004</version.org.kie-->
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<version.org.slf4j>1.7.26</version.org.slf4j>
<jax-rs-version>3.0.14.Final</jax-rs-version>
<version.object.model>0.0.65-SNAPSHOT</version.object.model>
<version.utils>0.0.12-SNAPSHOT</version.utils>
<version.workItemHandlers>2.0.6.41-SNAPSHOT</version.workItemHandlers>
<version.jakarta.persistence-api>2.2.3</version.jakarta.persistence-api>
<version.resteasy>6.0.0.Final</version.resteasy>
<!--version.jackson>2.12.4.redhat-00001</version.jackson-->
<version.jackson>2.12.3</version.jackson>
</properties>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${version.jackson}</version>
<!--scope>provided</scope-->
</dependency>
<dependency>
<groupId>jakarta.persistence</groupId>
<artifactId>jakarta.persistence-api</artifactId>
<version>${version.jakarta.persistence-api}</version>
</dependency>
<dependency>
<groupId>com.gok.ams</groupId>
<artifactId>ams-pam-util</artifactId>
<version>${version.utils}</version>
</dependency>
<dependency>
<artifactId>ams-pam-workitemhandlers</artifactId>
<version>${version.workItemHandlers}</version>
<groupId>com.gok.ams</groupId>
</dependency>
<dependency>
<groupId>org.jbpm</groupId>
<artifactId>jbpm-workitems-core</artifactId>
<version>${version.org.kie}</version>
</dependency>
<dependency>
<groupId>org.jbpm</groupId>
<artifactId>jbpm-services-api</artifactId>
<version>${version.org.kie}</version>
</dependency>
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>${version.org.kie}</version>
</dependency>
<dependency>
<groupId>org.kie.server</groupId>
<artifactId>kie-server-client</artifactId>
<version>${version.org.kie}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${version.org.slf4j}</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jaxrs</artifactId>
<version>3.0.14.Final</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-servlet-initializer</artifactId>
<version>${version.resteasy}</version>
<exclusions>
<exclusion>
<artifactId>resteasy-jaxrs</artifactId>
<groupId>org.jboss.resteasy</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-client</artifactId>
<version>${version.resteasy}</version>
</dependency>
The other possibility that I can think of is that this is related to this issue which may not have been fixed in PAM ; I have downgraded from Java 11 to Java 8 to have my the code running on my laptop look more like my company's integration environment.
You've got a dependency on two different versions of RESTEasy. You don't want to use 6.0.0.Final as that is Jakarta REST 3.0 which is Jakarta EE 9.1. You want to use the latest 3.15.x version as that is what JBoss EAP 7.4 uses. You also want to make sure you mark those dependencies as <scope>provided</scope> as they are provided by JBoss EAP 7.4.
I don't think you need the org.jboss.resteasy:resteasy-servlet-initializer at either.
A Maven Java project using geotools 15.2 generates an error : NoSuchFieldError: METER
pom.xml
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-data</artifactId>
<version>15.2</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-shapefile</artifactId>
<version>15.2</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>15.2</version>
</dependency>
When running the code, the error is generated when calling source.getFeatures(Filter.INCLUDE).
Stacktrace
java.lang.NoSuchFieldError: METER
at org.geotools.referencing.wkt.Parser.parseSpheroid(Parser.java:560)
at org.geotools.referencing.wkt.Parser.parseDatum(Parser.java:656)
at org.geotools.referencing.wkt.Parser.parseGeoGCS(Parser.java:867)
at org.geotools.referencing.wkt.Parser.parseProjCS(Parser.java:913)
at org.geotools.referencing.wkt.Parser.parseCoordinateReferenceSystem(Parser.java:225)
at org.geotools.referencing.wkt.Parser.parseCoordinateReferenceSystem(Parser.java:204)
I found out that SI.METER is linked to javax.measure but even if I add it to the project, the error is still present.
The result of mvn dependency:tree is
[INFO] +- com.vividsolutions:jts:jar:1.13:compile
[INFO] +- org.opengis:geoapi:jar:3.0.0:compile
[INFO] | \- javax.measure:jsr-275:jar:0.9.3:compile
[INFO] +- org.geotools:gt-data:jar:15.2:compile
[INFO] | +- org.geotools:gt-main:jar:15.2:compile
[INFO] | | \- org.geotools:gt-api:jar:15.2:compile
[INFO] | \- javax.media:jai_core:jar:1.1.3:compile
[INFO] +- org.geotools:gt-shapefile:jar:15.2:compile
[INFO] | \- org.jdom:jdom:jar:1.1.3:compile
[INFO] \- org.geotools:gt-swing:jar:15.2:compile
[INFO] +- org.geotools:gt-referencing:jar:15.2:compile
[INFO] | +- com.googlecode.efficient-java-matrix-library:core:jar:0.26:compile
[INFO] | +- commons-pool:commons-pool:jar:1.5.4:compile
[INFO] | +- org.geotools:gt-metadata:jar:15.2:compile
[INFO] | | \- org.geotools:gt-opengis:jar:15.2:compile
[INFO] | | \- net.java.dev.jsr-275:jsr-275:jar:1.0-beta-2:compile
[INFO] | +- jgridshift:jgridshift:jar:1.0:compile
[INFO] | \- net.sf.geographiclib:GeographicLib-Java:jar:1.44:compile
[INFO] +- org.geotools:gt-render:jar:15.2:compile
[INFO] | +- org.geotools:gt-coverage:jar:15.2:compile
[INFO] | | +- javax.media:jai_imageio:jar:1.1:compile
[INFO] | | +- it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15:compile
[INFO] | | | +- it.geosolutions.imageio-ext:imageio-ext-utilities:jar:1.1.15:compile
[INFO] | | | +- it.geosolutions.imageio-ext:imageio-ext-geocore:jar:1.1.15:compile
[INFO] | | | | \- it.geosolutions.imageio-ext:imageio-ext-streams:jar:1.1.15:compile
[INFO] | | | \- javax.media:jai_codec:jar:1.1.3:compile
[INFO] | | +- org.jaitools:jt-zonalstats:jar:1.4.0:compile
[INFO] | | +- org.jaitools:jt-utils:jar:1.4.0:compile
[INFO] | | +- it.geosolutions.jaiext.affine:jt-affine:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.algebra:jt-algebra:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandmerge:jt-bandmerge:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandselect:jt-bandselect:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandcombine:jt-bandcombine:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.border:jt-border:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.buffer:jt-buffer:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.crop:jt-crop:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.iterators:jt-iterators:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.lookup:jt-lookup:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.mosaic:jt-mosaic:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.nullop:jt-nullop:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.rescale:jt-rescale:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.scale:jt-scale:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.stats:jt-stats:jar:1.0.11:compile
[INFO] | | | \- com.google.guava:guava:jar:17.0:compile
[INFO] | | +- it.geosolutions.jaiext.translate:jt-translate:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.utilities:jt-utilities:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.warp:jt-warp:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.zonal:jt-zonal:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.binarize:jt-binarize:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.format:jt-format:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.colorconvert:jt-colorconvert:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.errordiffusion:jt-errordiffusion:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.orderdither:jt-orderdither:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.colorindexer:jt-colorindexer:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.imagefunction:jt-imagefunction:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.piecewise:jt-piecewise:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.classifier:jt-classifier:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.rlookup:jt-rlookup:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.vectorbin:jt-vectorbin:jar:1.0.11:compile
[INFO] | | \- commons-io:commons-io:jar:2.1:compile
[INFO] | \- org.geotools:gt-cql:jar:15.2:compile
[INFO] \- com.miglayout:miglayout:jar:swing:3.7:compile
Am I missing a dependency ?
The answer was found with the help of #IanTurton
I upgraded the geotools version from 15.2 to 20.5. I also added the gt-epsg-hsql jar. The pom.xml is now like this :
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-api</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-shapefile</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-epsg-hsql</artifactId>
<version>20.5</version>
</dependency>
I also updated my code to use org.locationtech.jts.geom.Geometry instead of com.vividsolutions
I came across this problem when trying to integrate kafka, storm and druid.
Their corresponding versions are:
kafka_2.11-0.8.2.2
druid-0.9.1.1
apache-storm-0.10.1
log4j-over-slf4j.jar and slf4j-log4j12.jar conflitct... but i don't have log4j-over-slf4j
This question almost resembles my question except that I got this error even though I have excluded slf4j-log4j12.jar. This is how the error looks like in storm log that corresponding to a KafkaSpout(storm.kafka.KafkaSpout):
2016-08-02 14:54:50.446 STDIO [ERROR] SLF4J: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError.
2016-08-02 14:54:50.446 o.s.c.ConfigurationObjectFactory [INFO] Using method itself for [task.warmingPeriod] on [com.metamx.tranquility.config.PropertiesBasedConfig#taskWarmingPeriod()]
2016-08-02 14:54:50.446 STDIO [ERROR] SLF4J: See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
2016-08-02 14:54:50.447 b.s.util [ERROR] Async loop died!
java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory
This is my output for mvn dependency:tree
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) # CloudStormDruid ---
[INFO] com.cloud.xxx:CloudStormDruid:war:1.0
[INFO] +- io.druid:tranquility-core_2.11:jar:0.8.2:compile
[INFO] | +- org.scala-lang:scala-library:jar:2.11.7:compile
[INFO] | +- com.metamx:scala-util_2.11:jar:1.11.6:compile
[INFO] | | +- com.metamx:loglady_2.11:jar:1.1.0-mmx:compile
[INFO] | | +- com.metamx:http-client:jar:1.0.3:compile
[INFO] | | +- com.metamx:emitter:jar:0.3.3:compile
[INFO] | | +- com.metamx:server-metrics:jar:0.2.6:compile
[INFO] | | +- joda-time:joda-time:jar:2.1:compile
[INFO] | | +- org.joda:joda-convert:jar:1.6:compile
[INFO] | | +- org.scalaj:scalaj-time_2.11:jar:0.5:compile
[INFO] | | +- org.skife.config:config-magic:jar:0.9:compile
[INFO] | | +- com.google.guava:guava:jar:16.0.1:compile
[INFO] | | +- org.yaml:snakeyaml:jar:1.11:compile
[INFO] | | +- org.jdbi:jdbi:jar:2.27:compile
[INFO] | | +- com.h2database:h2:jar:1.3.158:compile
[INFO] | | +- c3p0:c3p0:jar:0.9.1.2:compile
[INFO] | | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | +- org.apache.curator:curator-recipes:jar:2.6.0:compile
[INFO] | | \- org.apache.curator:curator-x-discovery:jar:2.6.0:compile
[INFO] | +- com.metamx:java-util:jar:0.27.4:compile
[INFO] | | \- net.sf.opencsv:opencsv:jar:2.3:compile
[INFO] | +- io.netty:netty:jar:3.10.5.Final:compile
[INFO] | +- com.twitter:util-core_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-function_2.11:jar:6.30.0:compile
[INFO] | | \- com.twitter:jsr166e:jar:1.0.0:compile
[INFO] | +- com.twitter:finagle-core_2.11:jar:6.31.0:compile
[INFO] | | +- com.twitter:util-app_2.11:jar:6.30.0:compile
[INFO] | | | \- com.twitter:util-registry_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-cache_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-codec_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-collection_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-hashing_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-jvm_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-lint_2.11:jar:6.30.0:compile
[INFO] | | +- com.twitter:util-logging_2.11:jar:6.30.0:compile
[INFO] | | \- com.twitter:util-stats_2.11:jar:6.30.0:compile
[INFO] | +- com.twitter:finagle-http_2.11:jar:6.31.0:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.12:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.4.6:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.6:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.6:compile
[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.4.6:compile
[INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-joda:jar:2.4.6:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.4.5:compile
[INFO] | | +- org.scala-lang:scala-reflect:jar:2.11.2:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | | \- com.google.code.findbugs:jsr305:jar:2.0.1:compile
[INFO] | +- io.druid:druid-server:jar:0.9.1:compile
[INFO] | | +- io.druid:druid-processing:jar:0.9.1:compile
[INFO] | | | +- io.druid:druid-common:jar:0.9.1:compile
[INFO] | | | | +- io.druid:druid-api:jar:0.9.1:compile
[INFO] | | | | | \- io.airlift:airline:jar:0.7:compile
[INFO] | | | | | \- com.google.code.findbugs:annotations:jar:2.0.3:compile
[INFO] | | | | +- org.apache.commons:commons-dbcp2:jar:2.0.1:compile
[INFO] | | | | +- commons-pool:commons-pool:jar:1.6:compile
[INFO] | | | | +- org.hibernate:hibernate-validator:jar:5.1.3.Final:compile
[INFO] | | | | | \- com.fasterxml:classmate:jar:1.0.0:compile
[INFO] | | | | +- javax.el:javax.el-api:jar:3.0.0:compile
[INFO] | | | | +- com.fasterxml.jackson.datatype:jackson-datatype-guava:jar:2.4.6:compile
[INFO] | | | | +- org.apache.logging.log4j:log4j-jul:jar:2.5:compile
[INFO] | | | | +- org.slf4j:jcl-over-slf4j:jar:1.7.12:compile
[INFO] | | | | +- net.java.dev.jets3t:jets3t:jar:0.9.4:compile
[INFO] | | | | | +- javax.activation:activation:jar:1.1.1:compile
[INFO] | | | | | \- com.jamesmurty.utils:java-xmlbuilder:jar:1.1:compile
[INFO] | | | | | \- net.iharder:base64:jar:2.3.8:compile
[INFO] | | | | \- org.antlr:antlr4-runtime:jar:4.5.1:compile
[INFO] | | | +- com.metamx:bytebuffer-collections:jar:0.2.4:compile
[INFO] | | | | +- com.metamx:extendedset:jar:1.3.9:compile
[INFO] | | | | \- org.roaringbitmap:RoaringBitmap:jar:0.5.16:compile
[INFO] | | | +- com.ning:compress-lzf:jar:1.0.3:compile
[INFO] | | | +- com.ibm.icu:icu4j:jar:4.8.1:compile
[INFO] | | | +- org.mozilla:rhino:jar:1.7R5:compile
[INFO] | | | \- org.mapdb:mapdb:jar:1.0.8:compile
[INFO] | | +- io.druid:druid-aws-common:jar:0.9.1:compile
[INFO] | | | \- com.amazonaws:aws-java-sdk:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-support:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-simpledb:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-storagegateway:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-route53:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-s3:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-importexport:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-sts:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-sqs:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-rds:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-redshift:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-elasticbeanstalk:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-glacier:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-iam:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-datapipeline:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-elasticloadbalancing:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-emr:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-elasticache:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-elastictranscoder:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-ec2:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-sns:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudtrail:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-logs:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cognitoidentity:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cognitosync:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-directconnect:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudformation:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudfront:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-kinesis:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-opsworks:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-ses:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-autoscaling:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudsearch:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudwatchmetrics:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-swf-libraries:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-codedeploy:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-codepipeline:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-kms:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-config:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-lambda:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-ecs:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-cloudhsm:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-ssm:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-workspaces:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-machinelearning:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-directory:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-efs:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-codecommit:jar:1.10.21:compile
[INFO] | | | +- com.amazonaws:aws-java-sdk-devicefarm:jar:1.10.21:compile
[INFO] | | | \- com.amazonaws:aws-java-sdk-core:jar:1.10.21:compile
[INFO] | | +- io.druid:druid-console:jar:0.0.3:compile
[INFO] | | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | +- org.glassfish:javax.el:jar:3.0.0:compile
[INFO] | | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.4.6:compile
[INFO] | | | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.4.6:compile
[INFO] | | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.4.6:compile
[INFO] | | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-smile-provider:jar:2.4.6:compile
[INFO] | | +- com.sun.jersey:jersey-server:jar:1.19:compile
[INFO] | | +- com.sun.jersey:jersey-core:jar:1.19:compile
[INFO] | | | \- javax.ws.rs:jsr311-api:jar:1.1.1:compile
[INFO] | | +- com.sun.jersey.contribs:jersey-guice:jar:1.19:compile
[INFO] | | | \- com.sun.jersey:jersey-servlet:jar:1.19:compile
[INFO] | | +- org.eclipse.jetty:jetty-server:jar:9.2.5.v20141112:compile
[INFO] | | | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | | | +- org.eclipse.jetty:jetty-http:jar:9.2.5.v20141112:compile
[INFO] | | | \- org.eclipse.jetty:jetty-io:jar:9.2.5.v20141112:compile
[INFO] | | +- org.eclipse.jetty:jetty-proxy:jar:9.2.5.v20141112:compile
[INFO] | | | +- org.eclipse.jetty:jetty-client:jar:9.2.5.v20141112:compile
[INFO] | | | \- org.eclipse.jetty:jetty-util:jar:9.2.5.v20141112:compile
[INFO] | | +- io.tesla.aether:tesla-aether:jar:0.0.5:compile
[INFO] | | | +- org.eclipse.aether:aether-spi:jar:0.9.0.M2:compile
[INFO] | | | +- org.eclipse.aether:aether-util:jar:0.9.0.M2:compile
[INFO] | | | +- org.eclipse.aether:aether-impl:jar:0.9.0.M2:compile
[INFO] | | | +- org.eclipse.aether:aether-connector-file:jar:0.9.0.M2:compile
[INFO] | | | +- io.tesla.aether:aether-connector-okhttp:jar:0.0.9:compile
[INFO] | | | | +- com.squareup.okhttp:okhttp:jar:1.0.2:compile
[INFO] | | | | \- org.apache.maven.wagon:wagon-provider-api:jar:2.4:compile
[INFO] | | | +- org.apache.maven:maven-aether-provider:jar:3.1.1:compile
[INFO] | | | | +- org.apache.maven:maven-model:jar:3.1.1:compile
[INFO] | | | | +- org.apache.maven:maven-model-builder:jar:3.1.1:compile
[INFO] | | | | +- org.apache.maven:maven-repository-metadata:jar:3.1.1:compile
[INFO] | | | | \- org.codehaus.plexus:plexus-utils:jar:3.0.15:compile
[INFO] | | | +- org.apache.maven:maven-settings-builder:jar:3.1.1:compile
[INFO] | | | | \- org.codehaus.plexus:plexus-interpolation:jar:1.19:compile
[INFO] | | | \- org.apache.maven:maven-settings:jar:3.1.1:compile
[INFO] | | +- org.eclipse.aether:aether-api:jar:0.9.0.M2:compile
[INFO] | | +- net.spy:spymemcached:jar:2.11.7:compile
[INFO] | | +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] | | +- org.eclipse.jetty:jetty-servlet:jar:9.2.5.v20141112:compile
[INFO] | | | \- org.eclipse.jetty:jetty-security:jar:9.2.5.v20141112:compile
[INFO] | | +- org.eclipse.jetty:jetty-servlets:jar:9.2.5.v20141112:compile
[INFO] | | | \- org.eclipse.jetty:jetty-continuation:jar:9.2.5.v20141112:compile
[INFO] | | +- com.ircclouds.irc:irc-api:jar:1.0-0014:compile
[INFO] | | +- com.maxmind.geoip2:geoip2:jar:0.4.0:compile
[INFO] | | | +- com.maxmind.maxminddb:maxminddb:jar:0.2.0:compile
[INFO] | | | \- com.google.http-client:google-http-client-jackson2:jar:1.15.0-rc:compile
[INFO] | | +- org.apache.derby:derby:jar:10.11.1.1:compile
[INFO] | | +- org.apache.derby:derbynet:jar:10.11.1.1:compile
[INFO] | | +- org.apache.derby:derbyclient:jar:10.11.1.1:compile
[INFO] | | \- org.apache.commons:commons-math3:jar:3.6.1:compile
[INFO] | +- com.google.inject:guice:jar:4.0:compile
[INFO] | +- com.google.inject.extensions:guice-servlet:jar:4.0:compile
[INFO] | +- com.google.inject.extensions:guice-multibindings:jar:4.0:compile
[INFO] | \- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] +- io.druid:tranquility-storm_2.11:jar:0.8.2:compile
[INFO] +- org.apache.storm:storm-core:jar:0.10.1:provided
[INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.21:provided
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:provided
[INFO] | | | \- org.ow2.asm:asm:jar:4.0:provided
[INFO] | | \- com.esotericsoftware.minlog:minlog:jar:1.2:provided
[INFO] | +- org.clojure:clojure:jar:1.6.0:provided
[INFO] | +- com.googlecode.disruptor:disruptor:jar:2.10.4:provided
[INFO] | +- org.apache.logging.log4j:log4j-api:jar:2.1:provided
[INFO] | +- org.apache.logging.log4j:log4j-core:jar:2.1:provided
[INFO] | +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.1:provided
[INFO] | +- org.slf4j:log4j-over-slf4j:jar:1.6.6:provided
[INFO] | \- org.apache.hadoop:hadoop-auth:jar:2.4.0:provided
[INFO] +- org.apache.storm:storm-kafka:jar:0.10.1:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- org.apache.curator:curator-framework:jar:2.5.0:compile
[INFO] | | \- org.apache.curator:curator-client:jar:2.5.0:compile
[INFO] | +- commons-lang:commons-lang:jar:2.5:compile
[INFO] | \- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] +- org.apache.kafka:kafka_2.11:jar:0.8.2.2:compile
[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.2:compile
[INFO] | +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] | +- net.sf.jopt-simple:jopt-simple:jar:3.2:compile
[INFO] | +- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.2:compile
[INFO] | +- com.101tec:zkclient:jar:0.3:compile
[INFO] | \- org.apache.kafka:kafka-clients:jar:0.8.2.2:compile
[INFO] | \- org.xerial.snappy:snappy-java:jar:1.1.1.7:compile
[INFO] +- com.jayway.jsonpath:json-path:jar:2.0.0:compile
[INFO] | \- net.minidev:json-smart:jar:2.1.1:compile
[INFO] | \- net.minidev:asm:jar:1.0.2:compile
[INFO] | \- asm:asm:jar:3.3.1:compile
[INFO] +- com.cloud.base:CloudUtil:jar:1.0:compile
[INFO] | +- com.aliyun.odps:odps-sdk-core:jar:0.20.7-public:compile
[INFO] | | +- com.aliyun.odps:odps-sdk-commons:jar:0.20.7-public:compile
[INFO] | | | \- com.jcabi:jcabi-aspects:jar:0.20.1:compile
[INFO] | | | +- com.jcabi:jcabi-log:jar:0.15:compile
[INFO] | | | \- org.aspectj:aspectjrt:jar:1.8.2:compile
[INFO] | | +- commons-codec:commons-codec:jar:1.9:compile
[INFO] | | +- net.sourceforge.javacsv:javacsv:jar:2.0:compile
[INFO] | | +- org.bouncycastle:bcprov-jdk15on:jar:1.52:compile
[INFO] | | \- com.google.protobuf:protobuf-java:jar:2.4.1:compile
[INFO] | +- org.apache.struts:struts2-core:jar:2.3.15.1:compile
[INFO] | | +- org.apache.struts.xwork:xwork-core:jar:2.3.15.1:compile
[INFO] | | | \- asm:asm-commons:jar:3.3:compile
[INFO] | | | \- asm:asm-tree:jar:3.3:compile
[INFO] | | +- org.freemarker:freemarker:jar:2.3.19:compile
[INFO] | | \- ognl:ognl:jar:3.0.6:compile
[INFO] | +- org.apache.struts:struts2-spring-plugin:jar:2.3.15.1:compile
[INFO] | +- org.apache.struts:struts2-convention-plugin:jar:2.3.15.1:compile
[INFO] | +- org.apache.struts:struts2-json-plugin:jar:2.3.15.1:compile
[INFO] | +- com.thoughtworks.xstream:xstream:jar:1.4.5:compile
[INFO] | | +- xmlpull:xmlpull:jar:1.1.3.1:compile
[INFO] | | \- xpp3:xpp3_min:jar:1.1.4c:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.2.1:compile
[INFO] | +- com.lowagie:itext:jar:2.1.7:compile
[INFO] | | +- bouncycastle:bcmail-jdk14:jar:138:compile
[INFO] | | +- bouncycastle:bcprov-jdk14:jar:138:compile
[INFO] | | \- org.bouncycastle:bctsp-jdk14:jar:1.38:compile
[INFO] | | +- org.bouncycastle:bcprov-jdk14:jar:1.38:compile
[INFO] | | \- org.bouncycastle:bcmail-jdk14:jar:1.38:compile
[INFO] | +- com.lowagie:itextasian:jar:1.0:compile
[INFO] | +- xerces:xercesImpl:jar:2.11.0:compile
[INFO] | | \- xml-apis:xml-apis:jar:1.4.01:compile
[INFO] | +- net.sourceforge.jexcelapi:jxl:jar:2.6.10:compile
[INFO] | +- commons-fileupload:commons-fileupload:jar:1.3:compile
[INFO] | +- org.apache.commons:commons-compress:jar:1.6:compile
[INFO] | | \- org.tukaani:xz:jar:1.4:compile
[INFO] | +- org.apache.poi:poi:jar:3.9:compile
[INFO] | +- org.apache.poi:poi-ooxml:jar:3.9:compile
[INFO] | | \- org.apache.poi:poi-ooxml-schemas:jar:3.9:compile
[INFO] | | \- org.apache.xmlbeans:xmlbeans:jar:2.3.0:compile
[INFO] | | \- stax:stax-api:jar:1.0.1:compile
[INFO] | +- jaxen:jaxen:jar:1.1.6:compile
[INFO] | +- net.sf.json-lib:json-lib:jar:jdk15:2.4:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | \- net.sf.ezmorph:ezmorph:jar:1.0.6:compile
[INFO] | +- com.alibaba:dubbo:jar:2.5.3:compile
[INFO] | +- org.apache.zookeeper:zookeeper:pom:3.4.5:compile
[INFO] | | \- jline:jline:jar:0.9.94:compile
[INFO] | +- redis.clients:jedis:jar:2.7.2:compile
[INFO] | | \- org.apache.commons:commons-pool2:jar:2.3:compile
[INFO] | +- org.springframework.data:spring-data-redis:jar:1.6.0.RELEASE:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.1:compile
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.3:compile
[INFO] | \- org.jdom:jdom:jar:1.1.3:compile
[INFO] +- javax.servlet:jstl:jar:1.2:compile
[INFO] +- commons-beanutils:commons-beanutils:jar:1.8.3:compile
[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] +- javax.servlet.jsp:jsp-api:jar:2.2.1-b03:provided
[INFO] +- javax.servlet:servlet-api:jar:2.5:provided
[INFO] +- org.slf4j:slf4j-api:jar:1.7.6:compile
[INFO] +- com.alibaba:fastjson:jar:1.1.24:compile
[INFO] +- org.aspectj:aspectjweaver:jar:1.7.0:compile
[INFO] +- com.alibaba:druid:jar:1.0.3:compile
[INFO] +- mysql:mysql-connector-java:jar:5.1.21:compile
[INFO] +- junit:junit:jar:4.10:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.1:test
[INFO] +- org.hibernate:hibernate-core:jar:4.1.7.Final:compile
[INFO] | +- antlr:antlr:jar:2.7.7:compile
[INFO] | +- org.jboss.logging:jboss-logging:jar:3.1.0.GA:compile
[INFO] | +- org.jboss.spec.javax.transaction:jboss-transaction-api_1.1_spec:jar:1.0.0.Final:compile
[INFO] | +- dom4j:dom4j:jar:1.6.1:compile
[INFO] | +- org.hibernate.javax.persistence:hibernate-jpa-2.0-api:jar:1.0.1.Final:compile
[INFO] | +- org.javassist:javassist:jar:3.15.0-GA:compile
[INFO] | \- org.hibernate.common:hibernate-commons-annotations:jar:4.0.1.Final:compile
[INFO] +- org.hibernate:hibernate-ehcache:jar:4.1.7.Final:compile
[INFO] | \- net.sf.ehcache:ehcache-core:jar:2.4.3:compile
[INFO] +- org.springframework:spring-core:jar:3.2.4.RELEASE:compile
[INFO] +- org.springframework:spring-context:jar:3.2.4.RELEASE:compile
[INFO] | \- org.springframework:spring-aop:jar:3.2.4.RELEASE:compile
[INFO] +- org.springframework:spring-jdbc:jar:3.2.4.RELEASE:compile
[INFO] | \- org.springframework:spring-tx:jar:3.2.4.RELEASE:compile
[INFO] +- org.springframework:spring-beans:jar:3.2.4.RELEASE:compile
[INFO] +- org.springframework:spring-web:jar:3.2.4.RELEASE:compile
[INFO] | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] +- org.springframework:spring-expression:jar:3.2.4.RELEASE:compile
[INFO] +- org.springframework:spring-orm:jar:3.2.4.RELEASE:compile
[INFO] +- org.projectreactor:reactor-spring:jar:1.0.1.RELEASE:compile
[INFO] | \- org.projectreactor:reactor-core:jar:1.0.1.RELEASE:compile
[INFO] | \- com.lmax:disruptor:jar:3.2.0:compile
[INFO] +- org.springframework:spring-test:jar:3.2.4.RELEASE:test
[INFO] \- org.springframework:spring-context-support:jar:3.2.4.RELEASE:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
I excluded slf4j-log4j12.jar for kafka as such in my pom.xml:
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>0.10.1</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.8.2.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
And I used maven-assembly-plugin to build a distribution bin for storm-nimbus to parse:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.cloud.storm.topology.TopologyService</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
What would be the cause of this problem? What would I have to do if I want to exclude slf4j-log4j12 and keep log4j-over-slf4j?
Thanks in Advance.
Find if there is a org.slf4j.impl.Log4jLoggerFactory class in your topology jar.
For example, activemq-all jar has this class, and will cause slf4j-log4j12.jar detected.
When you exclude it from your kafka dependency, you are excluding the logging dependencies storm requires that were coming transitively. To solve it, you need to declare it explicitly by including slf4j-api and log4j-over-sl4j in your dependency list, as below:
<dependencies>
<!-- (...) some other dependencies (...) -->
<dependency>
<!-- should be excluded from the shaded binary-->
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>${storm.version}</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>${storm-kafka.version}</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>${kafka.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
Just as a notice, I normally use maven-shade-plugin to build the binary package for nimbus, as this is the recommended way and makes your life easier by providing the ability to merge manifest entries. This can be done as below:
<plugins>
<!-- (...) some other plugins (...) -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<minimizeJar>false</minimizeJar>
<artifactSet>
<excludes>
<exclude>org.apache.storm:storm-core</exclude>
</excludes>
</artifactSet>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>MyTopologyMainClassFQDN</mainClass>
</transformer>
</transformers>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
i am trying to write to hdfs, i am sure i have same version hdfs clients in my pom.xml, some other dependencies are causing the issue. following are the details. i am using CDH 5.5.1 hadoop 2.6.0
if i just specify
conf.set("fs.defaultFS",
"localhost:8020");
it throws
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:8020/user/hdfs, expected: file:///
if i specify
conf.set("fs.default.name",
"localhost:8020");
it throws
org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
at org.apache.hadoop.ipc.Client.call(Client.java:1066)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at com.sun.proxy.$Proxy3.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:118)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:222)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:187)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1328)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:65)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1346)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:244)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:122)
maven dependency tree
+- junit:junit:jar:4.12:compile
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] +- com.googlecode.protobuf-java-format:protobuf-java-format:jar:1.4:compile
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.3:compile
[INFO] | \- org.apache.logging.log4j:log4j-api:jar:2.3:compile
[INFO] +- org.apache.avro:avro-tools:jar:1.7.7:compile
[INFO] | \- org.slf4j:slf4j-api:jar:1.6.4:compile
[INFO] +- org.apache.spark:spark-core_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.twitter:chill_2.11:jar:0.5.0:compile
[INFO] | | \- com.esotericsoftware.kryo:kryo:jar:2.21:compile
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | \- org.objenesis:objenesis:jar:1.2:compile
[INFO] | +- com.twitter:chill-java:jar:0.5.0:compile
[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:1.5.0:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
[INFO] | | +- commons-codec:commons-codec:jar:1.3:compile
[INFO] | | \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile
[INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:compile
[INFO] | | \- com.google.guava:guava:jar:14.0.1:compile
[INFO] | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.4.1:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.10:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.10:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile
[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.1.7:compile
[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.4.5:compile
[INFO] | +- com.typesafe.akka:akka-remote_2.11:jar:2.3.11:compile
[INFO] | | +- com.typesafe.akka:akka-actor_2.11:jar:2.3.11:compile
[INFO] | | | \- com.typesafe:config:jar:1.2.1:compile
[INFO] | | +- io.netty:netty:jar:3.8.0.Final:compile
[INFO] | | \- org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile
[INFO] | +- com.typesafe.akka:akka-slf4j_2.11:jar:2.3.11:compile
[INFO] | +- org.scala-lang:scala-library:jar:2.11.7:compile
[INFO] | +- org.json4s:json4s-jackson_2.11:jar:3.2.10:compile
[INFO] | | \- org.json4s:json4s-core_2.11:jar:3.2.10:compile
[INFO] | | +- org.json4s:json4s-ast_2.11:jar:3.2.10:compile
[INFO] | | \- org.scala-lang:scalap:jar:2.11.0:compile
[INFO] | | \- org.scala-lang:scala-compiler:jar:2.11.0:compile
[INFO] | | \- org.scala-lang.modules:scala-xml_2.11:jar:1.0.1:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1:compile
[INFO] | +- io.netty:netty-all:jar:4.0.29.Final:compile
[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.0:compile
[INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.4.4:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.4.4:compile
[INFO] | | +- org.scala-lang:scala-reflect:jar:2.11.2:compile
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile
[INFO] | +- oro:oro:jar:2.0.8:compile
[INFO] | +- org.tachyonproject:tachyon-client:jar:0.7.1:compile
[INFO] | | +- commons-lang:commons-lang:jar:2.4:compile
[INFO] | | +- org.apache.curator:curator-client:jar:2.1.0-incubating:compile
[INFO] | | +- org.tachyonproject:tachyon-underfs-hdfs:jar:0.7.1:compile
[INFO] | | \- org.tachyonproject:tachyon-underfs-local:jar:0.7.1:compile
[INFO] | +- net.razorvine:pyrolite:jar:4.4:compile
[INFO] | +- net.sf.py4j:py4j:jar:0.8.2.1:compile
[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile
[INFO] +- org.apache.spark:spark-sql_2.11:jar:1.5.0:provided
[INFO] | +- org.apache.spark:spark-catalyst_2.11:jar:1.5.0:provided
[INFO] | | \- org.codehaus.janino:janino:jar:2.7.8:provided
[INFO] | | \- org.codehaus.janino:commons-compiler:jar:2.7.8:provided
[INFO] | +- org.apache.parquet:parquet-column:jar:1.7.0:provided
[INFO] | | +- org.apache.parquet:parquet-common:jar:1.7.0:provided
[INFO] | | \- org.apache.parquet:parquet-encoding:jar:1.7.0:provided
[INFO] | | \- org.apache.parquet:parquet-generator:jar:1.7.0:provided
[INFO] | \- org.apache.parquet:parquet-hadoop:jar:1.7.0:provided
[INFO] | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:provided
[INFO] | \- org.apache.parquet:parquet-jackson:jar:1.7.0:provided
[INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.52:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:3.0.0-beta-2:compile
[INFO] +- com.databricks:spark-avro_2.11:jar:2.0.1:compile
[INFO] | \- org.apache.avro:avro:jar:1.7.6:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] | | | \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | | \- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile
[INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4:compile
[INFO] | | +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] | | | \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] | | +- com.sun.jersey:jersey-client:jar:1.9:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.8:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.8:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-aws:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- com.amazonaws:aws-java-sdk:jar:1.7.4:compile
[INFO] | \- org.apache.hadoop:hadoop-annotations:jar:2.6.0-cdh5.5.1:compile
[INFO] +- org.apache.kafka:kafka_2.11:jar:0.9.0-kafka-2.0.0:compile
[INFO] | +- com.101tec:zkclient:jar:0.7:compile
[INFO] | +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] | +- net.sf.jopt-simple:jopt-simple:jar:4.9:compile
[INFO] | +- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:compile
[INFO] | \- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] | \- jline:jline:jar:0.9.94:compile
[INFO] +- org.apache.kafka:kafka-clients:jar:0.9.0-kafka-2.0.0:compile
[INFO] +- commons-io:commons-io:jar:2.4:compile
[INFO] +- javax.xml.bind:jaxb-api:jar:2.2.11:compile
[INFO] +- org.jsoup:jsoup:jar:1.8.3:compile
[INFO] +- jdk.tools:jdk.tools:jar:1.7:compile
[INFO] +- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] \- com.google.code.gson:gson:jar:2.5:compile
how do i resolve this issue.
Yes because of old hadoop-core.jar inside some dependency hierarchy cause this issue. After spending some time and research I found something useful:
Please use following dependencies in your pom.xml.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>WordCount</groupId>
<artifactId>WordCount</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>3.2.0</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Sample java code which works well: AccessHDFS.java
package com.gpl.hadoop;
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class AccessHDFS {
public static void main(String args[]) {
String url = "hdfs://localhost:9000/user/hduser/input/test.txt";
FileSystem fs = null;
InputStream in = null;
try {
Configuration conf = new Configuration();
fs = FileSystem.get(URI.create(url), conf);
in = fs.open(new Path(url));
IOUtils.copyBytes(in, System.out, 4096, false);
} catch (Exception e) {
e.printStackTrace();
} finally {
IOUtils.closeStream(fs);
}
}
}
Alternatively you can add following repository with given dependency.
<repositories>
<repository> <id>mapr-releases</id>
<url>http://repository.mapr.com/maven/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>2.4.1-mapr-1408</version>
</dependency>
</dependencies>
Make sure you do select Force Update of Snapshots/Releases option to update maven project from:
Project --> Maven --> Update Project --> Select Force Update of Snapshots/Releases --> ok
Check that you have the correct dependency and version in your case: 2.6.0
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.6.0</version>
</dependency>
This should resolve your error. You could also exclude, hadoop-core in the dependency. See this blog for more on the above solution.
My local environment: OSX 10.9.2, java1.6
I use java api to connect hbase and maven to manage my project, I added Hbase-0.94.17 and Hadoop-core-1.0.4 into pom.xml, when I ran my .java program to connect Hbase I got the following error:
SLF4J: slf4j-api 1.6.x (or later) is incompatible with this binding.
SLF4J: Your binding is version 1.5.5 or earlier.
SLF4J: Upgrade your binding to version 1.6.x.
Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder;
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:128)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:107)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:295)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:269)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:281)
at org.apache.zookeeper.ZooKeeper.<clinit>(ZooKeeper.java:94)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:98)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:127)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:153)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:127)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1505)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:713)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:983)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:155)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:129)
at org.geogit.storage.hbase.MyLittleHBaseClient.main(MyLittleHBaseClient.java:29)
Here is what I got after ran mvm dependency:tree
INFO] --- maven-dependency-plugin:2.2:tree (default-cli) # geogit-hbase ---
[INFO] org.geogit:geogit-hbase:jar:0.8-SNAPSHOT
[INFO] +- org.geogit:geogit-core:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
[INFO] | +- com.vividsolutions:jts:jar:1.13:compile
[INFO] | +- org.geotools:gt-opengis:jar:10.5:compile
[INFO] | | +- net.java.dev.jsr-275:jsr-275:jar:1.0-beta-2:compile
[INFO] | | +- java3d:vecmath:jar:1.3.2:compile
[INFO] | | +- commons-pool:commons-pool:jar:1.5.4:compile
[INFO] | | \- javax.media:jai_core:jar:1.1.3:compile
[INFO] | +- org.geotools:gt-referencing:jar:10.5:compile
[INFO] | | +- org.geotools:gt-metadata:jar:10.5:compile
[INFO] | | \- jgridshift:jgridshift:jar:1.0:compile
[INFO] | +- org.geotools:gt-epsg-hsql:jar:10.5:compile
[INFO] | | \- hsqldb:hsqldb:jar:1.8.0.7:compile
[INFO] | +- org.geotools:gt-main:jar:10.5:compile
[INFO] | | \- org.geotools:gt-api:jar:10.5:compile
[INFO] | +- org.geotools:gt-cql:jar:10.5:compile
[INFO] | +- net.sourceforge.findbugs:jsr305:jar:1.3.7:compile
[INFO] | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | +- com.google.inject.extensions:guice-multibindings:jar:3.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.2:compile
[INFO] | \- com.ning:compress-lzf:jar:0.9.8:compile
[INFO] +- org.apache.hbase:hbase-client:jar:0.98.0-hadoop2:compile
[INFO] | +- org.apache.hbase:hbase-common:jar:0.98.0-hadoop2:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | \- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.apache.hbase:hbase-protocol:jar:0.98.0-hadoop2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.7:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- io.netty:netty:jar:3.6.6.Final:compile
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- org.slf4j:slf4j-log4j12:jar:1.6.1:compile
[INFO] | +- org.cloudera.htrace:htrace-core:jar:2.04:compile
[INFO] | | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] | | \- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.2.0:compile
[INFO] | | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | | +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | | +- org.codehaus.jettison:jettison:jar:1.0.1:compile (version managed from 1.1)
[INFO] | | | | \- stax:stax-api:jar:1.0.1:compile
[INFO] | | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO] | | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] | | +- commons-el:commons-el:jar:1.0:runtime
[INFO] | | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] | | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] | | | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] | | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | | \- org.tukaani:xz:jar:1.0:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.2.0:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-yarn-common:jar:2.2.0:compile
[INFO] | | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.2.0:compile
[INFO] | | | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | | | \- asm:asm:jar:3.1:compile
[INFO] | | | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile
[INFO] | | \- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.2.0:compile
[INFO] | \- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] +- org.geogit:geogit-mongodb:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.geogit:geogit-blueprints:jar:0.8-SNAPSHOT:compile
[INFO] | | +- com.tinkerpop.blueprints:blueprints-core:jar:2.4.0:compile
[INFO] | | | +- com.fasterxml.jackson.datatype:jackson-datatype-json-org:jar:2.1.2:compile
[INFO] | | | | +- com.fasterxml.jackson.core:jackson-core:jar:2.1.2:compile
[INFO] | | | | +- com.fasterxml.jackson.core:jackson-databind:jar:2.1.2:compile
[INFO] | | | | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.1.1:compile
[INFO] | | | | \- org.json:json:jar:20090211:compile
[INFO] | | | \- colt:colt:jar:1.2.0:compile
[INFO] | | | \- concurrent:concurrent:jar:1.3.4:compile
[INFO] | | \- com.tinkerpop.gremlin:gremlin-java:jar:2.4.0:compile
[INFO] | | \- com.tinkerpop:pipes:jar:2.4.0:compile
[INFO] | \- org.mongodb:mongo-java-driver:jar:2.11.3:compile
[INFO] +- com.google.guava:guava:jar:14.0.1:compile
[INFO] +- org.geogit:geogit-core:jar:tests:0.8-SNAPSHOT:test
[INFO] \- junit:junit:jar:4.10:test
[INFO] \- org.hamcrest:hamcrest-core:jar:1.1:test
And here I attach my pom.xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.geogit</groupId>
<artifactId>storage</artifactId>
<version>0.8-SNAPSHOT</version>
</parent>
<groupId>org.geogit</groupId>
<artifactId>geogit-hbase</artifactId>
<packaging>jar</packaging>
<name>HBase Storage for GeoGit objects</name>
<dependencies>
<dependency>
<groupId>org.geogit</groupId>
<artifactId>geogit-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.94.17</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.0.4</version>
</dependency>
<!-- is this necessary? -->
<!--
<dependency>
<groupId>com.boundlessgeo</groupId>
<artifactId>blongo</artifactId>
<version>0.1</version>
</dependency>
-->
<!-- is this necessary? -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<!-- Test scope dependencies -->
<dependency>
<groupId>org.geogit</groupId>
<artifactId>geogit-core</artifactId>
<version>${project.version}</version>
<scope>test</scope>
<classifier>tests</classifier>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
</profiles>
</project>
Have slf4j-api and choose a SLF4J binding of corresponding version, and add them as dependencies in your own POM. It is quite possibly some of your dependencies transitively resolve an incompatible version of SLF4J binding which caused the problem. Find which library is bringing in the unwanted SLF4J binding (you can easily do so by mvn dependency:tree), and add corresponding exclusion in your dependency to that lib.
(Normally SLF4J binding shouldn't be declared in library. It is in fact something wrong that you may want to report to the library developer once you found the actual cause)
Update
That's exactly what I am talking about:
From your dependency tree:
[INFO] +- org.geogit:geogit-core:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
.....
[INFO] +- org.apache.hbase:hbase-client:jar:0.98.0-hadoop2:compile
.....
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- org.slf4j:slf4j-log4j12:jar:1.6.1:compile
You got slf4j-log4j12 binding from hbase-client, and that's version 1.6.1
However, slf4j-api you have in your project is version 1.7.5. They are not compatible.
Proper way to solve is:
Exclude slf4j-log4j12 from your dependency of hbase-client
Declare a compatible SLF4J binding in your own project
You should also report this as a bug to zookeeper/hbase developer as it is not appropriate for a library to include SLF4J binding as compile scope dependency.