Server IPC version 9 cannot communicate with client version 4 - java

i am trying to write to hdfs, i am sure i have same version hdfs clients in my pom.xml, some other dependencies are causing the issue. following are the details. i am using CDH 5.5.1 hadoop 2.6.0
if i just specify
conf.set("fs.defaultFS",
"localhost:8020");
it throws
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:8020/user/hdfs, expected: file:///
if i specify
conf.set("fs.default.name",
"localhost:8020");
it throws
org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
at org.apache.hadoop.ipc.Client.call(Client.java:1066)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at com.sun.proxy.$Proxy3.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:118)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:222)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:187)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1328)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:65)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1346)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:244)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:122)
maven dependency tree
+- junit:junit:jar:4.12:compile
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] +- com.googlecode.protobuf-java-format:protobuf-java-format:jar:1.4:compile
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.3:compile
[INFO] | \- org.apache.logging.log4j:log4j-api:jar:2.3:compile
[INFO] +- org.apache.avro:avro-tools:jar:1.7.7:compile
[INFO] | \- org.slf4j:slf4j-api:jar:1.6.4:compile
[INFO] +- org.apache.spark:spark-core_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.twitter:chill_2.11:jar:0.5.0:compile
[INFO] | | \- com.esotericsoftware.kryo:kryo:jar:2.21:compile
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | \- org.objenesis:objenesis:jar:1.2:compile
[INFO] | +- com.twitter:chill-java:jar:0.5.0:compile
[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:1.5.0:compile
[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:1.5.0:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
[INFO] | | +- commons-codec:commons-codec:jar:1.3:compile
[INFO] | | \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile
[INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:compile
[INFO] | | \- com.google.guava:guava:jar:14.0.1:compile
[INFO] | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.4.1:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.10:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.10:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile
[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.1.7:compile
[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.4.5:compile
[INFO] | +- com.typesafe.akka:akka-remote_2.11:jar:2.3.11:compile
[INFO] | | +- com.typesafe.akka:akka-actor_2.11:jar:2.3.11:compile
[INFO] | | | \- com.typesafe:config:jar:1.2.1:compile
[INFO] | | +- io.netty:netty:jar:3.8.0.Final:compile
[INFO] | | \- org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile
[INFO] | +- com.typesafe.akka:akka-slf4j_2.11:jar:2.3.11:compile
[INFO] | +- org.scala-lang:scala-library:jar:2.11.7:compile
[INFO] | +- org.json4s:json4s-jackson_2.11:jar:3.2.10:compile
[INFO] | | \- org.json4s:json4s-core_2.11:jar:3.2.10:compile
[INFO] | | +- org.json4s:json4s-ast_2.11:jar:3.2.10:compile
[INFO] | | \- org.scala-lang:scalap:jar:2.11.0:compile
[INFO] | | \- org.scala-lang:scala-compiler:jar:2.11.0:compile
[INFO] | | \- org.scala-lang.modules:scala-xml_2.11:jar:1.0.1:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1:compile
[INFO] | +- io.netty:netty-all:jar:4.0.29.Final:compile
[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.0:compile
[INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.4.4:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.4.4:compile
[INFO] | | +- org.scala-lang:scala-reflect:jar:2.11.2:compile
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile
[INFO] | +- oro:oro:jar:2.0.8:compile
[INFO] | +- org.tachyonproject:tachyon-client:jar:0.7.1:compile
[INFO] | | +- commons-lang:commons-lang:jar:2.4:compile
[INFO] | | +- org.apache.curator:curator-client:jar:2.1.0-incubating:compile
[INFO] | | +- org.tachyonproject:tachyon-underfs-hdfs:jar:0.7.1:compile
[INFO] | | \- org.tachyonproject:tachyon-underfs-local:jar:0.7.1:compile
[INFO] | +- net.razorvine:pyrolite:jar:4.4:compile
[INFO] | +- net.sf.py4j:py4j:jar:0.8.2.1:compile
[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile
[INFO] +- org.apache.spark:spark-sql_2.11:jar:1.5.0:provided
[INFO] | +- org.apache.spark:spark-catalyst_2.11:jar:1.5.0:provided
[INFO] | | \- org.codehaus.janino:janino:jar:2.7.8:provided
[INFO] | | \- org.codehaus.janino:commons-compiler:jar:2.7.8:provided
[INFO] | +- org.apache.parquet:parquet-column:jar:1.7.0:provided
[INFO] | | +- org.apache.parquet:parquet-common:jar:1.7.0:provided
[INFO] | | \- org.apache.parquet:parquet-encoding:jar:1.7.0:provided
[INFO] | | \- org.apache.parquet:parquet-generator:jar:1.7.0:provided
[INFO] | \- org.apache.parquet:parquet-hadoop:jar:1.7.0:provided
[INFO] | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:provided
[INFO] | \- org.apache.parquet:parquet-jackson:jar:1.7.0:provided
[INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.52:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:3.0.0-beta-2:compile
[INFO] +- com.databricks:spark-avro_2.11:jar:2.0.1:compile
[INFO] | \- org.apache.avro:avro:jar:1.7.6:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] | | | \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | | \- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile
[INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4:compile
[INFO] | | +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] | | | \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | | +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] | | +- com.sun.jersey:jersey-client:jar:1.9:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.8:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.8:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-aws:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- com.amazonaws:aws-java-sdk:jar:1.7.4:compile
[INFO] | \- org.apache.hadoop:hadoop-annotations:jar:2.6.0-cdh5.5.1:compile
[INFO] +- org.apache.kafka:kafka_2.11:jar:0.9.0-kafka-2.0.0:compile
[INFO] | +- com.101tec:zkclient:jar:0.7:compile
[INFO] | +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] | +- net.sf.jopt-simple:jopt-simple:jar:4.9:compile
[INFO] | +- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:compile
[INFO] | \- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] | \- jline:jline:jar:0.9.94:compile
[INFO] +- org.apache.kafka:kafka-clients:jar:0.9.0-kafka-2.0.0:compile
[INFO] +- commons-io:commons-io:jar:2.4:compile
[INFO] +- javax.xml.bind:jaxb-api:jar:2.2.11:compile
[INFO] +- org.jsoup:jsoup:jar:1.8.3:compile
[INFO] +- jdk.tools:jdk.tools:jar:1.7:compile
[INFO] +- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] \- com.google.code.gson:gson:jar:2.5:compile
how do i resolve this issue.

Yes because of old hadoop-core.jar inside some dependency hierarchy cause this issue. After spending some time and research I found something useful:
Please use following dependencies in your pom.xml.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>WordCount</groupId>
<artifactId>WordCount</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>3.2.0</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Sample java code which works well: AccessHDFS.java
package com.gpl.hadoop;
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class AccessHDFS {
public static void main(String args[]) {
String url = "hdfs://localhost:9000/user/hduser/input/test.txt";
FileSystem fs = null;
InputStream in = null;
try {
Configuration conf = new Configuration();
fs = FileSystem.get(URI.create(url), conf);
in = fs.open(new Path(url));
IOUtils.copyBytes(in, System.out, 4096, false);
} catch (Exception e) {
e.printStackTrace();
} finally {
IOUtils.closeStream(fs);
}
}
}
Alternatively you can add following repository with given dependency.
<repositories>
<repository> <id>mapr-releases</id>
<url>http://repository.mapr.com/maven/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>2.4.1-mapr-1408</version>
</dependency>
</dependencies>
Make sure you do select Force Update of Snapshots/Releases option to update maven project from:
Project --> Maven --> Update Project --> Select Force Update of Snapshots/Releases --> ok

Check that you have the correct dependency and version in your case: 2.6.0
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.6.0</version>
</dependency>
This should resolve your error. You could also exclude, hadoop-core in the dependency. See this blog for more on the above solution.

Related

Spring #RestController not honoring #Validated

I recently upgraded to Spring Boot 2.7.0, and my the request body validations stopped working. From my research so far, I've learnt that the
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
is required after Spring Boot 2.3.0.
I've added the dependency and verified that it is the only validation jar on the classpath is jakarta.validation (not javax.validation).
Here's the Controller class:
#RestController
#RequestMapping("v1/services")
#Validated
public class ServiceController {
#PutMapping(
path = "/{id}",
consumes = APPLICATION_JSON_VALUE,
produces = APPLICATION_JSON_VALUE
)
#Validated(OnUpdate.class)
public ResponseEntity<Service> updateService(#PathVariable("id") int serviceId,
#Valid #RequestBody Service service) {
// Implementation omitted for brevity
}
}
Here is the domain class:
#Component
public class Service {
private int id;
NotNull(message = "Current status is required", groups = {OnCreate.class, OnUpdate.class})
private ServiceStatus currentStatus; // The field I want to validate
}
My partial pom.xml:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.0</version>
<relativePath/> <!-- lookup parent from repository --
</parent>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>jakarta.validation</groupId>
<artifactId>jakarta.validation-api</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
<version>7.0.4.Final</version>
</dependency>
My dependency tree:
[INFO] +- org.springframework.boot:spring-boot-starter:jar:2.7.0:compile
[INFO] | +- org.springframework.boot:spring-boot:jar:2.7.0:compile
[INFO] | | \- org.springframework:spring-context:jar:5.3.20:compile
[INFO] | +- org.springframework.boot:spring-boot-autoconfigure:jar:2.7.0:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-logging:jar:2.7.0:compile
[INFO] | | +- ch.qos.logback:logback-classic:jar:1.2.11:compile
[INFO] | | | \- ch.qos.logback:logback-core:jar:1.2.11:compile
[INFO] | | +- org.apache.logging.log4j:log4j-to-slf4j:jar:2.17.2:compile
[INFO] | | | \- org.apache.logging.log4j:log4j-api:jar:2.17.2:compile
[INFO] | | \- org.slf4j:jul-to-slf4j:jar:1.7.36:compile
[INFO] | +- jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile
[INFO] | +- org.springframework:spring-core:jar:5.3.20:compile
[INFO] | | \- org.springframework:spring-jcl:jar:5.3.20:compile
[INFO] | \- org.yaml:snakeyaml:jar:1.30:compile
[INFO] +- org.springframework.boot:spring-boot-starter-web:jar:2.7.0:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-json:jar:2.7.0:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-databind:jar:2.13.3:compile
[INFO] | | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.13.3:compile
[INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.13.3:compile
[INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.13.3:compile
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.13.3:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-tomcat:jar:2.7.0:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-core:jar:9.0.63:compile
[INFO] | | \- org.apache.tomcat.embed:tomcat-embed-websocket:jar:9.0.63:compile
[INFO] | +- org.springframework:spring-web:jar:5.3.20:compile
[INFO] | | \- org.springframework:spring-beans:jar:5.3.20:compile
[INFO] | \- org.springframework:spring-webmvc:jar:5.3.20:compile
[INFO] | +- org.springframework:spring-aop:jar:5.3.20:compile
[INFO] | \- org.springframework:spring-expression:jar:5.3.20:compile
[INFO] +- org.springframework.boot:spring-boot-starter-jdbc:jar:2.7.0:compile
[INFO] | +- com.zaxxer:HikariCP:jar:4.0.3:compile
[INFO] | | \- org.slf4j:slf4j-api:jar:1.7.36:compile
[INFO] | \- org.springframework:spring-jdbc:jar:5.3.20:compile
[INFO] | \- org.springframework:spring-tx:jar:5.3.20:compile
[INFO] +- org.springframework.boot:spring-boot-starter-validation:jar:2.7.0:compile
[INFO] | \- org.apache.tomcat.embed:tomcat-embed-el:jar:9.0.63:compile
[INFO] +- org.springframework.boot:spring-boot-devtools:jar:2.7.0:compile
[INFO] +- org.springdoc:springdoc-openapi-ui:jar:1.6.9:compile
[INFO] | +- org.springdoc:springdoc-openapi-webmvc-core:jar:1.6.9:compile
[INFO] | | \- org.springdoc:springdoc-openapi-common:jar:1.6.9:compile
[INFO] | | \- io.swagger.core.v3:swagger-core:jar:2.2.0:compile
[INFO] | | +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.13.3:compile
[INFO] | | +- io.swagger.core.v3:swagger-annotations:jar:2.2.0:compile
[INFO] | | \- io.swagger.core.v3:swagger-models:jar:2.2.0:compile
[INFO] | +- org.webjars:swagger-ui:jar:4.11.1:compile
[INFO] | \- org.webjars:webjars-locator-core:jar:0.50:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-core:jar:2.13.3:compile
[INFO] +- io.github.classgraph:classgraph:jar:4.8.147:compile
[INFO] +- jakarta.validation:jakarta.validation-api:jar:3.0.2:compile
[INFO] +- org.hibernate.validator:hibernate-validator:jar:7.0.4.Final:compile
[INFO] | +- org.jboss.logging:jboss-logging:jar:3.4.3.Final:compile
[INFO] | \- com.fasterxml:classmate:jar:1.5.1:compile
[INFO] +- com.h2database:h2:jar:2.1.212:runtime
[INFO] +- org.springframework.boot:spring-boot-starter-test:jar:2.7.0:test
[INFO] | +- org.springframework.boot:spring-boot-test:jar:2.7.0:test
[INFO] | +- org.springframework.boot:spring-boot-test-autoconfigure:jar:2.7.0:test
[INFO] | +- com.jayway.jsonpath:json-path:jar:2.7.0:test
[INFO] | | \- net.minidev:json-smart:jar:2.4.8:test
[INFO] | | \- net.minidev:accessors-smart:jar:2.4.8:test
[INFO] | | \- org.ow2.asm:asm:jar:9.1:test
[INFO] | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.3:compile
[INFO] | | \- jakarta.activation:jakarta.activation-api:jar:1.2.2:compile
[INFO] | +- org.assertj:assertj-core:jar:3.22.0:test
[INFO] | +- org.hamcrest:hamcrest:jar:2.2:test
[INFO] | +- org.junit.jupiter:junit-jupiter:jar:5.8.2:test
[INFO] | | +- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test
[INFO] | | +- org.junit.jupiter:junit-jupiter-params:jar:5.8.2:test
[INFO] | | \- org.junit.jupiter:junit-jupiter-engine:jar:5.8.2:test
[INFO] | +- org.mockito:mockito-core:jar:4.5.1:test
[INFO] | | +- net.bytebuddy:byte-buddy:jar:1.12.10:test
[INFO] | | +- net.bytebuddy:byte-buddy-agent:jar:1.12.10:test
[INFO] | | \- org.objenesis:objenesis:jar:3.2:test
[INFO] | +- org.mockito:mockito-junit-jupiter:jar:4.5.1:test
[INFO] | +- org.skyscreamer:jsonassert:jar:1.5.0:test
[INFO] | | \- com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:test
[INFO] | +- org.springframework:spring-test:jar:5.3.20:test
[INFO] | \- org.xmlunit:xmlunit-core:jar:2.9.0:test
[INFO] +- org.spockframework:spock-core:jar:2.1-groovy-3.0:test
[INFO] | +- org.codehaus.groovy:groovy:jar:3.0.10:test
[INFO] | \- org.junit.platform:junit-platform-engine:jar:1.8.2:test
[INFO] | +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] | +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
[INFO] | \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] +- org.spockframework:spock-spring:jar:2.1-groovy-3.0:test
[INFO] +- org.codehaus.groovy:groovy-json:jar:3.0.10:test
[INFO] \- junit:junit:jar:4.13.2:test
[INFO] \- org.hamcrest:hamcrest-core:jar:2.2:test
Lastly, I'm confident it's not the jakarta package because the following unit test passed:
import jakarta.validation.ConstraintViolation
import jakarta.validation.Validation
import jakarta.validation.Validator
def 'Null status on update should trigger violation'() {
given: 'I have an invalid service'
final LocalDate may4th2021 = LocalDate.of(2021, 5, 4)
Service serviceWithoutStatus = new Service(
id: 1,
contractId: 2,
dueDate: may4th2021,
currentStatus: null, // N.B. the violation
history: []
)
when: 'I validate the client'
Validator validator = Validation.byDefaultProvider()
.configure()
.messageInterpolator(new ParameterMessageInterpolator())
.buildValidatorFactory()
.getValidator()
Set<ConstraintViolation<Service>> violations = validator.validate(serviceWithoutStatus, OnUpdate.class)
then: 'There should be violations'
!violations.isEmpty()
violations.size() == 1
}
Edit: I am using Java 11
openjdk version "11.0.9.1" 2020-11-04 LTS
OpenJDK Runtime Environment Zulu11.43+1015-CA (build 11.0.9.1+1-LTS)
OpenJDK 64-Bit Server VM Zulu11.43+1015-CA (build 11.0.9.1+1-LTS, mixed mode)
Any help would greatly be appreciated. Thank you.
This issue was that the jakarta.validation-api does not work with #Validated. I had to use javax.validation
The code worked as expected after I removed the below dependencies:
<dependency>
<groupId>jakarta.validation</groupId>
<artifactId>jakarta.validation-api</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
<version>7.0.4.Final</version>
</dependency>

org.getools - NoSuchFieldError: METER

A Maven Java project using geotools 15.2 generates an error : NoSuchFieldError: METER
pom.xml
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-data</artifactId>
<version>15.2</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-shapefile</artifactId>
<version>15.2</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>15.2</version>
</dependency>
When running the code, the error is generated when calling source.getFeatures(Filter.INCLUDE).
Stacktrace
java.lang.NoSuchFieldError: METER
at org.geotools.referencing.wkt.Parser.parseSpheroid(Parser.java:560)
at org.geotools.referencing.wkt.Parser.parseDatum(Parser.java:656)
at org.geotools.referencing.wkt.Parser.parseGeoGCS(Parser.java:867)
at org.geotools.referencing.wkt.Parser.parseProjCS(Parser.java:913)
at org.geotools.referencing.wkt.Parser.parseCoordinateReferenceSystem(Parser.java:225)
at org.geotools.referencing.wkt.Parser.parseCoordinateReferenceSystem(Parser.java:204)
I found out that SI.METER is linked to javax.measure but even if I add it to the project, the error is still present.
The result of mvn dependency:tree is
[INFO] +- com.vividsolutions:jts:jar:1.13:compile
[INFO] +- org.opengis:geoapi:jar:3.0.0:compile
[INFO] | \- javax.measure:jsr-275:jar:0.9.3:compile
[INFO] +- org.geotools:gt-data:jar:15.2:compile
[INFO] | +- org.geotools:gt-main:jar:15.2:compile
[INFO] | | \- org.geotools:gt-api:jar:15.2:compile
[INFO] | \- javax.media:jai_core:jar:1.1.3:compile
[INFO] +- org.geotools:gt-shapefile:jar:15.2:compile
[INFO] | \- org.jdom:jdom:jar:1.1.3:compile
[INFO] \- org.geotools:gt-swing:jar:15.2:compile
[INFO] +- org.geotools:gt-referencing:jar:15.2:compile
[INFO] | +- com.googlecode.efficient-java-matrix-library:core:jar:0.26:compile
[INFO] | +- commons-pool:commons-pool:jar:1.5.4:compile
[INFO] | +- org.geotools:gt-metadata:jar:15.2:compile
[INFO] | | \- org.geotools:gt-opengis:jar:15.2:compile
[INFO] | | \- net.java.dev.jsr-275:jsr-275:jar:1.0-beta-2:compile
[INFO] | +- jgridshift:jgridshift:jar:1.0:compile
[INFO] | \- net.sf.geographiclib:GeographicLib-Java:jar:1.44:compile
[INFO] +- org.geotools:gt-render:jar:15.2:compile
[INFO] | +- org.geotools:gt-coverage:jar:15.2:compile
[INFO] | | +- javax.media:jai_imageio:jar:1.1:compile
[INFO] | | +- it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15:compile
[INFO] | | | +- it.geosolutions.imageio-ext:imageio-ext-utilities:jar:1.1.15:compile
[INFO] | | | +- it.geosolutions.imageio-ext:imageio-ext-geocore:jar:1.1.15:compile
[INFO] | | | | \- it.geosolutions.imageio-ext:imageio-ext-streams:jar:1.1.15:compile
[INFO] | | | \- javax.media:jai_codec:jar:1.1.3:compile
[INFO] | | +- org.jaitools:jt-zonalstats:jar:1.4.0:compile
[INFO] | | +- org.jaitools:jt-utils:jar:1.4.0:compile
[INFO] | | +- it.geosolutions.jaiext.affine:jt-affine:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.algebra:jt-algebra:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandmerge:jt-bandmerge:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandselect:jt-bandselect:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.bandcombine:jt-bandcombine:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.border:jt-border:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.buffer:jt-buffer:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.crop:jt-crop:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.iterators:jt-iterators:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.lookup:jt-lookup:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.mosaic:jt-mosaic:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.nullop:jt-nullop:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.rescale:jt-rescale:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.scale:jt-scale:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.stats:jt-stats:jar:1.0.11:compile
[INFO] | | | \- com.google.guava:guava:jar:17.0:compile
[INFO] | | +- it.geosolutions.jaiext.translate:jt-translate:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.utilities:jt-utilities:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.warp:jt-warp:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.zonal:jt-zonal:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.binarize:jt-binarize:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.format:jt-format:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.colorconvert:jt-colorconvert:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.errordiffusion:jt-errordiffusion:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.orderdither:jt-orderdither:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.colorindexer:jt-colorindexer:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.imagefunction:jt-imagefunction:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.piecewise:jt-piecewise:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.classifier:jt-classifier:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.rlookup:jt-rlookup:jar:1.0.11:compile
[INFO] | | +- it.geosolutions.jaiext.vectorbin:jt-vectorbin:jar:1.0.11:compile
[INFO] | | \- commons-io:commons-io:jar:2.1:compile
[INFO] | \- org.geotools:gt-cql:jar:15.2:compile
[INFO] \- com.miglayout:miglayout:jar:swing:3.7:compile
Am I missing a dependency ?
The answer was found with the help of #IanTurton
I upgraded the geotools version from 15.2 to 20.5. I also added the gt-epsg-hsql jar. The pom.xml is now like this :
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-api</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-shapefile</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>20.5</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-epsg-hsql</artifactId>
<version>20.5</version>
</dependency>
I also updated my code to use org.locationtech.jts.geom.Geometry instead of com.vividsolutions

java.lang.NoClassDefFoundError: com/amazonaws/services/s3/AmazonS3Client in Spark

I have a java spark process which I call using -
spark-submit --class MyClass target/MyJar.jar
The last part of this process is to write it locally and then copy it to s3 as it need to have specific name (can also be written to s3 and mv'ed there, for the sake of the question the error stays the same).
The code compiles and runs but when it get to the code snippet below I get the following error -
java.lang.NoClassDefFoundError: com/amazonaws/services/s3/AmazonS3 at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:348) at
org.apache.spark.util.Utils$.classForName(Utils.scala:229) at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:700)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused
by: java.lang.ClassNotFoundException:
com.amazonaws.services.s3.AmazonS3 at
java.net.URLClassLoader.findClass(URLClassLoader.java:381) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
public static void saveToS3(Dataset<Row> df, String outputBucket, String outputPath) throws IOException {
String tmpFile = "temp" + Long.toString(System.nanoTime());
df.coalesce(1).write().option("header", true).csv(tmpFile);
File directory = new File(tmpFile);
AmazonS3 s3client = new AmazonS3Client();
for (File file : directory.listFiles()) {
if (file.getName().startsWith("part_") || file.getName().endsWith("csv")) {
s3client.putObject(new PutObjectRequest(outputBucket, getS3path(outputPath), file));
}
file.delete();
}
directory.delete();
}
I use the following dependencies which might be relevant
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.7.4</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>2.7.2</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api-2.5</artifactId>
</exclusion>
</exclusions>
</dependency>
Any idea?
Dependency tree -
[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
[INFO] | | +- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile
[INFO] | | \- com.esotericsoftware:kryo-shaded:jar:3.0.3:compile
[INFO] | | +- com.esotericsoftware:minlog:jar:1.3.0:compile
[INFO] | | \- org.objenesis:objenesis:jar:2.1:compile
[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile
[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile
[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0:compile
[INFO] | | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0:compile
[INFO] | | | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.2.0:compile
[INFO] | | | | | \- com.google.inject:guice:jar:3.0:compile
[INFO] | | | | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | | | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0:compile
[INFO] | | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0:compile
[INFO] | | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0:compile
[INFO] | | \- org.apache.hadoop:hadoop-annotations:jar:2.2.0:compile
[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile
[INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
[INFO] | | \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile
[INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:compile
[INFO] | | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- com.google.guava:guava:jar:14.0.1:compile
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.4.1:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.16:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile
[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile
[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile
[INFO] | +- commons-net:commons-net:jar:2.2:compile
[INFO] | +- org.scala-lang:scala-library:jar:2.11.8:compile
[INFO] | +- org.json4s:json4s-jackson_2.11:jar:3.2.11:compile
[INFO] | | \- org.json4s:json4s-core_2.11:jar:3.2.11:compile
[INFO] | | +- org.json4s:json4s-ast_2.11:jar:3.2.11:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | | \- org.scala-lang:scalap:jar:2.11.0:compile
[INFO] | | \- org.scala-lang:scala-compiler:jar:2.11.0:compile
[INFO] | | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.1:compile
[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile
[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile
[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile
[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile
[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile
[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile
[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile
[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile
[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile
[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile
[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile
[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile
[INFO] | +- io.netty:netty-all:jar:4.0.42.Final:compile
[INFO] | +- io.netty:netty:jar:3.8.0.Final:compile
[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile
[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile
[INFO] | | +- org.scala-lang:scala-reflect:jar:2.11.7:compile
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile
[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile
[INFO] | +- oro:oro:jar:2.0.8:compile
[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile
[INFO] | +- net.sf.py4j:py4j:jar:0.10.4:compile
[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile
[INFO] | | \- org.scalatest:scalatest_2.11:jar:2.2.6:compile
[INFO] | | \- org.scala-lang.modules:scala-xml_2.11:jar:1.0.2:compile
[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile
[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile
[INFO] +- org.apache.hadoop:hadoop-aws:jar:2.7.2:compile
[INFO] | \- org.apache.hadoop:hadoop-common:jar:2.7.2:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.7.2:compile
[INFO] | | \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile
[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.spark:spark-sql_2.10:jar:2.1.0:compile
[INFO] | +- com.univocity:univocity-parsers:jar:2.2.1:compile
[INFO] | +- org.apache.spark:spark-sketch_2.10:jar:2.1.0:compile
[INFO] | +- org.apache.spark:spark-core_2.10:jar:2.1.0:compile
[INFO] | | +- com.twitter:chill_2.10:jar:0.8.0:compile
[INFO] | | +- org.apache.spark:spark-launcher_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-network-common_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-network-shuffle_2.10:jar:2.1.0:compile
[INFO] | | +- org.apache.spark:spark-unsafe_2.10:jar:2.1.0:compile
[INFO] | | +- org.json4s:json4s-jackson_2.10:jar:3.2.11:compile
[INFO] | | | \- org.json4s:json4s-core_2.10:jar:3.2.11:compile
[INFO] | | | \- org.json4s:json4s-ast_2.10:jar:3.2.11:compile
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.6.5:compile
[INFO] | +- org.apache.spark:spark-catalyst_2.10:jar:2.1.0:compile
[INFO] | | +- org.codehaus.janino:janino:jar:3.0.0:compile
[INFO] | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile
[INFO] | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile
[INFO] | +- org.apache.spark:spark-tags_2.10:jar:2.1.0:compile
[INFO] | | \- org.scalatest:scalatest_2.10:jar:2.2.6:compile
[INFO] | +- org.apache.parquet:parquet-column:jar:1.8.1:compile
[INFO] | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile
[INFO] | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile
[INFO] | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile
[INFO] | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile
[INFO] | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile
[INFO] +- com.amazonaws:aws-java-sdk:jar:1.7.4:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2:compile
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.3:compile
[INFO] | \- joda-time:joda-time:jar:2.9.9:compile (version selected from constraint [2.2,))
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.7.3:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.3:compile
[INFO] \- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.3:compile
You appear to be using a really old version of AWS SDK (1.7.4) from 2014. Your apache spark is 2.1, which is from 2016.
Commit history for the AmazonS3Client only goes back to 1.9, so it very well may not exist in your AWS SDK.
The reason your code compiles is that your code itself isn't the problem, but one of your dependencies is trying to use a more recent version of S3 client that your old SDK doesn't have. In other words, this isn't a code problem, it's a dependency management problem. If you're not directly using the SDK at all, it's safe to use a more appropriate AWS SDK 1.11.X, if you are, you'll have to coax your code to that point.
Typically, the spark-core module would point out its dependency to a version of aws, but I assume it was specifically excluded from its own so you could provide your own version of the SDK for your use without any sort of conflict, with the caveat that if you use too old of a dependency, it just breaks.
After massaging my pom file I was able to get to a stable with the following pom file and packaging with -
mvn clean compile assembly:single;
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.group</groupId>
<artifactId>arifact</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<jdk.version>1.8</jdk.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>2.8.2</version>
</dependency>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.11.229</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.group.artifact.MainClass</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
</project>

AbstractMethodError javax.ws.rs.core.UriBuilder

I know that this error kinda implicates dependency/runtime issues yet I can't seem to find the offending dependencies. As from my stackoverflow/google research this happens if you mix jersey versions or using jax-rs 2.0 with jersey 1.0 but all my jersey dependencies are 2.22.1 and I can see in the referenced libraries that jax-rs is used in version 2.0.1.
The maven dependencies I'm using are:
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-server</artifactId>
<version>2.22.1</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-common</artifactId>
<version>2.22.1</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>9.3.7.v20160115</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
<version>9.3.7.v20160115</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet</artifactId>
<version>2.22.1</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-http</artifactId>
<version>9.3.7.v20160115</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-jetty-http</artifactId>
<version>2.22.1</version>
</dependency>
The Java code used (if the code matters) (basically taken from an online tutorial):
ServletContextHandler context = new ServletContextHandler(
ServletContextHandler.SESSIONS);
context.setContextPath("/");
Server jettyServer = new Server(9090);
jettyServer.setHandler(context);
ServletHolder jerseyServlet = context.addServlet(
org.glassfish.jersey.servlet.ServletContainer.class, "/*");
jerseyServlet.setInitOrder(0);
// Tells the Jersey Servlet which REST service/class to load.
jerseyServlet.setInitParameter(
"jersey.config.server.provider.classnames",
EntryPoint.class.getCanonicalName());
try {
jettyServer.start();
jettyServer.join();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
jettyServer.destroy();
}
Trace of mvn dependency:tree:
[INFO]
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) # hadoopy ---
[INFO] ch.foob.mumi:hadoopy:jar:0.0
[INFO] +- junit:junit:jar:3.8.1:test
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.4:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | +- io.netty:netty-all:jar:4.0.23.Final:compile
[INFO] | +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] | | \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system
[INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- commons-net:commons-net:jar:3.1:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:compile
[INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.1.2:compile
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.1.2:compile
[INFO] | | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.10:compile
[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] | | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.7.1:compile
[INFO] | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | | \- org.apache.curator:curator-framework:jar:2.7.1:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.7.1:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] | | +- org.slf4j:slf4j-log4j12:jar:1.6.1:compile
[INFO] | | \- io.netty:netty:jar:3.7.0.Final:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.glassfish.jersey.core:jersey-server:jar:2.22.1:compile
[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.1:compile
[INFO] | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.1:compile
[INFO] | +- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b31:compile
[INFO] | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b31:compile
[INFO] | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b31:compile
[INFO] | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b31:compile
[INFO] | +- org.glassfish.hk2:hk2-locator:jar:2.4.0-b31:compile
[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile
[INFO] | \- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] +- org.glassfish.jersey.core:jersey-common:jar:2.22.1:compile
[INFO] | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.1:compile
[INFO] | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] +- org.eclipse.jetty:jetty-server:jar:9.3.7.v20160115:compile
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | \- org.eclipse.jetty:jetty-io:jar:9.3.7.v20160115:compile
[INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.3.7.v20160115:compile
[INFO] | \- org.eclipse.jetty:jetty-security:jar:9.3.7.v20160115:compile
[INFO] +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.1:compile
[INFO] | \- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.1:compile
[INFO] +- org.eclipse.jetty:jetty-http:jar:9.3.7.v20160115:compile
[INFO] | \- org.eclipse.jetty:jetty-util:jar:9.3.7.v20160115:compile
[INFO] \- org.glassfish.jersey.containers:jersey-container-jetty-http:jar:2.22.1:compile
[INFO] \- org.eclipse.jetty:jetty-continuation:jar:9.1.1.v20140108:compile

Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder

My local environment: OSX 10.9.2, java1.6
I use java api to connect hbase and maven to manage my project, I added Hbase-0.94.17 and Hadoop-core-1.0.4 into pom.xml, when I ran my .java program to connect Hbase I got the following error:
SLF4J: slf4j-api 1.6.x (or later) is incompatible with this binding.
SLF4J: Your binding is version 1.5.5 or earlier.
SLF4J: Upgrade your binding to version 1.6.x.
Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder;
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:128)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:107)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:295)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:269)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:281)
at org.apache.zookeeper.ZooKeeper.<clinit>(ZooKeeper.java:94)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:98)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:127)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:153)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:127)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1505)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:713)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:983)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:155)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:129)
at org.geogit.storage.hbase.MyLittleHBaseClient.main(MyLittleHBaseClient.java:29)
Here is what I got after ran mvm dependency:tree
INFO] --- maven-dependency-plugin:2.2:tree (default-cli) # geogit-hbase ---
[INFO] org.geogit:geogit-hbase:jar:0.8-SNAPSHOT
[INFO] +- org.geogit:geogit-core:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
[INFO] | +- com.vividsolutions:jts:jar:1.13:compile
[INFO] | +- org.geotools:gt-opengis:jar:10.5:compile
[INFO] | | +- net.java.dev.jsr-275:jsr-275:jar:1.0-beta-2:compile
[INFO] | | +- java3d:vecmath:jar:1.3.2:compile
[INFO] | | +- commons-pool:commons-pool:jar:1.5.4:compile
[INFO] | | \- javax.media:jai_core:jar:1.1.3:compile
[INFO] | +- org.geotools:gt-referencing:jar:10.5:compile
[INFO] | | +- org.geotools:gt-metadata:jar:10.5:compile
[INFO] | | \- jgridshift:jgridshift:jar:1.0:compile
[INFO] | +- org.geotools:gt-epsg-hsql:jar:10.5:compile
[INFO] | | \- hsqldb:hsqldb:jar:1.8.0.7:compile
[INFO] | +- org.geotools:gt-main:jar:10.5:compile
[INFO] | | \- org.geotools:gt-api:jar:10.5:compile
[INFO] | +- org.geotools:gt-cql:jar:10.5:compile
[INFO] | +- net.sourceforge.findbugs:jsr305:jar:1.3.7:compile
[INFO] | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | +- com.google.inject.extensions:guice-multibindings:jar:3.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.2:compile
[INFO] | \- com.ning:compress-lzf:jar:0.9.8:compile
[INFO] +- org.apache.hbase:hbase-client:jar:0.98.0-hadoop2:compile
[INFO] | +- org.apache.hbase:hbase-common:jar:0.98.0-hadoop2:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | \- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.apache.hbase:hbase-protocol:jar:0.98.0-hadoop2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.7:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- io.netty:netty:jar:3.6.6.Final:compile
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- org.slf4j:slf4j-log4j12:jar:1.6.1:compile
[INFO] | +- org.cloudera.htrace:htrace-core:jar:2.04:compile
[INFO] | | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] | | \- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.2.0:compile
[INFO] | | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | | +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | | +- org.codehaus.jettison:jettison:jar:1.0.1:compile (version managed from 1.1)
[INFO] | | | | \- stax:stax-api:jar:1.0.1:compile
[INFO] | | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO] | | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] | | +- commons-el:commons-el:jar:1.0:runtime
[INFO] | | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] | | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] | | | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] | | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | | \- org.tukaani:xz:jar:1.0:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.2.0:compile
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-yarn-common:jar:2.2.0:compile
[INFO] | | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.2.0:compile
[INFO] | | | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | | | \- asm:asm:jar:3.1:compile
[INFO] | | | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile
[INFO] | | \- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.2.0:compile
[INFO] | \- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] +- org.geogit:geogit-mongodb:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.geogit:geogit-blueprints:jar:0.8-SNAPSHOT:compile
[INFO] | | +- com.tinkerpop.blueprints:blueprints-core:jar:2.4.0:compile
[INFO] | | | +- com.fasterxml.jackson.datatype:jackson-datatype-json-org:jar:2.1.2:compile
[INFO] | | | | +- com.fasterxml.jackson.core:jackson-core:jar:2.1.2:compile
[INFO] | | | | +- com.fasterxml.jackson.core:jackson-databind:jar:2.1.2:compile
[INFO] | | | | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.1.1:compile
[INFO] | | | | \- org.json:json:jar:20090211:compile
[INFO] | | | \- colt:colt:jar:1.2.0:compile
[INFO] | | | \- concurrent:concurrent:jar:1.3.4:compile
[INFO] | | \- com.tinkerpop.gremlin:gremlin-java:jar:2.4.0:compile
[INFO] | | \- com.tinkerpop:pipes:jar:2.4.0:compile
[INFO] | \- org.mongodb:mongo-java-driver:jar:2.11.3:compile
[INFO] +- com.google.guava:guava:jar:14.0.1:compile
[INFO] +- org.geogit:geogit-core:jar:tests:0.8-SNAPSHOT:test
[INFO] \- junit:junit:jar:4.10:test
[INFO] \- org.hamcrest:hamcrest-core:jar:1.1:test
And here I attach my pom.xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.geogit</groupId>
<artifactId>storage</artifactId>
<version>0.8-SNAPSHOT</version>
</parent>
<groupId>org.geogit</groupId>
<artifactId>geogit-hbase</artifactId>
<packaging>jar</packaging>
<name>HBase Storage for GeoGit objects</name>
<dependencies>
<dependency>
<groupId>org.geogit</groupId>
<artifactId>geogit-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.94.17</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.0.4</version>
</dependency>
<!-- is this necessary? -->
<!--
<dependency>
<groupId>com.boundlessgeo</groupId>
<artifactId>blongo</artifactId>
<version>0.1</version>
</dependency>
-->
<!-- is this necessary? -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<!-- Test scope dependencies -->
<dependency>
<groupId>org.geogit</groupId>
<artifactId>geogit-core</artifactId>
<version>${project.version}</version>
<scope>test</scope>
<classifier>tests</classifier>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
</profiles>
</project>
Have slf4j-api and choose a SLF4J binding of corresponding version, and add them as dependencies in your own POM. It is quite possibly some of your dependencies transitively resolve an incompatible version of SLF4J binding which caused the problem. Find which library is bringing in the unwanted SLF4J binding (you can easily do so by mvn dependency:tree), and add corresponding exclusion in your dependency to that lib.
(Normally SLF4J binding shouldn't be declared in library. It is in fact something wrong that you may want to report to the library developer once you found the actual cause)
Update
That's exactly what I am talking about:
From your dependency tree:
[INFO] +- org.geogit:geogit-core:jar:0.8-SNAPSHOT:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
.....
[INFO] +- org.apache.hbase:hbase-client:jar:0.98.0-hadoop2:compile
.....
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] | | \- org.slf4j:slf4j-log4j12:jar:1.6.1:compile
You got slf4j-log4j12 binding from hbase-client, and that's version 1.6.1
However, slf4j-api you have in your project is version 1.7.5. They are not compatible.
Proper way to solve is:
Exclude slf4j-log4j12 from your dependency of hbase-client
Declare a compatible SLF4J binding in your own project
You should also report this as a bug to zookeeper/hbase developer as it is not appropriate for a library to include SLF4J binding as compile scope dependency.

Categories