We Keep Coding

Resolve guava conflict between spark core and azure key vault dependency

I am trying out a spark application in java which needs to connect to Azure Key-vault and fetch some secrets and process data after reading from a blob storage and push to a SQL Server table. Here is my pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.my.package</groupId>
<artifactId>hdinsightSample</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>hdinsightSample</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>2.4.7</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.12</artifactId>
<version>2.4.7</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-azure</artifactId>
<version>2.10.1</version>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.2.2.jre8</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-keyvault</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure</artifactId>
<version>1.3.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.my.package.hdinsightSample.App</mainClass>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/maven/**</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<relocations>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>repackaged.com.google.common</shadedPattern>
<includes>
<include>com.google.guava:*</include>
</includes>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
After running the jar file or running the code from Eclipse I get the following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: com.google.common.reflect.TypeToken
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:379)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:682)
at java.net.URLClassLoader.access$400(URLClassLoader.java:89)
at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1086)
at java.security.AccessController.doPrivileged(AccessController.java:739)
at java.net.URLClassLoader.findClass(URLClassLoader.java:589)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934)
at java.lang.ClassLoader.loadClass(ClassLoader.java:879)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:862)
at com.microsoft.azure.keyvault.KeyVaultClientImpl.getSecretDelegate(KeyVaultClientImpl.java:2938)
at com.microsoft.azure.keyvault.KeyVaultClientImpl.access$1800(KeyVaultClientImpl.java:92)
at com.microsoft.azure.keyvault.KeyVaultClientImpl$82.call(KeyVaultClientImpl.java:2928)
at com.microsoft.azure.keyvault.KeyVaultClientImpl$82.call(KeyVaultClientImpl.java:2924)
at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
at retrofit2.adapter.rxjava.CallArbiter.deliverResponse(CallArbiter.java:120)
at retrofit2.adapter.rxjava.CallArbiter.emitResponse(CallArbiter.java:102)
at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:46)
at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:24)
at rx.Observable.unsafeSubscribe(Observable.java:10142)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.subscribe(Observable.java:10238)
at rx.Observable.subscribe(Observable.java:10205)
at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:444)
at rx.observables.BlockingObservable.single(BlockingObservable.java:341)
at com.microsoft.azure.keyvault.KeyVaultClientImpl.getSecret(KeyVaultClientImpl.java:2868)
at com.microsoft.azure.keyvault.KeyVaultClient.getSecret(KeyVaultClient.java:862)
at com.my.package.hdsinsightSample.utils.KeyVaultUtils.getSecret(KeyVaultUtils.java:12)
at com.my.package.hdinsightSample.App.main(App.java:23)
Caused by: java.lang.ClassNotFoundException: com.google.common.reflect.TypeToken
at java.net.URLClassLoader.findClass(URLClassLoader.java:591)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934)
at java.lang.ClassLoader.loadClass(ClassLoader.java:879)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:862)
... 36 more
This is because to fetch the secret from key vault it uses guava jar but guava jar complied by default is 11.0.2 (present in spark-core dependecny) whereas the correct version it should be using is 20 (present in the Azure Key-Vault dependency). I tried shading the jar as shown in the pom above but the issue remains the same.
Kindly help.

I created a small project with the same dependencies and the dependency tree shows the Guava dependency actually comes from org.apache.hadoop:hadoop-azure:jar:2.10.1. Regardless of that, the simplest way to get rid of this problem is to explicitly state which Guava version you want your project to use in your POM (ideally the same as the one in Key Vault in this case):
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>
Additionally, there is a new set of libraries for Azure Key Vault with improved APIs and additional functionality, which also do not rely on Guava, so that is an alternative approach you can take if you also want to update your code base :)
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-security-keyvault-certificates</artifactId>
<version>4.1.3/version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-security-keyvault-keys/artifactId>
<version>4.2.3/version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-security-keyvault-secrets</artifactId>
<version>4.2.3/version>
</dependency>

Can't find class of postgresql driver

I'm trying to use HikariCP library for PostgreSQL connection pools in Java. I am using Maven, and I'm getting this error: java.lang.ClassNotFoundException: org.postgresql.Driver.
I have tried using different versions of the PostgreSQL driver, but none have worked to my advantage. (I have done more, but I've been faced with this problem, that I have not taken note of)
org.postgresql.ds.PGSimpleDataSource, org.postgresql.Driver and com.impossibl.postgres.jdbc.PGDataSource still produce this error, even though it is said in the HikariCP guide to use either the first or third. The second I found from research.
My maven:
(...)
<build>
<defaultGoal>clean package</defaultGoal>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<minimizeJar>true</minimizeJar>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<repositories>
<repository>
<id>spigotmc-repo</id>
<url>https://hub.spigotmc.org/nexus/content/groups/public/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.spigotmc</groupId>
<artifactId>spigot-api</artifactId>
<version>1.12.2-R0.1-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
<version>3.3.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.6.4</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.2.5</version>
</dependency>
</dependencies>
</project>
What I'm using to produce this error:
try {
Class.forName("org.postgresql.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.out.println("Failed to load.");
}
HikariConfig config = new HikariConfig("database.properties");
ds = new HikariDataSource(config);
(The constructor HikariDataSource(config); produces this error aswell)
I believe the reason why this is happening is that the driver is not being made into the classpath - however, all efforts I have tried can't seem to do this. This problem of the driver not being in the final jar (to my belief):
.
Actual error:
[20:16:35 WARN]: java.lang.ClassNotFoundException: org.postgresql.Driver
[20:16:35 WARN]: at java.net.URLClassLoader.findClass(Unknown Source)
[20:16:35 WARN]: at org.bukkit.plugin.java.PluginClassLoader.findClass(PluginClassLoader.java:152)
[20:16:35 WARN]: at org.bukkit.plugin.java.PluginClassLoader.findClass(PluginClassLoader.java:100)
[20:16:35 WARN]: at java.lang.ClassLoader.loadClass(Unknown Source)
[20:16:35 WARN]: at java.lang.ClassLoader.loadClass(Unknown Source)
[20:16:35 WARN]: at java.lang.Class.forName0(Native Method)
[20:16:35 WARN]: at java.lang.Class.forName(Unknown Source)
[20:16:35 WARN]: at me.test.kitpvp.Kitpvp.onEnable(Kitpvp.java:43)
[20:16:35 WARN]: at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
[20:16:35 WARN]: at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:337)
(...)
[20:16:35 INFO]: Failed to load.
[20:16:35 WARN]: 74 [Server thread] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
Edits
Adding <scope>compile</scope> produced same results - did not work.

You can specify a filter to include the driver artifact in the shaded JAR:
<configuration>
<minimizeJar>true</minimizeJar>
<filters>
<filter>
<artifact>org.postgresql:postgresql</artifact>
<includes>
<include>**</include>
</includes>
</filter>
</filters>
For more details: Selecting Contents for Uber JAR.
The issue occurs due to the (unfiltered) <minimizeJar>true</minimizeJar> configuration in your POM, and the fact that the driver class is not used at compile time.
maven-shade-plugin on minimizeJar:
When true, dependencies will be stripped down on the class level to only the transitive hull required for the artifact.

maven with bouncycastle jar error

I am using the following dependecy in my pom.xml from maven:
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.55</version>
</dependency>
After running the project inside eclipse everything is working fine.
By using the export feature from eclipse, creating a runnable jar file, including all dependencies also everything works fine.
No other parameters or configurations in eclipse are set.
Only when i am running a mvn built something seems to be wrong.
(Maven built runs without any error or warnings)
--> Dependencies are all included.
An error occurs in that moment when it tries to establish an https connection, the place where this bouncycastle library is used.
Error:
javax.net.ssl.SSLException: java.security.ProviderException: Could not derive key
at sun.security.ssl.Alerts.getSSLException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:553)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:412)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at plugin.KITIlias.executePost(KITIlias.java:117)
at plugin.KITIlias.login(KITIlias.java:45)
at control.IliasManager.login(IliasManager.java:27)
at control.IliasStarter.login(IliasStarter.java:31)
at control.LoginProvider$1.run(LoginProvider.java:56)
at java.lang.Thread.run(Unknown Source)
Caused by: java.security.ProviderException: Could not derive key
at sun.security.ec.ECDHKeyAgreement.engineGenerateSecret(ECDHKeyAgreement.java:133)
at sun.security.ec.ECDHKeyAgreement.engineGenerateSecret(ECDHKeyAgreement.java:163)
at javax.crypto.KeyAgreement.generateSecret(KeyAgreement.java:648)
at sun.security.ssl.ECDHCrypt.getAgreedSecret(Unknown Source)
at sun.security.ssl.ClientHandshaker.serverHelloDone(Unknown Source)
at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
at sun.security.ssl.Handshaker.processLoop(Unknown Source)
at sun.security.ssl.Handshaker.process_record(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
... 16 more
Caused by: java.security.InvalidAlgorithmParameterException
at sun.security.ec.ECDHKeyAgreement.deriveKey(Native Method)
at sun.security.ec.ECDHKeyAgreement.engineGenerateSecret(ECDHKeyAgreement.java:130)
... 25 more
Exception in thread "Thread-4" java.lang.NullPointerException
at plugin.KITIlias.executePost(KITIlias.java:125)
at plugin.KITIlias.login(KITIlias.java:45)
at control.IliasManager.login(IliasManager.java:27)
at control.IliasStarter.login(IliasStarter.java:31)
at control.LoginProvider$1.run(LoginProvider.java:56)
at java.lang.Thread.run(Unknown Source)
So for me it seems like something is missing during the maven built process.
Is it required to sign one of the jars?
my complete pom.xml is
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>X</groupId>
<artifactId>Y</artifactId>
<version>v1.1.0</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.7.2</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.5</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.controlsfx</groupId>
<artifactId>controlsfx</artifactId>
<version>8.20.8</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.55</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>view.Dashboard</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
I am getting the runnable jar with:
mvn clean install
EDIT:
Manifest file from eclipse export
Manifest-Version: 1.0
Rsrc-Class-Path: ./ junit-4.10.jar hamcrest-core-1.1.jar log4j-1.2.17.
jar jsoup-1.7.2.jar httpcore-4.4.5.jar httpclient-4.5.2.jar commons-c
odec-1.9.jar commons-logging-1.1.1.jar controlsfx-8.20.8.jar openjfx-
dialogs-1.0.2.jar bcprov-jdk15on-1.55.jar
Class-Path: .
Rsrc-Main-Class: view.Dashboard
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
-> seems like there are still dependencies to eclipse

Problem seems to be here that the dependency from bouncycastle must be signed. The jar is not signed anymore if maven repacks it inside another jar.
Solution is to keep all dependecies next to the jar file and add a classpathdirectory. Now the main jar can use directly the signed jar.
(However i would really prefer to have one single jar file)

Elasticsearch - Search fails after upgrading to 2.1.1

I am working on a Java EE 7 WebApplication (running in wildfly 10) using an embedded Elasticsearch.
After upgrading to elasticsearch 2.1.1 searching is not working anymore. (indexing seems to work)
Dependencies that worked:
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>5.2.1</version>
</dependency>
Upgraded dependencies:
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>5.3.1</version>
</dependency>
After indexing a very basic document (indexing seems to work), searching fails with the following exception:
org.elasticsearch.action.search.SearchPhaseExecutionException : all shards failed [Proxied because : Original exception caused: class java.io.NotSerializableException: org.elasticsearch.action.search.ShardSearchFailure]
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:228)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.onFailure(TransportSearchTypeAction.java:174)
at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:46)
at org.elasticsearch.transport.local.LocalTransport.handleException(LocalTransport.java:354)
at org.elasticsearch.transport.local.LocalTransport.handlerResponseError(LocalTransport.java:345)
at org.elasticsearch.transport.local.LocalTransport.messageReceived(LocalTransport.java:241)
at org.elasticsearch.transport.local.LocalTransportChannel$2.run(LocalTransportChannel.java:99)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: _byteSymbolCanonicalizer
at com.fasterxml.jackson.dataformat.smile.SmileFactory._createParser(SmileFactory.java:404)
at com.fasterxml.jackson.dataformat.smile.SmileFactory.createParser(SmileFactory.java:327)
at org.elasticsearch.common.xcontent.smile.SmileXContent.createParser(SmileXContent.java:107)
at org.elasticsearch.common.xcontent.smile.SmileXContent.createParser(SmileXContent.java:113)
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:817)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:651)
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:617)
at org.elasticsearch.search.SearchService.executeDfsPhase(SearchService.java:262)
at org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:360)
at org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:357)
at org.elasticsearch.transport.local.LocalTransport$2.doRun(LocalTransport.java:280)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
This is how my search looks like:
QueryStringQueryBuilder qb = QueryBuilders.queryStringQuery(queryString);
SearchResponse searchResponse = this.client.prepareSearch(INDEX_NAME)
.setTypes(typeName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(qb)
.setFrom(offset)
.setSize(limit)
.execute()
.actionGet();

I got around that problem by shading elasticsearch dependencies
1. I created a separate maven-module:
<project>
...
<dependencies>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.3.4</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.fasterxml.jackson</pattern>
<shadedPattern>shaded.elasticsearch.jackson</shadedPattern>
</relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" />
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
...
</project>
2. And then used that module as a dependency in my main project
<dependency>
<groupId>drivve</groupId>
<artifactId>elasticsearch-shaded</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>5.5.0</version><!-- be sure to match the version required by elasticsearch-shaded -->
</dependency>
Everything works fine now and is not that hard to managed.
More info on shading es dependencies can be found here.

Maven Spring Jar project java.lang.ClassNotFoundExceptiona

I have a strange problem with my Spring application. I use Eclipse and when i click RUn as Java Application my App works like a charm, but when i click maven build (mvn install) after packing to jar and try to run i get:
Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/c
ontext/ApplicationContext
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
at java.lang.Class.getMethod0(Unknown Source)
at java.lang.Class.getMethod(Unknown Source)
at sun.launcher.LauncherHelper.getMainMethod(Unknown Source)
at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.Applica
tionContext
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 6 more
I gues that the problem is in this line:
public static void main(String[] args) {
ModernSoft ms = new ModernSoft();
StringBuffer classpath = new StringBuffer();
ClassLoader applicationClassLoader = ms.getClass().getClassLoader();
if (applicationClassLoader == null) {
applicationClassLoader = ClassLoader.getSystemClassLoader();
}
URL[] urls = ((URLClassLoader)applicationClassLoader).getURLs();
for(int i=0; i < urls.length; i++) {
classpath.append(urls[i].getFile()).append("\r\n");
}
System.out.println(classpath.toString());
ApplicationContext ctx = new ClassPathXmlApplicationContext("classpath*:**/applicationContext*.xml");
//org.springframework.context.ApplicationContext ctx = new FileSystemXmlApplicationContext("C:\\Documents and Settings\\BK\\workspace\\ModernSoft\\src\\main\\resources\\applicationContext.xml");
DataSource ds = (DataSource) ctx.getBean("dataSource", DataSource.class);
String[] beans = ctx.getBeanDefinitionNames();
I have all jar in maven dependencies, but still i cant solve this wired problem.
Please help me.
UPDATE:
My pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>ModernSoft</groupId>
<artifactId>pl.modern</artifactId>
<version>0.0.1</version>
<name>ModernSoft</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifestFile>src/main/resources/META-INF/MANIFEST.MF</manifestFile>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
<finalName>ModernSoft</finalName>
</build>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>4.2.5.Final</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>3.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.26</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>3.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.3.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.eclipse.persistence</groupId>
<artifactId>javax.persistence</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>3.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>3.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.2.0.Final</version>
</dependency>
</dependencies>
</project>
UPDATE
Uberjar works like a charm
Now i have another problem, after run jar file i get:
Exception in thread "main" org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Unable to lo
cate Spring NamespaceHandler for XML schema namespace [http://www.springframework.org/schema/data/jpa]
Offending resource: class path resource [applicationContext.xml]
at org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:68)
at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:85)
at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:80)
at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.error(BeanDefinitionParserDelegate.java:318)
at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1435)
at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1428)
at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions(DefaultBeanDefinitionDocumentR
eader.java:185)
at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions(DefaultBeanDefinitionDocu
mentReader.java:139)
at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions(DefaultBeanDefinitionDocume
ntReader.java:108)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registerBeanDefinitions(XmlBeanDefinitionReader.java:493)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:334)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:302)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:
174)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:
209)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:
180)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:
243)
at org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:127)
at org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:93)
at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationCo
ntext.java:130)
at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:537)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
at pl.modern.ModernSoft.main(ModernSoft.java:27)
And also i cant add or change entities.
After make customerDao.save(Customer) i get only in hibernate
Hibernate: select max(id) from customers
But no record in DB shows up.

I believe that you are running jar file from command line using java -jar In this case you will need to specify all jars which needs to be included in classpath using -classpath option.
Another option would be using Maven uber jar (jar which contains all depencies.)

The various spring jars each contain their own spring.handlers and spring.schemas files named identically, meaning the shader/uberjar plugins will pick one of them and other namespaces will fail to work.
See Invalid or corrupt JAR File built by Maven shade plugin for a shader plugin configuration merging the otherwise conflicting files. That example does have the configuration element in a weird location, my version is:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<transformers>
<!-- must be SURE to do this with both spring.handlers and spring.schemas.
otherwise you won't be able to use them in the spring config files. -->
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
</transformers>
</configuration>
</plugin>