ElasticSearch 2.4 : Works well in IDE, but not as JAR - java

ElasticSearch 2.4 client work well when running from Intelli Idea Java IDE. When same code run via jar file java -jar <jar-path> gives following error.
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{<remote-ip>}{<remote-ip>:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1226)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:86)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:56)
at com.creo.datawarehouse.es.op.imp.IndexOperationImp.createIndex(IndexOperationImp.java:66)
at com.creo.datawarehouse.es.ElasticSearch.initialise(ElasticSearch.java:27)
at com.creo.datawarehouse.script.App.main(App.java:37)
POM dependency:
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.4.0</version>
</dependency>
This question have not any answer and what is the jar version for 2.4.0 ES version in this question.
Please let me know the probable reason or solution for this.

Finally after hours of debugging, found its solution. Before mentioning those, quick key notes.
if els server is 2.4.1, maven jar should also be 2.4.1
Add log4j dependencies for more logs in details (found from this).
Add shade in plugin section.
Nothing to do with IDE (may run initially but to run prod, have to follow above).
But haven't got answer for why it was running in IDE?
Java Maven Dependency
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.4.1</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<build>
<plugins>
[---]
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.4</version>
<configuration>
<finalName>injector-2.4.1</finalName>
<appendAssemblyId>false</appendAssemblyId>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>org.elasticsearch.demo.workshop.injector.runner.Generate</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals><goal>shade</goal></goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.your.classss.App</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
[--]
</plugins>
</build>

Related

How to produce an executable jar file with all maven dependencies?

I have a maven project and I would like to produce the jar file with all dependencies.
I use package shade plugin and the command mvn package to produce the jar file.
However, the produced jar file does not consider any of the dependencies that is pom.xml. The produced jar file keep give me exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/Level
Here is the content of pom.xml:
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>myProject-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>myProject-parser</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>info.picocli</groupId>
<artifactId>picocli</artifactId>
<version>${picoli.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${shade.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/log4j/log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.13.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.13.1</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>client</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${shade.version}</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>myProject-client-${project.version}</finalName>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>myProject.package.main</mainClass>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
</transformers>
<filters>
<!-- This is necessary to avoid a java.lang.SecurityException -->
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<!-- Jar file entry point -->
<mainClass>myProject.package.main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
Where is my mistake?
It is even does not consider <mainClass>myProject.package.main</mainClass> for the main class when I run the jar I have to specify the class name:
java -cp myApp.jar myProject.package.main
I looked over most of the questions and nothing solve my issue.
First of all, there are two plugins that you can use to create fat jars: maven-shade-plugin and maven-assembly-plugin.
The main difference between them is that maven-assembly just builds a jar containing all the dependency, while maven-shade also relocate these dependencies internally so that, if another project is depending on your jar, you won't risk having conflicts on dependencies.
If you don't expect yo use this artifact as a dependency of other projects, you can use maven-assembly-plugin for it. Here is an example of how to use it:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.1</version>
<configuration>
<finalName>${project.artifactId}</finalName>
<archive>
<manifest>
<mainClass>myProject.package.main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
If you really need to shade your dependencies, try to modify the manifest config of the plugin with this, at least it should solve the problem that you-re having with identifying the main class:
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>myProject.package.main</Main-Class>
<X-Compile-Source-JDK>${maven.compiler.source}</X-Compile-Source-JDK>
<X-Compile-Target-JDK>${maven.compiler.target}</X-Compile-Target-JDK>
</manifestEntries>
</transformer>
There are no groupId in the dependencies with org.apache.log4j, you have: org.apache.logging.log4j but its not the same as: org.apache.log4j

AWS Lambda Jar unable to zip after adding selenium dependencies in pom

This is a weird error. After adding the selenium dependencies to the pom of my maven project and upload it to a lambda, it says it is unable to unzip the file. However after removing the dependencies, the lambda is able to unzip the file just fine (however it comes up with a class not found afterwards). I have tried removing the dependencies one by one but each one triggers the error.
Any ideas on how to solve this?
Class not found error
org/openqa/selenium/WebDriver: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver
lambda cannot zip error
Calling the invoke API action failed with this message: Lambda was not able to unzip the file
The dependencies causing the issue
<dependency>
<groupId>org.seleniumhq.webdriver</groupId>
<artifactId>webdriver-common</artifactId>
<version>0.9.7376</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-chrome-driver</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.141.59</version>
</dependency>
updated dependancies (for Vishal)
<dependency>
<groupId>org.seleniumhq.webdriver</groupId>
<artifactId>webdriver-common</artifactId>
<version>0.9.7376</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-chrome-driver</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-api</artifactId>
<version>2.0rc2</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-remote-driver</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-support</artifactId>
<version>3.141.59</version>
</dependency>
Configuration
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
<forceJavacCompilerUse>true</forceJavacCompilerUse>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The shade plugin combines all dependencies with the developed code and plops them in one Uber JAR. The downside is that it can overwrite resource files, and doesn't play well with signed jars (in my experience at least).
I would recommend moving away from the shade plugin if at all possible.
That said, if you have to use it - you're issue may be with the combining the jar resources. There are many transformers that you can use to resolve this, and you'll need to investigate which one is really needed. I would start with something like this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>${executable.classifier}</shadedClassifierName>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>fully.qualified.ClassName</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
You can find more tranformers on the Apache plugin here
The alternative that I would suggest is Spring Boot, which uses the Jar-in-Jar structure with a custom ClassLoader to load classes from the internal jar(s).
This is the easier method due to not having to re-write files as the Shade plugin approach and it handles the dependencies a little better.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.3.6.RELEASE</version>
<configuration>
<classifier>${executable.classifier}</classifier>
<layout>ZIP</layout>
<mainClass>fully.qualified.ClassName</mainClass>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
Seriously, look at the simpler configuration!
NOTE: Most of this came from my own notes - version numbers may be a little old...
Try to tell your dependencies to output zip, maybe jar is messing up with things
Add this to maven-assembly-plugin configuration:
<formats>
<format>zip</format>
</formats>
For example:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
...
<configuration>
...
<formats>
<format>zip</format>
</formats>
</configuration>
</plugin>
Same is suggested here
I figured it out. The java selenium seemed to cause the major issue. Downgrading to 3.10 fixed the issue although I have no idea why.

I am adding Jacoco Maven Plugin in POM file of Apache Tika to get the Code Coverage. But i am facing issues doing so

The Apache Tika Project has a root directory within which there is a pom.xml present. Inside the Root directory there are submodules and module named "Tika-Parent". Inside Parent module there is also a pom.xml file.
Now i am adding jacoco Maven Plugin to Tika to get Code Coverage but i am facing problems because when i add Jacoco Maven Plugin to the Pom.xml in Root Directory. It builds successfully but gives an error :"Skipping JaCoCo execution due to missing execution data file:C:\Users\adity\IdeaProjects\tika-1.20\target\jacoco.exec"
I tried adding JaCOCO Plugin to both the pom.xml files together as well as one by one each also but no result.
Also when I Perform Maven Test on the pom.xml in root directory it builds successfully running all tests.
BUT when I Perform Maven Test on the Tika-parent's pom.xml it doesnt run any tests but builds successfully.
I am attaching below the Pom.xml in root directory as well as pom.xml from Tika-parent module inside the root directory.
I am using IntelliJ Community edition ( latest version on website).
Java version - JDK 8 ,
Maven version 3.
THese details are from Root directory pom.xml
<modules>
<module>tika-parent</module>
<module>tika-core</module>
<module>tika-parsers</module>
<module>tika-bundle</module>
<module>tika-xmp</module>
<module>tika-serialization</module>
<module>tika-batch</module>
<module>tika-app</module>
<module>tika-server</module>
<module>tika-translate</module>
<module>tika-langdetect</module>
<module>tika-example</module>
<module>tika-java7</module>
<module>tika-eval</module>
<module>tika-dl</module>
<module>tika-nlp</module>
</modules>
These are the properties
<jacoco.version>0.7.5.201505241946</jacoco.version>
<junit.version>4.12</junit.version>
<junit.platform.version>1.3.1</junit.platform.version>
<maven.antrun.version>1.8</maven.antrun.version>
<maven.assembly.version>3.1.0</maven.assembly.version>
<maven.bundle.version>4.0.0</maven.bundle.version>
<maven.failsafe.version>2.22.1</maven.failsafe.version>
<maven.javadoc.version>3.0.1</maven.javadoc.version>
<maven.scr.version>1.7.4</maven.scr.version>
<maven.surefire.version>2.22.1</maven.surefire.version>
<maven.shade.version>3.2.0</maven.shade.version>
<rat.version>0.12</rat.version>
These are Dependencies
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<version>${junit.platform.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>5.3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.4.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit-pioneer</groupId>
<artifactId>junit-pioneer</artifactId>
<version>0.3.0</version>
<scope>test</scope>
</dependency>
</dependencies>
Below are all Plugins in Build.Plugins.
This is the JACOCO Plugin
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>target/jacoco.exec</dataFile>
<outputDirectory>target/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
This is the Surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
</configuration>
</plugin>
THis is apache Rat Plugin
<plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
<version>${rat.version}</version>
<configuration>
<excludes>
<exclude>CHANGES.txt</exclude>
</excludes>
</configuration>
</plugin>
Rest there are no other plugins in this pom.xml
BELOW is pom.xml from Iika-Parent module in root directory.
These are the properties.
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>${project.build.sourceEncoding}
</project.reporting.outputEncoding>
<!-- plugin versions -->
<jacoco.version>0.7.5.201505241946</jacoco.version>
<junit.version>4.12</junit.version>
<junit.platform.version>1.3.1</junit.platform.version>
<forbiddenapis.version>2.6</forbiddenapis.version>
<groovy.maven.version>2.0</groovy.maven.version>
<maven.antrun.version>1.8</maven.antrun.version>
<maven.assembly.version>3.1.0</maven.assembly.version>
<maven.bundle.version>4.0.0</maven.bundle.version>
<maven.failsafe.version>2.22.1</maven.failsafe.version>
<maven.javadoc.version>3.0.1</maven.javadoc.version>
<maven.scr.version>1.7.4</maven.scr.version>
<maven.surefire.version>2.22.1</maven.surefire.version>
<maven.shade.version>3.2.0</maven.shade.version>
<rat.version>0.12</rat.version>
<!-- NOTE: sync tukaani version with commons-compress in tika-parsers -->
<poi.version>4.0.1</poi.version>
<commons.compress.version>1.18</commons.compress.version>
<commons.io.version>2.6</commons.io.version>
<commons.lang3.version>3.8.1</commons.lang3.version>
<gson.version>2.8.5</gson.version>
<osgi.core.version>6.0.0</osgi.core.version>
<cxf.version>3.2.7</cxf.version>
<slf4j.version>1.7.25</slf4j.version>
<jackson.version>2.9.7</jackson.version>
<!-- when this is next upgraded, see if we can get rid of
javax.activation dependency in tika-server -->
<jaxb.version>2.3.1</jaxb.version>
<cli.version>1.4</cli.version>
<lucene.version>7.5.0</lucene.version>
<mockito.version>2.23.4</mockito.version>
Additional Dependencies in this pom.xml ( rest all are same as earlier pom)
<dependency>
<groupId>biz.aQute</groupId>
<artifactId>bndlib</artifactId>
<version>1.50.0</version>
</dependency>
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.scr.annotations</artifactId>
<version>1.12.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
Below are all extra plugins in Build.plugins( rest all are same as earlier pom.xml)
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>2.7</version>
<configuration>
<generateBackupPoms>false</generateBackupPoms>
</configuration>
</plugin>
<plugin>
<groupId>de.thetaphi</groupId>
<artifactId>forbiddenapis</artifactId>
<!-- if this version contains commons-io 2.6, remove hard-coded commons-io version below -->
<version>${forbiddenapis.version}</version>
<configuration>
<targetVersion>${maven.compiler.target}</targetVersion>
<failOnUnresolvableSignatures>false</failOnUnresolvableSignatures>
<failOnUnsupportedJava>false</failOnUnsupportedJava>
<bundledSignatures>
<bundledSignature>jdk-unsafe</bundledSignature>
<bundledSignature>jdk-deprecated</bundledSignature>
<bundledSignature>jdk-non-portable</bundledSignature>
<bundledSignature>jdk-internal</bundledSignature>
<bundledSignature>commons-io-unsafe-${commons.io.version}</bundledSignature>
</bundledSignatures>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
<goal>testCheck</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>${maven.bundle.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.version}</version> <!-- versions greater than this don't like System.exit calls in tika-batch -->
<configuration>
<argLine>-Xmx3072m -Duser.timezone=UTC</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven.shade.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>2.5.3</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.scm</groupId>
<artifactId>maven-scm-provider-gitexe</artifactId>
<version>1.9.5</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.0.0-M2</version>
<executions>
<execution>
<id>enforce</id>
<configuration>
<rules>
<dependencyConvergence />
<requireMavenVersion>
<version>3.1</version>
</requireMavenVersion>
</rules>
</configuration>
<goals>
<goal>enforce</goal>
</goals>
</execution>
</executions>
</plugin>
The Jacoco Plugin is present in this pom file too which is same as earlier pom file.
I tried placing in this file and building it. Didnt work. THen tried placing jacoco plugin in Root pom file , didnt work. Gives the same above mentionde error.
Please HELP!!
in my code i insert jacoco plugin in root pom under pluginManagement tag:
<pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.9</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
And in every sub-module where I do unit test I insert the following pom.xml configuration:
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
You have to enable jacoco plugin only inside java module where there are unit test.

java.lang.NoSuchMethodError in Flink

I trying to read the a file using :
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet<String> line = env.readTextFile("file:///pathtofile/myfile.txt");
I get following error:
java.lang.NoSuchMethodError: org.apache.flink.api.common.io.DelimitedInputFormat: method <init>(Lorg/apache/flink/core/fs/Path;)V not found
I'm using flink version 1.3.2, java version "1.8.0_91"
There is a conflict with dependencies. Apache Flink loads many classes by default into its classpath.
Please read this article https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/debugging_classloading.html the last section
Resolving Dependency Conflicts with Flink using the maven-shade-plugin
Apache Flink loads many classes by default into its classpath. If a user uses a different version of a library that Flink is using, often 
IllegalAccessExceptions
 or 
NoSuchMethodError
 are the result.
So, I suggest to play with your pom.xml and use maven-shade-plugin and add correct relocation, as we have in example
<relocation>
<pattern>org.codehaus.plexus.util</pattern>
<shadedPattern>org.shaded.plexus.util</shadedPattern>
<excludes>
<exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
<exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
</excludes>
</relocation>
Are you getting this error in IntelliJ or Dashboard, if you are getting this error in IntelliJ then make sure you use the same Flink version in your pom.xml and also add dependency shading in the build like this
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerId>jdt</compilerId>
</configuration>
<dependencies>
<dependency>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-compiler-jdt</artifactId>
<version>0.21.0</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>org.codehaus.plexus.util</pattern>
<shadedPattern>org.shaded.plexus.util</shadedPattern>
<excludes>
<exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
<exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
</excludes>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>>
make sure to run maven clean install in the terminal after you make changes . On the other hand, If you are having this issue only in Dashboard not in intelliJ , then have a look here
I faced the similar issue, for me the problem was flink minor version mismatch. My local flink cluster was running flink-1.8.0 and my code expected the version to be flink-1.8.3. Switching to newer version solved this issue.
You need to check your build path, make sure the libs are there and import them properly
One possible cause for error "java.lang.NoSuchMethodError" is when we use different version of flink then what we have installed on our system. For me, I have Flink 1.3.2 and the version I was using was 1.1.1 . So I updated my pom file to have same version.
For the ones who're stuggling with the above issue, while using Flink 1.3.2, here's the entire pom which I was able to successfully build:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<!-- get all project dependencies -->
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<!-- MainClass in mainfest make a executable jar -->
<archive>
<manifest>
<mainClass>FlinktoLambda</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<!-- bind to the packaging phase -->
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>FlinktoLambda</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<!--added newly-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerId>jdt</compilerId>
</configuration>
<dependencies>
<dependency>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-compiler-jdt</artifactId>
<version>0.21.0</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>FlinktoLambda</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
<relocations>
<relocation>
<pattern>org.codehaus.plexus.util</pattern>
<shadedPattern>org.shaded.plexus.util</shadedPattern>
<excludes>
<exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
<exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
</excludes>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<name>my-app</name>
<url>http://maven.apache.org</url>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>FlinktoLambda</groupId>
<artifactId>my-app</artifactId>
<exclusions>
<exclusion>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-remote_2.10</artifactId>
</exclusion>
<exclusion>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-actor_2.10</artifactId>
</exclusion>
<exclusion>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-slf4j_2.10</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table_2.10</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.10</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.10</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala_2.10</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_2.10</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
Please change your main class accordingly in the shade plugin.
Another source of this error could be a mismatch of scala version between Flink and your application code (or your application dependencies that uses scala).
For instance, in my case, I was using Flink 1.7.1 and I had to update my scala dependencies from 2.11 to 2.12; I updated the artifcatId of the concerned dependencies as follows: from flink-scala_2.11 to flink-scala_2.12, flink-table_2.11 to flink-table_2.12, etc.
See here for more info.

maven package class not found selenium Web Driver

I'm creating maven application where I'm getting information from website.
I created maven package, but when I run jar using command:
"java -jar app-1.0-SNAPSHOT.jar" I get this error:
"java.lang.ClassNotFoundException: org.openqa.selenium.WebDriver"
It works good when I'm running it in IntelliJ.
This is my pom:
<groupId>app</groupId>
<artifactId>app</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.53.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-htmlunit-driver</artifactId>
<version>2.52.0</version>
</dependency>
<dependency>
<groupId>com.itextpdf</groupId>
<artifactId>itextpdf</artifactId>
<version>5.0.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>sample.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.10</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
<outputDirectory>${project.build.directory}/dependency-jars/</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Please help me find out what is wrong. Thanks!
there may be a chance of version mismatch as follows:
Starting with 2.53.0 you need to explicitly include HtmlUnitDriver as a dependency to include it. Version number of the driver is now tracking HtmlUnit itself.
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>htmlunit-driver</artifactId>
<version>2.53.1</version>
</dependency>
Keep same versions of Selenium-Java and HTMLUnitDriver.
OR
There might be a dependency on selenium-server-standalone.jar
If you are using DefaultSelenium (or the RemoteWebDriver implementation), you still need to start a Selenium server. The best way is to download the selenium-server-standalone.jar from the Selenium Downloads page and just use it. Furthermore you can also embed the Selenium server into your own project, if you add the following dependency to your pom.xml:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-server</artifactId>
<version>3.0.1</version>
</dependency>
Note: Be aware, that the selenium-server artifact has a dependency to the servlet-api-2.5 artifact, which you should exclude, if your project will be run inside a web application container.
Reference:
http://www.seleniumhq.org/download/maven.jsp

Categories