We Keep Coding

Hadoop NoSuchMethodError apache.commons.cli

I'm using hadoop-2.7.2 and I did a MapReduceJob with IntelliJ. In my job, I'm using apache.commons.cli-1.3.1 and I put the lib in the jar.
When I use the MapReduceJob on my Hadoop cluster I have a NoSuchMethodError:
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
I don't understand because the method exist in the class Option and the class Option is extracted from the commons-cli.jar to my application jar. Moreover, I don't have this issue with my others libraries.
Thank you for your time.

We were able to fix this error using maven class relocations. If you are using the shade plugin to build your jar add following to pom.xml under appropriate section:
<!-- necessary to fix NoSuchMethodError: org.apache.commons.cli.Option.builder -->
<relocations>
<relocation>
<pattern>org.apache.commons.cli</pattern>
<shadedPattern>org.shaded.commons.cli</shadedPattern>
</relocation>
</relocations>
Also an explicit reference to v1.3+ of commons-cli needs to be added at the TOP of the dependencies section before any dependency that may have a transitive reference to an older version of commons-cli.

The problem seems to be related to how the classloader is loading the classes. Because the static Builder class was in common-cli 1.4, while some of the hadoop dependencies were still referring to older version - the issue occurred.
In my case, the issue resolved by changing the sequence of jar file addition into the classpath in the shell script responsible for setting up environment before program execution. Earlier, I was adding the jar into classpath like
CLASSPATH=<Hadoop Jars>:<Common CLI jar>:$CLASSPATH
is changed to
CLASSPATH=<Common CLI jar>:<Hadoop Jars>:$CLASSPATH
and it fixed the problem.

We solved this issue with the next gradle configuration:
compile('org.apache.parquet:parquet-tools:1.9.0'){
exclude module:"commons-cli"
}

Error: No class Def found error while setting up Java as a Windows Service

Following is the error log I'm getting:
I'm using
wrapper-windows-x86-32-3.5.25-pro
in order to make my already developed application in Spring-boot. I wanted to execute this application as a windows service but I'm not.
This problem is specific to Tanukisoftware as I'm unable to configure it.
PFB the configuration I'm using for setting up the class path here:
wrapper.java.classpath.1=../lib/wrappertest.jar
wrapper.java.classpath.2=../lib/wrapper.jar
wrapper.java.classpath.3=../lib/slf4j-api-1.7.5
wrapper.java.classpath.4=../lib/myApp.jar
even if myApp.jar contains the slf4j classpath already and its running already. I think this is library specific problem and its not finding up the classpath even if I explicitly set it like did above.
Can you please let me know how it can be solved?

Did you try building a fat jar of your application ? You can add following lines to your Spring Boot project's pom.xml to repackage jar file:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
Besides, your following property line seems invalid:
wrapper.java.classpath.3=../lib/slf4j-api-1.7.5
Is it pointing a jar file actually ?

slf4j-api-1.7.5 should have a .jar extension. That is possibly confusing things.

Also, even just as a debugging step, try running your Windows service using Apache ProcRun to see if your results are any different.

NoClassDefError: SnakeYAML, with Maven

I'm having a runtime issue occurring with SnakeYAML and maven. For some reason, I'm getting a java.lang.NoClassDefFoundError with Yaml when I run my .jar.
Here is the stack trace:
Exception in thread "main" java.lang.NoClassDefFoundError: org/yaml/snakeyaml/Yaml
at my.package.TimberServer.loadConfiguration(TimberServer.java:56)
at my.package.TimberServer.<init>(TimberServer.java:38)
at my.package.Main.main(Main.java:17)
Caused by: java.lang.ClassNotFoundException: org.yaml.snakeyaml.Yaml
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 3 more
Here is the section in my pom.xml in which I declare the dependency (link to central repository):
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.14</version>
</dependency>
I don't declare any repository for it.
Here is the code that calls (and imports) the Yaml class:
import org.yaml.snakeyaml.Yaml;
// everything else is imported, yes
public class TimberServer {
public void loadConfiguration() throws IOException {
Yaml yaml = new Yaml(); // Error occurs HERE.
Map config = (Map) yaml.load(FileUtils.readFileToString(this.config));
Map<String, String> serverConfig = (Map<String, String>) config.get("server");
}
}
I get no errors when compiling with maven, which is the most confusing part to me - does Java not check for existent classes? I'm programming with an IDE (IntelliJ), which does not give any errors or warnings with the lines of code.

Are you including SnakeYaml in the JAR you're trying to run? If not, you'll need to 1) use Maven's uber-jar plugin to include it, 2) add the YAML JAR to your classpath when running the JAR or 3) use Maven's exec plugin to run your JAR so it can pick up the dependencies.

After help from #Seelenvirtuose in the comments, I came across this solution:
I was running mvn package alone - with no dependency plugin, which I learned is a necessity for packaging. However, in my specific case, the Uber jar plugin did not want to work right and I had to use the Assembly plugin.
Note: the resultant .jar from the Assembly plugin will be named target/yourProject-jar-with-dependencies.jar. I overlooked this.

Deploy with maven plugin on remote weblogic 12c

I have weblogic 12c on machine 192.168.1.3. I want to deploy ear from machine 192.168.1.2 with maven plugin:
<groupId>com.oracle.weblogic</groupId>
<artifactId>wls-maven-plugin</artifactId>
<version>12.1.1.0</version>
In the first phase I made in the project:
mvn wls:install
Plugin configuration looks like this:
<configuration>
<adminurl>t3://192.168.1.3:7001</adminurl>
<user>weblogic</user>
<password>welcome1</password>
<debug>true</debug>
<name>test-ear-dev01</name>
<remote>true</remote>
<upload>true</upload>
<advanced>true</advanced>
<failOnError>true</failOnError>
<artifactLocation>c:\Users\bartek\Downloads\wls1211_dev.zip</artifactLocation>
</configuration>
next I make
mvn wls:deploy
and I get following error
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Invalid file. Please provide an existing fully qualified path of the file.
[DEBUG] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Invalid file. Please provide an existing fully qualified path of the file.
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:719)
...
...
...
Caused by: org.apache.maven.plugin.MojoExecutionException: Invalid file. Please provide an existing fully qualified path of the file.
at weblogic.tools.maven.plugins.deploy.DeployerMojo.handleDeployerException(DeployerMojo.java:459)
...
Caused by: org.apache.maven.plugin.MojoExecutionException: Invalid file. Please provide an existing fully qualified path of the file.
at weblogic.tools.maven.plugins.deploy.DeployerMojo.getSourceParameter(DeployerMojo.java:434)
...
Can you tell me what I'm doing wrong, when deploy ear to a remote server

The configuration you show looks like what you used to install WebLogic with the install goal of the plugin. <artifactLocation> is a configuration element for that goal, not deploy, per the documentation.
For the deploy goal, replace <artifactLocation> with <source>, which contains the name of your ear file.

Maven AppAssembler not finding class

Attempting to modify an existing Java/Tomcat app for deployment on Heroku following their tutorial and running into some issues with AppAssembler not finding the entry class. Running target/bin/webapp (or deploying to Heroku) results in Error: Could not find or load main class org.stopbadware.dsp.Main
Executing java -cp target/classes:target/dependency/* org.stopbadware.dsp.Main runs properly however. Here's the relevant portion of pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.1.1</version>
<configuration>
<assembleDirectory>target</assembleDirectory>
<programs>
<program>
<mainClass>org.stopbadware.dsp.Main</mainClass>
<name>webapp</name>
</program>
</programs>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>assemble</goal>
</goals>
</execution>
</executions>
</plugin>
My guess is mvn package is causing AppAssembler to not use the correct classpath, any suggestions?

Your artifact's packaging must be set to jar, otherwise the main class is not found.
<pom>
...
<packaging>jar</packaging>
...
</pom>
The artifact itself is added at the end of the classpath, so nothing other than a JAR file will have any effect.

Try:
mvn clean package jar:jar appassembler:assemble

Was able to solve this by adding "$BASEDIR"/classes to the CLASSPATH line in the generated script. Since the script gets rewritten on each call of mvn package I wrote a short script that calls mvn package and then adds the needed classpath entry.
Obviously a bit of a hack but after a 8+ hours of attempting a more "proper" solution this will have to do for now. Will certainly entertain any more elegant ways of correcting the classpath suggested here.

I was going through that tutorial some time ago and had very similar issue. I came with a bit different approach which works for me very nicely.
First of all, as it was mentioned before, you need to keep your POM's type as jar (<packaging>jar</packaging>) - thanks to that, appassembler plugin will generate a JAR file from your classes and add it to the classpath. So thanks to that your error will go away.
Please note that this tutorial Tomcat is instantiated from application source directory. In many cases that is enough, but please note that using that approach, you will not be able to utilize Servlet #WebServlet annotations as /WEB-INF/classes in sources is empty and Tomcat will not be able to scan your servlet classes. So HelloServlet servlet from that tutorial will not work, unless you add some additional Tomcat initialization (resource configuration) as described here (BTW, you will find more SO questions talking about that resource configuration).
I did a bit different approach:
I run a org.apache.maven.plugins:maven-war-plugin plugin (exploded goal) during package and use that generated directory as my source directory of application. With that approach my web application directory will have /WEB-INF/classes "populated" with classes. That in turn will allow Tomcat to perform scanning job correctly (i.e. Servlet #WebServlet annotations will work).
I also had to change a source of my application in the launcher class:
public static void main(String[] args) throws Exception {
// Web application is generated in directory name as specified in build/finalName
// in maven pom.xml
String webappDirLocation = "target/embeddedTomcatSample/";
Tomcat tomcat = new Tomcat();
// ... remaining code does not change
Changes to POM which I added - included maven-war-plugin just before appassembler plugin:
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>exploded</goal>
</goals>
</execution>
</executions>
</plugin>
...
Please note that exploded goal is called.
I hope that small change will help you.
One more comment on that tutorial and maven build: note that the tutorial was written to show how simple is to build an application and run it in Heroku. However, that is not the best approach to maven build.
Maven recommendation is that you should adhere to producing one artifact per POM. In your case there are should two artifacts:
Tomcat launcher
Tomcat web application
Both should be build as separate POMs and referenced as modules from your parent POM. If you look at the complexity of that tutorial, it does not make much sense to split that into two modules. But if your applications gets more and more complex (and the launcher gets some additional configurations etc.) it will makes a lot of sense to make that "split". As a matter of fact, there are some "Tomcat launcher" libraries already created so alternatively you could use of one them.

You can set the CLASSPATH_PREFIX environment variable:
export CLASSPATH_PREFIX=target/classes
which will get prepended to the classpath of the generated script.

The first thing is that you are using an old version of appassembler-maven-plugin the current version is 1.3.
What i don't understand why are you defining the
<assembleDirectory>target</assembleDirectory>
folder. There exists a good default value for that. So usually you don't need it. Apart from that you don't need to define an explicit execution which bounds to the package phase, cause the appassembler-maven-plugin is by default bound to the package phase.
Furthermore you can use the useWildcardClassPath configuration option to make your classpath shorter.
<configuration>
<useWildcardClassPath>true</useWildcardClassPath>
<repositoryLayout>flat</repositoryLayout>
...
</configruation>
And that the calling of the generated script shows the error is depending on the thing that the location of the repository where all the dependencies are located in the folder is different than in the generated script defined.