We Keep Coding

Problem integrating web3j into maven clean install

I have a multi-module maven project where I want to generate java wrappers from .sol files, to achieve this I'm using web3j's maven plugin. Here are the (relevant sections of the) poms:
main pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.web3j</groupId>
<artifactId>web3j-maven-plugin</artifactId>
<version>4.9.4</version>
<executions>
<execution>
<goals>
<goal>generate-sources</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
child pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.web3j</groupId>
<artifactId>web3j-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>generate-sources</goal>
</goals>
</execution>
</executions>
<configuration>
<soliditySourceFiles>
<directory>${project.basedir}/src/main/resources/solidity</directory>
<includes>
<include>**/*.sol</include>
</includes>
</soliditySourceFiles>
<packageName>org.example.project-name.wrappers</packageName>
<outputDirectory>
<java>${project.build.directory}/generated-sources/web3j/java</java>
</outputDirectory>
</configuration>
</plugin>
</plugins>
</build>
I'm build the project with maven with the following command: call mvn clean -U install.
The files do generate, and in the correct location, but when maven begins the compile phase of install it runs into the following error:
Compilation failure
[ERROR] /C:/source/project/project-name/src/main/java/org/example/child/TestFile.java:[4,51]
package org.example.project-name.wrappers does not exist
TestFile is an empty file that tries to import one of the generated files.
I also have an openapi code generator plugin, it does generate files properly, and I run into no issues when importing those.
I didn't find any configuration options in the web3j plugin that I missed and I also didn't find any way to help mvn install or mvn compile consider the directory in which the wrappers are generated.
I tried manually extracting the bundled calls that install makes and manually interjecting the web3j:generate-sources:
call mvn clean
call mvn web3j:generate-sources
call mvn process-resources
call mvn compile
call mvn process-test-resources
call mvn test
call mvn package
call mvn install
call mvn deploy
But this too fails at compile. I'm assuming that the web3j plugin doesn't update a variable storing all generated sources, but that's just a guess and I don't know how I would fix that.

I managed to solve the issue using org.codehaus:build-helper-maven-plugin.
I added the plugin to the dependencies of my main pom and the relevant child (I use dependency management), then added the plugin to the child's pom:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated-sources/web3j</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
I also figured out that my assumption was correct: web3j's plugin does not update the project's sources directory.

Maven copy-dependencies + shade - classpath management

To package a maven project with its dependencies, among many solutions, one may use maven-dependency-plugin with its goal copy-dependencies to get the dependencies in a folder besides, or one may use maven-shade-plugin to get all the code in a single jar.
I actually do both: I choose to have external dependencies (e.g. apache commons) as external libs, and my own dependencies (I have a multi-module maven parent project) shaded into a unique jar.
And it works, except for the classpath. I copy-dependencies with option excludeGroupIds to exclude my own maven group id. I shade with option to include only my own maven group id. Before that, I jar with option to add classpath to the manifest. All set, it works. But my classpath also contains my own dependencies that were actually shaded in the final jar.
It is no big deal, because the result works even with this erroneous classpath. But I wonder if there is a simple means to have the correct classpath, in order not to expose my internal structure to my users.
Here is a basic example demonstrating the problem:
<groupId>com.foo.bar</groupId>
<artifactId>com.foo.bar.launcher</artifactId>
<dependencies>
<dependency>
<groupId>com.foo.bar</groupId>
<artifactId>com.foo.bar.utils</artifactId>
<version>0.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.8.1</version>
</dependency>
</dependencies>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<excludeGroupIds>com.foo.bar</excludeGroupIds>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<includes>
<include>com.foo.bar:*</include>
</includes>
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
The resulting manifest contains this:
Class-Path: lib/com.foo.bar.utils-0.0.1.jar lib/commons-lang3-3.8.1.jar while the com.foo.bar.utils one does not exist.

If you look into the following mvnrepository link maven shade plugin depends upon maven dependency tree. As per the above pom.xml maven dependency plugin, you have excluded com.foo.bar dependency. You can omit the maven-dependency-plugin to create fat jar. It is not mandatory to use in case of shade plugin.
You can use the following command to check and copy all the dependencies used in the project.
mvn dependency:copy-dependencies
https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-shade-plugin/3.2.1

maven generated jaxb classes - ClassNotFoundException

I have a maven model project, where I am generating jaxb class by maven command - clean install
and the jaxb classes are generated under target folder and jar file is generating under .m2 repository folder.
Now on my other project adding this jar as a dependency with proper group id and artifactId.
But I am getting ClassNotFoundException and compile error for those generated jaxb classes.
I am updating my question to add more details.
The Pom File of Model Project.
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5.1</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.jvnet.jaxb2.maven2</groupId>
<artifactId>maven-jaxb2-plugin</artifactId>
<version>0.8.3</version>
<executions>
<execution>
<id>spf-ssp-generate</id>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<generateDirectory>${project.build.directory}/jaxbclasses/pqr/xyz</generateDirectory>
<generatePackage>abc.vo.apply.v1</generatePackage>
<schemaIncludes>
<include>MyXSD.xsd</include>
</schemaIncludes>
</configuration>
</execution>
</executions>
<configuration>
<schemaDirectory>src/main/resources</schemaDirectory>
<extension>true</extension>
<args>
<arg>-XtoString</arg>
<arg>-Xequals</arg>
<arg>-XhashCode</arg>
</args>
<plugins>
<plugin>
<groupId>org.jvnet.jaxb2_commons</groupId>
<artifactId>jaxb2-basics</artifactId>
<version>0.6.4</version>
</plugin>
</plugins>
</configuration>
</plugin>
</plugins>
</build>
On clean install it generates the class files under.
target
jaxbclasses
pqr
xyz
with the package name - abc.vo.apply.v1
I have another Two Maven project(jar), suppose as, A & B.
Now I can use the jaxb model project as a maven dependency, and it compile fine.
Now My Web project is not a Maven project - it is a Liferay based on Ant.
I manually copy the A, B and The Jaxb Model project in to lib folder.
It compile fines. but I am getting ClassNotFoundException.

I am adding another answer, which I think is more accurate.
In order to make your JAXB project compile, I had to add this dependency:
<dependency>
<groupId>org.jvnet.jaxb2_commons</groupId>
<artifactId>jaxb2-basics</artifactId>
<version>0.6.5</version>
</dependency>
which obviously won't be automatically part of your classpath for Liferay.
When I ran mvn dependency:list, I got this:
org.jvnet.jaxb2_commons:jaxb2-basics-tools:jar:0.6.4:compile
org.jvnet.jaxb2_commons:jaxb2-basics:jar:0.6.4:compile
commons-lang:commons-lang:jar:2.2:compile
commons-logging:commons-logging:jar:1.1.1:compile
com.google.code.javaparser:javaparser:jar:1.0.8:compile
org.jvnet.jaxb2_commons:jaxb2-basics-runtime:jar:0.6.4:compile
commons-beanutils:commons-beanutils:jar:1.7.0:compile`
which means that you need to put these in the lib directory of your Liferay installation as well.

This is most likely due to the fact that the target directory is never included in the jar file by default. Try configuring the JAXB classes to be generated under (say) target/generated. Then, add this to the build plugin section of the POM:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-java-sources</id>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>

Maven shade plugin Packaging DLL

I have to add to my project a JNI module.
I install the module in Maven as two different artifact: the jar library:
mvn install:install-file -DgroupId=com.test -DartifactId=ssa -Dversion=1.0 -Dpackaging=jar -Dfile=ssa.jar
and the runtime library with the DLL
mvn install:install-file -DgroupId=com.sirio -Dpackaging=ddl -DartifactId=ssa-runtime -classifier=windows-x86 -Dversion=1.0 -Dfile=SSADll.dll
In my maven project I add these dependecies:
<dependency>
<groupId>com.test</groupId>
<artifactId>ssa</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.test</groupId>
<artifactId>ssa-runtime</artifactId>
<classifier>windows-${arch}</classifier>
<type>dll</type>
<version>1.0</version>
<scope>runtime</scope>
</dependency>
My problem is when I run the shade plugin goal to create a jar with dependencies, I get error:
Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.3:shade (default) on project ....: Error creating shaded jar: error in opening zip file sirio\ssa-runtime\1.0\ssa-runtime-1.0-windows-x86.dll
How can I tell the shade plugin to do not unpack the dll?

Maybe what you need is to package differently. Make the shaded jar with all java classes and libraries you use and then package this jar and the DLL together in a Zip file to be released. For this you can use the maven-assembly-plugin with a descriptor for your zip, like this:
In your pom.xml:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>zip.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
In zip.xml file:
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
<id>release</id>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>target</directory>
<includes>
<include>myapp.jar</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
<files>
<file>
<source>your.dll</source>
<fileMode>0644</fileMode>
</file>
</files>
</assembly>
That way you have all you need to release in this zip. I don't know if that's the best solution, but maybe it solves the problem for your use case.

this solution worked for my JavaFX-OpenCV project
package your project without DLL files.
copy and paste your DLL files in the jar file directory.
now you can run your application because all DLL files are now on the classpath of your jar application.
your directory should be like this :
/target/application.jar
/target/your_DLL_files.dll

Also being new to Maven, it took me a while to solve a similar problem. This answer may help others.
Using com.microsoft.sqlserver:mssql-jdbc_auth with mssql-jdbc_auth-10.2.3.x64.dll as a dependency.
Fortunately the dll is a separate artifact from the mssql jdbc jar.
To stop the shade plugin trying to open the dll as a zip file....
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>com.microsoft.sqlserver:mssql-jdbc_auth</exclude>
</excludes>
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
and then to put the dll into the same directory as the shaded jar file
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.2</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}</outputDirectory>
<includeArtifactIds>mssql-jdbc_auth</includeArtifactIds>
</configuration>
</execution>
</executions>
</plugin>

Maven add jars through systemPath/system but not added to war or anywhere else

I want to add a jar file through the systemPath from the local file-system relative to my project directory structure, not on a remote repository. I added the dependency declaration but maven doesn't do anything else with it.
In the declaration below, I want the jar file copied to my target web-inf/lib directory and also jarred as part of the war file. At present, that doesn't happen. How would I get the jar file copied to my war file?
This is the output from debug maven mode:
DEBUG] cglib:cglib-nodep:jar:2.2:test (setting scope to: compile)^M
DEBUG] Retrieving parent-POM: org.objenesis:objenesis-parent:pom:1.2 for project: null:objenesis:ja
DEBUG] org.objenesis:objenesis:jar:1.2:test (selected for test)^M
DEBUG] org.javap.web:testRunWrapper:jar:1.0.0:system (selected for system)^M
DEBUG] Plugin dependencies for:
...
<dependency>
<groupId>org.javap.web</groupId>
<artifactId>testRunWrapper</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${basedir}/lib/testRunWrapper.jar</systemPath>
</dependency>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webResources>
<resource>
<directory>WebContent</directory>
</resource>
</webResources>
</configuration>
</plugin>

OK, I did this: Note the directory structure at the bottom.
With the approach below, the jar file from the relative project path is treated as a first class citizen like the other jars. The listing below corrects my original problem. With the pom.xml listing below, the jar file is copied to my target directory.
<repositories>
<repository>
<id>JBoss</id>
<name>JBoss Repository</name>
<layout>default</layout>
<url>http://repository.jboss.org/maven2</url>
</repository>
<repository>
<id>my-local-repo</id>
<url>file://${basedir}/lib/repo</url>
</repository>
</repositories>
<dependency>
<groupId>testRunWrapper</groupId>
<artifactId>testRunWrapper</artifactId>
<version>1.0.0</version>
</dependency>
$ find repo
repo
repo/testRunWrapper
repo/testRunWrapper/testRunWrapper
repo/testRunWrapper/testRunWrapper/1.0.0
repo/testRunWrapper/testRunWrapper/1.0.0/testRunWrapper-1.0.0.jar

Using the maven dependency plugin does the job:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>compile</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/${project.build.finalName}/WEB-INF/lib</outputDirectory>
<includeScope>system</includeScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

Don't use system. To do what you want, just declare as a regular (compile) dependency and use mvn install:install-file into your local repository. Everything else will work as you want (lib will be copied, etc.) That will mean that the build will only work on your machine, however.
To properly fix this for your (internal) team, you will want to set up a repository (e.g. Artifactory, Nexus, or Archiva). This is almost a must for team use of Maven.
If this is for public (e.g. open source) use you can either mimic a repository via an http server or put up a real repository.

try something like this (using Ant plugin to manually put the jar to output directory):
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<copy file="${project.basedir}/pathToJAR.jar"
todir="${project.build.directory}/outputFileName/WEB-INF/lib"/>
</tasks>
</configuration>
</execution>
</executions>
</plugin>

AFAIK, system scoped dependencies are somewhat like those with provided scope and thus are not included in the target artifact. Why don't you install the dependency into your local repository instead?
From the doc:
system
This scope is similar to provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository.

In case this answer didn't work for you as it didn't for me and you know that system is a bad scope, you can try this solution where you are Installing the jar by using install-plugin (scroll down a bit), which installs the JAR into your actual local Maven-repository. Basically you only need to add this plugin to your pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>myGroupId</groupId>
<artifactId>myArtifactId</artifactId>
<version>myVersion</version>
<packaging>jar</packaging>
<file>${basedir}/lib/xxx.jar</file>
</configuration>
</execution>
</executions>
</plugin>
Fill in the appropriate values for groupId, artifactId and version and put your original jar file into the <project-home>/lib-directory and fix file above. You can add more execution-sections, but then don't forget to add ids there, like:
<execution>
<id>common-lib</id>
Everybody who updates from the code-repo needs to call mvn initialize once.
And all Eclipse-enthusiasts may add this to pom.xml, too, to get rid of errors in Eclipse:
<pluginManagement>
<plugins>
<!-- This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<versionRange>[2.4,)</versionRange>
<goals>
<goal>install-file</goal>
</goals>
</pluginExecutionFilter>
<action>
<execute></execute>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>

The problem with using a reference to the file system is that dependent projects will not be able to globally access this jar file. i.e. the dependent project's ${basedir} is different and thus the .jar file won't be found.
Global repositories on the other hand are universally accessible.