We Keep Coding

How can I use previously built jars in a multi-module maven project?

I have this maven multi-module project :
ModuleA
src
pom.xml
target
ModuleA-with-dependencies-shaded.jar (version 4.1 of lucene relocated)
ModuleB
src
pom.xml
target
ModuleB-with-dependencies.jar (version 7.5.0 of lucene)
ModuleDist
assembly
all.xml
pom.xml (shaded plugin for jar + assembly for Docker)
The dist pom plugins are configured like this :
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<shadedClassifierName>all</shadedClassifierName>
<shadedArtifactAttached>true</shadedArtifactAttached>
<transformers>
<!--remove models from jar see mitie -->
<transformer implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resource>.dat</resource>
</transformer>
</transformers>
<artifactSet>
<excludes>
<exclude>log4j:log4j:jar:</exclude>
</excludes>
</artifactSet>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<id>make-dist</id>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/assembly/all.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
And the assembly all.xml :
<assembly>
<id>all</id>
<formats>
<format>dir</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/main/docker</directory>
<outputDirectory>/</outputDirectory>
<includes>
<include>*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>lib</outputDirectory>
<includes>
<include>*.jar</include>
</includes>
</fileSet>
</fileSets>
</assembly>
As the ModuleA is having a transitive dependency on lucene 4.1 (but relocated so it won't clash with moduleB) and ModuleB is having a transitive dependency on lucene 7.5.0, I'd like to use the previously built and shaded jar of ModuleA in the maven shaded plugin of ModuleDist (because if I relocate lucene in ModuleDist it is relocating all lucene classes).
How could I do that ?
or is there another way of doing this ?

To add all your jars of the runtime dependencies to the assembly, add something like this:
<?xml version="1.0"?>
<assembly>
...
<dependencySets>
<dependencySet>
<outputDirectory>/lib</outputDirectory>
<unpack>false</unpack>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
</assembly>
Take a look at the assembly plugin documentation for more details.
A working example can be found here (no shading used).
My personal experience is that it's better to only have one final module that assembles all stuff. This way the shader doesn't have to disassemble and assembler everything over and over.
Shading is a big problem when using Jackson and Log4j2 because it breaks some extension lookup mechanisms that expect everything to be in a separate jar. I recommend not using it anymore.

Filtered runable fat .jar in maven with maven-assembly-plugin

I'm currently moving a small project in to Maven and I am having some trouble.
The setup is as follows
Where a setting file is used in a my resource folder for running the application in eclipse. But when i create a fat jar the setting file is packed in side the jar. I want to filter the setting file so it is outside the jar (I understand this makes the file brittle, but that is by design) and still have it available to the application from the same folder as the jar. I have achieved this with an eclipse runnable jar and and Ant file. But I cannot seem to get it to work with maven. Any insight you have would be very useful.
Her are examples of my plugin and filter file. Any Ideas on this would be great as this has been driving me insane
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<filters>
<filter>src/assembly/filter.properties</filter>
</filters>
<archive>
<manifest>
<mainClass>ie.business.project.artifact</mainClass>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
and my filter
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0
http://maven.apache.org/xsd/assembly-2.0.0.xsd">
<id>distribution</id>
<formats>
<format>jar</format>
</formats>
<files>
<file>
<source>settings.ini</source>
<outputDirectory>/</outputDirectory>
<filtered>true</filtered>
</file>
</files>
</assembly>

So I have tired something else. If I define an exclusion in my jar-plugin in addition to using maven-assembly, the setting file dose not end up in the fat jar. I'm unsure if this is the right way to do this but it has worked.
Please let me know if there in a better way to do this.
I added the following plugin to my pom
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<excludes>
<exclude>setting.ini</exclude>
</excludes>
</configuration>
</plugin>

It seems that you are not using the assembly plugin pointing to the assembly in your question. If you look at the plugin configuration you are using:
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
Actually you should be pointing to the relative path of the assembly file, like e.g:
<configuration>
<descriptors>
<descriptor>src/assembly/assembly.xml</descriptor>
</descriptors>
</configuration>
Now the assembly file for the creation of a fat jar should be changed to create the fat jar and exclude some files. This example is just to give you an idea:
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<fileSets>
<fileSet>
<directory>${basedir}</directory>
<excludes>
<exclude>settings.ini</exclude>
</excludes>
</fileSet>
</fileSets>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
</assembly>

Create zip in maven with additional files next to the jar

The only Maven experience I have is including other libraries so I need a very basic explanation about how I can achieve things in Maven with Eclipse.
I want to create my jar regulary. Then I want to take 3 further files and put all files together in 1 zip file. The content of my zip should look like this:
A.jar
B.txt
C.txt
D.bat
I know I have to insert certain goals in my pom.xml file. I already saw posts on Stack Overflow (i.e. here) regarding similar topics. I also tried the Maven documentation like here. Do I already need any Maven plugins or is still already Maven-core stuff?
But my skills are not enough yet to transfer this information for my case.

+ project
+ src/main/java
+ src/main/resources
+ src/main/config
+B.txt
+C.txt
+D.bat
+ src/main/assembly
+bin.xml
+pom.xml
bin.xml
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>bin</id>
<baseDirectory>/</baseDirectory>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/main/config</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>/</outputDirectory>
<includes>
<include>*.jar</include>
</includes>
</fileSet>
</fileSets>
</assembly>
poml.xml
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<configuration>
<descriptors>
<descriptor>src/main/assembly/bin.xml</descriptor>
</descriptors>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- append to the packaging phase. -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Output :
mvn clean package
+ artifactId-version.zip
+ B.txt
+C.txt
+D.txt
+artifactId-version.jar
Not added B.txt, C.txt, D.bat in src/main/resources because it's not good to have them in the CLASSSPATH

You need to use the maven-assembly-plugin when you need to create custom artifacts. I strongly suggest that you read Chapter 8. Maven Assemblies of the Maven book to get you started with using assemblies. This chapter contains in-depth explanations on the creation of Maven assemblies, and it is easy to read.
Basically, an assembly is created with the help of an assembly descriptor. The following assembly descriptor will include the project main artifact at the root of a zip archive. You can add more <file> or <fileSets> declaration here to add your custom files in your archive (the logic is the same)
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>assemby-id</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<files>
<file>
<source>${project.build.directory}/${project.build.finalName}.${project.packaging}</source>
<outputDirectory>/</outputDirectory>
</file>
</files>
</assembly>
Then you just need to declare this plugin in your POM. In this case, the plugin is bound to the package phase and configured with the descriptor above. By default, the assembly id is appended to the name of the archive: I removed it here.
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor> <!-- path to the descriptor -->
</descriptors>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
</execution>
</executions>
</plugin>
When running mvn clean package, you will see that a zip archive will have been created in the target directory.

Is the content of the Maven Assesmnly plugin's jar-with-dependencies descriptorRef documented as an example assembly descriptor file anywhere?

The following see How can I create an executable jar with dependencies using Maven? shows how to create an executable jar with the Maven plugin.
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>fully.qualified.MainClass</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
However I wish to exclude some of the dependencies which are in my POM. All I want to do is to add
<dependencySets>
<dependencySet>
<useStrictFiltering>true</useStrictFiltering>
<scope>compile</scope>
<excludes>
<exclude>com.excluded:artifact</exclude>
</excludes>
</dependencySet>
</dependencySets>
to the above but I cannot , because I need to add that in a separate assembly descriptor. What I want to do is exactly the same as jar-with-dependencies but with the exclusion. Is there somewhere where an equivalent assembly descriptor file is described so I can edit it? Is there any way to 'inherit' jar-with-dependencies and add my exclusions?

I think that you can use scope "compile" :
<dependency>
<groupId>com.xxxx</groupId>
<artifactId>xxxxxx</artifactId>
<version>3.x</version>
<scope>compile</scope>
</dependency>

See The Maven Documentation on pre defined descriptors for the equivalent of jar-with-depedencies.
So I can change the POM I am using to build 'super jar' as follows;
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<mainClass>my.Main</mainClass>
</manifest>
</archive>
<descriptors>
<descriptor>src/main/assembly/assembly.xml</descriptor>
</descriptors>
</configuration>
<!-- Uncomment this for automatic creation of the Complete jar
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
-->
</plugin>
And in the directory src/main/assembly/assembly.xml I can put the following which is essentially the same as the jar-with-dependencies descriptor with my exclusions;
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<!-- TODO: a jarjar format would be better -->
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
<excludes>
<exclude>com.local.depedency:artifact_id</exclude>
<exclude>transitive.depedency.so.way.down:artifact_id</exclude>
</excludes>
</dependencySet>
</dependencySets>
</assembly>

How do you create a standalone application with dependencies intact using Maven?

I have a desktop Java application built using Maven 2 (but I could upgrade to Maven 3 if that helps) with a number of open source dependencies. I'm now trying to package it up as a standalone to make it available to end users without them needing to install maven or anything else.
I've successfully used maven-assembly-plugin to build a single jar containing all dependencies but this is not really what I want because when using LGPL libraries you are meant to redistribute the libraries you are using as separate jars.
I want Maven to build a zip containing a jar with my code and a MANIFEST.MF that refers to the other jars I need together with the other jars. This seems like standard practice but I cannot see how to do it.
Here's an extract from my pom
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<encoding>UTF-8</encoding>
<compilerVersion>1.6</compilerVersion>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dfile.encoding=UTF-8</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<mainClass>com.company.widget.Main</mainClass>
<packageName>com.company.widget</packageName>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.3</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.company.widget.Main</mainClass>
<packageName>com.company.widget</packageName>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
EDIT:Taken on Kals idea
created a file called descriptor.xml
<?xml version="1.0" encoding="UTF-8"?>
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>distribution</id>
<formats>
<format>zip</format>
</formats>
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<outputDirectory>lib</outputDirectory>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
</assembly>
and pom contains:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.3</version>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor>
</descriptors>
<archive>
<manifest>
<mainClass>com.company.widget.cmdline.Main</mainClass>
<packageName>com.company.widget</packageName>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Now maintains the jar and puts them all in the lib folder, including my own code

You should check out the Maven Appassembler plugin. You can get a lot more robust package using it than rolling your own assembly.
It generates helpful startup scripts for Unix and Windows that allow you to set predefined JAVA VM options, commandline parameters and classpath.
It also has a concept of configuration directory where you can copy default configurations that user can later change. You can also set the configuration directory to be available on the classpath.
Dependencies can be saved in a Maven style "repo" or you can use a flat style "lib" directory.
You still need the assembly plugin for creating a zip or tar archive.
Here's an example appassembler configuration:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<programs>
<program>
<mainClass>com.mytools.ReportTool</mainClass>
<name>ReportTool</name>
</program>
</programs>
<assembleDirectory>${project.build.directory}/ReportTool</assembleDirectory>
<repositoryName>lib</repositoryName>
<repositoryLayout>flat</repositoryLayout>
</configuration>
<executions>
<execution>
<id>assembly</id>
<phase>package</phase>
<goals>
<goal>assemble</goal>
</goals>
</execution>
</executions>
</plugin>
To get a zip archive I use the this assembly:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.build.directory}/ReportTool</directory>
<outputDirectory>/</outputDirectory>
<includes>
<include>/**</include>
</includes>
</fileSet>
</fileSets>
</assembly>

Try creating a custom assembly descriptor and add a dependencySet and make sure you specify, unpack as false.
Use this as assembly descriptor,
<?xml version="1.0" encoding="UTF-8"?>
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>distribution</id>
<formats>
<format>zip</format>
</formats>
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<outputDirectory>lib</outputDirectory>
<useProjectArtifact>false</useProjectArtifact>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
</assembly>
Store this file to say src/main/assembly/assembly.xml and update your assembly plugin configuration in pom.xml like this.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2.1</version>
<executions>
<execution>
<id>assembly</id>
<phase>package</phase>
<goals>
<goal>attached</goal>
</goals>
<configuration>
<descriptor>${basedir}/src/main/assembly/assembly.xml</descriptor>
</configuration>
</execution>
</executions>
</plugin>
Here is assembly descriptor reference if you need anything more
http://maven.apache.org/plugins/maven-assembly-plugin/assembly.html

As far as I know it is possible to redistribute also LGPL libraries into your own packages as long as they are untouched. The maven assembly plugin creates a jar that contains the original jars inside a lib folder of the archive. So far you are fulfilling the LGPL.
Maybe this question gives some more information about this topic.
(Disclaimer: I'm not a lawyer, so please crosscheck this information ;) )

Add the following plugins in pom.xml. Check the value at mainClass,classpathPrefix,addClasspath tags.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<mainClass>org.apache.camel.spring.Main</mainClass>
<classpathPrefix>lib/</classpathPrefix>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptors>
<descriptor>src/assembly/some-assembly.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Create some-assembly.xml under src/assembly as below.
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>distribution</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>true</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>/</outputDirectory>
<includes>
<include>*.jar</include>
</includes>
</fileSet>
</fileSets>
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<outputDirectory>/lib</outputDirectory>
<useProjectArtifact>false</useProjectArtifact>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
Note that useProjectArtifact flag to false, unpack flag to false. If root folder inside zip file is not required,then one can make includeBaseDirectory to false.
This will create name-version-distribution.zip file. Inside zip file, there will be folder name-version. Inside this folder, your executable jar and lib folder containing all dependency jars will be present.
Check manifest.MF file of executable jar. It contains both main class and classpath information.