This question already has answers here:
Maven build [WARNING] we have a duplicate class
(8 answers)
Closed 5 years ago.
I would like to use Maven-shade-plugin to create uber-jar. But when I call mvn package command Maven reports that there are some overlapping classes. I am attaching all problematic overlapps, some of them are caused because older and new verion of a library (Log4J), but some of them seems to have the same classes - e.g. javax.mail and mailapi/smtp/imap et cetera.
What is the best to do in this situation? Is there some key how to decide which overlapping is safe to ignore a which needs to be correct?
- mailapi-1.4.3.jar, javax.mail-1.5.0.jar define 166 overlappping classes
- spring-2.5.6.SEC03.jar, spring-tx-3.1.4.RELEASE.jar define 176 overlappping classes:
- spring-beans-3.1.4.RELEASE.jar, spring-2.5.6.SEC03.jar define 283 overlappping classes:
- slf4j-log4j12-1.7.5.jar, slf4j-impl-2.0-beta2.jar define 3 overlappping classes:
- spring-2.5.6.SEC03.jar, spring-context-support-3.1.4.RELEASE.jar define 55 overlappping classes:
- aopalliance-1.0.jar, spring-2.5.6.SEC03.jar define 9 overlappping classes:
- imap-1.5.0.jar, javax.mail-1.5.0.jar define 87 overlappping classes:
- commons-logging-api-1.1.jar, commons-logging-1.1.3.jar define 19 overlappping classes:
- spring-2.5.6.SEC03.jar, spring-core-3.1.4.RELEASE.jar define 161 overlappping classes:
- spring-2.5.6.SEC03.jar, spring-context-3.1.4.RELEASE.jar define 326 overlappping classes:
- log4j12-api-2.0-beta3.jar, log4j-1.2.17.jar define 23 overlappping classes:
- spring-aop-3.1.4.RELEASE.jar, spring-2.5.6.SEC03.jar define 237 overlappping classes:
- spring-jdbc-3.1.4.RELEASE.jar, spring-2.5.6.SEC03.jar define 239 overlappping classes:
- quartz-1.8.6.jar, quartz-jobs-2.2.1.jar define 15 overlappping classes:
- smtp-1.5.0.jar, javax.mail-1.5.0.jar define 17 overlappping classes:
- spring-asm-3.1.4.RELEASE.jar, spring-2.5.6.SEC03.jar define 31 overlappping classes:
EDIT: this application "A" uses as a Maven dependency my another Java application - I'll call this app "B". This B application uses javax.mail ver 1.5.1. This library uses the first application too. But when I call mvn package command, Maven notices that javax.mail-api-1.5.1.jar, javax.mail-1.5.1.jar define 135 overlappping classes.
Is this problem and if so, how to solve it or can I ignore it?
The first thing to do is remove as many of the obvious causes of overlapping classes as you can. For instance:
You have dependencies on both spring 2.5.6 and spring 3.1.4, which will give you more problems than just in the shade plugin. Set up your module dependencies so you have just one version of spring. Use dependency exclusions if you have to (say you have transitive dependencies that you do not control).
Once the dependency version clashes are fixed, you can also configure which jars go into the uber-jar with the shade plugin configuration, as described at http://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html
Some of the jars probably contain all of the classes from their overlapping jars.
I suspect that commons-logging-1.1.3.jar has a superset of the classes declared in commons-logging-api-1.1.jar. If this is the case, you can exclude the api jar.
In response to the edited question, javax.mail-1.5.1.jar contains a superset of the classes in javax.mail-api-1.5.1.jar. Since these are clearly the same version and the overlapping classes should be identical, it will do no harm to build the shaded jar with the overlapping classes (it will take the classes from whichever jar it processes last). However, the build will be tidier and slightly faster if you exclude the api jar.
It's unlikely that you will need to retain conflicting versions of the classes in the shaded jar. If you do, the shade plugin also allows the relocation of classes, as described at http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html
I found the maven dependency tree plugin very useful to find out from where is the nested dependency coming from , and then add an exclusion for it.
$ **mvn dependency:tree -Dverbose -Dincludes=aopalliance**
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building petshop cli 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) # cli ---
[INFO] com.sample.petshop:cli:jar:1.0
[INFO] \- **org.springframework:spring-context**:jar:4.1.3.RELEASE:compile
[INFO] \- org.springframework:spring-aop:jar:4.1.3.RELEASE:compile
[INFO] \- **aopalliance:aopalliance**:jar:1.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.865s
[INFO] Finished at: Fri May 08 15:12:01 IST 2015
[INFO] Final Memory: 14M/223M
[INFO] ------------------------------------------------------------------------
The aopalliance jar is being referenced from spring-context-support which indicates that we could possibly exclude it.
This question duplicates this one, where you could find more examples.
But in case you are sure you don't have overlapping dependencies (like i had), cleaning project could help. See this answer for more details
Related
I have the following dependencies:
[INFO] +- org.apache.activemq:artemis-core-client:jar:2.19.1:compile
...
[INFO] | \- io.netty:netty-common:jar:4.1.79.Final:compile
[INFO] +- org.apache.activemq:artemis-jms-client-all:jar:2.19.1:compile
After a spring-boot-starter-parent and spring-cloud uplift the following message appeared after the application start:
***************************
APPLICATION FAILED TO START
***************************
Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
io.netty.buffer.AbstractReferenceCountedByteBuf.<init>(AbstractReferenceCountedByteBuf.java:50)
The following method did not exist:
io.netty.util.internal.ReferenceCountUpdater.setInitialValue(Lio/netty/util/ReferenceCounted;)V
The calling method's class, io.netty.buffer.AbstractReferenceCountedByteBuf, is available from the following locations:
jar:file:/app/lib/netty-buffer-4.1.79.Final.jar!/io/netty/buffer/AbstractReferenceCountedByteBuf.class
jar:file:/app/lib/artemis-jms-client-all-2.19.1.jar!/io/netty/buffer/AbstractReferenceCountedByteBuf.class
The calling method's class was loaded from the following location:
file:/app/lib/netty-buffer-4.1.79.Final.jar
The called method's class, io.netty.util.internal.ReferenceCountUpdater, is available from the following locations:
jar:file:/app/lib/artemis-jms-client-all-2.19.1.jar!/io/netty/util/internal/ReferenceCountUpdater.class
jar:file:/app/lib/netty-common-4.1.79.Final.jar!/io/netty/util/internal/ReferenceCountUpdater.class
The called method's class hierarchy was loaded from the following locations:
io.netty.util.internal.ReferenceCountUpdater: file:/app/lib/artemis-jms-client-all-2.19.1.jar
Action:
Correct the classpath of your application so that it contains compatible versions of the classes io.netty.buffer.AbstractReferenceCountedByteBuf and io.netty.util.internal.ReferenceCountUpdater
Since both the artemis-jms-client-all-2.19.1.jar and netty-common-4.1.79.Final.jar is in use and only the netty-common-4.1.79.Final.jar has the necessary method (setInitialValue()) I cannot just remove one of the JARs.
Is there a way to define the order of these?
Is this order issue could have come from the uplift? It was more lazy previously?
You have conflicting versions of the same dependency. When you keep your dependencies as they are, this will be only the first of a longer series of troubles to occur.
Try to use an updated version of artemis-jms-client that dependes on the version of netty-common you need.
Decide to either skip the netty-common dependency (since it is contained in the artemis-jms-client) or don't use the ...-all-... dependency of artemis-jms-client but rather us a version that uses transitive dependencies and loads the others via maven (or gradle or whatever build system you use).
Reordering classpath to resolve that kind of troubles is really a very delicate procedure I would never recommend to perform on third-party-dependencies.
I'm trying to setup a legacy (jboss-eap) project on quarkus
I cannot find a maven dependency to CXF but I know its a "provided" dependency in one of my common-libs since CXF is offered by JBOSS.
Now I run into a warning during build:
INFO] --- quarkus-maven-plugin:1.10.5.Final:build (default) # gmw_rest_dispatch ---
[INFO] [org.jboss.threads] JBoss Threads version 3.1.1.Final
[INFO] [org.hibernate.Version] HHH000412: Hibernate ORM core version 5.4.26.Final
[WARNING] [io.quarkus.arc.processor.BeanArchives] Failed to index org.apache.cxf.phase.AbstractPhaseInterceptor: Class does not exist in ClassLoader QuarkusClassLoader:Deployment Class Loader
[INFO] [io.quarkus.arc.processor.IndexClassLookupUtils] Class for name: org.apache.cxf.phase.AbstractPhaseInterceptor was not found in Jandex index. Please ensure the class is part of the index.
[WARNING] [io.quarkus.arc.processor.BeanArchives] Failed to index org.apache.cxf.feature.AbstractFeature: Class does not exist in ClassLoader QuarkusClassLoader:Deployment Class Loader
[INFO] [io.quarkus.arc.processor.IndexClassLookupUtils] Class for name: org.apache.cxf.feature.AbstractFeature was not found in Jandex index. Please ensure the class is part of the index.
I want my build as clean as possible. So I'd like to get rid of this warning. How can I achieve that?
As a side-note: I can't view what jandex writes to its jandex.idx. So I've no means to debug. Note that the mentioned (org.apache.xf.phase.AbstractPhaseIntercepter and org.apache.cxf.feature.AbstractFeature) classes are also not annotations (AFAIK).
I have a huge project that is a set of about 40 subprojects. It is built with an Ant script.
In each subproject I have some library dependencies, split in a set of folders (lets say: 'lib1', 'lib2', and so on).
It's a huge problem mantaining these libraries.
So, my team decided to move on a more modern built system. We decided to go on Gradle. At this moment, the only thing we will be performed with Gradle is the dependency management.
The lib sets ('lib1', 'lib2' etc) will be each of them put in different folders during the distribution phase. Some of them will not be distributed (are used on for compiling).
The structure so far is like this:
Root project
|- settings.gradle
|- project1
| |- build.gradle
|- project2
| |- build.gradle
...
The root project only includes the subprojects by include ':project1' and so on.
We have now two main challenges:
Download the projects dependencies in different folders depending on the dependency type and check if there are dependencies conflicts.
We presume (but not sure) that we will have to declare some configurations, like:
configurations{
lib1
lib2
compile.extendsFrom(lib1, lib2)
}
Just to be clear, I'd like to download each lib set in a folder like dist/lib1. The Ant script will continue to gather these libs and copying them in to the dest folder.
So, the following questions that remain:
How to declare the dependencies configurations (The above format doesn't seem to work, since compileJava does not include them (although compile configuration extends from them)).
How to download the dependencies in a given folder.
How to check dependencies from all subprojects at once.
Edit: I just found this question which answered questions number 1 and 2. But, when I ran this task:
task copyWebinf( type: Copy ) {
into "$buildDir/libs/WEB-INF"
from configurations.webinf
}
I noticed that I have some libs that I suspect are dupicates:
-rw-rw-r-- 1 thiago thiago 93395 Nov 7 18:00 xpp3-1.1.3.3.jar
-rw-rw-r-- 1 thiago thiago 24956 Nov 7 18:00 xpp3_min-1.1.4c.jar
and
-rw-rw-r-- 1 thiago thiago 246867 Nov 7 18:00 xstream-1.1.2.jar
-rw-rw-r-- 1 thiago thiago 411090 Nov 7 18:00 xstream-1.3.jar
These duplicates might be related to my question number 3 (dependency check).
I am using sbt-xjc plugin to generate java classes from XSD files. The plugin generates these classes under project/target/scala-2.10/xjc directory.
I need to create 2 jar files one with all .class files and another with all .java source files.
I am able to generate the jar file that has all .class files using sbt package but the issue is with sbt packageSrc, this command is looking only for folder those are in project/src/java folder and not considering files those are generated by sbt-xjc plugin under project/target/scala-2.10/xjc. Is there any configuration that i can provide that could help?
To know why this happens the command inspect tree packageSrc is helpful, it will also tell you what to change to have your sources included.
When executed should show you something like this:
> inspect tree packageSrc
[info] compile:packageSrc = Task[java.io.File]
[info] +-compile:packageSrc::packageConfiguration = Task[sbt.Package$Configuration]
[info] | +-compile:packageSrc::mappings = Task[scala.collection.Seq[scala.Tuple2[java.io.File, java.lang.String]]]
[info] | | +-compile:unmanagedSources = Task[scala.collection.Seq[java.io.File]]
[info] | | +-compile:unmanagedResources = Task[scala.collection.Seq[java.io.File]]
[info] | | +-compile:unmanagedResourceDirectories = List(/tmp/q-23437043/src/main/resources)
[info] | | +-*:baseDirectory = /tmp/q-23437043
[info] | | +-compile:unmanagedSourceDirectories = List(/tmp/q-23437043/src/main/scala, /tmp/q-23437043/src/main/java)
// more stuff but not relevant for us
You can see from there that SBT is using mappings key to know from where to take the files.
Knowing that we can take the generated files and add them to the mappings in packageSrc in your build.sbt:
import Path.flat
xjcSettings
def xjcSources(base: File) = base ** "*"
mappings in Compile in packageSrc ++= xjcSources((sourceManaged in (Compile, xjc)).value) pair flat
You can read more about Mappings and Paths to customize / control the result.
I've just started working with maven so if this is an idiotic or obvious question please bear with me.
I've got a soap server that I built for one of our servers, it requires things such as boneCP, slf4j, connector-java, so I added these to the POM.xml and when I just run or debug the soap server it works fine. However whenever I try to build it I get:
Exception in thread "main" java.lang.NoClassDefFoundError: com/ning/http/client/AsyncHttpProvider
at org.sonatype.aether.connector.async.AsyncRepositoryConnectorFactory.newInstance(AsyncRepositoryConnectorFactory.java:106)
at org.sonatype.aether.impl.internal.DefaultRemoteRepositoryManager.getRepositoryConnector(DefaultRemoteRepositoryManager.java:346)
at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:453)
at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:216)
at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:193)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:281)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:186)
at org.sonatype.aether.impl.internal.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:279)
at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:115)
at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getPluginDescriptor(DefaultMavenPluginManager.java:142)
at org.apache.maven.plugin.DefaultBuildPluginManager.loadPlugin(DefaultBuildPluginManager.java:59)
at org.apache.maven.plugin.prefix.internal.DefaultPluginPrefixResolver.resolveFromProject(DefaultPluginPrefixResolver.java:139)
at org.apache.maven.plugin.prefix.internal.DefaultPluginPrefixResolver.resolveFromProject(DefaultPluginPrefixResolver.java:122)
at org.apache.maven.plugin.prefix.internal.DefaultPluginPrefixResolver.resolve(DefaultPluginPrefixResolver.java:86)
at org.apache.maven.lifecycle.internal.MojoDescriptorCreator.findPluginForPrefix(MojoDescriptorCreator.java:262)
at org.apache.maven.lifecycle.internal.MojoDescriptorCreator.getMojoDescriptor(MojoDescriptorCreator.java:222)
at org.apache.maven.lifecycle.internal.DefaultLifecycleTaskSegmentCalculator.calculateTaskSegments(DefaultLifecycleTaskSegmentCalculator.java:106)
at org.apache.maven.lifecycle.internal.DefaultLifecycleTaskSegmentCalculator.calculateTaskSegments(DefaultLifecycleTaskSegmentCalculator.java:86)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:98)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: java.lang.ClassNotFoundException: com.ning.http.client.AsyncHttpProvider
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
... 32 more
I've searched on google but cant seem to find anything relating to this issue when using maven, only when declaring the classpath manually.
Am I missing something?
sorry if this is an obvious or stupid question, as I said I'm just starting out with maven.
EDIT:
Thanks for the help!
I ran the command to check dependencies and got
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) # auth ---
[INFO] com.v2.auth:auth:jar:0.0.1-SNAPSHOT
[INFO] +- junit:junit:jar:3.8.1:test
[INFO] +- ch.qos.logback:logback-classic:jar:1.0.13:compile
[INFO] | +- ch.qos.logback:logback-core:jar:1.0.13:compile
[INFO] | \- org.slf4j:slf4j-api:jar:1.7.5:compile
[INFO] +- com.jolbox:bonecp:jar:0.8.0.RELEASE:compile
[INFO] | +- com.google.guava:guava:jar:15.0:compile
[INFO] | \- (org.slf4j:slf4j-api:jar:1.7.2:compile - omitted for conflict with 1.7.5)
[INFO] \- mysql:mysql-connector-java:jar:5.1.6:compi
so it seems like the problem is coming from guava and logback not playing nice? so it looks like I need to find a way to exclude that slf4j version from being included in Guava?
A common problem with Maven is that sometimes you have dependencies conflicts, especially when you inherit dependencies from a parent POM.
The parent POM might define a version of the library, and your POM define a newer one. From here, there are two possibilities:
-The class not found is included in the older version, but not in the new one, and the new is the one that is being used. In fact, this could happen without the need of conflicting dependencies to exist: you can just have included the newest version instead of the old one.
-The class not found is included in the newer version, but the dependency management causes the old version to be the one used.
Check your dependencies map, and you will find if there is any conflict between versions.