We Keep Coding

Maven compile and provided scopes for unit testing of JPMS module with surefire-plugin

Trying to test my maven project (one parent pom and several modules (jpms modules)) I got annoying Error occurred in starting fork:
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project com.foo.plugin: There are test failures.
...
Error occurred in starting fork, check output in log
Process Exit Code: 1
org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
...
Error occurred in starting fork, check output in log
Process Exit Code: 1
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:690)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:285)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:248)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1217)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1063)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:889)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginM
A solution I found, for example, here suggested to set <forkCount>0</forkCount> in surefire-plugin configuration. However, such solution doesn't allow to run tests on module-path, so I went to this surefire issue.
Surefire developers (thanks to them) found out that the reason was in maven scope of the dependencies I used. In my project I have:
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
<version>2.1.6</version>
<scope>provided</scope> <!-- NOTE IT -->
</dependency>
<dependency>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
<version>2.3.2</version>
<scope>provided</scope> <!-- NOTE IT -->
</dependency>
And they said it had to be:
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
<version>2.1.6</version>
<scope>compile</scope> <!-- NOTE IT -->
</dependency>
<dependency>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
<version>2.3.2</version>
<scope>compile</scope> <!-- NOTE IT -->
</dependency>
And when I changed the scope my project was tested by surefire without problems. Both dependencies jakarta.ws.rs-api and jakarta.xml.bind-api are JPMS modules and are required by my JPMS modules.
So the question, is the problem in my code (provided is wrong if I want to run my tests for my JPMS module) or the problem is in surefire plugin?

NOTE: You can freely remove <scope>compile</scope> since no declaration of the scope in your POM means that the Maven will use the default value compile anyway.
The scope of provided is the project architecture aspect.
It is used in situations when the application server contains WS API (even if different arfifact file name). This way we mark it provided which makes the WAR file smaller and the WS API artifact would not be included.
In situations where you are also building serverless application and you are building your own container, then no scope of provided is needed to declare. There the artifact will appear within the Fat JAR which is what you expect.
So as you can see the dependency marked as provided copes with classpaths. It appears on Compiler classpath but it does not appear in runtime.
There is one more feature in the scope provided. It is the inheritance, where such of dependency cannot be inherited transitively.
If you have such dependency witht the scope provided in a POM, then the inheritance would not see the dependency in a dependent child POM.
For more information pls see Maven documentation.
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
This scope is only available on the compilation and test classpath, and is not transitive.

How to remove duplicate dependencies from pom?

I have a multi-module project in maven where some of the modules depend on other modules. Now the modules that act as dependencies have some of the dependencies which are already listed in the dependent module's pom.
Is there a quick way to identify such duplicate dependencies and remove them from the dependent module's pom?

A project's dependency tree can be expanded to display dependency conflicts.
Use command
mvn dependency:tree -Dverbose=true
to identify such duplicate dependencies. It shows all duplicates and conflicts in the pom.
Use the <exclusions> tag under the <dependency> section of the pom to exclude such duplicate dependencies.
<dependencies>
<dependency>
<groupId>sample.ProjectA</groupId>
<artifactId>Project-A</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion> <!-- declare the exclusion here -->
<groupId>sample.ProjectB</groupId>
<artifactId>Project-B</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>

If you are using eclipse as IDE then the duplicates can be seen in the dependency hierarchy of the concerned pom.xml. And using exclusions tag they can be ommitted.

You can use mvn depgraph:graph -Dincludes=ets.tkt -DshowDuplicates -Dscope:compile.
To use this plugin put this on your settings.xml
<settings>
. . .
<pluginGroups>
<pluginGroup>com.github.ferstl</pluginGroup>
</pluginGroups>
</settings>
When you run the previous console command, you can go to /target and you will find a .dot file. you can render this file using graphviz.
More info at https://github.com/ferstl/depgraph-maven-plugin

There is a JBoss tool to help with these issues: http://tattletale.jboss.org/
Unfortunately, it seems that is not under active development these days.

Maven transitive dependency not being deployed in TeamCity

I have a small maven web project. Its pom includes dependencies to other libraries. One of them has a dependency to slf4j-api.
When running maven's clean install from inside IntelliJ IDEA the resulting war file includes slf4j-api.jar in WEB-INF/lib
Now we have configured a TeamCity maven build for the project. It builds without any errors, but it doesn't include this specific transitive dependency (whereas it includes others).
Do you have any general hints to why this might happen?
Currently we added the dependency directly to the main pom even though we don't need it there...
In the main pom
<dependency>
<groupId>xxxxxxxxx</groupId>
<artifactId>mylibrary</artifactId>
<version>1.0.20</version>
</dependency>
in mylibrary's pom
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
I haven't specified any scope, so I suppose it's compile in any case.

Maven war plugin: Is all dependencies specified in pom get added to war?

I use maven-war-plugin in my pom to build my Vaadin application. My question is, if I added an unnecessary dependency (dependent library is not actually used in code) to my pom, Will maven-war-plugin still bundle the dependency into the war file it generates?

The answer to your question depends on the scope you specify in the <dependency> tag. Consider the following dependency tag:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>provided</scope>
</dependency>
The provided scope tells Maven to use the log4j JAR when compiling but to exclude it from the build, so it would not appear in your WAR. If, on the other hand, you used a scope of compile or runtime, then it would appear in the WAR.
If you don't specify any <scope>, then the default is compile, which means that the depedency will appear in the build output.

How to add maven dependencies for mahout and hadoop?

I am doing a project that has dependencies on some classes from the mahout and hadoop core jars. I was using javac with the classpath option to include them before, but someone suggested to me that I should use maven to build my project instead. However, I am not sure how to add the dependencies to these jar files which are located in my /usr/local directory.

<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.205.0</version> <!-- or whatever version -->
</dependency>
<dependency>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout-core</artifactId>
<version>0.5</version>
</dependency>

Add this to your pom:
<dependencies>
<dependency>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout-core</artifactId>
<version>0.5</version>
</dependency>
<dependency>
<groupId>some.group</groupId>
<artifactId>hadoop</artifactId>
<version>some.version</version>
</dependency>
</dependencies>
If you have a copy of the jar to be used for say the hadoop example above, execute this command:
mvn install:install-file -Dfile=/some/path/my-hadoop.jar -DgroupId=some.group -DartifactId=hadoop -Dversion=some.version -Dpackaging=jar

Have a look at the maven documentation, especially the part on dependency management. If you want to use Maven you should get to know the basics (one of which is dependency management).
Basially you define your project's dependencies in the <dependencies> section of your pom. Look up maven central (the most common online repository) for the dependencies you want or search for other online repositories that might contain them.
If you can't find them, add the dependencies you want anyways (think of a sensible group id, artifact id and version) and try to compile. Maven will complain about the dependencies missing and provide a basic command to put those dependencies into the local repository. Copy those commands and fill in the appropriate path to the jar file and maven will deploy that dependency in your local repository.
Note that you should first look for the dependencies in an online repository since otherwise you'd have to manually deploy each new version in your local repo.