We Keep Coding

Spring boot 2.2 activemq jetty conflict

I am trying to upgrade Spring boot to the version 2.2 together with jetty starter.
I get these errors due to jetty version conflict
The following method did not exist:
'org.eclipse.jetty.websocket.server.NativeWebSocketConfiguration org.eclipse.jetty.websocket.server.NativeWebSocketServletContainerInitializer.initialize(org.eclipse.jetty.servlet.ServletContextHandler)'
The method's class, org.eclipse.jetty.websocket.server.NativeWebSocketServletContainerInitializer, is available from the following locations:
jar:file:/some-dir/target/p3.0.166-SNAPSHOT.war!/WEB-INF/lib/jetty-all-9.4.19.v20190610-uber.jar!/org/eclipse/jetty/websocket/server/NativeWebSocketServletContainerInitializer.class
jar:file:/some-dir/target/p3.0.166-SNAPSHOT.war!/WEB-INF/lib/websocket-server-9.4.20.v20190813.jar!/org/eclipse/jetty/websocket/server/NativeWebSocketServletContainerInitializer.class
I have activemq dependency which brings in it's own jetty-all versioned 9.4.19 dependency which is in conflict with spring-boot 2.2 jetty (9.4.20)
And part of my pom.xml is:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!--
Jsp-api isn't standard in spring boot
-->
<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>javax.servlet.jsp-api</artifactId>
<version>${jsp.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<scope>provided</scope>
</dependency>
<!-- artefacts enable JSP running in spring-boot -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>apache-jsp</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>apache-jstl</artifactId>
</dependency>
<!--
Used to be a single artifact.
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.2</version>
Newer versions splits the interface and implementation.
This suggests to use a Glassfish implementation.
https://www.andygibson.net/blog/quickbyte/jstl-missing-from-maven-repositories/
The one we used had an Apache implementation, so going with that.
https://stackoverflow.com/a/24444342
-->
<dependency>
<groupId>org.apache.taglibs</groupId>
<artifactId>taglibs-standard-spec</artifactId>
<version>${taglibs.version}</version>
</dependency>
<dependency>
<groupId>org.apache.taglibs</groupId>
<artifactId>taglibs-standard-impl</artifactId>
<version>${taglibs.version}</version>
</dependency>
<!-- Unit test dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.easytesting</groupId>
<artifactId>fest-assert</artifactId>
<version>1.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<scope>test</scope>
</dependency>
<!-- html compressing is used by hrmanager in the JSP -->
<dependency>
<groupId>com.googlecode.htmlcompressor</groupId>
<artifactId>htmlcompressor</artifactId>
<version>1.5.2</version>
</dependency>
<!-- ApacheMQ HTTP jarfile set -->
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-http</artifactId>
</dependency>
</dependencies>
Any idea how I can fix this?

ActiveMQ is wrong here.
jetty-all is not meant to be used as a dependency in a project.
See https://www.eclipse.org/lists/jetty-users/msg06030.html
It only exists as a command line tool for the documentation to educate folks about basic featureset of Jetty.
It does not, and cannot, contain all of Jetty.
A single artifact with everything that Jetty produces is impossible.
As #Shilan pointed out, excluding jetty-all is the correct solution.
The use of the other appropriate dependencies (by spring-boot-starter-jetty it seems) will already pull in the correct Jetty transitive dependencies that you need.
You can use $ mvn dependency:tree from the command line to see this (before and after excluding jetty-all)
You might want to run one of the duplicate class finder maven plugins to see what other duplicate classes you have going on and correct for those as well.
https://github.com/ning/maven-duplicate-finder-plugin
https://github.com/basepom/duplicate-finder-maven-plugin

Include spark avoiding huge dependencies

I would like to include spark sql in my project. However, if doing so, the jar file becomes huge (over 120 MB) because Maven includes numerous dependencies.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.0</version>
</dependency>
Is there a way to minimize the included dependencies?

Depends on your use case. By default, maven includes all the dependencies of spark-sql in the uber jar. Based on your case, you may not use all of them. So you can exclude them from your dependency.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.0</version>
<exclusions>
<!-- to remove jackson-databind from your uber jar -->
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusions>
</exclusions>
</dependency>
But this won't help you if your application uses most of the features of spark-sql.
In many of the cases, the spark dependencies will be provided by the environment in which you are going to run your application(apart from standalone mode). In such cases, you can just flag spark-sql dependency as provided dependency as shown below,
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.0</version>
<scope>provided</scope>
</dependency>

How can I remove the old vulnerable Apache commons collection version dependency from my project's maven dependency tree?

My Java app project is being managed by Maven.
My project has a few library dependencies depending again on Apache commons collection 3.2.1 which is vulnerable - e.g. Apache commons configuration, velocity, etc.
(I can see it is being used by running mvn dependency:tree command.)
I did neither write any line of codes using Apache commons collection directly nor defined the dependency of it, but it's being used.
What could I do to remove its dependency and to force to use safe version - 3.2.2, 4.1.
For your information:
JIRA Bug - Arbitrary remote code execution with InvokerTransformer
Here is the part of my pom.xml, and I guess there's nothing remarkable.
...
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>org.apache.velocity</groupId>
<artifactId>velocity</artifactId>
<version>1.7</version>
</dependency>
...

Unless I am missing something obvious, just specifying dependency in your POM ought to be sufficient:
<dependencies>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.2</version>
<dependency>
...
</dependencies>
If you specify it a the top of your <dependencies> section, it will override any other transitive inclusion of commons-collections.
Of course, you may wind up with incompatibilities where other dependencies depend on the other version, but that's what unit tests are for, right? ;-)

What you need to do is exclude commons-collections from the affected dependencies and include the desired version in your dependencies directly.
Example pom.xml excerpt assuming commons-configuration uses the vulnerable commons-collections
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
<exclusions>
<exclusion>
<artifactId>commons-collections</artifactId>
<groupId>commons-collections</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.2</version>
<scope>runtime</scope>
</dependency>
For simplicity I didn't show configuring this in a root pom.xml in the dependency-management section.
The <scope> should be set to runtime since you mentioned not using the library directly.

I've added these lines in my pom.xml, but still commons-collections3.2 is getting downloaded..
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${apachecommonslang.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-collections</artifactId>
<groupId>commons-collections</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>commons-dbcp</groupId>
<artifactId>commons-dbcp</artifactId>
<version>${dbcp.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-collections</artifactId>
<groupId>commons-collections</groupId>
</exclusion>
</exclusions>
</dependency>

How to preference library in maven

I have imported two library in my project
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-web-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
now I want to use class HttpServletRequest from package javaee-web-api. Problem is that javax servlet contains this class too and application is using this instead of library what I want to. How I can change it ?

You cannot. Remove either one. Both classes are in the same namespace/package. HttpServletRequest is defined by a standard it should behave the same way.

Since you have a comment now that explains that the javax.servlet is coming from the parent pom then I don't think it is possible. It has been discussed before: Is there anyway to exclude artifacts inherited from a parent POM?
If you have any kind of control on the parent pom then you should try to change it to have an optional tag:
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
<optional>true</optional>
</dependency>
Then it won't be included as a transitive dependency for your child project.
http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html

Ok I found it. I can exclude this library:
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
and then it works

Java - getting jar dependencies right

I'm relatively new to Java & maven, and so to get to know my way around, I decided to do a project as a means for learning.
I picked a pretty common stack :
Java 1.6
Hibernate (with annotations)
Spring (with annotations)
JUnit 4
Tomcat
Oracle XE / In-mem hsqldb
By far one of the biggest problems I've experienced is getting the correct combination of jar versions to get a stable environment. It's an issue I'm still fighting with over two months later.
Quite often I get noSuchMethod or classNotFound exceptions thrown, and it turns out to be that Spring module A x.x.x is not compatible with Hibernate module B y.y.y. Or even, just as commonly, spring module A x.x.x is not compatible with spring module B y.y.y
I expected in starting from a clean slate, version dependencies should be minimal -- just grab the latest version and everything should work... but that has not been the case.
I expected that using maven would simplify this process, and no doubt it has.
But it's certainly be far from painless. I'd have thought that if module A requires a specific version of module B, that it be enforced somewhere along the line, and certinaly provide more meaningful messages that just "noSuchMethod".
Additionally, it seems that the only way I discover these problems is to try a new method call, get the dreaded noSuchMethod error, and start googling.
Have I missed something along the way here that has made this more difficult on myself than it needed to be?
For reference, here's the dependencies section of my pom...if you notice anything horrendously non-standard, please let me know!
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.5.6</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.5.6</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>ojdbc</groupId>
<artifactId>ojdbc</artifactId>
<version>14</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>1.4</version>
</dependency>
<dependency><!-- java bytecode processor -->
<groupId>javassist</groupId>
<artifactId>javassist</artifactId>
<version>3.8.0.GA</version>
</dependency>
<dependency>
<groupId>commons-dbcp</groupId>
<artifactId>commons-dbcp</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>1.8.0.7</version>
</dependency>
<dependency>
<groupId>org.dbunit</groupId>
<artifactId>dbunit</artifactId>
<version>2.4.4</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>2.5.6</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>2.5.6</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>2.5.6</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.4.0.GA</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-commons-annotations</artifactId>
<version>3.3.0.ga</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.3.1.GA</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>3.1.0.GA</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>3.4.0.GA</version>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.3</version>
</dependency>
</dependencies>
Thanks
Marty

One thing that I've found challenging is determining what is in each package, especially from Spring.
To that end, I've found Netbeans' support for maven to be outstanding in how it lets you know what libraries are pulled in by each requirement. 6.7 Beta contains a graphical tree which is outstanding, and m2eclipse also has a very nice graphical dependency tree. How else would you know that spring-orm includes, spring-beans, spring-core, spring-context, and spring-tx? You can ask maven for the dependencies using the dependency plugin from the command line, but the graphical representation is quite handy. dependency:tree is the goal you want to run. Obviously you can also run that from Netbeans or Eclipse.
So, as an example of one of your collisions:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.4.0.GA</version>
</dependency>
actually includes hibernate-commons-annotations-3.1.0.GA not 3.3. It also includes hibernate-core-3.3.0.SP1, not 3.3.1.GA.
I would start at your "biggest" component, and start to see what parts that already includes and only add what is missing. Even then, double check that you don't have a duplicate dependency and if need be, exclude the duplicate as shown in the answer to this question.

If you are using eclipse, then you should download the maven plugin from sonatype here http://m2eclipse.sonatype.org/.
This comes with a useful graphic visualisation of your dependencies (in particular transitive dependencies - dependencies you have not explicitly defined in your POM), and also shows conflicting dependencies.
Update: from the comments below, your mileage may vary.