maven - using local source instead of external dependency - java

If someone could help me out here it would save me a lot of time.
I maintain an open source library that gets pushed out to a sonatype repository. I make changes to that library a few times a day and push it out to the 1.0_snapshot build using mvn deploy.
Let's call it project1
I work constantly in another project that uses that library let's call it project2.
Right now, whenever i make changes to project 1 or 2, i need to first build and deploy project 1 to the repo, then build project 2 so it downloads a fresh copy of project1.jar
Project2 has Project1 as a dependency in a pom.xml:
<dependency>
<groupId>com.group</groupId>
<artifactId>project1</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
in order to build in a way where all of my changes can be tested, I have to do something like this:
mvn -f ./project1/pom.xml clean deploy
mvn -U -f ./project2/pom.xml clean package
this uploads my project1.jar to sonatype, then project2 downloads the new snapshot and builds it.
This is a simplified picture of what i'm doing on a larger scale, where my compiles take 5 minutes of up and downloads.
Question: What is the proper way to use maven so it knows to use the source of project1 in a dependency in project 2?

IDE:
install m2e in eclipse
import your both projects into workspace
from consumer project (right click > maven > enable workspace resolution)
this will put project2's classes in classpath from its target/classes instead of the actual jar
native straight maven:
You can create a maven project tree, if this two are open source project going through same build cycle it must have one already, if they are not related but related for your usecase then you can temporarily create a maven tree on top of these 2 projects and build the top parent it will build from bottom up in one command
it will find the leaf project, build it install it in maven's cache and now while building projectA it will refer it from maven's cache so no need to deploy to sonatype

You can also point project2 to build using offline mode :
mvn -o package
Then you drop the upload part of the project1 build to the remote repo.
Check the following link: Intro to repositories and How do I configure Maven for offline development?

Related

How can I recover maven dependency issues after changing my Intellij license user account to a new account?

I changed the Intellij license user account I have been using and all my maven dependencies are no longer visible to my spring boot project. I deleted the repository folder in C:\Users{My Username}.m2, executed mvn dependency:tree and run mvn clean install. This has recovered my maven dependencies downloaded online. However, the dependencies included as POM from other projects residing on my system are still not visible.
You need to install other projects so that they are in the local repository and visible to the current project.
Even without IntelliJ, you can simply go to each of your other local project folders and do a:
mvn dependency:resolve
Using that resolve mojo, that would be enough to download the dependencies of those projects back to your new %USERPROFILE%\.m2 cache folder.

Build Maven project with local dependencies

I downloaded old project, which based on Eclipse, Ant or something else, I don't know this build system (I use Maven\Gradle). Project sources was converted to Maven manually, but project had more jar libraries. They were imported in project with help of IDEA (Project Structure -> Modules -> Dependencies), but libraries defined only in .iml file.
When I tried to build project in .jar with help of Maven — Maven show errors, that it cannot find classes from libraries (but in sources all good). I found in Internet example, like in this code sample:
<dependency>
<groupId>example</groupId>
<artifactId>example</artifactId>
<version>examle</version>
<scope>system</scope>
<systemPath>${project.basedir}/libs/example.jar</systemPath>
</dependency>
But this don't help. Libraries are old too, and they created by old build tools, which don't write artifact id, group id and etc., so I don't know this information.
How I can write dependencies on local .jar libraries in pom.xml if I don't know information (group, artifact ids) about they?
UPD_0:
When I try to set only system path to library, I take this:
All you need is set correct path in the
<systemPath>${project.basedir}/libs/example.jar</systemPath>
group, artifact ids and version are user defined information. So you can define it as you wish.
Try to install this jar in ur local repository from command line / terminal like this. Then add the dependency with the package and version given by you in the command without scope as system
mvn install:install-file -Dfile=<path>/example.jar
-DgroupId=com.something
-DartifactId=example
-Dversion=<give some version>
-Dpackaging=jar
-DgeneratePom=true

maven- not downloading new added dependency in pom.xml file

I need new third party jar for reading csv in maven based project. So, I did entry in pom.xml for same as below.
<dependency>
<groupId>net.sf.opencsv</groupId>
<artifactId>opencsv</artifactId>
<version>2.0</version>
</dependency>
But when I run mvn install or mvn package command, It does not download newly added dependency and just build project and generate war.
I am trying to get the issue! Please share solution if anybody face this earlier!
Regards
Try running a forced update:
mvn clean install -U
The -U (uppercase U) forces maven to look at all dependencies and try to update them.
If the dependency is defined in a <dependencies> block that is within a <dependencyManagement> block, adding it without the version number to a <dependencies> block that is outside <dependencyManagement> may fix the problem.
This is because the purpose of <dependencyManagement> block is to manage dependency versions, and not to install the dependencies. See this other article: Differences between dependencyManagement and dependencies in Maven
I resolved this issue by following steps:
1). Remove concerned jar from local /m2 folder.
2). Run mvn eclipse:eclipse command
3). And last run: mvn clean install
Now I am looking for concerned jar in my project class path!
If you are using IntelliJ Idea as your editor then simply follow 3 simple steps:
Right click on your project
Select Maven (last option probably)
Select "Reload project"
And that's it, IntelliJ Idea will download the dependencies and now you can proceed further.
Try:
Menu -> Project -> Clean -> Select the project
Right Click on the project -> Maven -> Maven clean
Right Click on the project -> Maven -> Maven install
Happened to me and it has fixed my problem. Hope it helps you.
I deleted .m2 folder and then from eclipse ran maven install then took maven update project. It resolved my issue and jar file got downloaded.
You can usually resolve these errors by updating Maven dependencies as follows:
Right-click on your top-level project (not on the pom.xml file) in the Project Explorer view.
From the menu, choose Maven > Update project
Make sure ForceUpdate of Snapshots/Releases is checked, and click OK.
You'll see a progress indicator in the lower-right-hand corner of the application window. When the update completes, you should be able to generate code normally, and the error markers should disappear.
In IntelliJ
Right-click on your root folder of the project in the Project Explorer view. From the menu, choose Maven > Reload project.
After that, your new dependencies will be downloaded. Then you should be able to generate code normally, and all the error markers will disappear.

Maven: How to include jars in Eclipse, which are not available in repository?

I have copied the JARs into src\main\webapp\WEB-INF\lib.
I use eclipse. If I add the jars one-by-one to Project-> Java Build Path-> Add jars, then I do Project-> Maven-> Update Project Configuration, they are removed by Maven. And Eclipse shows errors, which contain " xxx cannot be resolved".
Env:
Eclipse Java EE IDE for Web Developers.
Version: Indigo Service Release 1
Build id: 20110916-0149
m2e - Maven Integration for Eclipse 1.0.100.20110804-1717
Note: I don't want to create my own Maven repository. It will be just used once.
How should I proceed ?
Not all libraries can be found in a public Maven repository, for example your own libraries or proprietary libraries. Anyway first you can search the Maven Repository, maybe some of them are there.
In case they are really not there, and you don't want to install Nexus or Artifactory, you can choose one of these two approaches:
install the jars in your local repository cache, no need to create a special repository. See the instructions on mkyong.
another approach is using system dependencies, you just put a path to reach the Jar in the filesystem. It's a less recommended approach, but if you really want it will work. Here is the official documentation.
Add this an a Systems Dependency.
In case you want to add this (this uses jquantlib as example) to the maven local repo use:
mvn install:install-file -Dfile=./jquantlib-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib -Dversion=0.2.4 -Dpackaging=jar
In order to do this I use a batch file
#echo off
cd lib
CMD /C "mvn install:install-file -Dfile=./jquantlib-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-helpers-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-helpers -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-samples-0.2.4-ubber.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-ubber -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-samples-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-samples -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./ta-lib-0.4.0.jar -DgroupId=com.tictactec -DartifactId=ta-lib -Dversion=0.4.0 -Dpackaging=jar"
You can use a similar script file on other systems.
I believe the system dependency approach shouldn't be used unless you don't have any other choice, and that's because you're loosing the whole 'build portability thing' here.
Of course you can store your jars in your source control system together with your project's source files, but I don't think its a good approach neither...
Using only install:install-file is not good enough - this would indeed deploy the jars in the proper format into your local repository, but what happens when you'll move to another computer and start to build your project there?
You will need to make this once more.
So, If you don't want to install nexus/artifactory (which is the best solution, I believe), you probably should create an another repository (just in a file system on some of your servers), and deploy the jars there (you can use mvn install:install-file as was suggested here, and then just copy the whole tree). Now you can configure apache web server and access the directory with all your jars via http. I don't believe its better then nexus/artifactory approach, but it can be a little be easier to do if you're familiar with apache web server.
In order to get your maven aware about this new repository you'll need to edit the %MAVEN_HOME%\conf\settings.xml file
There are atleast three approaches in which 3rd party JARs can be added to Maven projects.
Install manually using mvn install command
Adding the location of jar file in pom dependency with the the following tag system
Creating a 'dummy' maven repository pointing to jar location.
While approach 1 and 2 has been suggested above, I will focus on third approach which I find more cleaner and does not require any mvn command and works out of box from any IDE.
Step 1: Add the location of local 'dummy' repository in pom.xml
<repositories>
<repository>
<id>repo</id>
<name>repo</name>
<url>file:${project.basedir}/src/main/resources/lib</url>
</repository>
</repositories>
Here the 'dummy' repository location is the 'lib' folder of my project directory
Step 2 : Add the jar dependency into your pom.xml
<dependency>
<groupId>com.cloudera.impala</groupId>
<artifactId>impala-frontend</artifactId>
<version>0.1-SNAPSHOT</version>
</dependency>
choose any groupId but make sure that artifactId and version is of the format <artifactId>-<version>.jar ( Name of 3rd party jar)
Step 3 : Create the folder structure as per the groupId,artifactId and version mentioned in the Step 2 in your local 'dummy' repository. So in this case the folder struction would be /src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/
Place your jar in the version folder and build your project.
You will get the following output which treats your 'dummy' repository to be the provider of your 3rd party jar.
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/maven-metadata.xml
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.pom
[WARNING] The POM for com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT is missing, no dependency information available
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar
[WARNING] Could not validate integrity of download from file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar: Checksum validation failed, no checksums available
[WARNING] Checksum validation failed, no checksums available from repo for file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar
[INFO] Downloaded from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar (7.0 MB at 79 MB/s)
[INFO]
To add external JAR files not in the local repository simply right click on your main source folder and from the build path menu select: "configure build path", then navigate to the libraries tab and click "add external JAR files". next, locate the JAR (or zip) file you would like to add as a library and click ok.
Congratulations, you have now successfully added an external JAR (or zip) to your build path and you can now import any classes from that JAR file in your project without throwing an errorPicture How-To :) http://hostthenpost.org/uploads/541be8420657320c74489ff8d456ad08.png

Error opening Maven POM file dependency hierarchy in Eclipse - "Project read error"

When I open a POM file and click on the "Dependency Hierarchy" tab at the bottom, it gives me the error, "Project read error". It works with other projects in the same workspace, just not with this one. Any ideas?
EDIT
In response to #Yhn's answer.
Running the compile and package phases outside of Eclipse from the command-line work as expected. It compiles the application and builds the final WAR file.
Eclipse is indeed pointing to the default location of the Maven settings.xml file, so it should be aware of the custom repositories that are defined in it (my company has its own Maven repository).
I can open and edit the POM file from Eclipse, so it must have read/write permissions to the file.
The project is not configured in Eclipse as a Maven project, so I cannot run the package phase from Eclipse (I can only run it from the command-line).
I wonder if it has anything to do with the fact that I was having trouble building the project with Maven 3 because apparently some of the transitive dependencies are configured for Maven 1, which Maven 3 does not support (this is my theory anyway, based on some of the error messages). I can build the project with Maven 2, but I still get messages such as the following:
Downloading: http://dist.codehaus.org/mule/dependencies/maven2/org/codehaus/xfie/bcprov-jdk14/133/bcprov-jdk14-133.pom
[INFO] Unable to find resource 'org.codehaus.xfire:bcprov-jdk14:pom:133' in repsitory mule (http://dist.codehaus.org/mule/dependencies/maven2)
It must be able to find these dependences however, because it downloaded the JARs just fine and can build the application. It seems like the problem is that the dependencies don't have POM files associated with them, which is maybe why they cannot be used with Maven 3. This might also be why I cannot view the Dependency Hierarchy in Eclipse.
EDIT 2
I converted the project to a Maven project by going to "Configure > Convert to Maven Project". When I open the POM file, I see the error:
ArtifactDescriptorException: Failed to read artifact descriptor for woodstox:wst (Click for 140 more)
(woodstox:wst is another transitive dependency of the project). An error appears in the "Markers" view for seemingly every depedency and transitive dependency in my project. However, I can successfully build the project by doing a "Run As > Maven build". (Edit: This might be because this project has no Java source code, but the JARs of the dependencies correctly appear in the final WAR.) The Dependency Hierarchy still gives the same error--"Project read error".
About the "Unable to find resource" messages--but this only appears for a handful of transitive dependencies. The project has many more transitive dependencies, but these messages do not appear for them. It seems like, because the dependencies do not have POM files, that Maven tries to search for them every time the project is built. Is this normal not to have POMs??
How might I go about getting a repo manager? Is this something that would have to be installed on the company's Maven repository or can you install it on your own workstation?
I had this problem with some non-maven jars that I needed to include in my maven project. I put the jars in my local repository using this maven command:
mvn install:install-file -Dfile=/test/gov.nist.xccdf-1.2.jar -DgroupId=gov.nist -DartifactId=xccdf -Dpackaging=jar -Dversion=1.2
Then I referred to them as dependencies in my pom.xml:
<dependency>
<groupId>gov.nist</groupId>
<artifactId>xccdf</artifactId>
<version>1.2</version>
<type>jar</type>
<scope>compile</scope>
</dependency>
This worked fine for a while, but I must have upgraded something in eclipse, and I started getting the error you describe.
The fix is to take them out of the local repository, and redo the install-file asking maven to generate a pom for you:
mvn -DgeneratePom=true install:install-file -Dfile=/test/gov.nist.xccdf-1.2.jar -DgroupId=gov.nist -DartifactId=xccdf -Dpackaging=jar -Dversion=1.2
The command will cause a pom to be generated in the same directory (within your local repo) where the jar is placed.
In recent versions of eclipse maven support, the maven build is still done with maven 2 (if you right-click your pom.xml, choose run as mvn package for example). The dependency analysis is now done with an embedded maven 3 plugin, which doesn't like a dependency that has no pom.xml.
I had the same problem. If you made your project a maven project, you should run:
Right Mouse Click on project | Maven | Update Dependencies or
Right Mouse Click on project | Maven | Update Project Configuration
That worked for me.
Given the information through the comments:
mvn compile/mvn package work through console (I'm assuming outside of eclipse). Based on that, the POM file should be correct and dependencies can be resolved.
However; it could be that eclipse's settings aren't correct. For example, if eclipse has some own maven properties (instead of the default ~/.m2/settings.xml file); I can imagine it to fail resolving dependencies configured in the settings.xml used by maven itself. The settings for this should be in the Eclipse Preferences # Maven > User Settings.
Also make sure that Eclipse can actually read the file (not locked, correct rights, etc).
Should this seem to be correct; can you try to run the mvn goal compile from eclipse? You can do this by right-clicking the project and selecting run as > maven package. That way it should run maven with the same settings as eclipse is using, and might show any additional errors in it's configuration.
--
In response to additional information:
Basically it tells you - when using Maven 2 - that it can't find a dependency resource (in this case bcprov-jdk14) in the given repository (codehaus/mule). When I search that maven module (bcprov-jdk14) I find it in the Maven central repo (through our local nexus) # repo1.maven.org/maven2. Maybe that causes the error you're getting with Maven2?
And as the Maven 3 doc says; for Maven 1.x repo's; you should proxy them through a repo manager that can serve it to you as a maven 2 repository (I believe we do that here too for some repo's through Sonatype Nexus OSS)
ps.: Enabling maven dependency management in eclipse should be enough to be able to use maven run targets; if you would want to test it.
--
Sound like it can be multiple problems... given that the normal maven build works fine, one would think that maven can at least get the artifacts it depends upon and their dependencies... Yet, eclipse doesn't seem to be able to.
Double check if your Eclipse internet connection settings are correct? Since you were talking about a company repo, I'm assuming there will probably also be a proxy (I had to set up my Eclipse to use that one too; though for plugins (see next point)).
Another possibility, from experience, can be a rule-based proxy block on *.pom url requests (our proxy here blocks that to force usage of the local repo manager, how evil and annoying that is :(). You might want to try and open the .pom file it tries to download manually in your browser to see if this is the case (I can't, for example...).
To get the repo set up, you would probably have to ask the one responsible for the repository to add it as a maven2 proxy.
At the very least, it seems Eclipse is having issues getting the dependency poms (which are needed to build the dependency tree) from the internet. You could always try to ask some ICT crew (if they are capable enough...) about it at your company, maybe they have some useful hints.
right click on the project, Maven->Update Project->Tick "Force Update of Snapshots/Releases"
I had the same problem when I added a new dependency to pom.xml without network connection. After this, I had so many problems such as "Missing artifacts", "Missing Descriptor", or "Project Read error". I solved this
delete the bad dependency just added from local repository.
rebuild local index.
re-add the dependency
<properties>
<!-- ********************** -->
<!-- Plugin's properties -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.7</java.version>
</properties>
this pom setting worked for me
Usually this happens if there is a typo in the pom that Maven cannot understand and then it throws Project read error. Like in my case I copied a maven dependency snippet and I by passed the version. Instead of typing the actual version # it was getting it dynamically ${asciidoctorj.version} and my pom didn't have this in the properties section. So I got this error. Once I removed that and put a version number it started working.
I had this after copying a local repository folder from another machine. The fix was to go into the repository folder I copied and delete
_remote.repositories along with the 'LastUpdated' files then refresh the Maven dependencies in Eclipse (Alt+F5)
I faced same situation today. In my case it's caused by dependency name in wrong case. E.g.
Project A -> Project B -> Project C
In project B's pm file, I mistakenly specified the dependency artifacts name with "c" in stead of "C".
As in mac os, the files system is case insensitive. so I can build it in command line without detecting this mistake.
In Eclipse it breaks, but it provides very bad error message. In the error list, it says "dependency problem", but the name of dependency is empty. It also can't identify which line in the POM causes the problem.
When try to open "Effective POM" in POM viewer, it will show project read error. In the pop up error dialog box, it will show:
Could not read maven project
java.nio.channels.OverlappingFileLockException
All the messages are useless and misleading.
I finally detected this problem by submit to Jenkins CI after wasted hours of time.
I did maven clean install and the errors were gone
I faced the same issue of "Project read error" while clicking on "Dependency Hierarchy" tab. In my project I had a parent pom and child pom.
Parent pom had a property <appVersion>4.5<appVersion> that was being used in child pom
<version>${appVersion}<version>.
I selected the main project, right click and choose run as maven clean. The console display showed warning that version is using expression but should be a constant. Replacing ${appVersion} with constant value of 4.5 fixed the issue.

Categories