I have many dependencies in my project, almost all of which can be fetched from a remote repository. However, one of the repositories used contains a dependency that I build a custom version of locally using provided tools for that (hence why I'm not just building it to a custom location, figuring that out is a last resort).
I would like to fetch all my dependencies from remote repositories, except for the one I am building locally. I cannot remove the repository that contains it, as it contains more of my dependencies that I DO want to update.
In addition, this is a project shared with others, so a system scope and a systemPath attribute aren't going to work.
I cannot run maven in offline mode, as I do want to update every other dependency regularly.
Thanks.
Let me start with one suggestion, do not try to override existing coordinates with your custom logic and hence the build. It will always add to the confusion and worst runtime exception will make it hard to debug the problem. You should
Change the coordinates of this custom build project - Best possible solution. This will be treated as any other dependency.
If you can't change the coordinates, change the version. Do add a qualifier to make it future proof for further release of this library from original source.
Always prefer to add this dependency in dependency management section. This will always give this library a preference over dependency mediation.
You could use relocation mechanism of distribution management. This will give a useful message to people. There always is a shared settings.xml across a company.
Related
The general architecture
We have an internal java application (let's call it com.example.framework) acting as a kind of framework in the sense of being extensible through plugins. These plugins can serve various purposes. As an example, there will be a plugin for the support of the differnt database providers, e.g., MysqlPlugin, OraclePlugin and MssqlPlugin. On the other hand their might be support for exchange formats such as JSON or XML, etc.
Code splitting
The framework application is developed as a seperate multi-module java project with the parent group id com.example.framework having the API/SPI as a distinct child module. Therefore, the plugins have this api-module as a dependecy called com.example.framework.api, which works perfectly fine. Idealy, each plugin will have its own artifact under a group called com.example.framework.plugins such that I will only have those plugins installed that are really needed.
The problem to solve
To ease the developer experience, I would like to group plugins of similar functionality, which even might want to share a bit of code together to a git project while keeping some special ones alone. Now I wonder what the best way of structuring this in the maven system is.
Current idea
The best solution I could find is to also use the multi-module approach for grouped plugin projects to achieve splitted artifacts while being able to share code between two plugins. However, I am still confused about the groupId and version of the parent:
The naming convention of maven suggests to use a unique groupId for each project. This would mean to introduce another depth of naming, e.g. com.example.framework.plugins.sql.mysql, which would be inconvenient, since the name of a plugin is no longer sufficient to derive the full module name (with the a-priori knowledge of the common package name com.example.framework.plugins). So I wonder, whether the purpose of the convention is soley to avoid possible duplicates by design? Since I control the namespace and all plugins, I would make sure that there are no conflicts.
The actual question
If I were to remove the intermediate name layer and thus have multiple parent poms with the same groupId, what problems could arise? Since plugins would not even share versions, the parent has no real purpose and also no artifact on its own that could collide, or am I missing anything?
Or is my entire structure not ideal and I should adopt some other form? During my research I could not find any similar use-case.
Usually, different related projects share the same groupId. There is no problem in that. The linked Maven page is misleading.
I am facing one problem and not sure what is the best way to go,
So I have two git repositories with Spring projects which have to share the same database. (Both are spring projects with hibernate).
One of them is main project so-called MASTER which should modify all Hibernate entities and other I will call SLAVE which is secondary project and needs to read only from the same database.
Here is the small illustration what I have.
So the issue appears when I realized that need to keep the duplicate of entities in both master and slave.
I found two ways to go with this issue.
Using git submodules. Where I can have my entities to be an independent submodule.
Building a JAR from entity classes and include it in both projects.
This both solutions are not meeting my requirements which are:
The solution of submodules is not good because whenever I commit anything from the MASTER I want SLAVE to track that changes. Please note, I have 3 git branches for both projects, master, staging and production. So all the branches should have accordingly their version of entities.
The solution of the jars will work, just I do not find it nice and solid, as I should all the time build them and add a dependency for every project.
The development of these projects is done independently from each other.
Please, could you share your opinion on this issue?
I kind of sure that I am not only the one who is trying to achieve the same.
You should consider publishing your jar to a maven repository for easier exchange between the projects. You could even host your own like sonar nexus. https://www.sonatype.com/nexus-repository-sonatype
Personally, I think that managing the versions can be very annoying when you have multiple projects. Especially when you are testing something and you have to create a new jar and then publish it over and over again. However your project will be rebuildable and you can controll which project/module can use a newer version of your entity-dependency.
Specific Background
I have just switched from spring data neo4j 4.1.3 to 5.0.0
And this issue has arisen since I changed my pom file.
Maven install fails because "cannot find symbol ... class GraphRepository"
I am newer to Java Maven projects as a whole
Broad Question:
If I update maven dependencies on a given project from one version of something to another and a class that I have been using heavily now gives around 100 error codes saying that whole class is now missing... how do I have this not happen.
Specific Where I think I'm at
I am gonna have to remove every reference to the "GraphRepository" and change it to Neo4jRepository since "Also note that GraphRepository is deprecated and replaced by Neo4jRepository" - Neo4j 4.2 graph repo save method is now ambiguous
But, this just doesn't seem right. Do I really have to go through an entire project and change all that code just to update?
One full line of error:
[ERROR] /.../service/SupportModelServiceImpl.java:[10,49] cannot find symbol
symbol: class GraphRepository
location: package org.springframework.data.neo4j.repository
You cannot prevent external dependencies from introducing breaking changes. However you could write your code so that it takes minimal effort to update external dependencies.
I have observed that in practice not much care is given to dependencies as if they are free. Initially they are as good as free, but once you start stacking your dependencies and have transitive dependencies that conflict or you upgrade to a new version with breaking changes there comes a maintenance cost. I have seen projects where the web of dependencies is so complex that they should be rewritten completely from scratch, if not for management not understanding the concept of technical debt, living in the illusion that maintaining an existing (bad) version of the software is cheaper than writing a new one.
The only option you have to guard against external dependencies, is to encapsulate them in one way or another. This may involve some boiler plate code, though if this boiler plate code is minimal it may be well worth the effort.
Because I have seen projects with horrible dependencies, I have given it some thought how I could prevent such a dependency mess and made the following image:
External code, over which you have no control is in red. If you do not think about structuring your code, your code (in orange) will depend directly on the external code and is at risk for external changes. You can try to write code (in green) that has no dependencies on external code. The way you achieve this is that you define the external functionality that you need in your own interfaces. You have then some code (in orange) that implements these interfaces and has external dependencies. You inject the code with external dependencies through a dependency injection framework.
This approach limits the impact of external changes to only the code in orange. However it requires more planning than directly using dependencies everywhere in your code. And because more planning means more effort, it is often not put in practice.
This is not specific to maven, you will have this issue regardless of whatever build system you use.
but I do not understand why you would want this, a major version change (e.g 4.xx to 5.xx) means something is going to break, and you will have to make changes into your code.
I am trying to set up a local maven repository using nexus to store ALL of the maven dependency libraries to be use in an offline environment. We need to be able to point our maven to this repository instead of trying to query the internet for a needed project library. There is no option of using the internet in our environment so I need an easy(ish) way to download literally every library maven could need in order to populate our repository. I am not asking for an opinion on what is the easiest, I just need a few methodologies that would work because I have no idea where to start with this other than spending the next year clicking links and downloading. I have found methods online that explain how to store project specific dependencies but I can't find a way to get ALL of the dependencies. I also know these libraries are continuous changing and be upgraded, so any ideas on how to keep them up to date without individually clicking and downloading each one would be great as well. Thanks!
MavenCentral is huge. Downloading it not only takes ages, they will block you if you try.
I understand very well that you cannot allow your developers to access MavenCentral during their development process. Our solution for this problem is as follows:
We have a proxy of MavenCentral that is only accessible to a few admins. If a developer needs a dependency that is not on our development Nexus, he writes a ticket to the admins. They check the dependency and if they approve, they start a job that copies the required dependency (together with all transitive dependencies) from the MavenCentral proxy to the development Nexus. This way, the admins have full control over what the developers use, but you need not copy MavenCentral as a whole.
I have a system consisting of multiple web applications (war) and libraries (jar). All of them are using maven and are under my control (source code, built artifacts in Nexus,...). Let say that application A is using library L1 directly and L2 indirectly (it is used from L1). I can easily check the dependency tree top-down from the application, using maven's dependency:tree or graph:project plugins. But how can I check, who's using my library? From my example, I want to know, whether A is the only application (or library) using L1 and that L2 is used from L1 and from some other application, let say B. Is there any plugin for maven or nexus or should I try to write some script for that? What are your suggestions?
If you wish to achieve this on a repository level, Apache Archiva has a "used by" feature listed under project information
.
This is similar to what mvnrepository.com lists under its "used by" section of an artifact description.
Unfortunately, Nexus does not seem to provide an equivalent feature.
Now I suppose it would be a hassle to maintain yet another repository just for that, but then it would probably easier than what some other answers suggestions, such as writing a plugin to Nexus. I believe Archiva can be configured to proxy other repositories.
Update
In fact, there's also a plugin for Nexus to achieve the "used by" feature.
As far as I know nothing along these lines exists as an open source tool. You could write a Nexus plugin that traverses a repo and checks for usages of your component in all other components by iterating through all the pom's and analyzing them. This would be a rather heavy task to run though since it would have to look at all components and parse all the poms.
In a similar fashion you could do it on a local repository with some other tool. However it probably makes more sense to parse the contents of a repo manager rather than a local repository.
I don't think there's a Maven way to do this. That being said, there are ways of doing this or similar things. Here's a handful examples:
Open up your projects in your favorite IDE. For instance Eclipse will help you with impact analysis on a class level, which most of the time might be good enough
Use a simple "grep" on your source directory. This sounds a bit brusk (as well as stating the obvious), perhaps, but we've used this a lot
Use dependency analysis tools such as Sonargraph or Lattix
I am not aware of any public libraries for this job, so I wrote a customized app which does it for me.
I work with a distribution which involves more than 70 artifacts bundled together. Many times after modifying an artifact, I want to ensure changes are backward compatible (i.e. no compilation errors are introduced in dependent artifacts). To achieve this, it was crucial to know all dependents of modified artifact.
Hence, I wrote an app which scans through all artifacts under a directory(/subdirectories), extracts their pom.xml and searches (in dependency section of pom) for occurrence of modified artifact.
(I did this in java although shell/windows script can do this even more compactly.)
I'll be happy to share code on github, if that could be of any help.
One way that might suit your needs are to create a master-pom with all your maven projects. Then you run the following command on the master-pom:
mvn dependency:tree -DoutputType=graphml -DoutputFile=dependency.graphml
Open the generated file in yEd.
Used the instructions found here:
http://www.summa-tech.com/blog/2011/04/12/a-visual-maven-dependency-tree-view/
More interesting is probably: what would you do with this information? Inform the developers of A not to use library L1 or L2 anymore, because it has a critical bug?
In my opinion you should be able to create a blacklist of dependencies/parents/plugins on your repository manager. Once a project tries to deploy/upload itself with a blacklisted artifact, it should fail. I'm saying uploading and not downloading, because that might break a lot of projects. As far as I know, this is not yet available for any repository-manager.
One of the ways to approach this problem is outside Java itself : write an OS-level monitoring script that tracks each case of fopen() on the jar file under question! Assuming this is in a corporate environemnt, you might have to wait for a few weeks (!) to allow all using processes to access the library at least once!
On Windows, you might use Sysinternals Process Monitor to do this:
http://technet.microsoft.com/en-us/sysinternals/bb896645
On Unix variants, you would use DTrace or strace.
IMHO and also from my experience, looking for a technical solution for such a problem is often an overkill. If the reason why you want to know who is using your artifact(library) is because you want to ensure backward compatibility when you change an artifact or something similar, I think it is best done by communicating your changes using traditional channels and also encourage other teams who might be using your library to talk about it (project blogs, wiki, email, a well known location where documentations are put, Jour fixe etc.).
In theory, you could write a script that crawls though each project in your repository and then parses the maven build.xml (assuming they all use maven) and see whether they have defined a dependency to your artifact. If all the projects in your organization follows the standard maven structure, it should be easy to write one such script (though if any of those projects have a dependency to your artifact via a transitive dependency, things can get a bit more tricky).