I have a huge amount of local jar dependencies for legacy ant project. The names don't follow artifactId-version-classifier.jar pattern. I want to replace such jars with artifacts from central repository if possible.
Is there a way to do it?
That is usually a lot of work.
You can compute the checksums and look them up. You can also look up the name and the version (if you know it).
Furthermore, the project probably only uses a small fraction as direct dependencies. The rest are transitive dependencies. So if you figure out the direct dependencies (from looking at the code) than resolving those would be enough. Maven will find the transitive ones automatically.
Related
I see on maven repository, there are two types of dependencies while I was trying to add into my pom.xml of springBoot Application.
artifactId = aws-java-sdk and aws-java-sdk-bom
What's is the difference between the normal one and the one with the "bom". How do we know which one is a better fit for our requirement?
From the Maven guide:
Imports are most effective when used for defining a "library" of related artifacts that are generally part of a multiproject build. It is fairly common for one project to use one or more artifacts from these libraries. However, it has sometimes been difficult to keep the versions in the project using the artifacts in synch with the versions distributed in the library. The pattern below illustrates how a "bill of materials" (BOM) can be created for use by other projects.
Read more here: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms
I have an old ant-based build I must move to be Maven-based for our new build system. This old build is based on many well known libraries (Apache commons, etc.) which pose no problem to Maven, but it uses also about a dozen JARs in a subdirectory that are less well known or not known at all (can't find any information on) that I must include statically.
I have chosen this approach for including these more static JARs that I cannot get from Maven the way one would normally get them:
<repository>
<id>Pile of misfit JARs</id>
<url>file://${basedir}/lib</url>
</repository>
However, when it comes to creating <dependency> statements for these misfits, in the case of most of the JARs all I have is knowledge of the Java package paths in the code. I don't know how to discover what to use as groupId and artifactId. I have looked inside these JARs; some do have good information in manifests, but others do not or have only partial information.
(Incidentally, to use the <repository> construction above, I have to make use of $ mvn install:install-file ... and name groupId, artifactId and version too in addition to what I put into <dependency> in order to make the local, in-project repository work.)
I'm very grateful for any suggestions!
I will provide few outline.
Apart from well known jar files like Apache commons etc, you have to look into Manifest.mf file inside the unidentified jar files. You will find some name, company name etc.
Then go to mvnrepository.com and search for the name or company name which you found from the unidentified jar files.
If you get the actual artifact detail from mvnrepository.com, you can include group id, artifact id etc in maven dependency.
How ever all the above approach is time consuming.
There is another approach to get rid of. If you have latest version of Artifactory. Upload the unidentified jar files into Artifactory, it will either identify the details or it will generate the group id, artifact id version etc. If you want, you can provide your own artifact details also.
You can also refer to link.
http://roufid.com/3-ways-to-add-local-jar-to-maven-project/
The easiest way is to upload those jars to your Maven repo with some fake groupIds/artifactIds/versions using Maven deploy plugin. In this case you will be 100% sure that your Maven build uses the very same jars as your Ant build. Though you will still need to try to find our actual versions if you need to upgrade to newer version.
why use Maven when you have such quantity of local jars?
So we have a client that have a lot of private jars and custom jars.
For example commons-langMyCompanyCustom.jar which is commons-lang.jar with 10 more classes in it.
So on their environment we use 100% Maven without local dependencies.
But on our site we have the jars for development in Eclipse and have Maven build with the public ones, but we do not have permission to add their jars in our organizational repository.
So we want to use the Maven good things like: compile,test, build uber-jar, add static code analysis, generate java-docs, sources-jars etc. not to do this thinks one by one with the help of Eclipse.
So we have 70 jar some of them are public if I get the effective pom on their environment I found 50 of them in Maven Central, but the other 20 are as I called "custom" jars. I searched for decision of course but found this:
<dependency>
<groupId>sample</groupId>
<artifactId>com.sample</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/yourJar.jar</systemPath>
</dependency>
So for all 20 of them I have to add this in the development maven profile??
Is there a easy way like in Gradle where you can add all folder with its dependencies to the existing ones?
Also installing one by one in every developer's repo is not acceptable.
Please forget the system scope as mentioned before! Too problematic...
Ideally:
Ideally, all your developers have access to Repository Manager in your or their organization (if possible).
A central environment for your System Integration Testing, maybe?
Alternatively, you may have a central environment for testing where all the dependencies are provided. This approach can be used to simulate how a compilation would work as if it's in your client's environment. Plus you only setup jars once.
So on their environment we use 100% Maven without local dependencies.
But on our site we have the jars for development in Eclipse and have
Maven build with the public ones, but we do not have permission to add
their jars in our organizational repository.
According to what you're saying in the above-quoted excerpt I believe you want to have set in your build's pom.xml assuming that in the client setup the dependencies will be present.
Especially, as you indicate that the organization doesn't give you permission to add their jars in your repository, I would use the provided scope.
As stated in the Maven docs, the definition of a provided dependency is as followed:
This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. This scope is only available on the compilation and test classpath, and is not transitive.
So basically you assume that these dependencies will be present at your client's setup. However, this has some limitations. Meaning you can build solutions independently but cannot test it locally because you won't have the dependencies on your workstation.
If you won't even have access to the jars to configure your central environment ask if your client can provide a DEV/SIT environment.
None of the above? Inherit a parent pom.
To avoid the whole constant copy-paste process for every single (related) project, maven has the tools to centralize dependency and plugin configurations, one of such is by inheriting the configuration of a parent pom. As is explaining in the following documentation it is quite simple:
First you create a project with just a pom.xml where you define everything you wish to centralize (watch out, certain items have slight differences in their constructs);
Use as property of packaging tag the option pom: <packaging>pom</packaging>;
In the pom's that have to inherit these configurations set the parent configuration tags in <parent> ... </parent> (documentation is very clear with this);
Now everytime you update any "global" pom configuration only the parent version has to be updated on every project. As a result of this, you only need to configure everything once.
You can also apply this together with the abovementioned solutions, this way combining to find a solution that fits best to your needs.
But there is a big Maven world out there, so I advise a good read in its doc's to further acknowledge your possibilities. I remembered these situations because I've been in a similar situation you seem to be now.
Good luck!
Another alternative is the project RepoTree.
This one creates a Maven repository directory (not a server) from another directory which contains just the .jars. In other words, it creates the necessary .pom files and directory structure. It takes into account only the precise information from metadata contained in the archives (MANIFEST.MF, pom.xml).
Utility to recursively install artifacts from a directory into a local
Maven repository Based on Aether 1.7
This is 5 years old, but still should work fine.
TL;DR: MavenHoe creates a Maven repository server (not a directory) which serves the artefacts from a directory, guessing what you ask for if needed. The purpose is to avoid complicated version synchronizing - it simply takes whatever is closest to the requested G:A:V.
I have moved the MavenHoe project, which almost got lost with the decline of Google Code, to Github. Therefore I put it here for availability in the form of a full answer:
One of the options you have when dealing with conditions like that is to take whatever comes in form of a directory with .jar's and treat it as a repository.
Some time ago I have written a tool for that purpose. My situation was that we were building JBoss EAP and recompiled every single dependency.
That resulted in thousands of .jars which were most often the same as their Central counterpart (plus security and bug fixes).
I needed the tests to run against these artifacts rather than the Central ones. However, the Maven coordinates were the same.
Therefore, I wrote this "Maven repository/proxy" which provided the artifact if it found something that could be it, and if not, it proxied the request to Central.
It can derive the G:A:V from three sources:
MANIFEST.MF
META-INF/.../pom.xml
Location of the file in the directory, in combination with a configuration file like this:
jboss-managed.jar org/jboss/man/ jboss-managed 2.1.0.SP1 jboss-managed-2.1.0.SP1.jar
getopt.jar gnu-getopt/ getopt 1.0.12-brew getopt-1.0.12-brew.jar
jboss-kernel.jar org/jboss/microcontainer/ jboss-kernel 2.0.6.GA jboss-kernel-2.0.6.GA.jar
jboss-logging-spi.jar org/jboss/logging/ jboss-logging-spi 2.1.0.GA jboss-logging-spi-2.1.0.GA.jar
...
The first column is the filename in the .zip; Then groupId (with either slashes or dots), artifactId, version, artifact file name, respectively.
Your 70 files would be listed in this file.
See more information at this page:
https://rawgit.com/OndraZizka/MavenHoe/master/docs/README.html
The project is available here.
Feel free to fork and push further, if you don't find anything better.
I use dbus-java library in my own library. It depends on unix-java and some more. Those jars are not present in any maven repo.
How would I explicitly depend on all of these?
I see several options:
send jars to maven repo by myself (though it's not clear for me - how to preserve their groupId?)
package all the jar's into mine (which is obviously bad)
write in README: "apt-get install dbus-java-bin" and what to include in classpath... but it makes me really sad :(
Note: I came from Ruby land, so I'm relative new to all these weird Maven repos and confused by missing jars everywhere. In Ruby I was always sure that I will be able to retrieve all the gems either from rubygems or from a specified git repo (usually on github).
Could you explain how is better to distribute such libraries?
What I would do is to download the jars from the net and install them in my local-global repository.
(By this I mean the repository that is not local on my machine, but local to the company, often this is managed by Nexus).
You just need to set a pom with
<groupId>, <artifactId> and <version>.
Then, in your pom, you point to them in your dependencies list.
mvn deploy
By the way, if you wander what the groupId should be, you have two options:
com.yourcompany.trirdparty
or
com.whatever.the.original.groupid.is.groupId
Is there any way to reduce jar file size?
I want a tool that reduces the unused dependencies.
I use maven for dependency management.
A JAR file doesn't normally contain dependencies in the Maven sense. So you must be talking about:
a WAR or EAR or similar file,
a so-called UberJAR file produced by combining lots of JAR files; e.g. using the Maven shade plugin, or
dependencies at a finer granularity than Maven modules.
In the first two cases, you can keep out nominally dependent JARs by excluding them, either in the dependency specification, or in the war or shade plugin build descriptor. IIRC, the shade plugin also allows you to exclude specific packages and classes.
The last may require a separate tool to post-process the JAR file. Getting rid of unused classes is the kind of thing that an obfuscator can do. However, you need to be careful not to eliminate classes or class names that are used reflectively; e.g. by a DI / IoC framework or an AOP framework.
(Generally speaking, this kind of tool tries to figure out what classes are used by analysing the dependencies implied by .class file external references. DI / IoC / AOP and so on introduce other kinds of dependency that are not apparent in the .class file structure.)
If you like to know the dependencies your project uses just check the maven-dependency-plugin which can be used to analyze the used/unused dependencies.
Check your dependencies via:
mvn dependency:analyze
or take a look at the dep tree like this:
mvn dependency:tree
Or you can take a look into your ide (depending which one you use) for example with Eclipse (Indigo) and the m2e plugin you have a tab "Dependency Hierarchy" which shows the tree of dependencies incl. the transitive dependencies.
In some situation you have to be careful about dependencies which are used by DI frameworks which can't be analyzed by maven-dependency-plugin or by ide plugins.
pack200 can drastically reduce the JAR size. But it's hard to use with Maven and impossible to use with an EE container.
Why do you have unused dependencies?