When you have multiple projects that all use the same set of JAR libraries, it's tedious to include the same JARs over and over again with each project. If I'm working on 20 different projects, I'd rather not have 20 of the same exact set of JAR files lying around. What's the best way to make all those projects (and new projects as well) reference the same set of JARs?
I have some ideas, but each of them has some disadvantages:
Place all the JARs in a folder and have each project look in that folder.
Using Eclipse, create a "User Library" and have each project reference that user library.
Create a "Library" project that references each JAR, and have each project reference that library project.
Believe it or not, your 'tedious' approach is probably the simplest, cleanest and least time-consuming approach there is.
Before jumping on the maven bandwagon you should consider what is really wrong with doing things the way you are currently doing them. You mentioned that it is tedious and that you have a lot of jar files lying around. I created the build process on a large multi-module project using Maven then spent the next 18 months battling with it constantly. Believe me it was tedious and there were a lot of jar files lying around.
Since going back to Ant and committing jars to source control alongside the projects that use them it has been a much smoother ride.
I store a bunch of jar files in a single directory on my machine and then when I create a new project or need to add a new jar to an existing project it only takes about 30 seconds:
Copy the jar from JAR_REPO to project lib dir.
Add jar to build.properties
Add jar to classpath in build.xml
Add jar to build path in Eclipse.
Over the course of a project, that 30 seconds is insignificant, but it means I have a project that can be checked out of source control and just works without requiring any custom Eclipse configuration or Maven installations or user-specific setup.
This approach has saved me and my project team a huge amount of time, mainly because it is simple, reliable and easy to understand.
Update: Clarification prompted by comments
#Robert Munteanu: Thanks for the feedback and updated comments. This might sound a bit argumentative but I'm afraid I can't agree with you that Maven is simpler and clearer, or that it will save you time in the long run.
From your posting:
"I strongly believe that it's simpler and clearer to declare dependencies rather then manually include them. There is a small one-time cost associated with this - smaller for Ivy than for Maven - but in the long run it does pay off."
It may be easier to have Maven download a jar file for you than having to download it yourself but that's the only advantage. Otherwise Maven is not simpler, not clearer and its complexities and limitations will cost you in the long run.
Clarity
The two dependency declarations below do the same thing. I find the Ant one much clearer than the Maven one.
Ant Style:
<path id="compile.classpath">
<pathelement location="${log4j.jar}" />
<pathelement location="${spring.jar}" />
</path>
Maven Style:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>${spring.version}</version>
<scope>compile</scope>
</dependency>
Simplicity
With the Ant version you can hover over the ${log4j.jar} property and it will show you the absolute path to the jar file. You can search for usage of compile.classpath. There's not a whole lot more you need to know.
There is no question that Maven is more complex than the approach I'm suggesting. When you start out with Maven these are just some of the questions that need to be answered.
What does groupId mean?
What does artifactId mean?
Where does the jar come from?
Where is the jar now?
What is provided scope? Who's providing it?
How did that jar file end up in my WAR file?
Why does this dependency not have a version element?
I don't understand this error message. What on Earth does it mean?
Where on Earth did that jar file come from? I didn't declare it.
Why do I have 2 versions of the same jar file on my classpath?
Why does the project not build any more? Nothing has changed since the last time I built it.
How do I add a third-party jar that's not in the Maven repository?
Tell me again where I get that Eclipse plugin from.
Transitive Dependencies
"Another, smaller, benefit is the handling of transitive and conflicting dependencies."
In my experience, transitive dependencies are more trouble than they're worth. You end up with multiple versions of the same jar file and you end up with optional jar files that you don't want. I ended up declaring just about everything with provided scope to avoid the hassle.
The Long Term Payoff
"Focus on programming, not building."
I agree. Since going back to Ant and putting my jar files in source control I have been able to spend far less time dealing with build issues.
These are the things I spend less time doing:
Reading poor Maven documentation.
Reading even poorer Codehaus Mojo documentation.
Setting up shared internal repositories.
Educating team members.
Writing Maven plugins to fill the gaps.
Trying to workaround defective plugins (release, assembly).
Installing Eclipse plugins for Maven.
Waiting for the plugin to give me back control of Eclipse.
Anyways, sorry about the long posting. Maybe now that I've got that off my chest I can bring some closure to my long and painful Maven experience. :)
Use Maven or Ivy to handle these shared jars. If you're wary of changing your projects too much, you can simply use Ivy to manage the extra classpath for you.
Both have good Eclipse plugins:
m2eclipse
Maven classpath container http://img229.imageshack.us/img229/4848/mavendependencies.png
IvyDE
IvyDE classpath container http://img76.imageshack.us/img76/3180/cpnode.jpg
which I've used with good results.
You'll note that both of them reference jars outside the workspace, so the duplication is removed.
Update ( prompted by comments ):
My reason for recommending this approach is that I strongly believe that it's simpler and clearer to declare dependencies rather then manually include them. There is a small one-time cost associated with this - smaller for Ivy than for Maven - but in the long run it does pay off.
Another, smaller, benefit is the handling of transitive and conflicting dependencies. It's easy to forget why you need that commons-logging-1.1.jar in the classpath and whether you need to upgrade to 1.1.1. And also it's no fun to pull in all the depencies required for e.g. a Hibernate + Annotation + Spring combo. Focus on programming, not building.
It depends on your needs, but there are several viable options. My work uses an external folder and all projects reference that folder, which makes life easier running builds outside of eclipse. A user library is a slightly more plesant way of doing things, as long as you don't mind the slight eclipse dependancy. I don't see a whole lot of benefit to a library project on it's own, but if you have some sort of universal 'util' type project that all other projects already load, you could just put all the external jars in that project.
One approach is to put all your jar files in one location on your machine, in your eclipse ide, define an environment variable, say LIB_LOCATION that points to that directory and have your projects use the jars relative to that variable. This way, you get the ease of use, no multiple jars, portable across machines, as long as you have the variable defined correctly. I have been trying maven for a group of decent size projects and it seems I have to fight at least as much as I used to. Bugs and wired behaviors in the plug ins, m2eclipse and q4eclipse.
You might edit the "Installed JREs" to include your JAR file ("Add external JARs"), add the file to jdk\jre\lib\ext\ directory or specify a CLASSPATH environment variable containing the path to it.
I'd recommend the "library" project approach.
But even better -- a separate lib project per external jar -- this allows you to track deps between third-party jars and know what needs to change when you're upgrading a dependency.
Make sure you check in these projects so all users are using the same versions of third-party libs and so you can easily regenerate a version of software (use tags/labels in your version control to group which versions of which projects go together)
We have decided on a more tedious method but which allows us to have everything inhouse, but will probably only work well for a small set of developers.
Each set of jar files is set up as a Eclipse project named appropriately after the jar set, added to the build path, source jars and javadoc jars correctly set on each jar in the build path, and each project then includes those library projects needed for that project. The resulting multi-project workspace is then exported as a ProjectSet.psf file which can then be read in in a raw Eclipse bringing in the whole workspace again. We then have all the above projects in CVS including the jar files.
This has worked very wellf or us.
If you are in a larger organization the dependency handling in Maven may work well for you. You should definitively have a local cache of artifacts so the whole world doesn't stop if your internet connection is lost.
Also note that the new Eclipse 3.5 coming out this sommer, will have a "Create Runnable Jar" which can output the needed jars next to the generated runnable jar and set up the Class-PAth line in the Manifest correctly. I expect that to be a big time saver - check it out.
Related
I've written programs in several languages and have tutored students in computer science, but just starting to learn Java on my MacBook. Regarding this question, I'd be happy with any answer that points me to available information or tutorials that address my question; I'm capable of understanding advanced things.
I've been searching for the right IDE for me as well as something I can use with my students, and I've tried IntelliJ, Eclipse, and VS Code. Along the way I've installed external JARs to provide extra capabilities, such as Apache Commons.
Things are getting confusing. I've lost track of how I got to the present state in each IDE. I'd like to understand better how to know the overall Java environment that any given project is using on each of these IDEs, including any external JARs and where they are located. And I'd like to know if they borrow from the Java system environment.
My goal is to understand how my own system got to the way its currently configured, to update my configuration on a project-by-project basis, and to help my students get a matching configuration.
I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
Maven
Question: I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
If you really wanna work in a organised way and wanna focus completely on coding rather than looking for dependencies to work with , then try building your projects with Apache Maven. The magic wand of Maven projects are pom.xml file where all magic happens depending upon your wish.
Maven is a build automation tool used primarily for Java projects. Maven addresses two aspects of building software:
Describes and manages how software is built.
Describes and manages dependencies (various libraries used by your code).
Why Maven:
De facto standard
Able to compile, test, pack and distribute source code ( different Goals)
Robust dependency management (Most important from my point of view)
Extensible via plugin
Good community support and many fan boys around.
The big 3 IDEs (IntelliJ, NetBeans, and Eclipse) all having good
support for Maven, letting you use Maven as a substitute for their
own proprietary project definition and build process.
Maven famously caches all of its dependencies in the ~/.m2
directory, which is sometimes called the local Maven repository.
Maven local repository keeps your project's all dependencies (library jars,
plugin jars etc.). When you run a Maven build, then Maven automatically
downloads all the dependency jars into the local repository. It helps to
avoid references to dependencies stored on remote machine every time a
project is build.
You can simply deploy your project as JAR, WAR, or EAR file and use it on different IDEs or as standalone.
All IDEs need a way to know your project's dependencies. You can either tell them that yourself or let a build tool do that.
Manual dependency handling: by adding the jars to your project. This is probably the fastest way when working on a small project, with one developer, on a specific IDE, with few dependencies. Usually when telling the IDE that this .jar is a dependency of your project, the IDE stores that reference to a project-specific file (eg. in Eclipse the .classpath file which you can edit with a txt editor and see the dependencies yourself). However, it kind of locks your application to your IDE. Most IDEs have cross-IDE support for import and migration, but using both IDEs at the same time can be confusing when a dependency is added to one and has to be repetitively added to other as well. Furthermore, your dependencies have dependencies on their own. By adding manually your jars you are responsible to find and download their own dependencies as well.
Use a build tool: There are 3 standard such tools right now: Apache Ant with Ivy, Apache Maven and Gradle. All of them have support in the major IDEs for Java: IntelliJ IDEA, Eclipse and NetBeans. All of them use some extra build-tool specific files to store your project's configuration and subsequently configure your IDE and the IDE-specific files. That way, your project becomes IDE-agnostic, the IDE outsources the dependency handling to the build tool. These tools will download any direct or transitive dependencies of your project in a local directory or you can compile jars in a specified folder. From those, Ant is the oldest (with Ivy adding dependency handling support), Maven was developed after that and Gradle is the newest and probably the most flexible. In production however Maven is by far the most established one right now.
It would be also useful to look up the Standard Directory Layout. If you adhere to that, it will be easier to work/start with either Maven or Gradle.
Finally, you can search and find most of the free libraries in Maven-Central where conveniently their Ivy/Maven/Gradle script is added as well for you to use on your build-tool script. In many cases a .jar is provided as well if you prefer to manually add it as a dependency.
Regarding VS Code, I think it supports these tools through plugins but I'm not sure.
I've seen a lot of projects, even from big companies like Elephant Bird (Twitter) and Akela (Mozilla) that offer source and ask you to compile it yourself instead of also offering jars. Is there some benefit to compiling in your own environment instead of just downloading a jar someone else has compiled?
Dependencies are not in the same location or even have the same version on every machine. It is simpler to detect where they are at compile-time.
If there is any native code (sometimes just for optimization) in a project, there are probably platform-dependent flags that need to be set at compile-time.
The short answer is dependency management. Most public OSS Java projects offer jars by publishing them to Maven Central. You are expected to use a build system like Gradle, Ivy, or Maven to manage your dependencies - these tools will automatically download the library you want along with any of its dependent libraries and be smart about it, caching it on your local filesystem so if a library is shared across multiple libraries it won't be downloaded twice.
As for the example projects you listed, Elephant Bird is available via Maven Central whereas Akela tells you exactly how to create your own jar (perhaps it's not quite far along enough to justify going through the rigmarole of publishing to Maven Central):
Building
To make a jar you can do:
mvn package
To make a Hadoop MapReduce job jar with no defined main class in the manifest:
mvn assembly:assembly
Without an automatic build system its hard to maintain a current version of the jar file online. Including the jar file in the repository is generally not a good idea as users who clone it don't need the compiled jar, they want the code. So unless the publisher explicitly adds a jar file to a download location outside of the sourcecode repository and updates this file every time the application changes you have to compile it yourself. Automatic Build systems can help a publisher to provide a current compiled jar to it's users but for smaller projects it's not always sufficient to go through the trouble of setting one up.
I have a single very large codebase that compiles down to a JAR. I also use the shade plugin to compile it down and package it up with all dependencies. I also use the war plugin to get a WAR file.
With respect to the WAR file, once the goal is created I have a post-build event that simply copies the WAR file to its destination, so technically I'm good with that.
The problem I have is with the other two JARs. They both share the same artifactId since they're both built in a single pom.xml and this isn't acceptable for a number of reasons (including, but not limited to the fact that some caching of dependencies is pretty dumb about realizing that one JAR is the thin one and one is the full-dependency one).
What I need to do is create a pom.xml (or group thereof) suitable for builds (by Jenkins) that can use the same codebase (pulled from Github) but create two separate JAR files, each with their own artifactId.
Being a Maven novice, I've read through the beginning book and it seems to me that what I want is a parent pom.xml with two modules. But from what I can tell, each module means a separate directory with separate code. As I said, this is built from the same codebase. The only difference is one is built from the "regular" build, and the other is built using the "shade" plugin and goal.
The only other thing I can think of is build the "regular" JAR and then build the shaded JAR with a classifier of "full?" If this is the answer, may I humbly ask for some adult supervision on how to do this, as I'm not seeing how.
If that's not the answer, I suspect this must be a common problem, so again, some guidance would be very helpful!
The solution I came up with was to use a classifier for the "shaded" jar. Thus, the artifacts don't collide.
I then had an issue accessing it, but found the solution to that issue as well - How do I access a jar with a classifier?
I wanted to take a legacy project, and generate an initial pom.xml file for it, without converting the whole project. (The thinking is to take advantage of some Maven tasks without creating a big ripple effect in the automated builds that already include this project, which are not Maven projects.)
Is there some way to do this other than archetype:generate (in a phony directory), and then
copying the pom.xml to where I really want it?
AFAIK, no.
But if you know enough about Maven, you can typically write a POM file for a legacy project.
The complication is that the files / directories of a legacy project will typically not be organized in the Maven recommended way, so the normal generation tools won't work. However, "it is said" that Maven can cope with non-standard organisations ... if you write the POM file appropriately.
Another approach is to build the legacy project in the legacy way, and then manually add the resulting JAR files to your Maven repo with appropriate "coordinates" (i.e. group-id, artifact-id and version) so that your Maven projects can use them.
You seem to want to do an unorthodox thing with Maven, and while I like Maven, I find Maven is absolutely terrible at doing unorthodox things.
I would suggest using Gradle instead--especially since you just need to perform one task and don't need to build with it (and therefore learn it).
Because Gradle is a Groovy DSL, you can simply write some Groovy code to access a repository and copy it into a directory on the local file system.
(I will leave aside my personal abhorrence of putting compiled artifacts into source control.)
Gradle also has outstanding integration with Ant.
I might be missing something but how do you manage Java projects in eclipse that need a lot of Jar files. I know maven manages libraries well if there are new updates but maybe I'm missing something, is there a way that eclipse can update new jar files (it would be especially useful for projects using apache-commons, say).
I don't want to sound like asking for a feature request, but I'm looking at if there are ways to keep libraries jar files that a Java project uses to keep them updated automatically the way maven does. With more languages coming with this type of features, finding the right Jar files probably should be easier than this.
Eclipse doesn't manage your jar versions for you, and as far as I know it won't do any auto-updating of jars that have newer versions out there. There's simply not enough information or infrastructure for Eclipse to recognize that a given jar you've added to the classpath is eligible for updating and that you want it updated.
However, there is a Maven plugin for Eclipse called M2Eclipse, which will read a POM and construct a classpath out of jars it finds in the local repository and any remote repositories you've configured. It behaves largely like Maven does in terms of finding the latest version for a given jar (if you've specified a version range in your POM).
You can create user libraries and change their content when new versions are available. That way you do not at least need to change the build path of every project. Or you can load sources of the libraries from their svn and use their trunk version. Remember that you can select multiple projects and svn update them at once.