I wanted to take a legacy project, and generate an initial pom.xml file for it, without converting the whole project. (The thinking is to take advantage of some Maven tasks without creating a big ripple effect in the automated builds that already include this project, which are not Maven projects.)
Is there some way to do this other than archetype:generate (in a phony directory), and then
copying the pom.xml to where I really want it?
AFAIK, no.
But if you know enough about Maven, you can typically write a POM file for a legacy project.
The complication is that the files / directories of a legacy project will typically not be organized in the Maven recommended way, so the normal generation tools won't work. However, "it is said" that Maven can cope with non-standard organisations ... if you write the POM file appropriately.
Another approach is to build the legacy project in the legacy way, and then manually add the resulting JAR files to your Maven repo with appropriate "coordinates" (i.e. group-id, artifact-id and version) so that your Maven projects can use them.
You seem to want to do an unorthodox thing with Maven, and while I like Maven, I find Maven is absolutely terrible at doing unorthodox things.
I would suggest using Gradle instead--especially since you just need to perform one task and don't need to build with it (and therefore learn it).
Because Gradle is a Groovy DSL, you can simply write some Groovy code to access a repository and copy it into a directory on the local file system.
(I will leave aside my personal abhorrence of putting compiled artifacts into source control.)
Gradle also has outstanding integration with Ant.
Related
Using IntelliJ IDEA, it is often a straight-forward task to check out a project from GitHub and get it to build locally.
Once the base project is imported from GitHub, the IDE will download artifacts which will allow the main project to run. The dependencies can be examined by using File > Project Structure... > Libraries.
So the IDE downloads dependencies to allow running, but not to build from source. The IDE is able to decompile classes, but the IDE will not automatically download the source code for those dependency libraries so that the programmer can alter the code.
This question is about the steps required in the IDE and project configuration such that a selected dependency will be built locally, and thus allow the programmer to alter the code.
What I tried was to import the project on which the main project was dependent, as a separate project, then configure the main project to utilize the local project instead of the downloaded artifact. The first step, downloading the sub-project and getting it to compile, was completed successfully.
So I ended-up with two projects, the main one, and the one on which the main project depended. The task at hand, if this was the appropriate way to get this done, would be to open the main project and take some action to convince the main project to use the local build, rather than the downloaded runtime "jar".
First, I edited the POM.xml to comment out the dependency for the sub project. Checking File > Project Structure... > Libraries, I could see that it was gone, and the build now failed (as expected).
I went to File > Project Structure... > Libraries > + (plus sign) and poked around with New Module, and Import Module, but I was not able to convince the original project to use the locally available sub project. The result from various attempts was that code in the base project was not able to import from the sub project (unable to compile).
What are the specific steps to take in the IDE to get what was a runtime dependency "jar file" to instead to build locally, and use that instead?
Use a SNAPSHOT version for the dependency (you'll need to change this in both your project's pom.xml and the dependency's pom.xml, so if the current version is 2.0.0 change it to 2.0.0-SNAPSHOT)
Then you can edit the dependency's code and run mvn install in the dependency to provide the new version of the dependency to your project.
TLDR: there is no simple and straight-forward way of downloading a project's code and the code of its dependencies to rebuild it in one go as a complete chain.
Your project depends on several other artifacts. From the screenshot, we may assume that this is a maven project, at least we can be sure that there artifacts with compiled classes available for download, because this happends during build. You can view the compiled classes of your dependencies, because Intelli has the capability of decompressing jars and decompiling code, obviously, but the contents you are viewing is read-only.
Sidenote: Maven convention is to create 3 separate jars for each project. One with compiled classes, one with source files only and one with generated documentation. By default intellij may not download these, but you can force it (right-click on pom.xml -> maven -> Download sources and documentation). This will attach the actual source code instead of decompiled classes to your IDE, so it's much easier to understand the code - but still, there is no option to modify it - it's still read-only extract from some jar.
So what if you want to actually edit the source? You have 3 options, all with its own set of problems that need human intelligence to solve:
You extract the decompiled source from classes jar
You extract the attached source from sources jar
You check out git repository of the dependency
Now, beware of the downsides of each approach:
You can be sure that the decompiled source matches your project dependency 1:1. But decompiled code is not easy to read, missing comments, etc. Also, some projects may not ship their build scripts with the classes jar. Anything more complex than mvn clean install may turn out to be a blocker.
You can be reasonably sure the code matches your project dependency, but this actually is not a given. There is a chance of human error, causing the sources to actually not match the compiled classes (build from different revision or whatnot). Much depends on the quality of the project, the discipline put into the build process and care to avoid environment specific configuration that is not part of the source. The larger and older is the project, the less chances are you are able to recompile it successfully using only src jar.
A sane man's approach. You should have your build scripts, readmes, tutorials, etc. Except, of course, if we are talking some obscure company internal project with zero effort put in its maintenance. Surely, there are the same issues as before: not all projects are rebuilt easily on any environment. There may be steps upon steps required for your workstation to be configured as expected. Hopefully, self-respecting open-source java projects are easy to build, but again - not a given - not all project are open-source, not all are self-respecting.
Important note: When checking out the git repo of your dependency - you must also make sure that you are using correct revision. If the project is maintained with respect for git tags/branches naming convention - you are in luck. Not a given by any means.
All the above is enough to discourage any attempts to automatically decompose dependencies to compilable units by your IDE, and all the burden is put into you. So let's assume the best - our dependency is a simple, self-contained java application that is easily built using simple mvn clean install. You have it checked out in a separate project in your IDE. You identified correct git revision that matches version your project depends on.
Now let's apply your little change and test it. First thing you want to do is change pom.xml of your project to use a made up version of your dependency. It should be a -SNAPSHOT version for clarity and tidiness. You may of course build your modified dependency with real release version - but please be wary of how maven manages dependencies. If you install version 1.0 yourself - it stays in your local repo forever. You will forget about it, and will be using your fake 1.0 version when building all other dependent projects unless you manually locate and remove it from repo. So stick to 1.1-SNAPSHOT.
Now every time you need to apply a small fix to your dependency, execute mvn clean install in its repo, then make sure your actual project depends on the correct new SNAPSHOT version, execute your maven clean install and that's it.
Note that all this has very little to do with Intellij. You are not expected to modify any library paths, advanced project settings, or links to jars. Stick to modifying pom.xml and you are set.
I've written programs in several languages and have tutored students in computer science, but just starting to learn Java on my MacBook. Regarding this question, I'd be happy with any answer that points me to available information or tutorials that address my question; I'm capable of understanding advanced things.
I've been searching for the right IDE for me as well as something I can use with my students, and I've tried IntelliJ, Eclipse, and VS Code. Along the way I've installed external JARs to provide extra capabilities, such as Apache Commons.
Things are getting confusing. I've lost track of how I got to the present state in each IDE. I'd like to understand better how to know the overall Java environment that any given project is using on each of these IDEs, including any external JARs and where they are located. And I'd like to know if they borrow from the Java system environment.
My goal is to understand how my own system got to the way its currently configured, to update my configuration on a project-by-project basis, and to help my students get a matching configuration.
I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
Maven
Question: I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
If you really wanna work in a organised way and wanna focus completely on coding rather than looking for dependencies to work with , then try building your projects with Apache Maven. The magic wand of Maven projects are pom.xml file where all magic happens depending upon your wish.
Maven is a build automation tool used primarily for Java projects. Maven addresses two aspects of building software:
Describes and manages how software is built.
Describes and manages dependencies (various libraries used by your code).
Why Maven:
De facto standard
Able to compile, test, pack and distribute source code ( different Goals)
Robust dependency management (Most important from my point of view)
Extensible via plugin
Good community support and many fan boys around.
The big 3 IDEs (IntelliJ, NetBeans, and Eclipse) all having good
support for Maven, letting you use Maven as a substitute for their
own proprietary project definition and build process.
Maven famously caches all of its dependencies in the ~/.m2
directory, which is sometimes called the local Maven repository.
Maven local repository keeps your project's all dependencies (library jars,
plugin jars etc.). When you run a Maven build, then Maven automatically
downloads all the dependency jars into the local repository. It helps to
avoid references to dependencies stored on remote machine every time a
project is build.
You can simply deploy your project as JAR, WAR, or EAR file and use it on different IDEs or as standalone.
All IDEs need a way to know your project's dependencies. You can either tell them that yourself or let a build tool do that.
Manual dependency handling: by adding the jars to your project. This is probably the fastest way when working on a small project, with one developer, on a specific IDE, with few dependencies. Usually when telling the IDE that this .jar is a dependency of your project, the IDE stores that reference to a project-specific file (eg. in Eclipse the .classpath file which you can edit with a txt editor and see the dependencies yourself). However, it kind of locks your application to your IDE. Most IDEs have cross-IDE support for import and migration, but using both IDEs at the same time can be confusing when a dependency is added to one and has to be repetitively added to other as well. Furthermore, your dependencies have dependencies on their own. By adding manually your jars you are responsible to find and download their own dependencies as well.
Use a build tool: There are 3 standard such tools right now: Apache Ant with Ivy, Apache Maven and Gradle. All of them have support in the major IDEs for Java: IntelliJ IDEA, Eclipse and NetBeans. All of them use some extra build-tool specific files to store your project's configuration and subsequently configure your IDE and the IDE-specific files. That way, your project becomes IDE-agnostic, the IDE outsources the dependency handling to the build tool. These tools will download any direct or transitive dependencies of your project in a local directory or you can compile jars in a specified folder. From those, Ant is the oldest (with Ivy adding dependency handling support), Maven was developed after that and Gradle is the newest and probably the most flexible. In production however Maven is by far the most established one right now.
It would be also useful to look up the Standard Directory Layout. If you adhere to that, it will be easier to work/start with either Maven or Gradle.
Finally, you can search and find most of the free libraries in Maven-Central where conveniently their Ivy/Maven/Gradle script is added as well for you to use on your build-tool script. In many cases a .jar is provided as well if you prefer to manually add it as a dependency.
Regarding VS Code, I think it supports these tools through plugins but I'm not sure.
I'm using Maven to build and deploy jOOQ. I now want to start generating XJC-generated classes using appropriate Maven plugins (before, I used ant scripts). This works very nicely for me, but I'm afraid that some users wanting to build jOOQ themselves without Maven will now have trouble generating those missing sources. So I'd like to move those sources out of target/generated-sources/xjc and into src/main/java, in order to be able to put them under version control.
Is this a common practice?
How can I do it (should I use plugins to move the files on a low-level, or should I generate files directly into src/main/java)?
Do I have other options?
Note, the underlying XSD hardly ever changes, so I don't have to generate these sources every time I build...
This is a terrible idea, you should never put generated sources in version control because whatever they are being generated from can change, and then your code is out of sync.
Even if the XSD never changes, note I said never, not hardly ever, as you said, which is different, I would not put the sources under version control. Maven can be told to not generate the sources every time if the XSD hasn't changed.
If you go down the Maven route, then anyone building your stuff should use the same tool chain. This is not specific to Maven. If this was Ant, or a C++ project with a make file, you wouldn't want to do this either.
If you really want to provide a Maven free stand alone distribution, then have Maven generate that, there are plenty of plugins that will export all the artifacts as an archive that you can distribute. But don't compromise your build for some nebulous external requirement that may not exist.
Maybe an alternative is to use the Maven assembly plugin to zip all those sources and expose those as a versioned artifact. The dependency plugin would allow you to pull in those sources from a local or remote repository and make sure you have the correct version of those sources.
(In my opinion, leave it like it is. If Maven is your build system, everybody trying to build your code should use that)
I might be missing something but how do you manage Java projects in eclipse that need a lot of Jar files. I know maven manages libraries well if there are new updates but maybe I'm missing something, is there a way that eclipse can update new jar files (it would be especially useful for projects using apache-commons, say).
I don't want to sound like asking for a feature request, but I'm looking at if there are ways to keep libraries jar files that a Java project uses to keep them updated automatically the way maven does. With more languages coming with this type of features, finding the right Jar files probably should be easier than this.
Eclipse doesn't manage your jar versions for you, and as far as I know it won't do any auto-updating of jars that have newer versions out there. There's simply not enough information or infrastructure for Eclipse to recognize that a given jar you've added to the classpath is eligible for updating and that you want it updated.
However, there is a Maven plugin for Eclipse called M2Eclipse, which will read a POM and construct a classpath out of jars it finds in the local repository and any remote repositories you've configured. It behaves largely like Maven does in terms of finding the latest version for a given jar (if you've specified a version range in your POM).
You can create user libraries and change their content when new versions are available. That way you do not at least need to change the build path of every project. Or you can load sources of the libraries from their svn and use their trunk version. Remember that you can select multiple projects and svn update them at once.
When you have multiple projects that all use the same set of JAR libraries, it's tedious to include the same JARs over and over again with each project. If I'm working on 20 different projects, I'd rather not have 20 of the same exact set of JAR files lying around. What's the best way to make all those projects (and new projects as well) reference the same set of JARs?
I have some ideas, but each of them has some disadvantages:
Place all the JARs in a folder and have each project look in that folder.
Using Eclipse, create a "User Library" and have each project reference that user library.
Create a "Library" project that references each JAR, and have each project reference that library project.
Believe it or not, your 'tedious' approach is probably the simplest, cleanest and least time-consuming approach there is.
Before jumping on the maven bandwagon you should consider what is really wrong with doing things the way you are currently doing them. You mentioned that it is tedious and that you have a lot of jar files lying around. I created the build process on a large multi-module project using Maven then spent the next 18 months battling with it constantly. Believe me it was tedious and there were a lot of jar files lying around.
Since going back to Ant and committing jars to source control alongside the projects that use them it has been a much smoother ride.
I store a bunch of jar files in a single directory on my machine and then when I create a new project or need to add a new jar to an existing project it only takes about 30 seconds:
Copy the jar from JAR_REPO to project lib dir.
Add jar to build.properties
Add jar to classpath in build.xml
Add jar to build path in Eclipse.
Over the course of a project, that 30 seconds is insignificant, but it means I have a project that can be checked out of source control and just works without requiring any custom Eclipse configuration or Maven installations or user-specific setup.
This approach has saved me and my project team a huge amount of time, mainly because it is simple, reliable and easy to understand.
Update: Clarification prompted by comments
#Robert Munteanu: Thanks for the feedback and updated comments. This might sound a bit argumentative but I'm afraid I can't agree with you that Maven is simpler and clearer, or that it will save you time in the long run.
From your posting:
"I strongly believe that it's simpler and clearer to declare dependencies rather then manually include them. There is a small one-time cost associated with this - smaller for Ivy than for Maven - but in the long run it does pay off."
It may be easier to have Maven download a jar file for you than having to download it yourself but that's the only advantage. Otherwise Maven is not simpler, not clearer and its complexities and limitations will cost you in the long run.
Clarity
The two dependency declarations below do the same thing. I find the Ant one much clearer than the Maven one.
Ant Style:
<path id="compile.classpath">
<pathelement location="${log4j.jar}" />
<pathelement location="${spring.jar}" />
</path>
Maven Style:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>${spring.version}</version>
<scope>compile</scope>
</dependency>
Simplicity
With the Ant version you can hover over the ${log4j.jar} property and it will show you the absolute path to the jar file. You can search for usage of compile.classpath. There's not a whole lot more you need to know.
There is no question that Maven is more complex than the approach I'm suggesting. When you start out with Maven these are just some of the questions that need to be answered.
What does groupId mean?
What does artifactId mean?
Where does the jar come from?
Where is the jar now?
What is provided scope? Who's providing it?
How did that jar file end up in my WAR file?
Why does this dependency not have a version element?
I don't understand this error message. What on Earth does it mean?
Where on Earth did that jar file come from? I didn't declare it.
Why do I have 2 versions of the same jar file on my classpath?
Why does the project not build any more? Nothing has changed since the last time I built it.
How do I add a third-party jar that's not in the Maven repository?
Tell me again where I get that Eclipse plugin from.
Transitive Dependencies
"Another, smaller, benefit is the handling of transitive and conflicting dependencies."
In my experience, transitive dependencies are more trouble than they're worth. You end up with multiple versions of the same jar file and you end up with optional jar files that you don't want. I ended up declaring just about everything with provided scope to avoid the hassle.
The Long Term Payoff
"Focus on programming, not building."
I agree. Since going back to Ant and putting my jar files in source control I have been able to spend far less time dealing with build issues.
These are the things I spend less time doing:
Reading poor Maven documentation.
Reading even poorer Codehaus Mojo documentation.
Setting up shared internal repositories.
Educating team members.
Writing Maven plugins to fill the gaps.
Trying to workaround defective plugins (release, assembly).
Installing Eclipse plugins for Maven.
Waiting for the plugin to give me back control of Eclipse.
Anyways, sorry about the long posting. Maybe now that I've got that off my chest I can bring some closure to my long and painful Maven experience. :)
Use Maven or Ivy to handle these shared jars. If you're wary of changing your projects too much, you can simply use Ivy to manage the extra classpath for you.
Both have good Eclipse plugins:
m2eclipse
Maven classpath container http://img229.imageshack.us/img229/4848/mavendependencies.png
IvyDE
IvyDE classpath container http://img76.imageshack.us/img76/3180/cpnode.jpg
which I've used with good results.
You'll note that both of them reference jars outside the workspace, so the duplication is removed.
Update ( prompted by comments ):
My reason for recommending this approach is that I strongly believe that it's simpler and clearer to declare dependencies rather then manually include them. There is a small one-time cost associated with this - smaller for Ivy than for Maven - but in the long run it does pay off.
Another, smaller, benefit is the handling of transitive and conflicting dependencies. It's easy to forget why you need that commons-logging-1.1.jar in the classpath and whether you need to upgrade to 1.1.1. And also it's no fun to pull in all the depencies required for e.g. a Hibernate + Annotation + Spring combo. Focus on programming, not building.
It depends on your needs, but there are several viable options. My work uses an external folder and all projects reference that folder, which makes life easier running builds outside of eclipse. A user library is a slightly more plesant way of doing things, as long as you don't mind the slight eclipse dependancy. I don't see a whole lot of benefit to a library project on it's own, but if you have some sort of universal 'util' type project that all other projects already load, you could just put all the external jars in that project.
One approach is to put all your jar files in one location on your machine, in your eclipse ide, define an environment variable, say LIB_LOCATION that points to that directory and have your projects use the jars relative to that variable. This way, you get the ease of use, no multiple jars, portable across machines, as long as you have the variable defined correctly. I have been trying maven for a group of decent size projects and it seems I have to fight at least as much as I used to. Bugs and wired behaviors in the plug ins, m2eclipse and q4eclipse.
You might edit the "Installed JREs" to include your JAR file ("Add external JARs"), add the file to jdk\jre\lib\ext\ directory or specify a CLASSPATH environment variable containing the path to it.
I'd recommend the "library" project approach.
But even better -- a separate lib project per external jar -- this allows you to track deps between third-party jars and know what needs to change when you're upgrading a dependency.
Make sure you check in these projects so all users are using the same versions of third-party libs and so you can easily regenerate a version of software (use tags/labels in your version control to group which versions of which projects go together)
We have decided on a more tedious method but which allows us to have everything inhouse, but will probably only work well for a small set of developers.
Each set of jar files is set up as a Eclipse project named appropriately after the jar set, added to the build path, source jars and javadoc jars correctly set on each jar in the build path, and each project then includes those library projects needed for that project. The resulting multi-project workspace is then exported as a ProjectSet.psf file which can then be read in in a raw Eclipse bringing in the whole workspace again. We then have all the above projects in CVS including the jar files.
This has worked very wellf or us.
If you are in a larger organization the dependency handling in Maven may work well for you. You should definitively have a local cache of artifacts so the whole world doesn't stop if your internet connection is lost.
Also note that the new Eclipse 3.5 coming out this sommer, will have a "Create Runnable Jar" which can output the needed jars next to the generated runnable jar and set up the Class-PAth line in the Manifest correctly. I expect that to be a big time saver - check it out.