I'm using Maven to build and deploy jOOQ. I now want to start generating XJC-generated classes using appropriate Maven plugins (before, I used ant scripts). This works very nicely for me, but I'm afraid that some users wanting to build jOOQ themselves without Maven will now have trouble generating those missing sources. So I'd like to move those sources out of target/generated-sources/xjc and into src/main/java, in order to be able to put them under version control.
Is this a common practice?
How can I do it (should I use plugins to move the files on a low-level, or should I generate files directly into src/main/java)?
Do I have other options?
Note, the underlying XSD hardly ever changes, so I don't have to generate these sources every time I build...
This is a terrible idea, you should never put generated sources in version control because whatever they are being generated from can change, and then your code is out of sync.
Even if the XSD never changes, note I said never, not hardly ever, as you said, which is different, I would not put the sources under version control. Maven can be told to not generate the sources every time if the XSD hasn't changed.
If you go down the Maven route, then anyone building your stuff should use the same tool chain. This is not specific to Maven. If this was Ant, or a C++ project with a make file, you wouldn't want to do this either.
If you really want to provide a Maven free stand alone distribution, then have Maven generate that, there are plenty of plugins that will export all the artifacts as an archive that you can distribute. But don't compromise your build for some nebulous external requirement that may not exist.
Maybe an alternative is to use the Maven assembly plugin to zip all those sources and expose those as a versioned artifact. The dependency plugin would allow you to pull in those sources from a local or remote repository and make sure you have the correct version of those sources.
(In my opinion, leave it like it is. If Maven is your build system, everybody trying to build your code should use that)
Related
Using IntelliJ IDEA, it is often a straight-forward task to check out a project from GitHub and get it to build locally.
Once the base project is imported from GitHub, the IDE will download artifacts which will allow the main project to run. The dependencies can be examined by using File > Project Structure... > Libraries.
So the IDE downloads dependencies to allow running, but not to build from source. The IDE is able to decompile classes, but the IDE will not automatically download the source code for those dependency libraries so that the programmer can alter the code.
This question is about the steps required in the IDE and project configuration such that a selected dependency will be built locally, and thus allow the programmer to alter the code.
What I tried was to import the project on which the main project was dependent, as a separate project, then configure the main project to utilize the local project instead of the downloaded artifact. The first step, downloading the sub-project and getting it to compile, was completed successfully.
So I ended-up with two projects, the main one, and the one on which the main project depended. The task at hand, if this was the appropriate way to get this done, would be to open the main project and take some action to convince the main project to use the local build, rather than the downloaded runtime "jar".
First, I edited the POM.xml to comment out the dependency for the sub project. Checking File > Project Structure... > Libraries, I could see that it was gone, and the build now failed (as expected).
I went to File > Project Structure... > Libraries > + (plus sign) and poked around with New Module, and Import Module, but I was not able to convince the original project to use the locally available sub project. The result from various attempts was that code in the base project was not able to import from the sub project (unable to compile).
What are the specific steps to take in the IDE to get what was a runtime dependency "jar file" to instead to build locally, and use that instead?
Use a SNAPSHOT version for the dependency (you'll need to change this in both your project's pom.xml and the dependency's pom.xml, so if the current version is 2.0.0 change it to 2.0.0-SNAPSHOT)
Then you can edit the dependency's code and run mvn install in the dependency to provide the new version of the dependency to your project.
TLDR: there is no simple and straight-forward way of downloading a project's code and the code of its dependencies to rebuild it in one go as a complete chain.
Your project depends on several other artifacts. From the screenshot, we may assume that this is a maven project, at least we can be sure that there artifacts with compiled classes available for download, because this happends during build. You can view the compiled classes of your dependencies, because Intelli has the capability of decompressing jars and decompiling code, obviously, but the contents you are viewing is read-only.
Sidenote: Maven convention is to create 3 separate jars for each project. One with compiled classes, one with source files only and one with generated documentation. By default intellij may not download these, but you can force it (right-click on pom.xml -> maven -> Download sources and documentation). This will attach the actual source code instead of decompiled classes to your IDE, so it's much easier to understand the code - but still, there is no option to modify it - it's still read-only extract from some jar.
So what if you want to actually edit the source? You have 3 options, all with its own set of problems that need human intelligence to solve:
You extract the decompiled source from classes jar
You extract the attached source from sources jar
You check out git repository of the dependency
Now, beware of the downsides of each approach:
You can be sure that the decompiled source matches your project dependency 1:1. But decompiled code is not easy to read, missing comments, etc. Also, some projects may not ship their build scripts with the classes jar. Anything more complex than mvn clean install may turn out to be a blocker.
You can be reasonably sure the code matches your project dependency, but this actually is not a given. There is a chance of human error, causing the sources to actually not match the compiled classes (build from different revision or whatnot). Much depends on the quality of the project, the discipline put into the build process and care to avoid environment specific configuration that is not part of the source. The larger and older is the project, the less chances are you are able to recompile it successfully using only src jar.
A sane man's approach. You should have your build scripts, readmes, tutorials, etc. Except, of course, if we are talking some obscure company internal project with zero effort put in its maintenance. Surely, there are the same issues as before: not all projects are rebuilt easily on any environment. There may be steps upon steps required for your workstation to be configured as expected. Hopefully, self-respecting open-source java projects are easy to build, but again - not a given - not all project are open-source, not all are self-respecting.
Important note: When checking out the git repo of your dependency - you must also make sure that you are using correct revision. If the project is maintained with respect for git tags/branches naming convention - you are in luck. Not a given by any means.
All the above is enough to discourage any attempts to automatically decompose dependencies to compilable units by your IDE, and all the burden is put into you. So let's assume the best - our dependency is a simple, self-contained java application that is easily built using simple mvn clean install. You have it checked out in a separate project in your IDE. You identified correct git revision that matches version your project depends on.
Now let's apply your little change and test it. First thing you want to do is change pom.xml of your project to use a made up version of your dependency. It should be a -SNAPSHOT version for clarity and tidiness. You may of course build your modified dependency with real release version - but please be wary of how maven manages dependencies. If you install version 1.0 yourself - it stays in your local repo forever. You will forget about it, and will be using your fake 1.0 version when building all other dependent projects unless you manually locate and remove it from repo. So stick to 1.1-SNAPSHOT.
Now every time you need to apply a small fix to your dependency, execute mvn clean install in its repo, then make sure your actual project depends on the correct new SNAPSHOT version, execute your maven clean install and that's it.
Note that all this has very little to do with Intellij. You are not expected to modify any library paths, advanced project settings, or links to jars. Stick to modifying pom.xml and you are set.
I've written programs in several languages and have tutored students in computer science, but just starting to learn Java on my MacBook. Regarding this question, I'd be happy with any answer that points me to available information or tutorials that address my question; I'm capable of understanding advanced things.
I've been searching for the right IDE for me as well as something I can use with my students, and I've tried IntelliJ, Eclipse, and VS Code. Along the way I've installed external JARs to provide extra capabilities, such as Apache Commons.
Things are getting confusing. I've lost track of how I got to the present state in each IDE. I'd like to understand better how to know the overall Java environment that any given project is using on each of these IDEs, including any external JARs and where they are located. And I'd like to know if they borrow from the Java system environment.
My goal is to understand how my own system got to the way its currently configured, to update my configuration on a project-by-project basis, and to help my students get a matching configuration.
I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
Maven
Question: I'd also like advice on the right way, or simplest/cleanest way, to install external JARs.
If you really wanna work in a organised way and wanna focus completely on coding rather than looking for dependencies to work with , then try building your projects with Apache Maven. The magic wand of Maven projects are pom.xml file where all magic happens depending upon your wish.
Maven is a build automation tool used primarily for Java projects. Maven addresses two aspects of building software:
Describes and manages how software is built.
Describes and manages dependencies (various libraries used by your code).
Why Maven:
De facto standard
Able to compile, test, pack and distribute source code ( different Goals)
Robust dependency management (Most important from my point of view)
Extensible via plugin
Good community support and many fan boys around.
The big 3 IDEs (IntelliJ, NetBeans, and Eclipse) all having good
support for Maven, letting you use Maven as a substitute for their
own proprietary project definition and build process.
Maven famously caches all of its dependencies in the ~/.m2
directory, which is sometimes called the local Maven repository.
Maven local repository keeps your project's all dependencies (library jars,
plugin jars etc.). When you run a Maven build, then Maven automatically
downloads all the dependency jars into the local repository. It helps to
avoid references to dependencies stored on remote machine every time a
project is build.
You can simply deploy your project as JAR, WAR, or EAR file and use it on different IDEs or as standalone.
All IDEs need a way to know your project's dependencies. You can either tell them that yourself or let a build tool do that.
Manual dependency handling: by adding the jars to your project. This is probably the fastest way when working on a small project, with one developer, on a specific IDE, with few dependencies. Usually when telling the IDE that this .jar is a dependency of your project, the IDE stores that reference to a project-specific file (eg. in Eclipse the .classpath file which you can edit with a txt editor and see the dependencies yourself). However, it kind of locks your application to your IDE. Most IDEs have cross-IDE support for import and migration, but using both IDEs at the same time can be confusing when a dependency is added to one and has to be repetitively added to other as well. Furthermore, your dependencies have dependencies on their own. By adding manually your jars you are responsible to find and download their own dependencies as well.
Use a build tool: There are 3 standard such tools right now: Apache Ant with Ivy, Apache Maven and Gradle. All of them have support in the major IDEs for Java: IntelliJ IDEA, Eclipse and NetBeans. All of them use some extra build-tool specific files to store your project's configuration and subsequently configure your IDE and the IDE-specific files. That way, your project becomes IDE-agnostic, the IDE outsources the dependency handling to the build tool. These tools will download any direct or transitive dependencies of your project in a local directory or you can compile jars in a specified folder. From those, Ant is the oldest (with Ivy adding dependency handling support), Maven was developed after that and Gradle is the newest and probably the most flexible. In production however Maven is by far the most established one right now.
It would be also useful to look up the Standard Directory Layout. If you adhere to that, it will be easier to work/start with either Maven or Gradle.
Finally, you can search and find most of the free libraries in Maven-Central where conveniently their Ivy/Maven/Gradle script is added as well for you to use on your build-tool script. In many cases a .jar is provided as well if you prefer to manually add it as a dependency.
Regarding VS Code, I think it supports these tools through plugins but I'm not sure.
I've seen a lot of projects, even from big companies like Elephant Bird (Twitter) and Akela (Mozilla) that offer source and ask you to compile it yourself instead of also offering jars. Is there some benefit to compiling in your own environment instead of just downloading a jar someone else has compiled?
Dependencies are not in the same location or even have the same version on every machine. It is simpler to detect where they are at compile-time.
If there is any native code (sometimes just for optimization) in a project, there are probably platform-dependent flags that need to be set at compile-time.
The short answer is dependency management. Most public OSS Java projects offer jars by publishing them to Maven Central. You are expected to use a build system like Gradle, Ivy, or Maven to manage your dependencies - these tools will automatically download the library you want along with any of its dependent libraries and be smart about it, caching it on your local filesystem so if a library is shared across multiple libraries it won't be downloaded twice.
As for the example projects you listed, Elephant Bird is available via Maven Central whereas Akela tells you exactly how to create your own jar (perhaps it's not quite far along enough to justify going through the rigmarole of publishing to Maven Central):
Building
To make a jar you can do:
mvn package
To make a Hadoop MapReduce job jar with no defined main class in the manifest:
mvn assembly:assembly
Without an automatic build system its hard to maintain a current version of the jar file online. Including the jar file in the repository is generally not a good idea as users who clone it don't need the compiled jar, they want the code. So unless the publisher explicitly adds a jar file to a download location outside of the sourcecode repository and updates this file every time the application changes you have to compile it yourself. Automatic Build systems can help a publisher to provide a current compiled jar to it's users but for smaller projects it's not always sufficient to go through the trouble of setting one up.
I wanted to take a legacy project, and generate an initial pom.xml file for it, without converting the whole project. (The thinking is to take advantage of some Maven tasks without creating a big ripple effect in the automated builds that already include this project, which are not Maven projects.)
Is there some way to do this other than archetype:generate (in a phony directory), and then
copying the pom.xml to where I really want it?
AFAIK, no.
But if you know enough about Maven, you can typically write a POM file for a legacy project.
The complication is that the files / directories of a legacy project will typically not be organized in the Maven recommended way, so the normal generation tools won't work. However, "it is said" that Maven can cope with non-standard organisations ... if you write the POM file appropriately.
Another approach is to build the legacy project in the legacy way, and then manually add the resulting JAR files to your Maven repo with appropriate "coordinates" (i.e. group-id, artifact-id and version) so that your Maven projects can use them.
You seem to want to do an unorthodox thing with Maven, and while I like Maven, I find Maven is absolutely terrible at doing unorthodox things.
I would suggest using Gradle instead--especially since you just need to perform one task and don't need to build with it (and therefore learn it).
Because Gradle is a Groovy DSL, you can simply write some Groovy code to access a repository and copy it into a directory on the local file system.
(I will leave aside my personal abhorrence of putting compiled artifacts into source control.)
Gradle also has outstanding integration with Ant.
I might be missing something but how do you manage Java projects in eclipse that need a lot of Jar files. I know maven manages libraries well if there are new updates but maybe I'm missing something, is there a way that eclipse can update new jar files (it would be especially useful for projects using apache-commons, say).
I don't want to sound like asking for a feature request, but I'm looking at if there are ways to keep libraries jar files that a Java project uses to keep them updated automatically the way maven does. With more languages coming with this type of features, finding the right Jar files probably should be easier than this.
Eclipse doesn't manage your jar versions for you, and as far as I know it won't do any auto-updating of jars that have newer versions out there. There's simply not enough information or infrastructure for Eclipse to recognize that a given jar you've added to the classpath is eligible for updating and that you want it updated.
However, there is a Maven plugin for Eclipse called M2Eclipse, which will read a POM and construct a classpath out of jars it finds in the local repository and any remote repositories you've configured. It behaves largely like Maven does in terms of finding the latest version for a given jar (if you've specified a version range in your POM).
You can create user libraries and change their content when new versions are available. That way you do not at least need to change the build path of every project. Or you can load sources of the libraries from their svn and use their trunk version. Remember that you can select multiple projects and svn update them at once.