I have a simple problem but being a novice with Maven it's unclear which is the best practice to solve this problem.
A project I'm working has a dependency to manage archives which is the following: net.sf.sevenzipjbindings.
This dependency is made by two artifacts: a pure Java one, which provides the Java interface and another one which is the native underlying code which has many flavours according to the operating system.
At the moment I'm packaging the jar built by maven with all the natives for all the platforms but I'd like to be able to do something like:
when working (so compile goal if I understand correctly) on the project use the all-platforms artifact (so that I can work seamlessly from multiple operating systems
when packaging build 3 different JAR with dependencies composed by the Java interface + a specific platform artifact
This sounds rather simple from my point of view but I'm struggling to understand what I should do and what are the best practices in general. Since here there is no different or modular functionality (like isolating part of the code into a separated module), just a replacement for underlying dynamic libraries.
It sounds like you want a multi-module Maven build:
- myapp
- myapp-engine
- myapp-windows
- myapp-linux
The myapp-engine module should depend on external-library-api, and the myapp-windows module should depend on myapp-engine and external-library-windows. Since you're actually only running the application on a specific OS for any given launch (even in development), there's no reason not to launch the OS-specific module. Modern IDEs should have no trouble debugging into a separate module in a multi-module build.
Related
I'm trying to understand the difference(s) between structuring a project with the Java Platform Module System (JPMS) versus structuring a project using multi-poms.
Is the main difference that the JPMS encapsulates the code while a multi-pom project separates project dependencies?
I've searched on google but I haven't found a good explanation on the differences, but rather I see the word module getting used interchangably.
The computing industry often recycles terms. Context is everything.
The two kinds of “module” you present are unrelated, orthogonal issues.
The Java Platform Module System (JPMS) is a way to identify to the Java compiler namespaces amongst all the classes and methods available at runtime.
Multi-module in Apache Maven is a way to conjoin into one project what could be handled as separate projects. Each module in the project has its own POM with its own dependency and build settings, yet all can be managed as one super-project when combined as a Maven multi-module. Each module results in producing an artifact, such as a JAR or WAR file.
Very simple apps in Java may use neither.
Ideally new Java apps would use the JPMS, but it is still technically optional. In a perfect world, JPMS would have been included in the original Java, but was in fact added only recently, in Java 9. JPMS is quite handy if your app will run as a standalone, with a JVM bundled, because you can include a JVM that has been stripped down to only the parts actually used by your particular app (see jlink, jpackage, and related tools enabled by JPMS).
Maven multi-module projects are generally only used for complicated projects such as an app that includes a piece of functionality which may be spun-off for use in other projects. Or a multi-module Maven project may be good for an app that combines both a frontend user-interface module along with a backend business-logic module where we want to manage each part separately yet combine them into a single deliverable, such as a Vaadin Flow web app.
I can see how you could become confused, as both have to do with arranging classes. To oversimplify, Maven modules are about compile-time (dependency management and build automation) while Java Platform Module System is about runtime.
I’ve read that Gradle is more adept at managing a multi-module project. You might consider switching from Maven to Gradle for your multi-module projects. Gradle obtains your dependencies from Maven repositories.
Never found a really satisfactory solution to this. How do you do it? I am looking for inspiration for new approaches.
For context, assume I write a generator that takes a project resource and generates a code file. But it could be any other project support tool - validator, converter, deployer etc. Often manually triggered actions that are not running as part of normal build.
Such tools typically require a few dependencies that are not required by the project itself at runtime.
Strategies that I have applied or considered in the past:
add tool dependency to project anyway, and either mark it "provided" or filter it out during the packaging process (this is what I usually do, but now I am in danger of adding normal project code that uses the tool dependency, potentially resulting in an error that only manifests during runtime)
use a script (trying hard to avoid scripts and their hidden dependencies and complexities)
create separate support projects (trying hard to avoid project explosion, especially for seemingly small tasks that are handled by a few lines of code)
subprojects / modules (only vaguely aware of this option, never really tried it)
maven plugin that is run with a profile with separate dependencies (trying to avoid the separate project required to maintain the custom plugin)
Inspiration from answers and comments
separate tools project shared by multiple projects
I just realized that maven and eclipse already solved exactly this problem for a very specific "tool": test code.
Test code often needs additional dependencies not used by the application itself.
People obviously invested quite a bit to keep the "test / tool" infrastructure within the same project, as opposed to creating a separate test-project:
separate source locations (src/main/java, src/test/java)
separate resource locations (src/main/resources, src/test/resources)
a full-blown separate maven dependency scope "test", complete with transitive resolution
separate compilation phases (compile / test) with separate dependency trees
eclipse supports special junit launch configurations that are able to correctly resolve the test dependencies
probably more stuff that I am not aware of currently
So, I am strongly considering to program all my supporting tools as "junit test cases".
I am planning to create and commit shared junit launch configs for the team that execute just one specific "test case", which will run the tool logic instead of testing.
The problem I have to solve is to avoid running these dummy tests during the normal maven test phase.
Also, writing this, I realize that there is even another such system already in place: the maven plugin infrastructure, that also has a separate dependency resolution mechanism. Although, so far it seems necessary or normal to create separate projects to create plugins. I will look into ways of writing and building project specific maven plugins without needing to create separate projects. I am thinking about generating the pom.xml needed for plugin compilation on the fly, and including all the test dependencies.
From Jigsaw Project:
Make it easier for developers to construct and maintain libraries and
large applications, for both the Java SE and EE Platforms.
I'm trying to learn what project Jigsaw is and till now it seems that the goal of Project Jigsaw somewhat overlaps with what we did using Maven (or Gradle) dependency management:
Is it a threat to build tools like Maven?
Or my understanding is wrong and project Jigsaw is going to complement these build tools in some way?
Very simplified answer
After Jigsaw, public will be public only within the JAR scope. To see the class outside the JAR it must be exported.
Java will force modularization because any inter module interaction will have to be specified in the module-info file.
For example, if you produce a WAR it will remain almost unchanged but all JARs packages in the WAR must define a module-info (or not define it and be treated as automatic or unnamed modules).
Maven has 2 main features: dependency management and building:
Dependency management means Maven can determine versions of libraries
and download them from repositiories.
Building means Maven can compile code and package it into artifacts.
To conclude: Maven will still be responsible for building, but one must learn how to compile and package using Jigsaw modules.
Modules are not in any way a threat to build tools. Modules complement build tools because build tools construct a dependency graph of artifacts and their versions at build time while modules enforce dependencies of artifacts/modules (not including versions) at build time and run time.
From the State of the Module System:
"A module’s declaration does not include a version string, nor
constraints upon the version strings of the modules upon which it
depends. This is intentional: It is not a goal of the module system
to solve the version-selection problem, which is best left to build
tools and container applications."
Well here is an interesting experience i had since last couple of weeks structuring my maven multi module project.
When i decided to use maven for my build life cycle management i had couple of reason that i wished to choose maven.
a. Mostly development teams are divided so that each team can work on separate Module within the project like Team-A to work on User Management System, Team-B to work on Authorization System, Team-C to work on Document Management System...and so on. Each team has java developers, testers, UI experts etc.
So the maven Project structure should be such that each team can independently work on their respective modules. They must be able to code, compile, build, test, deploy their module without having to compile, test modules belonging to other teams.
And thus i came to conclusion that each development module of the maven multi-module project must represent a Functional Module
After some discussions on forums i found people suggesting me to follow layered approach were child modules must be layers like controller-layer,service-layer,dao-layer etc. I did not pay heed to this advice because this not solving my purpose of teams working on individual module. This way for large project the build and deployment time for each team during development increases which does impact the project time-lines. sometimes the build and deploy time is upto 30 minutes say if there are 10 to 11 modules in the project.
But i did pay heed to a suggestion that keeping DAO layer separate for each module is not a good idea as DAO is highly granular and reused by other modules. and so the dependency of one module on other would would any how become greater.
I found a solution to this problem by creating a common module and moving DAOs and DOMAIN to the common module which will be inherited as a dependency by each module. And this seems to be a more viable option. Now the Project Structure looks like this.
Now when i build the project and run the webapp on server, It complains 404, Resource Not Found. I found that this is because the WEB-INF/classes folder is missing, src/main/java is missing in web-app module. I searched and found couple of links that suggested it is Deployment Assembly issue in Eclipse. So i need to manually create these folders and add in the deployment assembly because maven does not do it.
But the bigger questions are
do i need to move the Controller classes like com.mycompany.usermgmtsys.controller.UserMgmtController etc.. to src/main/java Or maven should find the controllers from the module jars included as dependency in WEB-INF/lib.
I dont want to do this i.e. putting java file in web-app. i want all the controllers should be available to the web-app as dependency for example WEB-INF/lib/usermgmtsystem.jar. But then wouldnt the Tomcat be looking for controllers in classes folder.
I dont know what should i do ? Any suggestions would be appreciated.
Its the way the eclipse render maven based project. It generally creates two structure. One based on master pom (parent project) and others based on individual module pom. however doing changes in any structure will reflect in the other one. As a practice I do changes in individual module folder structures and is more easy to read too.
Personally I try to avoid multi-module projects as, if you're using the Maven Release Plugin, you are locked into releasing all your modules together.
While this may sound like a convenience the problem arises when you need to do bug fix release to one of the modules - you end up releasing all the modules, not just the module with the bug fix, incrementing their version even though they haven't changed.
You also take a hit if you're running CI with multi-module projects - you're build typically runs over all modules from you root pom but if you're working in a particular module, you end up taking the hit of building those that haven't changed, in effect losing some of the benefits that the modularization was meant to provide.
So, go with independent modules but, and this is the important bit, create a common 'dependency' pom used by each.
A 'dependency' pom is a pom that standardizes all the dependencies across your projects and is different in that those dependencies are specified in the dependencyManagement section rather than the dependencies section (it also sets up standard plugin config, etc). This allows your project poms to specify the dependency pom as their parent and then declare the dependencies they need minus the versions, which are picked up from the 'dependency' pom and thus standardized across your projects.
If you are still concerned about being able to built everything, this can be achieved with a simple batch-file.
This is a good question. There are many aspects that must be considered for a useful project layout. I'd like to try to answer one which you didn't mention. Is your app extensible by users? If it is, then consider creating a separate module for your public API layer (service interfaces, DTOs used by those services, and Exceptions thrown by the services).
In our app, we have several maven modules per functional area. The idea is that a group worked on a feature within just one functional area and this isolation kept them messing with sources being modified by another group. Each functional area is broken down further in maven sub-modules we call "api", "domain", and "service" - we don't lump services/controllers, domain, and exceptions into a single module. The api module contains those classes we want to expose to customers for their customizations. Our service layer is the implementation of those interfaces. Further, we do not allow one module's service to call another module's service as this would bypass our service orchestration layer where customer can attach extensions to our services. Using separate maven modules per functional area helps enforce this.
We have other modules (internal-api, web, adapter) but they don't really add to this topic.
I figured out the issue. Controllers are presentation-layer components. The dispatcher expects the presentation layer components in the WEB-INF/classes folder in the target rather than looking for it in the lib. I am not sure if this is valid only for maven based structuring in eclipse. So finally these are the changes i have made
a. Created a src/main/java source folder in web-app. It is not generated by default in web-app module.
b. Add packages and respective controllers in the src/main/java folder.
So the final structure that i have (i am not pasting exact eclipse snapshot, this is generalized view)
I am working in a small team (3 persons) on several modules (about 10 currently). The compilation, integration and management of build versions is becoming more and more tedious.
I am looking for a good build / integration tool to replace / complete Ant.
Here is the description of our current development environment :
- Several modules depending on each over and on third party JARs
- Some may export JARS, some export WARS, some export standalone, runnable JARS (with Fat-Jar)
- Javadoc for all of them
- We work with eclipse
- Custom Ant script for each module. Many redundant information between the eclipse configuration and Ant scripts. For example, for the standalone Fat-JAR, we have listed all the recursive dependencies, whereas ideally, it could clearly be imported from the eclipse configuration.
- The source code is versioned using SVN
Here is what I would like a perfect integration tool to do for me :
Automatize the releases and versioning of modules. Ideally, the integration tool should detect if a new version is needed. For example, if I want to release a project A that depends on a project B, and if I have made small changes on the project B locally, then the integration tool should first release a new version of B as well and make A based on it.
Integrate strongly with eclipse, so that it could get the dependencies between modules and third party libs from its configuration. BTW, I would like to continue to configure build path with eclipse without updating some other ".xml" stuff. I saw that Gradle can generate eclipse project files from its configuration, but the counterpart would be great.
Enable a "live" and transparent development on local projects. I mean that I often make small changes on the core / common projects while developing the main / "leaf" projects. I would like to have my changes on core projects immediately available to leaf projects without the need of publishing (even locally) the JARs of my core projects.
Store all versions of the releases of my module on an external server. The simplest (shares folder / Webdav) would be the best. A nice web page with list of modules and delivered artifacts would be great too.
I have looked around for many things. From Ant4eclipse (to integrate the Eclipse configuration into my Ant script), to the Maven / Ivy / Gradle tools.
I am a bit confused.
Here is what I have understood so far:
- Maven is a great / big tool, but is somewhat rigid and obliges you to bend to its structure and concepts. It is based on description rather than on scripting. If you go out of the path, you have to develop you own plugins.
- Ivy is less powerful than maven, it handles less stuff but is more flexible.
- Gradle is in-between. It is general purpose. It enables scripting as well as "convention based" configuration. It integrates Ant and extends it.
So at this point I am looking for actual testimonials from real users.
What tools do you use ? How ? Do you have the same needs as me ?
Does it ease your life or get into the way ?
Are there sample some use cases, or workspace skeletons out there that I could use as a starting point to see what these tools are capable of ?
Sorry for the length of this message.
And thanks in advance for you advice.
Kind regards,
Raphael
Automatize the releases and versioning of modules (...)
The concepts of versioning and repository are built-in with Maven and they could fit here.
Maven supports SNAPSHOT dependencies. When using a snapshot, Maven will periodically try to download the latest available snapshot from a repository when you run a build. SNAPSHOT are typically used when a project is under active development.
Maven 2 also supports version ranges (I do not really recommend them but that's another story) which allow for example to configure A to depend on version [4.0,) of B (any version greater than or equal to 4.0). If you build and release a new version of B, A would use it.
Integrate strongly with eclipse
The m2eclipse plugin provides bi-directional synchronization with Eclipse.
Enable a "live" and transparent development on local projects.
The m2eclipse plugin supports "workspace resolution": if project A depend on project B and if project B is in the workspace, you can configure A to depend on B sources and not on B.jar (that's the default mode if I'm not wrong). So a change on B sources would be directly visible, without the need to build B.jar.
Store all versions of the releases of my module on an external server.
As mentioned earlier, this is actually a central concept of Maven (you don't even have the choice) and deploying through file:// or dav:// are both supported.
To sum up, Maven is (probably) not the only candidate but I'm sure it would fit:
Your project isn't that exotic or complex, there is nothing scaring from your description (some refactoring of the structure will probably be required but this shouldn't be a big deal).
Maven also brings a workflow based on best practices.
m2eclipse provides strong integration with the IDE.
But Maven has some learning curve.
CI tools? To me, there's only one: the Hudson CI.
I've setup a software development environment for Java once, with the components:
eclipse IDE
mercurial
bugzilla
maven
Nexus
Hudson CI
and some apache, mysql, php, perl, python, .. for integration.
The hudson was not integrated with eclipse and that was on purpose, because I wanted to build on a separate server. for all the other tools I had a perfect cross integration (like: mylyn on eclipse to talk with bugzilla, m2eclipse for using maven eclipse, a lot of plugins for hudson, ...)
We've been starting to integrate Gradle into our build process, and I can add to the answers posted already that Gradle would also work. Your assumptions are mostly correct, gradle is more off the cuff, but is powerful and allows for scripting and such within the build itself. It seems that most things maven can do, gradle does as well.
Now for your individual points:
Versioning: gradle supports dependency maps, versioning, and if you add in a CI server, you can trigger automated/dependent builds. For example, almost all of our 'deliverables' are .wars, but we have several code libs (.jars) and one executable .jar in development. One configuration is to make the the wars and the "fat-jar" dependent on the shared code libs. Then, when the shared libs are updated, bump the versions on the shared libs, test the consuming projects, then use Hudson's ability to fire dependent projects to redeploy those. There are other ways, but that seems to work best for us, for now.
Integrate strongly with eclipse: You're right, gradle can generate the eclipse files. We tend to only use the eclipseCp (to update .classpath) task once we get going, as only classpath needs changed. It's kind of quirky (grabs your default JRE, so make sure it's right, doesn't add exported="true" if you need it), but gets you 99% of the way there.
Enable a "live" and transparent development on local projects: This is one I'm not sure about. I've only hacked around gradle in this case; by removing the artifact in the consuming project and marked the shared project as such in eclipse, then reverted afterwards.
Store all versions of the releases of my module on an external server: simple and many approaches are supported, similar to Maven.
As far as examples, the docs for gradle are good, as well as the example projects that come with the full zip. They'll get you up and running fairly quickly.
Have a look at Ant Ivy. http://ant.apache.org/ivy/
There are no silver bullets, but in my experience Maven is a great project management tool. Personally, I like to use a comibnation of subversion (for version control), maven (for project/build management) and hudson (for continuous build/integration).
I find the convention brought by maven is really useful for context switching, and great for dependency management. It can be frustrating if jars aren't in the repositories, but you can install them locally and when you're ready you can host your own private repository which mirrors other places. I have had a good experience using sonar.nexus by http://www.sonatype.com/ . They also provide an excellenmt free book to get you started.
It might seem like overkill now, but setting up a good build / test / integrate / release environment now, will pay dividends later. It's is always harder to retro-fit, and it's something you can replicate easily.
Lastly, I happen to prefer Netbeans integration for maven, but that's just me :)
Some of your topics are part of deployment and release management.
You could check out a product like: Xebia DeployIt
(with an personal edition which is free)