I am facing one problem and not sure what is the best way to go,
So I have two git repositories with Spring projects which have to share the same database. (Both are spring projects with hibernate).
One of them is main project so-called MASTER which should modify all Hibernate entities and other I will call SLAVE which is secondary project and needs to read only from the same database.
Here is the small illustration what I have.
So the issue appears when I realized that need to keep the duplicate of entities in both master and slave.
I found two ways to go with this issue.
Using git submodules. Where I can have my entities to be an independent submodule.
Building a JAR from entity classes and include it in both projects.
This both solutions are not meeting my requirements which are:
The solution of submodules is not good because whenever I commit anything from the MASTER I want SLAVE to track that changes. Please note, I have 3 git branches for both projects, master, staging and production. So all the branches should have accordingly their version of entities.
The solution of the jars will work, just I do not find it nice and solid, as I should all the time build them and add a dependency for every project.
The development of these projects is done independently from each other.
Please, could you share your opinion on this issue?
I kind of sure that I am not only the one who is trying to achieve the same.
You should consider publishing your jar to a maven repository for easier exchange between the projects. You could even host your own like sonar nexus. https://www.sonatype.com/nexus-repository-sonatype
Personally, I think that managing the versions can be very annoying when you have multiple projects. Especially when you are testing something and you have to create a new jar and then publish it over and over again. However your project will be rebuildable and you can controll which project/module can use a newer version of your entity-dependency.
Related
We recently migrated from SVN, with most code in a single repo, to git, with most projects in their own repos (about 70 of them). We build about a dozen different apps from this java source. The apps all run on *nix servers. We use maven and nexus to build. Many of us are struggling with developing features when that feature touches more than one repo. Here are a few of the challenges:
The developer has to branch each repo separately - we use the same name for all branches for one feature to make tracking less difficult.
One must update poms of all repos to point to the updated versions of each repo's artifact. If multiple people are working on the same branch, there can be a lot of merging others pom changes. When I commit a change to a repo, then the artifact is renamed to "-SNAPSHOT" which means more pom updates.
Changes need to be pushed in the right order or our automated builds will fail, e.g: repo A depends on a change to repo B; if repo A is pushed before repo B is built and deployed, then repo A won't build.
The person reviewing the feature has to look at changes in multiple repos.
When the feature is merged from its branch to, say, master, One has to remember all the repos that were touched.
It looks like switching to a mostly monorepo approach might be best, tho there are some drawbacks there:
Building the entire codebase with maven takes a looong time. (Why can't maven be more like make, only building things that have changed or whose dependencies have changed?)
Each push kicks off a big set of builds and many unit tests rather than just one repo's artifact build and test.
The developers who generally work in one or two repos prefer this new multi-repo world and will resist a change back.
I've looked into git submodules and sub trees, which don't seem to solve many of our issues (Not sure about Google Repo). Some of us use tools like "mu" to help. It would be sweet if there was a toolkit that would help developers maintain versions in poms, and track changes across repos.
Let me know if you have a set of procedures or tools you use to ease development in this kind of environment.
with most projects in their own repos (about 70 of them).`
For me this is where the problems start. My vote goes for minimising this number significantly.
If you really don't want a single repo (1 repo gets my vote) then you could separate the code base into n*change_often repos with 1*change_rarely repo. Keeping the n small is important. This way you would avoid rebuilding the bits that change rarely.
Also, even with the a single repo you don't need to reference everything by source and use binaries for base libraries. When a base library changes the person making the change could also update all the references in one go so that that all projects are up to date.
I have a very well structured big project which is maintained in SVN in my team. And we not want to move to Git - which will further integrated with continious integration servers like Teamcity or Jenkins (yet to decide).
When I saw the svn I found that since svn allows you to create tags from anywhere - It has a lot of tags which only contains single-single projects.
The codebase is so huge that I import all the code in one git repo. One approach would be for each project - I can create seperate repos, or combine some of the project to one repo (not sure)
The thing I want to achieve is I want to retain all the history and break the repo so that it becomes easy to manage. (Also it ll be helpful in future while integration with jenkins for automated builds on every commit)
How can I ensure to retain history, have all the existing tags, and move to git
git-svn is the obvious solution to help with this migration but, in my opinion, this is better for a one-off migration from SVN to git.
If you want to keep both the SVN and git repositories alive and allow commits to both (keeping them both in sync), I would recommend SubGit.
When we moved from svn to git I found this stackoverflow post very helpful, specifically the second answer. I assume from your description that everything is under a single trunk. That does make it a little more complicated. I have not tried moving individual projects before, but you may be able to do it in one of two ways:
Migrate the entire svn repo, tags, branches, etc. to your local git repo (before you push to the remote). Then break up the projects by following the suggestions in (this) stackoverflow post. This should give you individual repos that you can then push to the remote repos.
You may be able to alter the steps when running the svn to git conversion and specify an individual project, but that seems dangerous and confusing since the tags/branches won't necessarily line up.
I have two Java projects with same domain objects.First project is the administration of a webapp. And second project is the webapp.
I've chosen this approach in order to allow deployment of administration without downtime for my webapp.
So both projects use same database. I'm using spring-data and marking entities with #Entity.
My question is: is there any way to replicate domain objects in each project?
For example creating another maven module with the domain objects and mark as a dependency. (But in this case #Entity will still work?).
the way is just as you said it - create a maven module (usually called datamodel, infomodel or something along those lines) that contains all of your JPA classes (#Entity classes).
this model can either be a completely separate 3rd project (more work) or, more likely, pick one of the 2 projects as the "owner" of the module and the other project will simply list it as a dependency. in both cases you'll need to think about things like version compatibility (what happens when you update administration but not the webapp and the entities changed? who updates the database, how do you make sure the older code can still read/write it?)
as for working, JPA classes work just fine in their own jar.
Like you have raised is a good approximation, separating two different maven projects.
What you're trying to do is very similar to the structure of Broadleaf Commerce. It is a multi-module project using Maven and Spring, is open source, so you can look at how it is structured to see if it helps.
Here you have another guide of how to implement it step by step. Hope it helps.
It seams that you will need at least three modules.
1st - the domain module with the enitity annotated domain classes;
2nd - the application itself witch depends on the domain module;
3rd - the adm module witch also depends on the domain.
Now that you have a multi-module maven project you should have a 4th project formally listing the other three as its child modules.
P.s.: Resist to the temptation of creating separate git repositories and evolving the versions of the modules separately.(just an advice)
I have a large scale project I am working on at the moment using Eclipse. Normally, as a one man team, these problems would not be an issue, but as our team is not one person we need to be able to break up pieces of the project to be worked on by certain team members.
In simplicity, let's say I have two layers to be separated apart:
1. Each DAO is a separate Java project, to be worked upon individually
2. The web-tier service layer contains all of our service endpoints and must be able to reference all of the DAOs. This layer runs on Tomcat as a dynamic web project, and utilizes Adobe LiveCycle Data Services as the piece that handles creation and management of endpoints.
Now, the issue we are running into is that when we create a DAO and unit test it individually it runs great. But when we reference it into our service project and try to run it we begin to get all kinds of issues related to the fact that we have two different versions of certain jars referenced in and as such we begin to have errors when running the server.
As a result, we know we can solve the issue by pulling the problem jars and ensuring that this is not an issue again in the future, but as I said before this is a large scale project with multiple people working on it and we don't want to be spending our time weeding out dependency issues when under the gun.
We are looking for recommendations on where to proceed for alternative solutions? Our team is new to JavaEE and as such we don't have much of a bearing on what we can use to tie everything together in it, or if it is a viable solution. Should we be looking at turning our DAOs into EJBs and deploying them in an EAR library? If so, where would our service layer lie, and would the service layer be able to reference the DAO classes since the EJB maintains it's own classpath (from what we have read?) Are we looking down the wrong path, or are we completely wrong in our current understanding of JavaEE?
Any assistance is greatly appreciated. We are still in the framework stage of this project and we want to be sure that we will be able to maintain it in the long run.
I second the Maven recommendation. That can add all sorts of sanity to your project structure.
Maven can even generate Eclipse workspaces via mvn eclipse:eclipse
An important clarification on the EJBs note. As of ava EE 6 is you no longer need to separate EJBs from Servlets and can use them together in the very same jar in the war file.
So understand from that that using EJBs or not no longer has any impact on packaging or classloaders as it once did. These are now separate decisions. EARs and classloader separation should now be viewed as a feature you might want to use should you want classloader separation and the complexity it brings. Most applications simply do not need that and are more than fine with just a war file containing servlets, ejbs, jpa entities, cdi beans, jaxrs services and whatever else you need. You are free to decide how you want to separate them or if you want to bother separating them at all.
EJBs do make great DAOs due to transaction management, something you don't get from plain Tomcat but can be made available in Tomcat via TomEE and works fine in Eclipse. You should consider EJBs for that reason, not for dependency reasons.
Side note, as you're new to Java EE, you might find this helpful:
http://openejb.apache.org/examples-trunk/index.html
In order to have things organized when working with Java EE in teams of 1+ people I could suggest:
Use Maven to manage your build process and library dependencies.
Maven has a small learning curve, but once you grasp it you will be grateful. By using Maven you no longer depends on Eclipse to manage your classpath.
A thing about it that I think is really helpful when working in teams is the install feature. Suppose you are woking on the version 1.0 of an EJB module, say core-ejb-module-1.0, and you've got it to a stable state and want everyone working in the project to refer to it from now on.
You then run a maven command like this on it: mvn clean package install
Maven will clean this module, compile it, run tests, create the jar and then install it to a repository that you define. Could be any computer in your company.
Now you may tell the guys working on other projects to update this dependency version on their .pom file and in the next build they run, before compiling, maven will download this library and then use it. Really neat. No more classpath hell.
(There are other ways to always automatically refer to the latest library as stated in this post, but there are some caveats. Anyway it's just an example.)
Use JPA/EJB instead of DAO Pattern.
Some people say DAO meaning any sort of data access, others really mean that they use the DAO Pattern to access objects. If that is your case, you no longer need to use it when using JPA. (At least for most common scenarios).
In my case, I have a generic EntityService which is capable of doing CRUD operations on any Entity and has a centralized query management. Then every EJB's that should perform database related operations may inject this guy and do its job.
As a suggestion, with Maven, you project could be organized as such:
core project structure
core (The pom root)
core-ejb-module (Includes all generic EJB's, like the EntityService for instance.)
core-jpa-module (Includes all JPA generic definitions, like Interfaces, MappedSuperclasses and such.)
core-jsf-module (Includes all JSF generic definitions, like abstract controllers, generic converters and wrappers for FacesContext, etc..)
Now that you have a core generic module setup, you could create:
an application structure
app (The pom root)
app-ear-module (Includes all other modules in this application. Shared jars goes in the ear /lib folder, so all other modules could reference to them.)
app-ejb-module-a (Includes EJB's for the business layer of your application. It uses the core-ejb-module)
app-ejb-module-b (You may have lots of ejb modules. You may even have a project that contains only ejb modules. Other apps will declare their dependency on them via Maven.)
app-jpa-module (Contains definitions for JPA Entities that represents you database tables. Depends on the core-jpa-module)
app-web-module (Holds the pages, Controllers and Converters for this application.)
I think you got the idea. Things tend to be loosely coupled and you may organize your projects as you like.
This is just a simple example to illustrate. I didn't explained a lot about Maven but if you're interested I think it will help you indeed.
I hope it gives you some ideas and may help you in any way.
[]'s
If you can run all the sub-components using the same set of dependencies, you may find it helpful to migrate to a Maven build.
With Maven, you can define a top-level project that manages all the 3rd party dependency versions in one place, so all modules are built, tested and deployed against the same library versions. You are also likely to find Maven a good fit for the multi-module approach you have adopted, as it ensures that a project is rebuilt correctly if one of its dependencies changes.
You would still be able to use dynamic web projects as before; Eclipse will automatically deploy the DAOs as part of the service project (IIRC you need to characterise the DAOs as utility modules).
If you do go down the EJB root, you are correct that each EAR will get its own class-loader, and can therefore use a varying set of dependencies. However, in your position I would tend to look at improving your dependency management first - it'll probably be cheaper and easier.
I have recently joined a project that is using multiple different projects. A lot of these projects are depending on each other, using JAR files of the other project included in a library, so anytime you change one project, you have to then know which other projest use it and update them too. I would like to make this much easier, and was thinking about merging all this java code into one project in seperate packages. Is it possible to do this and then deploy only some of the packages in a jar. I would like to not deploy only part of it but have been sassked if this is possible.
Is there a better way to handle this?
Approach 1: Using Hudson
If you use a continuous integration server like Hudson, then you can configure upstream/downstream projects (see Terminology).
A project can have one or several downstream projcets. The downstream projects are added to the build queue if the current project is built successfully. It is possible to setup that it should add the downstream project to the call queue even if the current project is unstable (default is off).
What this means is, if someone checks in some code into one project, at least you would get early warning if it broke other builds.
Approach 2: Using Maven
If the projects are not too complex, then perhaps you could create a main project, and make these sub-projects child modules of this project. However, mangling a project into a form that Maven likes can be quite tricky.
If you use Eclipse (or any decent IDE) you can just make one project depend on another, and supply that configuration aspect in your SVN, and assume checkouts in your build scripts.
Note that if one project depends on a certain version of another project, the Jar file is a far simpler way to manage this. A major refactoring could immediately means lots of work in all the other projects to fix things, whereas you could just drop the new jar in to each project as required and do the migration work then.
I guess it probably all depends on the specific project, but I think I would keep all the projects separate. This help keep the whole system loosely coupled. You can use a tool such as maven to help manage all the dependencies between the projects. Managing dependencies like this is one of maven's main strengths.
Using Ant as your build tool, you can package your project any way that you want. However, leaving parts of your code out of the distribution seems like it would be error prone; you might accidentally leave out necessary classes (presumably, all of your classes are necessary).
In relation to keeping your code in different projects, I have a loose guideline. Keep the code that changes together in the same project and package it in its own jar file. This works best when some of your code can be broken out into utility libraries that change less frequently than your main application.
For example, you might have an application where you've generated web service client classes from a web service WSDL (using something like the Axis library). The web service interface will likely change infrequently, so you don't want to have the regeneration step reoccurring all the time in your main application build. Create a separate project for this piece so that you only have to recreate the web service client classes when the WSDL changes. Create a separate jar and use it in your main application. This style also allows other projects to reuse these utility modules.
When following this style, you should place a version number in the jar manifest so that you can keep track of which applications are using which versions of your module. Depending on how far you want to take this, you could also keep a text file in the jar that details the changes that have occurred for each revision (much like an open source library).
It's all possible (we had the same situation some years ago). How hard or easy it'll be depends on your IDE (refactoring, merging, organizing new project) and you build tool (deploying). We used IDEA as IDE and Ant as build tool and it wasn't too hard. One sunday (nobody working+committing), 2 people on one computer.
I'm not sure what you mean by
"deploy only some of the packages in a jar"
I think you will need all of them at runtime, won't you? As I understood they depend on each other.