Suppose you are working on a big project, which is run on some application server (let's say Tomcat, but it may be also Jboss, or Jetty, or something else). The project consists of a several wars, while each war contains a lot of jar. The whole thing is built using Maven and it takes a lot of time to build it all.
Now, suppose a developer makes a change in just one module that produces one small jar. To continue working and test the change, the developer needs to replace this jar in the relevant wars and restart server (sometimes it's sufficient to redeploy wars). It's much faster then rebuilding the whole application. I have seen a lot of developers (including myself) creating shell scripts for this task.
However, it could be much nicer, if it could be done automatically using maven. Let's say when running "mvn install" the plugin will also go to some predefined location (e.g. ${tomcat}/webapps) and search for all appearances of myjar.jar and replace them with a new version (We have multiple jars, remember?)
Does anyone know about such a plugin? Or may be about some other tool that can do the same task? Or some better idea how to do it?
Updated: Btw, If I don't find a solution, I'll probably implement it myself. So please let me know if you are interested. I'll need some beta testers :)
Updated: So I created the plugin myself. See http://code.google.com/p/replace-file-plugin/ Any feedback is appreciated.
Check out JRebel. It rocks, reduces development/deployment time a lot. There is a 30 day free trial, so check it out for free. I purchased my own license, it took my company a few weeks to approve purchasing it, and I couldn't wait.
Seriosuly man, it rocks. Read about it, it can do what you want.
Well, any features I asked for in this question, I have implemented myself. See http://code.google.com/p/replace-file-plugin/
I'll appreciate any feedback.
If you can write your script as a Ant build, you can use the Maven Antrun plugin to embed the Ant build in your build lifecycle.
Let's narrow the scope to one and only war file, I think you'd scale this up to your scenario yourself with no trouble.
You may try this option: create a final assembly module which runs fast and combines all dependee modules into one war-file. Then, if you initially do mvn clean package install for the whole build, you'd have to mvn clean package install the changed dependee module and re-run the final assembly module to quickly get grab on the updated war. This is quite stateful build scheme (and thus error prone), but after some getting familiar with this approach to change propagation you'd probably think this is the most simple idea that might work, and work faster.
Related
Over the last couple of months I have restructured (renamed) my Maven modules of my project. I discovered some unexpected behavior with respect to how Jenkins treats outdated Maven modules (i.e., they no longer exist in pom.xml files) produced by the build job.
This is depicted in the following screenshot:
As can be seen several modules are skipped (= grey bubble). Sadly, however from the UI, I have no chance to delete them by any means. Deleting the workspace does not help either. Actually, the problem exists in other build jobs as well in case a renaming of one ore more modules was conducted in the pom structures.
The next screenshot shows a German version of the menu options.
Despite I used the latest Jenkins version (2.23), there seems to be no possibility to clean up the ui / modules from this project.
Does anybody have an idea how I can achieve removing this obsolete artifacts without a fresh setup of the whole project or even the complete build server installation.
EDIT-1:
There is an offical issue for that at the Jenkins Project, which seems to be unresolved up to day.
EDIT-2:
I've tried the work arround mentioned in the issue using http://hudson-installation/job/the-project/deleteAllDisabledModules in order to obtain the delete view. However, this page does not exist.
I think this answer from Amedee Van Gasse provides at least a workaround to your problem.
He suggests removing the modules marked as "(übersprungen)" by hand using a Groovy script. His solution also implies why the use of http://hudson-installation/job/the-project/doDeleteAllDisabledModules won't work: The property disabled isn't properly set for the respective modules.
Goto:
http://jenkins-ci/job/<projectname>/<package>$<modulename>/
and click delete.
look at:
http://jenkins-ci/job/<projectname>/modules
It looks like you are in the menu of a build. You first have to go back to the project "Zurück zum Projekt". Afterward you can delete the project (Assuming you have the necessary permissions).
Some things you could try as a last resort:
Delete the job, and then recreate it.
Reinstall Jenkins.
Create a new FreeStyle job and use the same command as you expect developers to use:
mvn clean install
This behaviour is an outstanding bug in the Jenkins Maven Plugin
https://issues.jenkins-ci.org/browse/JENKINS-4561
For me it raises a deeper question of why would you ever want to use a plugin for this functionality. It is a DevOps principle to ensure that the developers use and understand the same techniques as are used to build for release.
I have a web project. I build it using maven(complete all stages in every module, then archiving each module to *.jar and then making war-file).
If I change one line of code in one class I need to run maven build script and it takes ~5 minutes.
How can I see my changes without building the whole application if I changes are within one class and one method?
You should use IntelliJ IDEA for building your project and its Artifacts in the exploded form, so that the classes can be reloaded with hotswap.
With such configuration you can update your application much faster. Also check the tutorials.
For even faster updates consider using JRebel.
You could consider using JRebel. You will need to have a valid license.
You can also can try open source tools just take a look this hotswaping tutorial:
http://www.asjava.com/core-java/how-to-hotswap-java-code-into-jvm-redefinition-example/
IDE misconfiguration is a big source of inefficient time use in our team.
I wanted to know if other teams have tried to check the health of the eclipse workspace with continuous integration.
Eclipse is open source and extensible, and most (all?) of its files are in xml. So it should not be difficult to add a step to continuous integration that checks the health of the workspace, such as no missing Jar files, no errors, etc.
What we have is a separate ant script to do the real builds that go to QA and to the customers. This ant script is run with continuous integration and we have put in place a few simple checks that catch most big showstoppers.
The workspace configuration is a different story and we sometimes detect problems when it's too late (the dev left home).
EDIT: Note that we share our Eclipse config files.
There is some information on building with Eclipse from the command line here.
(Should be a comment, but I can't).
I don't see why you want to do that. Eclipse complains loudly if anything is broken, so leave it to the developer.
What you should consider instead, in my opinion is to write tests that check that everything is as you expect it to in the building process of those builds from source code that the developer has checked in the source repository.
If a build breaks due to a jar is missing in the build, add a check. If a build breaks because it is dependent on a certain feature in the JVM, add a check.
Only ship builds outside of the development team that pass all tests. Those builds that fail, should be fixed by the developer introducing the change that broke the build.
Since you are using Ant, you can create a custom task that verifies the following files against pre-defined ones. If they don't match, report problem:
workspace/.metadata/*.* (whichever configurations you think are important)
workspace/project/.classpath
workspace/project/.project
workspace/project/.settings/*.* (whichever configurations you think are important)
Of course, these files include some hard-coded paths, so you can use regular expressions in the pre-defined files.
If you want to check only simple things like "the project doesn't compile", then just compile the project in the ant script (using the javac task) and see if there are errors.
Another thing - continuous integration should better be IDE-agnostic. I.e. you must have a IDE-less environment (a CI Engine) that compiles the project. Imagine the following:
three developers, one of them accidentally removed a jar from his Eclipse, but the project in the repository is compiling. No need to report problems in that case
one of the developers adds a new jar and commits. The others have not updated. No problems are reported in there workspaces, although after they update, they might get the problem.
That all said, I think you'd better look at Hudson, which is a continuous integration engine. Thus you won't be dependent on IDE settings for your builds.
I'm soon going to check in the very first commit of a new Java project. I work with Eclipse Ganymede and a bunch of plug ins are making things a little bit easier.
Previously I've been part of projects where the entire Eclipse project was checked in. It's quite convenient to get the project settings after a check out. However this approach still was not problem free:
I strongly suspect that some Eclipse configuration files would change without user interaction (from when I used Eclipse Europa), making them appear as changed (as they were changed, but not interactively) when it's time to do a commit.
There are settings unique to each development machine as well as settings global for all developers on a project. Keeping these apart was hard.
Sometime if the Eclipse version was different from others Eclipse would get angry and mess up the project configuration. Another case is that it change the format so it gets updated, and if commited messes up the configuration for others.
For this specific project I have another reason not to commit the project files:
There might be developers who prefer NetBeans which will join the project later. However they won't join within the coming months.
How do you organize this? What do you check into versioning control and what do you keep outside? What do you consider best practice in this kind of situation?
At a minimum you should be check-in the .project and .classpath files. If anybody on your team is hard-coding an external JAR location in the .classpath you should put them up against the wall and shoot them. I use Maven to manage my dependencies but if you are not using maven you should create user libraries for your external JARs with with a consistent naming convention.
After that you need to consider things on a plug-in by plug-in basis. For example I work with Spring so I always check-in the .springBeans and likewise for CheckStyle I always check-in the .checkstyle project.
It gets a bit trickier when it comes to the configuration in the .settings folder but I generally check-in the following if I change the default settings for my project and want them shared with the rest of the team:
.settings/org.eclipse.jdt.ui.prefs - it contains the settings for the import ordering
.settings/org.eclipse.jdt.core.prefs - it contains the settings for the compiler version
In general I haven't noticed Ganymede modifying files without me modifying the project preferences.
I recommend to use maven so that the entire life cycle is outside of any IDE. You can easily create an eclipse project with it on the command line and you can use whatever you want, if it's not eclipse. It has it's quirks but takes out a lot of bitterness when it comes to dependencies and build management.
In our world, we check in the entire Eclipse project and the entire parallel but separate Netbeans project. Our motivations for this were entirely focused on "when I do a checkout, I want a functional configuration immediately afterward." This means that we had to do some work:
Create runnable configurations for each primary IDE (people like what they like). This includes main class, working directory, VM parameters, etc.
Create useful start up scripts for all of our relevant scenarios.
Create edited datasets that don't cause the checkout to take too much longer (it's a big project).
This philosophy was worth cash money (or at least labor hours which are almost more valuable) when our new hire was able to check out the project from Subversion into Eclipse and immediately run a functional system with a (small) real data set without any fuss or bother on his part.
Follow up: this philosophy of "make the new guy's life easier" paid off again when he changed IDEs (he decided to try Netbeans after using Eclipse for quite a long time and decided to stick with it for a while). No configuration was required at all, he just opened the Netbeans project in the same directory that Eclipse had been pointing to. Elapsed switchover time: approximately 60 seconds.
I only ever check in things are done by humans, anything else that is generated (whether automaticly or not) should be easy to regenerate again and is liable to change (as you've stated). The only exeption to this is when the generated files are hard (requires alot of human intervention ;) ) to get it right. How ever things like this should really be automated some how.
Try to port your project to a build system like maven. It has everything you need to get the same experience of the project on every machine you use.
There are plugins for just everything. Like the eclipse plugin. You just type "mvn eclipse:eclipse" and the plugin generates your entire ready to work eclipse project.
To give the answer to your question. Never check in files that are not being used by your project at any time in the development cycle. That means that metadata files like eclipse properties etc. should never be checked in in a SCM.
I like checking in the .project, .classpath, and similar files only if they will be identical on any Eclipse user's machine anyway. (People using other IDEs should be able to check out and build your project regardless, but that issue is orthogonal to whether or not to check in Eclipse-only files.)
If different users working on the project will want to make changes or tweaks to their .project or .classpath or other files, I recommend that you do not check them into source control. It will only cause headaches in the long run.
I use IntelliJ, which has XML project files. I don't check those in, because they change frequently and are easy to recreate if I need to.
I don't check in JAR files. I keep those in a separate repository, a la Maven 2.
I don't check in WARs or JARs or javadocs or anything else that can be generated.
I do check in SQL and scripts and Java source and XML config.
I'd suggest having the actual project files ignored by the version control system due to the downsides you mentioned.
If there is enough consistent information in the project settings that there would be benefit from having it accessible, copy it to a location that Eclipse doesn't treat as special, and you'll have it available to work with on checkout (and copy back to where Eclipse will pay attention to it). There is a decent chance that keeping the actual project files separate from the controlled ones will result in loss of synch, so I'd only suggest this if there is clear benefit from having the settings available (or you're confident that you'll be able to keep them synchronised)
In our case, we used to check in the project files (.project and .classpath) to make it easy for all developers to create their project workspace. A common preferences file and team project set were located in source control as well, so creating your workspace was as simple as import preferences and import team project set. This worked very well, but does rely on everyone having a consistent environment, any customizations would have to be applied after the basic workspace is created.
We still do this for the most part, but Maven is now used so of course dependency management is handled via Maven instead. To avoid conflicting information, the .project and .classpath were removed from source control and are now generated via maven goals before we import the team project set. This would easily allow for different environments, as you would simply need scripts to generate the IDE specific portions based on the Maven configuration.
PS-For ease of maintenance though, I prefer having everyone use the same environment. Anything else inevitably becomes a full time maintenance job for someone.
Netbeans 6.5 has an improved Eclipse project import which is supposed to sync changes from Netbeans back to Eclipse: http://wiki.netbeans.org/NewAndNoteWorthyNB65#section-NewAndNoteWorthyNB65-EclipseProjectImportAndSynchronization
Don't. Only check in the source code of your projects.
As a response to:
"There are settings unique to each development machine as well as settings global for all developers on a project. Keeping these apart was hard."
Eclipse offers a number of ways to keep local settings manageable: Java Classpath Variables (Java > Build Path > Classpath Variables) are one, 'Linked Resources' (General > Workspace > Linked Resources) are another http://help.eclipse.org/stable/index.jsp?topic=/org.eclipse.platform.doc.user/concepts/concepts-13.htm Creating a README that states which settings to set before building/running the project works pretty well in my opinion.
Now how to make sure your continuous build system understands the changes that were made to the eclipse settings, thats another issue... (I have a separate build.xml for ant that I keep up to date by hand)
What is the best way to allow a team of programmers to use Netbeans, Eclipse and IntelliJ on the same project, thus eliminating the "which IDE is better" question.
Which files should or should not be checked into source code control?
I think the best way is to make the build process independent of IDE. This means that your project should not rely on any IDE-specific files to build, but rather use an external build system, like Apache Maven, Apache Ant, or even make or custom scripts. Maven is supported by most popular Java IDEs, either directly or via plug-ins.
If you don't want to use an external build systems, you should at least make the project as easy to set up as possible (i.e. by having standard folders for shared libraries and other dependencies). When I have working on teams with multiple IDEs in the past, I spent by far the most time on resolving dependencies as the prerequisites for building the project changed over time. In the worst case you may even end up with developers not bothering to get the latest version from the version control repository, since they think setting up the new project is such a hassle.
If your project has many library dependencies, I think its a good idea to make these available in binary form in the version control repository. That way people don't have to resolve all the dependencies of the dependencies and so on just to build a single project. This does however require that you have someone responsible for keeping the "official" binaries up-to-date whenever they change. (This is pretty much the same philosophy used by the Maven repository, but the principles can be applied manually even when not using Maven.)
Well, that's a pretty self-answering question.
The files to not check into source control are files that have to do with the IDEs themselves.
Leave it to the developers to generate these files.
If you use Maven, it can generate the files such as Eclipse's .project and .classpath for you. Eclipse in general is very easy to use with a basic file structure (with the new Java Project option).
I think Maven has Netbeans support as well, not sure about IntelliJ though.
Maven's site is maven.apache.org.
For each IDE that has more than one developer, check-in all the supporting files. Why re-invent the wheel at every desk.
I have done this with many different IDEs, and I have yet to see a filename conflict.
In fact, even when only a single developer uses a particular IDE, it is to his/her advantage to version the supporting files, for the same reason that you version the other files in your development environment: history, diffing, comments, etc.
For Eclipse, that would be .classpath and .project files.
My team uses Maven, and developers are discouraged from checking in Eclipse-specific files. Because they can be generated from Maven, these files are redundant.
Also, checking project-specific files seems like it would save time, but it usually winds up being a pain because of variations in different developers' workstations, resulting in wasted time resolving conflicts in the IDE-specific files. The only way to get around that is to force everyone to set up their environment the same way, which goes against the IDE-agnostic approach.
There are many considerations when using multiple toolsets within the same project team. For example, my team has Java developers using IntelliJ and most of the front end (JSP/CSS/HTML) developers using eclipse. We are in the process of migrating the Eclipse users to IntelliJ because of some IntelliJ plugins that we have developed that provide extended support for our environment. We're not going to develop the plugins for multiple platforms, so we are standardizing on IntelliJ across the board.
In terms of specific files, I can speak to IntelliJ. We have checked in our .ipr files and our .iml files. Do not check in .iws files. If you also have Eclipse users, configure your IntelliJ project to read/store dependency information in the .classpath file and commit that to your VCS.
We intentionally support multiple IDEs from the same SVN repository. Our thinking was that we want to ensure that, if a new person joined the team or someone had to start working on a new machine, we wanted them to be able to checkout the codebase, import it to the IDE and immediately have a work-able configuration.
What that means on the developer end is that they should not commit their changes to the IDE files. Everything else (e.g., src, test, lib and so forth) becomes the set that we normally update and commit every day.
The side benefit is that we have completely eliminated the IDE wars here: Netbeans and Eclipse people live in perfect harmony (looking askance at the IntelliJ people, but hey... ;-).
For more comments and answers on this topic see this question (How do you handle different Java IDEs and svn?)
We rename our IDE files for checkin with an extra extension .deletethis or similar. When a new person checks out the project, they simply strip off the extra extension and are good to go. This way we avoid source control conflicts with the project files as people tweak their environments. And you don't have to worry about educating new developers to not check in those files.
Typically, I would consider this a bad idea. I'm not sure what kind of environment this is (perhaps open source?), but it would really suck to support multiple IDEs. One thing I would recomend if this is unavoidable, would be to standardize your builds in ant scripts. If you have a large set of dependencies, this may be the easiest way to get a predictable build across all platforms.
If one of the IDEs happens to be RAD (based on eclipse), there is an entire folder called .settings that you would not want to include in the SCM.