Related
For past 4 years, I have been programming with Eclipse (for Java), and Visual Studio Express (for C#). The IDEs mentioned always seemed to provide every facility a programmer might ask for (related to programming, of course).
Lately I have been hearing about something called "build tools". I heard they're used almost in all kind of real world development. What are they exactly? What problems are they designed to solve? How come I never needed them in past four years? Are they kind of command-line stripped down IDEs?
What are build tools?
Build tools are programs that automate the creation of executable
applications from source code (e.g., .apk for an Android app). Building
incorporates compiling,linking and packaging the code into a usable or
executable form.
Basically build automation is the act of scripting or automating a
wide variety of tasks that software developers do in their day-to-day
activities like:
Downloading dependencies.
Compiling source code into binary code.
Packaging that binary code.
Running tests.
Deployment to production systems.
Why do we use build tools or build automation?
In small projects, developers will often manually invoke the build
process. This is not practical for larger projects, where it is very
hard to keep track of what needs to be built, in what sequence and
what dependencies there are in the building process. Using an
automation tool allows the build process to be more consistent.
Various build tools available(Naming only few):
For java - Ant,Maven,Gradle.
For .NET framework - NAnt
c# - MsBuild.
For further reading you can refer following links:
1.Build automation
2.List of build automation software
Thanks.
Build tools are tools to manage and organize your builds, and are very important in environments where there are many projects, especially if they are inter-connected. They serve to make sure that where various people are working on various projects, they don't break anything. And to make sure that when you make your changes, they don't break anything either.
The reason you have not heard of them before is that you have not been working in a commercial environment before. There is a whole lot of stuff that you have probably not encountered that you will within a commercial environments, especially if you work in software houses.
As others have said, you have been using them, however, you have not had to consider them, because you have probably been working in a different way to the usual commercial way of working.
Build tools are usually run on the command line, either inside an IDE or completely separate from it.
The idea is to separate the work of compiling and packaging your code from creation, debugging, etc.
A build tool can be run on the command or inside an IDE, both triggered by you. They can also be used by continuous integration tools after checking your code out of a repository and onto a clean build machine.
make was an early command tool used in *nix environments for building C/C++.
As a Java developer, the most popular build tools are Ant and Maven. Both can be run in IDEs like IntelliJ or Eclipse or NetBeans. They can also be used by continuous integration tools like Cruise Control or Hudson.
Build tools are generally to transform source code into binaries - it organize source code, set compile flags, manage dependencies... some of them also integrate with running unit test, doing static analysis, a generating documentation.
Eclipse or Visual Studio are also build systems (but more of an IDE), and for visual studio it is the underlying msbuild to parse visual studio project files under the hood.
The origin of all build systems seems like the famous 'make'.
There are build systems for different languages:
C++: make, cmake, premake
Java: ant+ivy, maven, gradle
C#: msbuild
Usually, build systems either using a propriety domain specific language (make, cmake), or xml (ant, maven, msbuild) to specify a build. The current trend is using a real scripting language to write build script, like lua for premake, and groovy for gradle, the advantage of using a scripting is it is much more flexible, and also allows you the to come up with a set of standard APIs(as build DSL).
These are different types of processes by which you can get your builds done.
1. Continuous Integration build: In this mainly developers check-in their code and right after their check-in a build initiates for building of the recent changes so we should know whether the changes done by the developer has worked or not right after the check-in is done. This is preferred for smaller projects or components of the projects. In case where multiple teams are associated with the project or there are a large no. of developers working on the same project this scenario becomes difficult to handle as if there are 'n' no. of check-in’s and the build fails at certain points it becomes highly difficult to trace whether all the breakage has occurred because of one issue or with multiple issues so if the older issues are not addressed properly than it becomes very difficult to trace down the later defects that occurred after that change. The main benefit of these builds is that we get to know whether a particular check-in is successful or not.
2. Gated check-in builds: In this type of check in a build is initiated right after the check in is done keeping the changes in a shelve sets. In this case if the build succeeds than the shelve-set check-in gets committed otherwise it will not be committed to the Team Foundation Server. This gives a slightly better picture from the continuous integration build as only the successful check-in's are allowed to get committed.
3. Nightly builds: This is also referred as Scheduled builds. In this case we schedule the builds to run for a specific time in order to build the changes. All the previous uncommitted changes from the last build are built during this build process. This is practiced when we want to check in multiple times but do not want a build every time we check in our code so we can have a fixed time or period in which we can initiate the build for building of the checked-in code.
The more details about these builds can be found at the below location.
Gated-check in Builds
Continuous Integration Builds
Nightly Builds
Build Process is a Process of compiling your source code for any errors using some build tools and creating builds(which are executable versions of the project). We(mainly developers) do some modifications in the source code and check-in that code for the build process to happen. After the build process it gives two results :
1. Either build PASSES and you get an executable version of your project(Build is ready).
2. It fails and you get certain errors and build is not created.
There are different types of build process like :
1. Nightly Build
2. gated Build
3. Continuous integration build etc.
Build tools help and automates the process of creating builds.
*So in Short Build is a Version of Software in pre-release format used by the Developer or Development team to gain confidence for the final result of their Product by continuously monitoring their Product and solving any issues early during the development process.*
You have been using them - IDE is a build tool. For the command line you can use things like make.
People use command line tools for things like a nightly build - so in the morning with a hangover the programmer has realised that the code that he has been fiddling with with the latest builds of the libraries does not work!
"...it is very hard to keep track of what needs to be built" - Build tools does not help with that all. You need to know what you want to build. (Quoted from Ritesh Gun's answer)
"I heard they're used almost in all kind of real-world development" - For some reason, software developers like to work in large companies. They seem to have more unclear work directives for every individual working there.
"How come I never needed them in past four years". Probably because you are a skilled programmer.
Pseudo, meta. I think build tools do not provide any really real benefit at all. It is just there to add a sense of security arising from bad company practices, lack of direction - bad software architectural leadership leading to bad actual knowledge of the project. You should never have to use build tools(for testing) in your project. To do random testing with a lack of knowledge of the software project does not give any sort of help at all.
You should never ever add something to a project without knowing it's purpose, and how it will work with the other components. Components can be functional separate, but not work together. (This is the responsibility of the software architect I assume).
What if 4-5 components are added into the project. You add a 6th component. Together with the first added component, it might screw up everything. No automatic would help to detect that.
There is no shortcut other than to think think think.
Then there is the auto download from repositories. Why would you ever want to do that? You need to know what you download, what you add to the project. How do you detect changes in versions of the repositories? You need to know. You can't "auto" anything.
What if we were to test bicycles and baby transports blindfolded with a stick and just randomly hit around with it. That seems to be the idea of build tool testing.
I'm sorry there are no shortcut
https://en.wikipedia.org/wiki/Scientific_method
and
https://en.wikipedia.org/wiki/Analysis
I am curious to understand how Java tests its APIs. Let's say,I am interested in the class ConcurrentHashMap, will there be any unit tests for this class? If so, is it available for public?
When you say "Java" you probably mean the Java Development Kit (JDK), which comes as OracleJDK and OpenJDK (OracleJDK is essentially OpenJDK with a few extras). OpenJDK is open-source; and the source code for all of its projects can be found here:
http://hg.openjdk.java.net/
In particular, here is a browsable version of the jdk7 project directory.
I am curious to understand how Java tests its APIs. Let's say,I am interested in the class ConcurrentHashMap, will there be any unit tests for this class? If so, is it available for public?
Yes, for a list of all jdk7 tests look in jdk7/jdk/test.
If you are interested in ConcurrentHashMap tests, look in jdk7/jdk/test/java/util/concurrent/ConcurrentHashMap:
NOTE: The JDK tests may look a bit awkward because it does not use JUnit, it uses JTreg.
Any class packaged inside jdk like ConcurrentHashMap is tested and you don't need to worry almost all the cases. If you implementing a well defined interfaces then there are test cases available which can run against your API and check whether it's complacency. In addition if you define a new API which you will expose, you have to write your test cases using JUnit or TestNg. Here the purpose of your test cases should be checking whether your API giving expected results. If you cover all the combination of using your API then you can identify when there is a regression.
Offtopic : Integration testing suppose to test the whole project when all the components are integrated.
[Update] If you really need to test ConcurrentHashMap you can use a sample project which uses ConcurrentHashMap, please keep in mind that you better choose multi-thread project.
You probably won't be able to get the source (and thus the testing code) for Oracle's Java. However, since OpenJDK theoretically follows the same specifications for behavior as Oracle Java (at least for all the parts of it that I'm familiar with), you could perhaps use their unit tests to test your library. The source for OpenJDK's jdk8 can be browsed here (see the test directory for tests), and you can find other versions of java at their main page.
However, as Erranda points out, you should really ask yourself if you want to test the existing libraries. They are probably more thoroughly tested than anything you'll end up writing already, and unless you want to recompile Java from source after finding an issue (which could lead to compatibility problems), what good would finding a bug in the libraries do?
What is the best way to handle situation, when a Java desktop application (executable JAR) requires specific (or newer) version/update of Java, to not get some NoClassDef exceptions?
Details: I have JavaFX 8 desktop application, build with Maven and com.zenjava.javafx-maven-plugin to executable JAR. I'm using new features of JDK 8u40. I would like to nicely handle situation, when someone is trying to run it with older Java version.
First, most low-end approach is to check
System.getProperty("java.version");
at the start of main function, parse it with a couple of "ifs" to extract is it major >= 8 and update >= 40, if not show some popup instead of app. But it seems to me like mediocre solution.
I wonder are there better ways? I'd love to know more about direct code solutions, external libs, additional jar launchers or Maven options for this purpose.
Edit: additional info - I would like to avoid platform-specific solutions
The solution which mostly works: just create a native bunde with included JRE. This is the default when calling mvn jfx:native, so you shouldn't worry about the class being there, because you bundled that specific version.
That is the main reason why some are using that option. Just imagine the situation you are require Java 8 but the user just uses Java 7 and so on, it makes the bundle bigger, indeed, but it preserves you from such problems.
Disclaimer: I'm the maintainer of the javafx-maven-plugin, there are example-projects ;) but no cookies yet.
While i'm developing in C/C++ and Java, i simply make a compile.bat script that does everything, that's fine for me. Why should i use make and why should i use ant?
Suppose you have 1000 source files and change just one of them. With your .bat script you will have to recompile the lot, with make you recompile just the one that changed. This can save quite a bit (read hours on a big project) of time.
Even better, if you change one of your header files, make will re-compile only the source files that use that header.
These are the two main features that have meant make and its offspring are used for all serious software development with compiled languages.
With a build file, you can automate more than just compiling your code; you can run unit tests, gather metrics, package build artifacts for deployment, and more.
An advantage, of sorts, of Ant is that it inspired tools for other platforms - NAnt for .NET, Phing for PHP. They do the same things, and work in the same way.
As long as you develop for yourself under Windows: suit yourself.
But if you start to develop with others Ant and Make are a standard to describe how your App is built.
There may be several reasons:
- because you are not the only one in the project
- because somebody will have to care about the build script when you left
- because a compile.bat script is not platform independent
- because the policy of the project defines the build technology, e.g. for the entire enterprise
I recently read a funny article about build tools. For you only the first part may be of any interest (before the maven bashing starts)
For one thing, make and ant both keep track of which files have already been compiled, so it does not redo work if it isn't needed.
You gain some platform-neutrality (how can I run your batch-script on my linux-box). But more important: build-tools have support for dependency-resolution. Scripts lack this and have to build it on their own.
Say you have targets A, B and C. B and C both depend on A. In an script they call the subroutine A. If you now create a new target D, that depends on B and C, than you will execute A twice. Build-tools recognize this and execute A only once.
Because this situation is typical for task involving compiling, building a distribution, testing, building documentation etc. build-tools are useful for software-development.
It would be possible to use a .bat file -- or bash or another flavor of shell script -- to do everything Ant can do, I believe. But it's far easier to do many things in Ant...
create/move/delete files and directories
apply filter tokens
run unit tests
package a jar, zip, or war properly
Many projects need to do these things, in addition to compiling, when they build.
Also Ant is platform-independent, so you can collaborate with those who use other operating systems.
But why stop at Ant? Apache Maven offers even more compelling features.
Make and Ant really come into their own when used with automated build systems or continuous integration.
The requirements for production builds, test builds and development builds are often different, and on a multi-person team, different developers set up their development environments in different ways.
Having a definitive build process solves the "Works on My Machine" (WOMM) problems that arise whenever the application needs to be built on a different machine from the one that it was developed on. It is the ultimate arbiter when one developer checks in code that doesn't work on another developer's machine. If the build process doesn't produce working software, then the first developer's check-in was broken. If it does produce working software then the second developer's development environment is broken - unless proven otherwise.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am trying to revamp our build process, which is currently a gigantic Ant build.xml that calls into other ant build files and executes several Java classes to perform more complex logic that would be impossible/scary to attemp in Ant.
Background:
experience in Java and Ant, some Groovy
Windows platforms
Goals:
run as a combination of command line cron and when a servlet is posted to
as simplified as possible, fewest languages and bouncing between techs
I need higher level logical power that a language like Java provides and Ant is pretty easy and we use the filtering to override default properties files for different clients. Mostly I'm wondering if there is something other than Ant/Java that people use.
Except the Ant you mentioned and the scarry make/autotools, the mainstream tools are:
SCons
Jam
CMake
Maven
I use SCons, because it is python based, well-funded and elegant.
Jam seems to be the most pragmatic one.
I don't know too much about CMake.
Maven may be the choice for you as it is Java centric and more high level than Ant.
More you can find at wikipedia: List of built tools
If you pursue Maven, then you will have two problems: a complex build and learning the f#*#ing "magic" of Maven. Maven just makes the problem worse because it is obtuse and overly-complicated.
I inherited a legacy Maven 1.x build at a large Fortune 500 company. I used Maven 2.x by choice on many other projects in recent years. I evaluated Maestro, in hopes that it might make Maven tractable. My conclusion, like many other peoples' (check the 'net), is that Maven is a big step in the wrong direction. It definitely is not an improvement over Ant.
I have used Ant for MANY years, including writing a large open-source library of Ant helper scripts. I have also extensively used its .NET cousin nAnt. However, Ant has two major failings. One, XML is simply not the right place to be doing build tasks. Two, Ant and XML do not scale well to large, complex builds. In fact, I have written a lot here at SO about my experiences in that arena (and with Maven).
Industry leaders have concluded that a build is just another application, and should be approached using general application tools. However, since it involves system-level and cross-platform functionality, most development languages/platforms are not properly suited (which includes Java, and therefore Ant and Maven). That also excludes .NET.
I spent two years looking for an alternative, and I found it: Python. It has the right combination of system-level access, cross-platform portability, simplicity, readability, power, robustness, and maturity. SCons, buildbot, setuptools/easyinstall, and base Python are my current target platform for the build process. When necessary, integration with Ant, Maven, and any other such tool is easy. Meanwhile, I can use these tools for the core of any build on any platform with any source language. No more roadblocks, no more crazy complexity, no more supposedly-helpful "declarative" scripting, no more black-box f#*#ing "magic".
If you can't switch to Python, then try Ant + Ivy (at apache.org). It gives you Maven's cool repository without most of Maven's evils. That is what I am doing as well, where necessary and suitable.
Best wishes.
Also take a look at
Gant
Gradle
While more general purpose build systems like SCons are very powerful, the Java support is somewhat limited in comparison to systems specially tailored to build Java projects.
I like using Rake as you can fall back on the power of the whole Ruby language and it's framework library when needed.
I use Ant, taking advantage of its macro feature. If you layout your project in a consistent mannner, you can eliminate a lot of the duplication by writing macros.
I've been building up an Antlib containing macros and custom tasks that I reuse across multiple projects.
Alternatively, some people swear by Maven. Other people just swear about Maven.
I use Maven, not just for build, I also use their release/dist plugin.
In a pair of commands I can have code that was in a source control to build, package and release.
The release plugin handles updating the version numbers, dist handles how to put everything together and zip it.
Ant looks hard when compared to Maven. Sure there is a learning curve with Maven, but reading a pom.xml is far easier than reading a build.xml.
Maven needs to be far less verbose.
I like Ant but only if you spend the time to write your own Java plugins to encapsulate complex actions. It isn't hard but unfortunately most people try to write their logic in XML w/the ant-contrib stuff. I think this is a huge mistake.
I've heard good things about rake and the groovy tools (mentioned in another comment) but I've got no experience with them.
If you're trying to script together several steps in a life-cycle you might be better off using a process automation based build server like AnthillPro (Cruise, BuildForge and Electric-Commander are the others in this space).
Another place to ask this kind of question is the CITCON mailing list. CITCON is a conference on Continuous Integration and Testing and the associated mailing list has turned into a really great community around these kinds of topics.
(I'm an organizer for CITCON but a labor of love not a profit maker. It really does have a really helpful mailing list. If I was pimping something for money it would be The CI Guys. ;-) )
Stick with Ant since you're building Java. I've mixed Ant/SCons for some JNI work, but in general I'd stay with Ant especially since you've got an existing build setup in Ant. Porting to Maven will be like shoving a square peg through a wall with no holes.
Embrace your custom Java logic for any and consider writing proper Ant tasks instead of executing external Java code. I solved some very complex parts of our build process by simply extending Ant to do exactly what I needed, ex. managing icon resources for a large gui project or injected subversion information directly info jar manifests (thank you SVNKit)
I'd go with Ant any day of the week.
It's not perfect; XML is very verbose and implementing any logic at all is almost impossible, but even the most junior engineer on the team can at least understand what the ant file is doing within a day.
Complex logic can be refactored using java and integrated in ant, if you so wish. Ant gives you all the power of java:)
Dependency resolution is difficult no matter what system you use. With ant, the best solutions seem to be either a lib directory in which all your jars are stored, or an internal web server from which the libraries are copied at build time.
I also have some experience with both Maven 1 and Maven 2. That experience left me with the feeling that Maven is awesome for hobby projects, but can have complications for software you need to maintain over time.
I see two important problems with Maven:
Any dependency you import using Maven may change over time without you knowing it, resulting in weird problems.
You import the licenses of not only the software you import directly using Maven, but also the licenses used by the libraries which are indirectly imported
In short, I dislike the fact that the build depends on the time it is started. That can be a true problem when producing a bugfix release a year after the production release.
These problems can of course be managed (maybe using a nexus proxy) but you need to consider them before rebuilding the build system. At my company we decided to use Ant for all new project and try to port maven 1 and 2 to ant whenever the occasion presents itself. It's just too difficult to keep it working.
My advice, if you and your team know how to deal with ant, try to refactor your ant file and don't jump on some other build tool. It just takes too much time to get it right; time you could spend making money and surviving as a company :)
I use Rake everywhere I can. Where you need to build java code you might use it with jruby, or look at something like: buildr
We use luntbuild. It's a web app that's very easy to use. It will check out from CVS/SVN, automatically increment the version/build number, execute build tasks in ant scripts and tag your repository with the new version. Can schedule automatic builds, have it email you or contact you via IM, also has security and build dependencies.
We're still using version 1.2.3 but I see it's up to 1.6.0. It just works.
http://luntbuild.javaforge.com/
EDIT: Re-reading your question I see now that you're looking for something to replace Ant. In our case Ant is not really a problem. We have projects setup in Netbeans, which uses Ant for building and we just have to implement some hooks into the existing scripts provided by Netbeans which is very easy to do.
EDIT: Looks like you can call Ant from Groovy. That would be nice because then you can re-use all the tasks that already exist for Ant.
http://groovy.codehaus.org/Using+Ant+from+Groovy
Try FinalBuilder
It provides a GUI interface to whatever your build process may be and has a native action for almost everything, plus an action studio where you can create your own.
With a bit of planning can be very streamlined.
I have to add to these one solution that I find I can't live without ... its called Hudson. It only takes a few seconds to setup and you can usually get away with most of whatever your existing ANT files are already doing.
Additionally, Hudson provides a great way to "cron" builds, execute test cases, and generate "artifacts" (e.g. build products) in a way that anyone can download.
It is hard to capture all that it can do for you ... so just try it out ... you will no be disappointed.
Maven is really good choice to do large builds...in the majority of the cases where it doesn't work is very simple. The people misunderstand the concepts of Maven. If your are working with the "maven way" you will end up with smaller modules which gives you a better architecture in your software. On the other hand things like Hudson will support you in reducing build times by using Maven, cause Hudson supports to build only changed modules which is not supported by any other build tool. The problem with Maven is to learn and understand the concepts of Maven for example the structure of a project(folders etc.) or only a single artifact etc. The build-cycle will support you in different areas: compiling, packaging, deployment and release which is not supported by other tools (only if you implement it by hand...I've wrote many large Ant scripts to reach this)...Other problem like changes over the time are caused by ignoring the best practice and pin pointing versions which are used.
I use mainly Ant, but Maven is also a good tool. With ant you can do anything you want.
At our company, we created a general purpose ant builder that does a lot of things: Build, Compess images, minify, generate documentation, pack files.. It is open source and we are open to improvements. You can get it here:
https://github.com/edertone/TurboBuilder