Coverity for Java static analysis - java

I'd like to get comments from people who have used or evaluated Coverity for statically analysing Java-code. I know it's popular in C/C++ world, but is it worth spending the money for Java analysis or am I better off with PMD, Findbugs and other Open Source tools?

If you aren't using anything today, I would start off with Findbugs and PMD. They are easy to install and use. Concentrate on reviewing and fixing correctness errors with Findbugs first - they recommend starting with High and Medium severity correctness errors as the checkers have very low false positives and you will get a good return on your time. Get developers to use PMD to cleanup the code, and the Findbugs plugin in Eclipse to review new code. Working incrementally will get the developers to understand and buy-in to the usefulness of these tools.
Coverity's Java checkers are still weak compared to their C/C++ checkers. We use Findbugs, PMD, Coverity and Klocwork because they all have different strengths and we are paranoid. If you aren't paranoid, you could stick with open source tools and get a lot of value. Or if you need security checking: then Klocwork or especially Fortify should do a more thorough job for you.

I'll chip in with a somewhat relevant answer. I've used Klocwork for the both Java and C code. Klocwork is a close competitor of Coverity... cost-wise they are about the same ( look carefully, Klocwork looks cheaper until you actually buy what you need ), and feature-wise they fight back and forth.
For C/C++, it's great. For Java.... well, it helped find a lot of resource leaks ( #$#^#ing Java developers seem to forget that resources like file handles aren't garbage collected ), but it doesn't seem to find many "critical" bugs. It's probably because the language itself does help protect against some of the more basic but hard to find errors ( array overflows, pointer corruption, etc ).
Get Coverity in to run a demo, they are more than happy to. See what sort of things they find.

I'll add a limited me-too to the preceding answers, somewhat restricted by the Coverity NDA I'm bound by. Coverity Prevent has an impressive public track record for finding bugs in open source C/C++ code, but their Java product is a lot newer. (Coverity has a press release on my former employer, so I can say that it did help find and fix lots of bugs in our C/C++ code, more than I'd found in all my previous career in bug hunting.) FindBugs does an impressive job on Java code, and you can't beat the price. But the big point has already been made: try out both of them on your real code before you buy. There's no substitute for reality, and the conventional wisdom in static analysis is that there's surprisingly little overlap in what the tools discover.

As others have said, the best way to decide is to try all these tools out.
Coverity recently announced a hosted static-analysis-as-a-service product called Code Spotter (https://code-spotter.com/), currently in beta. It's using the same analysis engine as the Coverity enterprise product, but it is wrapped in a different (simplified) user interface. Since this is a hosted service, it is very easy to play with it to get a sense of the Coverity analysis capabilities.
At the time of this writing, Code Spotter is Java-only, but other Coverity supported languages should be coming soon.

Related

Tool that finds pieces of code without thorough coverage of exceptional states

I would like to find a good tool that will analyze my code and find possible problems with missing exceptional states handling, like null checking, numerical overflows ect..
Any recommendations?
Edit: this was retlated to testing too i.e. how to find spots that are weakly covered by tests.
Personally, I use PMD and Findbugs:
http://pmd.sourceforge.net/
http://findbugs.sourceforge.net/
There's a lot of overlap between those two, but you can configure custom rulesets and rules for both of them, and you can also code your own rules if you want.
Unfortunately, what any static analysis tools can do with regards to finding potential problems with exceptional states is somewhat limited. Having said that, both of these tools has pointed out problems in code I've worked on that would have wasted a lot of time to debug if a client had found the problem after we released.
[EDIT] To cover the dynamic part of the code, use EclEmma and unit tests. EclEmma uses Emma internally, which you could use without Eclipse as well.
A good IDE like Netbeans or Eclipse should do that as wide as it's possible. The Compilers messages should help, too.
The rest is up to you (that means: testing, testing and testing).

Scala as the new Java?

I just started exploring Scala in my free time.
I have to say that so far I'm very impressed. Scala sits on top of the JVM, seamlessly integrates with existing Java code and has many features that Java doesn't.
Beyond learning a new language, what's the downside to switching over to Scala?
Well, the downside is that you have to be prepared for Scala to be a bit rough around the edges:
you'll get the odd cryptic Scala compiler internal error
the IDE support isn't quite as good as Java (neither is the debugging support)
there will be breaks to backwards compatibility in future releases (although these will be limited)
You also have to take some risk that Scala as a language will fizzle out.
That said, I don't think you'll look back! My experiences are positive overall; the IDE's are useable, you get used to what the cryptic compiler errors mean and, whilst your Scala codebase is small, a backwards-compatibility break is not a major hassle.
It's worth it for Option, the monad functionality of the collections, closures, the actors model, extractors, covariant types etc. It's an awesome language.
It's also of great personal benefit to be able to approach problems from a different angle, something that the above constructs allow and encourage.
Some of the downsides of Scala are not related at all to the relative youth of the language. After all, Scala, has about 5 years of age, and Java was very different 5 years into its own lifespan.
In particular, because Scala does not have the backing of an enterprise which considers it a strategic priority, the support resources for it are rather lacking. For example:
Lack of extensive tutorials
Inferior quality of the documentation
Non-existing localization of documentation
Native libraries (Scala uses Java or .NET libraries as base for their own)
Another important difference is due to how Sun saw Java and EPFL sees Scala. Sun saw Java as a product to get enterprise customers. EPFL sees Scala as a language intended to be a better language than existing ones, in some particular respects (OOxFunctional integration, and type system design, mostly).
As a consequence, where Sun made JVM glacially-stable, and Java fully backward compatible, with very slow deprecation and removal of features (actually, removal?), JAR files generated with one version of Scala won't work at all with other versions (a serious problem for third party libraries), and the language is constantly getting new features as well as actually removing deprecated ones, and so is Scala's library. The revision history for Scala 2.x, which I think is barely 3 years old, is impressive.
Finally, because of all of the above, third party support for Scala is incipient. But it's important to note, though, that JetBrains, which makes money out of selling the IntelliJ IDEA IDE, has supported Scala for quite some time, and keeps improving its support. That means, to me, that there is demand for third party support, and support is bound to increase.
I point to the book situation. One year ago there was no Scala book on the market. Right now there are two or three introductory Scala books on the market, about the same number of books should be out before the end of the year, and there is a book about a very important web framework based on Scala, Lift.
I bet we'll see a book about ESME not too far in the future, as well as books about Scala and concurrency. The publishing market has apparently reached the tipping point. Once that happens, enterprises will follow.
I was unshackled from the J2EE leash last year wanted to do something new after 12 years of Java in the enterprise building very large system for some of the worlds biggest companies.
I had tried Ruby on Rails in the past. After building a few sample apps I did not like the feel of it or the fact that I would have to write a ton of unit tests to cover stuff that is normally done by a compiler.
Groovy on Grails was my next port of call. I have to say I do like this but it suffers from the same dynamic typing problems as ROR. Don't get me wrong I am not putting Grails down as it is an excellent framework and I will still use it. Each has its own place IMO.
I then jumped on Scala and have now built a hybrid application based on Scala and Spring MVC. At first working with Scala is difficult but it gets easier and more productive the more time you put into it. I've reached a tipping point where I now want to invest time in Lift as well.
The combination of "Programming in Scala" and David Pollak's "Beginning Scala" books is good for learning the language, the latter with a less academic bent.
Scala is still young and has some way to go. I think it has a bright future and I see momentum is already picking up. Recently one of the creators of the Groovy language said in a blog post he would never have bothered designing Groovy if Scala had been around at the time.
I think some more work on better Java API integration/wrapping will give Scala the boost it needs to win more followers. The basic integration is there already but I think its could be polished a bit more.
Yes IDE support is there but it is basic at the moment. The powerfully refactoring support of Intellij is not there yet and I miss that a lot. The compiler + IDE support with a mix of of other plugins is not mature yet. I sometimes get very weird internal compiler errors caused by how Scala sits with JDO enhancement for the Goggle app engine. However these are little things that can be easily fixed. Early adaptation of new technologies and languages always comes with a little pain. But this bit of pain can produce great pleasure in the future.
If I look at the capabilities of Scala compared to early Java its miles ahead. When I moved from C++ to Java the JVM was not ready yet regarding scalability. There used to be lots of weird crash and burn JVM core dumps on various OSes. All of this has now been fixed in Java and the JVM is rock solid. Scals runs in the JVM so it has been given a massive head start on native platform integration. Its standing on the shoulders of giants!
After years of building and supporting enterprise applications my vote is for a language where a compiler can catch most of the non functional bugs before even unit tests are built. I love the type checking mixed with the power of functional programming. I like the fact that I am doing OO++.
I think the development community will decide if Scala is the future or not. The downside of adopting Scala now would be if it did not pick up momentum and adaptation. It would be very difficult to maintain an Scala code base with very few Scala developers around. However I watched Java come from the skunk works into the enterprise to replace C++ and it was all pushed from the bottom up by the developer community. Time will tell for Scala but currently it has my vote.
Beyond learning a new language, what's the downside to switching over
to Scala?
Thinking, thinking, thinking..... nope, there is none :-)
I'll tell you my little personal experience, and how I found that it wasn't so easy to integrate Scala with existing Java libraries:
I wanted to get started with something easy, and as I thought that Scala was very well suited for scientific computation I wanted to do a little wrapper around JAMA (Java Matrix library)... My initial approach was to extend the Matrix type with a Scala class and then overload the arithmetic operators and call the Java native methods, but:
The Matrix class doesn't provide a default constructor (without arguments)
The Scala class needs one primary constructor
I thought one good primary constructor could be the one accepting an Array[Array[Double]] (first thing that sucks, that syntax is much more verbose and hard to read than Double[][])
As far as I know by reading the manuals, the parameters of the primary constructor are also implicitly fields of the class, so I would end with one Array[Array[Double]] in the Scala subclass and another double[][] in the Java superclass, which is pretty redundant.
I think I could have used an empty primary constructor that initialized the superclass with some default values (for example, a [[0]]), or just make an adapter class that used the Jama.Matrix as a delegate, but if a language is supposed to be elegant and seamless integrated with another, that kind of things shouldn't happen.
Those are my two cents.
I don't think there are any downsides. Actually learning new language is very helpful for broadening your programming knowledge. You might gain from Scala such things as generic classes, variance annotations, upper and lower type bounds, inner classes and abstract types as object members, compound types, explicitly typed self references, views, and polymorphic methods.
It consistently breaks backwards compatibilty.
Community size is small.
IDE support isn't there yet.
Otherwise its fine.
It is just a young language, it will get there eventually.
Great for hobbyists, not ready for enterprise.
The two, by which I mean four, biggest downsides I'm seeing are:
Many working as developers in the professional community aren't trained and are unwilling to learn how to use a functional language, they won't even give it a go so they can understand why it's a better approach. This means you'll always be fighting an uphill battle getting adoption until it's mandated at the corporate level.
RDBMS integration is still a bit spotty. Plenty of solutions, but nothing that really sticks out as becoming a standard. For me though, this might be an advantage rather than a disadvantage. JPA2 is a mess and causes more issues than problems it solves. Hibernate criteria queries aren't much better.
IDE support is still lagging, but mostly in the area of debugging at this point. Code inspection is doing pretty good (at least in IntelliJ).
You'll never want to write another line of Java again! Likely you'll want to punch a wall or break something when forced back into the awkward syntax of Java.
The answers here are somewhat dated circa 2022 so I thought that I would contribute with an update. I have been working in a tech company that started using Scala at about the same time this question was originally asked. I recently blogged about lessons learned in that shop when trying to teach Java developers how to work with a code base written in idiomatic Scala so this topic is top-of-mind for me right now.
Just about all of the maturity issues in tooling, educational
collateral, and integration are gone. Version 2 of Scala is just as
rock solid a programming language as version 8 or 11 of Java.
Some of the most obvious advantages in switching to Scala are no longer
relevant because those language features have been added to newer
versions of Java.
Where Scala continues to outshine Java is in terms
of modularity and readability which affords idiomatic Scala as able
to handle code complexity better than Java.
That improved code scalability comes at the cost of a higher
learning curve which makes Scala less attractive to junior developers
who tend to make up the majority of your engineering group.

Should I study Scala? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am an experienced C++ programmer with average Python skills. The reasons I studied Python in the first place were:
to get a different perspective on programming (static vs dynamic, interpreted vs compiled, etc.)
to increase the breadth of projects that I can work on (Python allows me to do web development, develop for Symbian phones or knock up quick system administration scripts)
to complement my C++ skills.
I think that Python is great and I believe that I have achieved the above goals. I will continue to use it for small projects, scripts and web development.
I doubt that I can use it for medium to large projects though. While the dynamic typing is convenient, it allows a certain class of bugs that I find disturbing. Unit testing and linting can alleviate this problem, but static typing completely eliminates it.
After looking at some programming languages, I think that Scala looks like a good candidate:
I like the type inference and it runs on the JVM so it should be available wherever the JVM is available. I can also learn more about functional programming when using it.
But... I also have some doubts, and this is where I hope that the Stack Overflow community can help:
Portability: Linux and Windows at least I hope. What about mobile phones, is it possible to get it to run there?
C++ compatibility: can I mix C++ code with Scala? (JNI?)
Programming paradigm: I don't feel comfortable with switching to functional programming (FP) at this time. Can I use object oriented and procedural with some FP at first and then change the proportions as I learn?
Tool chain maturity: what's your experience with IDEs and debuggers? I'm using Eclipse right now and it seems OK.
Learning speed: considering my experience, how fast do you think that I can reach a workable level with Scala?
Deployment: how exactly do you deploy a Scala program? Is it a jar, is it an executable?
Finally, what do you think that are some of Scalas disadvantages?
Portability: Linux and Windows at least I hope. What about mobile phones, did anyone succeed in getting it to run there?
Yes. There is quite some movement about Scala on Android. As for J2ME, I saw something in that respect, but not much. There is some code pertaining to J2ME on the source code repository. I'm not sure how viable it is, but it looks to me that there isn't much demand for that.
I'll also mention that there is/was a pool on Scala-Lang about the desired target platforms, and J2ME was one of them, very low on the totem pole.
C++ compatibility: can I mix C++ code with Scala? (JNI?)
As well as you can mix C++ with Java, for whatever that is worth. If you haven't any experience with that, you can just read the Java resources, as anything in them will be applicable with Scala with no changes (aside Scala syntax).
Programming paradigm: I don't feel comfortable with switching to FP at this time. Can I use OO and procedural with some FP at first and then change the proportions as I learn?
Definitely, yes. Scala goes out of it's way to make sure you don't need to program in a functional style. This is the main criticism of Scala from functional folks, as a matter of fact: some do not consider a language functional unless it forces the programmer to write in functional style.
Anyway, you can go right on doing things your way. My bet, though, is that you'll pick up functional habits without even realizing they are functional.
Perhaps you can look at the Matrices series in my own blog about writing a Matrix class. Even though it looks like standard OO code, it is, in fact, very functional.
Tool chain maturity: what's your experience with IDEs and debuggers? I'm using Eclipse right now and it seems ok.
IDEA (IntelliJ), NetBeans and Eclipse all have good support for Scala. It seems IDEA's is the best, and NetBeans/Eclipse keep frog-jumping each other, though NetBeans has certainly been more stable than Eclipse of late. On the other hand, the support on Eclipse is taking a very promising route that should produce results in the next 6 months or so -- it's just that it's a bumping route. :-)
Some interesting signs of Scala tooling for these enviroments is the fact that the Eclipse plugin in development uses AOP to merge more seamlessly with the whole IDE, that the NetBeans plugin is being completely rewritten in Scala, and that there's a Scala Power Pack on IDEA that supports, among other things, translating Java code into Scala code.
The EMACS folks have extensive tools for Scala as well, and lots of smaller editors have support for it too. I'm very comfortable with jEdit's support for small programs and scripts, for instance.
There is also good Maven support -- in fact, the standard way to install Lift is to install maven, and then build a Lift archetype. That will pull in an appropriate Scala version. There's an scala:cc target that will do triggered recompilation as well.
Speaking of recompilation, neither Maven, and particularly nor Ant do a good job at identifying what needs to be recompiled. From that problem sprung SBT (Simple Build Tool), written in Scala, which solves that problem through the use of Scala compiler plugin. SBT uses the same project layout as Maven, as well as Maven/Ivy repositories, but project configurations are done in Scala code instead of XML -- with support for Maven/Ivy configuration files as well.
Learning speed: considering my experience, how fast do you think that I can reach a workable level with Scala?
Very fast. As a purely OO language, Scala already introduces some nice features, comparable to some stuff that's present in C++ but not Java, though they work in different fashion. In that respect, once you realize what such features are for and relate them to C++ stuff, you'll be much ahead of Java programmers, as you'll already know what to do with them.
Deployment: how exactly do you deploy a Scala program? Is it a jar, is it an executable?
The same thing as Java. You can deploy JARs, WARs, or any other of Java targets, because the scala compiler generate class files. In fact, you use Java's jar to generate a Scala's JAR file from the class files, and the Maven targets for Lift support building WAR files.
There is an alternative for script files, though. You can call "scala" to run Scala source code directly, similar to a Perl of Shell script. It can also be done on Windows. However, even with the use of a compilation daemon to speed up execution, start up times are slow enough that effective use of Scala in a heavy scripting environment needs something like Nailgun.
As for Scala's disadvantages, take a look at my answer (and other's) in this Stack Overflow question.
Scala is an evolving language well worth to invest in, especially if you are coming from Java world. Scala is widely covered at Artima. See this article from Bill Venners and also read about Twitter and Scala.
Regarding your questions:
Java can run wherever there is a JVM. No luck with the mobile phones however. You need a full JRE, not the subset that is available there.
This is possible with JNI. If something is possible with Java, then it is possible with Scala. Scala can call Java classes.
Functional programming is a strong point of Scala - you do need to learn it. However you could also start using it without taking full advantage of it and work your way with it.
There is a plug-in of Eclipse. It is not best, but it will do the job. More details here.
If you are experienced, I would say really fast. I recommend that you find a book to start with.
See this faq entry for deployment.
Programming paradigm: I don't feel comfortable with switching to FP at this time. Can I use OO and procedural with some FP at first and then change the proportions as I learn?
Scala has full support for imperative programming, writing programs with no FP elements in it is a breeze (however, FP is useful and worth learning anyway).
Learning speed: considering my experience, how fast do you think that I can reach a workable level with Scala?
Quickly. There is a number of interesting features in Scala that may be not familiar to people coming from a C++, Java environment, like for example some of the features of the typing system. Some argue that the fact that there is a lot to learn in Scala before you know all of it is a problem with the language; I disagree. The presence of those feature is an advantage of the language. The more features the merrier. After all, you don't have to use them all at once, just like you don't have to buy everything that is being sold in the store.
Learning speed: considering my experience, how fast do you think that I can reach a workable level with Scala?
I also come from a C++ background, one thing I noticed is that since you will write a lot less code as compared to C++ for a comparable task, your learning will be expedited as you will get more done in the same time period. This was the same phenomenon that I experienced with Ruby.
Actually - if I were you - I'd study programming paradigms instead of languages. Of course you have to study an example language to study the paradigm. Knowing the drawbacks & benefits of different paradigms enables you to view your problems from a different side and makes you a better programmer (even in the languages you already know).
Picking up a language of a paradigm already known is a relativly easy task if needed. Since Scala is FP (at least you mentioned it) and C++/Python is OOP, it's a good language for you, I'd say.
You should register for this course by the Creator of Scala himself.
https://www.coursera.org/course/progfun
James Strachan (productive Java open source developer, for those not in the loop) has an interesting discussion of Scala here, and why he feels it's a progression from Java (the langauge, not the platform).
Scala looks like it's gaining a lot of traction. I don't think it's a flash in the pan, and is currently on my list of languages to learn (partly for the functional aspect)
Here's an anecdotal evidence regarding learnability of Scala.
In our company, we got several interns from U.Waterloo. They were told to write in Scala; never saw it before.
They picked up Scala and Lift remarkably fast; now they are producing Scala code; it may be not perfect, but nobody's perfect.
So, the fact that a manager does not know Scala may be not the best argument when you decide on adoption.

Can Scala survive without corporate backing?

I was wondering whether Scala will get the takeup it deserves without explicit corporate backing (I was thinking by Sun/Oracle but I suppose it could be someone else, such as Google).
With Sun's recent decision not to include closures in JDK7, couldn't they put their weight behind Scala as the Java alternative for those wishing to have a more expressive language? After all, it seems to me that they should care most about the JVM, not Java per se. Does anyone think this is likely?
Apart from the examples of previous languages that have succeeded without initial corporate backing, I think Microsoft's promotion of functional programming on the .NET platform may indirectly help Scala gain adoption. Since the Java and .NET ecosystems are seen as close rivals, people aware of F# and the functional additions to C# may be inclined to look for JVM analogs, and to me Scala seems best equipped to fill that role.
You need to be more specific. Thrive in what context?
I think Scala's community is near the critical mass that it needs to be a self-sustainable open source project even if its primary institutional backer, the EPFL, suddenly had a change of heart; and there is currently every sign that it will reach this critical mass. I think Scala will be with us and actively maintained for a long time.
A more pressing issue if the type of uses for which it is suited. The Scala compiler and standard library are far from perfect. When you start pushing the language or a portion of the library there is still a decent chance that you will find bugs. This is improving by leaps and bounds, but it in itself isn't the core of the problem.
The problem is in order to get fixes you pretty much have to upgrade to the next version of Scala as upgrades come out. The problem with that is that most version upgrades contain breaking changes, thus in order to obtain fixes you are likely to have to change your own code. There's also the binary incompatibility problem, which means all your Scala dependencies have to change versions, too.
This could be severe problem if you have a lot of dependencies on other Scala libraries (unlikely - there aren't many yet), if you are subject to severe infrastructure bureaucracy, or worse, you're a product-oriented company that needs to distribute fixes to customer with severe infrastructure bureaucracy.
In order for Scala to be viable in such situations long-term, someone will have to start back porting fixes to earlier versions so that people don't have to perform breaking upgrades just to get some fixes. I'm sure this will happen, because it really wouldn't be that hard, but it will probably require someone seeing a business opportunity, because let's face it, backporting changes and doing regression testing isn't exactly exiting work.
I think Scala will be fine. It has a lot of mind share now that Twitter has partially moved to it. I think it's unlikely Sun will concentrate on anything but Java, but they are probably happy that other languages are targeting the JVM.
I think the closure thing is lame, by the way. I love closures.
I think it will thrive because it doesn't have corporate bozos 'backing' them. I personally like the idea that a language is so independent, and yet very good and well planned and thought out.
I think Google is likely to pick it up, but I don't think it will be an official backing. I'm sure Google will find a cool project to use Scala with.
Corporate backing is only necessary for a language to succeed if it doesn't have a lot of support from working programmers. Perl, Python, and Ruby all grew up without support from the Man (well, Ruby had the Japanese Man, but it wasn't a lot of help in the US).
I've not used Scala, but Twitter's endorsement of it suggests that it has enough programmer support to thrive even without a corporate big brother.
Of course it will survive.
There are bundles of languages lying around, being supported by univerities, working groups, hobbiests, and programmers, up and down the world, and out in the wild. In some cases the fact their not supported by the Man, means they have some chance of survival, perhaps more to do with their cachet or maybe they server an important reserarch aspect.

How do commercial Java static analysis tools compare with the free ones? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I'm familiar with a handful of the free static analysis tools available for Java, such as FindBugs and PMD. What I'd like to know is how the commercial products such as Klocwork and Coverity stack up against these. What are their strengths and weaknesses?
We use a suite of open source and commercial static analysis tools. The different tools find different kinds of bugs and some are tuned for lower false positive rates, at the expense of possibly missing some real problems.
In my experience, Findbugs does a good job of finding real problems, especially if you focus on Correctness errors as their team suggests. Recently the developers of Findbugs have added some basic security vulnerability checks as well. Coverity has a low false positive rate especially if you don't turn on their experimental checkers, and Coverity Prevent includes a good tracking database for trend/cluster analysis. I am not convinced yet that their threading checkers (static or dynamic) work - at least they haven't found anything interesting for us. Klocwork Developer for Java returns higher false positives, but we find they have the strongest security checking of these tools. So it depends on whether your priority is quality checking (Findbugs, Coverity) or security vulnerability analysis (Klocwork, or Fortify). Some of our developers also use PMD to support source code reviews, as it helps with general code cleanup.
A recent project conducted with NIST called "SATE: Static Analysis Tool Exposition" reviewed a wide variety of different tools and their underlying approaches.
https://samate.nist.gov/index.php/SATE.html and other references to this project such as at OWASP.
The general finding is that different tools have different strengths and weaknesses, so use more than one if you want to do a thorough job.
I'll suggest you to try SONAR an open source software quality management tool, dedicated to continuously analyze and measure source code quality.
This soft take the result from code analysis tool, consolidate that results and give you access to an user friendly interface.
The one feature you will most certainly find in a commercial static analysis tool (and that you will not find easily in a freeware analysis tool, at least in 2008, at the time of the OP) is
Reporting: Measures software quality trends over time
As explained in this question about code metrics, any static code analysis in itself in not always meaningful, because you could have:
too many "defects" to fix
too many categories of defect reported
You need the ability to do some triage, and you need to check if a particular defect is occurring less and less over time or not, in order to help you prioritize what to fix.
This is especially true on legacy project with thousands of classes: you do not fix defect on many files just like that, without having a good reason. That reason can be deduced from a good reporting and trend analysis you will not find with freeware tools.
Update: from 2012 (4 years later), Sonar (Now in 2018 named "SonarQube") "Historical Information" (aka "Time Machine") in its 4.x and 5.x series.
Note those project dashboards were dropped in SonarQube 6.1 (Sept. 2016): see this thread.
Those dashboard would need to be re-created manually through a custom page.
SonarQube 6.5 restores a bit of those dashboards with the Activity page, which gets (several predefined and one customisable) charts to display the evolution of a project.
I have not had direct experience with Findbugs or PMD but have met plenty of people who have compared them with Klocwork and Coverity.
My general take on the feedback has been:
Findbugs and PMD are more "tool-ish". The type of thing you'd run on your desktop. It finds a wide range of potential problems but tends to be noisy, meaning false positives and "I don't care" varieties. It does find some good stuff. I've heard mixed feedback on its long term use. Some feel that the ROI on a free tool is infinite however there is a true cost to false positives.
Not surprisingly, Klocwork and Coverity, which cost money, tend to be more solution oriented that can also scales better to work with teams, has a more efficient, easier to use UI and tends to be less noisy. It seems their analysis is doing deeper inspection and therefore coming up with better results if you did a side by side comparison. When adopting a tool across a team, you'll have various levels of enthusiasm for using a tool and the noise factor is a big issue that prevents widespread adoption. Of course there are things like having support to back you up, etc.
In general, because Findbugs and PMD are free, you see that as a first option. Many companies see value and choose Coverity or Klocwork for a longer term solution although I see also running Findbugs and PMD. They tend to find different things and so if your goal is to find and fix as much as possible, it's good to have a combination of both.
Disclosure: I work for Code Integrity Solutions (codeintegritysolutions.com) which is a partner of Coverity.
here's a list of commercial analysis tools : http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis#Java_2
coverity has several tools :
http://www.coverity.com/html/coverity-readiness-manager-java.html : this should be on par with findbugs and PMD but with better presentation
prevent : http://www.coverity.com/html/prevent-for-java.html : low FALSE POSITIVES.
thread analyzer : http://www.coverity.com/html/coverity-thread-analyzer-java.html : this is what is absent in most open source tools.

Categories