We Keep Coding

Measure Java Program Performance

I had an old application, a JAR file, that went through some enhancements. Basically some parts of the code had to be modified along with modifying some of the logic.
Comparing the OLD version against the NEW version, the NEW version is about 2X slower than the old one.
I'm trying to narrow down whats causing the slow down, but I'm finding myself measuring the time for certain for-loops using System.println with System.currentTimeMillis(). This is really getting very tedious.
Is there a Java performance tool that will help me in figuring out why the NEW JAR is about 2X slower than the old one?
Thanks in advance.

JProfiler has the capability to compare CPU snapshots. Record the execution for the old and the new JAR file and save snapshots (if the JVM exits at the end, configure a "JVM exit" trigger that saves a snapshot).
Then open the snapshot comparison window with "Session->Compare Snapshots in New Window" and add the two snapshot. A hot spots comparison will look like this (a view filter is set in this case):
It will immediately show you which methods are responsible for the increase in execution time.
Another way to analyze the differences in execution time is the call tree comparison which will look like this:
Disclaimer: My company develops JProfiler.

You should use a profiler. This will show you which methods are taking the most time (and what is calling them), without you having to guess which ones to measure.
Java comes with a built-in profiler called hprof, but see also:
https://stackoverflow.com/questions/14762/please-recommend-a-java-profiler
5 things you didn't know about ... Java performance monitoring
The JConsole and VisualVM tools

Depending on how long-running the process is, I'd think about Visual VM 1.3.3. If you download all the plugins, you'll be able to see heap, threads, objects, etc. That ought to help, and it won't cost a dime.
I believe it assumes the Oracle/Sun JVM.

A profiler tool like YourKit or something to measure performance reliably like Hyperic's Sigar is a good canditate for your case. Have a look at those tools.
The former will find bottlenecks in your code and/or memory leaks (not all of them) while as the latter is an API that you can measure performance reliably since Oracle's JVM & OpenJDK have no way of getting perfomance metrics reliably/consistently/accurately (like CPU wall clock time or CPU time spent from the application, memory usage, application threads, etc).
By default, Java provides packages for these things.
For example:
java.lang.management.ManagementFactory
java.lang.management.ThreadMXBean
but depending on your case they may or may not be adequate (keep in mind they are OK for most cases unless we are talking about something critical).

Automatic Runtime Performance Regression Test in Java

I'm looking for ways to detect changes in runtime performance of my code in an automatic way. This would act in a similar way that JUnit does, but instead of testing the code's functionality it would test for sudden changes in speed. As far as I know there are no tools right now to do this automatically.
So the first question is: Are there any tools available which will do this?
Then the second questions is: If there are no tools available and I need to roll my own, what are the issues that need to be addressed?
If the second question is relevant, then here are the issues that I see:
Variability depending on the environment it is run on.
How do detect changes since micro benchmarks in Java have a large variance.
If Caliper collects the results, how to get the results out of caliper so that they can be saved in a custom format. Caliber's documentation is lacking.

I have just come across http://scalameter.github.io/ which looks appropriate, works in scala and java.

Take a look at Caliper CI, I put out version 2.0 yesterday as a Jenkins plugin.

I don't know any separate tools to handle this, but JUnit has an optional parameter called timeout in the #Test-annotation:
The second optional parameter, timeout, causes a test to fail if it
takes longer than a specified amount of clock time (measured in
milliseconds). The following test fails:
#Test(timeout=100) public void infinity() {
while(true);
}
So, you could write additional unit-tests to check that certain parts work "fast enough". Of course, you'd need to somehow first decide what is the maximum amount of time a particular task should take to run.
-
If the second question is relevant, then here are the issues that I
see:
Variability depending on the environment it is run on.
There will always be some variability, but to minimize it, I'd use Hudson or similar automated building & testing server to run the tests, so the environment would be the same each time (of course, if the server running Hudson also does all other sorts of tasks, these other tasks still could affect the results). You'd need to take this into account when deciding the maximum running time for tests (leave some "head room", so if the test takes, say, 5% more to run than usually, it still wouldn't fail straight away).
How do detect changes since micro benchmarks in Java have a large variance.
Microbenchmarks in Java are rarely reliable, I'd say test larger chunks with integration tests (such as handling a single http-request or what ever you have) and measure the total time. If the test fails due to taking too much time, isolate the problematic code by profiling, or measure and log out the running time of separate parts of the test during the test run to see which part takes the largest amount of time.
If Caliper collects the results, how to get the results out of caliper so that they can be saved in a custom format. Caliber's
documentation is lacking.
Unfortunately, I don't know anything about Caliper.

Identifying which parts of my Java program need optimization

I want to optimize my Java source code, but I am not sure what parts of my code may be optimized. Is there any tool or any technique through which I can optimize my Java source code?
I know the basic fundamentals of code optimization, but I have a large number of lines and very complex source code (Java classes).
Can I optimize my code with the help of a Java profiler?
Thanks in Advance!

What do you mean by optimising your code? DO you want to refactor your code to simplify it. In that case I suggest you use your IDE to help refactor the code.
If you want to minimise its memory / CPU consumption, I would suggest you use a profiler like VisualVM or YourKit to idenify where resources are being consumed.
A code analysis tool can also help pick up the obvious performance issues. I have code analysis on as I type in my IDE which helps me pick up those issues as I write it.

Performance optimization - yes, a profiler may help. At least it can show you those areas in your application that take an unexpected amount of CPU time.
But before starting to apply changes - take care not to do some microoptimization. Look at improving algorithms first. Sometimes we use nested for loops while a task can be done with a single one (linear time). Double check if you use the correct collection types. Then have a look if you accidentally create more objetcs than needed (object creation in loops is a typical reason for performance problems).

Their are several tools for static code analyzes (to do code style / code convention / best practises / bug "optimization" of your code).
Checkstyle
PMD
Findbugs
I would recommend using Sonar. It covers them all and is easy to setup - even on a local machine (unzip and start). It is best used with maven projects but also possible for NON maven projects.

which tool should be used in order to measure performance of a program unit?

i need to measure performance of my program unit. I am using hibernate as ORM tool. i want a tool that is capable enough to measure the time taken per method invoked and excluding the database loads???
Please help

This is what a profiler does. VisualVM is a free one, but if you want mroe detail as to the timings and behaviour of JDBC queries I suggest you look at YourKit which can anlyse the queries in more depth.

JConsole is a graphical monitoring tool to monitor the Java Virtual Machine and java applications both on a local or remote machine.
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

for a quick and dirty hack, you can use http://www.clarkware.com/software/JUnitPerf.html, a junit based microbenchmark framework. You can write your code, mock out (or even use a real database), and run it to get a benchmark. This benchmark is essentially good only for testing a single (or very few) methods, and in their specific uses, not a general profiler.

Your question isn't quite clear to me. Do you wonder which part of your application takes the time? Or do you want to observe the time a certain part of your code takes. In the first case use a proper file. VisualVM and YourKit are both fine profilers, I've used them before and found them very helpfull. In the latter case, I would try a tool like Perf4J which allows you to annotate a method and observe its average runtime, its standard deviation and other thing in realtime or afterwards.

Java Tools for combining runtime execution with coverage

Are there any free tools for Java (preferably Eclipse) that can give metrics on both how frequently code is executed (based on a recorded run) and do a side by side with coverage? I'm not sure if there is already a metric for measuring this but it seems it would be an interesting one to see.

do you mean running the application as in production ?
if in dev environ ...not sure if this is what you are looking for - http://insectj.sourceforge.net/

Are you looking for a Java profiler (something that gives you function call counts and elapsed execution times)? If so, you might want to look at the Eclipse Test & Performance Tools Platform. It doesn't look like it will give you a side-by-side comparison of two pieces of code, but it might be worth looking at.

The SD Java Profiler collects profiling data in a way that generalizes test coverage. So if you have its profile data displayed, you have the test coverage data effectively displayed too; there's no need for a side-by-side comparison.
Not free, and doesn't plug into eclipse yet, but does use a Java GUI.