I want to do some research to improve my programmer skills by seeing see how long a method takes to finish.
But I don't want to write any code in the java files I am testing because I have a lot of methods to test (for some of the code I do not have permission to edit the code) so if possible I just want to "watch" the methods.
For example:
public void methodZero(){
methodOne();
methodTwo();
}
Should ideally print something like:
Time of methodZero = x. Time of methodOne = y. Time of methodTwo = z.
Question: Can I measure timing like this? Is there some additional info that would be important to me, like memory use?
Timing a method in isolation is usually completely meaningless, for a start you need statistical samples. If you want to get run times for a method you have to take a lot of things into consideration and give as much context to the target of your timing as possible.
For example, suppose your method has a return value, but you don't use it in any way in your benchmark - you are possibly going to encounter "dead-code elimination" by the JVM, which makes your timings meaningless. The JVM does many clever things like this (openjdk.net: Performance techniques used in the Hotspot JVM), all of which confuse and complicate taking meaningful timings.
A good library to use to help you do this (that also has good documentation and tutorials) is JMH.
Just as important as actual timings, but often related to taking meaningful timings, is measuring algorithm space and time complexity. This allows you to understand how your algorithm will grow as the input dataset changes in size. For example MethodA might be faster on a small input array and impractically slow on an input array 100x bigger, whereas MethodB may take the same time regardless of array size.
If you want to find out where in a program you should start looking to improve performance you can use a profiler (e.g. eclipse.org: An introduction to profiling Java applications). This will help you identify things such as: high memory usage, high GC usage, total method time (e.g. low method call count but high execution time, or high call count and small but significant execution time). However profilers will impact on your program's execution time as well.
TL;DR: profiling and performance testing is hard. Often you aren't really measuring what you think you are measuring.
long startTime = System.nanoTime();
methodOne();
long endTime = System.nanoTime();
long duration = (endTime - startTime);
You as a programmer should be more interested in complexity (Usually Time Complexity, but sometimes you should also worry about Space Complexity) of your algorithm rather than actual time taken to execute the program.
Have a read here about analysing algorithm complexity
Related
Suppose I have some code, and I want to optimize it to the maximum, disregarding readability, maintainability, etc.
For this purpose, is there anyway to figure out how much time every basic action takes? I suppose this may be CPU dependent, but I'm not sure.
I mean stuff like cycling through a for, assignments a=24, mathematical operations 2+25, etc.
Context matters. There aren't fixed costs for various Java language constructs that you can just add up to get even an approximation of a useful run-time estimate. The answer you seem to be hoping for doesn't exist.
Even if you did manage to correctly design a microbenchmark to measure some if()s vs. a switch, for example, the results would be heavily dependent on the surrounding code, the details of what the cases were, and the predictability of the branch. Making microbenchmarks is hard. You often end up measuring something other than what you intend to, unless you look at the machine instructions that actually run in your loop. It's also easy for a good compiler to optimize away your loop if you don't use the results, but then it's hard to use the results in a way that doesn't create more overhead than what you're trying to measure.
A good JIT-compiler JVM should generate machine code that's not too much worse than what you'd hope for, so if you have a good sense of how C compiles to ASM, that could be useful for java.
If you want to learn what is fast vs. slow on modern x86 microarchitectures, have a look at Agner Fog's guides.
A good profiling tool can help you figure out whether your code is CPU-bound or bottlenecked on memory (cache misses: memory bandwidth or latency), or branch mispredicts. I haven't done this for Java, but the standard tools (like Linux's perf) probably work as long as you use a long enough run to hide the overhead of the JVM startup.
You could do something like:
long startTime = System.currentTimeMillis();
// ---- Your code ----
long endTime = System.currentTimeMillis()
System.out.println("The previous line took " + (endTime - startTime) + " ms");
I am trying to compare the accuracy of timing methods with C++ and Java.
With C++ I usually use CLOCKS_PER_SEC, I run the block of code I want to time for a certain amount of time and then calculate how long it took, based on how many times the block was executed.
With Java I usually use System.nanoTime().
Which one is more accurate, the one I use for C++ or the one I use for Java? Is there any other way to time in C++ so I don't have to repeat the piece of code to get a proper measurement? Basically, is there a System.nanoTime() method for C++?
I am aware that both use system calls which cause considerable latencies. How does this distort the real value of the timing? Is there any way to prevent this?
Every method has errors. Before you spend a great deal of time on this question, you have to ask yourself "how accurate do I need my answer to be"? Usually the solution is to run a loop / piece of code a number of times, and keep track of the mean / standard deviation of the measurement. This is a good way to get a handle on the repeatability of your measurement. After that, assume that latency is "comparable" between the "start time" and "stop time" calls (regardless of what function you used), and you have a framework to understand the issues.
Bottom line: clock() function typically gives microsecond accuracy.
See https://stackoverflow.com/a/20497193/1967396 for an example of how to go about this in C (in that instance, using a usec precision clock). There's the ability to use ns timing - see for example the answer to clock_gettime() still not monotonic - alternatives? which uses clock_gettime(CLOCK_MONOTONIC_RAW, &tSpec);
Note that you have to extract seconds and nanoseconds separately from that structure.
Be careful using System.nanoTime() as it is still limited by the resolution that the machine you are running on can give you.
Also there are complications timing Java as the first few times through a function will be a lot slower until they get optimized for your system.
Virtually all modern systems use pre-emptive multi threading and multiple cores, etc - so all timings will vary from run to run. (For example if control gets switched away from your thread while it in the method).
To get reliable timings you need to
Warm up the system by running around the thing you are timing a few hundred times before starting.
Run the code for a good number of times and average the results.
The reliability issues are the same for any language so apply just as well to C as to Java so C may not need the warm-up loop but you will still need to take a lot of samples and average them.
I'm doing CPU profiling in VisualVM and look at the results in the call tree.
I have some method, taking a total time X, which is spent in the method itself (Self time), and in subroutines called from the method.
When I add up the times spent in the subroutines, plus the Self time, why doesn't the result equal the total time spent in the method? Note that I'm not talking about milliseconds, but more like 50% or several minutes missing in the balance.
It is very difficult to use "self time" to learn anything meaningful except in tiny programs with very shallow call trees.
CPU-only time is also not very useful in any kind of complex program, which can easily spend a large fraction of time in hidden I/O.
It's better to look at
inclusive time, not self time
wall clock, not cpu time
as percent, not as absolute seconds or milliseconds
It's even better to get line-level resolution, not just function or method.
Here's the method I use to find out why time is being spent and how to improve it, and here's an example of what has been done with it.
Here's a more extensive discussion of the issues.
you need both the total time and inherent (self) time...you also should avoid call trees and instead look at clock timing from a namespace hierarchy perspective (packages, classes, methods, even dynamic tags & marks)
Here is a series of articles which detail what a thorough performance investigation looks like especially when dealing with huge stacks depths beyond +2000 and billion of method invocations in a very short period:
http://www.jinspired.com/solutions/case-studies/scala-compiler
note that each method looks to validate the findings of other methods used...and more importantly there is no single right performance model...there are many right performance models depending on what is being asked and what can be changed...the only bad performance model I know of is one that is sample based in this context though when you have nothing else to do and in a hurry anything little piece of information helps
I want to capture the time take to go from statement A to Statement B in a Java class. In between these statements there are many web service calls made. I wanted to know if there is some stop watch like functionality in java that i could use to capture the exact time?
Kaddy
This will give you the number of nanoseconds between the two nanoTime() calls.
long start = System.nanoTime();
// Java statements
long diff = System.nanoTime() - start;
For more sophisticated approaches there are several duplicate questions that address Stopwatch classes:
Java performance timing library
Stopwatch class for Java
#Ben S's answer is spot on.
However, it should be noted that the approach of inserting time measurement statements into your code does not scale:
It makes your code look a mess.
It makes your application run slower. Those calls to System.nanoTime() don't come for free!
It introduces the possibility of bugs.
If your real aim is to try and work out why your application is running slowly so that you decide what what to optimize, then a better solution is to use a Java profiler. This has the advantage that you need to make ZERO changes to your source code. (Of course, profiling doesn't give you the exact times spent in particular sections. Rather, it gives you time proportions ... which is far more useful for deciding where to optimize.)
System.currentTimeMillis will get it in milliseconds and nanoTime in nanosceconds.
If you're trying to compare the performance of different techniques, note that the JVM environment is complex so simply taking one time is not meaningful. I always write a loop where I execute method 1 a few thousand times, then do a System.gc, then execute method 2 a few thousands times, then do another System.gc, then loop back and do the whole thing again at least five or six times. This helps to average out time for garbage collection, just-in-time compiles, and other magic things happening in the JVM.
Java gives access to two method to get the current time: System.nanoTime() and System.currentTimeMillis(). The first one gives a result in nanoseconds, but the actual accuracy is much worse than that (many microseconds).
Is the JVM already providing the best possible value for each particular machine?
Otherwise, is there some Java library that can give finer measurement, possibly by being tied to a particular system?
The problem with getting super precise time measurements is that some processors can't/don't provide such tiny increments.
As far as I know, System.currentTimeMillis() and System.nanoTime() is the best measurement you will be able to find.
Note that both return a long value.
It's a bit pointless in Java measuring time down to the nanosecond scale; an occasional GC hit will easily wipe out any kind of accuracy this may have given. In any case, the documentation states that whilst it gives nanosecond precision, it's not the same thing as nanosecond accuracy; and there are operating systems which don't report nanoseconds in any case (which is why you'll find answers quantized to 1000 when accessing them; it's not luck, it's limitation).
Not only that, but depending on how the feature is actually implemented by the OS, you might find quantized results coming through anyway (e.g. answers that always end in 64 or 128 instead of intermediate values).
It's also worth noting that the purpose of the method is to find the two time differences between some (nearby) start time and now; if you take System.nanoTime() at the start of a long-running application and then take System.nanoTime() a long time later, it may have drifted quite far from real time. So you should only really use it for periods of less than 1s; if you need a longer running time than that, milliseconds should be enough. (And if it's not, then make up the last few numbers; you'll probably impress clients and the result will be just as valid.)
Unfortunately, I don't think java RTS is mature enough at this moment.
Java time does try to provide best value (they actually delegate the native code to call get the kernal time). However, JVM specs make this coarse time measurement disclaimer mainly for things like GC activities, and support of underlying system.
Certain GC activities will block all threads even if you are running concurrent GC.
default linux clock tick precision is only 10ms. Java cannot make it any better if linux kernal does not support.
I haven't figured out how to address #1 unless your app does not need to do GC. A decent and med size application probably and occasionally spends like tens of milliseconds on GC pauses. You are probably out of luck if your precision requirement is lower 10ms.
As for #2, You can tune the linux kernal to give more precision. However, you are also getting less out of your box because now kernal context switch more often.
Perhaps, we should look at it different angle. Is there a reason that OPS needs precision of 10ms of lower? Is it okay to tell Ops that precision is at 10ms AND also look at the GC log at that time, so they know the time is +-10ms accurate without GC activity around that time?
If you are looking to record some type of phenomenon on the order of nanoseconds, what you really need is a real-time operating system. The accuracy of the timer will greatly depend on the operating system's implementation of its high resolution timer and the underlying hardware.
However, you can still stay with Java since there are RTOS versions available.
JNI:
Create a simple function to access the Intel RDTSC instruction or the PMCCNTR register of co-processor p15 in ARM.
Pure Java:
You can possibly get better values if you are willing to delay until a clock tick. You can spin checking System.nanoTime() until the value changes. If you know for instance that the value of System.nanoTime() changes every 10000 loop iterations on your platform by amount DELTA then the actual event time was finalNanoTime-DELTA*ITERATIONS/10000. You will need to "warm-up" the code before taking actual measurements.
Hack (for profiling, etc, only):
If garbage collection is throwing you off you could always measure the time using a high-priority thread running in a second jvm which doesn't create objects. Have it spin incrementing a long in shared memory which you use as a clock.