I am running some JUnit tests on a single thread and they are failing in a non-deterministic way. I had one person tell me that the optimizing JVM (Oracle Hotspot 64-Bit 17.1-b03) is executing the instructions out of order for speed. I have trouble believing that the java spec would allow that, but I can't find the specific reference.
Wikipedia states that a single thread must enforce within-thread as-if-serial so I shouldn't have to worry about execution order differing from what I wrote.
http://en.wikipedia.org/wiki/Java_Memory_Model#The_memory_model
Example code:
#Test
public void testPersistence() throws Exception
{
// Setup
final long preTestTimeStamp = System.currentTimeMillis();
// Test
persistenceMethod();
// Validate
final long postTestTimeStamp = System.currentTimeMillis();
final long updateTimeStamp = -- load the timestamp from the database -- ;
assertTrue("Updated time should be after the pretest time", updateTimeStamp >= preTestTimeStamp);
assertTrue("Updated time should be before the posttest time", updateTimeStamp <= postTestTimeStamp);
}
void persistenceMethod()
{
...
final long updateTime = System.currentTimeMillis();
...
-- persist updateTime to the database --
...
}
When this test code is run it has completely non-deterministic behavior, sometimes it passes, sometimes if fails on the first assert, and sometimes it fails on the second assert. The values are always within a millisecond or two of each other so it isn't that the persistence is just failing completely. Adding a Thread.sleep(2); between each statement does decrease the number of times the test fails, but doesn't eliminate the failures completely.
Is it possible that this is the fault of the JVM or is it more likely that the database (MsSql) is doing some sort of rounding of the stored data?
The possibility that the JVM is executing statements out of order is so remote that I think you can pretty much dismiss it. If the JVM had a bug like that, it would be showing up in a lot of places besides this one program of yours.
It is true that currentTimeMillis is not guaranteed to actually be accurate to the millisecond. But the possibility that the clock would run backwards is almost as remote as the possibility that the JVM is executing statements out of order.
I've written many, many programs that test how long it takes a function I'm interested in to execute by taking the currentTimeMillis before it starts, executing the function, getting currentTimeMillis when it's done, and subtracting to find an elapsed time. I have never had such a program give me a negative time.
Some possibilities that occur to me:
There's an error in your code to save the timestamp to the database or to read it back. You don't show that code, so we have no way to know if there's a bug there.
Rounding. I don't have a MySQL instance handy, so I'm not sure what the precision of a timestamp is. If it's not as precise as a millisecond, then this would readily explain your problem. For example, say it's only accurate to the second. You get pre time=01:00:00.1, update time=01:00:00.2, post time=01:00:00.4. But update time gets saved as 01:00:00 because that's the limit of precision, so when you read it back update time < re time. Likewise suppose the times are 01:00:00.4, 01:00:00.6, 01:00:00.7. Update time gets rounded to 01:00:01. So update time > post time.
Time zones. Default time zone is an attribute of a connection. If when you write the time you are set to, say, Eastern Time, but when you read it back you are on Pacific Time, then the order of the times will not be what you expected.
Instead of just looking at the relationships, why don't you print the values of all three timestamps? I'd print them as int's and also as Gregorian dates. Oh, and I'd print update time before saving and again after reading it back. Maybe something would become apparent.
If, for example, you see that the update time as read back always end with one or more zeros even when the time as saved had non-zero digits, that would indicate that your times are being truncated or rounded. If the time has read back differs from the time as written by an exact multiple of 1 hour, that might be a time zone problem. If post time is less than pre time, that either indicates a serious problem with your system clock or, more likely, a program bug that's mixing up the times. Etc.
Should be easy enough to determine whether mySql (0r your persistence code) is doing something. Have your persistenceMethod() return the value it persisted and compare with what you read. They surely should match.
I wonder whether it's the trustworthiness of currentTimeMillis() that's in question:
Returns the current time in milliseconds. Note that while the unit of
time of the return value is a millisecond, the granularity of the
value depends on the underlying operating system and may be larger.
For example, many operating systems measure time in units of tens of
milliseconds.
Given that you are doing a >= test, I can't quite see how that might manifest, but it makes me wonder exactly what times you are getting.
That is really strange. Java will certainly not rearrange statements and execute them in a different order if those statements might have side effects which affect subsequent statements.
I think this error happens because System.currentTimeMillis is not as precise as you think. The API documentation of that method says:
Returns the current time in milliseconds. Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
It sounds strange, but time might even seem to be going backwards in some cases, so the value that currentTimeMillis returns at one moment can be lower than what it returned an instant earlier. See this question: Will System.currentTimeMillis always return a value >= previous calls?
Related
This question already has answers here:
How do I write a correct micro-benchmark in Java?
(11 answers)
Closed 5 years ago.
My goal is to write a framework for measuring method execution or transaction time and for processing the measurements, i.e. storing, analysis etc. Transaction may include calls to external systems, with synchronously or asynchronously waiting for the results.
There already have been some questions around that topic, like
"How do I time a method's execution"
"Measure execution time for a Java method"
"System.currentTimeMillis vs System.nanoTime"
And all the answers boil down to three approaches for taking the time
System.currentTimeMillis()
System.nanoTime()
Instant.now() and Duration(since Java 8)
I know, all of these have some implications
System.currentTimeMillis()
The result of this method depends on the platform. On Linux you get 1ms resolution, of Windows you get 10ms (single core) ~15ms (multi core). So it's ok for measuring large running operations or multiple executions of short running ops.
System.nanoTime()
You get a high resolution time measure, with nano second precision (but not necessarily nano second accuracy) and you get an overflow after 292 years (I could live with that).
Instant.now() and Duration
Since Java 8 there is the new time API. An instant has a second and a nano second field, so it uses on top of the Object reference two long values (same for Duration). You also get nano second precision, depending on the underlying clock (see "Java 8 Instant.now() with nanosecond resolution?"). The instantiaion is done by invoking Instant.now() which maps down to System.currentTimeMillis() for the normal System clock.
Given the facts, it becomes apparent, that best precision is only achievable with System.nanoTime(), but my question targets more towards a best-practice for dealing with the measures in general, which not only includes the measure taking but also the measure handling.
Instant and Duration provide best API support (calculating, comparing, etc) but have os-dependend precision in standard case, and more overhead for memory and creating a measure (Object construction, deeper callstack)
System.nanoTime() and System.currentTimeMillis() have different levels of precision but only have basic "api" support (math operations on long), but are faster to get and smaller to keep in memory.
So what would be the best approach? Are there any implications I didn't think of? Are there any alternatives?
You are focusing too much on the unimportant detail of the precision. If you want to measure/profile the execution of certain operations, you have to make sure that these operation run long enough to make the measurement unaffected by one-time artifacts, small differences in thread scheduling timing, garbage collection or HotSpot optimization. In most cases, if the differences become smaller than the millisecond scale, they are not useful to draw conclusions from them.
The more important aspect is whether the tools are designed for your task. System.currentTimeMillis() and all other wall-clock based APIs, whether they are based on currentTimeMillis() or not, are designed to give you a clock which is intended to be synchronized with Earth’s rotation and its path around the Sun, which loads it with the burden of Leap Seconds and other correction measures, not to speak of the fact that your computer’s clock may be out of sync with the wall clock anyway and get corrected, e.g. via NTP updates, in the worst case jumping right when you are trying to measure your elapsed time, perhaps even backwards.
In contrast, System.nanoTime() is designed to measure elapsed time (exactly what you want to do) and nothing else. Since its return value has an unspecified origin and may even be negative, only differences between two values returned by this method make any sense at all. You will find this even in the documentation:
The values returned by this method become meaningful only when the difference between two such values, obtained within the same instance of a Java virtual machine, is computed.
So when you want to measure and process the elapsed time of your method execution or transactions, System.nanoTime() is the way to go. Granted, it only provides a naked long value, but it isn’t clear what kind of API support you want. Since points of time are irrelevant and even distracting here, you’ll have a duration only, which you may convert to other time units or, if you want to use the new time API, you can create a Duration object using Duration.ofNanos(long), allowing you to add and subtract duration values and compare them, but there isn’t much more you could do. You must not mix them up with wall-clock or calendar based durations…
As a final note, the documentation is a bit imprecise about the limitation. If you are calculating the difference between two values returned by System.nanoTime(), a numerical overflow isn’t bad per se. Since the counter has an unspecified origin, the start value of your operation might be close to Long.MAX_VALUE whereas the end value is close to Long.MIN_VALUE because the JVM’s counter had an overflow. In this case, calculating the difference will cause another overflow, producing a correct value for the difference. But if you store that difference in a signed long, it can hold at most 2⁶³ nanoseconds, limiting the difference to max 292 years, but if you treat it as unsigned long, e.g. via Long.compareUnsigned and Long.toUnsignedString, you may handle even 2⁶⁴ nanoseconds duration, in other words you can measure up to 584 years of elapsed time this way, if your computer doesn’t break in-between…
I would recommend using getThreadCpuTime from ThreadMXBean (see also https://stackoverflow.com/a/7467299/185031). If you want to measure the execution time of a method, you are most of the time not so much interested in the wall clock, but more on the CPU execution time.
I am writing a stress test that will issue many calls to a remote server. I want to collect the following statistics after the test:
Latency (in milliseconds) of the remote call.
Number of operations per second that the remote server can handle.
I can successfully get (2), but I am having problems with (1). My current implementation is very similar to the one shown in this other SO question. And I have the same problem described in that question: latency reported by using System.currentTimeMillis() is longer than expected when the test is run with multiple threads.
I analyzed the problem and I am positive the problem comes from the thread interleaving (see my answer to the other question that I linked above for details), and that System.currentTimeMillis() is not the way to solve this problem.
It seems that I should be able to do it using java.lang.management, which has some interesting methods like:
ThreadMXBean.getCurrentThreadCpuTime()
ThreadMXBean.getCurrentThreadUserTime()
ThreadInfo.getWaitedTime()
ThreadInfo.getBlockedTime()
My problem is that even though I have read the API, it is still unclear to me which of these methods will give me what I want. In the context of the other SO question that I linked, this is what I need:
long start_time = **rightMethodToCall()**;
result = restTemplate.getForObject("Some URL",String.class);
long difference = (**rightMethodToCall()** - start_time);
So that the difference gives me a very good approximation of the time that the remote call took, even in a multi-threaded environment.
Restriction: I'd like to avoid protecting that block of code with a synchronized block because my program has other threads that I would like to allow to continue executing.
EDIT: Providing more info.:
The issue is this: I want to time the remote call, and just the remote call. If I use System.currentTimeMillis or System.nanoTime(), AND if I have more threads than cores, then it is possible that I could have this thread interleaving:
Thread1: long start_time ...
Thread1: result = ...
Thread2: long start_time ...
Thread2: result = ...
Thread2: long difference ...
Thread1: long difference ...
If that happens, then the difference calculated by Thread2 is correct, but the one calculated by Thread1 is incorrect (it would be greater than it should be). In other words, for the measurement of the difference in Thread1, I would like to exclude the time of lines 4 and 5. Is this time that the thread was WAITING?
Summarizing question in a different way in case it helps other people understand it better (this quote is how #jason-c put it in his comment.):
[I am] attempting to time the remote call, but running the test with multiple threads just to increase testing volume.
Use System.nanoTime() (but see updates at end of this answer).
You definitely don't want to use the current thread's CPU or user time, as user-perceived latency is wall clock time, not thread CPU time. You also don't want to use the current thread's blocking or waiting time, as it measures per-thread contention times which also doesn't accurately represent what you are trying to measure.
System.nanoTime() will return relatively accurate results (although granularity is technically only guaranteed to be as good or better than currentTimeMillis(), in practice it tends to be much better, generally implemented with hardware clocks or other performance timers, e.g. QueryPerformanceCounter on Windows or clock_gettime on Linux) from a high resolution clock with a fixed reference point, and will measure exactly what you are trying to measure.
long start_time = System.nanoTime();
result = restTemplate.getForObject("Some URL",String.class);
long difference = (System.nanoTime() - start_time);
long milliseconds = difference / 1000000;
System.nanoTime() does have it's own set of issues but be careful not to get whipped up in paranoia; for most applications it is more than adequate. You just wouldn't want to use it for, say, precise timing when sending audio samples to hardware or something (which you wouldn't do directly in Java anyways).
Update 1:
More importantly, how do you know the measured values are longer than expected? If your measurements are showing true wall clock time, and some threads are taking longer than others, that is still an accurate representation of user-perceived latency, as some users will experience those longer delay times.
Update 2 (based on clarification in comments):
Much of my above answer is still valid then; but for different reasons.
Using per-thread time does not give you an accurate representation because a thread could be idle/inactive while the remote request is still processing, and you would therefore exclude that time from your measurement even though it is part of perceived latency.
Further inaccuracies are introduced by the remote server taking longer to process the simultaneous requests you are making - this is an extra variable that you are adding (although it may be acceptable as representative of the remote server being busy).
Wall time is also not completely accurate because, as you have seen, variances in local thread overhead may add extra latency that isn't typically present in single-request client applications (although this still may be acceptable as representative of a client application that is multi-threaded, but it is a variable you cannot control).
Of those two, wall time will still get you closer to the actual result than per-thread time, which is why I left the previous answer above. You have a few options:
You could do your tests on a single thread, serially -- this is ultimately the most accurate way to achieve your stated requirements.
You could not create more threads than cores, e.g. a fixed size thread pool with bound affinities (tricky: Java thread affinity) to each core and measurements running as tasks on each. Of course this still adds any variables due to synchronization of underlying mechanisms that are beyond your control. This may reduce the risk of interleaving (especially if you set the affinities) but you still do not have full control over e.g. other threads the JVM is running or other unrelated processes on the system.
You could measure the request handling time on the remote server; of course this does not take network latency into account.
You could continue using your current approach and do some statistical analysis on the results to remove outliers.
You could not measure this at all, and simply do user tests and wait for a comment on it before attempting to optimize it (i.e. measure it with people, who are what you're developing for anyways). If the only reason to optimize this is for UX, it could very well be the case that users have a pleasant experience and the wait time is totally acceptable.
Also, none of this makes any guarantees that other unrelated threads on the system won't be affecting your timings, but that is why it is important to both a) run your test multiple times and average (obviously) and b) set an acceptable requirement for timing error's that you are OK with (do you really need to know this to e.g. 0.1ms accuracy?).
Personally, I would either do the first, single-threaded approach and let it run overnight or over a weekend, or use your existing approach and remove outliers from the result and accept a margin of error in the timings. Your goal is to find a realistic estimate within a satisfactory margin of error. You will also want to consider what you are going to ultimately do with this information when deciding what is acceptable.
I wish to calculate the time passed in milliseconds from a specific time in Java.
The classic way it to use System.currentTimeMillis(); for the starting time, and then use this again with the previous one to get the elapsed time. I wish to do something similar to this, but NOT rely on the system time for this.
If I rely on the system time, the user of the program could manipulate the system clock to hack the program.
I have tried using code similar to the following:
int elapsed = 0;
while (true) {
Thread.sleep(10);
elapsed += 10;
}
This works, but I it is not too reliable in the case that the computer lags and then locks up for a second or two.
Any ideas anyone?
You want to utilize System.nanoTime. It has no relation to the system clock. It can only be used to track relative time which seems to be all you want to do.
In an effort to prevent this answer from just being a link to another answer here is a short explanation.
From Documentation
public static long nanoTime() Returns the current value of the most
precise available system timer, in nanoseconds.
This method can only be used to measure elapsed time and is not
related to any other notion of system or wall-clock time. The value
returned represents nanoseconds since some fixed but arbitrary time
(perhaps in the future, so values may be negative). This method
provides nanosecond precision, but not necessarily nanosecond
accuracy. No guarantees are made about how frequently values change.
Differences in successive calls that span greater than approximately
292 years (263 nanoseconds) will not accurately compute elapsed time
due to numerical overflow.
Yet another link to timer information: https://blogs.oracle.com/dholmes/entry/inside_the_hotspot_vm_clocks
You could use Java's Timer class to spawn a "check" callback at some specific precision, lets say every 500ms. This callback would not be used to determine that 500ms actually did pass. You would call System.nanoTime in the callback and compare it to the last time you called System.nanoTime. That would give you a fairly accurate representation of the amount of time that has passed regardless of the wall clock changing.
You can take a look here: System.currentTimeMillis vs System.nanoTime
Is there a Java API/suggestion to use instead of System.currentTimeMillis() to get current time in milli second precision on windows - requirement is two subsequent calls with a sleep time of 1ms in between should give two different time - currently i need to explicitly sleep for 15 ms to get different times
Don't attempt to use time to create unique values. Use the database to generate a unique id (key I'm assuming) for the record. Either use an auto incrementing field or create a separate table with a single record holding the counter that you can lock and update safely.
While you may get a solution that works, counting on timing to prevent a clash of resources will eventually catch up to you.
Since Java 1.5 you can use System.nanoTime() for micro benchmarks with higher precision. As the fixed time this is based on may change (see Javadoc for the method), it might make sense to combine it with System.currentTimeMillis(), e.g.
String time = System.currentTimeMillis() + "" + System.nanoTime();
It's a windows limitation. If you call System.currentTimeMillis() on other operating systems you get much higher precision.
My advise is don't use a time stamp as your source of uniqueness. Use an oracle sequence as it was designed for this problem. Otherwise use the thread name + timetamp (yuk).
OR you can use System.nanoTime(), but it's only useful for time differences, not absolute time.
Since java 1.5, you can use the java.util.UUID to generate unique IDs.
e.g
public static void main(String[] args)
{
System.out.println("uuid=" + UUID.randomUUID().toString());
System.out.println("uuid=" + UUID.randomUUID().toString());
}
The resolution of the currentTimeMillis() call is dependent on the underlying operating system, and should not be relied on for creating unique stamps in your situation. Consider having a UID-singleton which can give you a long value which is incremented by one for each call, and then use that.
Why do you need the times to be unique?
Take the time at the start of the transaction then add one MS for each insert.
It's important here to distinguish accuracy from precision. System.currentTimeMillis() has millisecond precision, but no guarantee whatsoever on accuracy, since it gets this from the underlying OS, and the OS gets it from the hardware, and different hardware clocks have different accuracies.
Using this for data versioning is a bad idea, since even if you had millisecond accuracy, you'd still run the risk of the occasional clash if two things happened in the same millisecond.
Although this is not directly related to the question, I've undestood from the comments that the original attempt is to generate some version identifiers.
This is, of course, a bad idea, as detailed by other posters here.
If you can't use a database, then a better idea would be to use an AtomicInteger or AtomicLong -- then you can invoke getAndIncrement() or incrementAndGet() and not worry about any timing issues that might arise.
The 15ms limitation on absolute time resolution is a feature of your operating system & interrupt rate. I'm aware that for Linux there is a kernel patch to increase the resolution to 1ms (possibly even micro sec?), not sure about Windows though. As others have commented, relative times can be resolved using System#nanoTime() (currently for micro sec precision). Either way, you should consider using db keys (or similar) for assigning unique keys.
Links
Timer resolution in Java
Increasing OS clock resolution
Java gives access to two method to get the current time: System.nanoTime() and System.currentTimeMillis(). The first one gives a result in nanoseconds, but the actual accuracy is much worse than that (many microseconds).
Is the JVM already providing the best possible value for each particular machine?
Otherwise, is there some Java library that can give finer measurement, possibly by being tied to a particular system?
The problem with getting super precise time measurements is that some processors can't/don't provide such tiny increments.
As far as I know, System.currentTimeMillis() and System.nanoTime() is the best measurement you will be able to find.
Note that both return a long value.
It's a bit pointless in Java measuring time down to the nanosecond scale; an occasional GC hit will easily wipe out any kind of accuracy this may have given. In any case, the documentation states that whilst it gives nanosecond precision, it's not the same thing as nanosecond accuracy; and there are operating systems which don't report nanoseconds in any case (which is why you'll find answers quantized to 1000 when accessing them; it's not luck, it's limitation).
Not only that, but depending on how the feature is actually implemented by the OS, you might find quantized results coming through anyway (e.g. answers that always end in 64 or 128 instead of intermediate values).
It's also worth noting that the purpose of the method is to find the two time differences between some (nearby) start time and now; if you take System.nanoTime() at the start of a long-running application and then take System.nanoTime() a long time later, it may have drifted quite far from real time. So you should only really use it for periods of less than 1s; if you need a longer running time than that, milliseconds should be enough. (And if it's not, then make up the last few numbers; you'll probably impress clients and the result will be just as valid.)
Unfortunately, I don't think java RTS is mature enough at this moment.
Java time does try to provide best value (they actually delegate the native code to call get the kernal time). However, JVM specs make this coarse time measurement disclaimer mainly for things like GC activities, and support of underlying system.
Certain GC activities will block all threads even if you are running concurrent GC.
default linux clock tick precision is only 10ms. Java cannot make it any better if linux kernal does not support.
I haven't figured out how to address #1 unless your app does not need to do GC. A decent and med size application probably and occasionally spends like tens of milliseconds on GC pauses. You are probably out of luck if your precision requirement is lower 10ms.
As for #2, You can tune the linux kernal to give more precision. However, you are also getting less out of your box because now kernal context switch more often.
Perhaps, we should look at it different angle. Is there a reason that OPS needs precision of 10ms of lower? Is it okay to tell Ops that precision is at 10ms AND also look at the GC log at that time, so they know the time is +-10ms accurate without GC activity around that time?
If you are looking to record some type of phenomenon on the order of nanoseconds, what you really need is a real-time operating system. The accuracy of the timer will greatly depend on the operating system's implementation of its high resolution timer and the underlying hardware.
However, you can still stay with Java since there are RTOS versions available.
JNI:
Create a simple function to access the Intel RDTSC instruction or the PMCCNTR register of co-processor p15 in ARM.
Pure Java:
You can possibly get better values if you are willing to delay until a clock tick. You can spin checking System.nanoTime() until the value changes. If you know for instance that the value of System.nanoTime() changes every 10000 loop iterations on your platform by amount DELTA then the actual event time was finalNanoTime-DELTA*ITERATIONS/10000. You will need to "warm-up" the code before taking actual measurements.
Hack (for profiling, etc, only):
If garbage collection is throwing you off you could always measure the time using a high-priority thread running in a second jvm which doesn't create objects. Have it spin incrementing a long in shared memory which you use as a clock.