I've inherited a project in which monetary amounts use the double type.
Worse, the framework it uses, and the framework's own classes, use double for money.
The framework ORM also handles retrieval of values from (and storage to) the database. In the database money values are type number(19, 7), but the framework ORM maps them to doubles.
Short of entirely bypassing the framework classes and ORM, is there anything I can do to calculate monetary values precisely?
Edit: yeah, I know BigDecimal should be used. The problem is that I am tightly tied to a framework that where, e.g., the class framework.commerce.pricing.ItemPriceInfo has members double mRawTotalPrice; and double mListPrice. My company's application's own code extends, e.g, this ItemPriceInfoClass.
Realistically, I can't say to my company, "scrap two years of work, and hundreds of thousands of dollars spent, basing code on this framework, because of rounding errors"
If tolerable, treat the monetary type as integral. In other words, if you're working in the US, track cents instead of dollars, if cents provides the granularity you need. Doubles can accurately represent integers up to a very large value (2^53) (no rounding errors up to that value).
But really, the right thing to do is bypass the framework entirely and use something more reasonable. That's such an amateur mistake for the framework to make - who knows what else is lurking?
I didn't see you mention refactoring. I think that's your best option here. Instead of throwing together some hacks to get things working better for now, why not fix it the right way?
Here's some information on double vs BigDecimal. This post suggests using BigDecimal even though it is slower.
Plenty of people will suggest using BigDecimal and if you don't know how to use rounding in your project, that is what you should do.
If you know how to use decimal rounding correctly, use double. Its many orders of magnitude faster, much clear and simpler and thus less error prone IMHO. If you use dollars and cents (or need two decimal places), you can get an accurate result for values up to 70 trillion dollars.
Basically, you won't get round errors if you correct for it using approriate rounding.
BTW: The thought of rounding errors strikes terror into the heart of many developers, but they are not random errors and you can manage them fairly easily.
EDIT: consider this simple example of a rounding error.
double a = 100000000.01;
double b = 100000000.09;
System.out.println(a+b); // prints 2.0000000010000002E8
There are a number of possible rounding strategies. You can either round the result when printing/displaying. e.g.
System.out.printf("%.2f%n", a+b); // prints 200000000.10
or round the result mathematically
double c = a + b;
double r= (double)((long)(c * 100 + 0.5))/100;
System.out.println(r); // prints 2.000000001E8
In my case, I round the result when sending from the server (writing to a socket and a file), but use my own routine to avoid any object creation.
A more general round function is as follows, but if you can use printf or DecimalFormat, can be simpler.
private static long TENS[] = new long[19]; static {
TENS[0] = 1;
for (int i = 1; i < TENS.length; i++) TENS[i] = 10 * TENS[i - 1];
}
public static double round(double v, int precision) {
assert precision >= 0 && precision < TENS.length;
double unscaled = v * TENS[precision];
assert unscaled > Long.MIN_VALUE && unscaled < Long.MAX_VALUE;
long unscaledLong = (long) (unscaled + (v < 0 ? -0.5 : 0.5));
return (double) unscaledLong / TENS[precision];
}
note: you could use BigDecimal to perform the final rounding. esp if you need a specifc round method.
Well, you don't have that many options in reality:
You can refactor the project to use e.g. BigDecimal (or something that better suits its needs) to represent money.
Be extremely careful for overflow/underflow and loss of precision, which means adding tons of checks, and refactoring even larger proportion of the system in an unnecessary way. Not to mention how much research would be necessary if you are to do that.
Keep things the way they are and hope nobody notices (this is a joke).
IMHO, the best solution would be to simply refactor this out. It might be some heavy refactoring, but the evil is already done and I believe that it should be your best option.
Best,
Vassil
P.S. Oh and you can treat money as integers (counting cents), but that doesn't sound like a good idea if you are going to have currency conversions, calculating interest, etc.
I think this situation is at least minimally salvageable for your code. You get the value as a double via the ORM framework. You can then convert it to BigDecimal using the static valueOf method (see here for why) before doing any math/calculations on it, and then convert it back to double only for storing it.
Since you are extending these classes anyway, you can add getters for your double value that gets them as BigDecimal when you need it.
This may not cover 100% of the cases (I would be especially worried about what the ORM or JDBC driver is doing to convert the double back to a Number type), but it is so much better than just doing the math on the raw doubles.
However, I am far from convinced that this approach is actually cheaper for the company in the long run.
Related
I have written the following simple function that calculates the arctan of the inverse of an integer. I was wondering how to use BigDecimal instead of double to increase the accuracy of the results. I was also thinking of using a BigInteger to store the growing multiples of xSquare that the "term" value is divided by.
I have limited experience with the syntax for how to perform calculations on BigDecimals. How would I revise this function to use them?
/* Thanks to https://www.cygnus-software.com/misc/pidigits.htm for explaining the general calculation method
credited to John Machin.
*/
public static double atanInvInt(int x) {
// Returns the arc tangent of an inverse integer
/* Terminates once the remaining amount reaches zero or the denominator reaches 2101.
If the former happens, the accuracy should be determined by the number format used, such as double.
If the latter happens, the result should be off by at most one from the correct nearest value
in the seventh decimal place, if allowed by the accuracy of the number format used.
This likely only happens if the integer is 1.
*/
int xSquare = x*x;
double result = ((double)1)/x;
double term = ((double)1)/x;
int divisor = 1;
double midResult;
while ((term > 0)) {
term = term / xSquare;
divisor += 2;
midResult = result - term/divisor;
term = term /xSquare;
divisor += 2;
result = midResult + term/divisor;
if (divisor >= 2101) {
return ((result + midResult) / 2);
}
}
return result;
}
The BigDecimal provides very intuitive wrapper methods to provide all the different operations. you can have something like this to have an arbitrary precision of, for example, 99:
public static void main(String[] args) {
System.out.println(atanInvInt(5, 99));
// 0.197395559849880758370049765194790293447585103787852101517688940241033969978243785732697828037288045
}
public static BigDecimal atanInvInt(int x, int scale) {
BigDecimal one = new BigDecimal("1");
BigDecimal two = new BigDecimal("2");
BigDecimal xVal = new BigDecimal(x);
BigDecimal xSquare = xVal.multiply(xVal);
BigDecimal divisor = new BigDecimal(1);
BigDecimal result = one.divide(xVal, scale, RoundingMode.FLOOR);
BigDecimal term = one.divide(xVal, scale, RoundingMode.FLOOR);
BigDecimal midResult;
while (term.compareTo(new BigDecimal(0)) > 0) {
term = term.divide(xSquare, scale, RoundingMode.FLOOR);
divisor = divisor.add(two);
midResult = result.subtract(term.divide(divisor, scale, RoundingMode.FLOOR));
term = term.divide(xSquare, scale, RoundingMode.FLOOR);
divisor = divisor.add(two);
result = midResult.add(term.divide(divisor, scale, RoundingMode.FLOOR));
if (divisor.compareTo(new BigDecimal(2101)) >= 0) {
return result.add(midResult).divide(two, scale, RoundingMode.FLOOR);
}
}
return result;
}
For anyone who wanted to know why it was beneficial to pose this question to begin with: That is a fair question. I have written a rather long answer to it. I believe that writing this answer helped me to articulate to myself things about the BigDecimal class that are more intuitive now that I have Armando Carballo’s answer than they were before, so writing it was hopefully educational. I can only hope that reading it will be as well, though likely in a different way if at all.
The official documentation lists methods, but it doesn’t explain how they are used in the same way that Armando Carballo’s code demonstrates. For example, while the way the BigDecimal.divide method works is pretty intuitive, there is nothing in the official documentation that says “to take the mean of two numbers, not only should you have BigDecimals for those two numbers, but you should also create a BigDecimal equal to 2 and apply the BigDecimal.divide method to the result of a BigDecimal.add operation with the 2 BigDecimal as the input for the divisor.” This is something that is simple enough to be perfectly intuitive once you see it, but if you’ve never used object-oriented methods for the specific purpose of performing arithmetic before, it may be less intuitive the first time you are trying to figure out how to take the mean.
As another example, consider the idea that to figure out whether a number is greater than or equal to another number, instead of using a Boolean operator on the two numbers, you use a compareTo method that can give three possible outputs on one number with the other number as an input, then apply a Boolean operator to the output of that method. This makes perfect sense once you see it in action and have a quick sense of how the compareTo method works, but may be less obvious when you’re staring at a quick description of the compareTo method in the official documentation, even if the description is clear and you are able to figure out what the compareTo method will output with a given BigDecimal value calling the method and a given BigDecimal input as the comparison value. For anyone who has used compareTo methods with other classes besides BigDecimal extensively, this is probably obvious even if they’re new to the specific class, but if you haven’t used Booleans on the result of ANY compareTo method recently, it’s faster to see it.
When working with ints, you might very well write code a bit like this:
int x = 5;
x = x + 2;
System.out.println(x) // should be 7
Here, the “2” value was never declared to be an int. The result of the addition was the same as if we had declared y=2 and said that x = x+y instead of x = x+2, but with the above lines of code no named variable, or Integer object if we used those instead of primitive ints, was created for the “2”. With BigDecimal, on the other hand, since the BigDecimal.add method requires BigDecimals as inputs, it would be mandatory to create a BigDecimal equal to 2 in order to add 2. I don’t see anything in the official documentation that says “use this as a more accurate substitute for doubles, or for longs if you want something more versatile than BigInteger, but in addition to using it as a substitute for declared variables, also create BigDecimal objects equal to small integers that by themselves wouldn’t call for the use of the BigDecimal class so that you can use them in operations. Both your variables and the small values you are adding to them need to be BigDecimals if you want to use BigDecimals.”
Finally, let me explain something that has the potential to make the BigDecimal class more intimidating than it needs to be. Anyone who has ever worked with primitive arrays and tried to predict in advance at the time the array is created exactly how large it needs to be, or is familiar with how lower-level languages involve certain situations in which a programmer needs to know exactly how many bytes something takes up, may feel the need for caution when dealing with something that seems to demand a specified level of precision upfront. The documentation says this: “If no rounding mode is specified and the exact result cannot be represented, an exception is thrown; otherwise, calculations can be carried out to a chosen precision and rounding mode by supplying an appropriate MathContext object to the operation.” A newbie reading that sentence for the first time may be thinking that they are going to have to think extensively about rounding when writing their code for the first time or else face exceptions as soon as a value cannot be represented exactly, or that they are going to have to read the documentation on the MathContext object as well before using BigDecimal, which in turn might lead to reading IEEE standards that help grant an understanding of floating point numbers but are far removed from what the person actually wanted to code. Seeing that some of the constructors for BigDecimal take arrays as inputs and that others take a MathContext as an input, along with noticing that one of the constructors for the related BigInteger class takes a byte array as the input, may strengthen the feeling that using this object class requires a very fine understanding of the exact number of digits that will be used for the specific calculations the class is used for and that understanding MathContext is more or less essential to even the most basic use of the class. While I’m sure understanding MathContext is helpful, baby’s first BigDecimal project can actually work perfectly well without the need to learn this added functionality at the same time as the first use of the BigDecimal. Reading up on the scale parameter might also lead to the belief by a coder looking up info on the class for the first time that it is necessary to predict the order of magnitude of the answer in advance in order to use the class at all.
Armando Caballo’s commendable answer shows that these concerns of a hypothetical newbie are overblown, as while rounding mode does need to be specified fairly often and a consistent scale is often called as a parameter when using the divide method, the scale parameter is actually a fairly arbitrary specification of the desired accuracy in terms of number of decimal places and not something that requires pinpoint predictions about exactly what numbers the class will handle (unless the ultimate purpose for which the BigDecimal is being used requires a finely controlled level of accuracy, in which case it is fairly easy to specify). An “infinite” series of added and subtracted terms to compute an arc tangent was processed without ever declaring a MathContext object.
I'm trying to create a physical calculation program in Java. Therefore I used some formulas, but they always returned a wrong value. I split them and and found: (I used long so far.)
8 * 830584000 = -1945262592
which is obviously wrong. There are fractions and very high numbers in the formulas, such as 6.095E23 and 4.218E-10 for example.
So what datatype would fit best to get a correct result?
Unless you have a very good reason not to, double is the best type for physical calculations. It was good enough for the wormhole modelling in the film Interstellar so, dare I say it, is probably good enough for you. Note well though that, as a rough guide, it only gives you only 15 decimal significant figures of precision.
But you need to help the Java compiler:
Write 8.0 * 830584000 for that expression to be evaluated in double precision. 8.0 is a double literal and causes the other arguments to be promoted to a similar type.
Currently you are using integer arithmetic, and are observing wrap-around effects.
Reference: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
If you need perfect accuracy for large decimal numbers, BigDecimal is the way to go, though it will be at the cost of performance. If you know you numbers are not that large, you can use long instead which should be faster, but has a much more limited range and will require you to convert from and to decimal numbers.
As physics calculations involves a lot of floating point operations, float data type can be a good option in such calculations. I Hope it will help. :)
I came to know about the accuracy issues when I executed the following following program:
public static void main(String args[])
{
double table[][] = new double[5][4];
int i, j;
for(i = 0, j = 0; i <= 90; i+= 15)
{
if(i == 15 || i == 75)
continue;
table[j][0] = i;
double theta = StrictMath.toRadians((double)i);
table[j][1] = StrictMath.sin(theta);
table[j][2] = StrictMath.cos(theta);
table[j++][3] = StrictMath.tan(theta);
}
System.out.println("angle#sin#cos#tan");
for(i = 0; i < table.length; i++){
for(j = 0; j < table[i].length; j++)
System.out.print(table[i][j] + "\t");
System.out.println();
}
}
And the output is:
angle#sin#cos#tan
0.0 0.0 1.0 0.0
30.0 0.49999999999999994 0.8660254037844387 0.5773502691896257
45.0 0.7071067811865475 0.7071067811865476 0.9999999999999999
60.0 0.8660254037844386 0.5000000000000001 1.7320508075688767
90.0 1.0 6.123233995736766E-17 1.633123935319537E16
(Please forgive the unorganised output).
I've noted several things:
sin 30 i.e. 0.5 is stored as 0.49999999999999994.
tan 45 i.e. 1.0 is stored as 0.9999999999999999.
tan 90 i.e. infinity or undefined is stored as 1.633123935319537E16 (which is a very big number).
Naturally, I was quite confused to see the output (even after deciphering the output).
So I've read this post, and the best answer tells me:
These accuracy problems are due to the internal representation of floating > point numbers and there's not much you can do to avoid it.
By the way, printing these values at run-time often still leads to the correct results, at >least using modern C++ compilers. For most operations, this isn't much of an issue.
answered Oct 7 '08 at 7:42
Konrad Rudolph
So, my question is:
Is there any way to prevent such inaccurate results (in Java)?
Should I round-off the results? In that case, how would I store infinity i.e. Double.POSITIVE_INFINITY?
You have to take a bit of a zen* approach to floating-point numbers: rather than eliminating the error, learn to live with it.
In practice this usually means doing things like:
when displaying the number, use String.format to specify the amount of precision to display (it'll do the appropriate rounding for you)
when comparing against an expected value, don't look for equality (==). Instead, look for a small-enough delta: Math.abs(myValue - expectedValue) <= someSmallError
EDIT: For infinity, the same principle applies, but with a tweak: you have to pick some number to be "large enough" to treat as infinity. This is again because you have to learn to live with, rather than solve, imprecise values. In the case of something like tan(90 degrees), a double can't store π/2 with infinite precision, so your input is something very close to, but not exactly, 90 degrees -- and thus, the result is something very big, but not quite infinity. You may ask "why don't they just return Double.POSITIVE_INFINITY when you pass in the closest double to π/2," but that could lead to ambiguity: what if you really wanted the tan of that number, and not 90 degrees? Or, what if (due to previous floating-point error) you had something that was slightly farther from π/2 than the closest possible value, but for your needs it's still π/2? Rather than make arbitrary decisions for you, the JDK treats your close-to-but-not-exactly π/2 number at face value, and thus gives you a big-but-not-infinity result.
For some operations, especially those relating to money, you can use BigDecimal to eliminate floating-point errors: you can really represent values like 0.1 (instead of a value really really close to 0.1, which is the best a float or double can do). But this is much slower, and doesn't help you for things like sin/cos (at least with the built-in libraries).
* this probably isn't actually zen, but in the colloquial sense
You have to use BigDecimal instead of double. Unfortunately, StrictMath doesn't support BigDecimal, so you will have to use another library, or your own implementation of sin/cos/tan.
This is inherent in using floating-point numbers, in any language. Actually, it's inherent in using any representation with a fixed maximum precision.
There are several solutions. One is to use an extended-precision math package -- BigDecimal is often suggested for Java. BigDecimal can handle many more digits of precision, and also -- because it's a decimal representation rather than a 2's-complement representation -- tends to round off in ways that are less surprising to humans who are used to working in base 10. (That doesn't necessarily make them more correct, please note. Binary can't represent 1/3 exactly, but neither can decimal.)
There are also extended-precision 2's-complement floating-point representations. Java directly supports float and double (which are usually also supported by the hardware), but it's possible to write versions which support more digits of accuracy.
Of course any of the extended-precision packages will slow down your computations. So you shouldn't resort to them unless you actually need them.
Another may to use fixed-point binary rather than floating point. For example, the standard solution for most financial calculations is simply to compute in terms of the smallest unit of currency -- pennies, in the US -- in integers, converting to and from the display format (eg dollars and cents) only for I/O. That's also the approach used for time in Java -- the internal clock reports an integer number of milliseconds (or nanoseconds, if you use the nanotime call), which gives both more than sufficient precision and a more than sufficient range of values for most practical purposes. Again, this means that roundoff tends to happen in a way that matches human expectations... and again, that's less about accuracy than about not surprising the users. And these representations, because they process as integers or longs, allow fast computation -- faster than floating point, in fact.
There are yet other solutions which involve computing in rational numbers, or other variations, in an attempt to compromise between computational cost and precision.
But I also have to ask... Do you really NEED more precision than float is giving you? I know the roundoff is surprising, but in many cases it's perfectly acceptable to just let it happen, possibly rounding off to a less surprising number of fractional digts when you display the results to the user. In many cases, float or double are Just Fine for real-world use. That's why the hardware supports them, and that's why they're in the language.
Should we use double or BigDecimal for calculations in Java?
How much is the overhead in terms of performance for BigDecimal as compared to double?
For a serious financial application BigDecimal is a must.
Depends on how many digits you need you can go with a long and a decimal factor for visualization.
For general floating point calculations, you should use double. If you are absolutely sure that you really do need arbitrary precision arithmetic (most applications don't), then you can consider BigDecimal.
You will find that double will significantly outperform BigDecimal (not to mention being easier to work with) for any application where double is sufficient precision.
Update: You commented on another answer that you want to use this for a finance related application. This is one of the areas where you actually should consider using BigDecimal, otherwise you may get unexpected rounding effects from double calculations. Also, double values have limited precision, and you won't be able to accurately keep track of pennies at the same time as millions of dollars.
How much is the overhead in terms of performance for BigDecimal as compared to double?
A lot. For example, a multiplication of two doubles is a single machine instruction. Multiplying two BigDecimals is probably a minimum of 50 machine instructions, and has complexity of O(N * M) where M and N are the number of bytes used to represent the two numbers.
However, if your application requires the calculation to be "decimally correct", then you need to accept the overhead.
However (#2) ... even BigDecimal can't do this calculation with real number accuracy:
1/3 + 1/3 + 1/3 -> ?
To do that computation precisely you would need to implement a Rational type; i.e. a pair of BigInteger values ... and some thing to reduce the common factors.
However (#3) ... even a hypothetical Rational type won't give you a precise numeric representation for (say) Pi.
As always: it depends.
If you need the precision (even for "small" numbers, when representing amounts for example) go with BigDecimal.
In some scientific applications, double may be a better choice.
Even in finance we can't answer without knowing what area. For instance if you were doing currency conversions of $billions, where the conversion rate could be to 5 d.p. you might have problems with double. Whereas for simply adding and subtracting balances you'd be fine.
If you don't need to work in fractions of a cent/penny, maybe an integral type might be more appropriate, again it depends on the size of numbers involved.
Okay, here's my problem.
Basically I have this problem.
I have a number like .53999999.
How do I round it up to 54 without using any of the Math functions?
I'm guessing I have to multiply by 100 to scale it, then divide?
Something like that?
The issue is with money. let's say I have $50.5399999 I know how to get the $50, but I don't have how to get the cents. I can get the .539999 part, but I don't know how to get it to 54 cents.
I would use something like:
BigDecimal result = new BigDecimal("50.5399999").setScale(2, BigDecimal.ROUND_HALF_UP);
There is a great article called Make cents with BigDecimal on JavaWorld that you should take a look at.
You should use a decimal or currency type to represent money, not floating point.
Math with money is more complex than most engineers think (over generalization)
If you are doing currency calculations, I think you may be delving into problems that seem simple at their surface but are actually quite complex. For instance, rounding methods that are a result of business logic decisions that are repeated often can drastically affect the totals of calculations.
I would recommend looking at the Java Currency class for currency formatting.
Also having a look at this page on representing money in java may be helpful.
If this is homework, showing the teacher that you have thought through the real-world problem rather than just slung a bunch of code together that "works" - will surely be more impressive.
On a side note, I initially was going to suggest looking at the implementation of the Java math methods in the source code, so I took a look. I noticed that Java was using native methods for its rounding methods - just like it should.
However, a look at BigDecimal shows that there is Java source available for rounding in Java. So rather than just give you the code for your homework, I suggest that you look at the BigDecimal private method doRound(MathContext mc) in the Java source.
If 50.54 isn't representable in double precision, then rounding won't help.
If you're trying to convert 50.53999999 to a whole number of dollars and a whole number of cents, do the following:
double d = 50.539999; // or however many 9's, it doesn't matter
int dollars = (int)d;
double frac = d - dollars;
int cents = (int)((frac * 100) + 0.5);
Note that the addition of 0.5 in that last step is to round to the nearest whole number of cents. If you always want it to round up, change that to add 0.9999999 instead of 0.5.
Why would you not want to use any Math functions?
static long round(double a)
-Returns the closest long to the argument.
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Math.html
To represent money I would take the following advice instead of re-inventing the wheel:
http://www.javapractices.com/topic/TopicAction.do?Id=13
Try storing your currency as number of cents (you could abstract this to number of base curreny units) with a long.
Edit: Since this is homework, you may not have control over the types. Consider this a lesson for future projects
long money = 5054;
long cents = money % 100;
long dollars = money / 100; // this works due to integer/long truncation
System.out.printf("$%d.%02.d", dollars, cents);
You need to make the number .535 and compare that with your original number to see if you'll round up or down. Here's how you get .535 from .53999999 (should work for any number):
num = .53999999;
int_num = (int)(num * 100); // cast to integer, however you do it in Java
compare_num = (int_num + 0.5) / 100;
compare_num would be .535 in this case. If num is greater than or equal to compare_num, round up to int_num + 1. Otherwise round down simply to int_num.
Sean seems to have it, except, if you want to impose proper rules then you may want to throw in an if statement like so:
double value = .539999;
int result = (int) (value*100);
if(((value*100)%result)>.5)
result++;
I suggest you use long for rounding a double value. It won't matter for small numbers but could make a difference.
double d = 50.539999;
long cents = (long)(d * 100 + 0.5);
double rounded = cents/100;
What exactly are you trying to do? Are you always trying to go to two digits? Or are you always trying to fix things like 99999 at the end no matter what?
According to the comments, it is in fact the former: rounding to dollars and cents. So indeed just round(100*a)/100 is what you want. (The latter would be much more complicated...)
Finally, if you just want to extract the cents part, the following would work:
dollars_and_cents = round(100*a)/100
cents = (dollars_and_cents-(int)dollars_and_cents)*100
(or does java just have frac? In which case the last line would just be frac(dollars_and_cents)*100.