My current client is in the middle of hiring some Java developers and as I mentioned earlier in March (Interviewing – the importance of PASSION!) I’ve been doing some of the interviewing. One of the things we’ve done is to create a technical task to see how the candidates actually code. It’s a simple exercise that requires them to think through some of the basics of financial operations and one thing that has surprised me has been the common use of doubles to represent financial values. It’s been highlighted for some time that this is not a great thing to do but someone actually challenged me to show that it wasn’t.

So here we go…

package com.bloodredsun; public class DoubleOperation { public static void main(String[] args) { double t1 = 10.266d; double t2 = 10.0d; //Outputs 0.266 System.out.println(t1-t2); double h1 = 100.266d; double h2 = 100.0d; //Outputs 0.26600000000000534 System.out.println(h1-h2); } }

Ouch! That is not what we want but it is the classic behaviour of doubles. The inability to represent some decimals in the IEEE-754 format (as binary fractions) causes this. If we want correct precision the answer is to use BigDecimals but we have to remember to use Strings in the constructors or you end up with the same issues that you were trying to avoid.

package com.bloodredsun; import java.math.BigDecimal; public class BigDecimalOperation { public static void main(String[] args) { BigDecimal t1 = new BigDecimal("10.266"); BigDecimal t2 = new BigDecimal("10.0"); //Outputs 0.266 System.out.println(t1.subtract(t2)); BigDecimal h1 = new BigDecimal("100.266"); BigDecimal h2 = new BigDecimal("100.0"); //Outputs 0.266 System.out.println(h1.subtract(h2)); } }

That’s great but wouldn’t it be nice to use the normal operators rather than the overly-verbose method calls for the mathematical operations.

Now there is no way that we can do this in Java but if we let ourselves use another language on the JVM…

package com.bloodredsun object ScalaBigDecimalOperation { def main (args: Array[String]) { var t1 = BigDecimal("10.266") var t2 = BigDecimal("10.0") //Outputs 0.266 println(t1 - t2) var h1 = BigDecimal("100.266") var h2 = BigDecimal("100.0") //Outputs 0.266 println(h1 - h2) } }

Scala FTW!

PS if you want to know more about floating point operations have a read of What Every Computer Scientist Should Know About Floating-Point Arithmetic

EdAre you sure doubles are so bad in finance?? Really?

Each time I do alpha + beta I get a value signma + an error term epsilon. Provided epsilon is sufficiently less than the precision, you’re ok because you can just round the value to the appropriate accuracy (less than half epsilon I guess)

What’s the performance difference between big decimal and the scala version?

MartinAs you say, providing epislon is small enough it’s not a problem and in many cases we can live with it. The trouble arises when you have billions of calculations that sum up to something significant or when you are taking leveraged positions that can massively exacerbate any imprecision.

As for the performance of the Java and Scala version, they are identical since they both compile down to byte code. The difference is purely a syntactical one.

nraynaudIn the paper you mentioned, they explain how to cope with the precision, and this really funny fact that you can’t have the exact number, but that you can compute and represent the error anyways, and do stuff with it like http://en.wikipedia.org/wiki/Kahan_summation_algorithm .

So I still don’t see your point.

MartinPost authorMy point was that despite all the focus given to this topic, there are still developers who do not know about the imprecision of floating point values.

Like most rules, knowing this gives you the freedom to break it when you are able to, such as when the error is acceptably small and the performance benefit is a great enough advantage.

strattonHey Ed,

The problem with doubles as he shows is you can’t represent the numbers exactly. Imagine adding a whole bunch of numbers that should have added up to an exact value. It’s hard to verify your books if cents are gone here and there. When working with money, you need your numbers to be exact.

“That’s great but wouldn’t it be nice to use the normal operators rather than the overly-verbose method calls for the mathematical operations.”

As the author noted, with Scala BigDecimal has its operators overloaded so you can use normal + and – operators, while with java because BigDecimal isn’t a primitive, you need to use “.subtract()” and such.

RichDoubles are bad because they do not mean what naive programmers expect them to mean. Though in this case I think a workable solution is to pick a suitable precision and just use longs.

TristanIn the 238k range a float will(may) be off by a penny. This may not be an issue for day to day banking, Depending on the value of the whole part the precision of the fractional goes down. It turns out that at about 238000 the precision for the fractional goes down to ~0.005 which will impact financial transactions.

besides this is *your* money, is “shouldn’t be a problem” ok?

subesSee https://github.com/subes/invesdwin-util#decimal for an alternative to working with Double directly by using a fluent API around it that is very fast since being designed for financial strategy backtesting. It handles rounding for you *when actually needed*.

Chris FairhallFloating point in finance becomes a real problem when it comes to rounding.

In general the finance sector uses “half even” rounding.

If your calculation is supposed to end up as 36.295 and then rounded to 2DP it should go to 36.30.

If it actually ends up being 36.294999999999995, that rounds down to 36.29

John HaugelandAre … you retarded?

The *correct* answer is to do integer math on cents. Unambiguous, correct, primitive, fast, and no need to switch languages.

Why are Java people unable to cope with basic software topics?

hhhbecause they’re busy applying their design patterns.

Wouter LievensWhat if you need sub-cent accuracy? You could just move the imaginary decimal point, sure, but what if you don’t know the location of said point?

GavinWow, such ignorance….

Were you writing a calculator to use at the till in your toy shop?

Cent accuracy is far too coarse in finance, what are you going to do, just round to the nearest cent every time you need to find a fraction of a value?

Also, who are Java people? Java is just a language, stop with all the high school fan boy crap.

MartinPost authorInsulting, smug and wrong in one post – surely this is the ultimate internet post trifecta!

Integers are performant but all well and good until your precision changes and you realise you’ve coded yourself into a corner. And as Gavin mentioned, who says that cents are good enough? Or better yet if you’re dealing with massive numbers your chosen solution causes an overflow.

The reality is that you choose the right solution to the problem. If you can live with the imprecision, use a double. If you need specified precision (and don’t want to roll your own rounding), use something like a BigDecimal or create your own data object that can handle both sides of the decimal point independently. What you don’t do is present a solution as simplistic as yours and then get to claim that everyone else is an idiot.

EdSure, that works great if you are adding and subtracting monetary amounts. But that’s Accounting. We’re talking Finance here. What happens when interest rates come into the picture?

Consider:

Which outputs:

There’s a lot of disparity there, no matter how you choose to convert Decimals to “Dollars and Cents”.

EdEven in blockquote, that code looks hard to read. Try here:

https://gist.github.com/1042512

ChrisWhy not just impose a furthest place value — say, thousandths — and use ints (or longs if you’re worried about overflow)?

int t1 = 10266;

int t2 = 10000d;

//Outputs 266

System.out.println(t1-t2);

int h1 = 100266d;

int h2 = 100000d;

`//Outputs 266`

System.out.println(h1-h2);

Just make sure to divide by 1000 before displaying the results to the user.

RichEeek… nice comment until the division bit. The dollars and cents thing is a presentational issue, not an arithmetic one.

jmalcolmScala is pretty cool but this is not the best advertisement for it. After all, the following has worked in C# since version 1.0 (released in 2002)

using System;

class MainClass

{

public static void Main (string[] args)

{

var t1 = 10.266m;

var t2 = 10.0m;

// Outputs 0.266

Console.WriteLine(t1-t2);

var h1 = 100.266m;

var h2 = 100.0m;

`// Outputs 0.266`

Console.WriteLine(h1-h2);

}

}

Actually, I guess you would need to replace

`var`

with explicit`decimal`

back in the 1.0 days.I am not trashing on Scala though. As I said, Scala is pretty cool.

MartinPost authorAs I understand it, the m datatype is just an alias for System.Decimal so it actually amounts to the same thing 🙂

All I was trying to do was to have a little bit of fun with Scala to highlight that the implicit typing is a far better solution than Java’s, nothing more.

GorokonThis is not a big deal. Any developer unaware of the problems with floating point arithmetic is not worth feeding. Fixed point integers are good and they are faster too.

dvh@martin: this discussion raises the question: who interviewed you??

MartinPost authorDoes it? Why do you say that?

Derek D.Integers are your friends. They’ll never leave you for more exciting values. They’ll stay the same forever (unless you spend too much time adding to them, that is).

<3

Also, dear God, floats belong in the graphics department, keep them out of anything serious.

They're floaty like a viscous liquid, can't be trusted.

francisyou can also use Decimal in C#

so money variables would be declared like this; 2.50m

or you can just create a fixed point number class that will allow one to change the precision

AprogrammerIf you are doing simple retail like calcs it makes sense. But try using big decimals is an iterative calculation thousands of times (say like a Monte Carlo simulation) in a HFT business. You’ll be killed on latency and resources as you churn a truck load more objects causing high levels of gc whilst the Market moves is against you. Using doubles has it’s place, primitive longs better if you keep hold of the decimal and calculate that at the end.

If that was your criteria for hiring people then I despair……

MartinPost authorYou’re right about each approach, doubles/longs/BigDecimals, having their place. What I meant to highlight, and that particular point of the hiring process, was that each approach has its shortfall. As long as you are aware of the failings of each approach then that is all that matters but the candidates routinely don’t have this awareness.

Like many rules, it is lies-to-children. When you can appreciate the complexity of reality you are then able to break it for the right reason.

taraThis site and IEEE 754 pretty much cover all the financial number needs you may have.

http://speleotrove.com/decimal/

PatriceMartin,

This is a good post, you’re taking a lot of unjustified heat here IMO.

It’s a simple, general rule, which like some many others says “unless you know for a fact that you have a better answer, you should always use [BigDecimal] for [currency arithmetics] rather than double/long”.

A simple rule that keeps you out-of-trouble 99% of the time. It is intellectually honest, unlike some of the (expected) comments.

Yes there are other ways to do it, if you really need to (e.g. raw performance) and if you absolutely know what you’re doing, but you never claimed otherwise.

Your post made the programming masses a bit better, and it’s well worth a thank you. Keep ’em coming.

MartinPost authorThanks for the kind words Patrice 🙂

Quite a few people seemed to have missed the fact that I never said that doubles should never be used. I say that they are “not a great thing to do” but each data type has their own strengths and weaknesses and should be used when appropriate. The trick is, as you mention, having sufficient understanding of the complexities to know when to break the rule. My fault for a contentious title I guess!

AraThe problem with using BigDecimal is the overhead in performance. An arithmetic operation on a double (in most CPUs) is typically a one or two-instruction process. An arithmetic operation on a BigDecimal involves many many more instructions (check out the source for subtract() at http://www.docjar.com/html/api/java/math/BigDecimal.java.html).

By carefully controlling the rounding of doubles, it is possible to get accurate financial results without incurring the performance overhead. I know this for a fact because, 20 years ago, I used to write FORTRAN programs for scientific calculations that used FLOAT and DOUBLE PRECISION data types and produced accurate (within epsilon) results. BigDecimal may still be necessary, but only when adding really large numbers.

Keith ThompsonIf all you’re doing is adding and subtracting amounts of money, scaled integers (where 1 represents $0.01 and 100 represents $1.00) are good enough.

But as Ed points out, as soon as you start dealing with interest calculations, you have to deal with fractional cents, and you have to get the answers right.

As I understand it, there are regulations that specify exactly how these calculations must be done, with exact rules for rounding vs. truncating.

You shouldn’t even begin to write code that deals with interest calculations until you understand these regulations. Don’t assume that money can be expressed in whole numbers of cents, or even in real numbers of dollars. Don’t assume that the regulations match what your intuition tells you about how money *should* work. If your calculation yields a result that’s mathematically perfect, but it doesn’t match what the regulations require, your calculation is wrong.

(Disclaimer: I have little or no idea what these regulations actually say, or how to find them. It’s entirely possible that I’ve misunderstood the situation myself.)

florinI’d be curious to know what Quickbooks / Quicken uses.

JoePartical answer: go to a clearing house and ask them WHY they don’t want you to use float/double to perform their calculations. They don’t care about speed or efficiency, what they care about is accuracy. So fractional operations are performed on integer values and results are later scaled to obtain floating representation. When you’re shuffling millions for one account to another a few hundred times every night, a tiny rounding error is not so tiny.