Why we don’t use Doubles for Financial Calculations

32 Replies

My current client is in the middle of hiring some Java developers and as I mentioned earlier in March (Interviewing – the importance of PASSION!) I’ve been doing some of the interviewing. One of the things we’ve done is to create a technical task to see how the candidates actually code. It’s a simple exercise that requires them to think through some of the basics of financial operations and one thing that has surprised me has been the common use of doubles to represent financial values. It’s been highlighted for some time that this is not a great thing to do but someone actually challenged me to show that it wasn’t.

secured payday loans

So here we go…

package com.bloodredsun;

public class DoubleOperation {

    public static void main(String[] args) {

        double t1 = 10.266d;
        double t2 = 10.0d;

        //Outputs 0.266
        System.out.println(t1-t2);

        double h1 = 100.266d;
        double h2 = 100.0d;

        //Outputs 0.26600000000000534
        System.out.println(h1-h2);
    }
}

Ouch! That is not what we want but it is the classic behaviour of doubles. The inability to represent some decimals in the IEEE-754 format (as binary fractions) causes this. If we want correct precision the answer is to use BigDecimals but we have to remember to use Strings in the constructors or you end up with the same issues that you were trying to avoid.

package com.bloodredsun;

import java.math.BigDecimal;

public class BigDecimalOperation {

        public static void main(String[] args) {

        BigDecimal t1 = new BigDecimal("10.266");
        BigDecimal t2 = new BigDecimal("10.0");

        //Outputs 0.266
        System.out.println(t1.subtract(t2));

        BigDecimal h1 = new BigDecimal("100.266");
        BigDecimal h2 = new BigDecimal("100.0");

        //Outputs 0.266
        System.out.println(h1.subtract(h2));
    }
}

That’s great but wouldn’t it be nice to use the normal operators rather than the overly-verbose method calls for the mathematical operations.

Now there is no way that we can do this in Java but if we let ourselves use another language on the JVM…

package com.bloodredsun

object ScalaBigDecimalOperation {

  def main (args: Array[String]) {
    var t1 = BigDecimal("10.266")
    var t2 = BigDecimal("10.0")
    //Outputs 0.266
    println(t1 - t2)

    var h1 = BigDecimal("100.266")
    var h2 = BigDecimal("100.0")
    //Outputs 0.266
    println(h1 - h2)
  }
}

Scala FTW!

PS if you want to know more about floating point operations have a read of What Every Computer Scientist Should Know About Floating-Point Arithmetic

32 thoughts on “Why we don’t use Doubles for Financial Calculations”

  1. Ed

    Are you sure doubles are so bad in finance?? Really?
    Each time I do alpha + beta I get a value signma + an error term epsilon. Provided epsilon is sufficiently less than the precision, you’re ok because you can just round the value to the appropriate accuracy (less than half epsilon I guess)

    What’s the performance difference between big decimal and the scala version?

    Reply ↓
    1. Martin

      As you say, providing epislon is small enough it’s not a problem and in many cases we can live with it. The trouble arises when you have billions of calculations that sum up to something significant or when you are taking leveraged positions that can massively exacerbate any imprecision.

      As for the performance of the Java and Scala version, they are identical since they both compile down to byte code. The difference is purely a syntactical one.

      Reply ↓
        1. Martin Post author

          My point was that despite all the focus given to this topic, there are still developers who do not know about the imprecision of floating point values.

          Like most rules, knowing this gives you the freedom to break it when you are able to, such as when the error is acceptably small and the performance benefit is a great enough advantage.

          Reply ↓
    2. stratton

      Hey Ed,

      The problem with doubles as he shows is you can’t represent the numbers exactly. Imagine adding a whole bunch of numbers that should have added up to an exact value. It’s hard to verify your books if cents are gone here and there. When working with money, you need your numbers to be exact.

      “That’s great but wouldn’t it be nice to use the normal operators rather than the overly-verbose method calls for the mathematical operations.”

      As the author noted, with Scala BigDecimal has its operators overloaded so you can use normal + and – operators, while with java because BigDecimal isn’t a primitive, you need to use “.subtract()” and such.

      Reply ↓
    3. Rich

      Doubles are bad because they do not mean what naive programmers expect them to mean. Though in this case I think a workable solution is to pick a suitable precision and just use longs.

      Reply ↓
    4. Tristan

      In the 238k range a float will(may) be off by a penny. This may not be an issue for day to day banking, Depending on the value of the whole part the precision of the fractional goes down. It turns out that at about 238000 the precision for the fractional goes down to ~0.005 which will impact financial transactions.

      besides this is *your* money, is “shouldn’t be a problem” ok?

      Reply ↓
  2. John Haugeland

    Are … you retarded?

    The *correct* answer is to do integer math on cents. Unambiguous, correct, primitive, fast, and no need to switch languages.

    Why are Java people unable to cope with basic software topics?

    Reply ↓
    2. Wouter Lievens

      What if you need sub-cent accuracy? You could just move the imaginary decimal point, sure, but what if you don’t know the location of said point?

      Reply ↓
    3. Gavin

      Wow, such ignorance….

      Were you writing a calculator to use at the till in your toy shop?

      Cent accuracy is far too coarse in finance, what are you going to do, just round to the nearest cent every time you need to find a fraction of a value?

      Also, who are Java people? Java is just a language, stop with all the high school fan boy crap.

      Reply ↓
    4. Martin Post author

      Insulting, smug and wrong in one post – surely this is the ultimate internet post trifecta!

      Integers are performant but all well and good until your precision changes and you realise you’ve coded yourself into a corner. And as Gavin mentioned, who says that cents are good enough? Or better yet if you’re dealing with massive numbers your chosen solution causes an overflow.

      The reality is that you choose the right solution to the problem. If you can live with the imprecision, use a double. If you need specified precision (and don’t want to roll your own rounding), use something like a BigDecimal or create your own data object that can handle both sides of the decimal point independently. What you don’t do is present a solution as simplistic as yours and then get to claim that everyone else is an idiot.

      Reply ↓
    5. Ed

      Sure, that works great if you are adding and subtracting monetary amounts. But that’s Accounting. We’re talking Finance here. What happens when interest rates come into the picture?

      Consider:

      import java.math.BigDecimal;
      import java.math.RoundingMode;

      public class Main{
      private static Transformer floor = new BigDecToDiscreteViaFloor();
      private static Transformer ceil = new BigDecToDiscreteViaCeil();
      private static Transformer round = new BigDecToDiscreteViaRound();
      private static Transformer i = new BigDecIdentity();

      public static void main(String[] args){

      BigDecimal initialInvestment = new BigDecimal(“1000000″);
      BigDecimal annualRate = new BigDecimal(“0.07125″);
      int compoundingPeriodsPerYear = 4;
      int years = 10;

      BigDecimal test = new BigDecimal(“10.12345″);
      BigDecimal testA = new BigDecimal(“10.105″);
      BigDecimal testB = new BigDecimal(“10.115″);

      System.out.println(“To get some confidence in our Transformers:”);
      System.out.println(test + ” via Floor: ” + floor.transform(test));
      System.out.println(test + ” via Ceil: ” + ceil.transform(test));
      System.out.println(test + ” via Round: ” + round.transform(test));
      System.out.println(testA + ” via Round: ” + round.transform(testA));
      System.out.println(testB + ” via Round: ” + round.transform(testB));
      System.out.println(test + ” via Identity: ” + i.transform(test));

      System.out.println();

      BigDecimal correct = futureValue(initialInvestment, annualRate, compoundingPeriodsPerYear, years, i);
      BigDecimal viaFloor = futureValue(initialInvestment, annualRate, compoundingPeriodsPerYear, years, floor);
      BigDecimal viaCeil = futureValue(initialInvestment, annualRate, compoundingPeriodsPerYear, years, ceil);
      BigDecimal viaRound = futureValue(initialInvestment, annualRate, compoundingPeriodsPerYear, years, round);

      System.out.println( “What is the future value of $” + initialInvestment +
      ” invested at an annual rate of ” + annualRate +
      ” compounded ” + compoundingPeriodsPerYear + ” time(s) per year for ” +
      years + ” years?\n”);

      //Consider $1M invested for 10 years at 7.125 % interest,
      //compounded quarterly calculated with decimals
      System.out.println( “‘Correct’ value (to two decimal places):” + round.transform(correct) + “\n”);

      //Consider the same using integers (where integers are obtained
      //via the floor function on decimals)
      System.out.println(“Using floor (to two decimal places):” + round.transform(viaFloor) + “\n”);

      //Consider the same using integers (where integers are obtained
      //via the ceiling function on decimals)
      System.out.println(“Using ceiling (to two decimal places):” + round.transform(viaCeil) + “\n”);

      //Consider the same using integers (where integers are obtained
      //via the rounding)
      System.out.println(“Using round (to two decimal places):” + round.transform(viaRound) + “\n”);

      }

      public static BigDecimal futureValue( BigDecimal initialInvestment,
      BigDecimal annualRate,
      int compoundingPeriodsPerYear,
      int years,
      Transformer t) {

      return t.transform(initialInvestment).
      multiply(
      BigDecimal.ONE.
      add(
      t.transform(annualRate).
      divide(
      new BigDecimal(compoundingPeriodsPerYear)
      )
      ).pow( (compoundingPeriodsPerYear * years ) )
      );
      }

      private static interface Transformer {
      public T transform(T t);
      }

      private static class BigDecToDiscreteViaFloor implements Transformer {
      public BigDecimal transform(BigDecimal d) {
      return d.divide(BigDecimal.ONE, 2, RoundingMode.FLOOR);
      }
      }

      private static class BigDecToDiscreteViaCeil implements Transformer {
      public BigDecimal transform(BigDecimal d) {
      return d.divide(BigDecimal.ONE, 2, RoundingMode.CEILING);
      }
      }

      private static class BigDecToDiscreteViaRound implements Transformer {
      public BigDecimal transform(BigDecimal d) {
      return d.divide(BigDecimal.ONE, 2, RoundingMode.HALF_EVEN);
      }
      }

      private static class BigDecIdentity implements Transformer {
      public BigDecimal transform(BigDecimal d) {
      return d;
      }
      }
      }

      Which outputs:

      To get some confidence in our Transformers:
      10.12345 via Floor: 10.12
      10.12345 via Ceil: 10.13
      10.12345 via Round: 10.12
      10.105 via Round: 10.10
      10.115 via Round: 10.12
      10.12345 via Identity: 10.12345

      What is the future value of $1000000 invested at an annual rate of 0.07125 compounded 4 time(s) per year for 10 years?

      ‘Correct’ value (to two decimal places):2026334.83

      Using floor (to two decimal places):2001597.34

      Using ceiling (to two decimal places):2208039.66

      Using round (to two decimal places):2001597.34

      There’s a lot of disparity there, no matter how you choose to convert Decimals to “Dollars and Cents”.

      Reply ↓
  3. Chris

    Why not just impose a furthest place value — say, thousandths — and use ints (or longs if you’re worried about overflow)?


    int t1 = 10266;
    int t2 = 10000d;

    //Outputs 266
    System.out.println(t1-t2);

    int h1 = 100266d;
    int h2 = 100000d;

    //Outputs 266
    System.out.println(h1-h2);

    Just make sure to divide by 1000 before displaying the results to the user.

    Reply ↓
    1. Rich

      Eeek… nice comment until the division bit. The dollars and cents thing is a presentational issue, not an arithmetic one.

      Reply ↓
  4. jmalcolm

    Scala is pretty cool but this is not the best advertisement for it. After all, the following has worked in C# since version 1.0 (released in 2002)


    using System;

    class MainClass
    {
    public static void Main (string[] args)
    {
    var t1 = 10.266m;
    var t2 = 10.0m;

    // Outputs 0.266
    Console.WriteLine(t1-t2);

    var h1 = 100.266m;
    var h2 = 100.0m;

    // Outputs 0.266
    Console.WriteLine(h1-h2);
    }
    }

    Actually, I guess you would need to replace var with explicit decimal back in the 1.0 days.

    I am not trashing on Scala though. As I said, Scala is pretty cool.

    Reply ↓
    1. Martin Post author

      As I understand it, the m datatype is just an alias for System.Decimal so it actually amounts to the same thing :-)

      All I was trying to do was to have a little bit of fun with Scala to highlight that the implicit typing is a far better solution than Java’s, nothing more.

      Reply ↓
  5. Gorokon

    This is not a big deal. Any developer unaware of the problems with floating point arithmetic is not worth feeding. Fixed point integers are good and they are faster too.

    Reply ↓
  7. Derek D.

    Integers are your friends. They’ll never leave you for more exciting values. They’ll stay the same forever (unless you spend too much time adding to them, that is).

    <3

    Also, dear God, floats belong in the graphics department, keep them out of anything serious.

    They're floaty like a viscous liquid, can't be trusted.

    Reply ↓
  8. francis

    you can also use Decimal in C#
    so money variables would be declared like this; 2.50m

    or you can just create a fixed point number class that will allow one to change the precision

    Reply ↓
  9. Aprogrammer

    If you are doing simple retail like calcs it makes sense. But try using big decimals is an iterative calculation thousands of times (say like a Monte Carlo simulation) in a HFT business. You’ll be killed on latency and resources as you churn a truck load more objects causing high levels of gc whilst the Market moves is against you. Using doubles has it’s place, primitive longs better if you keep hold of the decimal and calculate that at the end.

    If that was your criteria for hiring people then I despair……

    Reply ↓
    1. Martin Post author

      You’re right about each approach, doubles/longs/BigDecimals, having their place. What I meant to highlight, and that particular point of the hiring process, was that each approach has its shortfall. As long as you are aware of the failings of each approach then that is all that matters but the candidates routinely don’t have this awareness.

      Like many rules, it is lies-to-children. When you can appreciate the complexity of reality you are then able to break it for the right reason.

      Reply ↓
  11. Patrice

    Martin,

    This is a good post, you’re taking a lot of unjustified heat here IMO.
    It’s a simple, general rule, which like some many others says “unless you know for a fact that you have a better answer, you should always use [BigDecimal] for [currency arithmetics] rather than double/long”.

    A simple rule that keeps you out-of-trouble 99% of the time. It is intellectually honest, unlike some of the (expected) comments.
    Yes there are other ways to do it, if you really need to (e.g. raw performance) and if you absolutely know what you’re doing, but you never claimed otherwise.

    Your post made the programming masses a bit better, and it’s well worth a thank you. Keep ‘em coming.

    Reply ↓
    1. Martin Post author

      Thanks for the kind words Patrice :-)

      Quite a few people seemed to have missed the fact that I never said that doubles should never be used. I say that they are “not a great thing to do” but each data type has their own strengths and weaknesses and should be used when appropriate. The trick is, as you mention, having sufficient understanding of the complexities to know when to break the rule. My fault for a contentious title I guess!

      Reply ↓
  12. Ara

    The problem with using BigDecimal is the overhead in performance. An arithmetic operation on a double (in most CPUs) is typically a one or two-instruction process. An arithmetic operation on a BigDecimal involves many many more instructions (check out the source for subtract() at http://www.docjar.com/html/api/java/math/BigDecimal.java.html).

    By carefully controlling the rounding of doubles, it is possible to get accurate financial results without incurring the performance overhead. I know this for a fact because, 20 years ago, I used to write FORTRAN programs for scientific calculations that used FLOAT and DOUBLE PRECISION data types and produced accurate (within epsilon) results. BigDecimal may still be necessary, but only when adding really large numbers.

    Reply ↓
  13. Keith Thompson

    If all you’re doing is adding and subtracting amounts of money, scaled integers (where 1 represents $0.01 and 100 represents $1.00) are good enough.

    But as Ed points out, as soon as you start dealing with interest calculations, you have to deal with fractional cents, and you have to get the answers right.

    As I understand it, there are regulations that specify exactly how these calculations must be done, with exact rules for rounding vs. truncating.

    You shouldn’t even begin to write code that deals with interest calculations until you understand these regulations. Don’t assume that money can be expressed in whole numbers of cents, or even in real numbers of dollars. Don’t assume that the regulations match what your intuition tells you about how money *should* work. If your calculation yields a result that’s mathematically perfect, but it doesn’t match what the regulations require, your calculation is wrong.

    (Disclaimer: I have little or no idea what these regulations actually say, or how to find them. It’s entirely possible that I’ve misunderstood the situation myself.)

    Reply ↓
  15. Joe

    Partical answer: go to a clearing house and ask them WHY they don’t want you to use float/double to perform their calculations. They don’t care about speed or efficiency, what they care about is accuracy. So fractional operations are performed on integer values and results are later scaled to obtain floating representation. When you’re shuffling millions for one account to another a few hundred times every night, a tiny rounding error is not so tiny.

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>