My JavaScript book is out! Don't miss the opportunity to upgrade your beginner or average dev skills.

Sunday, October 04, 2009

[Q&D] Fix Decimal Module Operator

This is a Q&D solution for some JavaScript precision problem when we use the module operator with floating point numbers.

The Problem

If we are dealing with the module operator we could have this case:
alert(
8.25 % .05
);
// 0.049999999999999545

We expected 0 but we have a floating point inconsistency instead. This is a well known problem, and not only with JavaScript, truly annoying when we have to show run-time computed money/trends operations.

The Solution

This comes from my reply in a JS forum, knowing that the module will be a simply floating point, let's say .1 to .001, if we remove floating point from the right side of the operation everything works just fine:

function mod(num, mod){
// Another WebReflection silly idea
var pow = Math.pow(10, (("" + mod).split(".")[1] || "").length);
return ((num * pow) % (mod * pow)) / pow;
};

alert(
mod(8.25, .05)
); // 0


alert(
mod(8.25, .1)
); // 0.5

That's it, have a nice Sunday

6 comments:

  1. I think it would be better to just work in cents all the time instead of working in Euros/Dollars. That way all your operations are protected from floating point inaccuracy instead of just the % operator.

    ReplyDelete
  2. this is a quick and dirty fix, not a best practice suggestion or a universal solution ... if people had this problem, they can fix it, that's the purpose of a Q&D code :)

    ReplyDelete
  3. thank you for the q&d, there's a typo in the code comments. // 0.5 should be // 0.05 :)

    ReplyDelete
  4. I appreciate the Q&D suggestion, but Erik is right. Use cents, not whole currency.

    Also, a spelling nit-pick :) The operator is called modulo.

    ReplyDelete
  5. guys, this is a solution for a forum question, that's it

    ReplyDelete
  6. Please keep posting the Q&D's. Those of use who don't work in JS every day forget the little tricks like ("" + castMeToString)

    ReplyDelete

Note: Only a member of this blog may post a comment.