Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 6 of 6
  1. #1
    New to the CF scene
    Join Date
    Sep 2005
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Unexpected result from addition

    I have what I thought was a simple while loop:

    while (i <= 1 && returnValue == 1) {
    alert("the value of i is " + parseFloat(i));
    if (testNum == i) {
    returnValue = 0; }
    i = i + .01;
    }

    i was set to .01 before this loop. testNum is supposed to be a number between .01 and 1, but that is not my problem. The alert was added for debugging purposes, and I've tried it with and without the parseFloat. It displays .01 through .05 OK, but instead of .06 it gives me .060000000000000005.

    I've tried this on IE, Firefox, and Netscape, on two different machines (on Win 2K and XP Professional). I can't believe it actually works this way, so it must be something I'm doing.

  • #2
    Regular Coder
    Join Date
    Aug 2005
    Location
    Toronto, ON, Canada
    Posts
    231
    Thanks
    0
    Thanked 0 Times in 0 Posts
    results are ok, as per http://www.ecma-international.org/pu...T/Ecma-262.pdf

    The Number Type

    The Number type has exactly 18437736874454810627 (that is, 264−253+3) values, representing the doubleprecision
    64-bit format IEEE 754 values as specified in the IEEE Standard for Binary Floating-Point
    Arithmetic, except that the 9007199254740990 (that is, 253−2) distinct “Not-a-Number” values of the IEEE
    Standard are represented in ECMAScript as a single special NaN value. (Note that the NaN value is
    produced by the program expression NaN, assuming that the globally defined variable NaN has not been
    altered by program execution.) In some implementations, external code might be able to detect a difference
    between various Non-a-Number values, but such behaviour is implementation-dependent; to ECMAScript
    code, all NaN values are indistinguishable from each other.
    There are two other special values, called positive Infinity and negative Infinity. For brevity, these values
    are also referred to for expository purposes by the symbols +∞ and −∞, respectively. (Note that these two
    infinite number values are produced by the program expressions +Infinity (or simply Infinity) and
    -Infinity, assuming that the globally defined variable Infinity has not been altered by program
    execution.)
    The other 18437736874454810624 (that is, 264−253) values are called the finite numbers. Half of these are
    positive numbers and half are negative numbers; for every finite positive number there is a corresponding
    negative number having the same magnitude.
    Note that there is both a positive zero and a negative zero. For brevity, these values are also referred to for
    expository purposes by the symbols +0 and −0, respectively. (Note that these two zero number values are
    produced by the program expressions +0 (or simply 0) and -0.)
    The 18437736874454810622 (that is, 264−253−2) finite nonzero values are of two kinds:
    18428729675200069632 (that is, 264−254) of them are normalised, having the form
    s × m × 2e
    where s is +1 or −1, m is a positive integer less than 253 but not less than 252, and e is an integer ranging
    from −1074 to 971, inclusive.
    The remaining 9007199254740990 (that is, 253−2) values are denormalised, having the form
    s × m × 2e
    where s is +1 or −1, m is a positive integer less than 252, and e is −1074.
    Note that all the positive and negative integers whose magnitude is no greater than 253 are representable in
    the Number type (indeed, the integer 0 has two representations, +0 and -0).
    A finite number has an odd significand if it is nonzero and the integer m used to express it (in one of the
    two forms shown above) is odd. Otherwise, it has an even significand.
    In this specification, the phrase “the number value for x” where x represents an exact nonzero real
    mathematical quantity (which might even be an irrational number such as π ) means a number value chosen
    in the following manner. Consider the set of all finite values of the Number type, with −0 removed and
    with two additional values added to it that are not representable in the Number type, namely 21024 (which is
    +1 × 253 × 2971) and −21024 (which is −1 × 253 × 2971). Choose the member of this set that is closest in value
    to x. If two values of the set are equally close, then the one with an even significand is chosen; for this
    purpose, the two extra values 21024 and −21024 are considered to have even significands. Finally, if 21024 was
    chosen, replace it with +∞; if −21024 was chosen, replace it with −∞; if +0 was chosen, replace it with −0 if
    and only if x is less than zero; any other chosen value is used unchanged. The result is the number value for
    x. (This procedure corresponds exactly to the behaviour of the IEEE 754 “round to nearest” mode.)
    Some ECMAScript operators deal only with integers in the range −231 through 231−1, inclusive, or in the
    range 0 through 232−1, inclusive. These operators accept any value of the Number type but first convert
    each such value to one of 232 integer values. See the descriptions of the ToInt32 and ToUint32 operators in
    sections 0 and 0, respectively.
    Last edited by rm-f; 09-24-2005 at 05:01 AM.
    rm -f /

  • #3
    New Coder
    Join Date
    Apr 2005
    Location
    Burnaby, BC, Canada
    Posts
    68
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I have come across this problem before also. I was using integers however, so the same solution won't work here.

    I'll play around with it a bit, and see what I can come up with.
    'Programming is the only artform that fights back!'

  • #4
    New Coder
    Join Date
    Apr 2005
    Location
    Burnaby, BC, Canada
    Posts
    68
    Thanks
    0
    Thanked 0 Times in 0 Posts
    This worked for me:

    Code:
    function round(num, digits)
    {
    	num1 = Math.pow(10,digits);
    	return Math.round(num*num1)/num1;
    }
    
    
    i=0.1;
    testNum = 0.17;
    returnValue = 1;
    
    while (i <= 1 && returnValue == 1) 
    {
    	alert("the value of i is " + i, 3);
    	
    	if (testNum <= i) 
    	{
    		returnValue = 0; 
    	}
    
    	i = round(i + 0.01, 3);
    }
    'Programming is the only artform that fights back!'

  • #5
    Regular Coder
    Join Date
    Aug 2005
    Location
    Toronto, ON, Canada
    Posts
    231
    Thanks
    0
    Thanked 0 Times in 0 Posts
    rm -f /

  • #6
    Master Coder
    Join Date
    Feb 2003
    Location
    Umeå, Sweden
    Posts
    5,575
    Thanks
    0
    Thanked 83 Times in 74 Posts
    Quote Originally Posted by Bruce.Burnell
    I can't believe it actually works this way, so it must be something I'm doing.
    Nope. It's just computers being computers. To a computer everything is binary, meaning that if you can't represent the number exactly in base 2, you have to approximate. And sometimes with approximations, you get holdovers from when a number doesn't really fit. Take 0.1 for instance. 0.1 can't be described by the computer as 1/2, nor as 1/4, nor as 1/8. Those are all larger than 0.1. But 1/16 is smaller than 0.1, so the number has to be 1/16 + something. And something is smaller than 1/16, so it'll try 1/32. Okay, 1/16 + 1/32 is not quite enough. It has to add further....

    As you can see, if a number can't be represented exactly in base 2, it will have to be a series of added together exactly represented numbers. But just like 1/3 can't be represented in base 10 exactly, 1/10 can't be exactly represented in base 2. And since the computer can't store an infinite number of progressively smaller numbers, it'll have to cut it off somewhere. And the problem is when one number with a certain cutoff point is added to another number with a different cutoff point, some fraction of the number can end up above the rounding point.
    liorean <[lio@wg]>
    Articles: RegEx evolt wsabstract , Named Arguments
    Useful Threads: JavaScript Docs & Refs, FAQ - HTML & CSS Docs, FAQ - XML Doc & Refs
    Moz: JavaScript DOM Interfaces MSDN: JScript DHTML KDE: KJS KHTML Opera: Standards


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •