Hello and welcome to our community! Is this your first visit?
Enjoy an ad free experience by logging in. Not a member yet? Register.

# Thread: Problem in arithemetic conversion

1. ## Problem in arithemetic conversion

Hi,

I have problem in basic arithemetic which makes me hectic in major issues.

i have to compare two numbers "a" and "b". And the values is as follows:
a = 18446744073709552000
b = 1.844674407371E+19

Both the values are equal.

But if i compare in javascript as if(a < b), sometimes it works well. But often there occurs problem since the value of "b" is in E power value. itz comparing as 1.844674407371 leaving E+19.

How to fix this?

Regards,
Viji.

• A Google search brough me to:-

http://www.irt.org/articles/js069/index.htm

One of the interesting features of JavaScript is, that the precision it uses to judge the accuracy of its calculation algorithms is noticeably lower than that of other programming languages. Despite it now being at revision 1.2, this standard has not yet been improved.

To give you an idea of the nature and depth of the problem, although for instance Authorware, Turbo Pascal, Excel and even Word6 will calculate the following correctly, multiplying:

0.119 by 100 in JavaScript evaluates to 11.899999.

0.14 by 100 in JavaScript evaluates to 14.0000000000002

Normally in JavaScript a value has 16 digits of precision, and this just seems to be the limitations inherent in the routines used.

Hope this helps.

• Actually, JavaScript has the same accuracy and the same mechanics as most other programming languages using standard double precision floating point numbers. What JavaScript lacks, however, is a functionality that hides accumulations of errors such as the one VBScript has.

See Eric Lippert's blog entry Why does JScript have rounding errors? for more on this.

For more on understanding why this is so, have a look at:
Floating Point Arithmetic, Part One
Benford's Law
Floating Point And Benford's Law, Part Two
Fun With Floating Point Arithmetic, Part Three
Fun with Floating Point Arithmetic, Part Four
Fun with Floating Point Arithmetic, Part Five
Fun With Floating Point Arithmetic, Part Six

•

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•