[Python-3000-checkins] r65627 - python/branches/py3k/Doc/tutorial/floatingpoint.rst

[georg.brandl]

Author: georg.brandl
Date: Sun Aug 10 14:16:45 2008
New Revision: 65627

Log:
Remove long integer output.


Modified:
python/branches/py3k/Doc/tutorial/floatingpoint.rst

Modified: python/branches/py3k/Doc/tutorial/floatingpoint.rst
==============================================================================
--- python/branches/py3k/Doc/tutorial/floatingpoint.rst (original)
+++ python/branches/py3k/Doc/tutorial/floatingpoint.rst Sun Aug 10 14:16:45 2008
@@ -173,24 +173,24 @@

2**52

- 4503599627370496L
+ 4503599627370496

2**53

- 9007199254740992L
+ 9007199254740992

2**56/10

- 7205759403792793L
+ 7205759403792794.0

That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The

q, r = divmod(2**56, 10)
r

- 6L
+ 6

Since the remainder is more than half of 10, the best approximation is obtained

q+1

- 7205759403792794L
+ 7205759403792794

Therefore the best possible approximation to 1/10 in 754 double precision is
that over 2\*\*56, or ::
@@ -211,7 +211,7 @@

7205759403792794 * 10**30 / 2**56

- 100000000000000005551115123125L
+ 100000000000000005551115123125

meaning that the exact number stored in the computer is approximately equal to
the decimal value 0.100000000000000005551115123125. Rounding that to 17