Understanding Windows Calculator Internal Variable Types

Asked 2 years ago, Updated 2 years ago, 35 views

As for the Windows calculator, it has both the characteristics of the double type and the decimal type.
Specifically,
If you add A.0.1+0.1+0.1..., the Double type will have an error and will no longer be 0.3, and the Decimal will be 0.3.
However, it will be 0.3 or 0.9 on the calculator.

If you calculate B.1/3*3 in order, it will be 1.0 for Double type and 0.9999999999999 for Decimal.
However, on the calculator, it will be exactly 1.In the middle, it was 0.333333333333333 as of 1/3 of the time.

In this way, calculators are very normal calculations, but they look good in the language type.
What kind of treatment are you doing?

windows

2022-09-30 20:18

2 Answers

According to When you change the insides, nobody notes | The Old New Thing, originally compliant with IEEE standard floating-point operations was rewritten to allow arbitrary precision operations only in four operations.The exact timing is not written, but I was able to confirm the behavior of 98SE at hand.

At that time, there seemed to be no real limit to the value range, and to what extent n! could be calculated?

What's the highest number you can put in the Windows calculator-AnandTech Forums

"However, as you can see on the last page of the above thread, Win7 says ""overflowed"" when it exceeds around 10^999."

What is the largest number that can be used in window 7 calculator-Microsoft Community


2022-09-30 20:18

For Microsoft Excel, 1/3*3 equals 1.The behavior in Excel is described in Excel may have incorrect floating point results.Perhaps the implementation is similar.


2022-09-30 20:18

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.