5.4.22
Ek computer real numbers ko finite precision ke saath store karta hai. Doubles mein lagbhag 15–16 significant decimal digits hote hain (53 bits of mantissa). Zyaadatar time yeh kaafi hota hai. Lekin do cheezein hamari school-math ki instincts ko tod deti hain:
Catastrophic cancellation : do lagbhag-equal numbers ko subtract karna un mein chupi hui tiny relative error ko amplify kar deta hai.Associativity failure : ( a + b ) + c = a + ( b + c ) + round karta hai.
WHY care? Ek formula mathematically perfect ho sakta hai phir bhi numerically garbage ho sakta hai. Yeh jaanna ki kahan precision khatam hoti hai, aapko formula rewrite karne mein madad karta hai taaki usse preserve kiya ja sake.
Definition IEEE-754 doubleEk double ek number ko is tarah represent karta hai:
x = ± ( 1. m 1 m 2 … m 52 ) 2 × 2 e 1.0 machine epsilon kehlaata hai:
ε mach = 2 − 52 ≈ 2.22 × 1 0 − 16 .
WHY 2 − 52 Implicit leading 1 ke baad 52 fraction bits hone ke saath, exponent 0 2 − 52 1.0
Key word hai relative : error number ki magnitude ke saath scale hoti hai, absolutely nahi.
Intuition Subtraction villain kyun haiHar stored number ek relative error ∼ u absolute error hai jo number ki size ke proportional hai. Jab aap do bade, lagbhag-equal numbers subtract karte ho, toh answer tiny hota hai lekin absolute errors bade rehte hain — isliye result ki relative error explode kar jaati hai.
Worked example Classic: quadratic formulaSolve karo x 2 − 200000 x + 1 = 0 x = 2 a − b ± b 2 − 4 a c a = 1 , b = − 200000 , c = 1 b 2 − 4 a c = 40000000000 − 4 ≈ 199999.99999 x + = 2 200000 + 199999.99999 ≈ 199999.999995 fine (similar-sign numbers add ho rahe hain).
"−" root: x − = 2 200000 − 199999.99999 catastrophic cancellation! Do ~200000 numbers subtract hokar ~0.00001 ban jaate hain.
Fix (WHY it works): identity x + x − = c / a stable root compute karo, phir
x − = a x + c . ∼ u Why this step? Multiplication/division kabhi relative error amplify nahi karte; hum dangerous subtraction ke around route karte hain.
f ( x ) = x 2 1 − cos x x = 0 = 2 1 x = 1 0 − 8 cos x ≈ 1 − 5 × 1 0 − 17 cos x 1 1 − cos x = 0 0 0.5 Total loss.
Fix: 1 − cos x = 2 sin 2 ( x /2 ) f ( x ) = x 2 2 s i n 2 ( x /2 ) = 2 1 ( x /2 s i n ( x /2 ) ) 2 . Why this step? Humne near-equals ki subtraction ko ek sin
+ round karta hai, aur rounding ko brackets ki parwah nahi
Floating addition commutative hai (a + b = b + a associative nahi . Kyun? Kyunki fl ( a + b )
Worked example Ek concrete breakMaano a = 1.0 b = 1 0 16 c = − 1 0 16
( a + b ) + c 1 + 1 0 16 1 0 16 1 ∼ 2 + c = 0 0 .a + ( b + c ) b + c = 0 + a = 1 1 .
Same numbers, sirf brackets → answers 100% differ karte hain.
Why this step? Ek chhote number ko ek bahut bade number mein add karna representable gap se neeche chala jaata hai aur vanish ho jaata hai (absorption).
Definition Kahan summation (lost bits recover karne ki trick)s = 0.0; c = 0.0
for x in data:
y = x - c # add back last lost bit
t = s + y # may lose low bits of y
c = (t - s) - y # recover exactly what was lost
s = t
WHY it works: (t - s) woh part hai y ka jo actually add hua ; y subtract karne se woh negated part milta hai jo drop hua tha. Hum use next iteration mein carry karte hain.
Common mistake "Floats compare karne ke liye bas== use karo."
Why it feels right: math mein, equal computations equal numbers dete hain. Why it's wrong: 0.1 + 0.2 == 0.3 False hai kyunki 0.1 , 0.2 , 0.3 0.3 ∼ 1 0 − 17 Fix: tolerance ke saath compare karo, jaise abs(x-y) <= atol + rtol*max(|x|,|y|) (Python ka math.isclose).
Common mistake "Zyaada terms / higher precision hamesha accuracy fix kar deti hai."Why it feels right: zyaada kaam = zyaada accuracy, usually. Why it's wrong: agar formula cancel karta hai, toh terms add karna ya float128 use karna sirf catastrophe ko delay karta hai; expression ki conditioning asli problem hai.
Fix: reformulate karo (identities use karo jaise 1 − cos x = 2 sin 2 ( x /2 ) x + 1 − x = x + 1 + x 1 avoid karne ke liye.
Common mistake "Floats associative hain kyunki math mein addition associative hai."Why it feels right: real-number axioms. Why it's wrong: rounding per operation hoti hai, isliye grouping change karta hai ki kaun se bits survive karte hain. Upar wala example 0 vs 1 deta hai.
Fix: careful order mein sum karo ya Kahan use karo; kabhi mat assume karo ki reorderings free hain.
Recall Active recall — answers cover karo
a − b ∣ a − b ∣ ∣ a ∣ + ∣ b ∣ Kya near-equal numbers ki float subtraction khud inaccurate hoti hai? → Nahi, yeh exact hai; yeh sirf prior error ko expose karti hai.
( a + b ) + c = a + ( b + c ) + ek aise jagah round karta hai jo partial sum ki magnitude par depend karta hai.Chhote quadratic root ka fix? → x − = c / ( a x + )
Kahan summation kya track karta hai? → Har addition mein lost low-order bits.
Recall Feynman: 12-saal ke bachche ko explain karoSocho tumhara calculator sirf 4 digits dikha sakta hai. 12340000 1 12340000 1 edge se gir gaya . Ab agar do giant numbers lagbhag same hain, jaise 12340000 12340000 0 asli answer ek tiny number ho sakta tha jo pehle hi kha liya gaya. Isliye big-minus-big dangerous hai: answer chhota hai lekin galtiyan badi hain. Aur 1 + (huge) + ( − huge ) 0 1 order answer badal deti hai!
"Near-twins subtract karo, digits kho do. Sums reorder karo, answer kho do."
Quadratic ke liye: "b (Same-sign root compute karo, doosra c / ( a x )
What is machine epsilon for IEEE-754 double? 2 − 52 ≈ 2.22 × 1 0 − 16 1.0
State the floating rounding model. fl ( x ) = x ( 1 + δ ) ∣ δ ∣ ≤ u = 2 1 ε mach
What is catastrophic cancellation? Do nearly-equal numbers subtract karne par significant digits ka loss; chhoti absolute errors badi relative errors ban jaati hain.
Relative-error amplification factor for a − b ∣ a − b ∣ ∣ a ∣ + ∣ b ∣ a → b
Is the subtraction step itself the source of error? Nahi — near-equal operands ke liye yeh exact hai (Sterbenz); yeh sirf operands mein already present error ko reveal karta hai.
Stable way to get the small root of x 2 + b x + c Large-magnitude root compute karo, phir x 2 = c / ( a x 1 )
Stable form of 1 − cos x 2 sin 2 ( x /2 )
Stable form of x + 1 − x Why is float addition not associative? Har + ek aise jagah round karta hai jo running magnitude par depend karta hai; alag groupings alag tarah round karti hain.
Give values where ( a + b ) + c = a + ( b + c ) a = 1 , b = 1 0 16 , c = − 1 0 16
What error does naive summation of n ∼ ( n − 1 ) u ∑ ∣ x i ∣ n
What does Kahan summation achieve and how? Sum error ko ∼ u n
Why is 0.1+0.2==0.3 False? Woh teeno binary mein exactly representable nahi hain; rounded sum, rounded 0.3 se ∼ 1 0 − 17
Correct way to compare floats? Tolerance test: abs(x-y) <= atol + rtol*max(|x|,|y|) (jaise math.isclose).
Rounding model fl x = x 1+delta
Absolute error grows with size
Catastrophic cancellation
Quadratic formula loses precision
Rewrite formula to avoid subtraction
