Machine epsilon — what it means in practice

4.8.3 · Maths › Numerical Methods

Machine epsilon exist KYUN karta hai?

Floating-point number KIYA hota hai?

Hum $ε_{mach} = 2^{- t}$ KAISE derive karte hain?

Figure — Machine epsilon — what it means in practice

Worked examples

Worked example 2 — Epsilon khud compute karo

$E = 1.0$ se shuru karo. Ise tab tak halve karte raho jab tak $1 + E > 1$ abhi bhi true rahe:

E = 1.0
while 1.0 + E > 1.0:
    E = E/2
E = 2*E   # last value that still mattered

Loop tab rukta hai jab $1 + E$ wapas $1$ par round ho jata hai, yaani jab $E$ gap se niche girta hai. Final $E \approx 2.22 \times 1 0^{- 16}$ . Yeh step kyun? Hum literally sabse choti perturbation ko probe kar rahe hain jise $1.0$ abhi bhi "feel" kar sake. Yahi definition hai.

Worked example 4 — Catastrophic cancellation epsilon ko amplify karta hai

$x = 1 0^{- 8}$ ke paas $f (x) = \frac{1 - cos x}{x ^{2}}$ compute karo. True value $\to 1/2$ . Yahan $1.0$ exactly store hota hai, lekin $cos x \approx 1 - 5 \times 1 0^{- 17}$ nearest float of $1$ par round ho jata hai, yaani $1$ se uski tiny deviation relative error $\sim ε_{mach} \approx 2 \times 1 0^{- 16}$ ki wajah se kho jaati hai. Do nearly-equal numbers ko subtract karna $cos x$ mein us rounding error ko ek huge relative error mein promote kar deta hai, aur $0$ ya $1.1$ jaisi garbage deta hai. Fix: Subtraction se bachne ke liye $\frac{2 sin ^{2} ( x /2 )}{x ^{2}}$ ki tarah rewrite karo. Yeh step kyun? Error literal $1.0$ se nahi aata (jo exact hai); yeh $cos x$ ko round karne se aata hai. Near-equal quantities ka subtraction phir surviving result ko kuch tiny se divide kar deta hai, relative error ko blow up karta hai.

Forecast-then-Verify

Verify: (1) $2^{- 23} \approx 1.19 \times 1 0^{- 7}$ . (2) $1$ ke paas; $1000 \approx 2^{10}$ ke paas gap $2^{10} ε_{mach}$ hai — gaps magnitude ke saath badhte hain. (3) Nahi — yeh underflow/denormals hai, ek alag limit (sabse chota normalized $\approx 2.2 \times 1 0^{- 308}$ , sabse chota subnormal $\approx 5 \times 1 0^{- 324}$ ).

Flashcards

Machine epsilon kya hai?

1.0

aur next representable float ke beech ka gap;

ε_{mach} = 2^{- t}

, number system ki relative precision.

IEEE double precision ke liye

ε_{mach}

ki value?

2^{- 52} \approx 2.22 \times 1 0^{- 16}

(

t = 52

mantissa bits).

ε_{mach} = 2^{- t}

kyun?

Mantissa mein

t

fractional bits hain;

1

se upar sabse chota increment last bit flip karna hai =

2^{- t}

ε_{mach}

aur sabse chote positive float mein fark?

Epsilon =

1

ke paas relative precision (~

1 0^{- 16}

); sabse chota positive float = underflow limit (sabse chota normalized ~

2.2 \times 1 0^{- 308}

, sabse chota subnormal ~

5 \times 1 0^{- 324}

). Alag concepts hain.

Unit roundoff

u

kya hai?

Max relative rounding error

= \frac{1}{2} ε_{mach} = 2^{- t - 1}

, kyunki rounding half a gap ke andar land karti hai.

0.1+0.2 != 0.3 kyun?

0.1, 0.2, 0.3 binary mein exactly representable nahi hain; rounding

ε_{mach}

order ka residue chhodti hai (observed

\approx 4.44 \times 1 0^{- 17}

Do floats compare karne ka sahi tarika?

∣ x - y ∣ \leq tol \cdot max (∣ x ∣, ∣ y ∣)

with

tol \sim 10 ε_{mach}

, kabhi == nahi.

Catastrophic cancellation kya hai?

Do nearly-equal numbers subtract karna tiny rounding-error ko large relative error mein promote kar deta hai.

Kya literal 1.0 exactly store hota hai?

Haan — 1.0 exactly representable hai;

1 - cos x

mein error

cos x

ko round karne se aata hai,

1.0

store karne se nahi.

Floats ke beech gap magnitude ke saath kaise badalta hai?

2^{e}

size ke number ke paas gap

2^{e} ε_{mach}

hai — gaps magnitude ke saath badhte hain.

Single precision (

t = 23

) ke liye

ε_{mach}

2^{- 23} \approx 1.19 \times 1 0^{- 7}

Recall Feynman: 12-saal ke bachche ko samjhao

Socho ek ruler jo number 1 ke paas sirf bahut paas-paas lines dikhata hai, lekin numbers ke bade hone ke saath lines aur door ho jaati hain. Computer sirf ek line par point kar sakta hai, kabhi do lines ke beech nahi. Machine epsilon number 1 ke bilkul baad waali do lines ke beech ka sabse chota gap hai. Agar aap computer se us gap se patli cheez mark karne kahein, toh woh nearest line par snap kar leta hai — woh tiny snapping error hi wajah hai ki $0.1 + 0.2$ thoda sa galat aata hai. Computer baavla nahi hai; uske paas bas ek sabse chota "pencil width" hai, aur epsilon wahi width hai.

Connections

Floating-point representation (IEEE-754) — jahan se $t$ , exponent, sign aate hain
Rounding error and unit roundoff — $u = \frac{1}{2} ε_{mach}$
Catastrophic cancellation — epsilon-noise kaise amplify hoti hai
Condition number — output error $\approx$ condition number $\times ε_{mach}$
Numerical stability of algorithms — error $\sim ε_{mach}$ rakhne ke liye design karna
Underflow and overflow — number range ki doosri limits