Foundations — Bias-variance trade-off

5.6.4 · D1 · Coding › Machine Learning (Aerospace Applications) › Bias-variance trade-off

Yeh page assume karta hai ki aapne Bias-Variance Trade-off ki koi bhi notation pehle nahi dekhi. Hum har symbol zero se build karte hain, us order mein jis order mein ideas ek doosre par depend karte hain. Upar se neeche padho; koi bhi cheez apni jagah aane se pehle nahi aayegi.

1. Dataset: page par dots

Kisi bhi formula se pehle, machine learning ke raw material ko picture karo: measurements. Aerospace mein hum ek wing ke liye measure kar sakte hain uska angle of attack (hawa mein kitna teda hai) aur woh kitna drag produce karta hai. Ek measurement matlab ek dot.

Yeh topic isko kyun chahta hai: bias aur variance dono ko measure kiya jaata hai is gap ke roop mein ki dot actually kahan hai aur hamaara model sochta hai woh kahan hona chahiye. Dots nahi, gaps nahi, story nahi.

2. True function $f(x)$ — nature ka qanoon

Nature dots ko randomly scatter nahi karti. Ek hidden rule hota hai: "is angle par, drag usually itna hoga." Woh hidden rule ek smooth curve hai jo dot cloud ke middle se guzarti hai.

Isko kyun chahiye: yahi woh target hai jise hum hit karne ki koshish kar rahe hain. Bias literally "hamaara aim $f(x)$ se kitna drift karta hai" hai.

3. Noise $ϵ$ aur irreducible error ${\sigma }^2$

Real measurements kabhi exactly true curve par nahi hoti. Hawa ka ek jhatka, sensor ki ek tapping — har cheez ek dot ko line se hata deti hai. Us random jhatke ka ek naam hai.

$ϵ$ ke baare mein do facts jinpar hum baar baar rely karenge:

• Uske wobbles average hokar kuch nahi bante: picture mein dots curve ke upar aur neeche equally often land karte hain.
• Uska typical size ek number se capture hota hai, ${\sigma }^2$.

4. Model $\hat{f}(x)$ — law ka hamaara best guess

Hum $f(x)$ nahi jaante. Hum sirf dots dekhte hain. Toh hum ek guess build karte hain: apni marzi ki ek curve (ek straight line, ek parabola, ek wiggly polynomial). Us guess ko ek hat milti hai.

5. Expectation $E[\cdot ]$ — "kaafi saari tries ka average"

Yeh page par sabse subtle symbol hai, toh hum dhheere jaate hain. Sochiye aap apna model ek baar nahi train karte — aap use hazaar baar train karte hain, har baar fresh batch of dots par jo usi fuzzy band se draw hue hain. Har baar aapko thodi si alag curve $\hat{f}(x)$ milti hai. Expectation poochta hai: un saari attempts mein average par kya hota hai?

Topic ko isko kyun chahiye: aap "predictions kitna change hoga" ke baare mein baat nahi kar sakte bina bahut saare datasets imagine kiye. Variance hi woh change hai; expectation woh tool hai jo "bahut saare datasets" ko ek precise idea banata hai.

6. Bias — aim ka off hona

Ab payoff assemble karo. Average model curve $E[\hat{f}(x)]$ ko truth $f(x)$ se compare karo.

Bias consistently same direction mein galat hone ke baare mein hai — ek million tries ka average bhi miss karta hai. Bahut simple model (curved truth ke liye straight line) mein high bias hota hai kyunki woh bend nahi kar sakta, chahe kitne bhi dots dedo.

7. Variance — aim ka twitchy hona

Variance inconsistent hone ke baare mein hai: alag dots par retrain karo aur bahut alag jawab milta hai. Bahut flexible model (15 dots ke through 10-wiggle polynomial) mein high variance hoti hai kyunki woh noise ko chase karta hai, aur noise har baar alag hoti hai.

8. Squared error aur MSE — woh score jo hum actually minimise karte hain

Models ko rank karne ke liye hume ek single number chahiye jo bataye woh kitne bure hain.

Parent note ka headline result yeh hai ki yeh ek number teen samajh mein aane wale pieces mein split hota hai: $\underset{\text{total error}}{\underbrace{E[(y-\hat{f}(x){)}^2]}}=\underset{\text{aim off}}{\underbrace{{\text{Bias}}^2}}+\underset{\text{aim twitchy}}{\underbrace{\text{Variance}}}+\underset{\text{noise floor}}{\underbrace{{\sigma }^2}}$

Us equation mein har symbol ab kuch aisa hai jo aap dekh sakte hain: square-areas ke averages ka ek sum equals ek steady-gap-squared plus swarm-fatness plus fuzzy-band-thickness. Poora topic ek line mein hai, aur aapne uska har letter build kiya.

9. Model complexity — woh dial jo hum ghumaate hain

Ek aakhri idea inhe sab ek saath jodhta hai: woh knob jo hum control karte hain.

Prerequisite map

Yeh foundation directly 5.6.03-Overfitting-and-Regularization (high variance matlab overfitting), 5.6.02-Training-Validation-Test-Sets (hum swarm ki error unseen dots par kaise measure karte hain), aur 5.6.05-Cross-Validation-Techniques (hum practice mein variance kaise estimate karte hain) mein feed hoti hai.

Equipment checklist

Khud ko test karo: right side cover karo aur reveal karne se pehle jawab do.