Bayes' theorem — derivation and applications: Exercises

2.7.9 · D4 · Maths › Statistics & Probability — Intermediate › Bayes' theorem — derivation and applications

Level 1 — Recognition

Yahan tumhe bas kaunsi quantity kaunsi hai woh spot karni hai. Koi mushkil arithmetic nahi — goal yeh hai ki kabhi bhi "cause diya hua" aur "evidence diya hua" mein confuse mat hona.

Ek weather app kehti hai: "Jab actually barish hoti hai, tab app ne barish forecast ki thi $90%$ time." Symbols mein, jahan $R =$ "barish hoti hai" aur $F =$ "app barish forecast karti hai", kaunsi conditional probability $0.90$ hai: $P (R ∣ F)$ ya $P (F ∣ R)$ ?

Recall Solution 1.1

Sentence pehle actual weather fix karta hai ("jab actually barish hoti hai") aur phir batata hai app kaise behave karti hai. Toh given (condition, bar ke baad wali cheez) $R$ hai. Iska matlab hai: $P (F ∣ R) = 0.90.$ Yeh ek likelihood hai — evidence (forecast) ki probability given cause (rain). Jo quantity ek user actually chahta hai, "usne barish forecast ki, toh actual barish kitni likely hai?", woh ulti wali $P (R ∣ F)$ hai, aur Bayes hi unhe connect karta hai.

Exercise 1.2

Disease-test setting mein, har phrase ko uske symbol se match karo: (a) " $1%$ logon ko disease hai", (b) "test $99%$ sensitive hai", (c) "tumhara test positive aaya; kya tum beemar ho?". $D$ = disease, $+$ = positive test use karo.

Recall Solution 1.2

(a) prior hai $P (D) = 0.01$ — kisi bhi test se pehle belief.
(b) likelihood hai $P (+ ∣ D) = 0.99$ — test kaise behave karta hai given tum beemar ho (sensitivity ek "true-positive rate" hai).
(c) posterior hai $P (D ∣ +)$ — positive dekhne ke baad updated belief. Yeh woh unknown hai jo Bayes compute karta hai.

Exercise 1.3

Do bags tumhare saare marbles partition karti hain: bag $A_{1}$ (red) aur bag $A_{2}$ (blue), jahan $P (A_{1}) = 0.7$ , $P (A_{2}) = 0.3$ . Abhi tak koi evidence nahi hai, kya in dono priors ka sum $1$ hona zaroori hai? Kyun?

Recall Solution 1.3

Haan: $0.7 + 0.3 = 1$ . Ek partition ka matlab hai causes mutually exclusive hain (ek marble exactly ek bag mein hai) aur exhaustive hain (koi aur bag nahi hai). Har marble exactly ek baar account hota hai, isliye priors ka sum $1$ hona zaroori hai. Agar nahi hota, toh Bayes ka denominator posteriors ko $1$ tak normalise kabhi nahi kar sakta.

Level 2 — Application

Ab tum formula mein numbers daalo, joints organize karne ke liye tree structure use karo.

Exercise 2.1

Ek factory line: $80%$ items Type-A hain, $20%$ Type-B hain. Type-A inspection mein $0.95$ probability se pass hota hai; Type-B $0.60$ probability se pass hota hai. Ek item pass hota hai. $P (Type-A ∣ pass)$ nikalo.

Recall Solution 2.1

Priors: $P (A) = 0.8, P (B) = 0.2$ . Likelihoods: $P (pass ∣ A) = 0.95, P (pass ∣ B) = 0.60$ .

Evidence (Law of Total Probability — pass A ya B se aa sakta hai): $P (pass) = (0.95) (0.8) + (0.60) (0.2) = 0.76 + 0.12 = 0.88.$ Bayes: $P (A ∣ pass) = \frac{0.76}{0.88} = \frac{19}{22} \approx 0.8636.$ Posterior ( $0.864$ ) prior ( $0.8$ ) se barely upar jaata hai — pass hona sirf thoda sa evidence hai Type-A ka, kyunki Type-A items lagbhag utni hi aasaani se pass hote hain jitna Type-B.

Exercise 2.2

Exercise 2.1 ke numbers reuse karo lekin item fail ho jaata hai. $P (Type-B ∣ fail)$ nikalo.

Recall Solution 2.2

Fail likelihoods complements hain: $P (fail ∣ A) = 1 - 0.95 = 0.05$ , $P (fail ∣ B) = 1 - 0.60 = 0.40$ .

Evidence: $P (fail) = (0.05) (0.8) + (0.40) (0.2) = 0.04 + 0.08 = 0.12.$ (Check: $0.88 + 0.12 = 1$ , toh passes aur fails saare items ko partition karte hain.)

Bayes: $P (B ∣ fail) = \frac{0.08}{0.12} = \frac{2}{3} \approx 0.6667.$ Failure Type-B ka strong evidence hai: chahe sirf $20%$ items Type-B hain, woh aath guna zyada baar fail hote hain, isliye woh failures mein dominate karte hain.

Exercise 2.3

Parent note ke Worked Example 1 ko redo karo lekin ek zyada specific test ke saath: $P (+ ∣ D) = 0.99$ , $P (+ ∣ \overset{ˉ}{D}) = 0.01$ (specificity $99%$ ), prevalence $P (D) = 0.01$ . $P (D ∣ +)$ nikalo.

Recall Solution 2.3

$P (+) = (0.99) (0.01) + (0.01) (0.99) = 0.0099 + 0.0099 = 0.0198.$ $P (D ∣ +) = \frac{0.0099}{0.0198} = 0.5.$ False-positive rate ko $5%$ se $1%$ tak karna posterior ko $16.7%$ (parent note) se exactly $50%$ tak le jaata hai. Specificity, sirf sensitivity nahi, woh hai jo rare disease par false positives ko control karta hai.

Level 3 — Analysis

Yahan tum teen ya zyada causes ki partition handle karte ho, ya kaise answer move karta hai iske baare mein reason karte ho.

Exercise 3.1

Teen factories chips supply karti hain: X ( $50%$ , defect rate $1%$ ), Y ( $30%$ , defect rate $2%$ ), Z ( $20%$ , defect rate $4%$ ). Ek chip defective hai. Har factory se aane ki posterior probability nikalo, aur most likely source batao.

Recall Solution 3.1

Joints $P (D ∣ \cdot) P (\cdot)$ : $X : (0.01) (0.50) = 0.005, Y : (0.02) (0.30) = 0.006, Z : (0.04) (0.20) = 0.008.$ Evidence: $P (D) = 0.005 + 0.006 + 0.008 = 0.019.$ Posteriors: $P (X ∣ D) = \frac{0.005}{0.019} \approx 0.2632, P (Y ∣ D) = \frac{0.006}{0.019} \approx 0.3158, P (Z ∣ D) = \frac{0.008}{0.019} \approx 0.4211.$ Inका sum $1$ hai (achha). Sabse kam chips banane ke bawajood, Z ek defect ka most likely source hai kyunki iska defect rate X ka char guna hai. Figure dekho: prior bars (widths) posterior tak shrink ho jaate hain jab hum defect rate se weight karte hain.

Figure — Bayes' theorem — derivation and applications

Exercise 3.2

Exercise 3.1 mein, bina recompute kiye decide karo: agar Factory Z ki defect rate aadhi hokar $2%$ ho jaaye, toh kya $P (Z ∣ D)$ upar jaayega ya neeche? Phir verify karo.

Recall Solution 3.2

Pehle reasoning: Z ki defect rate aadhi karna Z ke joint $P (D ∣ Z) P (Z)$ ko $0.008$ se $0.004$ tak shrink karta hai, jabki X aur Y unchanged hain. Ek chhota numerator ek chhote total ke saath Z ki share ko neeche kheenchta hai.

Verify: naye joints $X = 0.005, Y = 0.006, Z = 0.004$ ; evidence $= 0.015$ . $P (Z ∣ D) = \frac{0.004}{0.015} \approx 0.2667,$ $0.4211$ se neeche aa gaya. Y ab leader ban jaata hai ( $0.006/0.015 = 0.4$ ). Most likely cause fixed nahi hai — yeh prior aur likelihood ke product ko track karta hai.

Exercise 3.3

Ek spam filter: $40%$ email spam hai. Word "free" $60%$ spam mein aur $10%$ ham (non-spam) mein appear hota hai. Ek email mein "free" hai. $P (spam ∣ free)$ nikalo, phir batao ki "free" contain karne wale emails ka kitna fraction actually ham hai.

Recall Solution 3.3

Priors: $P (S) = 0.4, P (H) = 0.6$ . Likelihoods: $P (free ∣ S) = 0.6, P (free ∣ H) = 0.1$ . $P (free) = (0.6) (0.4) + (0.1) (0.6) = 0.24 + 0.06 = 0.30.$ $P (S ∣ free) = \frac{0.24}{0.30} = 0.8.$ Ham fraction complement hai, $P (H ∣ free) = 0.06/0.30 = 0.2$ . Yeh "ek word, ek update" step exactly ek Naive Bayes Classifier ka atom hai, jo aisi kai word-updates ko multiply karta hai.

Level 4 — Synthesis

Ab tum Bayes ko ek doosri idea ke saath combine karte ho: repeated evidence, complementary reasoning, ya Independent Events.

Exercise 4.1

Parent note wali same rare disease: $P (D) = 0.01$ , $P (+ ∣ D) = 0.99$ , $P (+ ∣ \overset{ˉ}{D}) = 0.05$ . Tum do baar positive test karte ho, aur dono tests tumhari true state given conditionally independent hain. $P (D ∣ + +)$ nikalo.

Recall Solution 4.1

Conditional independence ka matlab hai, ek fixed true state given, dono results multiply hote hain: $P (+ + ∣ D) = 0.9 9^{2} = 0.9801, P (+ + ∣ \overset{ˉ}{D}) = 0.0 5^{2} = 0.0025.$ Evidence: $P (+ +) = (0.9801) (0.01) + (0.0025) (0.99) = 0.009801 + 0.002475 = 0.012276.$ Bayes: $P (D ∣ + +) = \frac{0.009801}{0.012276} \approx 0.7984.$ Ek positive ne sirf $16.7%$ diya tha; ek doosra independent positive ise rocket karke $\approx 79.8%$ tak le jaata hai. Bayes chains: test 1 ke baad posterior test 2 ke liye prior ban jaata hai.

(Chaining view se cross-check: test 1 ke baad, prior $= 0.16 \overline{6}$ . Phir $P (D ∣ +) = \frac{0.99 \cdot 0.16667}{0.99 \cdot 0.16667 + 0.05 \cdot 0.83333} = \frac{0.165}{0.165 + 0.041667} = 0.7984$ — same answer.)

Exercise 4.2

Ek box mein do coins hain: ek fair ( $P (H) = 0.5$ ) aur ek two-headed ( $P (H) = 1$ ), equal prior ke saath choose kiya gaya. Tum ek coin draw karte ho, flip karte ho, aur heads aata hai. $P (two-headed ∣ H)$ nikalo. Phir usi coin ke agle flip ki probability nikalo ki heads aaye.

Recall Solution 4.2

Priors: har coin $0.5$ . Likelihoods: $P (H ∣ fair) = 0.5, P (H ∣ 2-head) = 1$ . $P (H) = (0.5) (0.5) + (1) (0.5) = 0.25 + 0.5 = 0.75.$ $P (2-head ∣ H) = \frac{0.5}{0.75} = \frac{2}{3} \approx 0.6667.$ Agla flip — dono coins ko unke posterior weights se mix karo: $P (H_{2} ∣ H_{1}) = 2-head \frac{2}{3} \cdot 1 + fair \frac{1}{3} \cdot \frac{1}{2} = \frac{2}{3} + \frac{1}{6} = \frac{5}{6} \approx 0.8333.$ Ek heads dekhne se tum two-headed coin ki taraf lean karne lage, jo agle heads ki expected chance $0.75$ se upar le jaata hai.

Exercise 4.3

Ek student ne ya toh padha (prior $0.7$ ) ya guess kiya (prior $0.3$ ). Ek padhne wala $4$ -option question sahi probability $0.9$ se answer karta hai; ek guesser pure chance se sahi karta hai, $1/4$ . Student correctly answer karta hai. $P (studied ∣ correct)$ nikalo.

Recall Solution 4.3

Guess likelihood: $P (correct ∣ guess) = \frac{1}{4} = 0.25$ . $P (correct) = (0.9) (0.7) + (0.25) (0.3) = 0.63 + 0.075 = 0.705.$ $P (studied ∣ correct) = \frac{0.63}{0.705} = \frac{0.63}{0.705} \approx 0.8936.$ Ek correct answer studying mein belief ko $0.70$ se $\approx 0.894$ tak nudge karta hai — evidence hai, lekin certainty nahi, kyunki guessers kabhi kabhi lucky ho jaate hain.

Level 5 — Mastery

Full modelling: tum partition, likelihoods banate ho, aur result interpret karte ho — koi scaffolding nahi.

Exercise 5.1

Ek urn mein $3$ red aur $2$ blue balls hain. Tum without replacement do balls draw karte ho. Given ki doosri ball red hai, probability nikalo ki pehli ball red thi. (Order matters; "pehle red" / "pehle blue" ko apni partition ke do causes ke roop mein use karo.)

Recall Solution 5.1

Priors (pehle draw ka colour): $P (R_{1}) = \frac{3}{5}, P (B_{1}) = \frac{2}{5}$ . Likelihoods "doosra red hai" ke liye, ab ek ball remove ho chuki hai: $P (R_{2} ∣ R_{1}) = \frac{2}{4} = \frac{1}{2}, P (R_{2} ∣ B_{1}) = \frac{3}{4} .$ Evidence: $P (R_{2}) = \frac{3}{5} \cdot \frac{1}{2} + \frac{2}{5} \cdot \frac{3}{4} = \frac{3}{10} + \frac{3}{10} = \frac{6}{10} = \frac{3}{5} .$ (Sanity check: symmetry se kisi bhi specific draw ka marginal chance red hone ka overall red fraction $3/5$ ke barabar hai — reassuring.) Bayes: $P (R_{1} ∣ R_{2}) = \frac{\frac{3}{10}}{\frac{6}{10}} = \frac{1}{2} .$ Yeh jaanna ki doosri ball red hai, pehli ball ka red hona uske prior $3/5$ se thoda kam likely banata hai — kyunki red pehla draw doosre ke liye kam reds chhodta hai.

Exercise 5.2

Ek signal source bit $0$ prior $0.6$ ke saath aur bit $1$ prior $0.4$ ke saath bhejta hai. Ek noisy channel bit ko $0.1$ probability se flip karta hai (kaunsa bit hai usse independent). Tumhe $1$ receive hota hai. Most likely kya bheja gaya tha, aur kis posterior probability ke saath?

Recall Solution 5.2

$1$ receive karne ki likelihoods: agar $1$ bheja gaya, koi flip nahi: $P (r = 1 ∣ s = 1) = 0.9$ ; agar $0$ bheja gaya, flip zaroori hai: $P (r = 1 ∣ s = 0) = 0.1$ . $P (r = 1) = (0.1) (0.6) + (0.9) (0.4) = 0.06 + 0.36 = 0.42.$ $P (s = 1 ∣ r = 1) = \frac{0.36}{0.42} = \frac{6}{7} \approx 0.8571.$ Most likely $1$ bheja gaya tha, posterior $\approx 0.857$ . Receiver ka optimal decision prior bit-frequency aur channel reliability dono use karta hai — yeh Bayesian decoding ka core hai.

Exercise 5.3

Bag $A$ : $70%$ balls green hain. Bag $B$ : $40%$ green. Tum randomly ek bag choose karte ho ( $0.5$ each), with replacement draw karte ho, aur sequence green, green, red observe karte ho. Kaunsa bag zyada likely hai, aur $P (A ∣ GGR)$ do.

Recall Solution 5.3

With replacement, bag given draws independent hain, toh likelihoods multiply hoti hain. $P (GGR ∣ A) = 0.7 \cdot 0.7 \cdot 0.3 = 0.147, P (GGR ∣ B) = 0.4 \cdot 0.4 \cdot 0.6 = 0.096.$ Evidence: $P (GGR) = (0.147) (0.5) + (0.096) (0.5) = 0.0735 + 0.048 = 0.1215.$ Bayes: $P (A ∣ GGR) = \frac{0.0735}{0.1215} \approx 0.6049.$ Bag $A$ zyada likely hai ( $\approx 0.605$ ). Do greens greener bag $A$ ki taraf kheenchte hain; ek red thoda wapas kheenchta hai, lekin itna nahi ki usse overturn kar sake.

Recall Jaane se pehle ek-line self-test

Upar har problem ne same skeleton follow kiya — ise recite karo. ::: Priors $P (A_{i})$ list karo; matching likelihoods $P (B ∣ A_{i})$ list karo; products sum karke evidence $P (B)$ nikalo; jo ek joint chahiye use $P (B)$ se divide karo. "PLE over E."

Connections

Bayes' theorem — derivation and applications — parent note jise ye exercises drill karte hain.
Conditional Probability — definition jis par har step dependent hai.
Law of Total Probability — har solution mein evidence denominator banata hai.
Independent Events — L4/L5 mein likelihoods multiply karne ke liye use hota hai.
Tree Diagrams — joints ke peeche bookkeeping picture.
Naive Bayes Classifier — spam problem (3.3) scaled up.
Prior and Posterior Distributions — jahan chained updates (4.1) generalize hote hain.

Exercises — Bayes' theorem — derivation and applications