Gaussian Mixture Models and EM algorithm

2.5.7 · AI-ML › Unsupervised Learning

Overview

Gaussian Mixture Models (GMM) ek probabilistic model hai jo assume karta hai ki data kai Gaussian distributions ke mixture se aata hai, jisme har ek ek cluster represent karta hai. Expectation-Maximization (EM) algorithm ek iterative method hai jo in Gaussians ke parameters find karta hai jab hume pata nahi hota ki kaunsa data point kis cluster ka hai.

Ise genres ki tarah socho: ek movie purely "action" ya "drama" nahi hoti — woh 60% action, 40% drama hoti hai. GMMs is blend ko naturally capture karte hain.

The Mathematical Foundation

$p(\mathbf{x})={\sum }_{k=1}^K{\pi }_k\mathcal{N}(\mathbf{x}|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)$

jahan:

• $K$ = mixture components (clusters) ki sankhya
• ${\pi }_k$ = component $k$ ka mixing coefficient (satisfies ${\sum }_{k=1}^K{\pi }_k=1$, ${\pi }_k\ge 0$)
• $\mathcal{N}(\mathbf{x}|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)$ = Gaussian distribution jiska mean ${\mathbf{\mu }}_k$ aur covariance ${\mathbf{\Sigma }}_k$ hai

Yeh form kyun? Hum keh rahe hain: "Cluster $k$ ko probability ${\pi }_k$ se chuno, phir us cluster ke Gaussian se ek point generate karo." Sum saare possible cluster choices par marginalize karta hai.

Gaussian Component Derive Karna

Multivariate Gaussian distribution hai:

$\mathcal{N}(\mathbf{x}|\mathbf{\mu },\mathbf{\Sigma })=\frac{1}{(2\pi {)}^{d/2}|\mathbf{\Sigma }{|}^{1/2}}\exp \left(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu }{)}^T{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathbf{\mu })\right)$

Yeh formula kyun?

• Exponent $-\frac{1}{2}(\mathbf{x}-\mathbf{\mu }{)}^T{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathbf{\mu })$ squared Mahalanobis distance hai — yeh measure karta hai ki $\mathbf{x}$, $\mathbf{\mu }$ se kitna dur hai, correlations ko account karte hue (jo $\mathbf{\Sigma }$ mein captured hai)
• Normalizing constant ensure karta hai ki $\int p(\mathbf{x})d\mathbf{x}=1$
• $(2\pi {)}^{d/2}$ aata hai $d$ dimensions par Gaussian integrate karne se
• $|\mathbf{\Sigma }{|}^{1/2}$ covariance ellipsoid ke "volume" ko account karta hai

The EM Algorithm: First Principles Se

PROBLEM: Hamare paas data $\mathbf{X}=\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_N\}$ hai lekin hume nahi pata ki kis cluster ne har point generate kiya. Hume $\mathbf{\theta }=\{{\pi }_k,{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k{\}}_{k=1}^K$ estimate karna hai.

Yeh mushkil kyun hai? Agar hume cluster assignments pata hoti, toh parameters estimate karna easy hota (sirf means aur covariances compute karo). Agar hume parameters pata hote, toh assignments compute karna easy hota (sirf Bayes' rule use karo). Hume dono nahi pata! Yeh ek chicken-and-egg problem hai.

Latent Variables Introduce Karna

Maano $z_i\in \{1,\dots ,K\}$ ${\mathbf{x}}_i$ ka hidden cluster label hai. Complete-data log-likelihood hogi:

$\log p(\mathbf{X},\mathbf{Z}|\mathbf{\theta })={\sum }_{i=1}^N{\sum }_{k=1}^{K}1(z_i=k)\left[\log {\pi }_k+\log \mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)\right]$

Yeh form kyun? Indicator $1(z_i=k)$ pick karta hai ki $i$ kis cluster ka hai, aur hum us choice (${\pi }_k$) ki probability aur us Gaussian ke under ${\mathbf{x}}_i$ ki probability ko log karte hain.

Lekin hum $\mathbf{Z}$ observe nahi karte! Toh hum incomplete-data log-likelihood maximize karte hain:

$\log p(\mathbf{X}|\mathbf{\theta })={\sum }_{i=1}^N\log \left({\sum }_{k=1}^K{\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)\right)$

Ise directly maximize karna mushkil hai kyunki sum ke log ka koi closed form nahi hota.

Jensen's inequality se, kisi bhi distribution $q(\mathbf{Z})$ ke liye:

$\log p(\mathbf{X}|\mathbf{\theta })=\log {\sum }_{\mathbf{Z}}p(\mathbf{X},\mathbf{Z}|\mathbf{\theta })$ $=\log {\sum }_{\mathbf{Z}}q(\mathbf{Z})\frac{p(\mathbf{X},\mathbf{Z}|\mathbf{\theta })}{q(\mathbf{Z})}$ $\ge {\sum }_{\mathbf{Z}}q(\mathbf{Z})\log \frac{p(\mathbf{X},\mathbf{Z}|\mathbf{\theta })}{q(\mathbf{Z})}=\mathcal{L}(q,\mathbf{\theta })$

Yeh kyun help karta hai? Lower bound $\mathcal{L}(q,\mathbf{\theta })$ maximize karna aasaan hai. Hum alternate karte hain:

1. E-step: $\mathbf{\theta }$ fix karo, $q$ choose karo taaki bound tight ho: $q(\mathbf{Z})=p(\mathbf{Z}|\mathbf{X},{\mathbf{\theta }}^{\text{old}})$
2. M-step: $q$ fix karo, $\mathcal{L}$ ko $\mathbf{\theta }$ ke respect mein maximize karo taaki ${\mathbf{\theta }}^{\text{new}}$ mile

E-Step: Responsibilities Compute Karna

Hume $p(z_i=k|{\mathbf{x}}_i,\mathbf{\theta })$ chahiye, jise responsibility ${\gamma }_{ik}$ kehte hain.

Bayes' rule se:

${\gamma }_{ik}=p(z_i=k|{\mathbf{x}}_i,\mathbf{\theta })=\frac{p(z_i=k)p({\mathbf{x}}_i|z_i=k,\mathbf{\theta })}{{\sum }_{j=1}^{K}p(z_i=j)p({\mathbf{x}}_i|z_i=j,\mathbf{\theta })}$

Yeh step kyun? Numerator: cluster $k$ ki prior probability times cluster $k$ ke under ${\mathbf{x}}_i$ ki likelihood. Denominator: saare clusters par total probability (normalization).

${\gamma }_{ik}=\frac{{\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)}{{\sum }_{j=1}^K{\pi }_j\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_j,{\mathbf{\Sigma }}_j)}$

${\gamma }_{ik}$ ka matlab kya hai? Probability ki point $i$ cluster $k$ se generate hua tha. Agar ${\gamma }_{i3}=0.8$, toh point $i$ 80% likely cluster 3 se hai.

M-Step: Parameters Update Karna

Hum expected complete-data log-likelihood maximize karte hain:

$Q(\mathbf{\theta },{\mathbf{\theta }}^{\text{old}})={\mathbb{E}}_{\mathbf{Z}|\mathbf{X},{\mathbf{\theta }}^{\text{old}}}[\log p(\mathbf{X},\mathbf{Z}|\mathbf{\theta })]$

Responsibilities substitute karne par:

$Q={\sum }_{i=1}^N{\sum }_{k=1}^K{\gamma }_{ik}\left[\log {\pi }_k+\log \mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)\right]$

${\mathbf{\mu }}_k$ update derive karna:

$\frac{\partial Q}{\partial {\mathbf{\mu }}_k}={\sum }_{i=1}^N{\gamma }_{ik}\frac{\partial }{\partial {\mathbf{\mu }}_k}\left[-\frac{1}{2}({\mathbf{x}}_i-{\mathbf{\mu }}_k{)}^T{\mathbf{\Sigma }}_k^{-1}({\mathbf{x}}_i-{\mathbf{\mu }}_k)\right]$

Yeh derivative kyun? Gaussian mein sirf quadratic term ${\mathbf{\mu }}_k$ par depend karta hai.

$={\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{\Sigma }}_k^{-1}({\mathbf{x}}_i-{\mathbf{\mu }}_k)$

Zero set karne par:

${\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{\Sigma }}_k^{-1}{\mathbf{x}}_i={\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{\Sigma }}_k^{-1}{\mathbf{\mu }}_k$

${\mathbf{\mu }}_k=\frac{{\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{x}}_i}{{\sum }_{i=1}^N{\gamma }_{ik}}=\frac{1}{N_k}{\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{x}}_i$

jahan $N_k={\sum }_{i=1}^N{\gamma }_{ik}$ cluster $k$ mein effective number of points hai.

Iska matlab kya hai? Naya mean saare points ka weighted average hai, jisme weights unki cluster $k$ ke liye responsibility hai. Points jo strongly cluster $k$ mein hain (${\gamma }_{ik}\approx 1$) mean ko apni taraf kheenchte hain; $k$ mein weakly hone wale points ka kam effect hota hai.

${\mathbf{\Sigma }}_k$ update derive karna:

Isi tarah (log-determinant aur quadratic terms ka derivative lete hue):

${\mathbf{\Sigma }}_k=\frac{1}{N_k}{\sum }_{i=1}^N{\gamma }_{ik}({\mathbf{x}}_i-{\mathbf{\mu }}_k)({\mathbf{x}}_i-{\mathbf{\mu }}_k{)}^T$

Kyun? Mean se squared deviations ka weighted average — covariance ki definition, responsibility-weighted.

${\pi }_k$ update derive karna:

Hum ${\sum }_{k=1}^K{N}_k\log {\pi }_k$ maximize karte hain subject to ${\sum }_{k=1}^K{\pi }_k=1$, Lagrange multiplier $\lambda$ use karke:

$\mathcal{L}={\sum }_{k=1}^K{N}_k\log {\pi }_k+\lambda \left(1-{\sum }_{k=1}^K{\pi }_k\right)$

$\frac{\partial \mathcal{L}}{\partial {\pi }_k}=\frac{N_k}{{\pi }_k}-\lambda =0⟹{\pi }_k=\frac{N_k}{\lambda }$

Suming over $k$: ${\sum }_{k=1}^K{\pi }_k=\frac{{\sum }_{k=1}^K{N}_k}{\lambda }=\frac{N}{\lambda }=1⟹\lambda =N$

${\pi }_k=\frac{N_k}{N}=\frac{1}{N}{\sum }_{i=1}^N{\gamma }_{ik}$

Iska matlab kya hai? Mixing coefficient wo fraction hai jo cluster $k$ ko assign ki gayi total responsibility hai.

Initialize: $K$ choose karo, ${\pi }_k$, ${\mathbf{\mu }}_k$, ${\mathbf{\Sigma }}_k$ randomly ya K-means se initialize karo

Convergence tak repeat karo:

E-step: Saare $i,k$ ke liye responsibilities compute karo: ${\gamma }_{ik}=\frac{{\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)}{{\sum }_{j=1}^K{\pi }_j\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_j,{\mathbf{\Sigma }}_j)}$

M-step: Saare $k$ ke liye parameters update karo: $N_k={\sum }_{i=1}^N{\gamma }_{ik}$ ${\mathbf{\mu }}_k=\frac{1}{N_k}{\sum }_{i=1}^N{\gamma }_{ik}{\mathbf{x}}_i$ ${\mathbf{\Sigma }}_k=\frac{1}{N_k}{\sum }_{i=1}^N{\gamma }_{ik}({\mathbf{x}}_i-{\mathbf{\mu }}_k)({\mathbf{x}}_i-{\mathbf{\mu }}_k{)}^T$ ${\pi }_k=\frac{N_k}{N}$

Convergence: Ruko jab log-likelihood change < threshold ho ya max iterations reach ho jaayein.

Worked Examples

Data: $\mathbf{X}=\{1,2,9,10\}$, $K=2$

Initialize:

• ${\pi }_1=0.5,{\pi }_2=0.5$
• ${\mu }_1=1.5,{\mu }_2=9.5$ (data endpoints se)
• ${\sigma }_1^2=1,{\sigma }_2^2=1$ (arbitrary small variance)

E-step iteration 1:

$x_1=1$ ke liye: $\mathcal{N}(1|1.5,1)=\frac{1}{\sqrt{2\pi }}{e}^{-0.125}\approx 0.352$ $\mathcal{N}(1|9.5,1)=\frac{1}{\sqrt{2\pi }}{e}^{-36.125}\approx 0$

Yeh values kyun? Plug into the Gaussian formula. Point 1, ${\mu }_1=1.5$ ke bahut paas hai (distance 0.5) lekin ${\mu }_2=9.5$ se bahut door (distance 8.5).

${\gamma }_{11}=\frac{0.5\times 0.352}{0.5\times 0.352+0.5\times 0}\approx 1.0$

Iska matlab kya hai? Point 1 essentially 100% cluster 1 ko assign hai.

Isi tarah:

• ${\gamma }_{21}\approx 1.0$ (point 2 → cluster 1)
• ${\gamma }_{31}\approx 0.0$ (point 9 → cluster 2)
• ${\gamma }_{41}\approx 0.0$ (point 10 → cluster 2)

M-step iteration 1:

$N_1={\gamma }_{11}+{\gamma }_{21}+{\gamma }_{31}+{\gamma }_{41}\approx 2.0$ $N_2\approx 2.0$

Kyun? Do points strongly cluster 1 mein hain, do cluster 2 mein.

${\mu }_1=\frac{1\times 1+1\times 2+0\times 9+0\times 10}{2}=\frac{3}{2}=1.5$

Yeh step kyun? Saare points ka weighted average, lekin sirf points 1 aur 2 ka cluster 1 ke liye non-zero weight hai.

${\mu }_2=\frac{0\times 1+0\times 2+1\times 9+1\times 10}{2}=9.5$

${\sigma }_1^2=\frac{1\times (1-1.5{)}^2+1\times (2-1.5{)}^2}{2}=\frac{0.25+0.25}{2}=0.25$

Kyun? Variance squared deviations ka weighted average hai. Dono points mean 1.5 se 0.5 door hain.

${\sigma }_2^2=\frac{1\times (9-9.5{)}^2+1\times (10-9.5{)}^2}{2}=0.25$

${\pi }_1={\pi }_2=\frac{2}{4}=0.5$

Result: Ek iteration ke baad, clusters tight ho gaye (${\sigma }^2$ 1 se 0.25 ho gaya), means stable hain. Algorithm jaldi converge hoga kyunki data well-separated hai.

Data:

• Cluster 1: $(0,0),(1,1),(0.5,0.5)$
• Cluster 2: $(4,4),(5,5),(4.5,4.5)$
• Ambiguous point: $(2.5,2.5)$

Convergence ke baad:

• ${\mathbf{\mu }}_1\approx (0.5,0.5)$
• ${\mathbf{\mu }}_2\approx (4.5,4.5)$
• Ambiguous point $(2.5,2.5)$ ke liye: ${\gamma }_{71}\approx 0.5,{\gamma }_{72}\approx 0.5$

50/50 kyun? Yeh point dono cluster centers se equidistant hai, isliye dono ke liye equal responsibility hai. Yahi GMM ka key advantage hai: soft assignments uncertainty capture karte hain.

Agar hum hard assignment force karte (jaise K-means), toh hum ise arbitrarily ek cluster mein daal dete, information kho dete. GMM kehta hai "yeh point genuinely ambiguous hai — yeh aadha dono clusters se hai."

Convergence check ke liye, hum compute karte hain:

$\log p(\mathbf{X}|\mathbf{\theta })={\sum }_{i=1}^N\log \left({\sum }_{k=1}^K{\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)\right)$

Point ${\mathbf{x}}_1=(1,1)$ ke liye $K=2$ ke saath:

$p({\mathbf{x}}_1)={\pi }_1\mathcal{N}({\mathbf{x}}_1|{\mathbf{\mu }}_1,{\mathbf{\Sigma }}_1)+{\pi }_2\mathcal{N}({\mathbf{x}}_1|{\mathbf{\mu }}_2,{\mathbf{\Sigma }}_2)$

Agar ${\pi }_1=0.6$, ${\pi }_2=0.4$, aur Gaussians $0.2$ aur $0.001$ evaluate karte hain:

$p({\mathbf{x}}_1)=0.6\times 0.2+0.4\times 0.001=0.12+0.004=0.1204$

Sum kyun? Total probability dono components ka mixture hai.

$\log p({\mathbf{x}}_1)=\log (0.1204)\approx -2.116$

Total log-likelihood ke liye saare points par sum karo. Agar yeh iterations ke beech $10^{-6}$ se kam change ho, toh hum converge ho gaye hain.

Common Mistakes and Fixes

Galat: ${\gamma }_{ik}={\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)$ compute karo aur directly use karo.

Kyun sahi lagta hai: Humne cluster $k$ ke under ${\mathbf{x}}_i$ ki probability compute ki, toh yahi responsibility honi chahiye!

Kyun galat hai: Denominator ke bina, ${\sum }_{k=1}^K{\gamma }_{ik}\ne 1$. Responsibilities conditional probabilities hain aur har point ke liye 1 sum honi chahiye.

Fix: Hamesha normalizing sum include karo: ${\gamma }_{ik}=\frac{{\pi }_k\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_k,{\mathbf{\Sigma }}_k)}{{\sum }_{j=1}^K{\pi }_j\mathcal{N}({\mathbf{x}}_i|{\mathbf{\mu }}_j,{\mathbf{\Sigma }}_j)}$

Galat: Ek cluster ek single point par collapse ho jaata hai, jisse $|{\mathbf{\Sigma }}_k|=0$ ho jaata hai, jo Gaussian evaluation break karta hai (divide by zero).

Kyun hota hai: Agar ek cluster mein saare points identical coordinates (ya numerical issues ki wajah se nearly identical) hain, toh covariance singular ho jaati hai.

Kyun dangerous hai: Algorithm crash ho jaata hai ya NaN values produce karta hai.

Fix: Regularization add karo ek small diagonal term daalke: ${\mathbf{\Sigma }}_k=\frac{1}{N_k}{\sum }_{i=1}^N{\gamma }_{ik}({\mathbf{x}}_i-{\mathbf{\mu }}_k)({\mathbf{x}}_i-{\mathbf{\mu }}_k{)}^T+ϵ\mathbf{I}$ jahan $ϵ=10^{-6}$ ek small constant hai. Yeh ensure karta hai ki ${\mathbf{\Sigma }}_k$ hamesha positive definite ho.

Steel-man: "Main artificial variance add nahi karna chahta — yeh data mein nahi hai!" Sach hai, lekin zero variance ke saath, model keh raha hai "yeh cluster sirf ek exact point generate karta hai infinite precision ke saath," jo physically unrealistic hai. Regularization measurement noise ya model uncertainty represent karta hai.

Galat: Saare ${\mathbf{\mu }}_k$ ko data dekhe bina uniform distribution se randomly initialize karo.

Kyun sahi lagta hai: Random initialization neural networks mein gradient descent ke liye kaam karta hai, toh yahan bhi kaam karna chahiye!

Kyun galat hai: EM initialization ke liye sensitive hai. Agar do means bahut paas se start hote hain, toh woh paas hi reh sakte hain (dono same cluster model karne ki koshish karte hain) jabki koi aur cluster bilkul miss ho jaata hai.

Fix: Initialization ke liye K-means++ use karo:

1. K-means convergence tak chalao (fast, simple)
2. K-means cluster centers ko ${\mathbf{\mu }}_k$ ki tarah use karo
3. K-means cluster covariances ko ${\mathbf{\Sigma }}_k$ ki tarah use karo
4. ${\pi }_k$ initialize karne ke liye K-means cluster sizes use karo

Kyun kaam karta hai: K-means ek achha rough clustering deta hai, aur EM usse soft assignments ke saath refine karta hai.

Galat: EM ek baar chalao, result lo, assume karo ki yeh best possible clustering hai.

Kyun sahi lagta hai: EM guaranteed hai ki har iteration mein likelihood badhega, toh yeh global maximum par converge hona chahiye!

Kyun galat hai: EM guaranteed hai ki local maximum par converge kare, necessarily global nahi. Alag-alag initializations alag local maxima par le jaate hain.

Fix: EM multiple times (5-10) alag random initializations ke saath chalao, sabse zyada final log-likelihood wala result chuno.

Steel-man: "Lekin yeh computationally expensive hai!" Haan, lekin zaroori hai. Log-likelihood landscape non-convex hai aur iske kai local maxima hain. Socho jaise neural networks train karna: tum alag random seeds try karte ho aur best chuno.

Clusters Ki Sankhya K Choose Karna

K-means ke unlike, GMM probabilistic hai, isliye hum model selection criteria use kar sakte hain:

Akaike Information Criterion (AIC): $\text{AIC}=2k-2\ln (\hat{L})$

jahan $k$ parameters ki sankhya hai, $\hat{L}$ maximum likelihood hai.

Bayesian Information Criterion (BIC): $\text{BIC}=k\ln (N)-2\ln (\hat{L})$

BIC, AIC se kyun behtar? BIC model complexity ko zyada heavily penalize karta hai (factor of $\ln (N)$ vs. 2). GMM ke liye, BIC aksar overfitting se bachne mein better perform karta hai.

Kaise use karo: $K=1,2,3,\dots$ ke saath GMMs fit karo aur BIC plot karo. Sabse kam BIC wala $K$ chuno.

GMM ke liye parameter count:

• $K-1$ mixing coefficients (constraint: sum to 1)
• $K\times d$ means
• $K\times d(d+1)/2$ covariance parameters (symmetric matrices)

Total: $k=(K-1)+Kd+K\frac{d(d+1)}{2}$

Connections

• K-Means Clustering: GMM, K-means ka soft version hai; K-means, GMM hai spherical covariances aur hard assignments ke saath
• Maximum Likelihood Estimation: EM mixture model ke under observed data ki likelihood maximize karta hai
• Bayes Theorem: Responsibilities Bayes' rule se compute ki jaati hain
• Expectation Maximization Algorithm: GMM general EM framework ka sabse common application hai
• Principal Component Analysis: GMM covariances data spread capture karti hain jaise PCA, lekin per-cluster
• Hidden Markov Models: HMMs, EM ko similar E-step (forward-backward) aur M-step updates ke saath use karte hain
• Variational Inference: Modern Bayesian GMs, variational EM use karte hain (zyada robust, automatic $K$ selection)
• Anomaly Detection: Points jo saare mixture components ke under kam probability rakhte hain woh outliers hain
• Density Estimation: GMs full probability density model karte hain, generative modeling ke liye useful

Ya socho: Evaluate, phir Modify. Tum evaluate karte ho ki points kahan belong karte hain, phir cluster parameters accordingly modify karte ho.

Tum guess se shuru karte ho: "Main sochta hoon chhote marbles red the aur bade wale blue."

Phir tum har marble dekhte ho aur kehte ho: "Yeh medium-sized hai... yeh probably 60% likely red aur 40% likely blue hai." Tum sure nahi ho, toh tum apne bets hedge karte ho.

Ab tum apna guess update karte ho: "Okay, agar main is marble ko 60% red treat karun, toh red marbles ki average size probably itni hogi..." Tum in partial assignments ke basis par reds aur blues ki average size recalculate karte ho.

Phir tum apne naye averages ke saath har marble phir dekhte ho: "Hmm, is naye average ke saath, shayad yeh marble actually 70% red hai, 60% nahi."

Tum baar baar jaate rehte ho — apna guess update karo ki kaunse marbles kaun sa color hain, phir apna guess update karo ki average red aur blue sizes kya hain — jab tak tumhare guesses badalna band nahi ho jaate. Yahi EM algorithm hai! Tum assignments estimate karne (E-step) aur parameters update karne (M-step) ke beech alternate kar rahe ho.

GMMs yahi data points aur clusters ke saath karte hain marbles aur colors ki jagah.

Flashcards

#flashcards/ai-ml

GMM ka matlab kya hai aur K-means par uska key advantage kya hai? :: Gaussian Mixture Model. Key advantage: soft assignments — har point ki har cluster mein belong karne ki probability hoti hai, uncertainty capture karte hue, ek cluster mein hard assignment ki jagah.

Kya

Concept Map