Question bank — Statistical estimation — MLE, method of moments

False. $L$ ek function hai $\theta$ ka fixed data ke liye; yeh $\theta$ ke upar $1$ tak integrate nahi hoti aur isko koi obligation nahi hai. Yeh data ki probability hai jo ulta padhi jaati hai, $\theta$ ke baare mein belief nahi — yeh ulta-seedha flip wahi hai jo Bayesian Estimation ek prior ke saath add karta hai.

False. log strictly increasing hai, isliye $\ell (\theta )=\log L(\theta )$ $L$ ke saath lockstep mein upar-neeche hota hai; dono same $\theta$ par peak karte hain. Log sirf algebra ko badalta hai, answer ko kabhi nahi.

False. Hume jo cheez actually care karni chahiye woh hai total error, $\text{MSE}=\text{Var}(\hat{\theta })+{\text{bias}}^2$. Ek thoda biased estimator jiska variance bahut kam ho, ek unbiased ko beat kar sakta hai — dekho Bias and Variance.

False. MLE likelihood ko optimize karta hai, unbiasedness ko nahi — alag goals hain. Normal MLE ${\hat{\sigma }}^2=\frac{1}{n}\sum (x_i-\bar{x}{)}^2$ $n$ se divide karta hai aur $E[{\hat{\sigma }}^2]=\frac{n-1}{n}{\sigma }^2<{\sigma }^2$ satisfy karta hai, isliye yeh neeche ki taraf biased hai.

False. Yeh exponential rate aur normal $\hat{\mu }$ ke liye coincide karte hain, lekin woh algebra ki kismat hai. $U(0,\theta )$ ke liye, MoM deta hai $\hat{\theta }=2\bar{x}$ jabki MLE deta hai $\hat{\theta }=\max x_i$ — bilkul alag recipes.

True. Yeh invariance property hai: kisi bhi function $g(\theta )$ ka MLE simply $g(\hat{\theta })$ hota hai. Yeh kaam karta hai kyunki axis ko $g$ se relabel karne se likelihood ka peak nahi hilta.

False. MoM equations ko blindly solve karta hai aur impossible values nikal sakta hai (ek negative variance, ek probability $1$ se zyada). Isko parameter ki support ki koi built-in respect nahi hai.

False. MLE consistent hota hai — yeh in probability converge karta hai, $\hat{\theta }\stackrel{P}{\to }\theta$ — matlab door hone ka chance zero ho jaata hai. Kisi bhi finite $n$ ke liye yeh abhi bhi wobble karta hai; "exactly" kabhi guarantee nahi hoti.

False. Ek constant (in $\theta$) poore curve ko upar ya neeche shift karta hai bina uska peak hilaaye; $\frac{d}{d\theta }(\ell +c)={\ell }^{\prime }$, isliye score equation untouched rehti hai. Isliye hum $-\frac{n}{2}\ln (2\pi )$ ko freely drop kar sakte hain.

True. $p$ unknowns ke saath tum pehle $p$ moments match karte ho — $p$ equations for $p$ unknowns. Kam equations system ko under-determined chhod deti hain; count forced hai.

Error avoidable pain hai, galat nahi: bahut saare chhote densities ka product numerically underflow karta hai aur product rule explode ho jaati hai. Pehle log lo — product ek sum ban jaata hai aur differentiation ek line mein ho jaata hai.

Error yeh hai ki $E[1/\bar{X}]\ne 1/E[\bar{X}]$ — tum expectation ko ek nonlinear function ke through nahi pull kar sakte (Jensen's inequality). Actually $\hat{\lambda }$ finite $n$ ke liye upar ki taraf biased hai.

Yahan differentiation fail ho jaati hai. Support $[0,\theta ]$ $\theta$ par depend karta hai; likelihood ${\theta }^{-n}$ hai jahan valid hai aur $0$ jab $\theta <\max x_i$, isliye jaise $\theta$ shrink hota hai yeh badhti hai aur zero par jump karti hai — koi interior critical point nahi. Maximum boundary par hai $\hat{\theta }=\max x_i$, inspection se milta hai.

${\ell }^{\prime }$ ka root ek minimum ya saddle bhi ho sakta hai. Tumhe confirm karna hoga ki ${\ell }^{\prime \prime }(\hat{\theta })<0$ (neeche curve karna = ek peak), aur yeh bhi check karna hoga ki parameter space ki koi boundary isse beat toh nahi kar rahi.

Dono $\frac{1}{n}\sum (x_i-\bar{x}{)}^2$ ke barabar hain — $n$ se division, jo biased hai. Unbiased sample variance $n-1$ se divide karta hai; yahan na MLE na MoM use produce karta hai.

Ek likelihood value $\theta$ ke sahi hone ke baare mein probability nahi hai. Yeh (proportional to) data ki probability hai us $\theta$ ke under; $\theta$'s ko compare karne ka matlab hai in values ko ek doosre se compare karna, na ki unhe $\hat{\theta }$ mein confidence ki tarah padhna.

Yeh large $n$ ke liye sirf approximately equal hote hain (yahi to Law of Large Numbers hai: $m_k\stackrel{P}{\to }{\mu }_k$). Finite data ke liye yeh differ karte hain, isliye MoM ek estimate hai, exact identity nahi.

Independent observations ka matlab hai ki unki joint probability multiply hoti hai: $P(\text{all data})=\prod f(x_i)$. Us joint probability ko maximize karna literally hai "kaunsa $\theta$ exactly is data ko least surprising banata hai."

Kyunki yeh monotonic hai — strictly increasing — isliye yeh order preserve karta hai: sabse bada $L$ sabse bade $\ell$ par map hota hai. Hum landscape ki height badal dete hain lekin uska summit kabhi nahi hilta.

Population moment ${\mu }_k=E[X^k]$ $\theta$ mein ek known formula hai; sample moment $m_k$ measurable hai. Kyunki $m_k\to {\mu }_k$ Law of Large Numbers se, unhe equal set karke solve karna humein true $\theta$ ke paas le jaata hai.

Kyunki $\bar{x}$ same data par fit hota hai, deviations $x_i-\bar{x}$ thodi chhoti hoti hain (mean data ki taraf pull hoti hai). Ek degree of freedom $\mu$ estimate karne mein kharch ho jaata hai, isliye humein $n-1$ se divide karna chahiye; MLE is baat ko ignore karta hai kyunki uska target likelihood hai, unbiasedness nahi.

MLE asymptotically efficient hota hai — large $n$ ke liye yeh sabse chhota possible variance achieve karta hai jo koi bhi estimator kar sakta hai (Cramér-Rao Lower Bound) — saath hi yeh consistent aur invariant bhi hai. MoM aasaan hai lekin often noisier hota hai aur legal range se bahar ja sakta hai.

Isse sirf moment formulas chahiye, likelihood maximization nahi, isliye yeh kaam karta hai jab density unknown ho ya likelihood intractable ho — ek fast, robust starting guess.

Kyunki "optimal" ek goal ke relative define hota hai. MLE likelihood optimize karta hai; unbiased estimator zero-bias optimize karta hai; ek minimum-MSE estimator dono ko trade off karta hai. Alag objectives, alag winners — jaise Bias and Variance explicitly batata hai.

$\hat{\theta }={\max }_i{x}_i$. Likelihood ${\theta }^{-n}$ (valid region par) jaise $\theta$ badhta hai shrink karti hai lekin har observation se $\ge$ rehni chahiye, isliye sabse chhota legal $\theta$ — sabse bada data point — jeetta hai. Yeh ek boundary maximum hai, ${\ell }^{\prime }=0$ ko dikhta hi nahi.

$\hat{p}=1$, $[0,1]$ ki ek boundary. Score equation $\frac{s}{p}-\frac{n-s}{1-p}=0$ ka koi interior root nahi hai (doosra term vanish ho jaata hai); likelihood $p^n$ $p$ ko uske edge tak push karke maximize hoti hai. Bilkul valid, bas ek interior critical point nahi hai.

$\bar{x}\to 0$ force karta hai $\hat{\lambda }\to \infty$ — ek degenerate estimate jo keh raha hai "rate bahut bada hai." Yeh bahut kam data ya ek ill-fitting model flag karta hai, koi usable number nahi.

Principle mein haan agar $\overline{x^2}<{\bar{x}}^2$ — lekin real data ke liye aisa nahi ho sakta kyunki $\frac{1}{n}\sum (x_i-\bar{x}{)}^2\ge 0$ hamesha. Lekin doosre parameters ke liye, MoM sach mein legal range se bahar ja sakta hai, jo iska notorious weakness hai.

Likelihood upar flat/bimodal hoti hai, isliye $\hat{\theta }$ poorly determined hota hai — data mein chhote changes isse bahut hilate hain (high variance). Ek sharp, peaked likelihood confident estimate karti hai; flat wali warn karti hai ki data $\theta$ ko barely constrain karta hai.

Nahi — ek point ke saath $\bar{x}=x_1$, isliye ${\hat{\sigma }}^2=0$. Tum ek single value se spread estimate nahi kar sakte; estimator degenerate ho jaata hai aur $n-1$ divisor toh zero se divide karega, honestly signal karta hua "data enough nahi hai."