WHY "partial"? In simple regression a slope mixes the direct effect of xand whatever else x correlates with. With several predictors in the model, βj measures xj's effect after the others have already explained what they can — its unique contribution.
Goal: choose β to minimise the sum of squared residuals (vertical gaps):
S(β)=∑i=1nεi2=∥y−Xβ∥2=(y−Xβ)⊤(y−Xβ).
Why squares? They are smooth (differentiable), penalise big misses harder, and give a unique closed-form answer.
Step 1 — expand.S=y⊤y−2β⊤X⊤y+β⊤X⊤Xβ.Why this step? The two cross terms y⊤Xβ and β⊤X⊤y are equal scalars (transpose of each other), so they combine into the −2 term.
Step 2 — differentiate w.r.t. the vector β and set to zero.
Using ∂β∂(a⊤β)=a and ∂β∂(β⊤Aβ)=2Aβ (for symmetric A=X⊤X):
∂β∂S=−2X⊤y+2X⊤Xβ=0.Why set to zero? At a minimum the gradient vanishes; S is convex (a quadratic with PSD Hessian 2X⊤X), so this stationary point is the global minimum.
Step 3 — the normal equations.X⊤Xβ^=X⊤y
Step 4 — solve (if X⊤X is invertible, i.e. predictors not perfectly collinear):β^=(X⊤X)−1X⊤y
Data: predict exam score y from hours studied x1 and hours slept x2.
i
x1
x2
y
1
1
6
50
2
2
7
65
3
3
5
70
4
4
8
85
We build X with a leading 1-column, form X⊤X (a 3×3 matrix) and X⊤y, then solve β^=(X⊤X)−1X⊤y.
Why include the 1-column? Without it we force the surface through the origin — usually wrong.
Why solve the linear system rather than eyeball? With 2+ predictors there's no simple ratio; the predictors share information, so all slopes must be solved jointly.
(The VERIFY block below solves this exact system symbolically.)
Suppose a fit gives y^=5+8x1+3x2 where x1=hours studied, x2=hours slept.
β^1=8: one extra hour of study adds 8 points, holding sleep constant. Why "holding constant" matters: if studious students also sleep more, a simple regression of y on x1 alone would mistakenly credit study with sleep's benefit too.
Predict for x1=3,x2=7: y^=5+24+21=50. Why: just plug in; the model is linear so it's a weighted sum.
Imagine guessing how tall a plant will grow from how much water and how much sunlight it gets. You collect many plants and draw the best flat "ramp" through the dots so the up-and-down distances to the ramp are tiniest. The ramp's steepness in the water direction tells you how much taller a plant gets per cup of water — pretending sunlight stays the same — and the steepness in the sunlight direction does the same for light. Multiple regression is just the math for finding that best ramp.
Multiple regression ka idea simple hai: tum ek output y ko predict karna chahte ho lekin ek nahi, kai predictors (x1,x2,…) se. Jaise exam score ko study hours aur sleep hours dono se predict karna. Simple regression me ek line fit hoti thi; yahan ek flat plane (hyperplane) fit hota hai us data-cloud ke beech me, taaki har point se plane tak ka vertical gap (residual) sabse chhota ho. Isi ko hum "least squares" kehte hain — squares isliye ki woh smooth hote hain aur ek hi unique answer dete hain.
Formula yaad rakhna: β^=(X⊤X)−1X⊤y. Ye derive hota hai sum of squared errors ko minimise karke — gradient ko zero set karo, normal equations X⊤Xβ^=X⊤y milte hain, phir invert karke beta solve karo. Geometry me yeh bas projection hai: tum y ko X ke column space pe perpendicular drop kar rahe ho.
Sabse important concept hai partial slope. β1 ka matlab hai "x1 ek unit badha to y kitna badhega, baaki sab predictors constant rakhte hue". Isliye yeh simple regression se zyada honest hai — kyunki agar do predictors aapas me related hain, simple regression unka effect mix kar deta hai.
Do bade traps: (1) R2 hamesha badhta hai jab tum naya predictor add karte ho, isliye adjusted R2 dekho. (2) Agar predictors aapas me bahut correlated hain (multicollinearity), to X⊤X almost non-invertible ho jaata hai aur coefficients pagal ho jaate hain. Aur yaad rakho — regression sirf association batata hai, causation nahi.
