4.9.23 · D1Multiple regression
Multiple regression data points ke ek cloud ko leta hai aur unke beech ek flat surface slide karta hai — taaki surface se upar-neeche ke gaps (differences) jitne ho sake utne chote rahein. Baaki sab — matrices, transposes, inverses — sirf ek tarah ki bookkeeping hai jo computer ko allow karti hai ki woh kai saari input variables ke liye woh best surface dhundh sake.
Is page par assume kiya gaya hai ki tumne kuch bhi nahi dekha. Hum har woh symbol naam denge jo parent note Multiple regression use karta hai, uske peeche ki picture draw karenge, aur batayenge ki is topic ko uske bina kaam kyon nahi chalega. Upar se neeche padho; har block mein sirf wahi symbols use hain jo uske upar already explain ho chuke hain.
Kisi bhi formula se pehle, hume ek observation ke baare mein baat karni hai aur yeh samajhna hai ki hum unhe kaisa count karte hain.
Definition Observation aur indexi observation measurements ki ek poori row hai: ek plant, ek student, ek ghar. Hum unhe line up karte hain aur har ek ko ek number dete hain. Chhota sa symbol i counter hai jo kehta hai "main kaun si row ki baat kar raha hoon". Agar humare paas n i 1 , 2 , 3 , … , n
n total number of observations (cloud mein kitne dots hain).y i i x ij i j do subscripts: pehla kehta hai kaun si row , doosra kehta hai kaun sa input .
Yeh kyun chahiye: regression har ek row ke liye ek prediction ko ek truth se compare karta hai, phir mistakes ko add karta hai. Counter ke bina hum yeh nahi keh sakte "row 3 par mistake kya thi".
Worked example Figure s01 kaise padheinLeft mein ek raw data table hai: har row ek observation hai, i Right mein wahi rows dots ban jaati hain ek picture mein, jisme horizontal axis = predictor x 1 y i = 2 x 2 , 1 = 2 y 2 = 65 Poora point yeh hai: numbers ki ek table aur dots ka ek cloud — yeh ek hi cheez hain — regression cloud par kaam karta hai.
y x 1 , … , x p
Response y Predictor x j y p number of predictors (kitne input columns hain).
Yeh sirf measured numbers hain — abhi kuch choose ya fit nahi kiya gaya hai. Aage hum inhe combine karenge, lekin pehle hume woh numbers introduce karne honge jinhe hum tune kar sakte hain.
β β j aisa number hai jo hum choose kar sakte hain fit ko accha banane ke liye.
β 0 intercept : predicted y har input zero ho — jahan surface vertical axis ko cross karti hai.β j j ≥ 1 x j slope : x j y
Ab jab ki hume fixed inputs x j β j
Definition "Linear" ka matlab kya hai (woh zaroori word)Predictors ka ek linear combination yeh hota hai: har predictor x j β j β 0 prediction = β 0 + β 1 x 1 + β 2 x 2 + ⋯ + β p x p koi bending nahi : na x 2 x x 1 x 2 flat surface hoti hai, curved nahi.
Topic ko yeh kyun chahiye: flatness hi woh cheez hai jo maths ko ek clean formula mein solvable banati hai. Ek curved fit ka koi simple closed-form answer nahi hota.
ε ε i error hai: y i explain nahi kar sakti — measurement noise, luck, hidden causes. Hum assume karte hain ki yeh average ho ke zero ho jata hai:
E [ ε i ] = 0 E [ ⋅ ] average value ". Yeh kehna ki average error zero hai, matlab surface systematically na zyada upar hai na zyada neeche.
Dono kyun matter karte hain: β solve karte hain; ε chota karne ki koshish karte hain. Poora game yeh hai ki β ε
Error ε i β measure kar sakte hain woh hai residual .
Definition Residual aur squared residualResidual real point aur humari surface ke beech ka vertical gap hai:
e i = y i − y ^ i y ^ i i prediction hai. Chhoti si hat ^ square karte hain, e i 2 S = ∑ i = 1 n e i 2 ∑ i = 1 n add karo , i n
Intuition Square kyun karein, sirf gap kyun nahi lein?
Raw gap negative ho sakta hai (point surface ke neeche); positives aur negatives cancel ho jaate aur bad fits chhup jaate. Squaring har gap ko positive banata hai.
Squaring ek bade miss ko kai chote misses se zyada punish karta hai, isliye surface wild errors avoid karti hai.
Squares smooth hote hain — tum calculus se inke bottom tak slide kar sakte ho. Absolute values mein ek sharp corner hoti hai aur solve karna mushkil hota hai.
Worked example Figure s02 kaise padheinBlack line humari fitted surface hai (1D mein yeh sirf ek line hai). Black dots real data hain; red vertical segments jo har dot ko line se jodte hain woh residuals e i = y i − y ^ i vertical hain — regression gap ko sirf y squared red lengths ka total sabse chota na ho jaye. Yahi Ordinary Least Squares ka dil hai.
n
Ek vector numbers ka ek single column hota hai. y y = y 1 y 2 ⋮ y n
Definition Parameter vectorβ ε parameter vector , aur bacha hua wiggles ko doosre column mein, error vector :
β = β 0 β 1 ⋮ β p ( length p + 1 ) , ε = ε 1 ε 2 ⋮ ε n ( length n ) . β ek saath saare coefficients aur bold ε ek saath saari errors — same numbers pehle jaisi, bas columns mein gather kiye gaye.
Definition Matrix aur design matrixX matrix numbers ka ek rectangle hota hai (rows × columns). Design matrix X leading column of 1's :
X = 1 1 ⋮ 1 x 11 x 21 x n 1 ⋯ ⋯ ⋯ x 1 p x 2 p ⋮ x n p n p + 1
Intuition 1's ka column kyun?Har 1 intercept β 0 β 0 har row ki prediction mein appear karta hai. Iske bina, surface origin se guzarni padti hai (sab inputs zero hon toh 0 predict karo) — jo almost hamesha galat hota hai.
y X β ε n y = X β + ε
Poori derivation teen operations par chalti hai. Hum har ek ko uski picture ke saath define karte hain.
Definition Matrix–vector productX β i X i β ( X β ) i = 1 ⋅ β 0 + x i 1 β 1 + ⋯ + x i p β p = y ^ i X β ek saath saari predictions hai. Isliye humne stack karne ki zehmat uthayi.
X ⊤ Transpose ek matrix ko uske diagonal ke upar se flip karta hai: rows columns ban jaati hain. Agar X n × ( p + 1 ) X ⊤ ( p + 1 ) × n Yeh kyon aata hai: X X ⊤ X y
∥ ⋅ ∥ v squared length hai
∥ v ∥ 2 = v ⊤ v = v 1 2 + v 2 2 + ⋯ + v n 2 n ∥ y − X β ∥ 2 residual vector ki squared length . Error minimize karna ek arrow ko chota karna hai.
Worked example Figure s03 kaise padheinShaded flat sheet X column space hai — har prediction X β y bahar nikalti hai. Red arrow residual y − X β perpendicular khadi hoti hai — woh perpendicular drop best fit hai, aur Orthogonal Projection ka seed.
Definition Dot product aur "perpendicular"Do vectors a , b dot product hai a ⊤ b = a 1 b 1 + ⋯ + a n b n zero ho, toh do vectors perpendicular (right angle par) hain .
Ab hum woh condition earn karte hain jo figure s03 ne geometrically dikhaayi thi. Yaad karo S ( β ) = ∥ y − X β ∥ 2
Definition Matrix inverseEk square matrix A inverse A − 1 A − 1 A = I I I v = v divide karne ka .
Common mistake "Har matrix invert ho sakti hai."Kyun sahi lagta hai: har non-zero number se divide kiya ja sakta hai.
Fix: ek matrix ka koi inverse nahi hota jab uske columns redundant hote hain (ek doosron ka combination hota hai). X ⊤ X Multicollinearity & VIF ka trap. Poori story ke liye Matrix Inverse dekho.
Last symbols measure karte hain ki fit kitni achi hai.
y ˉ y ˉ average hai: y ˉ = n 1 ∑ i y i sabse dumb possible predictor jo har input ko ignore karta hai.
Definition Teen sums of squares
S S T = ∑ i ( y i − y ˉ ) 2 y total spread uske mean ke aas-paas (kitna variation hai взагалі).S S E = ∑ i ( y i − y ^ i ) 2 error (jo surface miss kar gayi).S S R = ∑ i ( y ^ i − y ˉ ) 2 regression / explained part (jo surface ne capture kiya).
Least-squares fit ke liye yeh S S T = S S R + S S E
Ratio R 2 = S S R / S S T Coefficient of Determination .
Observation index i and n
Response y and predictors x
Beta dials and linear combination
Residual and sum of squares S
Vectors beta and epsilon and design matrix X
Matrix product transpose norm
Dot product and perpendicular
Mean and sums of squares SST SSE SSR
Multiple Regression estimator
Har arrow ka matlab hai "target samajhne se pehle source box samajhna zaroori hai." Final box parent topic Multiple regression hai, jo Gauss–Markov Theorem aur Positive Semidefinite Matrices ka darwaza bhi kholti hai.
Left side padho, zor se jawab do, phir reveal karo.
y i i Kaun sa observation (row) mean hai; yeh 1 n
"Linear" fitted surface ko kya force karta hai? Flat — koi curves nahi, inputs ke squared ya product terms nahi.
Residual e i Real point aur fitted surface ke beech ka vertical gap y i − y ^ i
Residuals ko add karne se pehle square kyun karte hain? Taaki positives aur negatives cancel na karein, bade misses ko punish kiya jaye, aur total calculus ke liye smooth rahe.
X Intercept β 0
Parameter vector β Woh column jo saare coefficients β 0 , … , β p p + 1
Transpose X ⊤ Rows ko columns mein flip karta hai taaki multiplication ke liye shapes align ho sakein.
Dot product ke terms mein do vectors perpendicular kab hote hain? Jab unka dot product a ⊤ b
Normal equations geometrically kya condition state karte hain? Residual X X ⊤ ( y − X β ^ ) = 0
Left se ( X ⊤ X ) − 1 β ^ Kyunki ( X ⊤ X ) − 1 ( X ⊤ X ) = I I β ^ = β ^
X ⊤ X Jab predictor columns redundant hon (perfect collinearity).
Least-squares fit ke teen sums of squares ko jodne wali identity batao. S S T = S S R + S S E
