Support vectors interpretation — Hinglish notes

2.4.6 · AI-ML › SVM, Naive Bayes & Probabilistic Models

Support vector kya hota hai?

WHAT yeh kehta hai: in points ka functional margin exactly $1$ hota hai — yeh dashed "margin lines" pe baithte hain, comfortably door nahi.

Kyon sirf support vectors matter karte hain — scratch se derive karna

Hum hard-margin SVM ke primal problem se shuru karte hain. WHY yeh objective? Hum classes ke beech sabse chauda "street" chahte hain; street ki half-width $1/∥ w ∥$ hai, isliye margin maximize karna = $∥ w ∥^{2}$ minimize karna.

$min_{w, b} \frac{1}{2} ∥ w ∥^{2} s.t. y_{i} (w^{⊤} x_{i} + b) \geq 1 \forall i .$

HOW hum constraints laate hain: multipliers $α_{i} \geq 0$ ke saath Lagrangian banao:

$L (w, b, α) = \frac{1}{2} ∥ w ∥^{2} - \sum_{i} α_{i} [y_{i} (w^{⊤} x_{i} + b) - 1] .$

Minus sign kyun? Hum ek aisa penalty subtract karte hain jo constraint violate hone pe badhta hai, isliye min–max saddle point constraints enforce karta hai.

Stationarity — derivatives zero karo:

$\frac{\partial L}{\partial w} = 0 \Rightarrow w = \sum_{i} α_{i} y_{i} x_{i} .$

Yeh step kyun? Yahi toh punch line hai: $w$ ek training points ka weighted sum hai, $α_{i}$ se weighted.

$\frac{\partial L}{\partial b} = 0 \Rightarrow \sum_{i} α_{i} y_{i} = 0.$

Ab KKT complementary slackness condition (interpretation ka dil):

$α_{i} [y_{i} (w^{⊤} x_{i} + b) - 1] = 0.$

HOW padhen ise: har point ke liye ya $α_{i} = 0$ ya bracket $= 0$ .

Agar ek point strictly apni side ke andar hai ( $y_{i} (w^{⊤} x_{i} + b) > 1$ ), toh bracket $\neq = 0$ , isliye hum majboor hain $α_{i} = 0$ rakhne ke liye → woh $w$ mein kuch contribute nahi karta.
Sirf margin pe wale points ( $y_{i} (w^{⊤} x_{i} + b) = 1$ ) ka $α_{i} > 0$ ho sakta hai → yahi support vectors hain.

Bias $b$ recover karna

WHY: stationarity se $w$ milta hai par $b$ nahi. Koi bhi support vector $x_{s}$ use karo (jahan margin exactly 1 ke barabar hai): $y_{s} (w^{⊤} x_{s} + b) = 1 \Rightarrow b = y_{s} - w^{⊤} x_{s} (using y_{s} = \pm 1, y_{s}^{2} = 1) .$ Practice mein numerical stability ke liye saare SVs pe average karo.

Soft margin: support vector ke teen flavors

Slack $ξ_{i}$ aur box constraint $0 \leq α_{i} \leq C$ ke saath:

Location	Condition	$α_{i}$
Margin se pare (safe)	$y_{i} f (x_{i}) > 1$	$α_{i} = 0$ (SV nahi)
Bilkul margin pe	$y_{i} f (x_{i}) = 1$	$0 < α_{i} < C$
Margin ke andar / misclassified	$y_{i} f (x_{i}) < 1$	$α_{i} = C$

Worked examples

Common mistakes

Flashcards

Algebraic condition kya hai jo ek support vector define karta hai?

Iska Lagrange multiplier

α_{i} > 0

(equivalently yeh margin pe/andar hai).

Non-support vectors decision boundary ko affect kyun nahi karte?

KKT complementary slackness

α_{i} = 0

force karta hai un points ke liye jo strictly margin se pare hain, isliye woh

w = \sum α_{i} y_{i} x_{i}

mein kuch contribute nahi karte.

w

ko training data ke terms mein likhो.

w = \sum_{i} α_{i} y_{i} x_{i}

, sirf support vectors pe nonzero.

Soft margin mein

α_{i} = C

kya indicate karta hai?

Point ek margin violator ya misclassified hai (margin ke andar / wrong side).

Support vector

x_{s}

b

recover kaise karo?

b = y_{s} - w^{⊤} x_{s}

(kyunki margin pe

y_{s} (w^{⊤} x_{s} + b) = 1

Support vectors ka bahut bada fraction kya suggest karta hai?

Overfitting / chhoti margin / poor generalization; SV fraction LOO error bound karta hai.

Kernelized decision rule kya hai?

f (x) = sign (\sum_{i \in S V} α_{i} y_{i} K (x_{i}, x) + b)

Recall Feynman: 12-saal ke bachche ko explain karo

Tum do teams ke beech tug-of-war khel rahe ho, aur beech mein chalk se ek line kheenchte ho. Sirf woh bacche jo har team ke sabse aage khade hain, line ke sabse paas, decide karte hain line kahan jayegi. Peechhe wale bacche line pe bilkul nahi kheenchte. Woh aage wale bacche "support vectors" hain — aur agar saare peechhe wale bacche ghar chale jayein, toh line ek inch bhi nahi hilegi.

Connections

Maximal margin classifier — jahan se $1/∥ w ∥$ margin aata hai
SVM dual formulation — $α_{i}$ yahin rehte hain
KKT conditions — complementary slackness interpretation drive karta hai
Kernel trick — $x_{i}^{⊤} x \to K$ SV sum ko rakhta hai
Soft margin & slack variables — $α_{i} = C$ case
Bias-variance tradeoff — SV count ek complexity signal ke roop mein