Object detection (R-CNN, YOLO, SSD) — Hinglish notes

3.4.13 · AI-ML › Convolutional Neural Networks

Overview

Object detection woh task hai jisme ek image mein multiple objects ko simultaneously locate (kahan hai?) aur classify (kya hai?) kiya jata hai. Image classification se alag (jo sirf "kya hai?" ka jawab deta hai) ya semantic segmentation se alag (jo har pixel ko label karta hai lekin instances ko alag nahi karta), object detection har detected object ke liye bounding boxes aur class labels output karta hai.

Figure — Object detection (R-CNN, YOLO, SSD)

Fundamental challenge yeh hai: ek classification network ko ek localization + classification system mein badalna jo variable number of objects ke liye kaam kare.

Evolution: Teen Paradigms

1. R-CNN Family (Region-based Methods)

Yeh architecture WHY?

"Kahan dekhna hai" (region proposals) aur "kya hai yeh" (classification) ko alag karta hai
CNNs ki fixed-size image classification mein strength ka faida uthata hai
Search space ko saare possible boxes (lakho) se kam karke promising candidates tak le aata hai

HOW kaam karta hai:

$Input Image Selective Search {R_{1}, R_{2}, \dots, R_{n}} Warp to fixed size CNN SVM Class + Box$

Har region $R_{i}$ ko:

Ek fixed size par Warp kiya jata hai (AlexNet ke liye 227×227)
CNN se pass kiya jata hai ek feature vector extract karne ke liye
SVM se classify kiya jata hai
Ek bounding box regressor se refine kiya jata hai

\hat{G}_{x} \hat{G}_{y} \hat{G}_{w} \hat{G}_{h} = P_{w} \cdot d_{x} (P) + P_{x} = P_{h} \cdot d_{y} (P) + P_{y} = P_{w} \cdot exp (d_{w} (P)) = P_{h} \cdot exp (d_{h} (P))

Yeh formulas WHY?

Translation ( $d_{x}, d_{y}$ ): Offset ko box width/height ke fraction ke roop mein (scale-invariant)
Scale ( $d_{w}, d_{h}$ ): Exponential positive width/height ensure karta hai, learning aasaan karta hai (log-space zyada linear hoti hai)

Network $d_{x}, d_{y}, d_{w}, d_{h}$ seekhta hai yeh minimize karke:

L_{reg} = i \in {x, y, w, h} \sum smooth_{L_{1}} (d_{i} - t_{i})

jahan $t_{i}$ true transformation targets hain:

t_{x} = \frac{G _{x} - P _{x}}{P _{w}}, t_{y} = \frac{G _{y} - P _{y}}{P _{h}}, t_{w} = lo g \frac{G _{w}}{P _{w}}, t_{h} = lo g \frac{G _{h}}{P _{h}}

Fast R-CNN improvement: CNN ko 2000 baar run karne ki jagah, poori image ke liye ek baar features extract karo, phir har proposal ke liye fixed-size features nikalne ke liye RoI (Region of Interest) pooling use karo.

Image CNN Feature Map RoI Pool Fixed-size features FC layers Class + Box

Faster R-CNN improvement: Selective Search ko ek learned Region Proposal Network (RPN) se replace karo.

Har anchor ke liye:

Objectness score: $p_{obj} = σ (z_{obj})$ (kya koi object hai?)
Box refinement: $(d_{x}, d_{y}, d_{w}, d_{h})$ jaise upar bataya

Loss function dono ko combine karta hai:

$L = \frac{1}{N _{cls}} \sum_{i} L_{cls} (p_{i}, p_{i}^{*}) + λ \frac{1}{N _{reg}} \sum_{i} p_{i}^{*} L_{reg} (d_{i}, t_{i})$

Anchor boxes WHY?

Bina resizing ke multiple scales/aspect ratios handle karte hain
Box regression ke liye reference coordinates provide karte hain
Typical: 3 scales × 3 ratios = 9 anchors per position

NMS (Non-Maximum Suppression) ke baad, top 300 rakho. Har ek ke liye:

7×7 RoI pooled features extract karo
Do fully connected layers
Output: 21 classes (20 objects + background) + 4×21 box offsets

Yeh step kyun? 7×7 pooling features ko standardize karta hai chahe proposal ka size kuch bhi ho. 4×21 offsets class-specific box refinement allow karte hain (ek "car" box ko "person" box se alag adjustments chahiye ho sakte hain).

2. YOLO (You Only Look Once)

Single-stage WHY?

Speed: Hazaron ki jagah ek forward pass
Global context: Poori image dekhta hai, background false positives kam karta hai
Trade-off: Chote objects par thodi kam accuracy

HOW YOLO kaam karta hai:

Image ko $S \times S$ grid mein divide karo (jaise 7×7). Har grid cell predict karta hai:

$B$ bounding boxes (har ek mein 5 values: $x, y, w, h, confidence$ )
$C$ class probabilities

Output tensor: $S \times S \times (B \cdot 5 + C)$

YOLO v1 ke liye: $7 \times 7 \times (2 \cdot 5 + 20) = 7 \times 7 \times 30$

(x, y, w, h, confidence)

jahan:

$(x, y)$ : Box center grid cell ke relative (values 0–1)
$(w, h)$ : Box size image ke relative (values 0–1)
$confidence = P (object) \cdot IoU_{pred}^{truth}$

Yeh confidence formula WHY?

Agar koi object nahi: $P (object) = 0$ , confidence 0 honi chahiye
Agar object hai: Confidence dikhata hai box kitna fit hai (IoU)
Training mein, yeh supervised hai. Test time par, predicted hota hai.

Har grid cell yeh bhi predict karta hai:

P (C_{i} ∣ object)

Final class-specific confidence:

P (C_{i}) \cdot confidence = P (C_{i} ∣ object) \cdot P (object) \cdot IoU

L = λ_{coord} i = 0 \sum S^{2} j = 0 \sum B 1_{ij}^{obj} [(x_{i} - \overset{x}{^}_{i})^{2} + (y_{i} - \overset{y}{^}_{i})^{2}] + λ_{coord} i = 0 \sum S^{2} j = 0 \sum B 1_{ij}^{obj} [(w_{i} - \overset{w}{^}_{i})^{2} + (h_{i} - \hat{h}_{i})^{2}] + i = 0 \sum S^{2} j = 0 \sum B 1_{ij}^{obj} (C_{i} - \hat{C}_{i})^{2} + λ_{nobj} i = 0 \sum S^{2} j = 0 \sum B 1_{ij}^{noobj} (C_{i} - \hat{C}_{i})^{2} + i = 0 \sum S^{2} 1_{i}^{obj} c \in classes \sum (p_{i} (c) - \overset{p}{^}_{i} (c))^{2}

Har term WHY?

Localization loss (x, y): Box centers par MSE. Sirf us box ko penalize karo jo object ke liye "responsible" hai ( $1_{ij}^{obj} = 1$ agar cell $i$ mein box $j$ ka highest IoU hai).
Size loss (w, h): $w, h$ use karo kyunki bade boxes mein chote deviations utne matter nahi karte jitne chote boxes mein karte hain. Square root errors ko scales ke across zyada comparable banata hai.
Confidence loss (object hai): Penalize karo jab object ho lekin confidence kam ho.
Confidence loss (koi object nahi): Kam weight diya ( $λ_{noobj} = 0.5$ ) kyunki zyaatar boxes mein koi object nahi hota — gradient ko overwhelm hone se rokta hai.
Classification loss: Objects wale cells ke liye class probabilities par standard MSE.

$λ_{coord} = 5$ localization ki importance badhata hai.

Cell (3, 4) mein ek dog ka center hai:

Box 1 predict karta hai: $(0.6, 0.4, 0.3, 0.5, 0.85)$ (cell aur image ke relative)
Cell predict karta hai: $P (dog ∣ object) = 0.92$
Final: $0.92 \times 0.85 = 0.782$ confidence "dog" ke liye

Yeh step kyun? Cell (3, 4) responsible hai kyunki object ka center us cell mein hai. Highest IoU wala box (box 1) training mein penalize hota hai. 0.782 final score "kya koi object hai?" aur "woh kya hai?" ko combine karta hai.

YOLO v2/v3 improvements:

Anchor boxes (Faster R-CNN jaisa): Absolute coordinates ki jagah anchors se offsets predict karo
Multi-scale predictions: 3 alag feature map resolutions par detect karo (chote, medium, bade objects ke liye)
Better backbone: Darknet-53 (ResNet-jaisa skip connections ke saath)

3. SSD (Single Shot MultiBox Detector)

Multi-scale WHY?

Early layers (high resolution): Chote objects detect karte hain
Late layers (low resolution): Bade objects detect karte hain
Har layer ek scale range mein specialize karti hai

HOW kaam karta hai:

CNN ke alag-alag stages se 6 different feature maps lo (jaise 38×38, 19×19, 10×10, 5×5, 3×3, 1×1). Har map mein har location ke liye predict karo:

k default boxes \times (c + 4)

jahan $c$ = classes ki sankhya, 4 = box offsets.

Localization:

\overset{g}{^}_{c x} \overset{g}{^}_{cy} \overset{g}{^}_{w} \overset{g}{^}_{h} = d_{w} \cdot l_{c x} + d = d_{h} \cdot l_{cy} + d_{cy} = d_{w} \cdot exp (l_{w}) = d_{h} \cdot exp (l_{h})

Faster R-CNN box regression jaisa: default box $d$ ke relative offsets $l_{c x}, l_{cy}, l_{w}, l_{h}$ predict karo.

Classification:

c_{i} = softmax (z_{i})

har class $i$ ke liye (background including).

L (x, c l, g) = \frac{1}{N} (L_{conf} (x, c) + α L_{loc} (x, l, g))

jahan:

$N$ = matched default boxes ki sankhya
$α = 1$ (weight balance)

Localization loss (Smooth L1):

L_{loc} (x, l, g) = i \in Pos \sum m \in {c x, cy, w, h} \sum x_{ij}^{k} smooth_{L_{1}} (l_{i}^{m} - \overset{g}{^}_{j}^{m})

Confidence loss (Softmax):

L_{conf} (x, c) = - i \in Pos \sum x_{ij}^{p} lo g (\overset{c}{^}_{i}^{p}) - i \in Neg \sum lo g (\overset{c}{^}_{i}^{0})

jahan $\overset{c}{^}_{i}^{p} = \frac{e x p ( c _{i}^{p} )}{\sum _{p} e x p ( c _{i}^{p} )}$

Smooth L1 WHY?

smooth_{L_{1}} (x) = {0.5 x^{2} ∣ x ∣ - 0.5 if ∣ x ∣ < 1 otherwise

Jab error chota ho (|x| < 1): L2 jaisa behave karta hai (gradients vanish nahi karte)
Jab error bada ho (|x| ≥ 1): L1 jaisa behave karta hai (outliers ke liye kam sensitive, zyada stable)

Feature map 5×5 (late layer):

Receptive field: ~300 pixels
Default boxes: 6 boxes per location (scales 0.6-0.9, zyada aspect ratios)
Detect karne ke liye achha: cars, buses, bade objects

Yeh step kyun? Ek chota object (jaise 30×30 pixels) 5×5 map mein barely visible hoga (sirf 0.5 cells), lekin 38×38 map mein achhi tarah represent hoga (4-5 cells). Multi-scale matching ensure karta hai ki har object wahan predict ho jahan woh sabse zyada salient ho.

Sabhi Methods Mein Common Techniques

Non-Maximum Suppression (NMS)

Sabhi detectors kai overlapping boxes output karte hain. NMS har object ke liye sirf best box rakhta hai.

Yeh WHY kaam karta hai? High IoU wale boxes ek hi object detect kar rahe hain. Sabse confident wala rakho, redundant detections discard karo.

IoU (Intersection over Union) ki Derivation:

Do boxes $A$ aur $B$ diye hue:

IoU (A, B) = \frac{Area ( A \cap B )}{Area ( A \cup B )} = \frac{Area ( A \cap B )}{Area ( A ) + Area ( B ) - Area ( A \cap B )}

Yeh metric WHY?

IoU = 1: Perfect overlap (same box)
IoU = 0: Koi overlap nahi
IoU ∈ [0, 1]: Box similarity geometrically measure karta hai
Standard: IoU > 0.5 evaluation ke liye "match" maana jata hai

Hard Negative Mining

Zyaatar default boxes/anchors mein background hoti hai, jisse class imbalance hota hai.

Solution: Matching ke baad, boxes ko confidence loss se sort karo aur top-K negatives rakho (jaise 3:1 ratio negative:positive).

WHY? Iske bina, network har jagah "background" predict karna seekh leta hai aur kabhi objects detect nahi karta. Hard negatives sabse confusing backgrounds hain (high loss) — inpar focus karne se discrimination improve hoti hai.

Comparison: Kab Kya Use Karein

| Method | Speed (FPS) | Accuracy (mAP) | Best For | Trade-off | |--------|-------------|----------|--------| | Faster R-CNN | ~7 FPS | High | High-accuracy applications (medical, autonomous vehicles) | Slow, two-stage | | YOLO v3 | ~30 FPS | ~55% | Real-time video (surveillance, sports) | Chote objects miss karta hai | | SSD | ~25 FPS | ~68% | Balanced speed/accuracy (mobile apps) | Middle ground |

Yeh trade-offs WHY?

Two-stage (R-CNN): Region proposals → alag classification = zyada computation, better localization
Single-stage (YOLO/SSD): Direct prediction = faster, lekin precise localization seekhna mushkil
Multi-scale (SSD): Chote objects par YOLO se better, lekin YOLO se slower

Common Mistakes

Yeh sahi kyun lagta hai: Dono 0 aur 1 ke beech hain, dono detection quality se related hain.

Fix yeh hai:

Objectness/Confidence: "Kya yahan koi bhi object hai?" (binary)
Classification: "Kaunsa object hai?" (multi-class)
Final score = objectness × classification

YOLO mein: Har grid cell ek set class probabilities predict karta hai (sabhi boxes ke liye shared) aur per-box confidences. Class-specific scores ke liye inhe multiply karo.

Steel-man: Confusion isliye hoti hai kyunki Faster R-CNN mein, yeh RPN (objectness) aur R-CNN head (classification) mein combine ho jaate hain, jo separation ko kam obvious banata hai.

Yeh sahi kyun lagta hai: MSE standard regression loss hai.

Fix yeh hai: Do errors consider karo:

Box 1: 10×10 ko 15×15 predict kiya (error = 5)
Box 2: 100×100 ko 105×105 predict kiya (error = 5)

w, h par MSE se: Dono ka loss = 25.

Lekin Box 2 ka error sirf 5% relative error hai, jabki Box 1 ka 50% hai!

$w, h$ par MSE se:

Box 1: $10 \approx 3.16$ , $15 \approx 3.87$ , error ≈ 0.71, loss ≈ 0.5
Box 2: $100 = 10$ , $105 \approx 10.25$ , error ≈ 0.25, loss ≈ 0.06

Ab Box 1 ka loss 8× bada hai, jo correctly bade relative error ko reflect karta hai.

Steel-man: "Absolute error equally matter karna chahiye" wali intuition similar scale ki quantities ke liye kaam karti hai, lekin bounding boxes orders of magnitude span karte hain. Square root range compress karta hai, loss ko relative error ke proportional banata hai.

Yeh sahi kyun lagta hai: Zyada anchors → zyada coverage → ground truth match hone ki better chance.

Fix yeh hai:

Computational cost: Anchors ki sankhya ke saath linear (slower training, inference)
Label assignment ambiguity: Bahut zyada anchors ke saath, multiple anchors same object ko equally match karte hain → inconsistent gradients
Overfitting: Anchor-specific patterns seekhne ke liye zyada parameters

Sweet spot: 3 scales × 3 aspect ratios = 9 anchors zyaatar object variations capture karta hai. Papers mein empirically validated hai.

Steel-man: Intuition "more capacity = more learning power" se aati hai, jo model depth/width ke liye kaam karta hai. Lekin anchors hand-designed priors hain, learned features nahi — zyada hamesha better nahi, smart selection zaroori hai.

Active Recall

Recall Object detection ko ek 12 saal ke bachche ko samjhao

Imagine karo tum "Where's Waldo?" khel rahe ho lekin computer kar raha hai.

Purana tarika (R-CNN): Computer kehta hai, "Dekho, kya Waldo is box mein hai... nahi. Is box mein? Nahi. Is mein? Shayad..." Woh ek-ek karke 2000 boxes check karta hai! Bahut slow.

Naya tarika (YOLO): Computer picture ko ek grid mein divide karta hai, jaise checkerboard. Har square chillata hai, "Mujhe yahan ek insaan dikha! Confidence: 85%!" aur "Mujhe yahan ek kutta dikha! 92%!" Sab ek saath! Bahut fast.

Aur bhi better tarika (SSD): YOLO jaisa, lekin multiple grids use karta hai — badi cheezein (gaadiyaan) ke liye ek mota grid aur choti cheezein (chehre) ke liye ek barik grid. Jaise choti details ke liye magnifying glass aur door ki cheezein dekhne ke liye binoculars use karna.

Computer hazaron labeled pictures dikhane se seekhta hai: "Billi yahan hai, uske around ek box banao." Dheere-dheere, woh billi-shapes, gaadi-shapes, insaan-shapes seekhta hai. Jab galti karta hai, hum batate hain, "Box thoda baya hona chahiye" ya "Yeh billi nahi, kutta hai!" aur woh adjust karta hai.

Jadoo yeh hai: Woh sirf kya hai nahi bolta (classification), balki kahan hai bhi bolta hai (localization) — har object ke around boxes draw karta hai!

Ya yun yaad rakho: Refined, Yappy, Scalable

Connections

Convolutional Neural Networks: Backbone feature extractors (VGG, ResNet, Darknet)
Image Classification: Object detection classification ko localization ke saath extend karta hai
Semantic Segmentation: Detection mein pixel-level labeling vs. box-level
Transfer Learning: Detectors aksar ImageNet-pretrained backbones se start karte hain
Anchor Boxes: Faster R-CNN, YOLO v2+, SSD mein core technique
Non-Maximum Suppression: Duplicate detections remove karne ke liye post-processing
Intersection over Union (IoU): Evaluation metric aur matching criterion
Batch Normalization: Deep detector backbones mein training stabilize karta hai
Data Augmentation: Detection ke liye critical (random crops, flips, color jitter)
Instance Segmentation: Detection se agle step (Mask R-CNN)

Flashcards

#flashcards/ai-ml

Image classification aur object detection mein fundamental difference kya hai? :: Image classification poori image ke liye ek single label output karta hai. Object detection multiple bounding boxes with class labels output karta hai, "kya" (classification) aur "kahan" (localization) dono solve karta hai.

R-CNN mein do stages kya hain?

1) Region proposal (Selective Search ~2000 candidate boxes generate karta hai), 2) Classification (CNN features extract karta hai, SVM har region classify karta hai).

YOLO size loss mein square roots kyun use karta hai?

Loss ko relative error ke liye zyada sensitive banane ke liye. 10-pixel box par 5-pixel error, 100-pixel box se zyada bura hai. Square roots range compress karte hain taaki loss percentage error ke saath scale kare.

Faster R-CNN mein anchor boxes ka purpose kya hai?

Anchor boxes multiple scales aur aspect ratios par reference coordinates provide karte hain. Network absolute coordinates ki jagah anchors se offsets predict karta hai, learning aasaan hoti hai aur size variation handle hoti hai.

Faster R-CNN mein bounding box center offset formula derive karo.

Proposal

P

aur ground truth

G

diye hue:

\hat{G}_{x} = P_{w} \cdot d_{x} + P_{x}

jahan

d_{x}

learned offset hai. Yeh offset ko scale-invariant banata hai (box width ka fraction).

YOLO mein confidence score kya hai?

confidence = P (object) \times IoU_{pred}^{truth}

. Yeh represent karta hai ki object exist karne ki probability aur box kitna fit hai. Training mein supervised; test time par predicted.

SSD multiple feature maps kyun use karta hai?

Early feature maps (high resolution) chote objects detect karte hain. Late feature maps (low resolution, bade receptive fields) bade objects detect karte hain. Multi-scale predictions objects ko appropriate scales se match karti hain.

Non-Maximum Suppression kya hai aur kyun zaroori hai?

NMS duplicate detections remove karta hai. Algorithm: Boxes ko confidence se sort karo, iteratively highest-confidence box rakho aur sabhi overlapping boxes (IoU > threshold) remove karo. Zaroori hai kyunki detectors har object ke liye kai overlapping boxes output karte hain.

SSD mein Hard Negative Mining kya hai?

Zyaatar default boxes mein background hoti hai (class imbalance). Hard negative mining highest loss wale negatives (sabse confusing backgrounds) select karta hai taaki training mein 3:1 negative:positive ratio maintain ho.

IoU (Intersection over Union) derive karo.

IoU (A, B) = \frac{Area ( A \cap B )}{Area ( A \cup B )} = \frac{Area ( A \cap B )}{Area ( A ) + Area ( B ) - Area ( A \cap B )}

. Box overlap measure karta hai; 1 = perfect match, 0 = koi overlap nahi.

Localization mein Smooth L1 loss kyun use hoti hai?

smooth_{L_{1}} (x) = 0.5 x^{2}

agar

∣ x ∣ < 1

, warna

∣ x ∣ - 0.5

. Chote errors ke liye, L2 jaisa behave karta hai (gradients vanish nahi karte). Bade errors ke liye, L1 jaisa behave karta hai (outliers ke liye kam sensitive, zyada stable training).

One-stage aur two-stage detectors mein trade-off kya hai?

Two-stage (Faster R-CNN): Zyada accurate, better localization, lekin slower (~7 FPS). One-stage (YOLO, SSD): Faster (25-30 FPS), real-time capable, lekin chote objects par kam accuracy.

YOLO grid cells mein detection task kaise divide karta hai?

Har grid cell B bounding boxes (x, y, w, h, confidence ke saath) aur C class probabilities predict karta hai. Final output:

S \times S \times (B \cdot 5 + C)

tensor. Training mein sirf wahi cells penalize hoti hain jinmein object centers hain.

Faster R-CNN mein Region Proposal Network (RPN) kya hai?

RPN CNN feature maps par slide karta hai, har position par predict karta hai: k anchor boxes with objectness scores aur box refinements. Slow Selective Search ko ek learned proposal generator se replace karta hai. Loss classification (object/nahi) aur regression combine karta hai.

YOLO objectness aur class probability ko multiply kyun karta hai?

Final class-specific score =

P (class ∣ object) \times P (object) \times IoU

. "Kuch bhi hai kya" (objectness) ko "kya hai yeh" (classification) aur "box kitna fit hai" (IoU) ke saath combine karta hai. Object presence ko marginalize out karta hai.