1.4.5 · AI-ML › Python & Scientific Computing

Overview

Broadcasting NumPy ka ek mechanism hai jo different shapes wale arrays par element-wise operations karne deta hai, bina explicitly data ko replicate kiye. Yeh fast, memory-efficient vectorized code ka secret hai—lekin tabhi, jab tum samjho ki yeh kab aur kaise kaam karta hai.

Ise painting ki tarah socho: tumhare paas ek bada canvas hai (large array) aur ek chhota stamp (small array). Broadcasting tumhe exactly batata hai kab aur kaise tum us stamp ko canvas par "tile" kar sakte ho—bina stamp ki physical copies banaye.

The Three Broadcasting Rules

1. Rule 1 (Equality): Dimensions equal hain, YA
2. Rule 2 (Singleton): Unmen se ek 1 hai, YA
3. Rule 3 (Missing): Ek dimension exist hi nahi karti (array mein kam dimensions hain)

Agar saare dimension pairs mein se kam se kam ek rule satisfy ho, to broadcasting succeed ho jaati hai. Warna, tumhe ValueError: operands could not be broadcast together milega.

Rightmost Dimension Se Kyun Shuru Karein?

Yeh is baat ke saath align karta hai ki NumPy arrays ko memory mein kaise store karta hai (row-major order). Rightmost dimension sabse fast change hoti hai jab tum iterate karte ho. Right se start karna memory access patterns ko predictable aur cache-friendly banata hai.

Step 1: Shapes ko right-align karo (missing dimensions ko left mein 1 se pad karo)

A:      a₁  a₂  a₃  a₄
B:   1  b₁  b₂  b₃  b₄  (if B has fewer dims)

Step 2: Har dimension pair (aᵢ, bᵢ) ke liye:

• Agar aᵢ == bᵢ: Output dimension aᵢ hai
• Agar aᵢ == 1: Output dimension bᵢ hai (A ko stretch karo)
• Agar bᵢ == 1: Output dimension aᵢ hai (B ko stretch karo)
• Warna: Error

Step 3: Result shape aligned shapes ka element-wise maximum hai.

$\text{result.shape}[i]=\max (A.\text{shape}[i],B.\text{shape}[i])$

jahan hum missing dimensions ko 1 maante hain.

First Principles Se Derivation: Yeh kaam kyun karta hai? Element-wise operations ko elements ke pairs chahiye. Broadcasting ek virtual grid banata hai jahan:

• Agar dimension size 1 hai, to woh single slice axis ke along repeat hoti hai
• Agar dimension missing hai, to array us axis mein size 1 ki tarah behave karta hai
• Output ko har dimension mein badi size accommodate karni chahiye

Worked Examples

A = np.array(1, 2, 3, [4, 5, 6]]) # shape (2, 3) b = 10 # shape () — a 0-D scalar

result = A + b

Shape analysis:

A (2, 3)

b: shape () — has NO axes at all

Missing axes get padded with 1 on the LEFT:

Aligned: (2, 3)

(1, 1) → stretches to (2, 3)

Result: (2, 3)

**Yeh step kyun?** Scalar `b` ka shape `()` hai — yeh **zero-dimensional** hai (koi bhi axes nahi), length 1 ka 1-D array nahi. Broadcasting ke dauran, NumPy missing axes ko left mein 1 se pad karta hai, isliye `()` effectively `(1, 1)` ban jaata hai jab `(2, 3)` ke against align hota hai:
- Dimension -1 (rightmost): `3 vs missing → 1` → Rule 3 phir Rule 2, stretch karke 3
- Dimension -2: `2 vs missing → 1` → Rule 3 phir Rule 2, stretch karke 2

Result: A ke har element mein 10 add ho jaata hai.

> [!example] Example 2: Matrix + Column Vector (Normalization Pattern)
> ```python
> X = np.array([[1, 2, 3],
>               [4, 5, 6],
>               [7, 8, 9]])     # shape (3, 3)

mean = np.array([[2],
                 [5],
                 [8]])         # shape (3, 1)

centered = X - mean
# Shape analysis:
# X:    (3, 3)
# mean: (3, 1)
# Dimension -1: 3 vs 1 → Rule 2, stretch mean's columns to 3
# Dimension -2: 3 vs 3 → Rule 1, already match
# Result:       (3, 3)

Yeh step kyun? Column vector (3, 1) mein last dimension mein ek singleton hai. Broadcasting us column ko horizontally 3 baar repeat karta hai (bina memory copy kiye). X ki har row mein se uska corresponding mean value subtract ho jaata hai.

Mental model: mean ko ek stencil ki tarah socho jo X ki har row ke along slide karta hai.

Reshape to force outer product structure

a_col = a[:, np.newaxis] # shape (3, 1) outer = a_col * b # b remains (4,)

Shape analysis:

a_col: (3, 1)

b: (4,) → right-aligned: (1, 4) effectively

Actually, (4,) right-aligns as:

(3, 1)

( 4) ← treated as (1, 4) when aligned

Dimension -1: 1 vs 4 → stretch a_col

Dimension -2: 3 vs missing (1) → stretch b

Result: (3, 4)

**Yeh step kyun?** `a` ko `(3, 1)` mein reshape karke, hum broadcasting ke liye explicit dimensions create karte hain. `(3, 1)` array right mein stretch karta hai, `(4,)` array left mein stretch karta hai (`(1, 4)` ki tarah treat hota hai), aur saare pairwise products create hote hain.

Result:

10 20 30 40 [ 20 40 60 80] [ 30 60 90 120]]

> [!example] Example 4: Broadcasting Failure
> ```python
> A = np.array([[1, 2, 3]])  # shape (1, 3)
> B = np.array([[1],
>               [2]])         # shape (2, 1)

result = A + B  # This WORKS! → (2, 3)

# But this fails:
C = np.array([[1, 2]])     # shape (1, 2)
D = np.array([[1, 2, 3]])  # shape (1, 3)

# result = C + D  # ValueError!
# Dimension -1: 2 vs 3 → NEITHER equal, singleton, nor missing

Pehla kyun kaam karta hai? (1, 3) aur (2, 1):

• Dimension -1: 3 vs 1 → Rule 2, B ko stretch karo
• Dimension -2: 1 vs 2 → Rule 2, A ko stretch karo
• Result: (2, 3)

Doosra kyun fail karta hai? (1, 2) aur (1, 3):

• Dimension -1: 2 vs 3 → Koi bhi rule apply nahi hota (equal nahi, koi bhi 1 nahi)

X = np.random.rand(100, 3)  # 100 samples, 3 features
weights = np.array([0.5, 0.3, 0.2])  # shape (3,)
 
# This WORKS as expected: (100, 3) and (3,)
# dim -1: 3 vs 3 → match! Each column scaled by its weight
weighted = X * weights

Kyun sahi lagta hai: Tum soch rahe ho "3 features × 3 weights = match!", jo tabhi sahi hai jab 3 rightmost dimension par land kare.

Mistake wala version (transposed data):

X = np.random.rand(3, 100)  # 3 features, 100 samples (transposed)
weights = np.array([0.5, 0.3, 0.2])  # shape (3,)
 
# X * weights  → ValueError!
# Right-align: (3, 100) and (3,) → (3, 100) vs (1, 3) effectively
# dim -1: 100 vs 3 → NO rule applies → ValueError

Mistake kyun hoti hai: Broadcasting hamesha right se align karta hai. Yahan (3,) weights last dimension 100 ke against align hote hain, 3 ke against nahi. Kyunki 100 ≠ 3 aur koi bhi 1 nahi hai, NumPy ValueError raise karta hai.

Fix: Weights ko column ki tarah reshape karo taaki woh correct axis ke along broadcast karein:

result = X * weights[:, np.newaxis]  # weights becomes (3, 1)
# (3, 100) and (3, 1) → dim -1: 100 vs 1 (stretch), dim -2: 3 vs 3 (match)
# Result: (3, 100) ✓

Operation likhne se pehle hamesha mentally right-alignment visualize karo.

A = np.array([[[1, 2]]])  # shape (1, 1, 2)
B = np.array([[3], [4]])  # shape (2, 1)
 
# Student sochta hai: Yeh bilkul alag lagte hain, yeh fail hoga!
# Actually works:
result = A + B  # shape (1, 2, 2)

Kyun sahi lagta hai: Shapes (1, 1, 2) aur (2, 1) rank aur size dono mein bilkul alag lagte hain.

Kyun galat hai:

• Right-align: (1, 1, 2) aur (2, 1) → B ko left mein pad karo → (1, 1, 2) aur (1, 2, 1)
• Dimension -1: 2 vs 1 → Rule 2 (singleton), stretch karke 2
• Dimension -2: 1 vs 2 → Rule 2 (singleton), stretch karke 2
• Dimension -3: 1 vs 1 → Rule 1 (equal), 1 rehta hai
• Result: (1, 2, 2)

Fix: Yaad rakho ki 1 magic number hai. Size 1 wali koi bhi dimension kisi bhi doosre size ke saath match karne ke liye stretch ho sakti hai, aur missing dimensions ko left mein 1 se pad kiya jaata hai.

Active Recall

100 toy cars khareedne ki bajay (ek har block ke liye), jo expensive aur wasteful hoga, tum ek magic rule use karte ho: "Zaroorat padne par ek car magically har block ke paas appear ho sakti hai." Bas yahi broadcasting hai!

NumPy ke paas teen simple rules hain yeh check karne ke liye ki tumhari "magic appearance" kaam karti hai ya nahi:

1. Agar dono ke paas ek direction mein same number of blocks hain, great!
2. Agar ek side mein sirf 1 block hai (jaise tumhari single car), to woh magically doosri side se match karne ke liye apne aap copy ho jaati hai.
3. Agar ek side ke paas woh direction hi nahi hai (jaise ek single number ki koi rows ya columns nahi hoti), to hum pretend karte hain ki size 1 hai aur rule 2 use karte hain.

Tum hamesha right se left check karte ho (jaise numbers padhna: units, tens, hundreds). Agar har direction teen tests mein se ek pass kare, magic kaam karta hai! Agar nahi, to NumPy kehta hai "Sorry, yeh magic nahi ho sakta."

Socho: "Right Equation, Stretch it!"

Connections

• 1.4.03-NumPy-array-indexing-slicing: Broadcasting aksar slicing operations ke baad aata hai
• 1.4.04-NumPy-vectorization: Broadcasting hi woh cheez hai jo mismatched shapes par vectorized operations ko possible banata hai
• 1.4.06-NumPy-performance-optimization: Broadcasting ko samajhna unnecessary np.tile() ya np.repeat() calls se bachata hai
• 2.1.02-Feature-normalization-standardization: Broadcasting efficient batch normalization enable karta hai
• 3.2.04-Matrix-operations-in-neural-networks: Weight matrices har layer mein bias vectors ke saath broadcast karte hain
• 1.2.04-Memory-efficient-Python-patterns: Broadcasting memory-heavy array duplication se bachata hai

#flashcards/ai-ml

Concept Map