Matrix multiplication rules and dimensionality

1.1.5 · AI-ML › Linear Algebra Essentials

Matrix multiplication kya hai?

Shapes kya batate hain:

m \times n A n \times p B = m \times p C

Do inner numbers ( $n$ aur $n$ ) match karne chahiye aur "consume" ho jaate hain; do outer numbers ( $m$ aur $p$ ) survive karte hain aur answer ki shape bante hain.

Figure — Matrix multiplication rules and dimensionality

Rule ko first principles se derive karna

Hum sirf $c_{ij} = \sum_{k} a_{ik} b_{k j}$ state nahi karte. Hum ise discover karte hain.

Dimensions match kyun karni chahiye: $b_{j}$ $R^{n}$ mein rehta hai (iske $n$ entries hain, ek $B$ ki har row ke liye). $A$ ke liye ek $n$ -vector ko lena possible ho, $A$ mein exactly $n$ columns hone chahiye. Yahi poora reason hai "columns of $A$ = rows of $B$ " rule ka. Yeh ek plug fits socket condition hai.

Key algebraic rules (aur WHY)

Property	Holds?	Kyun
Associative $(A B) C = A (B C)$	✅	Functions ka composition associative hota hai
Distributive $A (B + C) = A B + A C$	✅	Transformation ki linearity
Commutative $A B = B A$	❌ (generally)	"Pehle rotate phir stretch" ≠ "pehle stretch phir rotate"
Identity $A I = I A = A$	✅	$I$ = "kuch mat karo" transform
Transpose $(A B)^{⊤} = B^{⊤} A^{⊤}$	✅	Order reverse hota hai (proof neeche)

Worked examples

Common mistakes

Active recall

Recall Khud test karo (answers chhupao)

$A B$ exist karne ki kya condition hai? → columns of $A$ = rows of $B$ .
$(m \times n) (n \times p)$ ki shape? → $m \times p$ .
$c_{ij}$ ka formula? → $\sum_{k} a_{ik} b_{k j}$ .
Kya matrix mult commutative hai? → Nahi (generally).
$(A B)^{⊤}$ kya hai? → $B^{⊤} A^{⊤}$ .

Recall Feynman: ek 12-saal ke bacche ko explain karo

Socho do vending machines ek line mein hain. Pehli machine 4 coins leti hai aur 5 snacks deti hai. Doosri machine exactly 5 snacks leti hai aur 2 drinks deti hai. Yeh sirf isliye connect hoti hain kyunki pehli ka output (5) doosri ke input (5) se match karta hai. Combined machine 4 coins leti hai, 2 drinks deti hai — beech wala 5 andar "gayab" ho jaata hai. Har output nikalne ke liye, tum ek recipe ki poori row ko doosri ki poori column ke saath mix karte ho, pairs ko multiply karke add karte ho.

Connections

Vectors and Vector Spaces — ek matrix in spaces ke vectors par act karti hai.
Linear Transformations — matrix mult = inka composition.
Identity and Inverse Matrices — "kuch mat karo" aur "undo karo" transforms.
Neural Network Forward Pass — stacked $W x$ operations.
Dot Product — har entry $c_{ij}$ ek dot product HI hai.
Transpose Properties — order-reversal rule.

A B

product kab defined hota hai?

Jab

A

ke columns ki sankhya

B

ki rows ki sankhya ke barabar ho.

(m \times n) (n \times p)

ki shape kya hai?

m \times p

(inner dims match hoke cancel ho jaate hain, outer dims survive karte hain).

A B

ki entry

c_{ij}

ka formula do.

c_{ij} = \sum_{k = 1}^{n} a_{ik} b_{k j}

A

ki row

i

aur

B

ke column

j

ka dot product.

Kya matrix multiplication commutative hai?

Nahi;

A B \neq = B A

generally (unki shapes bhi alag ho sakti hain).

A

ke columns

B

ki rows ke kyun barabar hone chahiye?

Kyunki

A

B

ke har column par act karna hota hai, jo

R^{n}

mein rehta hai;

A

ko ek

n

-vector accept karne ke liye exactly

n

columns chahiye.

(A B)^{⊤}

kya hai?

B^{⊤} A^{⊤}

— order reverse ho jaata hai.

Matrix mult Hadamard product se kaise alag hai?

Matrix mult row·column dot products use karta hai (maps ka composition); Hadamard

⊙

element-wise hota hai.

y = W x

layer mein jahan

W

3 \times 4

hai,

x

aur

y

ki shape kya hai?

x

4 \times 1

hai,

y

3 \times 1

hai.

Column view:

A B

ka column

j

kiske barabar hai?

A

times

B

ka column

j

Kya matrix multiplication associative hai?

Haan,

(A B) C = A (B C)

, kyunki linear maps ka composition associative hota hai.