Matrix multiplication rules and dimensionality

What is matrix multiplication?

WHAT the shapes tell you:

\underbrace{A}_{m\times n}\;\underbrace{B}_{n\times p}\;=\;\underbrace{C}_{m\times p}

The two inner numbers ( $n$ and $n$ ) must match and get "consumed"; the two outer numbers ( $m$ and $p$ ) survive to become the shape of the answer.

Figure — Matrix multiplication rules and dimensionality

Deriving the rule from first principles

We don't just state $c_{ij}=\sum_k a_{ik}b_{kj}$ . We discover it.

Why the dimensions must match: $\mathbf{b}_j$ lives in $\mathbb{R}^n$ (it has $n$ entries, one per row of $B$ ). For $A$ to eat an $n$ -vector, $A$ must have exactly $n$ columns. That is the entire reason for the "columns of $A$ = rows of $B$ " rule. It is a plug fits socket condition.

Key algebraic rules (and WHY)

Property	Holds?	Why
Associative $(AB)C = A(BC)$	✅	Composition of functions is associative
Distributive $A(B+C)=AB+AC$	✅	Linearity of the transformation
Commutative $AB = BA$	❌ (generally)	"Rotate then stretch" ≠ "stretch then rotate"
Identity $AI=IA=A$	✅	$I$ = "do nothing" transform
Transpose $(AB)^\top=B^\top A^\top$	✅	Order reverses (proof below)

Worked examples

Common mistakes

Active recall

Recall Test yourself (hide answers)

What condition allows $AB$ to exist? → columns of $A$ = rows of $B$ .
Shape of $(m\times n)(n\times p)$ ? → $m\times p$ .
Formula for $c_{ij}$ ? → $\sum_k a_{ik}b_{kj}$ .
Is matrix mult commutative? → No (in general).
What is $(AB)^\top$ ? → $B^\top A^\top$ .

Recall Feynman: explain to a 12-year-old

Imagine two vending machines lined up. The first machine takes 4 coins and spits out 5 snacks. The second machine takes exactly 5 snacks and spits out 2 drinks. They only connect because the first's output (5) matches the second's input (5). The combined machine takes 4 coins, gives 2 drinks — the 5 in the middle "disappears" inside. To find each output, you mix a whole row of one recipe with a whole column of the other, multiplying pairs and adding them up.

Connections

Vectors and Vector Spaces — a matrix acts on vectors from these spaces.
Linear Transformations — matrix mult = composition of these.
Identity and Inverse Matrices — the "do nothing" and "undo" transforms.
Neural Network Forward Pass — stacked $W\mathbf{x}$ operations.
Dot Product — each entry $c_{ij}$ IS a dot product.
Transpose Properties — order-reversal rule.

When is the product

AB

defined?

When the number of columns of

A

equals the number of rows of

B

What is the shape of

(m\times n)(n\times p)

m\times p

(inner dims match and cancel, outer dims survive).

Give the formula for entry

c_{ij}

AB

c_{ij}=\sum_{k=1}^{n} a_{ik}b_{kj}

= row

i

A

dotted with column

j

B

Is matrix multiplication commutative?

No;

AB\ne BA

in general (they may even have different shapes).

Why must columns of

A

equal rows of

B

Because

A

must act on each column of

B

, which lives in

\mathbb{R}^n

;

A

needs exactly

n

columns to accept an

n

-vector.

What is

(AB)^\top

B^\top A^\top

— the order reverses.

How is matrix mult different from the Hadamard product?

Matrix mult uses row·column dot products (composition of maps); Hadamard

\odot

is element-wise.

In a layer

\mathbf{y}=W\mathbf{x}

with

W

being

3\times4

, what shape is

\mathbf{x}

and

\mathbf{y}

\mathbf{x}

4\times1

\mathbf{y}

3\times1

Column view: column

j

AB

equals?

A

times column

j

B

Is matrix multiplication associative?

Yes,

(AB)C=A(BC)

, because composition of linear maps is associative.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, matrix multiplication ko samajhne ka sabse easy tarika hai: har matrix ek machine hai jo vectors ko transform karti hai. Jab hum $A \times B$ karte hain, matlab pehle $B$ wali transformation lagao, phir uske output pe $A$ wali transformation lagao. Isliye rule aata hai — $A$ ke columns aur $B$ ke rows barabar hone chahiye. Kyun? Kyunki $B$ ka har column ek $n$ -dimension ka vector hai, aur $A$ ko us $n$ -vector ko "khaana" hai, to $A$ ke paas exactly $n$ columns hone hi chahiye. Yahi hai "plug fits socket" wali baat.

Shape yaad rakhne ka mantra: "Inners match, outers survive." Agar $A$ hai $m\times n$ aur $B$ hai $n\times p$ , to beech wale $n$ aur $n$ match karke gayab ho jaate hain, aur bahar wale $m$ aur $p$ bach jaate hain — result banta hai $m\times p$ . Har ek entry $c_{ij}$ nikaalne ke liye $A$ ki $i$ -th row ko $B$ ki $j$ -th column ke saath dot product karo (pairs multiply karo, phir add karo).

Ek important cheez: matrix multiplication commutative nahi hai, yaani $AB$ aur $BA$ generally alag hote hain — kabhi to inki shape hi alag hoti hai! Jaise "pehle mozey pehno phir joote" vs "pehle joote phir mozey" — order matters. Aur ek common galti: matrices ko element-by-element multiply mat karo (woh Hadamard product hai, alag cheez). Matrix mult hamesha row-column dot product se hota hai.

ML mein yeh core hai bhai — neural network ka har layer $\mathbf{y}=W\mathbf{x}$ ek matrix-vector multiplication hi to hai, aur layers stack karna matrices ko multiply karke compose karna hai. Isliye dimensionality ka hisaab bilkul solid hona chahiye, warna "shape mismatch" error har jagah aayega.

Test yourself — Linear Algebra Essentials

Connections