Scalars, vectors, matrices, and tensors definitions

1.1.1 · AI-ML › Linear Algebra Essentials

WHY karte hain care? (80/20)

WHAT hain ye? (rank ke hisaab se definitions)

Key organizing concept hai rank (a.k.a. order ya number of axes): ek element ko address karne ke liye kitne indices chahiye.

Figure — Scalars, vectors, matrices, and tensors definitions

HOW count karte hain elements? (scratch se derive karo)

Step 1 — Scalar. Kitne numbers? Bas ek. $N_{0} = 1$ Ye step kyun? Rank 0 ka matlab hai koi bhi index kisi cheez pe range nahi karta.

Step 2 — Vector of length $n$ . Index $i$ run karta hai $1 \to n$ tak. $N_{1} = n$ Ye step kyun? Ek index, $n$ choices.

Step 3 — Matrix $m \times n$ . $m$ rows mein se har ek ke liye, $n$ columns hain. $N_{2} = m \cdot n$ Ye step kyun? Independent choices multiply hoti hain (fundamental counting principle).

Step 4 — Tensor $d_{1} \times \dots \times d_{k}$ . Generalize karo: har axis length ko multiply karo. $N_{k} = r = 1 \prod k d_{r}$ Ye step kyun? Har index ek independent multiplicative factor contribute karta hai — bilkul nested loops ki tarah.

Transpose: ek rank-2 operation

Kyun: ye grid ko main diagonal ke across reflect karta hai. Ek vector $v \in R^{n}$ (ek column) transpose hokar ek row $v^{⊤} \in R^{1 \times n}$ ban jaata hai. Ek scalar apna khud ka transpose hota hai ( $a^{⊤} = a$ ).

Worked Examples

Common Mistakes (Steel-manned)

Flashcards

Rank-

k

tensor ke ek element ko kitne indices address karte hain?

Exactly

k

Ek scalar kis rank ka tensor hai?

Rank 0.

R^{n}

mein vector ki shape vs

R^{m \times n}

mein matrix ki shape?

Vector:

(n,)

, rank 1. Matrix:

(m, n)

, rank 2.

d_{1} \times \dots \times d_{k}

tensor mein total elements ki sankhya?

\prod_{r = 1}^{k} d_{r}

(A^{⊤})_{ij}

kiske barabar hota hai?

A_{j i}

(do indices swap karo).

64 RGB

28 \times 28

images ke batch ki shape?

(64, 28, 28, 3)

, ek rank-4 tensor.

1 \times n

matrix ek

n

-vector kyun NAHI hai?

Alag rank (2 vs 1) aur shape; ye alag tarah se broadcast/multiply karte hain.

"Tensor rank/order" vs "matrix rank" — kya fark hai?

Order = axes ki sankhya; matrix rank = linearly independent rows/columns ki sankhya.

N

elements wale float32 tensor ki memory?

4 N

bytes.

Ek scalar ka transpose?

Scalar khud hi (

a^{⊤} = a

Recall Feynman: 12-saal ke bachche ko samjhao

Numbers ke boxes imagine karo. Scalar ek sticky note pe ek number hai. Vector lockers ki ek single row hai — batao kaunsa locker (ek number) aur uski value mil jaati hai. Matrix lockers ki poori ek wall hai — aap row aur column batate ho (do numbers). Tensor bahut saari walls, floors, aur buildings wali ek giant building hai — aap numbers dete rehte ho jab tak ek akele locker ko point nahi kar lete. Jinhe aapko bolna padta hai ek value dhundne ke liye, woh "rank" hai. Saari sizes ko multiply karo aur pata chal jaata hai kitne lockers (numbers) total hain.

Connections

Matrix multiplication — matching inner dimensions chahiye, inhi shapes par based.
Broadcasting rules — scalars/lower-rank tensors higher-rank ones ke saath kaise combine karte hain.
Transpose and symmetry
Tensor reshaping and flattening — $\prod d_{r}$ elements ko rearrange karna.
Dot product and matrix-vector product
Rank of a matrix (linear independence) — "rank" ka doosra matlab.