Vector addition, scalar multiplication, and geometric interpretation

1. What is a vector? (WHAT)

WHY a column? Because matrix–vector multiplication (later chapters) is defined so a matrix acts on a column from the left. Consistency now saves pain later.

2. Vector Addition (HOW + WHY)

Derivation from first principles

A vector represents a displacement. If $\mathbf{u}$ moves me from origin $O$ to point $A$ , and $\mathbf{v}$ is another displacement, then to add them I continue from $A$ by $\mathbf{v}$ , landing at $B$ .

The $x$ -displacement of the whole trip = (x-move of $\mathbf{u}$ ) + (x-move of $\mathbf{v}$ ) = $u_1 + v_1$ .
Same logic each axis.

Therefore addition must be componentwise — it's not a definition pulled from a hat, it's forced by "displacements accumulate along each axis independently."

Figure — Vector addition, scalar multiplication, and geometric interpretation

3. Scalar Multiplication (HOW + WHY)

Derivation

Scaling means "do the same displacement $c$ times." Doing $\mathbf{v}$ twice = $\mathbf{v}+\mathbf{v}$ , whose components are $2v_1,\dots$ . Extending from integers to reals by continuity gives $c\mathbf{v}=(cv_1,\dots,cv_n)$ .

4. Combining them: the workhorse of ML

5. Worked Examples

6. Common Mistakes (Steel-manned)

7. Feynman & Mnemonic

Recall Explain to a 12-year-old (click to reveal)

Imagine walking. A vector is an instruction: "go 3 steps east, 4 steps north." Adding two instructions means do the first walk, then the second — you end up somewhere, and that final spot is the sum. Scalar multiplication is like a speed dial: multiply by 2 = take a walk twice as long in the same direction; multiply by $-1$ = walk backwards the same distance. That's it — everything fancy in AI is just millions of these walks added up.

8. Active Recall

How do you add two vectors?

Componentwise:

(\mathbf u+\mathbf v)_i = u_i+v_i

; geometrically tip-to-tail.

Why is vector addition componentwise (first principles)?

Displacements along each axis accumulate independently, so each coordinate adds separately.

What is

c\mathbf v

Multiply every component by

c

(cv_1,\dots,cv_n)

; stretches/flips the arrow.

Formula for the length of

c\mathbf v

\|c\mathbf v\| = |c|\,\|\mathbf v\|

Why

|c|

and not

c

\|c\mathbf v\|

Length is non-negative; a negative scalar flips direction but scales length by

|c|

State the triangle inequality for vectors.

\|\mathbf u+\mathbf v\| \le \|\mathbf u\| + \|\mathbf v\|

, equality iff same direction.

What is a linear combination?

c_1\mathbf v_1+\cdots+c_k\mathbf v_k

— scaled sum of vectors; the core operation of a neuron.

What must be true to add two vectors?

They must have the same dimension.

Geometric meaning of

\tfrac12(\mathbf u+\mathbf v)

Midpoint of the segment joining the tips of

\mathbf u

and

\mathbf v

Is vector addition commutative?

Yes:

\mathbf u+\mathbf v=\mathbf v+\mathbf u

(parallelogram = both tip-to-tail orders).

9. Connections

Vectors and vector spaces — the axioms these operations obey
Linear combinations, span, and basis — built directly from add + scale
Dot product and vector norms — magnitude generalizes here
Matrix-vector multiplication — a matrix output is a linear combination of its columns
Neural network forward pass — weighted sum = scale + add, repeated
Word embeddings — averaging/subtracting embeddings uses these exact operations

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, ek vector basically ek arrow hai jo origin se kisi point tak jaata hai — usme direction bhi hai aur length (magnitude) bhi. AI-ML me har data point, har embedding, har feature-row ek vector hi hota hai, isliye ye foundation hai.

Do main operations yaad rakho. Pehla addition: dono vectors ko component-wise jodo — x ke saath x, y ke saath y. Geometrically ise "tip-to-tail" socho: pehle vector chalo, uske tip se doosra vector chalo, jahan pahunche wahi answer. Kyun component-wise? Kyunki har axis ka movement alag-alag add hota hai, ye definition kisi ne random nahi banayi.

Doosra scalar multiplication: ek number $c$ se poore vector ko multiply karo, matlab har component ko $c$ se guna. $c$ agar 2 hai toh arrow double lamba, $0.5$ toh aadha, aur negative $c$ toh direction ulti ho jaati hai. Length ka formula: $\|c\mathbf v\| = |c|\,\|\mathbf v\|$ — dhyan do, yaha $|c|$ hai kyunki length kabhi negative nahi hoti, sirf direction flip hoti hai.

Ek important galti se bacho: $\|\mathbf u + \mathbf v\|$ hamesha $\|\mathbf u\| + \|\mathbf v\|$ ke barabar nahi hota — arrows ek doosre ko cancel bhi kar sakte hain (triangle inequality). Jab tumhe ye do cheezein — scale karo aur add karo — samajh aa jaayengi, tab neural network ka "weighted sum" bilkul trivial lagega, kyunki wo bas yehi operations baar-baar hain.

Test yourself — Linear Algebra Essentials

Connections