Orthogonal sets and orthonormal basis

1. Definitions, built up carefully

WHAT is the difference? Orthogonal = perpendicular only. Orthonormal = perpendicular and length 1. You make an orthogonal set orthonormal by dividing each vector by its own length (normalizing).

2. WHY orthogonal sets are automatically independent (derive it)

Derivation from scratch. Suppose $c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + \dots + c_p\mathbf{u}_p = \mathbf{0}.$ Take the dot product of both sides with one fixed vector $\mathbf{u}_j$ : $\mathbf{u}_j\cdot\Big(\sum_i c_i\mathbf{u}_i\Big) = \mathbf{u}_j\cdot\mathbf{0}=0.$ Distribute the dot product: $\sum_i c_i(\mathbf{u}_j\cdot\mathbf{u}_i) = 0.$ Why this step? Because of orthogonality, every term dies except $i=j$ (those dot products are $0$ ). What survives: $c_j(\mathbf{u}_j\cdot\mathbf{u}_j) = c_j\|\mathbf{u}_j\|^2 = 0.$ Since $\mathbf{u}_j\neq\mathbf{0}$ , we have $\|\mathbf{u}_j\|^2>0$ , forcing $c_j=0$ . This holds for every $j$ , so all coefficients are zero → independent. $\blacksquare$

3. THE payoff: coordinates by dot product (derive it)

HOW we get it (first principles). Because it's a basis, some coefficients exist: $\mathbf{y}= c_1\mathbf{u}_1+\dots+c_p\mathbf{u}_p.$ We don't yet know the $c_i$ . Dot both sides with $\mathbf{u}_j$ : $\mathbf{y}\cdot\mathbf{u}_j = \sum_i c_i(\mathbf{u}_i\cdot\mathbf{u}_j).$ Why this step? Same trick as before — orthogonality nukes all cross terms, leaving only $c_j\,(\mathbf{u}_j\cdot\mathbf{u}_j)$ : $\mathbf{y}\cdot\mathbf{u}_j = c_j\,(\mathbf{u}_j\cdot\mathbf{u}_j)\;\Rightarrow\; c_j = \frac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}.$ Each coordinate is decoupled — no system to solve. That is the whole magic.

Figure — Orthogonal sets and orthonormal basis

4. Orthogonal matrices

WHY $U^{\mathsf T}U=I$ ? The $(i,j)$ entry of $U^{\mathsf T}U$ is exactly $\mathbf{u}_i\cdot\mathbf{u}_j$ . Orthonormality says that's $1$ on the diagonal, $0$ off — i.e. the identity.

5. Worked examples

Worked example Example A — verify orthogonality, then find coordinates

Let $\mathbf{u}_1=\begin{bmatrix}3\\1\\1\end{bmatrix}$ , $\mathbf{u}_2=\begin{bmatrix}-1\\2\\1\end{bmatrix}$ , $\mathbf{u}_3=\begin{bmatrix}-1\\-4\\7\end{bmatrix}$ , and $\mathbf{y}=\begin{bmatrix}6\\1\\-8\end{bmatrix}$ .

Step 1 — check orthogonal. Why? So we can use the dot-product shortcut. $\mathbf{u}_1\cdot\mathbf{u}_2 = -3+2+1 = 0$ . $\mathbf{u}_1\cdot\mathbf{u}_3=-3-4+7=0$ . $\mathbf{u}_2\cdot\mathbf{u}_3=1-8+7=0$ . ✓ Orthogonal basis of $\mathbb{R}^3$ .

Step 2 — compute coordinates. Why this formula? Section 3 says $c_j=\dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$ .

$\mathbf{y}\cdot\mathbf{u}_1=18+1-8=11$ , $\ \mathbf{u}_1\cdot\mathbf{u}_1=9+1+1=11 \Rightarrow c_1=1.$
$\mathbf{y}\cdot\mathbf{u}_2=-6+2-8=-12$ , $\ \mathbf{u}_2\cdot\mathbf{u}_2=1+4+1=6 \Rightarrow c_2=-2.$
$\mathbf{y}\cdot\mathbf{u}_3=-6-4-56=-66$ , $\ \mathbf{u}_3\cdot\mathbf{u}_3=1+16+49=66 \Rightarrow c_3=-1.$

Step 3 — check. $1\mathbf{u}_1-2\mathbf{u}_2-1\mathbf{u}_3 = \begin{bmatrix}3+2+1\\1-4+4\\1-2-7\end{bmatrix}=\begin{bmatrix}6\\1\\-8\end{bmatrix}=\mathbf{y}.$ ✓

6. Common mistakes (Steel-manned)

7. Active recall

What is an orthogonal set?

A set of vectors where every distinct pair has dot product 0 (

\mathbf{u}_i\cdot\mathbf{u}_j=0

for

i\neq j

What extra condition makes a set orthonormal?

Each vector is a unit vector (

\|\mathbf{u}_i\|=1

), i.e.

\mathbf{u}_i\cdot\mathbf{u}_i=1

Why is a nonzero orthogonal set automatically linearly independent?

Dotting

\sum c_i\mathbf{u}_i=0

with

\mathbf{u}_j

leaves

c_j\|\mathbf{u}_j\|^2=0

, forcing

c_j=0

Coordinate formula in an orthogonal basis?

c_j=\dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}

Coordinate formula in an orthonormal basis?

c_j=\mathbf{y}\cdot\mathbf{u}_j

(denominator is 1).

For a matrix U with orthonormal columns, what is

U^{\mathsf T}U

The identity

I

When does

U^{-1}=U^{\mathsf T}

hold?

When U is square with orthonormal columns (an orthogonal matrix).

Why do orthonormal transforms preserve length?

\|U\mathbf{x}\|^2=\mathbf{x}^{\mathsf T}U^{\mathsf T}U\mathbf{x}=\mathbf{x}^{\mathsf T}\mathbf{x}=\|\mathbf{x}\|^2

How do you turn an orthogonal set into an orthonormal one?

Normalize: divide each vector by its own norm.

Common misnomer trap with "orthogonal matrix"?

It actually requires orthonormal (unit) columns and a square shape.

Recall Feynman: explain to a 12-year-old

Imagine arrows pointing in directions that are all at perfect right angles, like the edges of a room (left-right, forward-back, up-down). If someone stands somewhere in the room and you want to describe where they are, you don't solve a puzzle — you just ask "how far along the floor?" and "how high up?" separately, because the directions don't interfere. A dot product is exactly that "how far along this arrow" measurement. Orthonormal also means each arrow is exactly one step long, so your answers are already in the right units. That's why this coordinate system is the easiest one in the world.

Connections

Dot product and norms — the measuring tool behind everything here.
Gram-Schmidt process — how you build an orthogonal/orthonormal basis from any basis.
Orthogonal projection — Section 3's formula is literally a sum of projections.
Least squares — orthonormal bases make the normal equations trivial.
QR decomposition — $Q$ has orthonormal columns; $R$ stores the coordinates.
Eigenvalues and eigenvectors — symmetric matrices have orthonormal eigenbases (Spectral Theorem).

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, basis ka matlab hai ek coordinate system. Lekin agar basis "random" ho, to kisi vector ke coordinates nikalne ke liye tumhe poora linear system solve karna padta hai — boring aur lamba kaam. Yahin par orthonormal basis game changer hai: jab saare basis vectors aapas mein perpendicular hain aur length exactly 1 hai, to har coordinate sirf ek dot product se mil jaata hai. Koi system solve nahi, bas $c_j = \mathbf{y}\cdot\mathbf{u}_j$ .

Agar basis sirf orthogonal hai (length 1 nahi) to thoda sa adjust karo: $c_j = \dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$ . Magic tareeka same hai — dono taraf $\mathbf{u}_j$ se dot karo, orthogonality ki wajah se baaki saare terms zero ho jaate hain, aur sirf $c_j$ bach jaata hai. Isi reason se nonzero orthogonal vectors automatically linearly independent bhi hote hain, alag se check karne ki zaroorat nahi.

Matrix language mein, agar $U$ ke columns orthonormal hain to $U^{\mathsf T}U = I$ , kyunki $(i,j)$ entry to bas $\mathbf{u}_i\cdot\mathbf{u}_j$ hi hai. Square case mein iska matlab $U^{-1}=U^{\mathsf T}$ — inverse free mein mil gaya! Aise transforms length aur angle ko preserve karte hain (rotation/reflection), isliye numerically bhi safe rehte hain.

Ek common galti yaad rakho: "orthogonal matrix" naam dhoka deta hai — actually usme columns orthonormal (unit length) hone chahiye aur matrix square hona chahiye. Aur jab basis sirf orthogonal ho to denominator $\mathbf{u}_j\cdot\mathbf{u}_j$ ko bhoolna mat. Bas itna sambhal lo, baaki sab smooth hai.

1. Definitions, built up carefully

2. WHY orthogonal sets are automatically independent (derive it)

3. THE payoff: coordinates by dot product (derive it)

4. Orthogonal matrices

5. Worked examples

6. Common mistakes (Steel-manned)

7. Active recall

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Linear Algebra (Full)

Connections