Gram-Schmidt orthogonalization — algorithm: Question bank

Before the questions, three tiny reminders so every symbol is earned:

True or false — justify

Order matters: swapping

v_{1}

and

v_{2}

can change the resulting orthogonal set.

True —

u_{1}

is always the first input untouched, so a different first vector seeds a different chain and generally gives different (though still orthogonal, same-span) results.

Gram–Schmidt preserves the span at every stage:

span (u_{1}, \dots, u_{k}) = span (v_{1}, \dots, v_{k})

True — each

u_{k}

v_{k}

minus a combination of earlier vectors, an invertible triangular change of basis, so no direction is ever gained or lost.

If the inputs are already mutually orthogonal, Gram–Schmidt leaves them unchanged (up to normalization).

True — every projection coefficient

⟨ v_{k}, u_{j} ⟩

is already zero, so nothing gets subtracted and

u_{k} = v_{k}

The output

u_{k}

is orthogonal to all previous

u_{j}

, not just the immediately preceding one.

True — we subtract the shadow on every settled

u_{j}

at once, and because those

u_{j}

are mutually orthogonal the cross terms vanish, killing overlap with each simultaneously.

An orthogonal set of nonzero vectors is automatically linearly independent.

True — if a combination sums to zero, dotting with any

u_{i}

isolates its coefficient times

⟨ u_{i}, u_{i} ⟩ \neq = 0

, forcing every coefficient to be zero.

Normalizing first (using

e_{j}

) and normalizing last give the same final orthonormal basis.

True — both routes produce the same orthogonal directions; the only difference is when you divide by lengths, and that never rotates a vector.

Gram–Schmidt requires the vectors to live in

R^{n}

with the usual dot product.

False — it works in any inner product space (e.g. functions with

⟨ f, g ⟩ = \int f g

); only the meaning of

⟨ \cdot, \cdot ⟩

changes.

If some

u_{k}

comes out as the zero vector, you made an arithmetic mistake.

False — a zero

u_{k}

is the algorithm correctly reporting that

v_{k}

was linearly dependent on earlier vectors; the inputs, not your arithmetic, were the problem.

The

R

in the resulting QR decomposition can have zeros on its diagonal.

False for independent inputs — a diagonal zero would mean some

∥ u_{k} ∥ = 0

, i.e. a dependent vector, which is exactly what independence forbids.

Spot the error

"I'll orthogonalize

v_{2}

by subtracting its projection onto

v_{1}

instead of

u_{1}

." — where's the flaw?

Here

u_{1} = v_{1}

so it happens to be fine for the second vector, but the habit fails from

v_{3}

on: projecting onto raw

v_{2}

(not orthogonal to

v_{1}

) leaves cross terms

⟨ v_{j}, v_{i} ⟩ \neq = 0

and the perpendicularity proof collapses.

"After finding

u_{1}

, I normalized it to

e_{1}

, then computed

u_{2} = v_{2} - \frac{⟨ v _{2} , e _{1} ⟩}{⟨ e _{1} , e _{1} ⟩} e_{1}

." — what's redundant?

The denominator: once

e_{1}

is a unit vector

⟨ e_{1}, e_{1} ⟩ = 1

, so it should just be

u_{2} = v_{2} - ⟨ v_{2}, e_{1} ⟩ e_{1}

; keeping the denominator isn't wrong, only mixing conventions carelessly is.

"My set came out orthogonal but I forgot to check it spans the original space." — is that a real worry?

No — the recursion is a triangular, invertible reshuffle of the same vectors, so the span is guaranteed preserved; you never need to re-verify span, only orthogonality if you fear arithmetic slips.

"I got

⟨ u_{2}, u_{1} ⟩ = 0.0001

instead of exactly 0, so my method is broken." — diagnosis?

The method is exact; a tiny nonzero is rounding error from finite-precision arithmetic, a known weakness of classical Gram–Schmidt that motivates the numerically stabler modified variant.

"To orthonormalize I divided each

u_{k}

⟨ u_{k}, u_{k} ⟩

." — fix it.

You divided by the squared length; the norm is

∥ u_{k} ∥ = ⟨ u_{k}, u_{k} ⟩

, so use

e_{k} = u_{k} / ⟨ u_{k}, u_{k} ⟩

or the resulting vectors won't be unit length.

"I projected

v_{3}

onto

u_{1}

and

u_{2}

but used

⟨ v_{3}, u_{2} ⟩

over

⟨ v_{3}, v_{2} ⟩

in one denominator." — what broke?

The denominator must be

⟨ u_{j}, u_{j} ⟩

, the settled vector dotted with itself; mixing in

v

's or numerators from different terms scales the shadow wrongly and destroys orthogonality.

Why questions

Why do we subtract the shadow onto

u_{j}

and never onto

v_{j}

Only the

u_{j}

are mutually orthogonal, so their overlaps are independent and removing each separately leaves a clean residual; the

v_{j}

still overlap each other, so subtracting their shadows reintroduces overlap.

Why is the projection scalar forced to be exactly

c = \frac{⟨ v , u ⟩}{⟨ u , u ⟩}

It's the unique value making the leftover perpendicular: imposing

⟨ v - c u, u ⟩ = 0

and solving for

c

leaves no freedom, so orthogonality picks

c

, not our taste.

Why must the input vectors be linearly independent?

A dependent

v_{k}

lies entirely in the span of earlier vectors, so subtracting all its shadows removes everything, giving

u_{k} = 0

which cannot be normalized (division by zero).

Why does orthogonality make coordinates "decouple"?

In an orthonormal basis the coordinate of a vector along

e_{k}

is simply

⟨ v, e_{k} ⟩

, computable independently of the others — no simultaneous equations, which is exactly why Least squares loves orthogonal bases.

Why is Gram–Schmidt "equivalent" to QR decomposition?

The orthonormal outputs form

Q

's columns, and the projection coefficients you compute along the way are precisely the entries of the upper-triangular

R

that rebuilds the original vectors.

Why normalize at all if we already have perpendicular vectors?

Perpendicularity handles direction; normalizing handles length, and only unit vectors give the clean formulas

⟨ v, e_{k} ⟩

for coordinates and an actual Orthonormal basis.

Why does the perpendicularity proof rely on induction?

Each step assumes the earlier

u_{j}

are already mutually orthogonal to collapse the sum; that assumption must be established for the previous vectors first, which is exactly an inductive chain.

Edge cases

What happens if you feed in a single vector

v_{1}

alone?

Nothing to remove —

u_{1} = v_{1}

, and normalization gives

e_{1} = v_{1} / ∥ v_{1} ∥

; the algorithm is trivially complete.

What if two input vectors are identical,

v_{2} = v_{1}

The overlap is total, so

u_{2} = v_{1} - proj_{u_{1}} (v_{1}) = 0

, correctly flagging that the pair is dependent.

What if an input vector is the zero vector?

It has no direction, its norm is zero, so it fails independence immediately and cannot be normalized — the input set was invalid from the start.

What if two inputs are already perpendicular but not unit length?

Gram–Schmidt subtracts nothing (their projection coefficient is zero) and only the final normalization step rescales them to unit length.

In an infinite-dimensional inner product space (say polynomials), does Gram–Schmidt still terminate?

For any finite list of inputs, yes — the recursion runs exactly

k

times; the space being infinite-dimensional never matters as long as your input set is finite.

If all inputs lie in a 2-D plane inside

R^{3}

, what does the third output look like?

The third vector is dependent on the first two, so its residual is

u_{3} = 0

; the algorithm produces only a 2-D orthogonal set, matching the true dimension of their span.

Recall One-line survival summary

Every trap on this page dissolves into one sentence. The single mantra ::: "Subtract the shadow on every settled $u_{j}$ , keep the leftover, and let a zero leftover mean dependence."

Connections

Orthogonal projection — the shadow operation every trap circles back to
Linear independence — what a zero $u_{k}$ is really reporting
Inner product spaces — why the method outlives the ordinary dot product
QR decomposition — where the coefficients and $Q$ columns end up
Orthonormal basis — the object normalization delivers
Least squares — the payoff of decoupled orthogonal coordinates