4.5.35 · D1Linear Algebra (Full)

Foundations — Gram-Schmidt orthogonalization — algorithm

2,524 words11 min readBack to topic

Before you can read the parent note Gram–Schmidt orthogonalization — algorithm, you must own every symbol it fires at you. Below, each symbol gets three things: plain words → the picture → why the topic needs it. They are ordered so each one leans on the last (fittingly).


1. A vector, and the arrow it draws

The little numbers inside, and , are called components. In the first component tells you how far right, the second how far up.

To name the components without committing to specific numbers, we write : here is the first component (how far right) and is the second component (how far up). Warning about the two uses of subscripts — read carefully:

  • When the letter under the vector is the same as the whole vector (like inside ), the subscript picks out a component of that one vector.
  • When we later list several different vectors as , the subscript is a name tag meaning "vector number 1, vector number 2, ..."

The context always tells you which: are we opening up one vector's coordinates, or naming a family of vectors? To keep the two crystal-clear, this page uses for components and reserves whole-vector names for the family discussion below.

Figure — Gram-Schmidt orthogonalization — algorithm
Figure: the arrow drawn from the origin. The black horizontal leg is the first component (3, how far right) and the black vertical leg is the second component (1, how far up); the red arrow itself is the vector. The two legs and the arrow form a right triangle.


2. Length, written

For we have , so . The double bars are just "length of what's inside" — never confuse them with the single bars of absolute value.


3. The inner product — the heart of everything

This is the most important symbol in the whole chapter, so we build it slowly.

Figure — Gram-Schmidt orthogonalization — algorithm
Figure: a black reference arrow lies flat. Three red arrows share the origin: one leaning the same way as (inner product positive), one standing straight up at a right angle to — marked with a small square and labelled — and one leaning backwards (inner product negative). The picture shows the inner product sliding from positive, through zero at the right angle, to negative.

Why does this specific formula detect perpendicularity? It helps to name the opening between the two arrows. Let (the Greek letter "theta") stand for the angle between the two arrows, measured where their tails meet — when they point the same way and when they form a perfect corner. With that name in hand, the inner product secretly equals where is the cosine of that angle. When the arrows are perpendicular, and , dragging the whole product to zero. You don't need this cosine formula to use Gram–Schmidt, but it explains why is the same statement as "right angle."

For a fuller treatment see Inner product spaces; for now, "multiply matching components and add, zero means perpendicular" is all you need.


4. Orthogonal and orthonormal

The word splits as ortho (right-angle) + normal (unit length). A unit-length vector is written ; the letter is the traditional name for a clean, normalized direction. See Orthonormal basis.


5. Projection — the shadow

This is the one operation the parent note calls "the building block."

Figure — Gram-Schmidt orthogonalization — algorithm
Figure: a black arrow acts as the floor; a black arrow leans above it. The red arrow is , lying flat along — the shadow of . A dashed black segment drops straight from the tip of down to the tip of the red shadow, meeting the floor at a right angle (marked with a small square); that dashed leftover is the perpendicular part.

The formula (derived in full in the parent note) is Read it as: "how much agrees with " divided by "how much agrees with itself", times the direction . The fraction is just a number (a scalar), and multiplying by it stretches or shrinks to reach the foot of the shadow. Deep dive available at Orthogonal projection.


6. Linear independence


7. The recursion symbols: , the base case, and

The parent's key formula has two parts. First, a base case that tells you where to start: Then, the general step for every later vector : Without the base case you would never get going, because the sum needs earlier 's to exist. Two pieces of notation deserve unpacking.

The word recursion means "each new answer is defined using the earlier answers": feeds , and feed , and so on. You cannot compute without first having .


8. Span and "the same subspace"


How these foundations feed the topic

The map below reads top to bottom: the plain arrow (vector) supports the two measurements we build on it — the inner product (agreement) and the norm (length). Those two give us the test for orthogonality and the idea of unit length, which combine into orthonormal. Inner product and norm together also build the projection (shadow), whose whole purpose is that we can subtract the shadow. Finally, subtraction, the loop, the requirement of linear independence, and the promise of an unchanged span all feed into Gram–Schmidt itself. If your browser cannot render the diagram, that sentence is the diagram in words.

Vector = arrow

Inner product agreement

Norm length of v

Orthogonal means inner product = 0

Orthonormal unit length

Projection = shadow

Subtract the shadow

Linear independence

Gram-Schmidt

Summation over earlier uj

Span same subspace


Equipment checklist

Before opening the parent note, make sure you can answer each of these out loud.

What does draw, and what do the numbers mean?
An arrow from the origin; = how far right, = how far up.
In , what does the subscript mean, versus in a list ?
Inside one vector it picks out a component; across a list it is a name tag for "vector number 1, 2, 3."
How do you compute for ?
, the Pythagorean length of the arrow.
Compute .
.
What value of the inner product signals a right angle?
Exactly — orthogonal means .
What is in ?
The angle between the two arrows, measured where their tails meet.
Why does mean perpendicular?
Because , and .
What does equal?
, the length of squared.
In plain words, what is ?
The shadow casts on the line through — the part of that leans along .
When is the projection formula undefined, and why?
When , since would force division by zero.
What does "orthonormal" add on top of "orthogonal"?
Every vector also has length .
What does tell you to do?
Add up the expression for (a "do it for every earlier vector" loop).
What is the base case that starts Gram–Schmidt?
— the first vector is taken as-is.
What breaks if the input vectors are linearly dependent?
Some becomes the zero vector, and you can't normalize (division by zero).
What does it mean that output vectors "span the same subspace"?
They sweep out the exact same sheet/space as the inputs; Gram–Schmidt only re-angles them inside it.

Connections