Worked examples — Singular Value Decomposition (SVD) — full derivation

Everything below leans on the parent recipe: the four-step SVD recipe. When we say "form $A^{\top }A$" we mean the gateway from Positive Semi-Definite Matrices whose eigenbasis comes from the Spectral Theorem.

---

The scenario matrix

Every matrix you could be handed falls into one of these case classes. The last column says which worked example nails that cell.

#	Case class	What makes it tricky	Covered by
C1	Square, full-rank, symmetric	Do ${\sigma }_i$ equal the eigenvalues? Signs!	Example 1
C2	Square, negative eigenvalue	$\lambda (A)<0$ but $\sigma \ge 0$	Example 2
C3	Square, complex eigenvalues (rotation-like)	$A$ has no real eigenvectors	Example 3
C4	Diagonal with mixed signs	Is $\Sigma$ just $	A
C5	Tall, rank-deficient ($m>n$, $r<n$)	Zero $\sigma$, null space, extend $U$	Example 5
C6	Wide, full row rank ($m<n$, $r=m$)	Null space lives in the $v$'s	Example 6
C7	Wide, rank-deficient ($m<n$, $r<m$)	Zeros in BOTH the $u$'s and $v$'s	Example 7
C8	Zero / degenerate matrix	${\sigma }_1=0$, limiting case	Example 8
C9	Word problem (data / stretching)	Read geometry off the ${\sigma }_i$	Example 9

Notation reminder, so no symbol is unearned:

• $A^{\top }$ = transpose: flip $A$ across its diagonal (rows become columns).
• $\Vert x\Vert$ = length of vector $x$, $\sqrt{x_1^2+x_2^2+\dots }$.
• ${\delta }_{ij}=1$ if $i=j$, else $0$ (the "are-they-the-same-index?" switch).
• $\det$ = determinant; solving $\det (M-\lambda I)=0$ finds eigenvalues $\lambda$ of $M$ (see Eigenvalues and Eigenvectors).
• $r$ = rank = number of nonzero singular values = number of truly independent directions $A$ keeps.
• $e_i$ and $\Sigma$: defined in the callout above.

---

Example 1 — C1: symmetric, full-rank square

Forecast: guess the singular values before reading. Diagonal, positive... does $\Sigma$ just copy $A$?

1. Form $A^{\top }A$. $A^{\top }A=\left(\begin{array}{cc}2& 0\\ 0& 3\end{array}\right)\left(\begin{array}{cc}2& 0\\ 0& 3\end{array}\right)=\left(\begin{array}{cc}4& 0\\ 0& 9\end{array}\right).$ Why this step? It is the safe symmetric gateway from the parent recipe.
2. Eigenvalues. Already diagonal: ${\lambda }_1=9,{\lambda }_2=4$. So ${\sigma }_1=\sqrt{9}=3,{\sigma }_2=\sqrt{4}=2$. Why this step? ${\sigma }_i=\sqrt{{\lambda }_i}$; ordering biggest-first means ${\sigma }_1=3$.
3. Build $V$. Eigenvector of $\lambda =9$ is $e_2=(0,1{)}^{\top }$; of $\lambda =4$ is $e_1=(1,0{)}^{\top }$. So $V=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$. Why this step? We must list the $v_i$ in the same order as the ${\sigma }_i$.
4. Define $u_i=\frac{1}{{\sigma }_i}A{v}_i$. $u_1=\frac{1}{3}A(0,1{)}^{\top }=\frac{1}{3}(0,3{)}^{\top }=(0,1{)}^{\top }$; $u_2=\frac{1}{2}A(1,0{)}^{\top }=\frac{1}{2}(2,0{)}^{\top }=(1,0{)}^{\top }$. So $U=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$. Why this step? Defining $u_i$ from $v_i$ locks the correct sign (avoids the parent's "sign trap").

Verify: $U\Sigma V^{\top }=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\left(\begin{array}{cc}3& 0\\ 0& 2\end{array}\right)\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)=\left(\begin{array}{cc}2& 0\\ 0& 3\end{array}\right)=A.$ ✓ Note $\sigma$'s got reordered from $A$'s entries — copying is not automatic.

---

Example 2 — C2: a negative eigenvalue

Forecast: $A$ has eigenvalue $-3$. Will a singular value be $-3$?

1. Form $A^{\top }A$. $\left(\begin{array}{cc}-3& 0\\ 0& 1\end{array}\right)\left(\begin{array}{cc}-3& 0\\ 0& 1\end{array}\right)=\left(\begin{array}{cc}9& 0\\ 0& 1\end{array}\right)$. Why this step? Squaring kills the sign — that is exactly the point.
2. Eigenvalues of $A^{\top }A$: $9,1$. So ${\sigma }_1=3,{\sigma }_2=1$. Why this step? ${\sigma }_i=\sqrt{{\lambda }_i(A^{\top }A)}\ge 0$ always, unlike $\lambda (A)$.

Verify: ${\sigma }_1=3=|-3|$, not $-3$. The reflection (negative eigenvalue) is stored inside $U$, not $\Sigma$. Sanity: $\Vert A{e}_1\Vert =\Vert (-3,0{)}^{\top }\Vert =3={\sigma }_1$. ✓ This directly answers the parent's mistake box: singular values are never the eigenvalues of $A$.

---

Example 3 — C3: complex eigenvalues (a rotation)

Forecast: a rotation turns every vector without changing its length. What should the ${\sigma }_i$ be?

Figure s01 (Example 3). Left: the blue unit circle of inputs. Right: the same circle after applying $A$ (red) — still a unit circle, and the yellow arrow shows one input landing at the same length. Use it in step 3: no direction is stretched, so every ${\sigma }_i=1$.

1. Recall the trap. $A$'s eigenvalues are $e^{\pm i30^{\circ }}$ — complex, so $A$ has no real eigenvectors. Eigen-decomposition fails over $\mathbb{R}$. SVD does not care. Why this step? Motivates why $A^{\top }A$ (real, symmetric) rescues us.
2. Form $A^{\top }A$. Any rotation $R$ satisfies $R^{\top }R=I$, so $A^{\top }A=I=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$. Why this step? Rotation columns are orthonormal, giving the identity.
3. Eigenvalues of $I$: the characteristic polynomial is $(1-\lambda {)}^2=0$, a repeated root $\lambda =1$ (multiplicity two). So ${\sigma }_1={\sigma }_2=\sqrt{1}=1$. Why this step? $\sqrt{1}=1$: a rotation stretches nothing — matching figure s01.
4. Build $V,U$. With $A^{\top }A=I$, any orthonormal $V$ works — take $V=I$. Then $u_i=A{v}_i$, so $U=A$. Why this step? $\Sigma =I$, so $A=U\cdot I\cdot I^{\top }=A$ trivially. The whole matrix hides in the rotation $U$.

Verify: $A^{\top }A=I$ and both singular values equal $1$ (the characteristic polynomial $(1-\lambda {)}^2$ has the single distinct root $1$). Geometrically, the red circle in figure s01 never grew. ✓

---

Example 4 — C4: diagonal with mixed signs

Forecast: which entry becomes ${\sigma }_1$?

1. Form $A^{\top }A$. Diagonal squared: $\text{diag}(16,0,4)$. Why this step? Gateway; diagonals square entrywise.
2. Eigenvalues $16,4,0\Rightarrow \sigma =4,2,0$. Rank $r=2$. Why this step? Order biggest-first: ${\sigma }_1=4,{\sigma }_2=2,{\sigma }_3=0$.
3. $V$: columns are standard basis reordered to match — $v_1=e_1,v_2=e_3,v_3=e_2$. Why this step? $V$'s columns must appear in the same order as the ${\sigma }_i$ — $e_1$ carries $\lambda =16$, $e_3$ carries $\lambda =4$, and $e_2$ carries $\lambda =0$, so they line up as ${\sigma }_1,{\sigma }_2,{\sigma }_3$.
4. $U$: $u_1=\frac{1}{4}A{e}_1=\frac{1}{4}(-4,0,0{)}^{\top }=(-1,0,0{)}^{\top }$; $u_2=\frac{1}{2}A{e}_3=(0,0,1{)}^{\top }$; complete with $u_3=(0,1,0{)}^{\top }$. Why this step? The sign $-1$ in $u_1$ absorbs $A$'s negative entry.

Verify: singular values are $\{4,2,0\}=\{|-4|,|2|,|0|\}$ sorted — absolute values, then descending. ✓

---

Example 5 — C5: tall, rank-deficient

Forecast: the second column is exactly twice the first... how many nonzero $\sigma$?

1. Form $A^{\top }A$. $\left(\begin{array}{ccc}1& 2& 2\\ 2& 4& 4\end{array}\right)\left(\begin{array}{cc}1& 2\\ 2& 4\\ 2& 4\end{array}\right)=\left(\begin{array}{cc}9& 18\\ 18& 36\end{array}\right)$. Why this step? $A$ is $3\times 2$ (not square), so it has no eigenvalues of its own; $A^{\top }A$ is the $2\times 2$ symmetric gateway we can diagonalize.
2. Eigenvalues: $\det \left(\begin{array}{cc}9-\lambda & 18\\ 18& 36-\lambda \end{array}\right)={\lambda }^2-45\lambda =0\Rightarrow \lambda =45,0$. So ${\sigma }_1=\sqrt{45}=3\sqrt{5},{\sigma }_2=0$. Rank $r=1$. Why this step? One zero eigenvalue confirms the dependent columns.
3. $V$: for $\lambda =45$: $(9-45)x+18y=0\Rightarrow x=y/2$, so $v_1=\frac{1}{\sqrt{5}}(1,2{)}^{\top }$. For $\lambda =0$: $v_2=\frac{1}{\sqrt{5}}(2,-1{)}^{\top }$. Why this step? $v_2$ (with ${\sigma }_2=0$) spans the null space: $A{v}_2=0$.
4. $U$: $u_1=\frac{1}{3\sqrt{5}}A{v}_1=\frac{1}{3\sqrt{5}}\cdot \frac{1}{\sqrt{5}}(5,10,10{)}^{\top }=\frac{1}{3}(1,2,2{)}^{\top }$. Then extend by Gram–Schmidt to $u_2,u_3\perp u_1$; e.g. $u_2=\frac{1}{\sqrt{5}}(2,-1,0{)}^{\top }$, $u_3=\frac{1}{\sqrt{45}}(2,4,-5{)}^{\top }$. Why this step? $u_2,u_3$ span the left null space — the directions $A$ never reaches.

Verify: $\Vert A{v}_1\Vert =\Vert \frac{1}{\sqrt{5}}(5,10,10{)}^{\top }\Vert =\frac{1}{\sqrt{5}}\sqrt{225}=\frac{15}{\sqrt{5}}=3\sqrt{5}={\sigma }_1$. ✓ And $A{v}_2=\frac{1}{\sqrt{5}}A(2,-1{)}^{\top }=\frac{1}{\sqrt{5}}(0,0,0{)}^{\top }=0$. ✓

---

Example 6 — C6: wide, full row rank

Forecast: more input directions ($n=3$) than output ($m=2$). Where does the extra dimension go?

1. Form $A^{\top }A$ ($3\times 3$): $\left(\begin{array}{ccc}9& 0& 0\\ 0& 0& 0\\ 0& 0& 16\end{array}\right)$. Why this step? $A^{\top }A$ is always $n\times n$; here that is $3\times 3$.
2. Eigenvalues $16,9,0\Rightarrow {\sigma }_1=4,{\sigma }_2=3,{\sigma }_3=0$. Rank $r=2$. Why this step? ${\sigma }_i=\sqrt{{\lambda }_i}$, ordered biggest-first; the single zero eigenvalue tells us one input direction is killed even though the two rows are independent.
3. $V$: $v_1=e_3,v_2=e_1,v_3=e_2$ (the $e_2$ direction has $\sigma =0$). Why this step? $v_3=e_2$ is the null direction: $A{e}_2=(0,0{)}^{\top }$.
4. $U$ ($2\times 2$): $u_1=\frac{1}{4}A{e}_3=(0,1{)}^{\top }$; $u_2=\frac{1}{3}A{e}_1=(1,0{)}^{\top }$. $U=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$. Why this step? Only $r=2$ left singular vectors are needed and $m=2$, so $U$ is already complete.
5. $\Sigma$ is $2\times 3$: $\left(\begin{array}{ccc}4& 0& 0\\ 0& 3& 0\end{array}\right)$. Why this step? $\Sigma$ takes the shape of $A$ — the parent's third mistake box.

Verify: $U\Sigma V^{\top }$: $V^{\top }$ has rows $v_1^{\top },v_2^{\top },v_3^{\top }$. Reconstruction gives back $\left(\begin{array}{ccc}3& 0& 0\\ 0& 0& 4\end{array}\right)$. Check ${\sigma }_1^2+{\sigma }_2^2=16+9=25=3^2+4^2=$ sum of squared entries of $A$ (Frobenius). ✓

---

Example 7 — C7: wide AND rank-deficient

Forecast: wide and the rows are dependent. How many nonzero $\sigma$, and how many zero directions in total?

1. Form $A^{\top }A$ ($3\times 3$): $\left(\begin{array}{cc}1& 2\\ 1& 2\\ 0& 0\end{array}\right)\left(\begin{array}{ccc}1& 1& 0\\ 2& 2& 0\end{array}\right)=\left(\begin{array}{ccc}5& 5& 0\\ 5& 5& 0\\ 0& 0& 0\end{array}\right)$. Why this step? The safe $n\times n$ gateway; it will expose two zero eigenvalues (one from the wide shape, one from the dependent rows).
2. Eigenvalues: the top-left $2\times 2$ block $\left(\begin{array}{cc}5& 5\\ 5& 5\end{array}\right)$ gives ${\lambda }^2-10\lambda =0\Rightarrow \lambda =10,0$; the third row/column contributes another $\lambda =0$. So $\lambda =10,0,0\Rightarrow {\sigma }_1=\sqrt{10},{\sigma }_2={\sigma }_3=0$. Rank $r=1$. Why this step? Only one nonzero singular value survives — the matrix truly keeps a single direction.
3. $V$: for $\lambda =10$: $(5-10)x+5y=0\Rightarrow y=x$, so $v_1=\frac{1}{\sqrt{2}}(1,1,0{)}^{\top }$. The null space (spanning $\sigma =0$) is $v_2=\frac{1}{\sqrt{2}}(1,-1,0{)}^{\top },v_3=(0,0,1{)}^{\top }$. Why this step? $v_2,v_3$ both satisfy $A{v}_i=0$ — the whole $2$-dimensional null space of a wide rank-deficient matrix.
4. $U$ ($2\times 2$): $u_1=\frac{1}{\sqrt{10}}A{v}_1=\frac{1}{\sqrt{10}}\cdot \frac{1}{\sqrt{2}}(2,4{)}^{\top }=\frac{1}{\sqrt{5}}(1,2{)}^{\top }$. Complete with $u_2=\frac{1}{\sqrt{5}}(2,-1{)}^{\top }$. Why this step? $u_2$ spans the left null space; $A$ never reaches that output direction.

Verify: $\Vert A{v}_1\Vert =\frac{1}{\sqrt{2}}\Vert (2,4{)}^{\top }\Vert =\frac{1}{\sqrt{2}}\sqrt{20}=\sqrt{10}={\sigma }_1$. ✓ Both $A{v}_2=\frac{1}{\sqrt{2}}A(1,-1,0{)}^{\top }=(0,0{)}^{\top }$ and $A{v}_3=A(0,0,1{)}^{\top }=(0,0{)}^{\top }$. ✓ Two zero directions confirm the double $\sigma =0$.

---

Example 8 — C8: the zero / degenerate matrix

Forecast: a matrix that sends everything to the origin. What is ${\sigma }_1$?

1. Form $A^{\top }A$ $=\left(\begin{array}{cc}0& 0\\ 0& 0\end{array}\right)$. Why this step? Even a fully degenerate matrix runs through the same gateway; $A^{\top }A$ is still symmetric and its eigenvalues still give the ${\sigma }_i$.
2. Eigenvalues both $0\Rightarrow {\sigma }_1={\sigma }_2=0$. Rank $r=0$. Why this step? No stretching in any direction — total collapse.
3. $V,U$: $\Sigma =0$, so any orthonormal $U,V$ work; take $U=V=I$. Why this step? With ${\sigma }_i=0$ we can never use $u_i=A{v}_i/{\sigma }_i$ (division by $0$), so we free-choose an orthonormal completion — the parent's Gram–Schmidt extension in the extreme.

Verify: $U\Sigma V^{\top }=I\cdot 0\cdot I=0=A$. ✓ This is the limit of shrinking every ${\sigma }_i\to 0$: the geometry degenerates to a point.

---

Example 9 — C9: a word problem (stretching data)

Forecast: the biggest possible $\Vert Ax\Vert$ over unit $x$ is exactly ${\sigma }_1$ — guess it.

Figure s02 (Example 9). Left: the blue unit circle of possible unit inputs, with the best direction $v_1$ (yellow) and the erased direction $v_2$ (red). Right: after $A$, the circle collapses to the red flat segment of half-length $5$ on the $x$-axis; the yellow arrow $A{v}_1=(5,0)$ is the maximal stretch, and $v_2$ lands on the origin. Read part (a) off the yellow arrow's length and part (b) off the red arrow.

1. Model. Max amplification $={\max }_{\Vert x\Vert =1}\Vert Ax\Vert ={\sigma }_1$; erased directions have $\sigma =0$. Why this step? This is the operational meaning of singular values.
2. Form $A^{\top }A$ $=\left(\begin{array}{cc}4& 0\\ 3& 0\end{array}\right)\left(\begin{array}{cc}4& 3\\ 0& 0\end{array}\right)=\left(\begin{array}{cc}16& 12\\ 12& 9\end{array}\right)$. Why this step? The gateway again: its top eigenvalue's square root is precisely that maximal amplification factor we were asked for, and its zero eigenvalue flags the erased direction.
3. Eigenvalues: ${\lambda }^2-25\lambda =0\Rightarrow \lambda =25,0$. So ${\sigma }_1=5,{\sigma }_2=0$. Why this step? ${\sigma }_1=5$ answers (a); ${\sigma }_2=0$ flags an erased direction.
4. Right vectors: $\lambda =25$: $(16-25)x+12y=0\Rightarrow y=\frac{3}{4}x$, $v_1=\frac{1}{5}(4,3{)}^{\top }$. $\lambda =0$: $v_2=\frac{1}{5}(3,-4{)}^{\top }$. Why this step? $v_2$ is the input direction with $\sigma =0$: the answer to (b).

Verify (a): feed the best input $v_1$: $A{v}_1=\frac{1}{5}A(4,3{)}^{\top }=\frac{1}{5}(25,0{)}^{\top }=(5,0{)}^{\top }$, length $5={\sigma }_1$. ✓ Verify (b): $A{v}_2=\frac{1}{5}A(3,-4{)}^{\top }=\frac{1}{5}(0,0{)}^{\top }=0$ — the direction $(3,-4)$ (i.e. $\perp$ to $(4,3)$) is erased. ✓ See figure s02: the blue unit circle becomes the red flat segment of half-length $5$ on the $x$-axis; the perpendicular blue arrow lands on the origin.

---

Recall Which cell did each example solve?

C1→Ex1, C2→Ex2, C3→Ex3, C4→Ex4, C5→Ex5, C6→Ex6, C7→Ex7, C8→Ex8, C9→Ex9. ::: Every row of the scenario matrix is covered, including both wide-full-rank and wide-rank-deficient.

Related pages worth revisiting after this: Low-Rank Approximation (keep only the top ${\sigma }_i$), Principal Component Analysis (PCA) (SVD of centred data), and Moore-Penrose Pseudoinverse (invert by $1/{\sigma }_i$ on nonzero $\sigma$).