Exercises — Subspaces — four fundamental subspaces of a matrix

WHAT/WHY: dimensions come straight from the dimension formulas $r,r,n-r,m-r$ with $m=3$, $n=5$, $r=2$.

• $\dim C(A)=r=2$, lives in output space ${\mathbb{R}}^3$.
• $\dim C(A^T)=r=2$, lives in input space ${\mathbb{R}}^5$.
• $\dim N(A)=n-r=5-2=3$, lives in input space ${\mathbb{R}}^5$.
• $\dim N(A^T)=m-r=3-2=1$, lives in output space ${\mathbb{R}}^3$. Check the sums: input pieces $2+3=5=n$ ✓; output pieces $2+1=3=m$ ✓.

False. $\dim N(A)=n-r$ and $\dim C(A)=r$; these are equal only when $n-r=r$, i.e. $n=2r$. In general they differ (they even live in different spaces ${\mathbb{R}}^n$ vs ${\mathbb{R}}^m$). The correct "same dimension" pairing is row space and column space, both $=r$.

Step 1 — Row reduce (WHY: pivots reveal rank and free variables). $R_2\to R_2-2R_1$: $\left[\begin{array}{ccc}1& 2& 2\\ 0& 0& 1\end{array}\right]$. Clear above pivot 2: $R_1\to R_1-2R_2$: $\left[\begin{array}{ccc}1& 2& 0\\ 0& 0& 1\end{array}\right]$. Pivots in columns $1$ and $3$, so $r=2$. Step 2 — Column space (WHY: pivot columns of the original $A$ span $C(A)$). $C(A)=\text{span}\{(1,2),(2,5)\}$. Two independent vectors in ${\mathbb{R}}^2$, so $C(A)={\mathbb{R}}^2$, $\dim =2$. Step 3 — Null space (WHY: free variable = column 2). From RREF: $x_1+2x_2=0$ and $x_3=0$. Free $x_2=1$: $x_1=-2,x_3=0$. Special solution $(-2,1,0)$. $N(A)=\text{span}\{(-2,1,0)\}$, $\dim =n-r=3-2=1$. Sanity: $2+1=3=n$ ✓.

Step 1 — Row reduce. $R_2\to R_2-2R_1$, $R_3\to R_3+R_1$: $\left[\begin{array}{cc}1& 3\\ 0& 0\\ 0& 0\end{array}\right]$. One pivot, $r=1$ ($m=3,n=2$). Column space: pivot is column 1 $\Rightarrow C(B)=\text{span}\{(1,2,-1)\}$, a line in ${\mathbb{R}}^3$, $\dim =1$. Row space: the nonzero RREF row $\Rightarrow C(B^T)=\text{span}\{(1,3)\}$, a line in ${\mathbb{R}}^2$, $\dim =1$. Null space: $x_1+3x_2=0$, free $x_2=1\Rightarrow x_1=-3$. $N(B)=\text{span}\{(-3,1)\}$, $\dim =n-r=2-1=1$. Orthogonality check (input space ${\mathbb{R}}^2$): row $(1,3)$ dotted with null $(-3,1)$ $=1(-3)+3(1)=0$ ✓ — exactly the perpendicularity from Orthogonal Complements.

Observe (WHY first): rows are $(1,2,3),(1,2,3),(2,4,6)$ — row 2 equals row 1, row 3 equals $2\times$ row 1. So $r=1$. (a) Left null space. $N(A^T)=\{y:A^{T}y=0\}$, equivalently the combinations of rows giving zero. We need $y_1{r}_1+y_2{r}_2+y_3{r}_3=0$ where all rows are multiples of $(1,2,3)$: $(y_1+y_2+2y_3)(1,2,3)=0\Rightarrow y_1+y_2+2y_3=0.$ One equation, $3$ unknowns $\Rightarrow$ two free directions, $\dim =m-r=3-1=2$ ✓. Free $(y_2,y_3)$: $y_2=1,y_3=0\Rightarrow (-1,1,0)$; $y_2=0,y_3=1\Rightarrow (-2,0,1)$. $N(A^T)=\text{span}\{(-1,1,0),(-2,0,1)\}$. (b) Solvability. By Solving Ax=b, $Ax=b$ is solvable $⟺b\in C(A)⟺b\perp N(A^T)$. So dot $b$ with each basis vector of $N(A^T)$ and set to zero: $-b_1+b_2=0$ and $-2b_1+b_3=0$, i.e. $\boxed{b_2=b_1,b_3=2b_1}$. Read-off: $b$ must be a multiple of $(1,1,2)$ — which is exactly the single column direction, as expected for rank $1$.

Rank: $\dim N(A)=n-r=3-r=0\Rightarrow r=3$ (full column rank). Left null space: $\dim N(A^T)=m-r=5-3=2$. This is nonzero, so there are forbidden output directions. Solvable for every $b$? No — $\dim C(A)=r=3<5=m$, so the column space is a 3-plane inside ${\mathbb{R}}^5$. Only the $b$ lying in that 3-plane (equivalently $b\perp N(A^T)$) are reachable. This is the setting where Least Squares & Projections steps in to find the best approximate answer.

Step 1 — Reduce. $R_2\to R_2-R_1$, $R_3\to R_3-R_1$: $\left[\begin{array}{cc}1& 1\\ 0& 1\\ 0& 2\end{array}\right]$; then $R_3\to R_3-2R_2$: $\left[\begin{array}{cc}1& 1\\ 0& 1\\ 0& 0\end{array}\right]$; clear up: $\left[\begin{array}{cc}1& 0\\ 0& 1\\ 0& 0\end{array}\right]$. Two pivots, $r=2$ ($m=3,n=2$). Column space: pivot columns $1,2$ of original $A$: $C(A)=\text{span}\{(1,1,1),(1,2,3)\}$, a plane in ${\mathbb{R}}^3$, $\dim 2$. Row space: $C(A^T)=\text{span}\{(1,0),(0,1)\}={\mathbb{R}}^2$, $\dim 2$. Null space: $r=n=2\Rightarrow N(A)=\{0\}$, $\dim 0$. Left null space: $\dim =m-r=3-2=1$. Solve $y_1(1,1,1)+y_2(1,2,3)=0$ read across columns of $A^T$: $y_1+y_2+y_3=0$ and $y_1+2y_2+3y_3=0$. Subtract: $y_2+2y_3=0\Rightarrow y_2=-2y_3$; then $y_1=-y_2-y_3=2y_3-y_3=y_3$. Take $y_3=1$: $y=(1,-2,1)$. $N(A^T)=\text{span}\{(1,-2,1)\}$. (b) Perpendicular check. Dot the left-null vector with each column: $(1,-2,1)\cdot (1,1,1)=1-2+1=0$ ✓; $(1,-2,1)\cdot (1,2,3)=1-4+3=0$ ✓. So the line $N(A^T)$ is the normal to the plane $C(A)$ — see the figure below. (c) Direction split of $b=(1,0,0)$. Since $C(A)\oplus N(A^T)={\mathbb{R}}^3$ and they are perpendicular, $b$ splits uniquely as $b=b_{\parallel }+b_{\perp }$ with $b_{\parallel }\in C(A)$ (a combination of $(1,1,1),(1,2,3)$) and $b_{\perp }$ a multiple of $(1,-2,1)$. The mere existence of this clean split is the Fundamental Theorem, Part 2 — the machinery behind Least Squares & Projections.

Plan (WHY): column space $=$ span of one vector means every column is a multiple of $(1,2,2)$, so $A=(1,2,2{)}^T{v}^T$ for some row vector $v=(v_1,v_2,v_3)$ (a rank-1 matrix). The null space is then all $x$ with $v\cdot x=0$. We must pick $v$ so that $N=\{x:v\cdot x=0\}$ equals the given plane — i.e. $v$ is the normal to that plane. Find $v$ (WHY: normal ⟂ both plane directions): need $v\cdot (1,-1,0)=0$ and $v\cdot (1,0,-1)=0$: $v_1-v_2=0$ and $v_1-v_3=0\Rightarrow v=(1,1,1)$. Assemble: $A=\left[\begin{array}{c}1\\ 2\\ 2\end{array}\right]\left[\begin{array}{ccc}1& 1& 1\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 2& 2& 2\\ 2& 2& 2\end{array}\right].$ Verify. Columns all $=(1,2,2)$: $C(A)=\text{span}\{(1,2,2)\}$, $\dim 1=r$ ✓. $Ax=0⟺x_1+x_2+x_3=0$, whose solution space is exactly $\text{span}\{(1,-1,0),(1,0,-1)\}$, $\dim 2=n-r$ ✓. $\dim C(A^T)=r=1$; $\dim N(A^T)=m-r=3-1=2$. Sums $1+2=3$ both sides ✓.

Strategy: show the null spaces match, $N(A^{T}A)=N(A)$, then convert via orthogonal complements. Step 1 — $N(A)\subseteq N(A^{T}A)$: if $Ax=0$ then $A^{T}Ax=A^{T}0=0$. Immediate. Step 2 — $N(A^{T}A)\subseteq N(A)$ (the clever half): suppose $A^{T}Ax=0$. Dot with $x$: $x^T{A}^{T}Ax=0\Rightarrow (Ax{)}^T(Ax)=0\Rightarrow \Vert Ax{\Vert }^2=0\Rightarrow Ax=0.$ WHY this works: $\Vert Ax{\Vert }^2$ is a sum of squares; it is zero only when the vector itself is zero — that is the one fact that pins $Ax$ down. So $N(A^{T}A)=N(A)$, both subspaces of ${\mathbb{R}}^n$. Step 3 — flip to row spaces. In ${\mathbb{R}}^n$, the row space is the orthogonal complement of the null space: $C(M^T)=N(M{)}^{\perp }$. Since $A^{T}A$ is symmetric, its row space equals its column space, and: $C(A^{T}A)=N(A^{T}A{)}^{\perp }=N(A{)}^{\perp }=C(A^T).$ Consequence: $\text{rank}(A^{T}A)=\dim C(A^T)=\text{rank}(A)$. ∎ This is the exact fact that makes the normal equations $A^{T}A\hat{x}=A^{T}b$ solvable in Least Squares & Projections.

Cover the answers. Dimension of $N(A)$ for $m\times n$ rank $r$? ::: $n-r$. Left null space lives in which space and is perpendicular to what? ::: ${\mathbb{R}}^m$, perpendicular to $C(A)$. $Ax=b$ solvable condition in terms of $N(A^T)$? ::: $b\perp N(A^T)$ (equivalently $b\in C(A)$). Rank of $A^{T}A$ versus rank of $A$? ::: Equal.