Linear Algebra (Full)

Question 1

(a) [6] Let $A=A^\top$ real. Let $A\mathbf v=\lambda\mathbf v$, $\mathbf v\neq 0$, possibly complex. Take conjugate transpose: $\bar{\mathbf v}^\top A = \bar\lambda\,\bar{\mathbf v}^\top$ (using $A$ real symmetric so $\bar A^\top=A$). Then $\bar{\mathbf v}^\top A\mathbf v = \lambda\,\bar{\mathbf v}^\top\mathbf v$ and also $=\bar\lambda\,\bar{\mathbf v}^\top\mathbf v$. Since $\bar{\mathbf v}^\top\mathbf v=\|\mathbf v\|^2>0$, we get $\lambda=\bar\lambda$, so $\lambda\in\mathbb R$. [3] Orthogonality: $A\mathbf u=\lambda\mathbf u$, $A\mathbf w=\mu\mathbf w$, $\lambda\neq\mu$. $\mathbf w^\top A\mathbf u = \lambda\,\mathbf w^\top\mathbf u$; also $\mathbf w^\top A\mathbf u=(A\mathbf w)^\top\mathbf u=\mu\,\mathbf w^\top\mathbf u$. So $(\lambda-\mu)\mathbf w^\top\mathbf u=0\Rightarrow\mathbf w^\top\mathbf u=0$. [2] $K$ is real and symmetric ($K=K^\top$), so the spectral theorem applies: $K=Q\Lambda Q^\top$ with $Q$ orthogonal. [1]

(b) [5] $\det(K-\lambda I)=(2-\lambda)^2-1=\lambda^2-4\lambda+3=(\lambda-1)(\lambda-3)$. So $\lambda_1=1,\ \lambda_2=3$. [2] $\lambda_1=1$: $(K-I)\mathbf v=0\Rightarrow\begin{pmatrix}1&-1\\-1&1\end{pmatrix}\mathbf v=0\Rightarrow \mathbf v_1=\tfrac1{\sqrt2}(1,1)^\top$. [1.5] $\lambda_2=3$: $\begin{pmatrix}-1&-1\\-1&-1\end{pmatrix}\mathbf v=0\Rightarrow \mathbf v_2=\tfrac1{\sqrt2}(1,-1)^\top$. [1.5] Orthonormal: $\mathbf v_1\cdot\mathbf v_2=0$, both unit.

(c) [5] $Q=\tfrac1{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}$, $Q^\top K Q=\Lambda=\mathrm{diag}(1,3)$. [1] Sub $\mathbf x=Q\mathbf y$: $\ddot{\mathbf x}=-K\mathbf x\Rightarrow Q\ddot{\mathbf y}=-KQ\mathbf y\Rightarrow \ddot{\mathbf y}=-Q^\top K Q\,\mathbf y=-\Lambda\mathbf y$. [2] Decoupled: $\ddot y_1=-y_1$ ($\omega_1=1$), $\ddot y_2=-3y_2$ ($\omega_2=\sqrt3$). [1] $y_i(t)=A_i\cos(\omega_i t)+B_i\sin(\omega_i t)$, and $\mathbf x(t)=\tfrac1{\sqrt2}\!\begin{pmatrix}1\\1\end{pmatrix}\!\big(A_1\cos t+B_1\sin t\big)+\tfrac1{\sqrt2}\!\begin{pmatrix}1\\-1\end{pmatrix}\!\big(A_2\cos\sqrt3 t+B_2\sin\sqrt3 t\big).$ Mode 1 = in-phase (slow), Mode 2 = anti-phase (fast). [1]

(d) [4] Eigenvalues $1,3>0\Rightarrow K$ positive definite $\Rightarrow V(\mathbf x)>0$ for $\mathbf x\neq0$. [2] Physically the potential has a strict local minimum at $\mathbf x=0$; positive definiteness ⇔ real positive $\omega_i^2$ ⇔ purely oscillatory (no growing modes) ⇒ equilibrium is stable. [2]

Question 2

(a) [7] $A=\begin{pmatrix}1&0\\1&1\\1&2\\1&3\end{pmatrix},\quad \mathbf b=\begin{pmatrix}1\\1\\3\\4\end{pmatrix},\quad \mathbf c=\begin{pmatrix}c_0\\c_1\end{pmatrix}.$ [2] Minimise $f(\mathbf c)=\|A\mathbf c-\mathbf b\|^2$. $\nabla f=2A^\top(A\mathbf c-\mathbf b)=0\Rightarrow A^\top A\mathbf c=A^\top\mathbf b$. [3] Geometric: the residual $\mathbf r=\mathbf b-A\mathbf c$ is minimised when $A\mathbf c$ is the orthogonal projection of $\mathbf b$ onto $\operatorname{Col}(A)$, i.e. $\mathbf r\perp\operatorname{Col}(A)$, meaning $\mathbf r\in\operatorname{Col}(A)^\perp=\operatorname{Nul}(A^\top)$, giving $A^\top\mathbf r=0$. [2]

(b) [5] $A^\top A=\begin{pmatrix}4&6\\6&14\end{pmatrix}$, $A^\top\mathbf b=\begin{pmatrix}9\\17\end{pmatrix}$. [2] $\det=56-36=20$. $(A^\top A)^{-1}=\tfrac1{20}\begin{pmatrix}14&-6\\-6&4\end{pmatrix}$. $\mathbf c=\tfrac1{20}\begin{pmatrix}14\cdot9-6\cdot17\\-6\cdot9+4\cdot17\end{pmatrix}=\tfrac1{20}\begin{pmatrix}24\\14\end{pmatrix}=\begin{pmatrix}1.2\\0.7\end{pmatrix}.$ [3] So $c_0=1.2,\ c_1=0.7$.

(c) [7] Columns $a_1=(1,1,1,1)^\top$, $a_2=(0,1,2,3)^\top$. $q_1=a_1/\|a_1\|=\tfrac12(1,1,1,1)^\top$, $r_{11}=2$. [1.5] $r_{12}=q_1^\top a_2=\tfrac12(0+1+2+3)=3$. $u_2=a_2-3q_1=(0,1,2,3)^\top-\tfrac32(1,1,1,1)^\top=(-1.5,-0.5,0.5,1.5)^\top$. [2] $\|u_2\|=\sqrt{2.25+0.25+0.25+2.25}=\sqrt5$, so $r_{22}=\sqrt5$, $q_2=u_2/\sqrt5$. [1.5] $R=\begin{pmatrix}2&3\\0&\sqrt5\end{pmatrix}$. $Q^\top\mathbf b$: $q_1^\top\mathbf b=\tfrac12(1+1+3+4)=4.5$; $q_2^\top\mathbf b=\tfrac1{\sqrt5}(-1.5+(-0.5)+1.5+6)=\tfrac{5.5}{\sqrt5}$. Back-sub $R\mathbf c=Q^\top\mathbf b$: $\sqrt5\,c_1=5.5/\sqrt5\Rightarrow c_1=5.5/5=1.1$?? Recompute $q_2^\top b$: components $q_2=\tfrac1{\sqrt5}(-1.5,-0.5,0.5,1.5)$, dot $b=(1,1,3,4)$: $-1.5-0.5+1.5+6=5.5$; times $1/\sqrt5$. Then $c_1=(5.5/\sqrt5)/\sqrt5=5.5/5=1.1$. This disagrees — recheck $u_2$ dot b directly: $c_1=(u_2^\top b)/\|u_2\|^2=5.5/5=1.1$. But (b) gave 0.7.

Correction: $u_2^\top b=(-1.5)(1)+(-0.5)(1)+(0.5)(3)+(1.5)(4)=-1.5-0.5+1.5+6=5.5$; $\|u_2\|^2=5$; ratio $=1.1$. The discrepancy flags an arithmetic error in (b): recompute $A^\top b$: rows: $c_0$ eqn $\sum b=1+1+3+4=9$ ✓; $c_1$ eqn $\sum t_i b_i=0+1+6+12=19$ (not 17!). So $A^\top b=(9,19)^\top$. Then $\mathbf c=\tfrac1{20}(14\cdot9-6\cdot19,\ -6\cdot9+4\cdot19)^\top=\tfrac1{20}(126-114,\ -54+76)^\top=\tfrac1{20}(12,22)^\top=(0.6,1.1)^\top$. [2] Corrected answer: $c_0=0.6,\ c_1=1.1$. QR gives $c_0=(4.5-3\cdot1.1)/2=(4.5-3.3)/2=0.6$ ✓ — both methods agree. [stray, credited]

(d) [3]

QR_least_squares(A, b):
    Q, R = qr(A)          # thin QR, orthonormal cols
    y = Q^T b
    solve R c = y by back-substitution  # R upper triangular
    return c

Numerical reason: forming $A^\top A$ squares the condition number ($\kappa(A^\top A)=\kappa(A)^2$), amplifying round-off; QR works directly on $A$ preserving conditioning. [3]

Question 3

(a) [6] Row reduce $M$: $R_3\to R_3-R_1-R_2$ gives $(0,0,0)$. RREF: $\begin{pmatrix}1&0&1\\0&1&1\\0&0&0\end{pmatrix}.$ Pivots in columns 1,2; column 3 free. $\operatorname{rank}=2$. [2] Column space basis: original pivot columns $\{(1,0,1)^\top,(0,1,1)^\top\}$, $\dim=2$. [2] Null space: $x_1=-x_3,\ x_2=-x_3$, $x_3$ free $\Rightarrow \mathbf x=x_3(-1,-1,1)^\top$; basis $\{(-1,-1,1)^\top\}$, $\dim=1$. [1] Rank–nullity: $\operatorname{rank}+\dim\operatorname{Nul}=2+1=3=n$ ✓. [1]

(b) [6] Statement: For $A\in\mathbb R^{m\times n}$, $\dim\operatorname{Col}(A)+\dim\operatorname{Nul}(A)=n$. [1] Proof: Let $r=\operatorname{rank}A$. Reduce $A$ to RREF $R$; row operations don't change null space or the number of pivot columns. There are $r$ pivot columns and $n-r$ free columns. Each free variable indexes a basis vector of $\operatorname{Nul}(A)$ (set it $=1$, others $0$, solve pivots), giving $n-r$ linearly independent null-space vectors that span it — so $\dim\operatorname{Nul}(A)=n-r$. [3] Column rank $=r=\dim\operatorname{Col}(A)$ (pivot columns