Linear Algebra (Full)

Question 1

(a) Start with $A$. $R_2 \to R_2 - 2R_1$, $R_3 \to R_3 + R_1$: $\begin{pmatrix} 1 & 2 & 0 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 1 & 1 \end{pmatrix}.$ $R_3 \to R_3 - R_2$: $\text{RREF} = \begin{pmatrix} 1 & 2 & 0 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix}.$ (3) — correct elimination steps (1), zero row (1), leading 1s in reduced form (1).

(b) Pivots in columns 1 and 3, so rank = 2. Basis for column space = columns 1 and 3 of original $A$: $\left\{\begin{pmatrix}1\\2\\-1\end{pmatrix}, \begin{pmatrix}0\\1\\1\end{pmatrix}\right\}.$ (3) — rank (1), identify pivot columns (1), original columns (1).

(c) Free variables $x_2, x_4$. From RREF: $x_1 = -2x_2 - x_4$, $x_3 = -x_4$. $\mathbf{x} = x_2\begin{pmatrix}-2\\1\\0\\0\end{pmatrix} + x_4\begin{pmatrix}-1\\0\\-1\\1\end{pmatrix}.$ Null space basis: those two vectors; nullity $=2$. Rank–nullity: $\text{rank} + \text{nullity} = 2+2 = 4 =$ number of columns. ✓ (4) — back-solve (1), two basis vectors (2), verification (1).

Question 2

(a) Characteristic polynomial: $\det(M-\lambda I) = (2-\lambda)^2 - 1 = \lambda^2 -4\lambda +3 = (\lambda-1)(\lambda-3)$. Eigenvalues $\lambda_1 = 3$, $\lambda_2 = 1$.

• $\lambda=3$: $(M-3I)=\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\Rightarrow \mathbf{v}_1=(1,1)^{\mathsf T}$.
• $\lambda=1$: $(M-I)=\begin{pmatrix}1&1\\1&1\end{pmatrix}\Rightarrow \mathbf{v}_2=(1,-1)^{\mathsf T}$.

Orthonormalize: $\hat{\mathbf v}_1 = \tfrac{1}{\sqrt2}(1,1)^{\mathsf T}$, $\hat{\mathbf v}_2 = \tfrac{1}{\sqrt2}(1,-1)^{\mathsf T}$. (5) — char poly (1), eigenvalues (1), eigenvectors (2), normalization (1).

(b) $Q = \frac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix},\quad D=\begin{pmatrix}3&0\\0&1\end{pmatrix},\quad M=QDQ^{\mathsf T}.$ Since $M$ is symmetric with all eigenvalues $>0$ (3 and 1), $q(\mathbf x)=\mathbf x^{\mathsf T}M\mathbf x$ is positive definite. (5) — $Q,D$ (2), $Q^{\mathsf T}=Q^{-1}$ orthogonality (1), positive-definite criterion via eigenvalues (2).

Question 3

(a) $\vec{PQ}=(2,1,-1)$, $\vec{PR}=(1,2,-2)$. $\vec{PQ}\times\vec{PR} = \begin{vmatrix} \mathbf i&\mathbf j&\mathbf k\\2&1&-1\\1&2&-2\end{vmatrix} = (1\cdot(-2)-(-1)\cdot2,\ (-1)\cdot1-2\cdot(-2),\ 2\cdot2-1\cdot1) = (0,3,3).$ $|\vec{PQ}\times\vec{PR}| = \sqrt{0+9+9}=3\sqrt2$. Area $=\tfrac12(3\sqrt2)=\dfrac{3\sqrt2}{2}$. (4) — two edge vectors (1), cross product (2), area (1).

(b) Normal $\mathbf n=(0,3,3)$, or simplified $(0,1,1)$. Plane through $P=(1,0,2)$: $0(x-1)+1(y-0)+1(z-2)=0 \Rightarrow y+z=2.$ (3) — normal (1), point substitution (1), equation (1).

(c) Distance $=\dfrac{|0+0-2|}{\sqrt{0+1+1}} = \dfrac{2}{\sqrt2}=\sqrt2$. (3) — distance formula (1), substitution (1), value (1).

Question 4

(a) $A=\begin{pmatrix}1&0\\1&1\\1&2\\1&3\end{pmatrix}$, $\mathbf y=(1,1,2,2)^{\mathsf T}$, $\mathbf c=(a,b)^{\mathsf T}$. $A^{\mathsf T}A=\begin{pmatrix}4&6\\6&14\end{pmatrix},\qquad A^{\mathsf T}\mathbf y=\begin{pmatrix}6\\11\end{pmatrix}.$ (Sums: $\sum x=6$, $\sum x^2=14$, $\sum y=6$, $\sum xy = 0+1+4+6=11$.) (4) — matrix $A$ (1), $A^{\mathsf T}A$ (1.5), $A^{\mathsf T}\mathbf y$ (1.5).

(b) Solve $\begin{pmatrix}4&6\\6&14\end{pmatrix}\begin{pmatrix}a\\b\end{pmatrix}=\begin{pmatrix}6\\11\end{pmatrix}$. Determinant $=4\cdot14-36=20$. $a=\frac{6\cdot14-6\cdot11}{20}=\frac{84-66}{20}=\frac{18}{20}=0.9,\quad b=\frac{4\cdot11-6\cdot6}{20}=\frac{44-36}{20}=\frac{8}{20}=0.4.$ Fit: $y=0.9+0.4x$. (4) — determinant (1), Cramer/elimination (2), values (1).

(c) Predictions: at $x=0,1,2,3$: $0.9, 1.3, 1.7, 2.1$. Residuals: $1-0.9=0.1$; $1-1.3=-0.3$; $2-1.7=0.3$; $2-2.1=-0.1$. RSS $=0.01+0.09+0.09+0.01=0.20$. (2) — predictions/residuals (1), RSS (1).

Question 5

(a) $B^{\mathsf T}B=\begin{pmatrix}3&4\\0&5\end{pmatrix}\begin{pmatrix}3&0\\4&5\end{pmatrix}=\begin{pmatrix}25&20\\20&25\end{pmatrix}$. Char poly: $(25-\lambda)^2-400 = \lambda^2 -50\lambda +225 = (\lambda-45)(\lambda-5)$. Eigenvalues $45$ and $5$. (4) — $B^{\mathsf T}B$ (2), eigenvalues (2).

(b) Singular values $\sigma_i=\sqrt{\lambda_i}$: $\sigma_1=\sqrt{45}=3\sqrt5$, $\sigma_2=\sqrt5$. (3)

(c) Direct: $\det B = 3\cdot5 - 0\cdot4 = 15$. Product of singular values: $3\sqrt5\cdot\sqrt5 = 3\cdot5 = 15$. $|\det B| = 15 = \sigma_1\sigma_2$. ✓ (3) — direct det (1), product (1), agreement (1).

[
  {"claim":"Q2 eigenvalues of M are 1 and 3","code":"M=Matrix([[2,1],[1,2]]); ev=sorted(M.eigenvals().keys()); result=(ev==[1,3])"},
  {"claim":"Q3 area of triangle PQR is 3*sqrt(2)/2","code":"P=Matrix([1,0,2]); Q=Matrix([3,1,1]); R=Matrix([2,2,0]); c=(Q-P).cross(R-P); area=c.norm()/2; result=simplify(area-3*sqrt(2)/2)==0"},
  {"claim":"Q4 least squares solution a=0.9, b=0.4","code":"A=Matrix([[1,0],[1,1],[1,2],[1,3]]); y=Matrix([1,1,2,2]); c=(A.T*A).inv()*A.T*y; result=(c==Matrix([Rational(9,10),Rational(2,5)]))"},
  {"claim":"Q4 RSS equals 0.2","code":"A=Matrix([[1,0],[1,1],[1,2],[1,3]]); y=Matrix([1,1,2,2]); c=(A.T*A).inv()*A.T*y; r=y-A*c; rss=(r.T*r)[0]; result=simplify(rss-Rational(1,5))==0"},
  {"claim":"Q5 singular values of B are 3*sqrt(5) and sqrt(5)","code":"B=Matrix([[3,0],[4,5]]); ev=sorted((B.T*B).eigenvals().keys()); svs=sorted([sqrt(e) for e in ev]); result=(svs==sorted([sqrt(5),3*sqrt(5)]))"}
]