Noesis
Level 4 — ApplicationLinear Algebra Essentials

Linear Algebra Essentials

60 marksprintable — key stays hidden on paper

Level: 4 (Application — novel problems, no hints) Time: 60 minutes Total Marks: 60

Question 1. (12 marks) A data preprocessing pipeline uses the matrix

A=[2013].A = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix}.

(a) Compute the eigenvalues and a set of eigenvectors of AA. (5)

(b) Write the eigendecomposition A=PDP−1A = P D P^{-1} explicitly (state PP, DD, P−1P^{-1}). (4)

(c) Using the eigendecomposition, compute A3A^{3} without directly multiplying AA three times. (3)

Question 2. (12 marks) Consider the feature vectors u=(3,4,0)\mathbf{u} = (3, 4, 0) and v=(1,2,2)\mathbf{v} = (1, 2, 2).

(a) Compute ∥u∥1\|\mathbf{u}\|_1, ∥u∥2\|\mathbf{u}\|_2, and ∥u∥∞\|\mathbf{u}\|_\infty. (3)

(b) Compute the angle θ\theta between u\mathbf{u} and v\mathbf{v} (give cos⁡θ\cos\theta exactly and θ\theta to the nearest degree). (4)

(c) Compute the vector projection of v\mathbf{v} onto u\mathbf{u}. (3)

(d) A recommender normalizes vectors to unit L2L_2 norm before taking dot products. Explain what geometric quantity the resulting dot product then measures. (2)

Question 3. (12 marks) Solve the linear system by Gaussian elimination and analyze it:

x + 2y + z &= 4 \\ 2x + 5y + 3z &= 7 \\ x + 3y + 2z &= 3 \end{aligned}$$ (a) Reduce the augmented matrix to row echelon form and find the solution. **(7)** (b) State the rank of the coefficient matrix and confirm consistency. **(3)** (c) Compute the determinant of the coefficient matrix and relate its value to invertibility. **(2)** --- **Question 4. (12 marks)** Let $M = \begin{bmatrix} 5 & 2 \\ 2 & 2 \end{bmatrix}$ be a covariance-like matrix. (a) Show $M$ is symmetric and determine whether it is positive definite using the leading principal minors test. **(5)** (b) Write the quadratic form $q(\mathbf{x}) = \mathbf{x}^T M \mathbf{x}$ for $\mathbf{x}=(x_1,x_2)$ explicitly as a polynomial. **(3)** (c) Evaluate $q$ at $\mathbf{x}=(1,-1)$ and confirm it is consistent with your answer in (a). **(2)** (d) State a relationship between the eigenvalues of a symmetric matrix and its positive definiteness. **(2)** --- **Question 5. (12 marks)** Consider the matrix $B = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 0 \end{bmatrix}$ (a $2 \times 3$ matrix). (a) Compute $B B^T$ and $B^T B$, and state their dimensions. **(4)** (b) Find the singular values of $B$ (use eigenvalues of $BB^T$). **(4)** (c) State $\operatorname{rank}(B)$ and give a basis for the null space of $B$. **(3)** (d) State one property connecting $\operatorname{trace}(BB^T)$ to the singular values. **(1)** --- *End of paper.*
Answer keyMark scheme & solutions

Question 1

(a) Characteristic polynomial: det⁡(A−λI)=(2−λ)(3−λ)−0=0\det(A-\lambda I) = (2-\lambda)(3-\lambda) - 0 = 0 (lower triangular). (1) Eigenvalues λ1=2\lambda_1 = 2, λ2=3\lambda_2 = 3. (1) For λ=3\lambda=3: (A−3I)v=0⇒[−1010]v=0⇒v1=0(A-3I)\mathbf{v}=0 \Rightarrow \begin{bmatrix}-1&0\\1&0\end{bmatrix}\mathbf{v}=0 \Rightarrow v_1=0, so v2=(0,1)\mathbf{v}_2=(0,1). (1.5) For λ=2\lambda=2: [0011]v=0⇒v1=−v2\begin{bmatrix}0&0\\1&1\end{bmatrix}\mathbf{v}=0 \Rightarrow v_1=-v_2, so v1=(1,−1)\mathbf{v}_1=(1,-1). (1.5)

(b) Ordering (λ=2,λ=3)(\lambda=2,\lambda=3): P=[10−11],D=[2003],P−1=[1011].P=\begin{bmatrix}1&0\\-1&1\end{bmatrix},\quad D=\begin{bmatrix}2&0\\0&3\end{bmatrix},\quad P^{-1}=\begin{bmatrix}1&0\\1&1\end{bmatrix}. (P columns = eigenvectors, D = eigenvalues.) (4) (1 each for P, D, P⁻¹, consistency)

(c) A3=PD3P−1A^3 = P D^3 P^{-1}, D3=diag⁡(8,27)D^3=\operatorname{diag}(8,27). (1) A3=[10−11][80027][1011]=[801927].A^3 = \begin{bmatrix}1&0\\-1&1\end{bmatrix}\begin{bmatrix}8&0\\0&27\end{bmatrix}\begin{bmatrix}1&0\\1&1\end{bmatrix}=\begin{bmatrix}8&0\\19&27\end{bmatrix}. (2)

Question 2

(a) ∥u∥1=3+4+0=7\|\mathbf{u}\|_1 = 3+4+0=7; ∥u∥2=9+16=5\|\mathbf{u}\|_2=\sqrt{9+16}=5; ∥u∥∞=4\|\mathbf{u}\|_\infty=4. (3)

(b) u⋅v=3+8+0=11\mathbf{u}\cdot\mathbf{v}=3+8+0=11. ∥v∥2=1+4+4=3\|\mathbf{v}\|_2=\sqrt{1+4+4}=3. (2) cos⁡θ=115⋅3=1115≈0.7333\cos\theta = \dfrac{11}{5\cdot 3}=\dfrac{11}{15}\approx0.7333, so θ≈42.8∘≈43∘\theta\approx 42.8^\circ \approx 43^\circ. (2)

(c) projuv=u⋅v∥u∥2u=1125(3,4,0)=(1.32,1.76,0)\text{proj}_\mathbf{u}\mathbf{v} = \dfrac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{u}\|^2}\mathbf{u}=\dfrac{11}{25}(3,4,0)=(1.32,1.76,0). (3)

(d) After unit-normalization u^⋅v^=cos⁡θ\mathbf{\hat u}\cdot\mathbf{\hat v}=\cos\theta; the dot product measures the cosine similarity (angle) between the vectors, independent of magnitude. (2)

Question 3

(a) Augmented matrix: [121425371323]\left[\begin{array}{ccc|c}1&2&1&4\\2&5&3&7\\1&3&2&3\end{array}\right] R2→R2−2R1R_2\to R_2-2R_1: (0,1,1,−1)(0,1,1,-1); R3→R3−R1R_3\to R_3-R_1: (0,1,1,−1)(0,1,1,-1). (2) R3→R3−R2R_3\to R_3-R_2: (0,0,0,0)(0,0,0,0). Echelon form: [1214011−10000].\left[\begin{array}{ccc|c}1&2&1&4\\0&1&1&-1\\0&0&0&0\end{array}\right]. (2) Free variable z=tz=t. From row 2: y=−1−ty=-1-t. From row 1: x=4−2y−z=4−2(−1−t)−t=6+tx=4-2y-z=4-2(-1-t)-t=6+t. (2) Solution: (x,y,z)=(6+t, −1−t, t)(x,y,z)=(6+t,\,-1-t,\,t), t∈Rt\in\mathbb{R} — infinitely many solutions. (1)

(b) Rank of coefficient matrix =2=2 (two nonzero rows). Rank of augmented =2=2 too, so the system is consistent (with 3−2=13-2=1 free parameter). (3)

(c) det⁡=1(5⋅2−3⋅3)−2(2⋅2−3⋅1)+1(2⋅3−5⋅1)=1(1)−2(1)+1(1)=0\det = 1(5\cdot2-3\cdot3)-2(2\cdot2-3\cdot1)+1(2\cdot3-5\cdot1) = 1(1)-2(1)+1(1)=0. (1) det⁡=0⇒\det=0 \Rightarrow matrix is singular / non-invertible, consistent with rank <3<3 and non-unique solution. (1)

Question 4

(a) MT=MM^T=M since off-diagonals equal (both 22) → symmetric. (1) Leading minors: M1=5>0M_1 = 5 > 0; M2=det⁡M=5⋅2−2⋅2=6>0M_2=\det M = 5\cdot2 - 2\cdot2 = 6 > 0. (3) Both positive → MM is positive definite (Sylvester's criterion). (1)

(b) q(x)=5x12+4x1x2+2x22q(\mathbf{x}) = 5x_1^2 + 4x_1x_2 + 2x_2^2. (Cross term =2⋅M12x1x2=4x1x2= 2\cdot M_{12}x_1x_2 = 4x_1x_2.) (3)

(c) q(1,−1)=5(1)+4(1)(−1)+2(1)=5−4+2=3>0q(1,-1)=5(1)+4(1)(-1)+2(1)=5-4+2=3>0, consistent with positive definiteness (positive for all nonzero x\mathbf{x}). (2)

(d) A symmetric matrix is positive definite iff all its eigenvalues are strictly positive (positive semidefinite iff all ≥0\geq 0). (2)

Question 5

(a) BBT=[101010][100110]=[2001] (2×2).BB^T = \begin{bmatrix}1&0&1\\0&1&0\end{bmatrix}\begin{bmatrix}1&0\\0&1\\1&0\end{bmatrix}=\begin{bmatrix}2&0\\0&1\end{bmatrix}\ (2\times2). (2) BTB=[101010101] (3×3).B^TB=\begin{bmatrix}1&0&1\\0&1&0\\1&0&1\end{bmatrix}\ (3\times3). (2)

(b) Eigenvalues of BBTBB^T are 22 and 11; singular values are their square roots: σ1=2, σ2=1\sigma_1=\sqrt2,\ \sigma_2=1. (4)

(c) Two nonzero singular values → rank⁡(B)=2\operatorname{rank}(B)=2. (1) Null space: solve Bx=0B\mathbf{x}=0: x1+x3=0, x2=0⇒x=t(1,0,−1)x_1+x_3=0,\ x_2=0 \Rightarrow \mathbf{x}=t(1,0,-1). Basis {(1,0,−1)}\{(1,0,-1)\}. (2)

(d) trace⁡(BBT)=∑iσi2=2+1=3\operatorname{trace}(BB^T)=\sum_i \sigma_i^2 = 2+1=3 (sum of squared singular values = Frobenius norm squared). (1)

[
{"claim":"A^3 = [[8,0],[19,27]] via eigendecomposition","code":"A=Matrix([[2,0],[1,3]]); result = (A**3 == Matrix([[8,0],[19,27]]))"},
{"claim":"cos angle u,v = 11/15","code":"u=Matrix([3,4,0]); v=Matrix([1,2,2]); c=(u.dot(v))/(u.norm()*v.norm()); result = simplify(c - Rational(11,15))==0"},
{"claim":"det of Q3 coefficient matrix is 0","code":"C=Matrix([[1,2,1],[2,5,3],[1,3,2]]); result = (C.det()==0)"},
{"claim":"M positive definite: minors 5>0 and det=6>0","code":"M=Matrix([[5,2],[2,2]]); result = (M[0,0]>0) and (M.det()==6)"},
{"claim":"singular values of B are sqrt(2) and 1","code":"B=Matrix([[1,0,1],[0,1,0]]); ev=sorted((B*B.T).eigenvals().keys()); result = (set(ev)=={1,2})"}
]