NumPy linear algebra — np.linalg.solve, eig, svd, norm, det: Exercises

This page is a self-testing ladder. Each problem states what to compute; the answer hides inside a collapsible [!recall]- callout so you can try first, then reveal. Difficulty climbs L1 Recognition → L2 Application → L3 Analysis → L4 Synthesis → L5 Mastery.

Everything here builds on the parent topic. If a word feels unfamiliar, the parent note defines it — but every symbol below is re-earned as we go.

Level 1 — Recognition

(Can you read the notation and pick the right tool?)

Exercise 1.1

You are given a square matrix A and a vector b. You want the x that satisfies $A x = b$ . Which one NumPy call do you use, and which call must you avoid?

Recall Solution 1.1

Use x = np.linalg.solve(A, b). Avoid np.linalg.inv(A) @ b.

$A$ is the machine, $b$ is where an arrow landed, $x$ is where it started. solve runs the machine backward directly.
Inverting first computes a whole matrix $A^{- 1}$ you don't need, costs more, and amplifies rounding error.

Exercise 1.2

For the vector $v = (3, - 4)$ , state the numeric value of np.linalg.norm(v) (the default). What does that number mean geometrically?

Recall Solution 1.2

Default norm is the Euclidean ( $ℓ_{2}$ ) length: $3^{2} + (- 4)^{2} = 9 + 16 = 25 = 5$ . Geometrically it is the straight-line distance from the origin to the tip of the arrow $(3, - 4)$ — the hypotenuse of a right triangle with legs $3$ and $4$ .

Exercise 1.3

np.linalg.eig(A) returns w, V. If you want the second eigenvector, do you write V[1, :] or V[:, 1]?

Recall Solution 1.3

V[:, 1] — eigenvectors are the columns of V. V[1, :] would grab a row, which is not an eigenvector.

Level 2 — Application

(Turn a small hand computation into the right call.)

Exercise 2.1

Solve by hand, then confirm the NumPy result: ${3 x + 2 y = 7 x + 2 y = 5$

Recall Solution 2.1

WHAT: subtract row 2 from row 1 to kill a variable. WHY: we want a triangular (one-unknown-at-a-time) system — this is Gaussian elimination. $R_{1} - R_{2} : (3 x - x) + (2 y - 2 y) = 7 - 5 \Rightarrow 2 x = 2 \Rightarrow x = 1.$ Back-substitute into row 2: $1 + 2 y = 5 \Rightarrow 2 y = 4 \Rightarrow y = 2.$ So $x = 1, y = 2$ .

A = np.array([[3.,2.],[1.,2.]]); b = np.array([7.,5.])
np.linalg.solve(A, b)   # array([1., 2.])

Exercise 2.2

Compute $det (4265)$ by hand. What does its sign and size tell you?

Recall Solution 2.2

WHAT/WHY: for a $2 \times 2$ matrix, $det = a d - b c$ is the signed area of the parallelogram spanned by the two columns $(4, 2)$ and $(6, 5)$ — it measures how much the machine scales area. (See figure below.) $det = (4) (5) - (6) (2) = 20 - 12 = 8.$ Positive and non-zero → the machine scales area by $8 \times$ , keeps orientation (no flip), and is invertible (a unique solve exists).

Figure — NumPy linear algebra — np.linalg.solve, eig, svd, norm, det

Exercise 2.3

For $v = (1, - 2, 2)$ , compute all three norms: $∥ v ∥_{2}$ , $∥ v ∥_{1}$ , $∥ v ∥_{\infty}$ .

Recall Solution 2.3

$∥ v ∥_{2} = 1^{2} + (- 2)^{2} + 2^{2} = 1 + 4 + 4 = 9 = 3.$
$∥ v ∥_{1} = ∣1∣ + ∣ - 2∣ + ∣2∣ = 5$ (total taxi-cab distance).
$∥ v ∥_{\infty} = max (1, 2, 2) = 2$ (the largest single component).

v=np.array([1.,-2.,2.])
np.linalg.norm(v), np.linalg.norm(v,1), np.linalg.norm(v,np.inf)  # 3.0, 5.0, 2.0

Level 3 — Analysis

(Reason about why a result comes out the way it does.)

Exercise 3.1

Find the eigenvalues of $A = (2112)$ by hand via the characteristic equation, then name the eigenvectors and explain their meaning.

Recall Solution 3.1

WHAT: an eigenvector $v \neq = 0$ satisfies $A v = λ v$ — the machine only scales it, never turns it. Rearranged: $(A - λ I) v = 0$ . WHY set the determinant to zero: for a nonzero $v$ to exist, $A - λ I$ must squash space (be singular), i.e. $det (A - λ I) = 0$ . $det (2 - λ 1 1 2 - λ) = (2 - λ)^{2} - 1 = 0.$ So $(2 - λ)^{2} = 1 \Rightarrow 2 - λ = \pm 1 \Rightarrow λ = 1 or λ = 3.$

For $λ = 3$ : $(A - 3 I) v = 0 \Rightarrow (- 1 1 1 - 1) v = 0 \Rightarrow v \propto (1, 1)$ .
For $λ = 1$ : $(1111) v = 0 \Rightarrow v \propto (1, - 1)$ .

Meaning: along the diagonal $(1, 1)$ the machine stretches by $3$ ; along the anti-diagonal $(1, - 1)$ it leaves length unchanged ( $λ = 1$ ). These two directions never rotate.

Exercise 3.2

A matrix has np.linalg.det(A) equal to 2.3e-17. A classmate concludes "the determinant is zero, so $A$ is singular." Steel-man and then correct this.

Recall Solution 3.2

The reasoning: in exact math, $det = 0 \Leftrightarrow$ singular, so a value this tiny looks like "zero with rounding." The correction: floating point almost never returns exactly $0$ , and the size of det is not scale-free — a genuinely invertible matrix scaled by $0.001$ everywhere has a determinant near $1 0^{- 9}$ per row and looks "small" without being singular. The reliable test is the condition number np.linalg.cond(A): large (say $> 1 0^{12}$ ) means untrustworthy solves. See Condition number and numerical stability.

Exercise 3.3

For $A = (30 0 - 2)$ , its eigenvalues are $3$ and $- 2$ . Its singular values are $3$ and $2$ . Explain why one is negative but both singular values are positive.

Recall Solution 3.3

Eigenvalues can be negative: $λ = - 2$ means the $y$ -axis direction is scaled by $- 2$ , i.e. stretched by $2$ and flipped. Singular values only report the stretch amount, never the flip, so they are $σ = ∣ - 2 ∣ = 2$ and $3$ . Formally, $σ_{i}^{2}$ are eigenvalues of $A^{⊤} A$ : here $A^{⊤} A = (9004)$ , whose eigenvalues $9, 4$ give $σ = 3, 2$ . Squaring erased the sign. See Eigenvalues and diagonalization.

Level 4 — Synthesis

(Combine two or more tools.)

Exercise 4.1

You must fit a line $y = m x + c$ through three points $(0, 1), (1, 2), (2, 2)$ — an over-determined system (3 equations, 2 unknowns). Set it up as $A θ = y$ and explain which NumPy tool solves it, and why solve cannot.

Recall Solution 4.1

Each point gives $m x_{i} + c = y_{i}$ . Stack them: $A = 012111, θ = (m c), y = 122 .$ np.linalg.solve needs a square $A$ — here $A$ is $3 \times 2$ , so it errors. We want the least-squares best fit: minimize $∥ A θ - y ∥_{2}$ . Use np.linalg.lstsq(A, y, rcond=None) (which uses SVD under the hood). See Least squares regression. Normal equations by hand: $θ = (A^{⊤} A)^{- 1} A^{⊤} y$ . $A^{⊤} A = (5333)$ , $A^{⊤} y = (0 \cdot 1 + 1 \cdot 2 + 2 \cdot 2 1 + 2 + 2) = (65)$ . Solve $(5333) θ = (65)$ : $det = 15 - 9 = 6$ , so $θ = \frac{1}{6} (3 - 3 - 3 5) (65) = \frac{1}{6} (18 - 15 - 18 + 25) = \frac{1}{6} (37) = (0.5 \frac{7}{6}) .$ Best line: $m = 0.5, c = 7/6 \approx 1.1667$ .

Exercise 4.2

Verify the identity $det (A) = λ_{1} λ_{2}$ for $A = (2112)$ (eigenvalues from Ex. 3.1), and also that the trace (sum of diagonal) equals $λ_{1} + λ_{2}$ .

Recall Solution 4.2

Eigenvalues were $λ = 1, 3$ .

Product: $λ_{1} λ_{2} = 1 \cdot 3 = 3$ . Determinant: $det = 2 \cdot 2 - 1 \cdot 1 = 3$ . ✓
Sum: $1 + 3 = 4$ . Trace: $2 + 2 = 4$ . ✓ WHY: the determinant is the total volume scaling, which is the product of the per-direction stretches (the eigenvalues); the trace equals their sum. These are cheap sanity checks on any eigen-computation.

Level 5 — Mastery

(Deep, multi-step, connects to real applications.)

Exercise 5.1

Compute the SVD of $A = 200003$ by hand: find the singular values, and give the rank. Then explain how truncating the smaller singular value would compress this matrix.

Recall Solution 5.1

WHAT: SVD writes $A = U Σ V^{⊤}$ ; singular values $σ_{i}$ are the diagonal of $Σ$ , obtained from $eig (A^{⊤} A)$ . $A^{⊤} A = (200003) 200003 = (4009) .$ Eigenvalues $4, 9$ → $σ = 9, 4 = 3, 2$ (sorted descending). Rank = number of nonzero singular values $= 2$ . Compression: keeping only the top singular value $σ_{1} = 3$ gives the best rank-1 approximation $σ_{1} u_{1} v_{1}^{⊤}$ ; the discarded energy is $σ_{2}^{2} = 4$ . This is exactly the idea behind low-rank image compression and Principal Component Analysis (PCA).

Exercise 5.2

The matrix $A = (12 2 4.0001)$ is almost singular. (a) Compute $det$ . (b) Argue why solving $A x = b$ here is dangerous, and which single NumPy call quantifies the danger.

Recall Solution 5.2

(a) $det = 1 \cdot 4.0001 - 2 \cdot 2 = 4.0001 - 4 = 0.0001$ . (b) The columns $(1, 2)$ and $(2, 4.0001)$ are nearly parallel — the parallelogram they span is a razor-thin sliver (area $= 0.0001$ ). A tiny change in $b$ swings $x$ wildly, because the machine barely spreads inputs across directions. np.linalg.det alone is misleading (it's small but nonzero); the honest measure is np.linalg.cond(A), the condition number, which is large here. See Condition number and numerical stability and Vector norms and metric spaces.

Exercise 5.3

A rotation matrix $R = (cos θ sin θ - sin θ cos θ)$ turns arrows without stretching. Predict its determinant, its singular values, and the nature of its eigenvalues (real or complex?), then explain each geometrically. Use $θ = 9 0^{\circ}$ as a concrete check.

Recall Solution 5.3

Determinant: $det = cos^{2} θ + sin^{2} θ = 1$ — area is preserved (a rotation squishes nothing).
Singular values: both $= 1$ ; a pure rotation stretches by $1$ in every direction ( $R^{⊤} R = I$ ).
Eigenvalues: generally complex ( $cos θ \pm i sin θ$ ). WHY: a real eigenvector would be a direction the machine doesn't turn — but a genuine rotation turns every real direction, so no real eigenvector can exist. The math signals this by producing complex eigenvalues.
Check $θ = 9 0^{\circ}$ : $R = (01 - 1 0)$ , $det = 1$ , singular values $1, 1$ , eigenvalues $\pm i$ . This is why we reach for svd (real, well-behaved) over eig on such matrices.

Recall One-line self-quiz

Which tool for an over-determined fit? ::: np.linalg.lstsq (least squares via SVD), never solve. Singular values of a rotation matrix? ::: all equal to $1$ . How to detect near-singularity reliably? ::: np.linalg.cond(A) is large — not a small det.