5.4.6 · D5Scientific Computing (Python)
Question bank — NumPy linear algebra — np.linalg.solve, eig, svd, norm, det
Every item below rests on five vocabulary anchors from the parent note, restated in plain words so nothing here is undefined:
True or false — justify
np.linalg.solve(A, b) first computes A^{-1} internally, so it's no more accurate than inv(A) @ b.
False —
solve runs LU factorization + back-substitution and never forms the inverse; skipping the inverse both saves work and avoids the extra error the inverse introduces.A matrix with a very small determinant (say ) is definitely close to non-invertible, so solving with it is dangerous.
False — determinant scales with the entries, so a matrix like has tiny det yet is perfectly well-behaved; use the condition number (
np.linalg.cond) to judge trust, not det.If then has no solution at all.
False — it has no unique solution; depending on it can have infinitely many (if lies in the collapsed image) or none, but never exactly one.
Every real square matrix has real eigenvalues.
False — a rotation matrix has no real eigenvectors (nothing stays pointing the same way), so its eigenvalues are complex;
np.linalg.eig will return complex numbers.Singular values can be negative if the matrix flips orientation.
False — singular values are always by definition; any flip/reflection is absorbed into the orthogonal factors or , not into .
The default np.linalg.norm(v) gives the sum of absolute values of the entries.
False — the default
ord=2 is the Euclidean length ; the sum of absolute values is the norm (ord=1).For any matrix, np.linalg.eig and np.linalg.svd give essentially the same information.
False —
eig needs a square matrix and reports scaling along possibly-complex, non-orthogonal directions; SVD works for any shape and always gives real, ordered, non-negative stretches along orthogonal directions.If is an eigenvalue of then so is .
False — there is no such symmetry in general; eigenvalues are just the roots of and can be any set of numbers (e.g. has , not ).
The singular values of equal the absolute values of its eigenvalues.
False in general — they coincide only for symmetric matrices; for a general , are square roots of eigenvalues of , which need not match .
np.allclose(A @ x, b) returning True guarantees x is the exact mathematical solution.
False — it confirms the residual is tiny, which is the right practical check, but for an ill-conditioned a tiny residual can still hide a large error in itself.
Spot the error
x = np.linalg.inv(A) @ b to solve a linear system.
The error is philosophical, not syntactic: it needlessly forms the inverse, which is slower and less accurate; write
x = np.linalg.solve(A, b) and keep inv only when you truly need the inverse matrix as an object.w, V = np.linalg.eig(A); first_vec = V[0, :] to grab the first eigenvector.
Wrong axis — NumPy stores eigenvectors as columns, so the -th one is
V[:, i]; V[0, :] is the first row, a mix of coordinates from all eigenvectors.U, s, Vt = np.linalg.svd(A); recon = U @ s @ Vt to rebuild A.
s is a 1-D array of singular values, not a matrix — you must expand it into a diagonal of 's shape (np.zeros_like(A) + np.fill_diagonal) before multiplying.if np.linalg.det(A) == 0: print("singular") to detect a singular matrix.
Floating point almost never lands on an exact ; test conditioning with
np.linalg.cond(A) (large ⇒ effectively singular) instead of comparing a float to zero.np.linalg.eig(A) where A is a 3×2 data matrix.
eig requires a square matrix and will raise LinAlgError; for a rectangular matrix use np.linalg.svd(A), which is defined for every shape.np.linalg.norm(M) expecting the largest entry of matrix M.
For a 2-D array the default is the Frobenius norm (), not the max entry; pass
ord=np.inf (max abs row sum) or np.max(np.abs(M)) if you want an extreme.Reconstructing A after svd but slicing U to U[:, :n] and forgetting the matching Vt.
Mixing the "full" and "economy" shapes breaks the product; either keep full , or pass
full_matrices=False so all three factors use the reduced, compatible shapes.Computing np.linalg.solve(A, B) where A is 3×3 and B is 3×5 and assuming it errors.
It does not error —
solve accepts a right-hand side with multiple columns and returns a 3×5 array whose -th column solves ; this is a feature, not a bug.Why questions
Why do we prefer solving directly instead of the textbook formula ?
Forming costs extra operations and injects rounding error into every entry; direct elimination reaches with fewer, more stable steps because it never manufactures the full inverse.
Why must hold for an eigenvalue?
We need a nonzero with ; if were invertible the only solution would be , so it must be singular, which is exactly .
Why does SVD exist for every matrix while eigendecomposition does not?
SVD builds its directions from and , which are symmetric positive-semidefinite for any (any shape), guaranteeing real orthogonal factors and non-negative ; eigendecomposition needs square, diagonalizable structure that many matrices lack.
Why are there several norms instead of one "correct" length?
Each encodes a different notion of size — straight-line distance (), total step-by-step distance (), worst single coordinate () — and the right one depends on what "big" should mean in your problem.
Why is np.linalg.cond a better invertibility gauge than det?
The condition number measures how much a solve can amplify errors regardless of overall scale, whereas
det conflates scale with degeneracy, so cond directly answers "can I trust this solve?".Why does the parent note say , and what does it buy us?
The orthogonal cancels (), leaving only the stretches squared; this shows the columns of are eigenvectors of and its eigenvalues, linking SVD back to eigen-analysis.
Why does a diagonal matrix have the coordinate axes as its eigenvectors?
A diagonal matrix scales each coordinate independently, so a vector lying on an axis is only stretched, never tilted — that "no tilt" property is exactly the eigenvector condition .
Edge cases
What does np.linalg.svd return for the zero matrix?
All singular values are (the map crushes everything to a point); and are still valid orthogonal matrices, just arbitrary since no direction is stretched more than another.
What is np.linalg.det of a non-square matrix?
Undefined — determinant needs a square matrix; NumPy raises
LinAlgError. Volume-scaling only makes sense when input and output live in the same-dimensional space.For an already-triangular matrix, what does the LU step inside solve do?
Almost nothing on the factorization side — an upper-triangular matrix is its own with , so
solve goes straight to back-substitution, which is why triangular systems are cheap.A matrix has a repeated eigenvalue (e.g. ) — how many independent eigenvectors?
For every nonzero vector is an eigenvector, so there is a full 2-D eigenspace; but defective matrices like share yet have only one direction, which is why
eig can be fragile there.What norm does an empty or single-element vector return?
np.linalg.norm([]) is (empty sum), and np.linalg.norm([-5]) is under every — with one entry all the norms coincide since there is nothing to combine.If two singular values are equal, is the SVD unique?
No — within the tied subspace the orthogonal directions can be rotated freely, so and are not unique even though the set of (and thus ) is fixed.
What happens to solve when is exactly singular?
It raises
LinAlgError: Singular matrix; when is merely near-singular it silently returns an answer whose error may be huge — which is why the residual check and cond matter.Recall One-line self-test
Which single tool would you reach for on a non-square matrix, and why? ::: np.linalg.svd — it is the only one of the five defined for any shape, giving real non-negative stretches without needing squareness.