NumPy linear algebra — np.linalg.solve, eig, svd, norm, det

1. Solving $Ax = b$ — `np.linalg.solve`

WHY not just invert? Mathematically $x = A^{-1}b$ . But computing $A^{-1}$ is slower and less accurate than directly solving. So we never write inv(A) @ b; we write solve(A, b).

import numpy as np
A = np.array([[2., 1.],
              [1., 3.]])
b = np.array([5., 10.])
x = np.linalg.solve(A, b)   # array([1., 3.])
np.allclose(A @ x, b)        # True  <- always verify!

2. Determinant — `np.linalg.det`

np.linalg.det(np.array([[2.,1.],[1.,3.]]))   # 5.0

3. Norms — `np.linalg.norm`

v = np.array([3., 4.])
np.linalg.norm(v)          # 5.0  (=sqrt(9+16))  default ord=2
np.linalg.norm(v, ord=1)   # 7.0
np.linalg.norm(v, ord=np.inf)  # 4.0
 
M = np.array([[1.,2.],[3.,4.]])
np.linalg.norm(M, 'fro')   # Frobenius = sqrt(sum of squares of all entries)

4. Eigenvalues / Eigenvectors — `np.linalg.eig`

A = np.array([[2., 0.],
              [0., 3.]])
w, V = np.linalg.eig(A)   # w = eigenvalues, columns of V = eigenvectors
# check column 0:
np.allclose(A @ V[:,0], w[0] * V[:,0])   # True

5. Singular Value Decomposition — `np.linalg.svd`

Figure — NumPy linear algebra — np.linalg.solve, eig, svd, norm, det

A = np.array([[3., 0.],
              [0., 2.],
              [0., 0.]])      # 3x2, NOT square -> eig would fail
U, s, Vt = np.linalg.svd(A)   # s is a 1D array of singular values
# reconstruct:
Sig = np.zeros_like(A)
np.fill_diagonal(Sig, s)
np.allclose(U @ Sig @ Vt, A)  # True

Recall Feynman: explain to a 12-year-old

A matrix is a stretchy-twisty machine for arrows.

solve: "I see where the arrow landed; where did it start?" (run the machine backward).
det: "How much bigger did the machine make a square?" If zero, it squished everything flat — you can't un-squish, so no backward.
norm: "How long is this arrow?"
eig: "Which arrows does the machine only make longer/shorter, never turn?"
svd: "Even for weird machines: first spin the arrow, then stretch it, then spin again." The stretch numbers are the singular values.

Flashcards

Why prefer np.linalg.solve(A,b) over np.linalg.inv(A)@b?

solve is faster (

O(n^3)

but with smaller constant) and numerically more stable; explicit inversion amplifies error and is wasteful.

What does

\det(A)=0

mean geometrically and algebraically?

The transform collapses volume to zero (maps to a lower dimension); algebraically

A

is singular / non-invertible, so

Ax=b

has no unique solution.

For a 2×2 matrix, write the determinant formula.

\det\begin{pmatrix}a&b\\c&d\end{pmatrix}=ad-bc

Define an eigenvector/eigenvalue.

v\neq0

with

Av=\lambda v

: a direction the matrix only scales (by

\lambda

), never rotates.

Characteristic equation for eigenvalues?

\det(A-\lambda I)=0

In NumPy w, V = np.linalg.eig(A), how do you get the i-th eigenvector?

V[:, i] — eigenvectors are the COLUMNS of V.

State the SVD of a general matrix.

A=U\Sigma V^\top

with

U,V

orthogonal and

\Sigma

diagonal of non-negative singular values

\sigma_1\ge\sigma_2\ge\dots

Relation between singular values and eigenvalues?

\sigma_i^2

are the eigenvalues of

A^\top A

(and of

AA^\top

Why use SVD instead of eig sometimes?

SVD works for ANY (even non-square) matrix and always gives real non-negative singular values; eig needs square matrices and may give complex results.

Default np.linalg.norm(v) computes which norm?

The Euclidean (

\ell_2

) norm

\sqrt{\sum v_i^2}

How do you check a computed solution x of Ax=b?

np.allclose(A @ x, b).

Which function judges trustworthiness of a solve better than det?

np.linalg.cond(A), the condition number.

Connections

NumPy arrays and broadcasting
Gaussian elimination and LU decomposition
Eigenvalues and diagonalization
Principal Component Analysis (PCA) — built on SVD
Least squares regression — solved via SVD / np.linalg.lstsq
Condition number and numerical stability
Vector norms and metric spaces

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, ek matrix ko ek "machine" samajho jo arrows (vectors) ko stretch aur twist karti hai. NumPy ka linalg module is machine ke saare kaam karta hai, aur andar se ye LAPACK (super-fast Fortran code) use karta hai, isliye tum sirf ek line likhte ho aur kaam ho jaata hai.

np.linalg.solve(A, b) ka matlab hai: $Ax=b$ ko ulta chalाना — landing point pata hai, starting arrow dhoondo. Yaha pe ek bada lesson: kabhi inv(A) @ b mat likho, hamesha solve use karo, kyunki inverse nikalna slow bhi hai aur error bhi badha deta hai. det batata hai machine ne area/volume kitna scale kiya — agar det=0 to sab kuch ek line me squish ho gaya, matlab matrix invertible nahi. norm arrow ki lambai (size) deta hai, default Euclidean length.

eig square matrix ke liye un special directions ko dhoondta hai jinko machine sirf stretch karti hai, rotate nahi — wahi hain eigenvectors, aur stretch amount eigenvalue $\lambda$ . Yaad rakho: NumPy me eigenvectors columns me aate hain, V[:, i], rows me nahi! svd sabse powerful hai — ye kisi bhi matrix (square ho ya nahi) ko todta hai: pehle ghumao ( $V^\top$ ), phir stretch karo ( $\Sigma$ ), phir ghumao ( $U$ ). Ye singular values $\sigma$ PCA, compression aur least-squares me kaam aate hain.

Practical tip jo exam aur real code dono me help karega: jo bhi solve/decompose karo, hamesha np.allclose(A @ x, b) ya reconstruction se check karo. Isse confidence aata hai ki numerics sahi hai. 80/20 rule: solve, svd, aur allclose — bas ye teen cheezein 80% kaam karwa dengi.

NumPy linear algebra — np.linalg.solve, eig, svd, norm, det

1. Solving $Ax = b$ — `np.linalg.solve`

2. Determinant — `np.linalg.det`

3. Norms — `np.linalg.norm`

4. Eigenvalues / Eigenvectors — `np.linalg.eig`

5. Singular Value Decomposition — `np.linalg.svd`

Flashcards

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Scientific Computing (Python)

Connections