Matrix Multiplication Linear Maps

The unifying principle

A linear map $T: \mathbb{R}^n \to \mathbb{R}^m$ is fully determined by what it does to the basis vectors $e_1, \dots, e_n$ . Because $T$ respects addition and scaling,

$T(x) = T\!\left(\sum_j x_j e_j\right) = \sum_j x_j\, T(e_j).$

Collect the images $T(e_j)$ as columns of a matrix $A$ , and the whole map becomes one product:

(Ax)_i = \sum_{j=1}^{n} A_{ij}\, x_j.

The single fact "composition = multiplication, associative but not commutative" is what unifies everything below.

How it shows up in each field

Maths — linear algebra

The map is the object. A change of basis $P$ conjugates a transform: $A' = P^{-1}AP$ . Eigenvectors $Av = \lambda v$ are directions the map only stretches, never rotates.

A dense layer is $h = \sigma(Wx + b)$ . The linear part $Wx$ is exactly a transformation of the feature vector; the nonlinearity $\sigma$ bends space between transforms so stacked linear maps don't collapse into one ( $W_2W_1$ ).

Hardware — quantum gates & signal processing

A quantum gate is a unitary matrix $U$ ( $U^\dagger U = I$ ) acting on a state vector; it's a rotation in complex Hilbert space, preserving $\|\psi\|=1$ .

Coding/CS — graphics & GPU computation

A GPU is a matrix-multiply engine. 3D graphics chains transforms via homogeneous $4\times4$ matrices: $\mathbf{p}' = P \cdot V \cdot M \cdot \mathbf{p}$ (model → view → projection). GPUs realize $C = AB$ as tiled dot-products so that thousands of output entries compute in parallel.

Worked example

// C[i][j] = sum_k A[i][k] * B[k][j]  — one thread per (i,j)
__global__ void matmul(float* A, float* B, float* C, int n){
  int i = blockIdx.y*blockDim.y + threadIdx.y;
  int j = blockIdx.x*blockDim.x + threadIdx.x;
  float s = 0;
  for (int k = 0; k < n; ++k) s += A[i*n+k]*B[k*n+j];
  C[i*n+j] = s;
}

Tensor Cores fuse a $4\times4$ tile multiply-accumulate into one instruction — the hardware primitive is the linear map.

Why this bridge matters

Composition intuition transfers everywhere. "Stacking transforms = multiplying matrices, and order matters" explains why quantum circuits, graphics pipelines, and deep networks all read right-to-left and are non-commutative.
Eigen-thinking unlocks ML and dynamics. Understanding eigenvalues as stretch factors explains vanishing/exploding gradients (spectral radius of $W$ ), PCA, and stability of iterated maps.
Unitarity is a design principle. Quantum's norm-preservation motivates orthogonal/unitary weight initialization in RNNs to avoid gradient decay.
The hardware lesson feeds back to math. GPUs make matmul the cheap operation, so ML architectures are literally shaped by "express it as $C=AB$ ." Knowing the compute cost is $O(n^3)$ (or $O(mnk)$ ) tells you why attention is expensive and why low-rank factorizations $A \approx UV^\top$ pay off.

Connections

01 Linear Maps & Change of Basis
02 Eigenvalues & Spectral Decomposition
07 Neural Network Layers & Attention
11 Quantum Gates & Unitary Evolution
14 3D Graphics Transform Pipeline
18 GPU Kernels & Tensor Cores

#bridge