NumPy arrays and vectorized operations

What Are NumPy Arrays?

Creating Arrays: From Scratch Derivation

HOW do we go from Python lists to ndarrays?

import numpy as np
 
# Method 1: From Python list
a = np.array([1, 2, 3, 4])  # shape (4,), dtype inferred as int64
 
# Method 2: Built-in constructors
zeros = np.zeros((3, 4))      # 3×4 array of 0.0
ones = np.ones((2, 5))        # 2×5 array of 1.0
arange = np.arange(0, 10, 2)  # [0, 2, 4, 6, 8], like range()
linspace = np.linspace(0, 1, 5)  # [0, 0.25, 0.5, 0.75, 1.0], 5 evenly spaced
 
# Method 3: Random
rand = np.random.rand(3, 3)   # uniform [0,1), shape (3,3)
randn = np.random.randn(100)  # standard normal, shape (100,)

WHY separate constructors? Each optimizes for a use case:

zeros/ones: Initialize weights, masks
arange: Iteration indices
linspace: Sampling continuous domains (plotting, integrals)
random: Monte Carlo, initialization

Vectorized Operations: The Core Principle

Formula Element-wise Operations

For arrays $\mathbf{a}, \mathbf{b}$ of compatible shapes, operations apply element-wise:

\mathbf{c} = \mathbf{a} + \mathbf{b}implies c_i = a_i + b_i

\mathbf{c} = \mathbf{a} \cdot \mathbf{b} \implies c_i = a_i \times b_i \quad \text{(Hadamard product)}

\mathbf{c} = f(\mathbf{a}) \implies c_i = f(a_i) \quad \text{for any ufunc } f

(\mathbf{A}\mathbf{B})_{ij} = \sum_{k=1}^{n} A_{ik} B_{kj}

(\mathbf{AB}){ij} = \sum{k=1}^{n} A_{ik} B_{kj}

In NumPy, `A @ B` dispatches to optimized BLAS (Basic Linear Algebra Subprograms) libraries like OpenBLAS or MKL, which tile the computation for cache efficiency and parallelize across cores. **WHY not use `*`?** Element-wise `*` is Hadamard product, used in different contexts (e.g., attention masks). `@` is unambiguous matrix multiplication.

Example 4: Solving Linear Systems

# Solve Ax = b
A = np.array([[3, 1], [1, 2]])
b = np.array([9, 8])
 
x = np.linalg.solve(A, b)  # [2., 3.], solves via LU decomposition
# Verify
print(A @ x)  # [9., 8.], matches b

Why this step?

solve uses LU factorization ( $\mathbf{A} = \mathbf{LU}$ ), then forward/backward substitution— $O(n^3)$ but numerically stable.
Direct np.linalg.inv(A) @ b is slower and less accurate (conditioning issues).

Recall Explain to a 12-year-old

Imagine you have a huge pile of Lego bricks (a million of them), and you want to paint each one red. The slow way: pick up each brick, paint it, put it down. Takes forever because you're doing one thing at a time.

NumPy is like having a magic conveyor belt: you dump all the bricks on it, press a button, and the belt runs all of them through a painting machine simultaneously. You get all painted bricks in one go—same result, 100 times faster.

Broadcasting is even cooler: say you have a checkerboard with 100 rows, and you want to add 10 points to every square in each column. Instead of loping through all 100 rows, you just tell NumPy "add this [10, 20, 30, ...] to every row," and it does it in one magic swoop by stretching your small list to match the big board.

Performance Tips

Avoid loops: Replace for i in range(n): result[i] = func(data[i]) with result = func(data).
Use in-place ops: a +=1 modifies memory directly, a = a + 1 creates new array.
Preallocate: result = np.empty(shape) then fill, rather than np.append in loop (which copies).
Profile: %timeit in Jupyter to compare approaches.

Connections

1.4.01-Python-fundamentals - NumPy builds on Python data structures
1.4.02-List-comprehensions-and-generators - Vectorization replaces comprehensions
1.4.05-Pandas-DataFrames - DataFrames are built on NumPy arrays
2.1.03-Matrix-operations-for-ML - Linear algebra with NumPy powers neural nets
2.2.04-Batch-gradient-descent - Vectorized gradient computation
3.1.02-Tensor-operations-in-PyTorch - PyTorch tensors mirror NumPy API

#flashcards/ai-ml

What is the primary reason NumPy vectorized operations are 10-100× faster than Python loops? :: NumPy pushes computation to optimized C/Fortran code that exploits CPU SIMD paralelism and avoids Python interpreter overhead (type checks, reference counting per element).

What does np.arange(0, 10, 2) return?

array([0, 2, 4, 6, 8]), integers from 0 to <10 with step 2 (like range but returns ndarray).

In NumPy, what is the difference between A * B and A @ B for2D arrays?

A * B is element-wise (Hadamard) product; A @ B is matrix multiplication (dot product of rows and columns).

When does NumPy broadcasting fail?

When arrays have incompatible shapes—for each dimension (from right), sizes must be equal or one must be 1. E.g., (3,4) and (5,) fail (4 ≠ 5, neither is 1).

Why are NumPy slices like A[1:3,:] views rather than copies?

For memory efficiency—slices share the underlying data buffer. Copying multi-GB arrays for every operation would be prohibitively slow.

What is the output shape when broadcasting (4, 1, 3) and (5, 3)?

(4, 5, 3). Align from right: (4,1,3) and (1,5,3) (prepend 1), then stretch dimensions of size 1.

Which NumPy function should you use to solve Ax = b and why not use inv(A) @ b?

np.linalg.solve(A, b) using LU decomposition—faster and numerically more stable than computing the inverse.

What does data[data > 0] do in NumPy?

Boolean indexing: returns a new array containing only elements of data that are greater than 0 (fancy indexing, creates a copy).

Why use np.linspace(0, 100) instead of np.arange(0, 1, 0.01)? :: linspace guarantees exactly 100 points including endpoints, avoiding floating-point errors. arange with float step can have rounding issues and might exclude the endpoint.

What is a ufunc and give two examples?

Universal function: operates element-wise on arrays with broadcasting support, implemented in compiled C. Examples: np.sin, np.exp, np.maximum.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

NumPy arrays ka sabse bada fayda hai vectorization—matlab ek baar mein pore array par operation ho jata hai, element-by-element loop ki zarurat nahi. Jaise ek list mein 1 million numbers hain, unhe square karna hai. Python loop mein ek-ek karke karenein toh 1 second lagega, lekin NumPy mein arr**2 likhte hi 10 milliseconds mein ho jaayega—100 guna tej! Kyun? Kyunki NumPy internally optimized C code use karta hai jo CPU ke parallel processing ko exploit karta hai, aur Python ke object overhead ko skip kar deta hai.

Broadcasting ek aur superpower hai—agar different size ke arrays ko add karna ho, NumPy automatically chhote array ko bada kar deta hai (logically, memory copy nahi karta). Example: ek 2×3 matrix mein har column mein same3 numbers add karne hain—ap sirf (3,) array dete ho, NumPy usse 2 rows mein repeat kar deta hai. Ye ML mein bahut kaam ata hai jaise dataset ko normalize karna (har feature ka mean subtract karna across all samples). Yeh chez manually loops se karne par code bhi slow aur mesy ho jata.

Ek common mistake: log sochte hain kisi bhi do arrays ko add kar sakte hain, lekin broadcasting ke rules hain—shapes compatible hone chahiye (har dimension mein size equal yaek size 1 honi chahiye). Agar rule break ho toh ValueError milega. NumPy views ka concept bhi samajhna zaroori hai—slice karne par nayi copy nahi banti, original data ka reference milta hai.Iska matlab slice modify karo toh original array bhi change hoga. Ye memory bachata hai lekin side-effects se savdhaan rehna padta hai. Yeh sab samajh ke hi ML code efficient aur bug-free likha ja sakta hai!

What Are NumPy Arrays?

Creating Arrays: From Scratch Derivation

Vectorized Operations: The Core Principle

Example 4: Solving Linear Systems

Performance Tips

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Python & Scientific Computing

Connections