Exercises — Functions, classes, and modules

Here b=10 is the default argument we defined above: if the caller does not supply b, Python fills in 10.

• add(5) → b defaults to 10 → 5 + 10 = 15.
• add(5, 2) → b is given as 2 → 5 + 2 = 7.

Output:

15
7

• Attribute: self.count — data the object remembers between calls.
• Method: tick — an action; it changes the remembered data and returns it.
• __init__ is the constructor we defined in the vocabulary above: it runs once when the object is created and sets the starting data (count = 0).

(A) and (B) are both correct — they are two valid import styles.

• (A) imports the whole module; you reach inside with metrics.accuracy.
• (B) imports just the one name; you call accuracy directly.
• (C) is invalid syntax — Python never writes import X from Y. The correct form is from Y import X.

Mean squared error = average of the squared gaps between truth and prediction. Using the two notation pieces from the vocabulary (the fraction bar = "divide", the summation sign = "add up as $i$ walks $1$ to $n$"), with $n$ points: $\text{MSE}=\frac{1}{n}{\sum }_{i=1}^n{\left(y_i^{\text{true}}-y_i^{\text{pred}}\right)}^2.$ In words: for each point, take (truth − prediction), square it, add them all, then divide by how many there are.

def compute_mse(y_true, y_pred):
    n = len(y_true)
    return sum((y_true[i] - y_pred[i])**2 for i in range(n)) / n

Plug in the numbers:

• gaps: $3-2=1$, $5-5=0$, $2-4=-2$.
• squares: $1,0,4$. Sum $=5$. Divide by $n=3$. $\text{MSE}=\frac{5}{3}\approx \mathrm{1.6667.}$

def compute_error(y_true, y_pred, squared=True):
    n = len(y_true)
    mse = sum((y_true[i] - y_pred[i])**2 for i in range(n)) / n
    return mse if squared else mse ** 0.5

RMSE just takes the square root: $\sqrt{5/3}=\sqrt{1.6667}\approx 1.2910$.

Euclidean distance is the straight-line length between two points. For vectors with parts indexed by $i$ (again, $\sum$ = "add up", the outer square root = "undo the squaring to get a length"), $d=\sqrt{{\sum }_i(x{1}_i-x{2}_i{)}^2}.$ Why the square root? The squared differences give an area-like quantity; the root brings us back to a length in the same units as the coordinates.

def euclidean(x1, x2):
    return sum((a - b)**2 for a, b in zip(x1, x2)) ** 0.5

Differences: $4-1=3$, $6-2=4$. Squares $9+16=25$. Root $=5.0$.

Read the figure below. Two points sit at the ends of a slanted line: the lower-left one is (1,2) and the upper-right one is (4,6). From (1,2) a horizontal leg (labelled dx = 3 in the figure) runs right, and a vertical leg (labelled dy = 4) runs up to reach (4,6); together they form a right angle. The slanted line closing the triangle is the distance itself, labelled d = 5. The formula is just the Pythagorean theorem on that right triangle: the slanted length is $\sqrt{3^2+4^2}=5$. Every leg, point, and angle carries a text label in the figure, so the colours are decoration, not the message — this is why Euclidean distance uses squares-then-root: it measures the hypotenuse of a right triangle whose legs are the coordinate differences.

Because recomputing on the test set = data leakage. Normalization asks "where does this value sit between the smallest and largest we have seen?" If the test set defines its own min/max, then information from the test set (its extremes) sneaks into how test samples are scaled — the model effectively "peeks" at data it is meant to be judged on. The honest pipeline: learn the scaling from training data, then apply that fixed scaling to test data. That is exactly the fit / transform split you see in scikit-learn API patterns.

For a constant feature (every value in that column identical), $\text{max}-\text{min}=0$. The scaling divides by that range, so you would divide by zero → nan/inf for that whole column. Setting the zero ranges to 1 makes the division safe. The numerator $X-\text{min}$ is also $0$ for a constant column, so $0/1=0$: every value in that column becomes 0, which is the sensible "this feature carries no information" answer.

(B) the class is more robust.

• A dictionary has no guarantee the right keys exist — a typo like layer['weight'] fails only at runtime, deep in a training loop.
• The class's constructor (__init__) guarantees every DenseLayer has weights and biases, and forward is bound to its own data — layer1.forward(x) automatically uses layer1's weights, no manual passing.
• Behavior travels with the data instead of floating in loose functions.

Contracts.

• Input: X is 2-D, shape (n_samples, n_features); each query row must have the same feature count as the training rows.
• Return: predict returns a NumPy array of labels, one per query row (never a bare Python list) — so downstream code can index and vectorize it consistently, matching scikit-learn API patterns.
• Edge cases: if k exceeds the number of training samples we clamp it down to that count (asking for more neighbours than exist is meaningless); ties in the vote are broken deterministically by picking the label that is smallest when sorted, so the same input always gives the same answer.

import numpy as np
from collections import Counter
 
class KNNClassifier:
    def __init__(self, k=3):
        self.k = k
    def fit(self, X, y):
        X = np.asarray(X)
        if X.ndim != 2:
            raise ValueError("X must be 2-D (n_samples, n_features)")
        self.X_train, self.y_train = X, np.asarray(y)
        return self                              # method chaining
    def _euclidean_distance(self, x1, x2):
        return np.sqrt(np.sum((x1 - x2)**2))
    def _vote(self, labels):
        counts = Counter(labels)
        top = max(counts.values())               # highest vote count
        tied = sorted(l for l, c in counts.items() if c == top)
        return tied[0]                            # deterministic tie-break
    def predict(self, X):
        X = np.asarray(X)
        if X.ndim != 2:
            raise ValueError("X must be 2-D (n_samples, n_features)")
        if X.shape[1] != self.X_train.shape[1]:
            raise ValueError("query features must match training features")
        k = min(self.k, self.X_train.shape[0])   # clamp k <= n_samples
        out = []
        for x in X:
            dists = [self._euclidean_distance(x, xt) for xt in self.X_train]
            nearest = np.argsort(dists)[:k]       # k closest indices
            out.append(self._vote([self.y_train[i] for i in nearest]))
        return np.array(out)                      # return-type contract

What happens for the query (2.5, 2.5)? Its three closest training points are (2,2), (3,3), (1,1) — all class A (the two diagonal points nearest (2.5,2.5) are (2,2) and (3,3), both a tiny diagonal step away; the next closest is (1,1); every B point sits far off along the diagonal). With all three nearest labelled A, the majority vote is unambiguous: A. No tie arises here, but had it, _vote would still return a single deterministic label.

Read the figure below. It plots the six training points and the query on the same axes. The three class-A points cluster near the lower-left; the three class-B points cluster near the upper-right; the query (drawn as a star, with the text label "query (2.5, 2.5)") sits right beside the class-A cluster. A dashed circle is drawn around the query enclosing exactly its 3 nearest neighbours — all of them class A. Each cluster and the query are labelled in text, so the picture reads even without colour: KNN works because points close together tend to share a label.

Returning the object enables method chaining: fit hands back the same instance, so you can immediately call the next method on it.

preds = KNNClassifier(k=3).fit(X_train, y_train).predict(X_test)

This is exactly the convention in scikit-learn API patterns — every estimator's fit returns self.

Why standardize? Subtracting $\mu$ centers the data at $0$; dividing by $\sigma$ (the typical spread) makes "one unit" mean "one standard deviation," so features on wildly different scales become comparable.

Shape contract: X is always 2-D, shape (n_samples, n_features), with n_samples >= 1. Keeping X 2-D means X.mean(axis=0) returns a 1-D array of per-column means (one entry per feature), so self.sigma is always an array we can index with self.sigma == 0 — never a bare scalar. That removes the single-feature edge case entirely.

import numpy as np
class StandardScaler:
    def fit(self, X):
        X = np.asarray(X, dtype=float)
        if X.ndim != 2:
            raise ValueError("X must be 2-D (n_samples, n_features)")
        if X.shape[0] < 1:
            raise ValueError("X must contain at least one row")
        self.mu    = X.mean(axis=0)          # 1-D: one mean per column
        self.sigma = X.std(axis=0)           # 1-D: one std  per column
        self.sigma[self.sigma == 0] = 1      # guard constant columns
        return self
    def transform(self, X):
        if not hasattr(self, "mu"):
            raise RuntimeError("call fit() before transform()")
        X = np.asarray(X, dtype=float)
        if X.ndim != 2 or X.shape[1] != self.mu.shape[0]:
            raise ValueError("X shape must match the fitted feature count")
        return (X - self.mu) / self.sigma
    def fit_transform(self, X):
        return self.fit(X).transform(X)      # reuse, no duplicate logic

Two deliberate edge-case guards: X.shape[0] < 1 rejects empty input before we try to average nothing, and the hasattr(self, "mu") check turns a call-before-fit into a readable RuntimeError instead of an obscure AttributeError. (A single row is allowed: its std is 0 for every column, and the $\sigma =0$ guard turns those into 1, so every value maps to 0 — the sensible "no spread" answer.)

Hand-compute for the column [[2],[4],[6],[8]].

• Mean $\mu =(2+4+6+8)/4=5$.
• Variance $=\frac{(2-5{)}^2+(4-5{)}^2+(6-5{)}^2+(8-5{)}^2}{4}=\frac{9+1+1+9}{4}=5$, so $\sigma =\sqrt{5}\approx 2.2361$.
• For $x=4$: $z=(4-5)/\sqrt{5}=-1/\sqrt{5}\approx -0.4472$.

# preprocessing.py
class StandardScaler: ...
 
# train.py
from preprocessing import StandardScaler
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)   # learns mu, sigma from TRAIN
X_test_s  = scaler.transform(X_test)        # reuses those mu, sigma

The guaranteeing property: fit stores $\mu ,\sigma$ as attributes on the object, and transform reads those stored attributes rather than recomputing. Calling only transform (never fit) on the test set means the training statistics are reused — no leakage. This is the same discipline as Data pipelines and preprocessing and underlies clean Software engineering for ML.

A function that produces a value must end with ::: return The special method that runs when an object is created (the constructor) ::: __init__ Scaling a test set with its own min/max causes ::: data leakage fit returns self to enable ::: method chaining To remember data across method calls, store it on ::: self Calling transform before fit should raise ::: a clear error (e.g. RuntimeError) KNN with an even k may hit a ::: tie, which must be broken deterministically