Foundations — Logistic regression and the sigmoid function

Before you can read the parent note comfortably, you must own every symbol it throws at you. Below, each symbol is introduced only after the ones it depends on. We start from a bare number line.

1. Numbers, intervals, and the symbol $\infty$

Figure s01 — two homes. The picture below sets up the entire story: the top cyan line is the wide home $(-\infty ,\infty )$ where a raw score lives; the amber box is the narrow home $[0,1]$ where a probability must live. Watch the white arrow — the whole job is to move a number from the line into the box.

2. A feature vector $x$ and its components $x_j$; the count $n$

Figure s02 — one example is one arrow. Below, a two-feature example ($n=2$) is drawn as a cyan arrow reaching the amber point $(x_1,x_2)$. The dashed white lines show each component $x_1,x_2$ as a coordinate along its own axis.

The letter $m$ counts the examples (rows of data); the letters $j$ and $n$ concern features ($n$ = how many features total, $j$ = which one). Keep them separate — $m$ is how many things, $n$ is how many properties each thing has.

3. Weights $w$, bias $b$, and the linear score $z$

4. The exponential $e^z$ and the natural log $\ln$

Figure s03 — exp and log are mirror images. The cyan curve is $e^z$ (always above the axis); the amber curve is $\ln u$ (only defined for positive $u$). They are reflections across the dashed white line $y=x$ — that reflection is what "inverse" means. Note the marked points $e^0=1$ and $\ln 1=0$.

5. Probability $p$, odds, and log-odds (the logit)

The symbol $y$ is the true label: $y=1$ means "actually yes", $y=0$ means "actually no". Bar $\mid$ inside $P(y=1\mid x)$ means "given / conditional on".

6. The sigmoid $\sigma$ and the model equation

Figure s04 — the S-shaped slide. Below is the sigmoid itself: a smooth S rising from near $0$ on the far left to near $1$ on the far right, passing through the amber dot at $(0,0.5)$. The steepest part is in the middle (near $z=0$); the two tails flatten out. This shape is the "slide" every score rides down.

7. Cost $J$, its gradient, and the training loop

Prerequisite outline (what feeds what)

Read this top-down; each line depends only on the ones above it.

1. Number line & intervals (§1) — gives us the two "homes" $(-\infty ,\infty )$ and $[0,1]$.
2. Feature vector $x$, count $n$ (§2) — one example as an arrow.
3. Exponential $e^z$ & log $\ln$ (§4) — the squash/undo pair.
4. Weighted sum $z=w^{\top }x+b$ (§3) — needs the feature vector; builds the raw score.
5. Probability $p$, odds, log-odds (§5) — needs $\ln$; the ladder from box to number line.
6. Sigmoid $\sigma (z)$ (§6) — needs $e$ and the log-odds ladder; inverts it, giving the model equation $P(y=1\mid x)=\sigma (z)$.
7. Slope ${\sigma }^{\prime }(z)=\sigma (1-\sigma )$ (§6) — needs the sigmoid.
8. Cost $J$ & gradients $\partial J/\partial w_j,\partial J/\partial b$ (§7) — needs the slope; drives Gradient Descent.
9. Trained logistic regression — the loop of §7 repeated until $J$ is small.

Equipment checklist

Read each as a question; the part after ::: is the answer to check yourself.