Logistic regression and the sigmoid function: Worked examples

Before we start, one reminder of the three symbols we lean on, so nothing is used unearned:

Recall The three symbols, in plain words

$z$ ::: the linear score, $w^{⊤} x + b$ . It is just a weighted sum of the inputs plus a constant. It can be any real number, negative or positive. $σ (z)$ ::: the sigmoid, $\frac{1}{1 + e ^{- z}}$ . A squishy slide that turns any $z$ into a number strictly between 0 and 1 — read as $P (y = 1 ∣ x)$ . $p - y$ ::: the error signal: your predicted probability minus the true label. Positive means you guessed too high, negative means too low. This is the whole engine of learning.

The scenario matrix

Every case logistic regression can throw at you falls into one of these cells. Each worked example below is tagged with the cell(s) it covers.

Cell	What makes it distinct	Covered by
A. Positive score	$z > 0$ , so $p > 0.5$ , predict class 1	Ex 1
B. Negative score	$z < 0$ , so $p < 0.5$ , predict class 0	Ex 2
C. The exact centre	$z = 0$ , so $p = 0.5$ — a tie, the boundary itself	Ex 3
D. Far tails / saturation	$∣ z ∣$ large, $p \to 0$ or $1$ , gradient nearly dies	Ex 4
E. Degenerate zero weights	$w = 0$ : model ignores input, always outputs $0.5$	Ex 5
F. Gradient step, over-confident wrong	learning nudges weights, sign of $(p - y)$	Ex 6
G. Real-world word problem	translate words → features → probability → decision	Ex 7
H. Log-odds / odds reading	interpret $z$ as additive evidence	Ex 8
I. Exam twist — 2-D boundary	draw the line $w^{⊤} x + b = 0$ in feature space	Ex 9
J. Symmetry check	$σ (- z) = 1 - σ (z)$ used to save work	Ex 10

Let us anchor the whole page in one picture of the slide, marking where each cell lives on it.

Figure — Logistic regression and the sigmoid function

Read the figure: the centre (cell C) sits at $z = 0, p = 0.5$ ; walk right into positive $z$ (cell A, blue region, $p > 0.5$ ); walk left into negative $z$ (cell B, red region); go far either way (cell D) and the curve flattens — the slope dies. That flattening is why the tails matter.

The worked examples

Worked example Example 4 — Cell D: far tails and the dying gradient

Same model. Predict for $x = 100$ , and inspect the slope of the slide there.

Forecast: Will $p$ be near 1? And will the model still want to learn there?

Score. $z = 2 (100) - 4 = 196$ . Why this step? A huge input pushes $z$ far into the positive tail.
Squash. $σ (196) = \frac{1}{1 + e ^{- 196}} \approx 1$ (differs from 1 by about $e^{- 196} \approx 1 0^{- 85}$ ). Why this step? $e^{- 196}$ is astronomically tiny, so $p$ is essentially 1 — total confidence.
Slope there. Using $σ^{'} (z) = σ (z) (1 - σ (z)) \approx 1 \cdot 1 0^{- 85}$ . Why this step? The gradient of the loss carries a factor tied to how far $p$ can still move. In the tail the slide is flat, so $σ^{'} \to 0$ : the machine barely responds. This is why a very wrong, very confident prediction under squared error stalls — cross-entropy fixes it because its error term is $(p - y)$ , which stays $O (1)$ .

Verify: $σ^{'} (z)$ is maximal at $z = 0$ (value $0.25$ ) and decays symmetrically; at $∣ z ∣ = 196$ it must be near 0. ✓

Worked example Example 6 — Cell F: one gradient step (over-confident, wrong)

Two features, $w = (0, 0), b = 0, α = 0.5$ . Data point $x = (1, 2), y = 1$ .

Forecast: We under-predict a positive. Which way do the weights move — up or down?

Predict. $z = 0 \Rightarrow p = σ (0) = 0.5$ . Why this step? Establish the current guess before scoring the mistake.
Error signal. $p - y = 0.5 - 1 = - 0.5$ . Why this step? Negative error means we guessed too low for a true positive — push $z$ up.
Per-weight gradient. $\partial J / \partial w_{1} = (p - y) x_{1} = (- 0.5) (1) = - 0.5$ ; $\partial J / \partial w_{2} = (- 0.5) (2) = - 1$ . Why this step? Gradient is (pred−target)×input; a larger input feels a larger nudge.
Update $w \leftarrow w - α grad$ : $w_{1} \leftarrow 0 - 0.5 (- 0.5) = 0.25$ ; $w_{2} \leftarrow 0 - 0.5 (- 1) = 0.5$ . Why this step? Subtracting a negative gradient raises the weight — next time $z$ will be higher for this point.

Verify: Recompute the new score for the same point: $z = 0.25 (1) + 0.5 (2) = 1.25$ , so $p = σ (1.25) \approx 0.777 > 0.5$ — we now correctly lean class 1. The step helped. ✓

Worked example Example 7 — Cell G: real-world word problem

A clinic uses $z = 0.08 (age) + 1.5 (tumor size in cm) - 6$ to model $P (malignant)$ . A patient is age 50 with a 2 cm tumor. Malignant?

Forecast: Guess whether this crosses 50% before computing.

Assemble the score. $z = 0.08 (50) + 1.5 (2) - 6 = 4 + 3 - 6 = 1$ . Why this step? Each feature contributes its weight × value; the bias $- 6$ is the baseline evidence with no features.
Squash. $σ (1) = \frac{1}{1 + e ^{- 1}} = \frac{1}{1 + 0.3679} \approx 0.731$ . Why this step? Convert the evidence score into an actual probability the clinician can act on.
Decide + report. $0.731 \geq 0.5 \Rightarrow$ flag malignant, 73% confidence — enough to warrant follow-up. Why this step? Units check: age in years × (per-year weight), size in cm × (per-cm weight) — both dimensionless log-odds contributions, summing to a pure number, as $σ$ requires.

Verify: $z = 1 > 0$ so $p > 0.5$ , consistent. If we drop tumor size to 0 cm, $z = 0.08 (50) - 6 = - 2$ , $p = σ (- 2) \approx 0.119$ — a small tumor sharply lowers the risk, as intuition demands. ✓

Worked example Example 8 — Cell H: reading log-odds and odds

A model outputs $z = w^{⊤} x + b = 1.1$ for an example. State the odds and the probability, and interpret.

Forecast: Odds and probability are different numbers — which is bigger here?

Odds. $\frac{p}{1 - p} = e^{z} = e^{1.1} \approx 3.004$ . Why this step? The parent derivation set $ln \frac{p}{1 - p} = z$ , so exponentiating recovers the odds directly.
Probability. $p = \frac{e ^{z}}{1 + e ^{z}} = \frac{3.004}{4.004} \approx 0.750$ . Why this step? Convert odds "3 to 1" into a fraction of the whole: $3/ (3 + 1)$ .
Interpret. Class 1 is about 3× more likely than class 0. Add one unit to a feature with weight $0.5$ and the log-odds rise by $0.5$ , multiplying the odds by $e^{0.5} \approx 1.65$ — evidence stacks additively on the log-odds scale.

Verify: Round-trip: $σ (1.1) = 1/ (1 + e^{- 1.1}) = 1/ (1 + 0.3329) \approx 0.750$ matches step 2. ✓

Worked example Example 9 — Cell I (exam twist): draw the 2-D decision boundary

Model $z = x_{1} + 2 x_{2} - 4$ . Where is the boundary, and which side is class 1?

Forecast: Is the boundary a curve or a straight line? Sketch it in your head first.

Set the boundary condition. Boundary is $p = 0.5 ⟺ z = 0 ⟺ x_{1} + 2 x_{2} - 4 = 0$ . Why this step? $σ$ crosses $0.5$ exactly at $z = 0$ ; the boundary lives entirely in the $z = 0$ locus.
Rewrite as a line. $x_{2} = \frac{4 - x _{1}}{2} = 2 - \frac{1}{2} x_{1}$ — a straight line, slope $- \frac{1}{2}$ , intercept $2$ . Why this step? Solving for $x_{2}$ shows it is linear in $x_{1}$ ; the sigmoid never bends feature space.
Pick the class-1 side. Test the origin $(0, 0)$ : $z = - 4 < 0 \Rightarrow p < 0.5$ , so the origin is class 0. The class-1 half-plane is the other side (larger $x_{1} + 2 x_{2}$ ). Why this step? One test point orients the whole half-plane.

Verify: Point $(4, 2)$ : $z = 4 + 4 - 4 = 4 > 0$ , class 1 — and indeed it lies above the line. Point $(0, 0)$ : class 0. See the figure. ✓

Connections

Logistic regression and the sigmoid function (index 2.2.9) — the parent this page drills into.
Cross-Entropy Loss — why the tails (Cell D) don't stall learning.
Gradient Descent — the engine behind Example 6's weight update.
Decision Boundaries — Example 9's straight-line boundary.
Linear Regression — the same $(p - y) x$ gradient shape.
Maximum Likelihood Estimation — where the loss came from.
Softmax Regression — the multi-class generalisation.
Neural Networks — sigmoid neurons stack into these.