Decision boundaries: Question bank — AI-ML deep dive

Before the traps, three anchors so no symbol is used unexplained:

$z = w^{⊤} x + b$ is the score — a plain weighted sum of the features plus an offset. Positive means "leaning class 1", negative means "leaning class 0".
$σ (z) = \frac{1}{1 + e ^{- z}}$ is the sigmoid — it squashes any score into a probability between 0 and 1. It is monotonic: bigger score always means bigger probability, never a wiggle back.
The decision boundary is the knife-edge set of points where the model ties at 50/50, which (for logistic regression) is exactly $z = 0$ .

True or false — justify

The boundary of a logistic regression model is a curved sigmoid shape.

False. The sigmoid bends the probability surface, but the boundary is where probability

= 0.5

, i.e.

z = 0

, and

z

is a straight-line function of

x

— a flat hyperplane.

Doubling both

w

and

b

moves the decision boundary.

False. The set

{w^{⊤} x + b = 0}

and

{2 w^{⊤} x + 2 b = 0}

are identical points; scaling only steepens the sigmoid (confidence), it never relocates the border.

Changing the classification threshold from 0.5 to 0.8 tilts the boundary to a new orientation.

False. A new threshold

t

gives

z = ln \frac{t}{1 - t}

, a constant; the hyperplane slides parallel to itself (new

b

) but keeps the same normal

w

, so orientation is unchanged.

A model with

z = x_{1}^{2} + x_{2}^{2} - 4

is a nonlinear model.

False (subtly). It is linear in its weights and features

(1, x_{1}^{2}, x_{2}^{2})

— the same linear machinery — it only looks curved when projected back onto the raw

(x_{1}, x_{2})

plane.

Every point exactly on the boundary is predicted as class 1.

False. On the boundary the model is genuinely tied (

P = 0.5

), so the label is undefined by the model itself; a convention (e.g. "

\geq 0.5 \to 1

") must be chosen, it is not implied by the geometry.

The vector

w

lies along the decision boundary.

False.

w

is the normal — perpendicular to the boundary — and points toward the class-1 side; moving along

w

increases

z

fastest.

If two features are perfectly correlated, the boundary is undefined.

False, but the weights become non-unique. Many

w

vectors give the same boundary line; the geometry is fine, only the numerical solution is unstable.

Adding more features can only make the boundary more flexible, never worse.

False. Extra (especially noisy) features let the boundary overfit, wiggling to trap training points; flexibility without regularization hurts generalization.

Spot the error

P = 0.5

reduces to

z = 0.5

Wrong:

P = σ (z) = 0.5

inverts to

z = 0

, not

z = 0.5

. Sigmoid outputs 0.5 at input 0 — you must undo the sigmoid, not read the number off directly.

"Since

σ

is smooth and curved, the boundary between classes is smooth and curved too."

Confuses probability space with feature space. The curve is in the

P

-vs-

z

graph; the boundary is a level set

z = 0

x

-space, which stays flat.

"To classify

(1, 1)

under

z = x_{1} + x_{2} - 3

, we need the sigmoid value first."

Unnecessary. Only the sign of

z

matters for the label;

z = 1 + 1 - 3 = - 1 < 0 \Rightarrow

class 0, no sigmoid arithmetic required.

"Plain logistic regression on

(x_{1}, x_{2})

can separate points inside a ring from points outside it."

Impossible — a ring separation is not linearly separable in raw coordinates. You must engineer a feature like

x_{1}^{2} + x_{2}^{2}

first, then the boundary becomes a circle.

"The distance of a point from the boundary is just

w^{⊤} x + b

Missing the normalization: signed distance is

\frac{w ^{⊤} x + b}{∥ w ∥}

. Without dividing by

∥ w ∥

, rescaling weights would fake a "farther" point.

"A softmax (multi-class) model has one boundary, like the two-class case."

For

K

classes you get several pairwise boundaries; the space is carved into

K

regions whose walls are the ties between adjacent classes, not a single hyperplane. See Softmax Classification.

Why questions

Why does the two-class boundary sit at probability 0.5 and not some other value?

The two class probabilities sum to 1, so they are equal exactly when each is 0.5 — that is the unique "no preference" knife-edge.

Why does

σ (z) = 0.5

correspond to exactly one score

z

Because the sigmoid is strictly monotonic (see Sigmoid Function); each output value is hit by one and only one input, so 0.5 maps back uniquely to

z = 0

Why is the logistic boundary linear even though logistic regression is a "nonlinear" model?

The nonlinearity (

σ

) only reshapes probabilities; the location of the tie depends solely on

z = 0

, and

z

is a straight-line function of the features.

Why does the magnitude of

w

control confidence but not boundary location?

A larger

∥ w ∥

makes

z

change faster as you step away from the boundary, so

σ

saturates quickly — sharper confidence — but

z = 0

is at the same place regardless of scale.

Why can adding the product feature

x_{1} x_{2}

create a diagonal or hyperbolic boundary?

The cross-term couples the two axes, so the level set

z = 0

becomes a conic; the raw-space shape rotates/curves even though it is still flat in the expanded feature space.

Why do Support Vector Machines and logistic regression share the same kind of boundary?

Both produce a hyperplane

w^{⊤} x + b = 0

; they differ only in which hyperplane they pick — SVM maximizes the margin, logistic regression maximizes likelihood. See Support Vector Machines.

Why is Linear Regression a poor classifier despite sharing the score

z

Its output is unbounded and its squared loss punishes confident-correct points, dragging the fitted line so the implied threshold sits in the wrong place; the boundary concept is the same, the fitting objective is mismatched.

Edge cases

What is the boundary when

w = 0

but

b \neq = 0

There is no boundary —

z = b

is a nonzero constant everywhere, so every point gets the same class; the model has collapsed to a single prediction.

What is the boundary when

w = 0

and

b = 0

z = 0

everywhere, so every point is tied at

P = 0.5

; the whole space is boundary and nothing is classified.

In 1D, what does the "hyperplane" boundary look like?

A single point on the number line (e.g.

x = 2

for

z = 2 x - 4

); one side predicts class 0, the other class 1.

If the data are perfectly separable, is the logistic boundary unique?

No — weights can grow without bound and multiple hyperplanes achieve zero training loss; regularization is needed to pin down a single, finite boundary.

What happens to the boundary if you shift the threshold all the way to

t \to 1

The constant

ln \frac{t}{1 - t} \to + \infty

, pushing the parallel hyperplane out to infinity — effectively the model refuses to ever predict class 1.

For a circular boundary

x_{1}^{2} + x_{2}^{2} = 4

, which side is class 1 if

z = x_{1}^{2} + x_{2}^{2} - 4

Outside the circle, where

z > 0

; inside,

z < 0

gives class 0 — the sign of the engineered score, not "inside vs bigger", decides.

What if a test point falls exactly at the center of a circular boundary?

With

z = x_{1}^{2} + x_{2}^{2} - 4

at the origin,

z = - 4 < 0

, firmly class 0 — the center is maximally far inside the ring, not on the fence.

Connections

Logistic Regression — the source of the $z = 0$ boundary these traps probe.
Sigmoid Function — monotonicity is why " $P = 0.5 \Leftrightarrow z = 0$ " (not $z = 0.5$ ).
Feature Engineering — the escape hatch for curved boundaries.
Support Vector Machines — same hyperplane, different chosen one.
Softmax Classification — the multi-boundary generalization.
Linear Regression — shares the score, mismatches the objective.