Softmax for output layers: Question bank — AI-ML deep dive

This is a conceptual drill for softmax. No heavy arithmetic — every item hunts a misconception or a boundary case. Read the prompt, answer out loud in a sentence, then reveal. If your reasoning differs from the answer's reasoning (even if the verdict matches), you have a gap worth closing.

Two facts are invoked so often below that we prove them once here, so the drill items can lean on them:

True or false — justify

Softmax outputs are guaranteed to lie strictly between 0 and 1, never exactly 0 or 1.

True in exact arithmetic. Each output is

e^{z_{i}} / \sum_{j} e^{z_{j}}

, and

e^{x} > 0

for every finite

x

, so the numerator is positive and strictly smaller than the sum. (In finite-precision floating point a very negative

z_{i}

can underflow

e^{z_{i}}

to exactly

0.0

— but that is a hardware rounding artifact, not the true value.)

If one logit is much larger than all others, softmax outputs an exact 1 for it.

False in exact math — it approaches 1 but never equals it. In floating point it rounds to 1.0, but that is numerical rounding, not the true value.

Adding 5 to every logit makes the winning class more confident.

False. By Fact A (shift-invariance) the common factor

e^{5}

cancels top and bottom, so every

p_{i}

is the identical number it was before — adding a constant to all logits changes nothing.

Multiplying every logit by 2 makes the winning class more confident.

True. Scaling logits is not the same as shifting: it does not cancel out. Multiplying by 2 is equivalent to setting temperature

T = \frac{1}{2} < 1

, which sharpens the distribution toward the argmax.

Softmax and argmax give the same winner (the same top class).

True. Softmax is monotonic, so the largest logit always maps to the largest probability — same winner. They differ only in that softmax is soft and differentiable, argmax is a hard non-differentiable pick.

A softmax probability of 0.99 means the model is 99% likely to be correct.

False. It is a relative score among the offered classes, not a calibrated truth. Neural nets are routinely overconfident; a wrong prediction can still read 0.99.

Softmax with

K = 2

classes (

K

= number of classes) is exactly a sigmoid.

True in effect. Two-class softmax depends only on the logit difference

z_{1} - z_{0}

, and

\frac{e ^{z_{1}}}{e ^{z_{0}} + e ^{z_{1}}} = σ (z_{1} - z_{0})

— the sigmoid of the gap.

You can feed already-normalized probabilities into softmax to "clean them up."

False. Applying softmax to values that already sum to 1 pushes them toward the uniform vector

[1/ K, \dots, 1/ K]

(e.g.

[0.7, 0.2, 0.1] \to [0.42, 0.30, 0.28]

), destroying the confidence you had — softmax is meant for raw logits, applied exactly once.

Softmax is a linear function of its inputs.

False. The exponential and the division make it nonlinear — that nonlinearity is precisely what amplifies gaps and produces a "soft winner."

If two logits are equal, their softmax probabilities are equal.

True. Equal inputs give equal

e^{z_{i}}

, hence equal shares — softmax is symmetric and respects ties exactly.

Spot the error

"To avoid negative probabilities, we normalize by writing

p_{i} = z_{i} / \sum_{j} z_{j}

The error: raw logits

z_{i}

can be negative (giving negative "probabilities") and their sum can be zero (division blows up). Exponentiating first,

e^{z_{i}}

, fixes both because it is always positive.

"Softmax subtracts the max only to change the answer to something safer."

The error: subtracting

max_{k} z_{k}

changes nothing in the output — it is exactly Fact A (shift-invariance) with

c = - max_{k} z_{k}

. It only prevents floating-point overflow of

e^{z}

; the probabilities are identical.

"For an image that is both a beach and sunny, softmax over {beach, sunny, indoor} handles it."

The error: softmax forces classes to compete (they sum to 1), so it cannot say "both." Multi-label problems need independent sigmoids (one probability per class, not sharing a pie) instead.

"The gradient

\partial L / \partial z_{k} = p_{k} - t_{k}

only works because cross-entropy is a special complicated case."

The error: it's the opposite — as shown in Fact B, the

lo g

in cross-entropy cancels the

e

in softmax, making the gradient simpler, not more special. That cancellation is exactly why the pair is used together.

"We divide by the sum of the logits to normalize."

The error: we divide by the sum of the exponentiated logits,

\sum_{j} e^{z_{j}}

. Dividing by the sum of raw logits reintroduces the sign and zero-sum problems.

"Since softmax outputs sum to 1, doubling one output and halving another keeps it valid."

The error: softmax outputs a specific distribution determined by the logits; you cannot hand-edit two values and expect it to correspond to any logit vector. "Sums to 1" is necessary, not sufficient.

"Applying softmax twice is harmless because it's idempotent like a normalize step."

The error: softmax is not idempotent. Each extra pass moves the vector closer to uniform

[1/ K, \dots, 1/ K]

(numerically: a sharp

[0.9, 0.08, 0.02]

becomes a duller

[0.44, 0.30, 0.26]

), discarding confidence information.

Why questions

Why exponentiate instead of, say, squaring the logits to force positivity?

Squaring loses sign/order information (

(- 3)^{2} = (+ 3)^{2}

), breaking monotonicity, and its gradient stalls at zero.

e^{z}

stays positive, strictly increasing, differentiable everywhere, and arises from the maximum-entropy assumption

lo g p_{i} \propto z_{i}

Why is softmax called a "soft" argmax rather than just argmax?

Argmax outputs a hard 0/1 pick with zero gradient almost everywhere, so it cannot be trained. Softmax gives a smooth, differentiable approximation that still favors the top class but passes usable gradients back through Backpropagation.

Why does the true class get a negative gradient

p_{y} - t_{y}

under softmax + cross-entropy?

For the true class

t_{y} = 1

(one-hot target) and

p_{y} < 1

, so

p_{y} - 1 < 0

; a negative gradient means "increase this logit," pushing the model to raise the correct class's score — exactly the desired learning direction (see Fact B for where

p_{k} - t_{k}

comes from).

Why can't we drop the exponential and use ReLU-then-normalize instead?

ReLU zeroes out all negative logits, so several classes could collapse to probability 0 and the gradient dies there. The exponential keeps every class alive with a nonzero, differentiable share.

Why does temperature

T

appear as division of logits,

z_{i} / T

, and not division of probabilities?

Dividing logits and then exponentiating scales the gaps in log-space, smoothly sharpening (

T < 1

) or softening (

T > 1

) the whole distribution while keeping it valid (see figure). Dividing probabilities directly would break the sum-to-1 guarantee.

Temperature

T

reshapes the same logits — look at figure s01 (bars sharpen as

T

falls, flatten as

T

rises).

The logits are fixed; only

T

changes. Small

T

concentrates mass on the top class; large

T

spreads it toward uniform — confidence is a dial, not a fixed property of the logits.

Edge cases

What does softmax output when all logits are equal (e.g.

[0, 0, 0]

The uniform distribution

[1/ K, \dots, 1/ K]

(

K

= number of classes). With no information favoring any class, softmax returns maximum entropy — total ignorance spread evenly.

What happens to softmax as one logit

\to + \infty

while others stay finite?

That class's probability approaches 1 and all others approach 0 — softmax degenerates toward a hard argmax in the limit, but never exactly reaches 0 or 1 for finite inputs.

What happens to softmax as one logit

\to - \infty

while others stay finite?

By symmetry with the

+ \infty

case,

e^{z_{i}} \to 0

, so that class's probability

p_{i} \to 0

and the surviving classes share the whole distribution. In exact math it approaches but never reaches 0; in floating point, a sufficiently negative logit underflows

e^{z_{i}}

to exactly

0.0

What happens as temperature

T \to \infty

Every

z_{i} / T \to 0

, so softmax approaches the uniform distribution

[1/ K, \dots, 1/ K]

regardless of the logits — infinite temperature erases all confidence (rightmost trend in figure s01).

What happens as temperature

T \to 0^{+}

The gaps blow up, and softmax collapses onto the single largest logit (a one-hot vector) — zero temperature reproduces hard argmax (leftmost trend in figure s01).

With logits

[1000, 1001, 1002]

, why does the naive formula fail and the shifted one not?

Naively

e^{1002}

overflows to inf and the ratio becomes NaN. Subtracting the max gives shifted logits

[- 2, - 1, 0]

whose largest exponent is

e^{0} = 1

— same exact probabilities, no overflow.

What is softmax of a single-element vector

[z_{1}]

(K=1)?

Always

[1.0]

, since

e^{z_{1}} / e^{z_{1}} = 1

for any

z_{1}

— with only one class there is nothing to be uncertain about.

If two classes tie for the largest logit, what does softmax do that argmax cannot?

Softmax splits the probability evenly between the tied classes (e.g.

0.5/0.5

over them), giving a well-defined, differentiable answer, whereas argmax must break the tie arbitrarily.

Connections

Cross-Entropy Loss — the log that cancels softmax's exp, giving the clean $p - t$ gradient.
Sigmoid Activation — the two-class / multi-label alternative when classes shouldn't compete.
Logits and Log-Odds — the raw inputs softmax is meant to consume.
Maximum Entropy Distributions — why the exponential form is the least-biased choice.
Temperature Scaling — dividing logits by $T$ to sharpen or soften the output.
Backpropagation — where the $p - t$ gradient travels backward.