1.2.8 · D5Calculus & Optimization Basics
Question bank — Convex vs non-convex functions
Parent: Convex vs non-convex functions · Hinglish: 1.2.08 Convex vs non-convex functions (Hinglish)
Symbols this page uses (build them once)
Everything below leans on four ideas. Pin them here so no line uses an undefined symbol.
True or false — justify
A twice-differentiable with at its minimum is convex.
False. Convexity needs on the whole domain; one point tells you nothing. has yet bends downward for .
If every local minimum of is also a global minimum, then must be convex.
False. It's a one-way street: convex (local global), but a non-convex can still have one basin. Concrete example: wiggles (its goes negative, so it is not convex) yet has a single lowest valley. Single valley does not force convexity.
A strictly convex function has exactly one global minimum.
True, if a minimum exists. Strict convexity forbids flat bottoms, so two distinct minimizers are impossible. But is strictly convex with no minimum at all — it just keeps decreasing.
The sum of two convex functions is convex.
True. Adding two "curve-above-chord" inequalities preserves the direction of , and summing PSD Hessians gives a PSD Hessian. This is why regularized losses like stay convex when is.
The product of two convex functions is convex.
False. and are both convex, and their product is not convex ( flips sign). Products do not generally preserve convexity — you need extra conditions (e.g. both non-negative and non-decreasing).
A function that is convex is automatically continuous on the interior of its domain.
True. Convexity forces the graph to lie below chords and above tangents, which squeezes out jumps on the interior. (Boundary points can still misbehave.)
If is convex then is concave.
True. Flipping the sign flips the master inequality into , which is exactly the definition of concave. That's why "cups upward" ↔ "caves downward."
Only linear functions are both convex and concave.
False, but close. It's the affine functions (, constant term allowed) — not just linear (). Affine curves equal their own chords and tangents, satisfying both inequalities with equality.
Every convex function is differentiable.
False. is convex but has a corner at (no derivative there). Convexity guarantees a subgradient, not a clean tangent.
A convex function of one convex variable composed with a convex function, , is always convex.
False. You also need to be non-decreasing. Example: (convex, decreasing) and gives , concave. Composition rules have sign/monotonicity conditions.
Spot the error
"The Hessian is , its trace is , so is PSD hence convex."
The test is every eigenvalue , not their sum. Here the eigenvalues are and ; the direction (the -axis) gives , so is indefinite. This is , a saddle, not a bowl.
" has a minimum at the origin because the gradient is zero there."
only marks a stationary point. It's a saddle: minimum along , maximum along . Zero gradient ≠ minimum without curvature info.
" for all (a straight-line graph, i.e. an affine function) means the function fails the convexity test."
An affine has , and is satisfied — affine functions are convex (weakly). Zero curvature passes; only negative curvature fails.
"Gradient descent converged and the gradient is tiny, so we're at the global minimum."
Tiny gradient means stationary, which on a non-convex loss could be a local min, a saddle, or a plateau. Only under convexity does certify globality. See Gradient Descent.
"The loss curve I plotted along one direction looks like a bowl, so the loss is convex."
A slice being convex says nothing about other directions. Convexity requires the Hessian PSD in all directions ; looks convex along yet is a saddle.
" for , so is not strictly convex and has no unique min."
is strictly convex overall and has a unique global min at ; the second-order test is only sufficient, and a single flat-curvature point doesn't break strictness.
"The Hessian at my minimum is positive definite, therefore the whole function is convex."
PD at one point only makes that point a local min. Convexity is the Hessian being PSD everywhere — check the entire domain, not the minimizer alone.
Why questions
Why does convexity make "local minimum global minimum"?
The first-order condition says the graph lies above every tangent; at a stationary point the tangent is flat (), so for all — no deeper point can exist. (See the tangent-floor figure above and Taylor Series Expansion.)
Why do we test the Hessian rather than just in many variables?
In multiple dimensions "curving up" must hold in every direction , which is exactly — the PSD condition. A single number can't capture all directions.
Why are neural-network losses non-convex?
Stacking nonlinear activations composes functions in ways that produce multiple valleys and saddles, so the Hessian is indefinite somewhere. Convexity's global guarantee then collapses — see Loss Functions.
Why do we love convex losses in ML?
They give one honest basin, so convex optimization guarantees the point you converge to is the best possible — no fear of fake bottoms. Linear/logistic regression and SVMs live here.
Why isn't "second derivative positive at the min" enough to call a function convex?
That's a purely local fact about one point; convexity is a global statement linking every pair of points via the chord. Local curvature can lie about the global shape.
Why can gradient descent stall near a saddle even though it's not a minimum?
Near a saddle the gradient shrinks toward zero (it's a stationary point), so steps become tiny even though a downhill escape direction exists along the negative-eigenvalue axis.
Edge cases
Is a constant function convex?
Yes. It's affine (), so it satisfies the chord inequality with equality — it's simultaneously convex and concave.
Is convex despite the corner at ?
Yes. The chord test holds everywhere; convexity does not require differentiability, only that the curve never rises above any chord.
Is convex, and does it have a global minimum?
Convex yes, minimum no. everywhere (convex), but the graph keeps falling as — convexity guarantees no fake minimum, not that a minimum exists.
At an inflection point like for , what does tell us?
Almost nothing on its own — changes sign through it, so the curve bends both ways. It's neither a min nor a max, and the function is not convex.
If a domain is non-convex (say two disjoint intervals), can the function still be "convex"?
No, the definition breaks. The chord between points in different pieces leaves the domain, so isn't even defined — convex functions require a convex domain first.
Does a Hessian with a zero eigenvalue (and the rest positive) still count as convex?
Yes, PSD allows zeros. Zero eigenvalue means a flat direction (like a trough), which is convex-but-not-strictly. Only a negative eigenvalue would break it.
Is convex, concave, or neither?
Neither. flips sign every , so the curve cups up on some intervals and caves down on others — the classic "most functions are neither" case.
Recall One-line survival kit
"Convex" is a global, everywhere claim: chord above curve / tangent below curve / Hessian PSD in all directions across the whole domain. Any trap that swaps "everywhere" for "at one point," or "all directions" for "one slice," or forgets the domain, is the error.