Question bank — Critical points and saddle points

False. Zero gradient only says the surface is flat to first order; it could equally be a maximum, saddle, or inflection point. You must inspect curvature (the Hessian) to decide.

True. If any partial derivative were nonzero you could step downhill and lower $f$, contradicting "minimum," so all slopes must vanish — that is the critical-point condition.

False. All-positive eigenvalues certify a local minimum (a bowl locally); a non-convex loss can have many such bowls. Only for a convex function does local imply global.

True for the clean (non-degenerate) case. Mixed signs are the definition of an indefinite Hessian, meaning $f$ climbs one way and descends another.

False. A critical point is a saddle if even one of the millions of eigenvalues is negative; getting all same-sign becomes exponentially unlikely, so saddles vastly outnumber minima.

False. $f(x)=x^4$ has $f^{\prime \prime }(0)=0$ yet $0$ is a genuine minimum. A zero second derivative just makes the test inconclusive — look at higher terms.

True. Momentum carries accumulated velocity, so the iterate keeps moving through the flat spot until the negative-curvature direction takes over.

True whenever the second partials are continuous (Clairaut/Schwarz theorem), since ${\partial }^{2}f/\partial x_i\partial x_j={\partial }^{2}f/\partial x_j\partial x_i$. Real eigenvalues follow from this symmetry.

False. A zero eigenvalue means the second-order test fails along that direction; the point could be a minimum, a saddle, or something needing third/fourth-order analysis (like a flat monkey-saddle).

The error is equating a flat gradient with optimality. It may be a saddle; check the smallest eigenvalue of $\mathbf{H}$ — if it is negative, an escape direction exists and the model is not optimal.

You never build the full matrix. Hessian-vector products $\mathbf{Hv}$ cost only $O(n)$ via autodiff, and iterative methods extract leading eigenvalues cheaply — see Second-Order Optimization.

The Hessian is $\left(\begin{array}{cc}2& 0\\ 0& -2\end{array}\right)$ with eigenvalues $+2,-2$ — mixed signs. It is a saddle: up along $x$, down along $y$, never a minimum.

False dichotomy. Here $f^{\prime \prime }(0)=0$ and $f^{\prime \prime \prime }(0)=6\ne 0$, giving an inflection point — the curve passes straight through, pausing but not turning around.

Reversed. All-negative eigenvalues make $\mathbf{H}$ negative definite, signalling a local maximum. Positive definite means all-positive → minimum.

Saddles are unstable equilibria: minibatch noise or finite precision perturbs the iterate off the exact saddle, and the negative-curvature direction then pulls it away. They slow, rarely trap.

The gradient is already $0$ there, so it tells you nothing new. Classification needs the second-order object — the Hessian and its eigenvalues.

Because $\nabla f({\mathbf{x}}^{\ast })=0$, so $\nabla f({\mathbf{x}}^{\ast }{)}^{\top }\mathbf{d}=0$ for every step $\mathbf{d}$; behaviour is then governed by the next term, the quadratic form $\frac{1}{2}{\mathbf{d}}^{\top }\mathbf{Hd}$.

Eigenvalues are the curvatures along the Hessian's principal (eigenvector) axes — a rotation-independent readout of "up or down" in each independent direction, which raw entries mix together.

A minimum needs all $n$ eigenvalues positive; for a roughly random symmetric Hessian each sign is like a coin flip, so all-same-sign has probability that decays exponentially in $n$.

Noise nudges the iterate off the exact flat point onto the negative-eigenvalue direction, where the loss genuinely decreases, letting the descent step do useful work again.

A zero eigenvalue means the quadratic form is flat along that direction, so second-order information cannot distinguish rise from fall — you must consult cubic or higher terms.

Near a saddle $\Vert \nabla f\Vert \approx 0$, so gradient steps become tiny and progress crawls, wasting many iterations before the escape direction dominates — a slowdown, not a dead end.

A genuine local (and global) minimum, even though $f^{\prime \prime }(0)=0$. The fourth-order term $x^4>0$ for $x\ne 0$ confirms the bowl; the quadratic test simply couldn't see it.

A degenerate saddle: the Hessian is the zero matrix (all ${\lambda }_i=0$), so second order is blind; the surface actually has three descending valleys, revealed only by cubic terms.

Yes. Along the zero-curvature direction the sign is decided by higher-order terms; if they turn negative there, the flat-then-down profile makes it a saddle despite the "almost bowl" Hessian.

Exactly one, and it is the unique global minimum — convexity forces the Hessian positive definite everywhere, ruling out maxima and saddles entirely.

The gradient definition breaks down; $0$ is still the minimum of $|x|$ but must be handled by subgradients, not by setting $\nabla f=0$. The smooth theory simply doesn't apply.

It degrades: escape-time results (e.g. Ge et al.) require ${\lambda }_{\min }<-c<0$ for some fixed gap $c$. As ${\lambda }_{\min }\to 0$, the downhill push weakens and escaping can take arbitrarily long — a near-flat plateau.