scipy.optimize — minimize, fsolve, curve_fit, linprog: Worked examples

This page is a case-hunting expedition. The parent note showed you the four engines and why they exist. Here we push every one of them into the corners: multiple roots, bad starting guesses, degenerate data, unbounded regions, and an exam-style twist. Nothing on this page uses a symbol we have not already earned in the parent, and where new ideas appear (local vs global, the residual as a picture) we build them from zero.

Before any code: three words you must own.

The scenario matrix

Every worked example below is tagged with a cell from this table. Together they cover the whole grid.

Cell	Tool	The scenario class it stress-tests
A	`minimize`	Smooth convex bowl — one global minimum, any seed works
B	`minimize`	Two valleys (non-convex) — seed decides the answer
C	`fsolve`	System with two roots — sign of seed picks the root
D	`fsolve`	Degenerate: no real root → what the solver returns
E	`curve_fit`	Nonlinear model with good `p0` → recovers parameters
F	`curve_fit`	Limiting case: perfect (noise-free) data → residuals $\to 0$
G	`linprog`	Bounded polytope → optimum at a corner
H	`linprog`	Unbounded region → solver reports failure, not a number
I	word problem	Real-world allocation, translate English → matrices
J	exam twist	Maximize and honour an equality constraint

Cell A — a smooth convex bowl (`minimize`)

Worked example A. One valley, any seed

Minimize $f (x, y) = (x - 3)^{2} + (y + 1)^{2}$ .

Forecast: where is the lowest point, and what is $f$ there? Guess before reading.

Steps

Recognise the shape. Each squared term is $\geq 0$ , so $f \geq 0$ always, and $f = 0$ only when both squares vanish. Why this step? A sum of squares is a bowl (see figure) — convex, one bottom. No seed can be trapped elsewhere.
The bottom is where each bracket is zero: $x - 3 = 0$ and $y + 1 = 0$ , i.e. $(3, - 1)$ . Why? Zero is the smallest value a square can take; achieving it in both directions minimises the whole sum.
Run it:

from scipy.optimize import minimize
f = lambda v: (v[0]-3)**2 + (v[1]+1)**2
res = minimize(f, x0=[0,0])
print(res.x, res.fun)   # ≈ [3, -1]  0.0

Why x0=[0,0]? A seed is mandatory, but for a convex bowl any seed rolls to the same bottom.

Verify: plug $(3, - 1)$ back: $(3 - 3)^{2} + (- 1 + 1)^{2} = 0$ . Lowest possible, matches res.fun. ✓

Cell B — two valleys, the seed decides (`minimize`)

Worked example B. Non-convex: where you start matters

Minimize $f (x) = x^{4} - 8 x^{2} + 3 x$ on the real line.

Forecast: this quartic has two dips. Will minimize find the deeper one from any start? Guess yes/no.

Steps

Look at the picture: two valleys, a hump between them. Why this step? minimize is local — it cannot see over the hump. The figure shows two seeds rolling into different bottoms.
Find the dips by hand: set $f^{'} (x) = 4 x^{3} - 16 x + 3 = 0$ . This cubic has three real roots, near $x \approx - 2.016$ , $x \approx 0.187$ , $x \approx 1.829$ (the middle one is the hump/maximum). Why? Stationary points are where the slope is zero — the parent note's rule. The two outer ones are the valleys.
Compare valley depths: $f (- 2.016) \approx - 22.05$ (deep, global), $f (1.829) \approx - 10.09$ (shallow, local only). Why? "Global minimum" = the lowest of all valleys; "local" = lowest only in its neighbourhood.
Run from two seeds:

from scipy.optimize import minimize
f = lambda v: v[0]**4 - 8*v[0]**2 + 3*v[0]
print(minimize(f, x0=[-2]).x)   # ≈ [-2.016]  global
print(minimize(f, x0=[ 2]).x)   # ≈ [ 1.829]  local trap

Why two seeds? To prove the answer depends on the seed — the standard fix for non-convex problems is to try several.

Verify: at $x \approx - 2.016$ the slope $f^{'} \approx 0$ (a genuine stationary point) and $f \approx - 22.05$ , which is below the other valley's $\approx - 10.09$ — so it is the global minimum. The VERIFY block confirms both numbers. ✓

Cell C — a system with two roots (`fsolve`)

Worked example C. Circle meets line — sign of seed picks the root

Solve $x^{2} + y^{2} = 25$ and $x - y = 1$ .

Forecast: a circle of radius 5 and a straight line cross at two points. Which one does fsolve return from [1,1] vs [-5,-5]?

Steps

Move everything to one side so each equation reads " $= 0$ ": $F_{1} = x^{2} + y^{2} - 25, F_{2} = x - y - 1.$ Why this step? fsolve hunts for where its output vector is zero. You must present the equations in that form.
Picture the geometry: circle ∩ line → two intersection points. Why? The figure shows both roots; fsolve converges to whichever is nearest the seed.
Solve by substitution to know the truth: $y = x - 1 \Rightarrow x^{2} + (x - 1)^{2} = 25 \Rightarrow 2 x^{2} - 2 x - 24 = 0 \Rightarrow x^{2} - x - 12 = 0 \Rightarrow (x - 4) (x + 3) = 0$ . So $x = 4, y = 3$ or $x = - 3, y = - 4$ . Why? Having the exact roots lets us verify the solver, not just trust it.
Run with two seeds:

from scipy.optimize import fsolve
def F(v):
    x, y = v
    return [x**2+y**2-25, x-y-1]
print(fsolve(F, x0=[ 1, 1]))   # ≈ [4, 3]
print(fsolve(F, x0=[-5,-5]))   # ≈ [-3, -4]

Why negative seed? Starting in the third quadrant lands you on the third-quadrant root — proving locality.

Verify: $4^{2} + 3^{2} = 25$ ✓ and $4 - 3 = 1$ ✓. Also $(- 3)^{2} + (- 4)^{2} = 25$ ✓, $- 3 - (- 4) = 1$ ✓.

Cell D — a degenerate system with no real root (`fsolve`)

Worked example D. What happens when there is nothing to find

Solve $x^{2} + 1 = 0$ (no real solution).

Forecast: there is no real $x$ with $x^{2} = - 1$ . Will fsolve crash, or lie, or warn?

Steps

Recognise the degeneracy: $x^{2} \geq 0$ always, so $x^{2} + 1 \geq 1 > 0$ . The curve never touches the $0$ line. Why this step? A root requires the curve to cross zero. Here it floats above — see the figure, the parabola sits entirely over the axis.
Run it and always ask for diagnostics:

from scipy.optimize import fsolve
F = lambda v: [v[0]**2 + 1]
x, info, ier, msg = fsolve(F, x0=[1.0], full_output=True)
print(x, ier)   # x stalls near 0; ier != 1 → not converged

Why full_output=True? ier==1 means "converged"; any other value means "do not trust x". This is the fix from the parent's mistake box. 3. Interpret: the solver drifts to the point of smallest $∣ F ∣$ (near $x = 0$ , where $F = 1$ ) but can never make it zero, so it flags failure. Why? Newton's step $- F / F^{'}$ can't drive a strictly-positive function to zero; the engine gives up.

Verify: at the returned $x \approx 0$ , $F = 0^{2} + 1 = 1 \neq = 0$ , and ier != 1. The solver correctly refuses to claim success. ✓ (We check that $x^{2} + 1 > 0$ for all real $x$ symbolically.)

Cell E — nonlinear fit with a good seed (`curve_fit`)

Worked example E. Recover the hidden parameters

Fit $y = a e^{- b x}$ to data generated with $a = 2.5, b = 1.3$ .

Forecast: if the model matches how the data was made, can curve_fit read off $a$ and $b$ ? Guess how close.

Steps

Write the model as a Python function of x then the parameters:

import numpy as np
from scipy.optimize import curve_fit
model = lambda x, a, b: a*np.exp(-b*x)

Why this signature? curve_fit demands f(x, *params) — independent variable first, unknowns after. 2. Fit with a sensible p0:

x = np.linspace(0, 4, 50)
rng = np.random.default_rng(0)
y = 2.5*np.exp(-1.3*x) + 0.02*rng.standard_normal(50)
popt, pcov = curve_fit(model, x, y, p0=[1, 1])
print(popt)                    # ≈ [2.5, 1.3]
print(np.sqrt(np.diag(pcov)))  # tiny → confident

Why p0? Nonlinear least squares is non-convex; a rough guess ( $a \sim 1, b \sim 1$ ) keeps Levenberg–Marquardt from wandering off.

Verify: popt lands within a few percent of $(2.5, 1.3)$ and the standard errors (root of the pcov diagonal) are small — the fit is trustworthy. We check the noise-free version exactly in Cell F.

Cell F — the limiting case: perfect data (`curve_fit`)

Worked example F. Zero noise → zero residuals

Fit a straight line $y = m x + c$ to exact points on $y = 2 x + 5$ .

Forecast: with no noise, what should the total squared error $S = \sum r_{i}^{2}$ be? Guess before computing.

Steps

Build clean data:

import numpy as np
from scipy.optimize import curve_fit
line = lambda x, m, c: m*x + c
x = np.array([0,1,2,3,4], float)
y = 2*x + 5           # exact, no noise
popt, _ = curve_fit(line, x, y, p0=[0,0])
print(popt)           # ≈ [2, 5]

Why this test? The limiting case (perfect data) is the sanity check every fitter must pass — the residual picture below shows every dot on the line.

2. Compute the residuals:

r_{i} = y_{i} - (m x_{i} + c)

. With

m = 2, c = 5

every

r_{i} = 0

. Why? The residual is the vertical gap; if the line passes through the dot, the gap is zero.

Verify: $S = \sum r_{i}^{2} = 0^{2} + 0^{2} + 0^{2} + 0^{2} + 0^{2} = 0$ . A linear model on exact linear data fits perfectly, as predicted. ✓ (Compare NumPy polyfit and Least Squares Regression — for a linear model this is the closed-form normal-equation answer, no iteration needed.)

Cell G — a bounded polytope, optimum at a corner (`linprog`)

Worked example G. Slide to the best corner

Maximize profit $40 x_{1} + 30 x_{2}$ subject to $x_{1} + x_{2} \leq 40$ , $2 x_{1} + x_{2} \leq 60$ , $x_{1}, x_{2} \geq 0$ .

Forecast: the feasible region is a polygon. Which corner wins, and what profit?

Steps

Sketch the feasible polygon (figure). Its corners are $(0, 0), (30, 0), (20, 20), (0, 40)$ . Why this step? A linear objective has a constant gradient — it never flattens inside the region, so the optimum sits at a vertex. See Linear Programming Simplex.
linprog only minimizes, so negate the profit coefficients:

from scipy.optimize import linprog
c = [-40, -30]                 # minimize -profit = maximize profit
A_ub = [[1,1],[2,1]]
b_ub = [40, 60]
res = linprog(c, A_ub=A_ub, b_ub=b_ub, bounds=[(0,None),(0,None)])
print(res.x, -res.fun)         # ≈ [20, 20]  1400

Why negate c? Maximizing $c^{⊤} x$ = minimizing $- c^{⊤} x$ ; report -res.fun to get the true profit. 3. Check each corner's profit by hand: $(30, 0) \to 1200$ , $(0, 40) \to 1200$ , $(20, 20) \to 1400$ . The interior corner wins. Why? Confirms the solver picked the true vertex, not just a feasible point.

Verify: at $(20, 20)$ : $20 + 20 = 40 \leq 40$ ✓, $2 (20) + 20 = 60 \leq 60$ ✓ (both tight — that's why it's a corner), profit $= 40 (20) + 30 (20) = 1400$ . ✓

Cell H — an unbounded region (`linprog`)

Worked example H. When "minimize" has no bottom

Minimize $- x_{1} - x_{2}$ subject to only $x_{1}, x_{2} \geq 0$ (no upper limits).

Forecast: we push $x_{1}, x_{2}$ up to make $- x_{1} - x_{2}$ ever more negative. Is there a smallest value? Guess.

Steps

See that nothing caps the variables: with no A_ub, they run to $+ \infty$ , so $- x_{1} - x_{2} \to - \infty$ . Why this step? An optimum exists only if the objective is bounded below on the feasible set. Here it is not.
Run and read the status:

from scipy.optimize import linprog
res = linprog(c=[-1,-1], bounds=[(0,None),(0,None)])
print(res.status, res.success)   # 3, False  → 'Problem is unbounded'

Why check .status? status==3 means unbounded; res.x is meaningless. Never trust .x without .success.

Verify: the solver reports success=False and status=3, i.e. it correctly refuses to return a finite corner because none exists. ✓ (No numeric answer to plug back — the absence of a number is the correct result.)

Cell I — real-world word problem (`linprog`)

Worked example I. Bakery: bread vs cake

A bakery earns ₹3 per loaf and ₹5 per cake. Each loaf needs 1 cup flour and 2 min oven; each cake needs 3 cups flour and 1 min oven. There are 24 cups flour and 16 oven-minutes. How many of each maximise revenue?

Forecast: flour favours bread, oven favours cake. Guess the mix.

Steps

Name variables: $x_{1}$ = loaves, $x_{2}$ = cakes. Objective (maximize): $3 x_{1} + 5 x_{2}$ . Why? Translating English into $c^{⊤} x$ is the whole skill — money coefficients go into $c$ .
Constraints as $\leq$ rows: flour $x_{1} + 3 x_{2} \leq 24$ ; oven $2 x_{1} + x_{2} \leq 16$ ; and $x_{1}, x_{2} \geq 0$ . Why? Each scarce resource is one inequality row of $A_{u b}$ , its total in $b_{u b}$ .
Find the binding corner by hand: solve $x_{1} + 3 x_{2} = 24$ and $2 x_{1} + x_{2} = 16$ together. From the second, $x_{2} = 16 - 2 x_{1}$ ; substitute: $x_{1} + 3 (16 - 2 x_{1}) = 24 \Rightarrow x_{1} + 48 - 6 x_{1} = 24 \Rightarrow - 5 x_{1} = - 24 \Rightarrow x_{1} = 4.8$ , then $x_{2} = 16 - 9.6 = 6.4$ . Why? The optimum sits where the two binding constraints cross — a vertex of the polygon.
Negate and solve:

from scipy.optimize import linprog
res = linprog(c=[-3,-5], A_ub=[[1,3],[2,1]], b_ub=[24,16],
              bounds=[(0,None),(0,None)])
print(res.x, -res.fun)   # ≈ [4.8, 6.4]  revenue 46.4
print(res.success, res.status)   # True  0  → optimal found

Why negate again? Same rule — linprog minimizes; we maximize revenue, so pass $- c$ and report -res.fun. Why print .success/.status? status==0 with success=True confirms a genuine optimum before we trust res.x.

Verify: at $(4.8, 6.4)$ : flour $4.8 + 3 (6.4) = 4.8 + 19.2 = 24 \leq 24$ ✓ (tight); oven $2 (4.8) + 6.4 = 9.6 + 6.4 = 16 \leq 16$ ✓ (tight); revenue $= 3 (4.8) + 5 (6.4) = 14.4 + 32 = 46.4$ . ✓

Cell J — the exam twist: maximize with an equality (`linprog`)

Worked example J. Forced to spend the whole budget

Maximize $2 x_{1} + 3 x_{2}$ subject to $x_{1} + x_{2} = 10$ (exact, an equality) and $x_{1}, x_{2} \geq 0$ .

Forecast: an equality is a line segment, not an area. The best point is an endpoint. Which one?

Steps

Spot the equality: use A_eq and b_eq, not A_ub. Why? A_ub is " $\leq$ "; an exact " $=$ " must go into the equality slots or you'd allow a whole triangle instead of one segment.
Since $x_{1} + x_{2}$ is fixed at 10, we want more of the higher-coefficient variable $x_{2}$ (coefficient 3 beats 2). So push $x_{2} = 10, x_{1} = 0$ . Why? Along the segment, swapping a unit of $x_{1}$ for $x_{2}$ gains $3 - 2 = + 1$ . Keep swapping to the endpoint.
Solve:

from scipy.optimize import linprog
res = linprog(c=[-2,-3], A_eq=[[1,1]], b_eq=[10],
              bounds=[(0,None),(0,None)])
print(res.x, -res.fun)   # ≈ [0, 10]  value 30

Why A_eq? It pins the solution to the constraint line exactly.

Verify: $(0, 10)$ : $0 + 10 = 10$ ✓ (equality holds), objective $= 2 (0) + 3 (10) = 30$ , which beats $(10, 0) \to 20$ and any mix. ✓

Recall The one-line rule behind every cell

Convex/one-valley ( $\to$ any seed, cell A/G) versus non-convex/multi-root ( $\to$ seed decides, cells B/C) — always ask "could there be more than one answer?" before trusting the first number.