Optimization — constrained, unconstrained, real-world problems: Foundations

This page assumes you have seen none of the notation the parent note uses. We build each symbol before it is allowed to appear. Read top to bottom — every idea leans only on the one before it.

1. A function $f (x)$ — the machine that turns numbers into numbers

The letter $f$ is just a name for the rule (like naming a recipe). The letter $x$ is a placeholder for whatever input we drop in. If the rule is "square it and add one," then $f (x) = x^{2} + 1$ , so $f (3) = 10$ .

Why the topic needs it. Optimization always asks "make something as big or small as possible." That something — volume, distance, profit — is a number that depends on a choice. A function is exactly "a number that depends on a choice."

Figure — Optimization — constrained, unconstrained, real-world problems

Look at the picture: the horizontal axis carries the input $x$ , the vertical axis carries the output $f (x)$ . The curve is the set of all points $(x, f (x))$ — a picture of every input paired with its output.

2. Maximum and minimum — the tallest and lowest points

Picture a hilly landscape drawn as a curve. Your eye finds the tops of hills (maxima) and bottoms of dips (minima) instantly. Optimization is the algebra that finds them without needing to eyeball the graph.

Local means "highest/lowest compared to nearby points only" — the top of one hill.
Global means "highest/lowest over the whole allowed range" — the tallest hill anywhere.

Why the topic needs it. "Maximize the volume" is literally "find the global maximum of the volume function." Naming peaks and valleys is naming the goal.

3. Slope — how steep the curve is, right now

Imagine zooming into a curve until the piece under your finger looks like a straight line. The slope is $\frac{rise}{run}$ of that little straight line: how far up you go divided by how far across.

In the figure, the pale-yellow line just kisses the curve at one point — this is the tangent line, the straight line that matches the curve's direction there. Its steepness is the slope at that point.

4. The derivative $f^{'} (x)$ — slope turned into its own function

The little mark $^{'}$ is called "prime." It does not mean subtraction or a new variable — it is a label saying "the slope-machine built from $f$ ."

Why we need a whole new function, not just one number. The slope of a curve changes as you move along it — steep here, flat there, downhill later. So the slope is itself something that depends on where you are, i.e. a function of $x$ . That is exactly what $f^{'} (x)$ records.

Why the topic needs it — and why this tool and not another. We could try plugging in thousands of $x$ -values and comparing heights, but that never proves we found the true peak. The derivative answers a sharper question — "where does the curve stop rising or falling?" — with a single equation $f^{'} (x) = 0$ . That is why calculus, not guesswork, powers optimization.

The figure stacks the two graphs: on top, the curve $f$ with its hill and valley; below, its derivative $f^{'}$ . Notice where the top curve is flat, the bottom curve crosses zero (the dashed line). That crossing is the mechanical fingerprint of an extremum.

5. The second derivative $f^{''} (x)$ — the slope of the slope

Think of $f^{'}$ as "how fast the height changes" and $f^{''}$ as "how fast the slope changes." (If height were position, $f^{'}$ is velocity and $f^{''}$ is acceleration.)

Why the topic needs it. A flat point ( $f^{'} = 0$ ) alone doesn't tell peak from valley from neither. The bending does. So $f^{''}$ is the tool that classifies a flat point once we have found it. See Second Derivative & Concavity.

6. Critical point — where the flat signal fires

The letter $c$ is just a name for "the special input we found," reused every time.

Why the topic needs it. Critical points are the candidates. The whole game becomes: list critical points, then classify each. This is Critical Points & Fermat's Theorem.

7. The endpoints — the edges of the allowed range

Many real problems only allow $x$ inside an interval $[a, b]$ (read "from $a$ to $b$ , endpoints included). The biggest or smallest value can sit at an edge even where the curve is not flat — like the highest point of a road being the top of the on-ramp, not a hill.

Why the topic needs it. To claim a global max/min you must compare critical points and both endpoints. That comparison is the Closed Interval Method (Global Extrema).

8. Two variables and the constraint — $g (x, y) = 0$

Sometimes the quantity depends on two free numbers, $x$ and $y$ , written $f (x, y)$ . A constraint is an extra rule they must obey, written $g (x, y) = 0$ — for example " $x + y = 10$ " rearranged to " $x + y - 10 = 0$ ." Here $g$ is a second function whose zero-set is the curve you're allowed to walk along.

The symbol $\nabla$ (an upside-down triangle, "nabla") is the two-variable cousin of the prime mark: it collects all the slopes at once.

Why the topic needs it. With a constraint you can't move freely — you're stuck on the curve $g = 0$ . You can improve $f$ until its uphill arrow $\nabla f$ points straight across the curve, with nothing left to gain along it. That is the geometric seed of Lagrange Multipliers (Multivariable) and connects to AM-GM Inequality.

9. Taylor expansion — zooming in with a polynomial

Written near a point $c$ : $f (c + h) \approx f (c) + f^{'} (c) h + \frac{1}{2} f^{''} (c) h^{2} .$ Here $h$ is "a tiny step away from $c$ ." When $f^{'} (c) = 0$ , the middle term vanishes and the change is governed by $\frac{1}{2} f^{''} (c) h^{2}$ — and since $h^{2} \geq 0$ , the sign of $f^{''} (c)$ decides everything. This is where the classification test comes from: see Taylor Series Expansion.

Prerequisite map

Read it as: functions give you a notion of peaks and valleys; slope leads to the derivative; the derivative gives critical points and (differentiated again) the second derivative; endpoints, gradient and Taylor all feed the final optimization machinery.

Equipment checklist

Cover the right side and test yourself before entering the parent note.

I can state in plain words what

f (x)

means

A rule that turns each input number

x

into exactly one output; read "

f

x

," no multiplication.

I can say what the slope at a point is

Rise over run of the tangent line — how much output changes per tiny step in input.

I know what

f^{'} (x)

is and what its sign tells me

The slope-function;

+

uphill,

-

downhill,

0

momentarily flat (candidate peak/valley).

I can read the second derivative

f^{''}

How the slope changes;

f^{''} > 0

cup (valley),

f^{''} < 0

cap (hilltop).

I can define a critical point

An input

c

where

f^{'} (c) = 0

f^{'}

doesn't exist — only a candidate extremum.

I know why endpoints matter

A global max/min can sit at the edge of the allowed range, not just where the curve is flat.

I understand what a constraint

g (x, y) = 0

An extra rule the inputs must obey — a curve you're forced to walk along.

I can describe the gradient

\nabla f

An arrow pointing in the steepest-uphill direction, packing all the slopes together.

I know what the Taylor expansion buys us

A polynomial approximation near a point, used to prove the cup/cap classification test.

Foundations — Optimization — constrained, unconstrained, real-world problems

1. A function $f (x)$ — the machine that turns numbers into numbers

2. Maximum and minimum — the tallest and lowest points

3. Slope — how steep the curve is, right now

4. The derivative $f^{'} (x)$ — slope turned into its own function

5. The second derivative $f^{''} (x)$ — the slope of the slope

6. Critical point — where the flat signal fires

7. The endpoints — the edges of the allowed range

8. Two variables and the constraint — $g (x, y) = 0$

9. Taylor expansion — zooming in with a polynomial

Prerequisite map

Equipment checklist

Connections