Exercises — Functions, limits, and continuity

The test is the vertical line test: draw any vertical line $x=c$; if it hits the curve more than once, one input gave two outputs → not a function.

• (a) $y=3x-5$: a straight line, every vertical line hits once → ✅ function.
• (b) $x=y^2$: solve for $y$, get $y=\pm \sqrt{x}$. Input $x=4$ gives $y=+2$ and $y=-2$ → ❌ not a function.
• (c) $y=|x|$: each $x$ gives exactly one non-negative output → ✅ function.
• (d) $x^2+y^2=9$ is a circle; $x=0$ gives $y=\pm 3$ → ❌ not a function.

• (a) As $x\to 0$ the output blows up to $\pm \infty$ → infinite discontinuity.
• (b) Factor: $\frac{(x-2)(x+2)}{x-2}=x+2$ for $x\ne 2$. Limit is $4$ but $f(2)$ is undefined ($0/0$) → removable (a hole at $(2,4)$).
• (c) Left limit $=0$, right limit $=1$; they disagree → jump.

(a) Plugging in gives $0/0$ → simplify. Factor the top: $x^2-4=(x-2)(x+2)$. $\frac{x^2-4}{x-2}=\frac{(x-2)(x+2)}{x-2}=x+2(x\ne 2).$ Why legal: the limit never touches $x=2$, so $x-2\ne 0$ and we may cancel. Now ${\lim }_{x\to 2}(x+2)=4$.

(b) Plugging in gives $0/0$ again, but there's no factor to cancel — the fix is to rationalize (multiply by the conjugate to kill the square-root difference): $\frac{\sqrt{x+9}-3}{x}\cdot \frac{\sqrt{x+9}+3}{\sqrt{x+9}+3}=\frac{(x+9)-9}{x(\sqrt{x+9}+3)}=\frac{x}{x(\sqrt{x+9}+3)}=\frac{1}{\sqrt{x+9}+3}.$ Why this tool: the conjugate turns $(\sqrt{}-3)(\sqrt{}+3)$ into a plain difference $(x+9)-9=x$, which cancels the troublesome $x$ in the denominator. Now substitute: $\frac{1}{\sqrt{9}+3}=\frac{1}{6}\approx 0.1667$.

• From the left ($x<0$): $s=x+1\to 0+1=1$.
• From the right ($x\ge 0$): $s=x-1\to 0-1=-1$.
• Left ($1$) $\ne$ right ($-1$) → the two-sided limit does not exist. This is a jump discontinuity of size $2$.

Goal: for any tolerance $\epsilon >0$ on the output, find a closeness $\delta >0$ on the input so that $0<|x-4|<\delta \Rightarrow |f(x)-7|<\epsilon$.

Step 1 — rewrite the output-gap using the input-gap. Why: $\delta$ controls $|x-4|$, so we want the output-gap expressed in that quantity. $|(3x-5)-7|=|3x-12|=3|x-4|.$ Step 2 — demand it be $<\epsilon$: $3|x-4|<\epsilon ⟺|x-4|<\epsilon /3$. Step 3 — choose $\boxed{\delta =\epsilon /3}$. Then $|x-4|<\delta \Rightarrow 3|x-4|<3\delta =\epsilon$. ✅ The limit is genuinely $7$, built from the definition rather than guessed.

Continuity at $3$ needs ${\lim }_{x\to 3}h=h(3)=k$. Compute the limit (it's $0/0$, so simplify): $\frac{x^2-9}{x-3}=\frac{(x-3)(x+3)}{x-3}=x+3$ for $x\ne 3$, so ${\lim }_{x\to 3}h=3+3=6$. Set $k=\lim =6$. We just plugged the hole with the limit value — condition 3 (heading $=$ arrival) now holds.

Form check: at $x=0$, $\sin 0/0=0/0$ → indeterminate, so we may not just substitute.

(i) Small-angle picture (see figure): for a tiny angle $x$ (in radians), the arc length and the vertical $\sin x$ nearly coincide, so $\sin x\approx x$ and the ratio $\to 1$.

(ii) L'Hôpital — why valid: the form is $0/0$ and both $\sin x$ and $x$ are differentiable near $0$, which are exactly L'Hôpital's preconditions. Differentiate top and bottom separately: ${\lim }_{x\to 0}\frac{\sin x}{x}={\lim }_{x\to 0}\frac{\cos x}{1}=\cos 0=1.$ Both routes agree: the limit is $1$.

Rule (Continuity of composite functions): if $u$ is continuous at $x$ and $\sigma$ is continuous at $u(x)$, then $\sigma \circ u$ is continuous at $x$.

• $u(x)=x^2$ is a polynomial → continuous everywhere.
• $\sigma (z)=1/(1+e^{-z})$: the denominator $1+e^{-z}$ is always $>1>0$ (an exponential is never negative), so we never divide by zero → $\sigma$ is continuous everywhere.
• Compose: continuous $\circ$ continuous $\Rightarrow$ $\sigma (x^2)$ is continuous for all real $x$. This chaining is exactly why a stack of continuous layers in a network stays continuous — the backbone of backprop.

At $x=0$: $\sigma (0^2)=\sigma (0)=\frac{1}{1+e^0}=\frac{1}{2}=0.5$.

(a) Continuity at $0$ — check all three:

1. $\mathrm{ReLU}(0)=\max (0,0)=0$ → exists.
2. Left: for $x<0$, value $=0\to 0$. Right: for $x>0$, value $=x\to 0$. Both sides head to $0$, so ${\lim }_{x\to 0}\mathrm{ReLU}=0$ → exists.
3. Limit $=0=\mathrm{ReLU}(0)$ → heading $=$ arrival. ✅ Continuous.

(b) Differentiability at $0$ — the derivative is the limit of the slope $\frac{\mathrm{ReLU}(0+t)-\mathrm{ReLU}(0)}{t}$.

• From the left ($t<0$): $\frac{0-0}{t}=0$ → slope $\to 0$.
• From the right ($t>0$): $\frac{t-0}{t}=1$ → slope $\to 1$. Left slope $0\ne 1$ right slope → no unique tangent, so not differentiable at $0$. Geometrically, a corner (see figure).

(c) At the single kink we pick any value in $[0,1]$ (a subgradient, commonly $0$); everywhere else ReLU is perfectly smooth, so descent has usable slopes almost everywhere and trains fine.

(a) Left limit: $x^2\to 1$. Right value/limit: $x^2+2\to 3$. Since $1\ne 3$, there is a jump of size $2$ → not continuous at $1$. (b) To glue the arrival at $1$, the right branch must approach the same $1$ from the right. Replace the constant: use $\ell (x)=x^2$ for $x\ge 1$ (drop the $+2$), giving right limit $1=$ left limit → continuous. (Any right branch with limit $1$ at $x=1$ works; $x^2$ is the minimal edit.) (c) A jump is a vertical cliff: near $x=1$ the "slope" is infinite/undefined, so the descent ball can't roll across smoothly — steps overshoot or diverge. Gradient descent needs the loss continuous (and mostly differentiable) so slopes are finite and meaningful — see Gradient Descent and Derivatives and gradients.

L1 Recognition — vertical line test & discontinuity types by inspection. L2 Application — kill $0/0$ by factoring / rationalizing; check both sides for jumps. L3 Analysis — prove limits with $\delta =\epsilon /(\text{slope})$; patch holes with the limit value. L4 Synthesis — pick the right tool (L'Hôpital only for $0/0,\infty /\infty$); compose continuity. L5 Mastery — continuous ⇏ differentiable; jumps (not kinks) break descent.