Level 2 — RecallCalculus & Optimization Basics

Calculus & Optimization Basics

⏱ 30 minutes40 marksprintable — key stays hidden on paper

Chapter: Calculus & Optimization Basics Difficulty Level: 2 — Recall / Standard textbook problems Time Limit: 30 minutes Total Marks: 40

Instructions

Answer all questions.
Show working where derivations are requested.
Use $...$ notation for mathematical expressions.

Q1. Define continuity of a function $f(x)$ at a point $x = a$ using the limit condition. [3 marks]

Q2. Differentiate the following, stating the rule used in each case: [4 marks]

(a) $f(x) = x^2 \sin x$
(b) $g(x) = \dfrac{x+1}{x-1}$

Q3. For $f(x,y) = x^2 y + 3xy^3$ , compute the partial derivatives $\dfrac{\partial f}{\partial x}$ and $\dfrac{\partial f}{\partial y}$ . [4 marks]

Q4. For $f(x,y) = x^2 + y^2$ , compute the gradient $\nabla f$ at the point $(1, 2)$ , and find the directional derivative at that point in the direction of the unit vector $\mathbf{u} = \left(\tfrac{3}{5}, \tfrac{4}{5}\right)$ . [4 marks]

Q5. State the definition of the Jacobian matrix for a vector-valued function $\mathbf{F}: \mathbb{R}^n \to \mathbb{R}^m$ . Then compute the Jacobian of $\mathbf{F}(x,y) = (x^2 y,\; x + y)$ . [4 marks]

Q6. For $f(x,y) = x^3 + y^3 - 3xy$ , compute the Hessian matrix $H$ . [4 marks]

Q7. Write the second-order Taylor series approximation of $f(x) = e^x$ about $x = 0$ , and use it to estimate $e^{0.1}$ . [4 marks]

Q8. (a) State the definition of a convex function on an interval. (b) State whether $f(x) = x^2$ and $g(x) = x^3$ are convex on $\mathbb{R}$ , giving a brief reason. [4 marks]

Q9. For $f(x) = x^3 - 3x$ , find all critical points and classify each as a local minimum, local maximum, or saddle/inflection using the second derivative test. [5 marks]

Q10. State the gradient descent update rule for minimizing $f(\theta)$ , and briefly explain the effect of choosing a learning rate that is (i) too large and (ii) too small. [4 marks]

END OF PAPER

Answer keyMark scheme & solutions

Q1. (3 marks) $f$ is continuous at $x=a$ if all three hold:

$f(a)$ is defined [1]
$\lim_{x \to a} f(x)$ exists [1]
$\lim_{x \to a} f(x) = f(a)$ [1]

Why: Continuity requires the function value and the limit to coincide with no gap/jump.

Q2. (4 marks)

(a) Product rule: $f'(x) = 2x \sin x + x^2 \cos x$ [2] (1 for rule, 1 for correct terms)
(b) Quotient rule: $g'(x) = \dfrac{(1)(x-1) - (x+1)(1)}{(x-1)^2} = \dfrac{-2}{(x-1)^2}$ [2]

Q3. (4 marks) $\dfrac{\partial f}{\partial x} = 2xy + 3y^3$ [2] $\dfrac{\partial f}{\partial y} = x^2 + 9xy^2$ [2]

Why: Treat the other variable as constant during each partial differentiation.

Q4. (4 marks) $\nabla f = (2x, 2y)$ , so at $(1,2)$ : $\nabla f = (2, 4)$ [2] Directional derivative $= \nabla f \cdot \mathbf{u} = 2\cdot\tfrac{3}{5} + 4\cdot\tfrac{4}{5} = \tfrac{6}{5} + \tfrac{16}{5} = \tfrac{22}{5} = 4.4$ [2]

Why: The directional derivative is the dot product of the gradient with the unit direction vector.

Q5. (4 marks) Definition: The Jacobian is the $m \times n$ matrix of first-order partial derivatives, $J_{ij} = \dfrac{\partial F_i}{\partial x_j}$ . [2] For $\mathbf{F}(x,y) = (x^2 y,\; x+y)$ : $J = \begin{pmatrix} 2xy & x^2 \\ 1 & 1 \end{pmatrix}$ [2]

Q6. (4 marks) First partials: $f_x = 3x^2 - 3y$ , $f_y = 3y^2 - 3x$ . [1] Second partials: $f_{xx} = 6x$ , $f_{yy} = 6y$ , $f_{xy} = f_{yx} = -3$ . [2] $H = \begin{pmatrix} 6x & -3 \\ -3 & 6y \end{pmatrix}$ [1]

Q7. (4 marks) Taylor: $f(x) \approx f(0) + f'(0)x + \tfrac{1}{2}f''(0)x^2 = 1 + x + \tfrac{x^2}{2}$ [2] At $x=0.1$ : $1 + 0.1 + 0.005 = 1.105$ [2]

Why: All derivatives of $e^x$ equal $e^0 = 1$ at $x=0$ . (True value $\approx 1.10517$ .)

Q8. (4 marks) (a) $f$ is convex on an interval if for all $x_1, x_2$ and $\lambda \in [0,1]$ : $f(\lambda x_1 + (1-\lambda)x_2) \le \lambda f(x_1) + (1-\lambda) f(x_2)$ [2] (Equivalently, $f'' \ge 0$ if twice differentiable.) (b) $f(x)=x^2$ : convex, since $f''=2 \ge 0$ everywhere. [1] $g(x)=x^3$ : not convex on $\mathbb{R}$ , since $g''=6x < 0$ for $x<0$ . [1]

Q9. (5 marks) $f'(x) = 3x^2 - 3 = 0 \Rightarrow x^2 = 1 \Rightarrow x = \pm 1$ [2] $f''(x) = 6x$ . [1] At $x=1$ : $f''=6>0 \Rightarrow$ local minimum (value $f(1)=-2$ ). [1] At $x=-1$ : $f''=-6<0 \Rightarrow$ local maximum (value $f(-1)=2$ ). [1]

Q10. (4 marks) Update rule: $\theta_{t+1} = \theta_t - \eta \nabla f(\theta_t)$ , where $\eta$ is the learning rate. [2] (i) Too large: may overshoot the minimum, causing oscillation or divergence. [1] (ii) Too small: very slow convergence, may take many iterations / get stuck near flat regions. [1]

[
  {"claim":"Q4 directional derivative = 4.4","code":"x,y=symbols('x y'); f=x**2+y**2; grad=Matrix([diff(f,x),diff(f,y)]).subs({x:1,y:2}); u=Matrix([Rational(3,5),Rational(4,5)]); result=(grad.dot(u)==Rational(22,5))"},
  {"claim":"Q2b derivative = -2/(x-1)**2","code":"x=symbols('x'); g=(x+1)/(x-1); result=simplify(diff(g,x)-(-2/(x-1)**2))==0"},
  {"claim":"Q7 Taylor estimate of e^0.1 = 1.105","code":"x=symbols('x'); T=1+x+x**2/2; result=T.subs(x,Rational(1,10))==Rational(1105,1000)"},
  {"claim":"Q9 f(1)=-2 local min, f(-1)=2 local max","code":"x=symbols('x'); f=x**3-3*x; f2=diff(f,x,2); result=(f.subs(x,1)==-2) and (f.subs(x,-1)==2) and (f2.subs(x,1)>0) and (f2.subs(x,-1)<0)"}
]