The best point on a curve
Tonight's idea
You want the largest \(f(x, y)\), but \((x, y)\) must stay on a curve \(g(x, y) = c\), the constraint. Draw the level curves of \(f\), where \(f\) is constant, like contours on a map, and walk along the constraint. Wherever a level curve crosses it you can keep walking to a higher level; you can only be stuck where one just touches.
Touching curves share a tangent, so they share a normal. The normal to a level curve of \(f\) is its gradient
\[\nabla f = \left(\frac{\partial f}{\partial x},\ \frac{\partial f}{\partial y}\right),\]
where \(\partial f / \partial x\) means: differentiate with respect to \(x\), treating \(y\) as a constant. (A small step \((dx, dy)\) along a level curve changes \(f\) by \(\frac{\partial f}{\partial x}dx + \frac{\partial f}{\partial y}dy = 0\), so the step is perpendicular to \(\nabla f\); see also gradient descent.) Likewise \(\nabla g\) is normal to the constraint. At the best point the two line up:
At a constrained maximum or minimum where \(\nabla g \ne 0\): \(\nabla f = \lambda\, \nabla g\) for some number \(\lambda\), together with \(g(x, y) = c\).
One equation per coordinate plus the constraint, and as many unknowns. The rule finds candidates, maxima and minima alike; you decide which is which. It works in any number of variables, where it earns its keep. Points where \(\nabla g = 0\) must be checked separately: with the constraint \(y^2 = 0\), the best value of \(y - x^2\) is at \((0, 0)\), but there \(\nabla g = (0, 0)\) and no \(\lambda\) works. Writing the same line as \(y = 0\) fixes it.
Watch one: the largest \(x + y\) on the unit circle
\(f = x + y\), constraint \(g = x^2 + y^2 = 1\). Then \(\nabla f = (1, 1)\) and \(\nabla g = (2x, 2y)\). Parallel means \(1 = 2\lambda x\) and \(1 = 2\lambda y\), so \(x = y\). With \(x^2 + y^2 = 1\) that gives \(x = y = \pm\tfrac{1}{\sqrt2}\). The plus sign is the maximum, \(f = \sqrt2\); the minus sign is the minimum, \(-\sqrt2\). In the figure, \(x + y = \sqrt2\) touches the circle at the top. Lines with \(|k| \lt \sqrt2\) cut it, \(k = -\sqrt2\) touches it at the bottom (the minimum), and \(|k| \gt \sqrt2\) misses it.
Your turn
- Maximise \(xy\) subject to \(x + y = 10\) with \(x, y > 0\). Write down the three equations, solve them, and keep the value of \(\lambda\) for the stretch. Check by substituting \(y = 10 - x\).
- Find the point on the line \(x + 2y = 5\) closest to the origin by minimising \(x^2 + y^2\). Give the point and the distance, and confirm the distance with the formula for the distance from a point to a line.
- Find the largest and smallest values of \(f = x^2 y\) on the circle \(x^2 + y^2 = 3\). The first gradient equation is \(2xy = 2\lambda x\); do not divide by \(x\) without asking whether \(x\) could be zero.
- A box sits inside a sphere of radius \(R\) with its eight corners on the sphere. With corner \((x, y, z)\) in the positive octant, the volume is \(8xyz\) and the constraint is \(x^2 + y^2 + z^2 = R^2\). Find the largest box, its side, and the fraction of the sphere it fills.
Stretch: what \(\lambda\) means
In exercise 1 you found \(\lambda = 5\). Now solve the same problem with the constraint \(x + y = c\) for general \(c\), call the maximum \(M(c)\), and compute \(\dfrac{dM}{dc}\) at \(c = 10\). What is \(\lambda\) telling you? Economists call it a shadow price.
Answers
- \(\nabla(xy) = (y, x)\) and \(\nabla(x + y) = (1, 1)\), so \(y = \lambda\), \(x = \lambda\), \(x + y = 10\): \(x = y = 5\), \(\lambda = 5\), maximum \(25\). Check: \(x(10 - x) = -(x - 5)^2 + 25\).
- \((2x, 2y) = \lambda(1, 2)\) gives \(y = 2x\); with \(x + 2y = 5\), \(x = 1\), \(y = 2\). Distance \(\sqrt{1 + 4} = \sqrt5\). Formula: \(\dfrac{\lvert 0 + 0 - 5 \rvert}{\sqrt{1^2 + 2^2}} = \sqrt5\).
- Equations: \(2xy = 2\lambda x\), \(x^2 = 2\lambda y\), \(x^2 + y^2 = 3\). If \(x \ne 0\), then \(\lambda = y\), so \(x^2 = 2y^2\) and \(3y^2 = 3\): \(y = \pm 1\), \(x = \pm\sqrt2\), giving \(f = 2\) (when \(y = 1\)) and \(f = -2\) (when \(y = -1\)). If \(x = 0\), then \(y = \pm\sqrt3\) and \(f = 0\). Largest \(2\), smallest \(-2\).
- \(\nabla(8xyz) = (8yz, 8xz, 8xy) = \lambda(2x, 2y, 2z)\). Multiply the three equations by \(x\), \(y\), \(z\) respectively: each left side becomes \(8xyz\), so \(2\lambda x^2 = 2\lambda y^2 = 2\lambda z^2\), and \(\lambda \ne 0\) (else the volume would be zero), so \(x = y = z = R/\sqrt3\). A cube of side \(2R/\sqrt3 \approx 1.155R\), volume \(\dfrac{8R^3}{3\sqrt3} \approx 1.540R^3\). The sphere has volume \(\tfrac43\pi R^3\), so the cube fills \(\dfrac{2}{\pi\sqrt3} \approx 36.8\%\) of it.
Stretch. With \(x + y = c\) the equations give \(x = y = c/2\), so \(M(c) = c^2/4\) and \(M'(c) = c/2\), which is \(5\) at \(c = 10\): equal to \(\lambda\). The multiplier is the rate at which the best achievable value improves per unit of extra constraint. Loosen the budget by a small \(h\) and the optimum rises by about \(\lambda h\).