Definition 5.5.1.
A critical point of a function which is not a local minimum or local maximum is called a saddle point.
Section 5.5 Hessians and the second derivative test
In Section 3.6, critical points are found by solving
\begin{equation*}
\nabla f(\mathbf a)=\mathbf 0.
\end{equation*}
This condition removes the linear term from the local approximation, but it does not determine whether \(\mathbf a\) is a local minimum, a local maximum, or a saddle point. The next term is quadratic. In Section 5.4, eigenvalues determined the sign of a quadratic form. We now apply that linear algebra to the Hessian.
Subsection Local behavior and quadratic approximation
A critical point need not be a local extremum. We first name the third possibility, then review the one-variable Taylor polynomial before writing its multivariable analogue.
The 3D graph of \(z=y^2-x^2\) is shown with the saddle point at the origin. One cross-section curves upward like a valley, while the perpendicular cross-section curves downward like a ridge. The surface is colored blue on one side and red on the other, making the opposing curvatures visible.
CC BY-NC-SA 4.0; source: OpenStax Figure 4.48.The second-order approximation explains this local behavior. We begin with the one-variable case.
For a review, see Section 6.3: Taylor and Maclaurin Series in Strang and Herman’s Calculus Volume 2.
1
openstax.org/books/calculus-volume-2/pages/6-3-taylor-and-maclaurin-seriesRecall that if a single-variable function \(f\) has \(n\) derivatives at \(x = a\text{,}\) then the \(n\)th Taylor polynomial for \(f\) at \(a\) is the function
\begin{align*}
p_n(x) \amp = f(a) + f'(a)(x - a) + \frac{f''(a)}{2!}(x - a)^2 + \cdots + \frac{f^{(n)}(a)}{n!}(x - a)^n\\
\amp = \sum_{k=0}^n \frac{f^{(k)}(a)}{k!}(x - a)^k\text{.}
\end{align*}
In particular, for \(n = 1\) we obtain the linear approximation
\begin{equation*}
p_1(x) = f(a) + f'(a) (x - a)
\end{equation*}
of \(f\text{,}\) and for \(n = 2\) we obtain the quadratic approximation
\begin{equation*}
p_2(x) = f(a) + f'(a) (x - a) + f''(a) \frac{(x-a)^2}{2}\text{.}
\end{equation*}
The Taylor polynomial \(p_n(x)\) approximates the function \(f(x)\) near \(x = a\text{,}\) with the approximation improving as \(n\) increases.
Activity 5.5.3. Taylor polynomials for a cube-root function (U5-LO5).
Consider the function \(f(x) = \sqrt[3]{x}\text{.}\)
Solution.
For \(f(x) = \sqrt[3]{x}\text{,}\) the values of the function and its first two derivatives at \(x = 8\) are as follows:
\begin{align*}
f(x) \amp = \sqrt[3]{x} \amp f(8) \amp = 2\\
f'(x) \amp = \frac{1}{3x^{2/3}} \amp f'(8) \amp = \frac{1}{12}\\
f''(x) \amp = \frac{-2}{9x^{5/3}} \amp f''(8) \amp = -\frac{1}{144}
\end{align*}
Thus, the first and second Taylor polynomials at \(x = 8\) are given by:
\begin{align*}
p_1(x) \amp = f(8) + f'(8)(x - 8)\\
\amp = 2 + \frac{1}{12}(x - 8)\\
p_2(x) \amp = f(8) + f'(8)(x - 8) + \frac{f''(8)}{2!}(x - 8)^2\\
\amp = 2 + \frac{1}{12}(x - 8) - \frac{1}{288}(x - 8)^2
\end{align*}
Activity 5.5.4. Using Taylor polynomials (U5-LO5).
Use these two polynomials to estimate \(\sqrt[3]{11}\text{.}\)
Solution.
Using the first Taylor polynomial at \(x = 8\text{,}\) we can estimate:
\begin{align*}
\sqrt[3]{11} \approx p_1(11) \amp = 2 + \frac{1}{12}(11 - 8)\\
\amp = 2.25
\end{align*}
Using the second Taylor polynomial at \(x = 8\text{,}\) we obtain:
\begin{align*}
\sqrt[3]{11} \approx p_2(11) \amp = 2 + \frac{1}{12}(11 - 8) - \frac{1}{288}(11 - 8)^2\\
\amp = 2 + \frac{1}{4} - \frac{1}{32}\\
\amp = 2.21875
\end{align*}
If \(a\) is a critical point of \(f\text{,}\) then the quadratic approximation of \(f\) is given by the expression
\begin{equation*}
f(a) + f''(a) \frac{(x-a)^2}{2}
\end{equation*}
If \(f''(a) \gt 0\text{,}\) then the quadratic approximation of \(f\) is an upward pointing parabola with a vertex at \((a,f(a))\text{.}\) This approximation has a minimum at this vertex, explaining \(a\) is a local minimum of the function \(f\) (since \(f\) is closely approximated by its quadratic approximation). Similarly, if \(f''(a) \lt 0\text{,}\) then the quadratic approximation is a downward pointing parabola with a maximum at \((a,f(a))\text{,}\) explaining why \(a\) is a local maximum of the function \(f\text{.}\)
In several variables, the gradient replaces the first derivative and the Hessian replaces the second derivative.
Definition 5.5.5. Quadratic approximation.
The quadratic approximation \(Q(\mathbf{x})\) of a scalar-valued function \(f\) near a point \(\mathbf{a}\) is the function defined by
\begin{equation*}
Q(\mathbf{x})
=
f(\mathbf{a})
+
\nabla f(\mathbf{a})\cdot(\mathbf{x}-\mathbf{a})
+
\frac12(\mathbf{x}-\mathbf{a})^T H_f(\mathbf{a})(\mathbf{x}-\mathbf{a})\text{.}
\end{equation*}
Remark 5.5.6.
The quadratic approximation of a function \(f\) refines the linear approximation via the gradient that we previously encountered. In particular, if the second derivatives of \(f\) are continuous, then
\begin{equation*}
\lim_{\mathbf{x} \to \mathbf{a}} \frac{f(\mathbf{x}) - Q(\mathbf{x})}{\| \mathbf{x} - \mathbf{a} \|^2} = 0\text{.}
\end{equation*}
If \(\mathbf{x}\) is close to \(\mathbf{a}\text{,}\) then \(\| \mathbf{x} - \mathbf{a} \|^2\) is much smaller than \(\| \mathbf{x} - \mathbf{a} \|\text{,}\) so that the quadratic approximation \(Q(\mathbf{x})\) to \(f\) is in general a much better approximation than the linear approximation \(L(\mathbf{x})\text{.}\)
Activity 5.5.7. Quadratic approximation of a logarithmic function (U5-LO5, U3-LO3).
Compute the gradient and the Hessian of the function \(f(x, y) = \ln(xy - 1)\) at the point \((x, y) = (1, 2)\text{,}\) use these values to determine the quadratic approximation to \(f\) at the point \((1,2)\text{,}\) and then use the quadratic approximation to estimate the value of \(f\) at \((1.2, 1.8)\text{.}\)
Solution.
We calculate that
\begin{equation*}
f_x = \frac{y}{xy - 1} \quad\text{and}\quad f_y = \frac{x}{xy - 1}\text{.}
\end{equation*}
So \((\nabla f)(1,2) = (2,1)\text{.}\) Next, we compute that
\begin{align*}
f_{xx} \amp = \frac{-y^2}{(xy - 1)^2} \amp
f_{yy} \amp = \frac{-x^2}{(xy - 1)^2} \amp
f_{xy} \amp = f_{yx} = \frac{-1}{(xy - 1)^2}\text{.}
\end{align*}
Evaluating at \((1, 2)\text{,}\) we find that
\begin{equation*}
H_f(1, 2) = \begin{bmatrix}
\frac{-4}{1} \amp \frac{-1}{1} \\
\frac{-1}{1} \amp \frac{-1}{1}
\end{bmatrix} = \begin{bmatrix}
-4 \amp -1 \\
-1 \amp -1
\end{bmatrix}\text{.}
\end{equation*}
Since
\begin{equation*}
f(1,2) = \ln(1) = 0\text{,}
\end{equation*}
using the Hessian and gradient, we find that the quadratic approximation of \(f\) near \((1,2)\) is given by
\begin{align*}
Q(x,y) \amp =
(2,1) \cdot ( x - 1, y - 2 ) + \frac{1}{2} \begin{bmatrix} x - 1 \amp y - 2 \end{bmatrix} \begin{bmatrix}
-4 \amp -1 \\
-1 \amp -1
\end{bmatrix} \begin{bmatrix} x - 1 \\ y - 2 \end{bmatrix}\\
\amp = 2(x - 1) + (y-2) - 2 (x - 1)^2 - (x-1)(y-2) - (1/2) (y-2)^2\text{.}
\end{align*}
So
\begin{align*}
Q(1.2,1.8) \amp = 2(0.2) + (-0.2) - 2 (0.2)^2 - (0.2)(-0.2) - (1/2) (-0.2)^2\\
\amp = 0.4 - 0.2 - 0.08 + 0.04 - 0.02\\
\amp = 0.14\text{.}
\end{align*}
The actual value is \(f(1.2,1.8)\approx0.1484\text{,}\) while the quadratic approximation is \(0.14\text{.}\) Thus the quadratic approximation is quite accurate. For comparison, the linear approximation \(L\) at \((1,2)\) is given by
\begin{equation*}
L(x,y) =2(0.2) + (-0.2) = 0.2\text{,}
\end{equation*}
which is far less accurate.
Activity 5.5.8. Quadratic approximation of a trigonometric function (U5-LO5, U3-LO3).
Let \(f(x, y) = \cos(x/y)\text{.}\) Use the gradient and Hessian to obtain a quadratic approximation of the function \(f(x, y)\) near \(\mathbf{a} = (0, 1)\text{.}\)
Solution.
We calculate that
\begin{equation*}
\nabla f(x,y) = \left( - \frac{\sin(x/y)}{y}, \frac{x \sin(x/y)}{y^2} \right)
\end{equation*}
In particular, \(\nabla f(0,1) = (0,0)\text{.}\) Next, we calculate that
\begin{align*}
\amp f_{xx} = -\frac{1}{y^2}\cos\left(\frac{x}{y}\right), \quad f_{yy} = \frac{x}{y^4}\left(2y\sin\left(\frac{x}{y}\right) + x\cos\left(\frac{x}{y}\right)\right)\\
\amp f_{xy} = f_{yx} = \frac{1}{y^3}\left(y\sin\left(\frac{x}{y}\right) + x\cos\left(\frac{x}{y}\right)\right)
\end{align*}
So
\begin{equation*}
H_f(0,1) = \begin{bmatrix} -1 \amp 0 \\ 0 \amp 0 \end{bmatrix}
\end{equation*}
Since \(f(0,1) = \cos(0) = 1\text{,}\) the quadratic approximation of \(f\) at \((0,1)\) is thus given by
\begin{equation*}
Q(x,y) = 1 - x^2/2\text{.}
\end{equation*}
Subsection Hessian eigenvalues and the general second derivative test
Suppose \(\mathbf a\) is a critical point. Then
\begin{equation*}
\nabla f(\mathbf a)=\mathbf 0,
\end{equation*}
so the quadratic approximation becomes
\begin{equation*}
f(\mathbf a+\mathbf h)\approx f(\mathbf a)+\frac12\mathbf h^T H_f(\mathbf a)\mathbf h.
\end{equation*}
Thus the local behavior is controlled by the quadratic form associated with the Hessian. Because the Hessian is symmetric, the spectral theorem gives an orthonormal basis of eigenvectors. The signs of the corresponding eigenvalues determine the sign of the quadratic term.
Theorem 5.5.9. Second derivative test.
Let \(f:\R^n\to\R\) be twice continuously differentiable near a critical point \(\mathbf a\text{,}\) and let
\begin{equation*}
H=H_f(\mathbf a).
\end{equation*}
-
If \(H\) has at least one strictly positive eigenvalue and at least one strictly negative eigenvalue, then \(\mathbf a\) is a saddle point.
In all other cases, the test is inconclusive.
Why is this true?.
Since \(\mathbf a\) is a critical point,
\begin{equation*}
f(\mathbf a+\mathbf h)-f(\mathbf a)=\frac12\mathbf h^TH\mathbf h+r(\mathbf h),
\end{equation*}
where
\begin{equation*}
\frac{r(\mathbf h)}{\|\mathbf h\|^2}\longrightarrow0\qquad\text{as }\mathbf h\longrightarrow\mathbf0.
\end{equation*}
Suppose first that every eigenvalue of \(H\) is positive, and let \(\mu>0\) be the smallest eigenvalue. By the quadratic-form diagonalization theorem,
\begin{equation*}
\mathbf h^TH\mathbf h\geq\mu\|\mathbf h\|^2.
\end{equation*}
For sufficiently small nonzero \(\mathbf h\text{,}\)
\begin{equation*}
|r(\mathbf h)|\leq\frac{\mu}{4}\|\mathbf h\|^2.
\end{equation*}
Therefore
\begin{equation*}
f(\mathbf a+\mathbf h)-f(\mathbf a)\geq\frac{\mu}{2}\|\mathbf h\|^2-\frac{\mu}{4}\|\mathbf h\|^2>0.
\end{equation*}
Hence \(\mathbf a\) is a strict local minimum. The negative-eigenvalue case follows by applying the same argument to \(-f\text{.}\)
Finally, suppose \(H\) has a positive eigenvalue \(\lambda_+\) with unit eigenvector \(\mathbf v_+\text{,}\) and a negative eigenvalue \(\lambda_-\) with unit eigenvector \(\mathbf v_-\text{.}\) Along the two lines through \(\mathbf a\text{,}\)
\begin{equation*}
f(\mathbf a+t\mathbf v_+)-f(\mathbf a)=\frac12\lambda_+t^2+o(t^2)>0
\end{equation*}
and
\begin{equation*}
f(\mathbf a+t\mathbf v_-)-f(\mathbf a)=\frac12\lambda_-t^2+o(t^2)\lt0
\end{equation*}
for all sufficiently small nonzero \(t\text{.}\) Nearby values therefore occur on both sides of \(f(\mathbf a)\text{,}\) so \(\mathbf a\) is a saddle point.
Warning 5.5.10.
A zero Hessian eigenvalue does not classify the critical point. It means the quadratic approximation has a flat direction, so higher-order terms may matter.
Activity 5.5.11. Predicting from Hessian eigenvalues (U5-LO5).
Suppose \(\mathbf{a}\) is a critical point of a scalar-valued function \(f\text{.}\) For each list of eigenvalues of \(H_f(\mathbf{a})\text{,}\) classify the critical point as a local minimum, local maximum, saddle point, or inconclusive.
-
\(\displaystyle 3,5\)
-
\(\displaystyle -2,-7\)
-
\(\displaystyle 4,-1\)
-
\(\displaystyle 0,2\)
Solution.
If the eigenvalues are \(3,5\text{,}\) then the Hessian is positive definite, so the critical point is a local minimum.
If the eigenvalues are \(-2,-7\text{,}\) then the Hessian is negative definite, so the critical point is a local maximum.
If the eigenvalues are \(4,-1\text{,}\) then the Hessian is indefinite, so the critical point is a saddle point.
If the eigenvalues are \(0,2\text{,}\) the test is inconclusive. The zero eigenvalue means the second-order approximation has a flat direction, so the second derivative test alone does not decide the local behavior.
Activity 5.5.12. Critical points of a quadratic in three variables (U5-LO5, U3-LO9).
Consider the function \(f(x,y,z) = x^2 + y^2 - xz\text{.}\) Find all critical points of \(f\) and determine the behavior of \(f\) near each critical point.
(a)
First find the critical points.
(b)
Now classify each critical point.
Solution.
The Hessian is the constant matrix
\begin{equation*}
H_f = \begin{bmatrix} 2 \amp 0 \amp -1 \\ 0 \amp 2 \amp 0 \\ -1 \amp 0 \amp 0 \end{bmatrix}\text{.}
\end{equation*}
The associated quadratic form is
\begin{align*}
\mathbf{h}^T H_f\mathbf{h}
\amp =
2h_1^2+2h_2^2-2h_1h_3.
\end{align*}
This quadratic form takes both positive and negative values. For example,
\begin{equation*}
\begin{bmatrix}0\\1\\0\end{bmatrix}^T
H_f
\begin{bmatrix}0\\1\\0\end{bmatrix}
=2>0,
\end{equation*}
but
\begin{equation*}
\begin{bmatrix}1\\0\\3\end{bmatrix}^T
H_f
\begin{bmatrix}1\\0\\3\end{bmatrix}
=
2-6=-4\lt 0.
\end{equation*}
Thus \(H_f\) is indefinite. By the second derivative test, the critical point \((0,0,0)\) is a saddle point.
Activity 5.5.13. Critical points of a cubic in three variables (U5-LO5, U3-LO9).
Classify the critical points of the function \(f(x,y,z) = xy + yz + xz + xyz\) using the second derivative test.
Solution.
We compute that
\begin{equation*}
\nabla f(x,y,z) =
\begin{bmatrix}
y + z + yz\\
x + z + xz\\
x + y + xy
\end{bmatrix}\text{.}
\end{equation*}
Notice that \(\nabla f(x,-1,z) = ( -1, x + z + xz, -1 )\) is non-vanishing, so that there are no critical points on the plane \(y = -1\text{.}\) Now suppose that \(\nabla f(x,y,z) = 0\) and \(y \neq -1\text{.}\) The third entry tells us that \(x = -y/(1 + y)\text{.}\) Substituting this into the second equation gives that \(-y/(1+y) + z - yz/(1+y) = 0\text{,}\) i.e., that \(z = y\text{.}\) Substituting this into the first equation gives that \(y^2 + 2y = 0\text{,}\) i.e., that \(y = 0\) or \(y = -2\text{.}\) Substituting back gives the two critical points of \(f\text{,}\) namely \((0,0,0)\) and \((-2,-2,-2)\text{.}\)
Next, we classify these critical points. We calculate that the Hessian of \(f\) is
\begin{equation*}
H_f(x,y,z) = \begin{bmatrix} 0 \amp 1 + z \amp 1 + y \\ 1 + z \amp 0 \amp 1 + x \\ 1 + y \amp 1 + x \amp 0 \end{bmatrix}\text{.}
\end{equation*}
In particular,
\begin{equation*}
H_f(0,0,0) = \begin{bmatrix} 0 \amp 1 \amp 1 \\ 1 \amp 0 \amp 1 \\ 1 \amp 1 \amp 0 \end{bmatrix}\text{,}
\end{equation*}
which has characteristic polynomial
\begin{align*}
\det \begin{bmatrix} \lambda \amp -1 \amp -1 \\ -1 \amp \lambda \amp -1 \\ -1 \amp -1 \amp \lambda \end{bmatrix} \amp = \lambda( \lambda^2 - 1) - (-1) (-\lambda - 1) + (-1) (1 + \lambda)\\
\amp = \lambda^3 - 3\lambda - 2\\
\amp = (\lambda + 1)(\lambda^2 - \lambda - 2)\\
\amp = (\lambda + 1)^2( \lambda - 2 )\text{.}
\end{align*}
So \(H_f(0,0,0)\) has eigenvalues \(-1\) and \(2\text{,}\) and so is indefinite, so that \(f\) has a saddle point at \((0,0,0)\text{.}\) We have
\begin{equation*}
H_f(-2,-2,-2) = \begin{bmatrix} 0 \amp -1 \amp -1 \\ -1 \amp 0 \amp -1 \\ -1 \amp -1 \amp 0 \end{bmatrix}\text{,}
\end{equation*}
which has characteristic polynomial
\begin{align*}
\det \begin{bmatrix} \lambda \amp 1 \amp 1 \\ 1 \amp \lambda \amp 1 \\ 1 \amp 1 \amp \lambda \end{bmatrix} \amp = \lambda( \lambda^2 - 1) - (\lambda - 1) + (1 - \lambda)\\
\amp = \lambda^3 - 3 \lambda + 2\\
\amp = (\lambda - 1)(\lambda^2 + \lambda - 2)\\
\amp = (\lambda - 1)^2(\lambda + 2)\text{,}
\end{align*}
so \(H_f(-2,-2,-2)\) has eigenvalues \(1\) and \(-2\text{,}\) and is thus an indefinite matrix. So \((-2,-2,-2)\) is also a saddle point of \(f\text{.}\)
Subsection The two-variable second derivative test
For a function of two variables, the Hessian at a critical point has the form
\begin{equation*}
H_f(a,b)=\begin{bmatrix}f_{xx}(a,b)\amp f_{xy}(a,b)\\f_{xy}(a,b)\amp f_{yy}(a,b)\end{bmatrix}.
\end{equation*}
Its determinant is
\begin{equation*}
D(a,b)=f_{xx}(a,b)f_{yy}(a,b)-f_{xy}(a,b)^2.
\end{equation*}
Since the determinant is the product of the two eigenvalues, \(D\) detects whether their signs agree or disagree. The standard two-variable test is therefore a shortcut for the general Hessian test.
Activity 5.5.14. The \(2\times2\) definiteness criterion (U5-LO4).
Show that a matrix
\begin{equation*}
A = \begin{bmatrix} a \amp b \\ b \amp d \end{bmatrix}
\end{equation*}
is positive definite if and only if \(ad - b^2\) is positive and \(a > 0\text{,}\) negative definite if and only if \(ad - b^2\) is positive and \(a \lt 0\text{,}\) and indefinite if and only if \(ad - b^2\) is negative. It follows from this that the two-variable test is a special case of the general second derivative test.
Solution.
Let \(\lambda_1\) and \(\lambda_2\) be the eigenvalues of \(A\text{.}\) A direct calculation gives
\begin{align*}
c_A(\lambda)
\amp=\det(\lambda I_2-A)\\
\amp=(\lambda-a)(\lambda-d)-b^2\\
\amp=\lambda^2-(a+d)\lambda+(ad-b^2).
\end{align*}
On the other hand, the characteristic polynomial is monic and has roots \(\lambda_1\) and \(\lambda_2\text{,}\) so
\begin{align*}
c_A(\lambda)
\amp=(\lambda-\lambda_1)(\lambda-\lambda_2)\\
\amp=\lambda^2-(\lambda_1+\lambda_2)\lambda+\lambda_1\lambda_2.
\end{align*}
Comparing coefficients gives
\begin{equation*}
a+d=\lambda_1+\lambda_2,
\qquad
ad-b^2=\lambda_1\lambda_2.
\end{equation*}
Suppose first that \(A\) is positive definite. Then \(\lambda_1,\lambda_2>0\text{,}\) so \(ad-b^2=\lambda_1\lambda_2>0\text{.}\) Also, by Theorem 5.4.13,
\begin{equation*}
a=\mathbf{e}_1^T A\mathbf{e}_1>0.
\end{equation*}
Conversely, suppose \(ad-b^2>0\) and \(a>0\text{.}\) Then \(ad>b^2\geq 0\text{,}\) so \(d>0\) as well. Hence
\begin{equation*}
\lambda_1+\lambda_2=a+d>0,
\qquad
\lambda_1\lambda_2=ad-b^2>0.
\end{equation*}
The positive product means that the two eigenvalues have the same sign, and the positive sum means that this sign is positive. Thus both eigenvalues are positive, so \(A\) is positive definite.
The negative-definite case is analogous. If \(A\) is negative definite, then \(\lambda_1,\lambda_2\lt 0\text{,}\) so \(ad-b^2>0\text{,}\) and Theorem 5.4.13 gives \(a=\mathbf{e}_1^T A\mathbf{e}_1\lt 0\text{.}\) Conversely, if \(ad-b^2>0\) and \(a\lt 0\text{,}\) then \(d\lt 0\text{.}\) Thus the eigenvalues have a positive product and a negative sum, so both eigenvalues are negative and \(A\) is negative definite.
Finally, \(A\) is indefinite if and only if its two eigenvalues have opposite signs. This holds if and only if \(\lambda_1\lambda_2\lt 0\text{,}\) which, by the coefficient comparison above, holds if and only if \(ad-b^2\lt 0\text{.}\)
Theorem 5.5.15. Two-variable second derivative test.
Let \(f:\R^2\to\R\) be twice continuously differentiable near a critical point \((a,b)\text{.}\) Define
\begin{equation*}
D(a,b)=f_{xx}(a,b)f_{yy}(a,b)-f_{xy}(a,b)^2.
\end{equation*}
-
If \(D(a,b)=0\text{,}\) the test is inconclusive.
Why is this true?.
Activity 5.5.16. Classifying a critical point of a quadratic (U5-LO5).
Use the two-variable second derivative test to determine if the critical point \((1,-2)\) of \(f(x,y) = x^2 + y^2 - 2x + 4y\) found in the earlier activity is a local maximum, a local minimum, or a saddle point.
Solution.
We apply the second derivative test. We compute the second-order partial derivatives of \(f\text{.}\) We calculate that
\begin{equation*}
f_{xx}(x,y) = 2 \quad f_{yy} = 2 \quad\text{and} \quad f_{xy}(x,y) = 0\text{.}
\end{equation*}
So the discriminant is the constant function
\begin{equation*}
D = f_{xx} f_{yy} - f_{xy}^2 = (2)(2) - (0)^2 = 4\text{.}
\end{equation*}
Since the discriminant is positive, by the second derivative test, the critical point \((1,-2)\) is a local minimum.
Activity 5.5.17. Classifying the origin for a quadratic (U5-LO5).
Use the two-variable second derivative test to analyze the critical point \((0,0)\) of \(f(x,y) = x^2 - y^2\) found in the earlier activity.
Solution.
We need to compute the second-order partial derivatives:
\begin{equation*}
f_{xx} = 2 \quad f_{yy} = -2 \quad \text{and} \quad f_{xy} = 0\text{.}
\end{equation*}
So the discriminant is also the constant function
\begin{equation*}
D = f_{xx} f_{yy} - f_{xy}^2 = (2)(-2) - (0)^2 = -4\text{.}
\end{equation*}
Since the discriminant is negative, the second derivative test tells us that \((0,0)\) is a saddle point of \(f\text{.}\)
Activity 5.5.18. Critical points of a quadratic with a cross term (U5-LO5, U3-LO9).
Find all local maxima, local minima, and saddle points for the function
\begin{equation*}
f(x,y) = x^2 + xy + y^2 - 6x + 6\text{.}
\end{equation*}
Solution.
We begin by finding the critical points of \(f\text{.}\) We calculate that
\begin{equation*}
f_x(x,y)=2x+y-6
\qquad\text{and}\qquad
f_y(x,y)=x+2y\text{.}
\end{equation*}
So the critical points satisfy
\begin{equation*}
2x+y-6=0
\qquad\text{and}\qquad
x+2y=0.
\end{equation*}
The second equation gives \(x=-2y\text{.}\) Substituting into the first equation gives
\begin{equation*}
2(-2y)+y-6=0,
\end{equation*}
so \(-3y=6\text{,}\) and therefore \(y=-2\text{.}\) Thus \(x=4\text{,}\) and the only critical point is \((4,-2)\text{.}\)
To classify this critical point, we apply the second derivative test. We calculate that
\begin{equation*}
f_{xx}(x,y)=2,\qquad f_{yy}(x,y)=2,\qquad f_{xy}(x,y)=1\text{.}
\end{equation*}
So the discriminant is the constant
\begin{equation*}
D = f_{xx} f_{yy} - f_{xy}^2 = (2)(2) - (1)^2 = 3\text{.}
\end{equation*}
Since \(D>0\) and \(f_{xx}>0\text{,}\) the critical point \((4,-2)\) is a local minimum.
Activity 5.5.19. Revisiting the gradient-descent quadratic (U5-LO5, U3-LO9).
Consider the function
\begin{equation*}
f(x,y)=x^2-3xy+3y^2+5y+2x.
\end{equation*}
Find all critical points of \(f\) and classify each as a local minimum, a local maximum, or a saddle point.
Solution.
We find the critical points by solving \(\nabla f(x,y)=\mathbf{0}\text{.}\) Since
\begin{equation*}
f_x(x,y)=2x-3y+2
\qquad\text{and}\qquad
f_y(x,y)=-3x+6y+5,
\end{equation*}
the critical points satisfy
\begin{equation*}
2x-3y+2=0
\qquad\text{and}\qquad
-3x+6y+5=0.
\end{equation*}
The first equation gives \(x=(3y-2)/2\text{.}\) Substituting this expression into the second equation gives
\begin{align*}
-3\left(\frac{3y-2}{2}\right)+6y+5 &=0,\\
\frac{3y}{2} &=-8,\\
y &=-\frac{16}{3}.
\end{align*}
It follows that
\begin{equation*}
x=\frac{3(-16/3)-2}{2}=-9,
\end{equation*}
so the only critical point is \((-9,-16/3)\text{.}\)
The Hessian matrix is the constant matrix
\begin{equation*}
H_f(x,y)=
\begin{bmatrix}
2 & -3 \\
-3 & 6
\end{bmatrix}.
\end{equation*}
Thus the discriminant is
\begin{equation*}
D=f_{xx}f_{yy}-f_{xy}^2=(2)(6)-(-3)^2=3>0.
\end{equation*}
Since \(D>0\) and \(f_{xx}=2>0\text{,}\) the second derivative test tells us that \((-9,-16/3)\) is a local minimum of \(f\text{.}\)
In the previous examples, the discriminant was constant because \(f\) was a quadratic function. Let us consider an example where the discriminant is nonconstant.
Activity 5.5.20. Critical points of a cubic in two variables (U5-LO5, U3-LO9).
Find all local maxima, local minima, and saddle points for the function \(f(x,y) = x^3 + y^2 + 2xy - 4x - 3y + 5\text{.}\)
Solution.
We begin by finding the critical point. We calculate that
\begin{equation*}
f_x(x,y) = 3x^2 + 2y - 4 \quad\text{and}\quad f_y(x,y) = 2y + 2x - 3\text{.}
\end{equation*}
Setting both equal to zero gives the pair of equations
\begin{equation*}
3x^2 + 2y - 4 = 0 \quad\text{and}\quad 2y + 2x - 3 = 0\text{.}
\end{equation*}
We solve the second equation for \(y\text{,}\) i.e., that \(y = 3/2 - x\text{,}\) and then substitute this into the first equation, which leads to the equation
\begin{equation*}
3x^2 + 2(3/2 - x) - 4 = 0\text{,}
\end{equation*}
which simplifies to the quadratic equation
\begin{equation*}
3x^2 - 2x - 1 = 0\text{.}
\end{equation*}
The quadratic equation tells us that the roots of this equation are given by the formula
\begin{equation*}
x = \frac{-(-2) \pm \sqrt{(-2)^2 - 4 (3)(-1)}}{2(3)} = 1/3 \pm \frac{\sqrt{16}}{6} = 1/3 \pm 2/3\text{.}
\end{equation*}
So \(x = 1\) or \(x = -1/3\text{.}\) For each such \(x\text{,}\) setting \(y = 3/2 - x\) gives us a critical point for \(f\text{,}\) so \(f\) has two critical points, i.e., \((1,1/2)\) and \((-1/3,11/6)\text{.}\)
To apply the second derivative test to each critical point, we calculate the discriminant of \(f\text{.}\) We find that
\begin{equation*}
f_{xx} = 6x \quad f_{yy} = 2 \quad \text{and}\quad f_{xy} = 2\text{.}
\end{equation*}
So the discriminant is the function
\begin{equation*}
D(x,y) = f_{xx}(x,y) f_{yy}(x,y) - f_{xy}(x,y)^2 = (6x)(2) - (2)^2 = 12x - 4\text{.}
\end{equation*}
So
\begin{equation*}
D(1,1/2) = 12(1) - 4 = 8 \quad\text{and}\quad D(-1/3,11/6) = 12(-1/3) - 4 = -8\text{.}
\end{equation*}
Since \(D(1,1/2)\) is positive, and \(f_{xx}(1,1/2) = 6\) is positive, the critical point \((1,1/2)\) is a local minimum of the function \(f\text{.}\) Since \(D(-1/3,11/6)\) is negative, this tells us that \((-1/3,11/6)\) is a saddle point of the function \(f\text{.}\)
Remark 5.5.21.
The following activity explains why we only look at \(f_{xx}\) in the first two cases of the second derivative test.
Activity 5.5.22. The discriminant and signs of second derivatives (U5-LO5).
Solution.
True. If \(D = f_{xx}f_{yy} - (f_{xy})^2 > 0\text{,}\) then \(f_{xx}f_{yy} > (f_{xy})^2 \geq 0\text{,}\) so \(f_{xx} f_{yy} > 0\text{.}\) Since \(f_{xx} f_{yy}\) is positive, this implies either both \(f_{xx}\) and \(f_{yy}\) are positive, or both \(f_{xx}\) and \(f_{yy}\) are negative. So \(f_{xx}\) and \(f_{yy}\) always have the same sign when the discriminant is positive.
Subsection Contour plots and saddle points
The Hessian tests classify local behavior algebraically. Contour plots and restrictions to lines show the same behavior geometrically.
Note 5.5.23.
A common way to detect a saddle point is to find two directions through the point such that the function increases in one direction and decreases in another. This is not a complete definition of every possible saddle point; it is a useful test in many examples.
For example, the function
\begin{equation*}
f(x,y)=x^2-y^2
\end{equation*}
has a saddle point at \((0,0)\text{.}\) Along the line \(y=0\text{,}\)
\begin{equation*}
f(x,0)=x^2,
\end{equation*}
which has a local minimum at \(x=0\text{.}\) Along the line \(x=0\text{,}\)
\begin{equation*}
f(0,y)=-y^2,
\end{equation*}
which has a local maximum at \(y=0\text{.}\) The contour plot below shows the same behavior.
In this example, the surface increases in one coordinate direction and decreases in the other. This is the behavior suggested by the saddle shape.
The contour plot shows level curves of \(f(x,y)=x^2-y^2\text{.}\) The zero contour consists of two diagonal lines crossing at the origin. Positive and negative contour values alternate across the four regions, indicating a saddle point at \((0,0)\text{.}\)
Remark 5.5.25.
More formally, if the lines are given in parametric form by
\begin{equation*}
\{\mathbf{a}+t\mathbf{d}_1\}
\qquad\text{and}\qquad
\{\mathbf{a}+t\mathbf{d}_2\},
\end{equation*}
then the one-variable restrictions
\begin{equation*}
g(t)=f(\mathbf{a}+t\mathbf{d}_1)
\qquad\text{and}\qquad
h(t)=f(\mathbf{a}+t\mathbf{d}_2)
\end{equation*}
can show opposite local behavior at \(t=0\text{.}\) In the example above, this verifies that nearby points occur on both sides of \(f(\mathbf{a})\text{.}\)
A translucent saddle surface for \(z=x^2-y^2\) is shown. The red curve along \(y=0\) opens upward and has a local minimum at the origin, while the blue curve along \(x=0\) opens downward and has a local maximum at the origin.
Activity 5.5.27. Classifying critical points from contours (U5-LO5).
For each point in the figure below, indicate whether it is a local maximum, a local minimum, or a saddle point.
The contour plot shows level curves of the illustrative function \(g(x,y)=2-(x^2-1)^2-(y^2-1)^2\text{.}\) Point \(P\) is at the center of nested low-value contours, point \(S\) is at the center of nested high-value contours, and points \(Q\) and \(R\) are placed where the contour pattern changes like a saddle.
Solution.
If we look at the collection of nested ovals centered on \(S\text{,}\) and inspect the numerical values labeling the level sets, we see those numbers are strictly increasing. These ovals seem to be honing in on a point \(S\) that is a local maximum. Likewise, the collection of nested ovals near \(P\) have function values going in a decreasing direction, suggesting \(P\) is a local minimum. We also see saddle points at \(R\) and \(Q\text{.}\) As we move away from \(R\) in the north-east direction, the function value increases, whereas moving away in the south-east direction causes the function value to decrease. Near saddle points, the contour plot forms an “X” shape dividing the space into \(4\) regions. Function values increase in one pair of opposite regions and decrease in the other pair. Near local extrema, level sets form oval shapes resembling ellipses.
