Skip to main content
Contents Dark Mode Prev Up Next \(\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\dimens}{\operatorname{dim}}
\DeclareMathOperator{\row}{\operatorname{row}}
\DeclareMathOperator{\col}{\operatorname{col}}
\newcommand{\im}{\operatorname{im}}
\newcommand{\nulls}{\operatorname{null}}
\newcommand{\minor}{\operatorname{minor}}
\newcommand{\spans}{\operatorname{span}}
\newcommand{\nullity}{\operatorname{nullity}}
\newcommand{\kers}{\operatorname{ker}}
\newcommand{\proj}{\operatorname{proj}}
\newcommand{\diag}{\operatorname{diag}}
\newcommand{\Tr}{\operatorname{Tr}}
\newcommand{\rank}{\operatorname{rank}}
\newcommand{\lt}{<}
\newcommand{\gt}{>}
\newcommand{\amp}{&}
\definecolor{fillinmathshade}{gray}{0.9}
\newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}}
\)
Section 6.7 Exercises
Exercise tags. The tags below identify the primary outcomes and competency types for each exercise group. Individual problems may also involve earlier outcomes.
Subsection Lagrange multipliers and parallel gradients
Primary outcomes. U6-LO1, U6-LO2.
Competencies. P+C+M.
Subsection Singular value decomposition
Primary outcomes. U6-LO3, U6-LO4, U6-LO5, U6-LO6.
Competencies. P+C.
Subsection Additional applied and computational problems
These exercises connect the main Unit 6 ideas: absolute extrema on closed bounded regions, Lagrange multipliers, maximum stretch, singular values, SVD, fundamental subspaces, rank-
\(k\) reconstruction, energy retained, redundant features, low-rank updates, and code interpretation. Solutions are collected in Appendix
C.6 .
Checkpoint 6.7.1 . Candidate table on a rectangle.
Let
\begin{equation*}
f(x,y)=x^2+y^2-2x+4y
\end{equation*}
on the rectangle
\begin{equation*}
D=\{(x,y):0\leq x\leq 3,\ -3\leq y\leq 1\}.
\end{equation*}
Find the critical point of
\(f\) in the interior of
\(D\text{.}\)
Find the boundary candidates on each of the four edges.
Include the four corner points.
Make a candidate table with the source of each candidate and the value of
\(f\text{.}\)
Find the absolute maximum and absolute minimum of
\(f\) on
\(D\text{.}\)
Tags. [U6-LO1 | P+C+R | Core]
Checkpoint 6.7.2 . Parallel gradients on a circle.
Let
\begin{equation*}
f(x,y)=3x+4y
\end{equation*}
and constrain the input to the circle
\begin{equation*}
x^2+y^2=25.
\end{equation*}
Write the constraint as
\(g(x,y)=25\text{.}\)
Solve
\begin{equation*}
\nabla f=\lambda\nabla g.
\end{equation*}
Find the constrained critical points.
Evaluate
\(f\) at the constrained critical points.
Which point gives the absolute maximum? Which point gives the absolute minimum?
Explain geometrically why
\(\nabla f\) is parallel to
\(\nabla g\) at those points.
Tags. [U6-LO2 | P+C+R | Core]
Checkpoint 6.7.3 . Quadratic form on the unit circle.
Checkpoint 6.7.4 . Maximum stretch of a matrix.
Let
\begin{equation*}
A=
\begin{bmatrix}
1\amp 2\\
2\amp 1
\end{bmatrix}.
\end{equation*}
Find the eigenvalues of
\(A^TA\text{.}\)
Find the singular values of
\(A\text{.}\)
Find a unit right singular vector corresponding to the largest singular value.
What is
\begin{equation*}
\max_{\|\mathbf{x}\|=1}\|A\mathbf{x}\|?
\end{equation*}
Which input direction is stretched most?
Tags. [U6-LO3, U6-LO6 | P+C+R | Core]
Checkpoint 6.7.5 . Reading an SVD.
Suppose
\begin{equation*}
A=U\Sigma V^T,
\end{equation*}
where
\begin{equation*}
U=
\begin{bmatrix}
1\amp 0\\
0\amp 1
\end{bmatrix},
\qquad
\Sigma=
\begin{bmatrix}
5\amp 0\amp 0\\
0\amp 2\amp 0
\end{bmatrix},
\end{equation*}
and
\begin{equation*}
V=
\begin{bmatrix}
1/\sqrt2\amp 0\amp 1/\sqrt2\\
1/\sqrt2\amp 0\amp -1/\sqrt2\\
0\amp 1\amp 0
\end{bmatrix}.
\end{equation*}
Thus the right singular vectors are
\begin{equation*}
\mathbf{v}_1=
\frac{1}{\sqrt2}
\begin{bmatrix}
1\\
1\\
0
\end{bmatrix},
\qquad
\mathbf{v}_2=
\begin{bmatrix}
0\\
0\\
1
\end{bmatrix},
\qquad
\mathbf{v}_3=
\frac{1}{\sqrt2}
\begin{bmatrix}
1\\
-1\\
0
\end{bmatrix}.
\end{equation*}
What are the singular values of
\(A\text{?}\)
What is
\(\operatorname{rank}(A)\text{?}\)
Compute
\(A\mathbf{v}_1\text{,}\) \(A\mathbf{v}_2\text{,}\) and
\(A\mathbf{v}_3\) using the SVD.
Which input direction is forgotten?
Which input directions are transmitted?
Tags. [U6-LO4, U6-LO5, U6-LO6 | C+R | Core]
Checkpoint 6.7.6 . Fundamental subspaces from an SVD.
Use the SVD data from the previous checkpoint.
Give an orthonormal basis for
\(\operatorname{row}(A)\text{.}\)
Give an orthonormal basis for
\(\operatorname{null}(A)\text{.}\)
Give an orthonormal basis for
\(\operatorname{col}(A)\text{.}\)
Give an orthonormal basis for
\(\operatorname{null}(A^T)\text{.}\)
Check that the dimensions agree with rank-nullity.
Tags. [U6-LO5 | C+R | Core]
Checkpoint 6.7.7 . Energy retained.
Suppose the nonzero singular values of a matrix are
\begin{equation*}
10,\ 4,\ 1,\ 0.5.
\end{equation*}
Compute the total squared singular-value energy.
Compute the energy retained by
\(k=1\text{.}\)
Compute the energy retained by
\(k=2\text{.}\)
Find the smallest
\(k\) that retains at least
\(95\%\) of the energy.
Explain what is discarded by that rank-
\(k\) reconstruction.
Tags. [U6-LO6, U6-LO7 | C+M | Core]
Checkpoint 6.7.8 . Code interpretation: rank-\(k\) reconstruction.
The following code computes a rank-one reconstruction.
import numpy as np
A = np.array([[3., 0.],
[0., 1.]])
U, s, Vt = np.linalg.svd(A, full_matrices=False)
k = 1
Ak = U[:, :k] @ np.diag(s[:k]) @ Vt[:k, :]
s, Ak
What are the singular values of
\(A\text{?}\)
Which singular value is kept when
\(k=1\text{?}\)
What is the shape of
U[:, :k]?
What is the shape of
np.diag(s[:k])?
What is the shape of
Vt[:k, :]?
Which input direction is discarded?
Tags. [U6-LO4, U6-LO7 | R+T | Core]
Checkpoint 6.7.9 . Debugging an SVD slice.
A student tries to compute a rank-one reconstruction using this code.
A = np.array([[3., 0.],
[0., 1.]])
U, s, Vt = np.linalg.svd(A, full_matrices=False)
k = 1
Ak_wrong = U[:, :k] @ np.diag(s[:k]) @ Vt[:, :k]
What should the last slice be?
Explain the shape problem.
Explain the mathematical problem.
Tags. [U6-LO4, U6-LO7 | C+T | Core]
Checkpoint 6.7.10 . Redundant features revisited by singular values.
Return to the Unit 2 redundant-feature matrix
\begin{equation*}
X=
\begin{bmatrix}
1\amp 2\amp 3\\
2\amp 4\amp 6\\
0\amp 1\amp 1\\
1\amp -1\amp 0
\end{bmatrix}.
\end{equation*}
The third column is the sum of the first two columns.
Verify that
\begin{equation*}
X
\begin{bmatrix}
1\\
1\\
-1
\end{bmatrix}
=
\mathbf{0}.
\end{equation*}
What does this say about
\(\operatorname{null}(X)\text{?}\)
What does this say about the rank of
\(X\text{?}\)
What singular value should appear because of this redundancy?
If \(X\mathbf{c}=\mathbf{y}\text{,}\) what is
\begin{equation*}
X\left(\mathbf{c}+t
\begin{bmatrix}
1\\
1\\
-1
\end{bmatrix}\right)?
\end{equation*}
Explain how this revisits the Unit 2 idea of nonunique coefficient vectors.
Tags. [U2-LO6, U6-LO5, U6-LO6 | C+R+M | Core]
Checkpoint 6.7.11 . Low-rank update storage.
Let
\begin{equation*}
W\in\mathbb R^{5000\times 3000},
\qquad
B\in\mathbb R^{5000\times 8},
\qquad
C\in\mathbb R^{8\times 3000}.
\end{equation*}
Define
\begin{equation*}
W_{\text{new}}=W+BC.
\end{equation*}
What is the shape of
\(BC\text{?}\)
What is the largest possible rank of
\(BC\text{?}\)
How many numbers are needed to store
\(W\text{?}\)
How many numbers are needed to store
\(B\) and
\(C\text{?}\)
Why is
\(BC\) called a low-rank update?
Does this mean
\(W_{\text{new}}\) must have low rank? Explain.
Tags. [U6-LO7 | C+M | Core]
