Section 6.8 Unit 6 highlights
Big question. What directions matter most in a constrained problem or matrix map?
Notation. \(\nabla f=\lambda\nabla g\text{,}\) \(\|\mathbf{x}\|=1\text{,}\) \(A^TA\text{,}\) \(A=U\Sigma V^T\text{,}\) \(\sigma_i\text{,}\) \(\mathbf{u}_i\text{,}\) \(\mathbf{v}_i\text{,}\) \(\operatorname{rank}(A)\text{,}\) \(\operatorname{row}(A)\text{,}\) \(\operatorname{col}(A)\text{,}\) \(\operatorname{null}(A)\text{,}\) \(\operatorname{null}(A^T)\text{,}\)
\begin{equation*}
A_k=U_k\Sigma_kV_k^T,
\end{equation*}
and
\begin{equation*}
\frac{\sigma_1^2+\cdots+\sigma_k^2}
{\sigma_1^2+\cdots+\sigma_r^2}.
\end{equation*}
Learning outcomes. U6-LO1 through U6-LO7 ask you to compare constrained candidates, interpret parallel gradients, compute singular values and singular vectors, read and construct SVDs, identify fundamental subspaces, interpret stretch factors, and explain rank-\(k\) approximation or compression.
Concepts. Closed bounded region, boundary candidate, absolute extremum, constrained critical point, Lagrange multiplier, parallel gradients, quadratic form, maximum stretch, singular value, right singular vector, left singular vector, SVD, fundamental subspaces, rank-\(k\) reconstruction, energy retained, redundant feature, low-rank update.
Results. A continuous function on a closed bounded region has absolute extrema. To find absolute extrema on a region, compare interior and boundary candidates. For a constraint \(g(\mathbf{x})=k\text{,}\) the Lagrange condition
\begin{equation*}
\nabla f=\lambda\nabla g
\end{equation*}
means the objective gradient is parallel to the constraint gradient. For symmetric \(B\text{,}\) the extrema of
\begin{equation*}
\mathbf{x}^TB\mathbf{x}
\end{equation*}
on the unit sphere occur at eigenvectors of \(B\text{.}\) Applying this to \(B=A^TA\) gives singular values and right singular vectors. The SVD
\begin{equation*}
A=U\Sigma V^T
\end{equation*}
separates input directions, stretch factors, and output directions. Positive singular values identify transmitted directions; zero singular values identify forgotten directions. Truncated SVD keeps the largest singular directions.
Toolbox skills. U6-LO1 through U6-LO7 use candidate tables, boundary restrictions, Lagrange equations, \(A^TA\text{,}\) eigenvalues, eigenvectors, singular values, singular vectors, rank from SVD, fundamental subspaces, energy retained, NumPy SVD shapes, slicing such as
U[:, :k] and Vt[:k, :], and low-rank update storage counts.
Main applications. Constrained optimization on closed bounded sets, maximum stretch of a matrix map, dominant directions in data matrices, redundant-feature detection, SVD compression, rank-\(k\) reconstruction, energy retained, and model-matrix low-rank updates.
Connections. Unit 1 supplies vectors, matrix maps, matrix-vector products, shapes, and code-reading habits. Unit 2 supplies rank, null spaces, reachable outputs, forgotten directions, and redundant features. Unit 3 supplies gradients, level sets, tangent directions, closed sets, and bounded sets. Unit 4 supplies orthonormal bases and projection-like approximation language. Unit 5 supplies eigenvalues, eigenvectors, symmetric matrices, and quadratic forms. Unit 7 returns to best approximation, polynomial inner products, and low-rank structure.
Study anchors. U6-LO1: Absolute extrema on closed bounded regions. U6-LO2: Lagrange multipliers and parallel gradients. U6-LO3 and U6-LO6: Maximum stretch and singular values. U6-LO4 and U6-LO5: Singular value decomposition. U6-LO7: Low-rank approximation and compression. Code interpretation and exam-style checks: Applications and computation recap and linked notebook.
Applications and computations readiness checklist. Given a short Unit 6 calculation, code snippet, table, or diagram, I can decide whether boundary candidates are needed; set up Lagrange equations; interpret \(\nabla f=\lambda\nabla g\text{;}\) read \(A^TA\) as a maximum-stretch calculation; explain why singular values are square roots of eigenvalues of \(A^TA\text{;}\) read
U, s, Vt; check the identity \(A\mathbf{v}_i=\sigma_i\mathbf{u}_i\text{;}\) find rank from singular values; identify row, column, and null-space directions from SVD; compute energy retained; explain what rank-\(k\) reconstruction keeps and discards; debug an incorrect SVD slice; and distinguish a low-rank update from a low-rank matrix.
Common mistakes. Ignoring boundary candidates; treating constrained critical points as final answers without comparing values; writing \(\nabla f=\lambda\nabla f\text{;}\) forgetting that \(\nabla g\) must be nonzero for the usual Lagrange condition; confusing \(U\text{,}\) \(V\text{,}\) and \(V^T\text{;}\) forgetting that singular values are nonnegative and usually sorted from largest to smallest; using \(s\) instead of \(s^2\) for energy retained; using
Vt[:, :k] instead of Vt[:k, :]; treating a small numerical singular value as automatically exact zero; assuming a rank-\(k\) reconstruction is exact; or assuming \(W+BC\) must have low rank just because \(BC\) has low rank.
