Skip to main content

MATH 345: Linear Algebra and Optimization

Sebastien Roch with Jacob Denson and Kaitlyn Phillipson

Section 7.5 Exercises

Learning outcomes. The labels below identify the learning outcomes for each exercise group. Individual problems may also involve earlier outcomes.
🔗

Subsection Abstract vector spaces

Learning outcomes. U7-LO1, U7-LO2.
🔗
  • Nicholson 6.1.1(b)
     1 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.01%3A_Examples_and_Basic_Properties/6.1E%3A_Examples_and_Basic_Properties_Exercises
    🔗
    🔗
  • Nicholson 6.1.2(f)
     2 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.01%3A_Examples_and_Basic_Properties/6.1E%3A_Examples_and_Basic_Properties_Exercises
    🔗
    🔗
  • Nicholson 6.1.6(b)
     3 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.01%3A_Examples_and_Basic_Properties/6.1E%3A_Examples_and_Basic_Properties_Exercises
    🔗
    🔗
🔗

Subsection Subspaces, bases, and coordinates in nonstandard spaces

Learning outcomes. U7-LO1, U7-LO2.
🔗
  • Nicholson 6.2.2(d)
     4 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.02%3A_Subspaces_and_Spanning_Sets/6.2E%3A_Subspaces_and_Spanning_Sets_Exercises
    🔗
    🔗
  • Nicholson 6.2.2(f)
     5 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.02%3A_Subspaces_and_Spanning_Sets/6.2E%3A_Subspaces_and_Spanning_Sets_Exercises
    🔗
    🔗
  • Nicholson 6.3.2(d)
     6 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/06%3A_Vector_Spaces/6.03%3A_Linear_Independence_and_Dimension/6.3E%3A_Linear_Independence_and_Dimension_Exercises
    🔗
    🔗
🔗

Subsection Inner product spaces

Learning outcomes. U7-LO3, U7-LO4.
🔗
  • Nicholson 10.1.1(b)
     7 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/10%3A_Inner_Product_Spaces/10.01%3A_Inner_Products_and_Norms/10.1E%3A_Inner_Products_and_Norms_Exercises
    🔗
    🔗
  • Nicholson 10.1.1(d)
     8 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/10%3A_Inner_Product_Spaces/10.01%3A_Inner_Products_and_Norms/10.1E%3A_Inner_Products_and_Norms_Exercises
    🔗
    🔗
  • Nicholson 10.1.3(d)
     9 
    math.libretexts.org/Bookshelves/Linear_Algebra/Linear_Algebra_with_Applications_(Nicholson)/10%3A_Inner_Product_Spaces/10.01%3A_Inner_Products_and_Norms/10.1E%3A_Inner_Products_and_Norms_Exercises
    🔗
    🔗
🔗

Subsection Additional exercises

These exercises connect the main Unit 7 ideas: abstract vector spaces, matrix and polynomial examples, inner products, projection equations, Taylor and least-squares approximations, Gram-Schmidt in polynomial spaces, sampled polynomial regression, two-variable planes, and code interpretation. Solutions are collected in Appendix C.7.
🔗

Additional exercise 7.5.1. Matrix-space coordinates.

Let
\begin{equation*} E_{11}= \begin{bmatrix} 1\amp0\\ 0\amp0 \end{bmatrix}, \quad E_{12}= \begin{bmatrix} 0\amp1\\ 0\amp0 \end{bmatrix}, \quad E_{21}= \begin{bmatrix} 0\amp0\\ 1\amp0 \end{bmatrix}, \quad E_{22}= \begin{bmatrix} 0\amp0\\ 0\amp1 \end{bmatrix}. \end{equation*}
Let
\begin{equation*} A= \begin{bmatrix} 2\amp-1\\ 0\amp3 \end{bmatrix}. \end{equation*}
🔗
  1. Write \(A\) as a linear combination of \(E_{11},E_{12},E_{21},E_{22}\text{.}\)
    🔗
    🔗
  2. Write the coordinate vector of \(A\) in the ordered basis \((E_{11},E_{12},E_{21},E_{22})\text{.}\)
    🔗
    🔗
  3. What is \(\dim M_{22}\text{?}\)
    🔗
    🔗
  4. What is the zero vector in \(M_{22}\text{?}\)
    🔗
    🔗
Learning outcomes. U7-LO2.
🔗
🔗

Additional exercise 7.5.2. Polynomial coordinates and subspaces.

Let \(p(x)=2-3x+x^3\text{.}\)
🔗
  1. Write the coordinate vector of \(p\) in the ordered basis \((1,x,x^2,x^3)\text{.}\)
    🔗
    🔗
  2. Find a basis for \(U=\{p\in\mathcal P_{\le3}:p(0)=0\}\text{.}\)
    🔗
    🔗
  3. Find a basis for \(W=\{p\in\mathcal P_{\le3}:p(0)=p'(0)=0\}\text{.}\)
    🔗
    🔗
  4. Explain why \(U\) and \(W\) are subspaces.
    🔗
    🔗
Learning outcomes. U7-LO1, U7-LO2.
🔗
🔗

Additional exercise 7.5.3. Derivative and evaluation as linear maps.

Define
\begin{equation*} D:\mathcal P_{\le3}\to\mathcal P_{\le2},\qquad D(p)=p', \end{equation*}
and
\begin{equation*} E_0:\mathcal P_{\le3}\to\mathbb R,\qquad E_0(p)=p(0). \end{equation*}
Use the ordered bases \((1,x,x^2,x^3)\) for \(\mathcal P_{\le3}\) and \((1,x,x^2)\) for \(\mathcal P_{\le2}\text{.}\)
🔗
  1. Explain why \(D\) is linear.
    🔗
    🔗
  2. Find the matrix of \(D\) in these bases.
    🔗
    🔗
  3. Explain why \(E_0\) is linear.
    🔗
    🔗
  4. Find a basis for the kernel of \(E_0\text{.}\)
    🔗
    🔗
Learning outcomes. U7-LO1, U7-LO2.
🔗
🔗

Additional exercise 7.5.4. Matrix inner product.

Use the matrix inner product \(\langle A,B\rangle=\operatorname{tr}(A^TB)\text{.}\) Let
\begin{equation*} A= \begin{bmatrix} 1\amp2\\ 0\amp-1 \end{bmatrix}, \qquad B= \begin{bmatrix} 3\amp0\\ 4\amp1 \end{bmatrix}. \end{equation*}
🔗
  1. Compute \(\langle A,B\rangle\text{.}\)
    🔗
    🔗
  2. Compute \(\|A\|\text{.}\)
    🔗
    🔗
  3. Compute \(d(A,B)\text{.}\)
    🔗
    🔗
  4. Are \(A\) and \(B\) orthogonal?
    🔗
    🔗
Learning outcomes. U7-LO3.
🔗
🔗

Additional exercise 7.5.5. Same polynomial space, different inner products.

In \(\mathcal P_{\le1}\text{,}\) let \(p(x)=1+x\) and \(q(x)=1-x\text{.}\)
🔗
  1. Compute \(\langle p,q\rangle_{\mathrm{coef}}\text{.}\)
    🔗
    🔗
  2. Compute \(\langle p,q\rangle_{L^2}\text{,}\) where \(\langle p,q\rangle_{L^2}=\int_{-1}^{1}p(x)q(x)\,dx\text{.}\)
    🔗
    🔗
  3. Are \(p\) and \(q\) orthogonal for the coefficient inner product?
    🔗
    🔗
  4. Are \(p\) and \(q\) orthogonal for the \(L^2[-1,1]\) inner product?
    🔗
    🔗
  5. What does this show about the phrase “orthogonal polynomials”?
    🔗
    🔗
Learning outcomes. U7-LO3.
🔗
🔗

Additional exercise 7.5.6. Projection equations for the continuous least-squares line.

Let \(p(x)=1+2x+x^2+x^3\text{,}\) and \(U=\operatorname{span}\{1,x\}\text{.}\) Use the \(L^2[-1,1]\) inner product.
🔗
  1. Compute the Gram matrix for the ordered basis \((1,x)\text{.}\)
    🔗
    🔗
  2. Compute the right-hand side vector \(\mathbf b\) for projecting \(p\) onto \(U\text{.}\)
    🔗
    🔗
  3. Solve \(G\mathbf c=\mathbf b\text{.}\)
    🔗
    🔗
  4. What is the continuous least-squares line?
    🔗
    🔗
Learning outcomes. U7-LO4, U7-LO5.
🔗
🔗

Additional exercise 7.5.7. Taylor line versus fixed-interval least squares.

Let \(f(x)=x^2\text{.}\)
🔗
  1. Find the Taylor line of \(f\) at \(0\text{.}\)
    🔗
    🔗
  2. Find the \(L^2[-1,1]\)-projection of \(f\) onto \(\mathcal P_{\le1}=\operatorname{span}\{1,x\}\text{.}\)
    🔗
    🔗
  3. Explain why the answers differ.
    🔗
    🔗
Learning outcomes. U7-LO5.
🔗
🔗

Additional exercise 7.5.8. Taylor projection from a local inner product.

On \(\mathcal P_{\le3}\text{,}\) define
\begin{equation*} \langle p,q\rangle_0 = \sum_{k=0}^{3} \frac{p^{(k)}(0)}{k!} \frac{q^{(k)}(0)}{k!}. \end{equation*}
Let \(p(x)=a_0+a_1x+a_2x^2+a_3x^3\text{.}\) Show that the orthogonal projection of \(p\) onto \(\mathcal P_{\le1}\) is \(a_0+a_1x\text{.}\) Then rewrite the answer using \(p(0)\) and \(p'(0)\text{.}\)
🔗
Learning outcomes. U7-LO3, U7-LO5.
🔗
🔗

Additional exercise 7.5.9. Shrinking least squares.

Let \(P_t f\) denote the \(L^2[-t,t]\)-projection of \(f\) onto \(\operatorname{span}\{1,x\}\text{.}\)
🔗
  1. Show that \(P_t(x^2)=\frac{t^2}{3}\text{.}\)
    🔗
    🔗
  2. Show that \(P_t(x^3)=\frac{3t^2}{5}x\text{.}\)
    🔗
    🔗
  3. What happens to both projections as \(t\to0\text{?}\)
    🔗
    🔗
  4. How does this connect to Taylor approximation?
    🔗
    🔗
Learning outcomes. U7-LO5.
🔗
🔗

Additional exercise 7.5.10. Gram-Schmidt in \(\mathcal P_{\le3}\).

Use the inner product \(\langle f,g\rangle=\int_{-1}^{1}f(x)g(x)\,dx\text{.}\) Apply Gram-Schmidt to \(1,\ x,\ x^2,\ x^3\) through the \(x^3\) step.
🔗
  1. Explain why \(1\) and \(x\) are orthogonal.
    🔗
    🔗
  2. Derive \(x^2-\frac13\text{.}\)
    🔗
    🔗
  3. Derive \(x^3-\frac35x\text{.}\)
    🔗
    🔗
  4. Why does an orthogonal basis make projection easier?
    🔗
    🔗
Learning outcomes. U7-LO6.
🔗
🔗

Additional exercise 7.5.11. Code interpretation: polynomial regression.

Consider the code:
🔗
A = np.column_stack([np.ones_like(xs), xs, xs**2])
c = np.linalg.lstsq(A, y, rcond=None)[0]
r = y - A @ c
A.T @ r
  1. What polynomial subspace is represented by the columns of A?
    🔗
    🔗
  2. What does the vector c store?
    🔗
    🔗
  3. What does A @ c store?
    🔗
    🔗
  4. What does r store?
    🔗
    🔗
  5. What does A.T @ r check?
    🔗
    🔗
  6. Why does \(A^T\mathbf r\approx\mathbf0\) not mean \(\mathbf r\approx\mathbf0\text{?}\)
    🔗
    🔗
Learning outcomes. U7-LO4, U7-LO5.
🔗
🔗

Additional exercise 7.5.12. Two-variable tangent plane and sampled design matrix.

Let \(f(x,y)=1+2x-y+x^2+xy+2y^2+x^3\text{.}\)
🔗
  1. Compute the tangent plane to \(f\) at \((0,0)\text{.}\)
    🔗
    🔗
  2. Write the residual \(r_T=f-T\text{.}\)
    🔗
    🔗
  3. What conditions do \(r_T(0,0)\) and \(\nabla r_T(0,0)\) satisfy?
    🔗
    🔗
  4. For sampled points \((x_i,y_i)\text{,}\) write the design matrix for fitting a plane \(q(x,y)=\alpha+\beta x+\gamma y\text{.}\)
    🔗
    🔗
  5. Write the sampled residual-orthogonality equations.
    🔗
    🔗
Learning outcomes. U7-LO5.
🔗
🔗
🔗
🔗
PrevTopNext