Unit 4 Orthogonality and least squares
Orthogonality turns geometry into computation. In this unit, projection becomes the mechanism for best approximation, and regression becomes a linear algebra problem.
Big question. When \(A\mathbf{x}=\mathbf{b}\) has no solution, what is the best approximate solution?
Learning outcomes. By the end of this unit, students should be able to:
-
U4-LO1. Use dot products and norms to analyze orthogonality, normalize vectors, and compute vector expansions in orthogonal or orthonormal bases.
-
U4-LO2. Construct orthogonal and orthonormal bases using Gram-Schmidt, and compute orthogonal complements.
-
U4-LO3. Compute orthogonal projections onto subspaces and solve closest-point problems.
-
U4-LO4. Formulate least squares as projection onto a column space, identify the residual, and explain residual orthogonality.
-
U4-LO5. Derive and solve the normal equations \(A^TA\hat{\mathbf{x}}=A^T\mathbf{b}\text{,}\) and use them in small regression problems.
-
U4-LO6. Compute and interpret QR factorizations, and use QR to solve linear systems and least-squares problems.
-
U4-LO7. Interpret projection and least-squares computations in code, including regression, QR methods, and checks such as
A.T @ r, and explain what the outputs do and do not prove.
Toolbox skills. Compute dot products, test orthogonality, normalize vectors, apply Gram-Schmidt, compute projections, identify residuals, derive and solve normal equations, fit a line by least squares, check \(A^T\mathbf{r}\approx \mathbf{0}\text{,}\) and use QR in small least-squares problems.
