Unit 4 Orthogonality and least squares
Orthogonality turns geometry into computation. In this unit, projection becomes the mechanism for best approximation, and regression becomes a linear algebra problem.
Big question. When \(A\mathbf{x}=\mathbf{b}\) has no solution, what is the best approximate solution?
Learning outcomes. By the end of this unit, students should be able to:
Core outcomes.
-
U4-LO1. Use orthogonality and orthonormal bases to decompose vectors and compute coefficients.
-
U4-LO2. Construct orthogonal and orthonormal bases using Gram-Schmidt, and compute orthogonal complements.
-
U4-LO3. Compute orthogonal projections onto subspaces and solve closest-point problems.
-
U4-LO4. Formulate least squares as projection onto a column space, identify the residual, and explain residual orthogonality.
-
U4-LO5. Derive and solve the normal equations \(A^TA\hat{\mathbf{x}}=A^T\mathbf{b}\text{,}\) and use them in small regression problems.
-
U4-LO6. Compute and interpret QR factorizations, and use QR to solve linear systems and least-squares problems.
-
U4-LO7. Interpret computational least-squares checks such as
A.T @ rand explain what they do and do not prove.
Toolbox skills. Compute dot products, test orthogonality, normalize vectors, apply Gram-Schmidt, compute projections, identify residuals, derive and solve normal equations, fit a line by least squares, check \(A^T\mathbf{r}\approx \mathbf{0}\text{,}\) and use QR in small least-squares problems.
Application and computation outcomes. Interpret Lab U4
[U4-LO4, U4-LO5, U4-LO7 | C+M+T | Core] by reading projection computations, design matrices, least-squares coefficients, residual orthogonality, normal equations, QR checks, and code outputs such as A.T @ r.
