Section 7.6 Unit 7 highlights
Subsection Mathematical quick reference
Vector spaces, bases, and coordinates.
Real vector space. A set with addition and real scalar multiplication satisfying closure, associativity and commutativity of addition, a zero vector and additive inverses, both distributive laws, \(a(b\mathbf v)=(ab)\mathbf v\text{,}\) and \(1\mathbf v=\mathbf v\text{.}\) A vector need not be a numerical column.
Subspace test. A subset \(U\subseteq V\) is a subspace if it contains the zero vector and is closed under addition and scalar multiplication.
Basis and coordinates. A basis is an independent spanning list. For an ordered basis \(\mathcal B=(\mathbf v_1,\ldots,\mathbf v_k)\text{,}\)
\begin{equation*}
\mathbf v=\sum_{j=1}^k c_j\mathbf v_j,\qquad [\mathbf v]_{\mathcal B}=\begin{bmatrix}c_1\\\vdots\\c_k\end{bmatrix},\qquad \dim V=k.
\end{equation*}
Independence means the zero vector has only the all-zero coefficient representation.
Matrix and polynomial spaces. \(M_{m,n}\) has dimension \(mn\text{,}\) with basis matrices having one entry \(1\) and all others zero. The space \(\mathcal P_{\le d}\) has basis \((1,x,\ldots,x^d)\) and dimension \(d+1\text{.}\) The coordinates of \(p(x)=\sum_{j=0}^d c_jx^j\) are its coefficients in that order.
Inner products and geometry.
Real inner product. The rule \(\langle\mathbf u,\mathbf v\rangle\) is symmetric, linear in each argument, and positive definite:
\begin{equation*}
\langle\mathbf v,\mathbf v\rangle\geq0,\qquad \langle\mathbf v,\mathbf v\rangle=0\quad\Longleftrightarrow\quad\mathbf v=\mathbf0.
\end{equation*}
Norm, distance, and orthogonality.
\begin{equation*}
\|\mathbf v\|=\sqrt{\langle\mathbf v,\mathbf v\rangle},\qquad d(\mathbf u,\mathbf v)=\|\mathbf u-\mathbf v\|,\qquad \mathbf u\perp\mathbf v\quad\Longleftrightarrow\quad\langle\mathbf u,\mathbf v\rangle=0.
\end{equation*}
Normalize a nonzero vector by dividing by its norm.
Matrix inner product. For matrices of the same size,
\begin{equation*}
\langle A,B\rangle=\Tr(A^TB)=\sum_{i,j}a_{ij}b_{ij}.
\end{equation*}
The trace \(\Tr(C)\) of a square matrix is the sum of its diagonal entries.
Polynomial inner products. For coefficient lists \(a_j,b_j\text{,}\)
\begin{equation*}
\langle p,q\rangle_{\mathrm{coef}}=\sum_{j=0}^d a_jb_j,\qquad \langle p,q\rangle_{L^2}=\int_{-1}^1p(x)q(x)\,dx.
\end{equation*}
The sampled rule \(\sum_i p(x_i)q(x_i)\) is an inner product on a chosen polynomial space exactly when no nonzero polynomial in that space vanishes at every sample point. Otherwise it fails positive definiteness.
Projection and Gram matrices.
Best approximation. In a real inner product space, the projection \(\mathbf q\) of \(\mathbf f\) onto a finite-dimensional subspace \(U\) is the unique vector satisfying
\begin{equation*}
\mathbf q\in U,\qquad \mathbf f-\mathbf q\perp U.
\end{equation*}
It uniquely minimizes \(\|\mathbf f-\mathbf u\|\) over \(\mathbf u\in U\text{.}\) Changing the inner product can change the projection.
Gram-matrix equations. For a basis \((\phi_0,\ldots,\phi_m)\) of \(U\text{,}\) write \(q=\sum_{j=0}^m c_j\phi_j\) and solve
\begin{equation*}
G\mathbf c=\mathbf b,\qquad G_{ij}=\langle\phi_j,\phi_i\rangle,\quad b_i=\langle f,\phi_i\rangle.
\end{equation*}
A genuine inner product and an independent basis make \(G\) positive definite and invertible.
Orthogonal basis shortcut. If the nonzero \(\phi_j\) are pairwise orthogonal, then
\begin{equation*}
q=\sum_{j=0}^m\frac{\langle f,\phi_j\rangle}{\langle\phi_j,\phi_j\rangle}\phi_j.
\end{equation*}
Three approximation conditions.
Taylor matching. For a polynomial \(p\text{,}\) the degree-\(m\) Taylor polynomial at \(a\) is
\begin{equation*}
T_mp(x)=\sum_{j=0}^m\frac{p^{(j)}(a)}{j!}(x-a)^j.
\end{equation*}
Its residual has zero derivatives through order \(m\) at \(a\text{.}\)
Continuous least squares. Over an interval \([a,b]\) with \(a\lt b\text{,}\) minimize \(\int_a^b(f-q)^2\,dx\) over \(q\in U\text{.}\) The residual satisfies \(\int_a^b(f-q)\phi_j\,dx=0\) for every basis function.
Sampled least squares. With \(A_{ij}=\phi_j(x_i)\) and \(y_i=f(x_i)\text{,}\) minimize \(\|A\mathbf c-\mathbf y\|^2\text{.}\) The equations are
\begin{equation*}
A^TA\widehat{\mathbf c}=A^T\mathbf y,\qquad \mathbf r=\mathbf y-A\widehat{\mathbf c},\qquad A^T\mathbf r=\mathbf0.
\end{equation*}
Coefficients are unique when \(A\) has independent columns. Neither continuous nor sampled least squares generally agrees with Taylor matching.
Local projection and shrinking intervals.
Derivative-based inner product. On \(\mathcal P_{\le d}\text{,}\) fix \(a\) and positive weights \(w_j\text{:}\)
\begin{equation*}
\langle p,q\rangle_{\mathrm{jet},a}=\sum_{j=0}^d w_j\frac{p^{(j)}(a)}{j!}\frac{q^{(j)}(a)}{j!}.
\end{equation*}
For \(0\leq m\leq d\text{,}\) projection onto \(\mathcal P_{\le m}\) is exactly \(T_mp\text{:}\) retain the first \(m+1\) coefficients in powers of \(x-a\text{.}\)
Affine projection on a shrinking interval. For a polynomial \(p\) and \(t>0\text{,}\) its \(L^2[-t,t]\)-projection onto \(\operatorname{span}\{1,x\}\) is
\begin{equation*}
P_t p(x)=a_t+b_tx,\qquad a_t=\frac1{2t}\int_{-t}^t p(x)\,dx,\qquad b_t=\frac3{2t^3}\int_{-t}^t xp(x)\,dx.
\end{equation*}
As \(t\to0\text{,}\) \(a_t\to p(0)\) and \(b_t\to p'(0)\text{,}\) so the coefficients approach those of the Taylor line.
Gram–Schmidt in an inner product space.
Orthogonalize an independent list. Starting with \(v_0,\ldots,v_m\text{,}\) set
\begin{equation*}
u_0=v_0,\qquad u_j=v_j-\sum_{i=0}^{j-1}\frac{\langle v_j,u_i\rangle}{\langle u_i,u_i\rangle}u_i.
\end{equation*}
The vectors \(u_j\) are nonzero and pairwise orthogonal, with the same successive spans as the input list. Normalize separately by \(q_j=u_j/\|u_j\|\) for an orthonormal basis. Use the specified inner product in every coefficient and norm.
Subsection Common mistakes
Assuming vectors must be numerical columns; forgetting that the zero vector in a polynomial space is the zero polynomial; confusing degree \(d\) with dimension \(d+1\text{;}\) treating an affine set as a subspace; changing a basis without changing coordinates; or using a sampled rule as an inner product without checking positive definiteness.
Using “best approximation” without specifying an inner product or error measure; confusing local Taylor matching with fixed-interval or sampled least squares; interpreting residual orthogonality as a zero residual; applying ordinary dot products where a different inner product was specified; or calling an orthogonal basis orthonormal before normalizing it.
