Section 2.8 Unit 2 highlights
Subsection Mathematical quick reference
Systems and row reduction.
Matrix equation. For \(A\in\mathbb R^{m\times n}\text{,}\) solving \(A\mathbf x=\mathbf b\) means finding \(\mathbf x\in\mathbb R^n\) for a specified \(\mathbf b\in\mathbb R^m\text{.}\) Use the augmented matrix \([A\mid\mathbf b]\) from Definition 2.2.26.
Elementary row operations. Interchange rows, multiply a row by a nonzero scalar, or add a multiple of one row to a different row. Apply each operation to every column of the augmented matrix, including the right-hand side. These operations preserve the solution set by Theorem 2.2.32.
Echelon forms. In REF, zero rows are last, the first nonzero entry of each nonzero row is a leading one (pivot), and successive pivots move to the right as one moves down the rows. Entries below each pivot are zero. RREF additionally requires each pivot to be the only nonzero entry in its column. See Definition 2.2.35 and Algorithm 2.2.40.
Classify and solve. After row reduction, a row \([0\ \cdots\ 0\mid d]\) with \(d\neq0\) means no solution. Otherwise the system is consistent. Pivot columns to the left of the vertical line give leading or basic variables; the other variable columns give free variables. Back-substitute from REF or read the equations from RREF to express the basic variables in terms of the free variables, as in Algorithm 2.2.48.
Number of parameters. A consistent system with \(n\) variables and \(r\) pivot columns to the left of the vertical line has \(n-r\) free parameters. If \(r=n\text{,}\) the solution is unique; if \(r\lt n\text{,}\) there are infinitely many solutions. See Theorem 2.2.50.
Homogeneous systems and basic solutions. A homogeneous system has the form \(A\mathbf x=\mathbf0\) and always has the trivial solution \(\mathbf x=\mathbf0\text{.}\) Set one free variable to \(1\) and the others to \(0\) in turn, then solve for the basic variables. This gives one basic solution for each free variable. Every homogeneous solution is a unique linear combination of these basic solutions; see Definition 2.2.63.
General solution. If \(\mathbf x_p\) is one solution of \(A\mathbf x=\mathbf b\text{,}\) then
\begin{equation*}
\{\mathbf x:A\mathbf x=\mathbf b\}=\mathbf x_p+\operatorname{null}(A).
\end{equation*}
Thus every other solution differs from \(\mathbf x_p\) by a homogeneous solution. See Fact 2.3.20.
Lines, planes, and subspaces.
Line. Through \(\mathbf p\) with direction \(\mathbf v\neq\mathbf0\text{:}\)
\begin{equation*}
\mathbf x=\mathbf p+t\mathbf v,\qquad t\in\mathbb R.
\end{equation*}
In \(\mathbb R^3\text{,}\) if all three direction components are nonzero, the symmetric form is
\begin{equation*}
\frac{x_1-p_1}{v_1}=\frac{x_2-p_2}{v_2}=\frac{x_3-p_3}{v_3}.
\end{equation*}
If a direction component is zero, retain the corresponding constant-coordinate equation instead of dividing by zero. See Definition 2.2.18.
Plane in three dimensions. Through \(\mathbf p\) with normal \(\mathbf n\neq\mathbf0\text{:}\) \(\mathbf n\cdot(\mathbf x-\mathbf p)=0\text{.}\) For \(\mathbf n=(a,b,c)\text{,}\) this is the scalar equation \(ax+by+cz=d\text{,}\) where \(d=\mathbf n\cdot\mathbf p\text{.}\) Equivalently, \(\mathbf x=\mathbf p+s\mathbf u+t\mathbf v\text{,}\) with \(s,t\in\mathbb R\text{,}\) for two independent directions in the plane. See Definition 2.2.14.
Plane intersections. Combine the plane equations into one system and row-reduce its augmented matrix. Two planes with nonparallel normals meet in a line; planes with parallel normals either coincide or have no intersection. See Warning 2.2.23.
Subspace test. A subset of \(\mathbb R^n\) is a subspace if it contains \(\mathbf0\) and is closed under addition and scalar multiplication. Homogeneous solution sets are subspaces. See Definition 2.3.1.
Span. \(\operatorname{span}\{\mathbf v_1,\ldots,\mathbf v_k\}\) is the set of all linear combinations of the listed vectors. To test membership of \(\mathbf b\text{,}\) row-reduce the augmented matrix for \([\mathbf v_1\ \cdots\ \mathbf v_k]\mathbf c=\mathbf b\text{.}\) The span is a subspace; see Definition 2.3.22 and Theorem 2.3.26.
Independence, bases, and coordinates.
Linear independence. The list \(\mathbf v_1,\ldots,\mathbf v_k\) is independent exactly when
\begin{equation*}
c_1\mathbf v_1+\cdots+c_k\mathbf v_k=\mathbf0\quad\Longrightarrow\quad c_1=\cdots=c_k=0.
\end{equation*}
Test this by solving the homogeneous system with these vectors as columns. A nontrivial solution shows dependence; see Algorithm 2.4.8.
Basis and dimension. A basis is an independent spanning set. Every basis of a finite-dimensional subspace has the same number of vectors, its dimension. The empty set is a basis of \(\{\mathbf0\}\text{,}\) which has dimension \(0\text{.}\) See Definition 2.4.13 and Theorem 2.4.14.
Basis tests and construction. If \(\dim(U)=d\text{,}\) then a set of \(d\) vectors in \(U\) is a basis exactly when it is independent, and exactly when it spans \(U\text{.}\) An independent set in \(U\) can be extended to a basis, and a spanning set for \(U\) can be reduced to a basis. See Fact 2.4.19 and Basis and dimension.
Comparing dimensions. If \(U\subseteq W\) are subspaces, then \(\dim(U)\leq\dim(W)\text{.}\) If their dimensions are equal, then \(U=W\text{.}\)
Coordinates. For an ordered basis \(B=(\mathbf b_1,\ldots,\mathbf b_k)\) of \(U\) and a vector \(\mathbf v\in U\text{,}\)
\begin{equation*}
\mathbf v=c_1\mathbf b_1+\cdots+c_k\mathbf b_k,\qquad C_B(\mathbf v)=\begin{bmatrix}c_1\\\vdots\\c_k\end{bmatrix}.
\end{equation*}
Solve for the unique coefficient vector. Its entries depend on the order of the basis, and \(C_B(\mathbf b_i)=\mathbf e_i\text{.}\) See Definition 2.4.29.
Fundamental spaces, rank, and nullity.
Image, column, row, and null spaces. Let \(A\in\mathbb R^{m\times n}\text{.}\) Its reachable outputs form the image space, which is also the span of its columns:
\begin{equation*}
\operatorname{im}(A)=\operatorname{col}(A)=\{A\mathbf x:\mathbf x\in\mathbb R^n\}\subseteq\mathbb R^m.
\end{equation*}
Its row space \(\operatorname{row}(A)\) is the span of its rows, a subspace of \(\mathbb R^n\text{.}\) Its forgotten input directions form
\begin{equation*}
\operatorname{null}(A)=\{\mathbf x\in\mathbb R^n:A\mathbf x=\mathbf0\}.
\end{equation*}
See Definition 2.3.13 and Definition 2.4.33.
Basis calculations. Locate pivot columns using REF or RREF, then take those columns from the original \(A\) for a column-space basis. Row operations preserve row space, so the nonzero rows of REF or RREF give a row-space basis. Basic solutions of \(A\mathbf x=\mathbf0\) give a null-space basis. See Theorem 2.4.39 and Theorem 2.4.53.
A basis for a span. Place the given generators in the rows of a matrix, reduce to REF, and take the nonzero rows as a basis, as in Activity 2.4.38. If the requested basis must be a subset of the original generators, place them in columns instead and select the original pivot columns.
Rank and nullity. Rank is the common dimension of the column and row spaces, while nullity is the dimension of the null space:
\begin{equation*}
\operatorname{rank}(A)=\dim\operatorname{col}(A)=\dim\operatorname{row}(A),
\end{equation*}
\begin{equation*}
\operatorname{nullity}(A)=\dim\operatorname{null}(A).
\end{equation*}
See Definition 2.4.42 and Definition 2.4.49. To compute rank, count the pivots in REF or RREF, by Theorem 2.4.45. Nullity equals the number of free variables in \(A\mathbf x=\mathbf0\text{.}\)
Rank–nullity. For \(A\in\mathbb R^{m\times n}\text{,}\) Theorem 2.4.53 gives
\begin{equation*}
\operatorname{rank}(A)+\operatorname{nullity}(A)=n.
\end{equation*}
The number of columns, \(n\text{,}\) is the input dimension: transmitted dimensions plus forgotten input dimensions account for all input dimensions.
Existence and uniqueness. \(A\mathbf x=\mathbf b\) is consistent exactly when \(\mathbf b\in\operatorname{col}(A)\text{.}\) For a consistent system, the solution is unique exactly when \(\operatorname{null}(A)=\{\mathbf0\}\text{.}\)
Different inputs, same output.
\begin{equation*}
A\mathbf x=A\mathbf y\quad\Longleftrightarrow\quad\mathbf x-\mathbf y\in\operatorname{null}(A).
\end{equation*}
If \(\mathbf z\neq\mathbf0\) lies in the null space, then varying \(t\in\mathbb R\) in \(\mathbf x+t\mathbf z\) produces distinct inputs with the same output as \(\mathbf x\text{.}\) See Activity 2.4.50.
Redundant features. A linear dependence among the columns of a data matrix \(X\) gives a nonzero vector \(\mathbf z\) with \(X\mathbf z=\mathbf0\text{.}\) Consequently,
\begin{equation*}
X(\mathbf c+t\mathbf z)=X\mathbf c,\qquad t\in\mathbb R.
\end{equation*}
Different coefficient vectors can make identical predictions, so those predictions do not determine individual coefficients uniquely. Rank counts the independent feature directions retained; nullity counts independent coefficient changes invisible to the data matrix. See Example 2.4.57 and Activity 2.6.4.
Inverses and determinants.
Inverse. A square matrix \(A\) is invertible (nonsingular) if there is a matrix \(A^{-1}\) with
\begin{equation*}
AA^{-1}=A^{-1}A=I_n.
\end{equation*}
The inverse is unique and reverses the map: \(A\mathbf x=\mathbf b\) has the solution \(\mathbf x=A^{-1}\mathbf b\text{.}\) See Definition 2.5.1 and Theorem 2.5.11.
Invertibility criteria. For a square \(n\times n\) matrix \(A\text{,}\) each of the following is equivalent to invertibility:
-
The columns are independent, or equivalently span \(\mathbb R^n\text{;}\) equivalently they form a basis of \(\mathbb R^n\text{.}\)
-
The rows are independent, or equivalently span \(\mathbb R^n\text{;}\) equivalently they form a basis of \(\mathbb R^n\text{.}\)
-
\(\operatorname{null}(A)=\{\mathbf0\}\text{,}\) so \(A\mathbf x=\mathbf0\) has only the trivial solution.
-
\(\det(A)\neq0\text{.}\)
Two-by-two determinant and inverse.
\begin{equation*}
\det\begin{bmatrix}a\amp b\\c\amp d\end{bmatrix}=ad-bc.
\end{equation*}
When \(ad-bc\neq0\text{,}\)
\begin{equation*}
\begin{bmatrix}a\amp b\\c\amp d\end{bmatrix}^{-1}=\frac1{ad-bc}\begin{bmatrix}d\amp-b\\-c\amp a\end{bmatrix}.
\end{equation*}
See Theorem 2.5.13.
Minors and cofactors. The minor \(\operatorname{minor}(A)_{ij}\) is the determinant of the submatrix obtained by deleting row \(i\) and column \(j\text{,}\) and
\begin{equation*}
\operatorname{cof}(A)_{ij}=(-1)^{i+j}\operatorname{minor}(A)_{ij}.
\end{equation*}
Cofactor expansion. For a fixed row \(i\text{,}\)
\begin{equation*}
\det(A)=\sum_{j=1}^n a_{ij}\operatorname{cof}(A)_{ij}.
\end{equation*}
For a fixed column \(j\text{,}\)
\begin{equation*}
\det(A)=\sum_{i=1}^n a_{ij}\operatorname{cof}(A)_{ij}.
\end{equation*}
All choices give the same determinant. Choose a row or column with many zeros to reduce the computation; see Fact 2.5.33.
Determinant rules. Swapping two rows changes the sign; scaling one row by \(c\) scales the determinant by \(c\text{;}\) adding a multiple of a different row leaves it unchanged. See Theorem 2.5.45. A zero row or column, or two equal rows or columns, gives determinant zero. For a triangular matrix, multiply the diagonal entries. For \(n\times n\) matrices,
\begin{equation*}
\det(AB)=\det(A)\det(B),\qquad \det(A^T)=\det(A).
\end{equation*}
Scaling the whole matrix scales all \(n\) rows:
\begin{equation*}
\det(cA)=c^n\det(A).
\end{equation*}
See Fact 2.5.43.
Inverse identities. If \(A,B\) are invertible matrices of the same size, then \((AB)^{-1}=B^{-1}A^{-1}\) and \(\det(A^{-1})=1/\det(A)\text{.}\) See Theorem 2.5.7 and Theorem 2.5.50.
Optional: computing an inverse by row reduction. Row-reduce \([A\mid I_n]\) to \([I_n\mid A^{-1}]\text{.}\) If the completed reduction does not give \(I_n\) on the left, \(A^{-1}\) does not exist. This general computation method is the optional material in Optional: Calculating Inverses; see Algorithm 2.5.56.
Subsection Common mistakes
-
Counting the augmented column as a variable column, missing an inconsistent row, or omitting free parameters. Only columns to the left of the vertical line represent variables.
-
Using columns of REF or RREF for a column-space basis. Use the corresponding columns of the original matrix; reduced nonzero rows are appropriate for a row-space basis.
-
Confusing row space and column space. For an \(m\times n\) matrix, they lie in \(\mathbb R^n\) and \(\mathbb R^m\text{,}\) respectively, even though their dimensions agree.
-
Assuming a spanning set is a basis without checking independence, or assuming three or more pairwise nonparallel vectors must be independent. See Warning 2.4.7.
-
Calling a translated line or plane a subspace without checking the origin and closure. In particular, the solution set of \(A\mathbf x=\mathbf b\) with \(\mathbf b\neq\mathbf0\) does not contain the origin.
-
Treating every relationship between features as linear redundancy. For example, area is the product of width and height, not a linear combination of them; see Warning 2.4.59.
-
Concluding that identical predictions imply identical coefficients. A nonzero null-space direction permits coefficient changes with no change in predictions.
-
Applying square-matrix inverse or determinant tests to a rectangular matrix, or confusing the determinant effect of scaling one row with scaling the entire matrix.
-
Reading a zero-based pivot index as a one-based mathematical column number. Add one when describing the column number, but use the original zero-based index in Python; see Activity 2.4.41.
-
Treating
A.nullspace()as a single vector. It returns a list of basis column matrices; select a vector from a nonempty list before multiplying. An empty list means the null space is \(\{\mathbf0\}\text{;}\) see Activity 2.4.56.
Subsection Connections
-
Unit 3 reuses normal vectors for tangent planes and null spaces for directions with zero change under a local linear map.
-
Unit 5 uses determinants to find eigenvalues, null spaces to find eigenspaces, and bases and invertibility to diagonalize matrices.
-
Unit 7 extends subspaces, bases, dimension, and coordinates to abstract vector spaces, including spaces of polynomials.
