Subspaces

Section 2.3 Subspaces

Subsection Definition and first examples

The outputs a matrix can “reach” form what is known as a subspace. The directions a matrix “forgets” also form a subspace. The definitions below make this precise.

🔗

We are already familiar with geometric spaces living inside bigger ones, like lines lying in the plane, or planes lying in 3-space. We are now going to formalize this notion algebraically. Recall the definition of the set \(\R^n\) in Definition 1.1.1.

🔗

Definition 2.3.1. Subspaces.

A set \(U\) of vectors in \(\R^n\) is called a subspace of \(\R^n\) if it satisfies the following properties:

The zero vector (written as \(\mathbf{0}\)) is in \(U\text{.}\)

🔗
If \(\mathbf{x}\) is in \(U\) and \(y\) is in \(U\text{,}\) then \(\mathbf{x} + \mathbf{y}\) is in \(U\text{.}\) This means \(U\) is closed under addition.

🔗
If \(\mathbf{x}\) is in \(U\text{,}\) then \(a \mathbf{x}\) is in \(U\) for every scalar \(a\text{.}\) This means \(U\) is closed under scalar multiplication.

🔗

🔗

Note 2.3.2. Subspaces are closed under linear combinations.

Let \(U\) be a subspace. If \(\mathbf{u},\mathbf{v}\in U\) and \(a,b\in\R\text{,}\) closure under scalar multiplication gives \(a\mathbf{u}\in U\) and \(b\mathbf{v}\in U\text{.}\) Closure under addition then gives

\begin{equation*} a\mathbf{u}+b\mathbf{v}\in U. \end{equation*}

🔗

More generally, if \(\mathbf{u}_1,\ldots,\mathbf{u}_k\in U\) and \(c_1,\ldots,c_k\in\R\text{,}\) each vector \(c_i\mathbf{u}_i\) lies in \(U\text{.}\) Adding these vectors one at a time shows that

\begin{equation*} c_1\mathbf{u}_1+\cdots+c_k\mathbf{u}_k\in U. \end{equation*}

Thus every finite linear combination of vectors in a subspace remains in that subspace.

🔗

Example 2.3.3.

The whole set \(\R^n\) is a subspace of itself.

🔗

Example 2.3.4.

The set containing only the zero vector, i.e., the set \(\{ \mathbf{0} \}\text{,}\) is a subspace of \(\R^n\) (called the trivial subspace or zero subspace).

🔗

Activity 2.3.5. The zero vector and subspace conditions (U2-LO4).

Verify that \(\{ \mathbf{0} \}\) is a subspace of \(\R^n\text{.}\)

🔗

Solution.

To verify that \(\{ \mathbf{0} \}\) is a subspace, we must verify three properties:

Contains the zero vector: 🔗: The zero \(\mathbf{0}\) is evidently an element of \(\{ \mathbf{0} \}\text{.}\)
Closure under addition: 🔗: If \(\mathbf{x}, \mathbf{y} \in \{ \mathbf{0} \}\text{,}\) then \(\mathbf{x} = 0\) and \(\mathbf{y} = 0\text{,}\) so \(\mathbf{x} + \mathbf{y} = 0\text{,}\) which is an element of \(\{ \mathbf{0} \}\text{.}\)
Closure under scalar multiplication: 🔗: If \(\mathbf{x} \in \{ \mathbf{0} \}\) and \(a\) is a scalar, then \(\mathbf{x} = 0\text{,}\) and \(a\mathbf{x} = a \mathbf{0} = 0\) is in \(\{ \mathbf{0} \}\text{.}\)

Since \(\{ \mathbf{0} \}\) satisfies all three of the necessary conditions, it is a subspace.

🔗

Definition 2.3.6. Proper Subspaces.

Any subspace of \(\R^n\) other than \(\R^n\) or \(\{ 0 \}\) is a proper subspace of \(\R^n\text{.}\)

🔗

Example 2.3.7.

Planes and lines through the origin in \(\R^3\) are all subspaces of \(\R^3\text{.}\)

🔗

Example 2.3.8. Lines and planes through the origin (U2-LO4).

(a)

Verify that planes through the origin in \(\R^3\) are all proper subspaces of \(\R^3\text{.}\)

🔗

Solution.

Let \(R\) be a plane through the origin. Then there exist scalars \(a,b,c\text{,}\) not all zero, such that \(R\) is the set of points \((x,y,z)\) satisfying the equation \(ax + by + cz = 0\text{.}\) The vector

\begin{equation*} \mathbf{n} = \begin{bmatrix} a \\ b \\ c \end{bmatrix} \end{equation*}

is a normal vector for the plane, and we can write \(R\) as the set of vectors \(\mathbf{v} \in \R^3\) such that \(\mathbf{n} \cdot \mathbf{v} = 0\text{.}\)

🔗

Let us verify that \(R\) satisfies the conditions to be a subspace:

Contains the zero vector 🔗: \(\mathbf{0} \in R\) because \(\mathbf{n} \cdot \mathbf{0} = 0\text{.}\)
🔗
Closure under addition 🔗: If \(\mathbf{v} \in R\) and \(\mathbf{w} \in R\) then to verify \(\mathbf{v} + \mathbf{w} \in R\) we must argue that \(\mathbf{n} \cdot ( \mathbf{v} + \mathbf{w}) = 0\text{.}\) Indeed,

\begin{equation*} \mathbf{n} \cdot (\mathbf{v} + \mathbf{w}) = \mathbf{n} \cdot \mathbf{v} + \mathbf{n} \cdot \mathbf{w} = 0\text{.} \end{equation*}

🔗
Closure under scalar multiplication 🔗: If \(\mathbf{v} \in R\text{,}\) and \(a\) is a scalar, to verify \(a \mathbf{v} \in R\) we must argue that \(\mathbf{n} \cdot (a \mathbf{v}) = 0\text{.}\) Indeed,

\begin{equation*} \mathbf{n} \cdot (a \mathbf{v}) = a ( \mathbf{n} \cdot \mathbf{v}) = a (0) = 0\text{.} \end{equation*}

🔗

We have verified the three properties required for \(R\) to be a subspace.

🔗

(b)

Verify that lines through the origin in \(\R^3\) are subspaces of \(\R^3\text{.}\)

🔗

Solution.

Let \(L\) be a line through the origin in \(\R^3\text{.}\) If \(L\) has direction vector \(\mathbf{d}\text{,}\) then \(L = \{ t \mathbf{d}: t \in \R \}\text{,}\) i.e., the line \(L\) consists of all scalar multiples of \(\mathbf{d}\text{.}\)

🔗

Let us verify that \(L\) satisfies the conditions to be a subspace:

Contains the zero vector 🔗: \(\mathbf{0} \in L\) because \(\mathbf{0} = 0 \mathbf{d}\text{.}\)
🔗
Closure under addition 🔗: Suppose \(\mathbf{v} \in L\) and \(\mathbf{w} \in L\text{.}\) Then \(\mathbf{v}\) and \(\mathbf{w}\) are both scalar multiples of \(\mathbf{d}\text{,}\) i.e., \(\mathbf{v} = t_1 \mathbf{d}\) and \(\mathbf{w} = t_2 \mathbf{d}\text{.}\) Then

\begin{equation*} \mathbf{v} + \mathbf{w} = t_1 \mathbf{d} + t_2 \mathbf{d} = (t_1 + t_2) \mathbf{d}\text{.} \end{equation*}

Thus \(\mathbf{v} + \mathbf{w}\) is a scalar multiple of \(\mathbf{d}\text{,}\) and so \(\mathbf{v} + \mathbf{w} \in L\text{.}\)

🔗
Closure under scalar multiplication 🔗: If \(\mathbf{v} \in L\text{,}\) and \(a\) is a scalar, then we may write \(\mathbf{v} = t \mathbf{d}\text{,}\) and then

\begin{equation*} a \mathbf{v} = a (t\mathbf{d}) = (at) \mathbf{d}\text{.} \end{equation*}

So \(a \mathbf{v}\) is a scalar multiple of \(\mathbf{d}\text{,}\) and thus \(a \mathbf{v} \in L\text{.}\)

🔗

We have verified the three properties required for \(L\) to be a subspace.

🔗

Note 2.3.9.

Lines and planes which do not pass through the origin in \(\R^3\) are not subspaces of \(\R^3\text{.}\) Nonetheless, if \(\Pi\) is a plane, and we consider the set of displacement vectors

\begin{equation*} \Delta\Pi = \{ \mathbf{v}_1 - \mathbf{v}_0 : \mathbf{v}_0,\mathbf{v}_1 \in \Pi \}\text{,} \end{equation*}

then \(\Delta\Pi\) forms a plane passing through the origin, and so is a subspace of \(\R^3\text{.}\) A similar result holds for sets of displacement vectors given by a line not passing through the origin.

🔗

Example 2.3.10. A product condition on coordinates (U2-LO4).

Show that the set

\begin{equation*} U = \left\{ \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} : x_1 x_2 = 0 \right\} \end{equation*}

is not a subspace of \(\R^2\text{.}\)

🔗

Solution.

It suffices to show one of the conditions required to be a subspace does not hold. \(U\) does contain the zero vector, and \(is\) closed under scalar multiplication. But \(U\) is not closed under addition. Define

\begin{equation*} \mathbf{x} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \quad\text{and}\quad \mathbf{y} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \text{.} \end{equation*}

Then \(\mathbf{x} \in U\) and \(\mathbf{y} \in U\text{.}\) But

\begin{equation*} \mathbf{x} + \mathbf{y} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} \end{equation*}

is not in \(U\text{,}\) since \((1)(1) \neq 0\text{.}\) Since \(U\) fails to be closed under addition, it is not a subspace of \(\R^2\text{.}\)

🔗

Activity 2.3.11. A set described by two parameters (U2-LO4).

Let \(W\) be the set of all vectors in \(\R^3\) of the form

\begin{equation*} \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix}\text{,} \end{equation*}

where \(r,s \in \R\text{.}\) Is \(W\) a proper subspace of \(\R^3\text{?}\)

🔗

Solution.

We check whether the required properties hold in order to be a subspace:

Contains the zero vector: 🔗: The zero vector is in \(W\text{,}\) since
\begin{equation*} \mathbf{0} = \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix} \end{equation*}
when \(r = 0\) and \(s = 0\text{.}\)
Closed under addition: 🔗: If
\begin{equation*} \mathbf{v} = \begin{bmatrix} r_1 \\ s_1 \\ r_1 + 2s_1 \end{bmatrix} \quad\text{and}\quad \mathbf{w} = \begin{bmatrix} r_2 \\ s_2 \\ r_2 + 2s_2 \end{bmatrix} \end{equation*}
Then
\begin{align*} \mathbf{v} + \mathbf{w} \amp = \begin{bmatrix} r_1 + r_2 \\ s_1 + s_2 \\ r_1 + 2s_1 + r_2 + 2s_2 \end{bmatrix}\\ \amp = \begin{bmatrix} r_1 + r_2 \\ s_1 + s_2 \\ (r_1 + r_2) + 2(s_1 + s_2) \end{bmatrix}\\ \amp = \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix}\text{,} \end{align*}
where \(r = r_1 + r_2\) and \(s = s_1 + s_2\text{.}\) Thus \(\mathbf{v} + \mathbf{w} \in W\text{.}\)
Closed under scalar multiplication: 🔗: If
\begin{equation*} \mathbf{v} = \begin{bmatrix} r_1 \\ s_1 \\ r_1 + 2s_1 \end{bmatrix}\text{,} \end{equation*}
and \(c\) is a scalar, then
\begin{equation*} c \mathbf{v} = \begin{bmatrix} cr_1 \\ cs_1 \\ c(r_1 + 2s_1) \end{bmatrix} = \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix}\text{,} \end{equation*}
with \(r = cr_1\) and \(s = cs_1\text{.}\) Thus \(c \mathbf{v} \in W\text{.}\)

Thus \(W\) is a subspace of \(\R^3\text{.}\)

🔗

To check that \(W\) is a proper subspace of \(\R^3\text{,}\) we must show that \(W \neq \R^3\) and \(W \neq \{ \mathbf{0} \}\text{.}\) The vector

\begin{equation*} \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} \end{equation*}

is not in \(W\text{,}\) since if it were of the form

\begin{equation*} \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix}\text{,} \end{equation*}

then \(r = 1\) and \(s = 1\text{,}\) and the third entry would have to be \(3\text{.}\) Thus \(W\) does not contain every vector, and thus cannot be equal to \(\R^3\text{.}\) But it also contains some non-zero vectors, such as

\begin{equation*} \begin{bmatrix} 1\\ 1 \\ 3 \end{bmatrix}\text{,} \end{equation*}

and so \(W \neq \{ \mathbf{0} \}\text{.}\)

🔗

Thus \(W\) is a proper subspace of \(\R^3\text{.}\)

🔗

Note 2.3.12.

To show a set is a subspace, you tend to need to parameterize all elements of the set in order to show they all satisfy some property (closure under addition or scalar multiplication), and so your argument should use variables. To show a set is not a subspace, you need to find one or more counterexamples that show that some property does not hold for all elements; a single explicit counterexample usually suffices.

🔗

Subsection Null space and image space

The following important examples are representative of two canonical ways of describing a subspace.

🔗

Definition 2.3.13. Null Space and Image Space.

Let \(A\) be an \(m \times n\) matrix. The null space of \(A\text{,}\) denoted \(\nulls(A)\text{,}\) is the set of vectors \(\mathbf{x}\) in \(\R^n\) which are solutions to the equation \(A \mathbf{x} = \mathbf{0}\text{,}\) i.e.,

\begin{equation*} \nulls(A) = \{ \mathbf{x} \in \R^n : A \mathbf{x} = \mathbf{0} \}\text{.} \end{equation*}

The image space of \(A\text{,}\) denoted \(\im(A)\text{,}\) is the set of vectors \(\mathbf{y}\) in \(\R^m\) such that \(A\mathbf{x} = \mathbf{y}\) has a solution, i.e.,

\begin{equation*} \im(A) = \{ A\mathbf{x} : \mathbf{x} \in \R^n \}\text{.} \end{equation*}

🔗

Note 2.3.14.

The null space lives in the input space. It records input directions that \(A\) sends to \(\mathbf{0}\text{.}\)

🔗

The image space lives in the output space. It records outputs that \(A\) can reach.

🔗

Table 2.3.15.

🔗

Object	Lives in	How to read it
\(\operatorname{null}(A)\)	\(\mathbb R^n\)	input directions \(A\) sends to \(\mathbf{0}\)
\(\operatorname{im}(A)\)	\(\mathbb R^m\)	outputs \(A\) can reach

Two set diagrams show the null space inside the input space mapping to zero, and the whole input space mapping onto the image inside the output space. — Figure 2.3.16. The null space consists of inputs sent to the zero vector. The image consists of all outputs reached from the input space.
🔗

Activity 2.3.17. Revisiting the height map with subspace language (U2-LO3).

Return to Activity 2.1.1. Let

\begin{equation*} P= \begin{bmatrix} 1&0&0\\ 0&1&0 \end{bmatrix}. \end{equation*}

Describe the output subspace and the subspace of input directions that \(P\) sends to \(\mathbf{0}\text{.}\)

🔗

Solution.

For

\begin{equation*} \mathbf{x} = \begin{bmatrix} x\\ y\\ z \end{bmatrix}, \qquad P\mathbf{x} = \begin{bmatrix} x\\ y \end{bmatrix}. \end{equation*}

🔗

As \(x\) and \(y\) vary freely, every vector in \(\mathbb R^2\) occurs as an output. Therefore

\begin{equation*} \operatorname{im}(P)=\mathbb R^2. \end{equation*}

🔗

The directions that \(P\) sends to \(\mathbf{0}\) are the solutions to \(P\mathbf{x}=\mathbf{0}\text{.}\) This requires

\begin{equation*} x=0,\qquad y=0, \end{equation*}

while \(z\) is free. Thus

\begin{equation*} \operatorname{null}(P) = \left\{ \begin{bmatrix} 0\\ 0\\ t \end{bmatrix} : t\in\mathbb R \right\}. \end{equation*}

This subspace records the height direction.

🔗

Theorem 2.3.18.

For any \(m \times n\) matrix, the null space of \(A\) is a subspace of \(\R^n\text{,}\) and the image space of \(A\) is a subspace of \(\R^m\text{.}\)

🔗

Why is this true?.

We start by showing the null space of \(A\) is a subspace. Recall the properties discussed in Theorem 1.3.5:

Contains the zero vector 🔗: \(\mathbf{0} \in \nulls(A)\) because \(\mathbf{A} \mathbf{0} = \mathbf{0}\text{.}\)
🔗
Closure under addition 🔗: Suppose \(\mathbf{v} \in \nulls(A)\) and \(\mathbf{w} \in \nulls(A)\text{.}\) Then \(A \mathbf{v} = \mathbf{0}\) and \(A \mathbf{w} = \mathbf{0}\text{.}\) So

\begin{equation*} A (\mathbf{v} + \mathbf{w}) = A \mathbf{v} + A \mathbf{w} = \mathbf{0} + \mathbf{0} = \mathbf{0}\text{.} \end{equation*}

Thus \(\mathbf{v} + \mathbf{w} \in \nulls(A)\text{.}\)

🔗
Closure under scalar multiplication 🔗: If \(\mathbf{v} \in \nulls(A)\text{,}\) and \(a\) is a scalar, then

\begin{equation*} A(a \mathbf{v}) = a (A \mathbf{v}) = a \mathbf{0} = \mathbf{0} \text{.} \end{equation*}

Thus \(a \mathbf{v} \in \nulls(A)\text{.}\)

🔗

So \(\nulls(A)\) is a subspace of \(\R^n\text{.}\)

🔗

Next, we show that \(\im(A)\) is a subspace.

Contains the zero vector 🔗: \(\mathbf{0} \in \im(A)\) because \(\mathbf{A} \mathbf{0} = \mathbf{0}\text{.}\)
🔗
Closure under addition 🔗: Suppose \(\mathbf{v} \in \im(A)\) and \(\mathbf{w} \in \im(A)\text{.}\) Then the equations \(A\mathbf{x} = \mathbf{v}\) and \(A \mathbf{x} = \mathbf{w}\) both have solutions, i.e., there exists \(\mathbf{x}_1\) and \(\mathbf{x}_2\) such that \(A\mathbf{x}_1 = \mathbf{v}\) and \(A\mathbf{x}_2 = \mathbf{w}\text{.}\) Then

\begin{equation*} A(\mathbf{x}_1 + \mathbf{x}_2) = A\mathbf{x}_1 + A\mathbf{x}_2 = \mathbf{v} + \mathbf{w}\text{,} \end{equation*}

so the equation \(A \mathbf{x} = \mathbf{v} + \mathbf{w}\) has a solution, and so \(\mathbf{v} + \mathbf{w} \in \im(A)\text{.}\)

🔗
Closure under scalar multiplication 🔗: If \(\mathbf{v} \in \im(A)\text{,}\) and \(a\) is a scalar, then we may write \(\mathbf{v} = A\mathbf{x}_1\) for some \(\mathbf{x}_1\text{.}\) But then

\begin{equation*} A(a\mathbf{x}_1) = a (A\mathbf{x}_1) = a \mathbf{v}\text{,} \end{equation*}

so the equation \(A\mathbf{x} = a \mathbf{v}\) has a solution, so that \(a \mathbf{v} \in \im(A)\text{.}\)

🔗

Thus \(\im(A)\) is a subspace of \(\R^m\text{.}\)

🔗

Warning 2.3.19.

The solution set of \(A\mathbf{x}=\mathbf{0}\) is a subspace. The solution set of \(A\mathbf{x}=\mathbf{b}\) for \(\mathbf{b}\neq\mathbf{0}\) is usually not a subspace; when it is nonempty, it is a shifted copy of \(\nulls(A)\text{.}\)

🔗

Fact 2.3.20. General solution of a linear system.

Let \(A\) be an \(m\times n\) matrix. If \(\mathbf{x}_p\) is one solution of \(A\mathbf{x}=\mathbf{b}\text{,}\) then

\begin{equation*} \{\mathbf{x}\in\mathbb R^n:A\mathbf{x}=\mathbf{b}\}=\mathbf{x}_p+\nulls(A). \end{equation*}

That is, the solutions are exactly the vectors \(\mathbf{x}_p+\mathbf{z}\) with \(\mathbf{z}\in\nulls(A)\text{.}\)

🔗

Why is this true?.

Since \(A\mathbf{x}_p=\mathbf{b}\text{,}\) for any \(\mathbf{x}\in\mathbb R^n\text{,}\)

\begin{align*} A\mathbf{x}=\mathbf{b}\amp\Longleftrightarrow A(\mathbf{x}-\mathbf{x}_p)=\mathbf{0}\\ \amp\Longleftrightarrow \mathbf{x}-\mathbf{x}_p\in\nulls(A). \end{align*}

Thus \(A\mathbf{x}=\mathbf{b}\) exactly when \(\mathbf{x}=\mathbf{x}_p+\mathbf{z}\) for some \(\mathbf{z}\in\nulls(A)\text{.}\)

🔗

From here on, homogeneous equations play a special role. To solve \(A\mathbf{x}=\mathbf{0}\) by row reduction, we could write the augmented matrix \([A\mid \mathbf{0}]\text{.}\) Since the right-hand side stays zero throughout the row operations, the coefficient matrix \(A\) carries the essential information.

🔗

Subsection Spanning Sets

Even though subspaces of \(\R^n\) can have infinitely many vectors, it is possible to represent them with a finite amount of data.

🔗

Activity 2.3.21. Writing a parametric set as a span (U2-LO4).

Recall from Activity 2.3.11 that

\begin{equation*} W = \left\{ \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix} : r,s \in \R \right\}. \end{equation*}

Show that there are two vectors (describe them explicitly), so that every vector in \(W\) can be written as a linear combination of those two vectors.

🔗

Solution.

We may write

\begin{equation*} \begin{bmatrix} r \\ s \\ r + 2s \end{bmatrix} = r \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} + s \begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}\text{.} \end{equation*}

Thus every vector in \(W\) can be written as a linear combination of

\begin{equation*} \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}\quad\text{and}\quad \begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}\text{.} \end{equation*}

🔗

Definition 2.3.22. The Span of a Set of Vectors.

The set of all linear combinations of a set of vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\) in \(\R^n\) is called the span of the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\text{,}\) and is denoted

\begin{equation*} \spans \{ \mathbf{x}_1, \dots, \mathbf{x}_k \} = \{ t_1 \mathbf{x}_1 + \cdots + t_k \mathbf{x}_k : t_1,\dots,t_k \in \R \}\text{.} \end{equation*}

If \(V = \spans \{ \mathbf{x}_1, \dots, \mathbf{x}_k \}\text{,}\) we say that \(V\) is spanned by the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\text{,}\) and that the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\) span the space \(V\text{.}\)

🔗

Example 2.3.23.

Recall the standard basis \(\{ \mathbf{e}_1, \dots, \mathbf{e}_n \}\) of \(\R^n\) from Definition 1.3.14. Note that if

\begin{equation*} \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}\text{,} \end{equation*}

then

\begin{equation*} \mathbf{x} = x_1 \mathbf{e}_1 + x_2 \mathbf{e}_2 + \dots + x_n \mathbf{e}_n\text{.} \end{equation*}

This means that \(\R^n = \spans \{ \mathbf{e}_1,\dots,\mathbf{e}_n \}\text{.}\)

🔗

For an image space, spanning vectors come from the columns of the matrix. If

\begin{equation*} A=[\mathbf{a}_1\ \mathbf{a}_2\ \cdots\ \mathbf{a}_n] \end{equation*}

and

\begin{equation*} \mathbf{x} = \begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{bmatrix}, \end{equation*}

then

\begin{equation*} A\mathbf{x} = x_1\mathbf{a}_1+x_2\mathbf{a}_2+\cdots+x_n\mathbf{a}_n. \end{equation*}

Thus every output \(A\mathbf{x}\) is a linear combination of the columns of \(A\text{.}\)

🔗

Activity 2.3.24. A spanning set for an image space (U2-LO3, U2-LO4).

Find a spanning set for the image space of the matrix

\begin{equation*} A = \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp -1 \amp -3 \amp 2 \\ 2 \amp 1 \amp 7 \amp -4 \end{bmatrix} \text{.} \end{equation*}

🔗

Solution.

Let

\begin{equation*} \mathbf{x} = \begin{bmatrix} t_1\\ t_2\\ t_3\\ t_4 \end{bmatrix}. \end{equation*}

Then

\begin{equation*} A\mathbf{x} = t_1 \begin{bmatrix} 1\\ 0\\ 2 \end{bmatrix} + t_2 \begin{bmatrix} -2\\ -1\\ 1 \end{bmatrix} + t_3 \begin{bmatrix} -4\\ -3\\ 7 \end{bmatrix} + t_4 \begin{bmatrix} 3\\ 2\\ -4 \end{bmatrix}. \end{equation*}

🔗

Therefore the image space is spanned by the columns of \(A\text{:}\)

\begin{equation*} \operatorname{im}(A) = \operatorname{span} \left\{ \begin{bmatrix} 1\\ 0\\ 2 \end{bmatrix}, \begin{bmatrix} -2\\ -1\\ 1 \end{bmatrix}, \begin{bmatrix} -4\\ -3\\ 7 \end{bmatrix}, \begin{bmatrix} 3\\ 2\\ -4 \end{bmatrix} \right\}. \end{equation*}

🔗

This is a spanning set. It is not claiming to be minimal.

🔗

Activity 2.3.25. A spanning set for a null space (U2-LO3, U2-LO4).

Find a spanning set for the null space of the matrix

\begin{equation*} A = \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp -1 \amp -3 \amp 2 \\ 2 \amp 1 \amp 7 \amp -4 \end{bmatrix}\text{.} \end{equation*}

🔗

Practice this activity in The Elimination Scratchpad

The Elimination Scratchpad

🔗

Solution.

To solve this problem, we begin by finding the basic solutions to the linear system \(A \mathbf{x} = 0\text{,}\) where

\begin{equation*} \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix}\text{.} \end{equation*}

Using the Gaussian elimination algorithm, we row reduce:

\begin{align*} \amp \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp -1 \amp -3 \amp 2 \\ 2 \amp 1 \amp 7 \amp -4 \end{bmatrix}\\ \amp \xrightarrow{R_3 \to R_3 - 2R_1} \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp -1 \amp -3 \amp 2 \\ 0 \amp 5 \amp 15 \amp -10 \end{bmatrix}\\ \amp \xrightarrow{R_3 \to R_3 + 5R_2} \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp -1 \amp -3 \amp 2 \\ 0 \amp 0 \amp 0 \amp 0 \end{bmatrix}\\ \amp \xrightarrow{R_2 \to -R_2} \begin{bmatrix} 1 \amp -2 \amp -4 \amp 3 \\ 0 \amp 1 \amp 3 \amp -2 \\ 0 \amp 0 \amp 0 \amp 0 \end{bmatrix}\\ \amp \xrightarrow{R_1 \to R_1 + 2R_2} \begin{bmatrix} 1 \amp 0 \amp 2 \amp -1 \\ 0 \amp 1 \amp 3 \amp -2 \\ 0 \amp 0 \amp 0 \amp 0 \end{bmatrix} \end{align*}

The free variables are \(x_3\) and \(x_4\text{.}\) Setting \(x_3 = s\) and \(x_4 = t\text{,}\) we obtain that \(A\mathbf{x} = 0\) when \(x_1 = -2s + t\) and \(x_2 = -3s + 2t\text{.}\) Thus a general solution to this equation is given by

\begin{equation*} \begin{bmatrix} -2s + t \\ -3s + 2t \\ s \\ t \end{bmatrix} = s \begin{bmatrix} -2 \\ -3 \\ 1 \\ 0 \end{bmatrix} + t \begin{bmatrix} 1 \\ 2 \\ 0 \\ 1 \end{bmatrix}\text{.} \end{equation*}

Thus the null space is spanned by

\begin{equation*} \mathbf{v}_1 = \begin{bmatrix} -2 \\ -3 \\ 1 \\ 0 \end{bmatrix} \quad\text{and}\quad \mathbf{v}_2 = \begin{bmatrix} 1 \\ 2 \\ 0 \\ 1 \end{bmatrix} \text{.} \end{equation*}

🔗

To summarize: basic solutions of \(A\mathbf{x}=\mathbf{0}\) form a spanning set for \(\operatorname{null}(A)\text{,}\) and the columns of \(A\) form a spanning set for \(\operatorname{im}(A)\text{.}\)

🔗

Theorem 2.3.26.

Let \(U = \spans \{ \mathbf{x}_1,\dots,\mathbf{x}_k \}\) be a set in \(\R^n\text{.}\) Then

\(U\) is a subspace of \(\R^n\) containing each of the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\text{.}\)

🔗
If \(W\) is a subspace of \(\R^n\) and each of the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\) is in \(W\text{,}\) then \(U\) is a subset of \(W\text{,}\) i.e., \(U \subseteq W\text{.}\)

🔗

🔗

Why is this true?.

Define an \(n \times k\) matrix \(A\text{,}\) with the vectors \(\mathbf{x}_1,\dots,\mathbf{x}_k\) as columns. Then \(U = \im(A)\text{,}\) and is thus a subspace of \(\R^n\text{.}\) Since \(\mathbf{x}_i = A \mathbf{e}_i\) for \(1 \leq i \leq k\text{,}\) the set \(U\) contains each of the vectors \(\mathbf{x}_i\text{.}\)

🔗

Conversely, suppose \(W\) is a subspace of \(\R^n\) and \(\mathbf{x}_1,\dots,\mathbf{x}_k \in W\text{.}\) Since \(W\) is closed under addition and scalar multiplication, all of the linear combinations of \(\mathbf{x}_1,\dots,\mathbf{x}_k\) are elements of \(W\text{,}\) and so \(U \subseteq W\text{.}\)

🔗

Activity 2.3.27. Testing membership in a span (U2-LO4).

In \(\R^3\text{,}\) consider the two vectors

\begin{equation*} \mathbf{v}_1 = \begin{bmatrix} 2 \\1 \\ 1 \end{bmatrix} \quad\text{and}\quad \mathbf{v}_2 = \begin{bmatrix} 1 \\ -1 \\ 3 \end{bmatrix}\text{.} \end{equation*}

Determine if the vector

\begin{equation*} \mathbf{w} = \begin{bmatrix} 1 \\ 5 \\ -7 \end{bmatrix} \end{equation*}

belongs to \(\spans \{ \mathbf{v}_1, \mathbf{v}_2 \}\text{.}\)

🔗

Solution.

We need to determine if \(\mathbf{w}\) is a linear combination of the vectors \(\mathbf{v}_1\) and \(\mathbf{v}_2\text{,}\) i.e., if it is possible to find scalars \(a\) and \(b\) such that \(\mathbf{w} = a \mathbf{v}_1 + b \mathbf{v}_2\text{.}\) Comparing entries of the vectors in each side of the equation, we obtain a system of equations in \(a\) and \(b\text{,}\) i.e.,

\begin{align*} 2a + b \amp = 1\\ a - b \amp = 5\\ a + 3b \amp = -7 \end{align*}

🔗

Practice this system in The Elimination Scratchpad

The Elimination Scratchpad

🔗

Apply Gaussian elimination to the augmented matrix:

\begin{equation*} \left[\begin{array}{cc|c} 2 \amp 1 \amp 1 \\ 1 \amp -1 \amp 5 \\ 1 \amp 3 \amp -7 \end{array}\right]. \end{equation*}

The first entry in column 1 is already nonzero. Following Algorithm 2.2.40, first scale row 1 to obtain a leading one, then eliminate the entries below it. Repeat this process on the remaining rows:

\begin{align*} R_1\to \tfrac{1}{2}R_1 \quad \amp \left[\begin{array}{cc|c} 1 \amp \tfrac{1}{2} \amp \tfrac{1}{2} \\ 1 \amp -1 \amp 5 \\ 1 \amp 3 \amp -7 \end{array}\right]\\ \substack{R_2\to R_2-R_1\\R_3\to R_3-R_1} \quad \amp \left[\begin{array}{cc|c} 1 \amp \tfrac{1}{2} \amp \tfrac{1}{2} \\ 0 \amp -\tfrac{3}{2} \amp \tfrac{9}{2} \\ 0 \amp \tfrac{5}{2} \amp -\tfrac{15}{2} \end{array}\right]\\ R_2\to -\tfrac{2}{3}R_2 \quad \amp \left[\begin{array}{cc|c} 1 \amp \tfrac{1}{2} \amp \tfrac{1}{2} \\ 0 \amp 1 \amp -3 \\ 0 \amp \tfrac{5}{2} \amp -\tfrac{15}{2} \end{array}\right]\\ R_3\to R_3-\tfrac{5}{2}R_2 \quad \amp \left[\begin{array}{cc|c} 1 \amp \tfrac{1}{2} \amp \tfrac{1}{2} \\ 0 \amp 1 \amp -3 \\ 0 \amp 0 \amp 0 \end{array}\right]. \end{align*}

🔗

The last row represents \(0=0\text{,}\) so it imposes no additional condition. The second row gives \(b=-3\text{.}\) Back substitution into the first row, \(a+\tfrac{1}{2}b=\tfrac{1}{2}\text{,}\) gives \(a=\tfrac{1}{2}-\tfrac{1}{2}b=2\text{.}\) Thus the system is consistent, with solution \(a=2\) and \(b=-3\text{.}\) Therefore \(\mathbf{w}=2\mathbf{v}_1-3\mathbf{v}_2\text{,}\) so \(\mathbf{w}\) belongs to \(\spans\{\mathbf{v}_1,\mathbf{v}_2\}\text{.}\)

🔗

Activity 2.3.28. Testing membership in another span (U2-LO4).

In \(\R^3\text{,}\) consider the two vectors

\begin{equation*} \mathbf{v}_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} \quad\text{and}\quad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}\text{.} \end{equation*}

Determine if the vector

\begin{equation*} \mathbf{w} = \begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix} \end{equation*}

belongs to \(\spans \{ \mathbf{v}_1, \mathbf{v}_2 \}\text{.}\)

🔗

Solution.

We seek scalars \(a\) and \(b\) such that \(\mathbf{w}=a\mathbf{v}_1+b\mathbf{v}_2\text{.}\) Comparing coordinates gives the system

\begin{align*} a \amp = 1\\ a+b \amp = 2\\ a+2b \amp = 4 \end{align*}

🔗

Practice this system in The Elimination Scratchpad

The Elimination Scratchpad

🔗

Apply Gaussian elimination to the augmented matrix:

\begin{equation*} \left[\begin{array}{cc|c} 1 \amp 0 \amp 1 \\ 1 \amp 1 \amp 2 \\ 1 \amp 2 \amp 4 \end{array}\right]. \end{equation*}

Following Algorithm 2.2.40, eliminate below the pivots in columns 1 and 2:

\begin{align*} \substack{R_2\to R_2-R_1\\R_3\to R_3-R_1} \quad \amp \left[\begin{array}{cc|c} 1 \amp 0 \amp 1 \\ 0 \amp 1 \amp 1 \\ 0 \amp 2 \amp 3 \end{array}\right]\\ R_3\to R_3-2R_2 \quad \amp \left[\begin{array}{cc|c} 1 \amp 0 \amp 1 \\ 0 \amp 1 \amp 1 \\ 0 \amp 0 \amp 1 \end{array}\right]. \end{align*}

🔗

The last row represents \(0=1\text{,}\) so the system is inconsistent. Therefore \(\mathbf{w}\) does not belong to \(\spans\{\mathbf{v}_1,\mathbf{v}_2\}\text{.}\)

🔗

Fact 2.3.29. Spanning and solvability.

Let \(A\) be an \(m\times n\) matrix with columns \(\mathbf{c}_1,\ldots,\mathbf{c}_n\text{.}\) Then \(A\mathbf{x}=\mathbf{b}\) has a solution for every \(\mathbf{b}\in\R^m\) if and only if \(\spans\{\mathbf{c}_1,\ldots,\mathbf{c}_n\}=\R^m\text{.}\)

🔗

Why is this true?.

As noted in the earlier discussion of matrix columns,

\begin{equation*} A\mathbf{x}=x_1\mathbf{c}_1+\cdots+x_n\mathbf{c}_n. \end{equation*}

Thus \(A\mathbf{x}=\mathbf{b}\) has a solution exactly when \(\mathbf{b}\) is a linear combination of the columns of \(A\text{.}\) By Definition 2.3.22, this holds for every \(\mathbf{b}\in\R^m\) exactly when the columns span \(\R^m\text{.}\)

🔗

The next section studies when a spanning set has redundant vectors and how to remove them.

🔗

Prev Top Next