Skip to main content

MATH 345: Linear Algebra and Optimization

Sebastien Roch with Jacob Denson and Kaitlyn Phillipson

Section 7.1 Abstract vector spaces

Earlier units worked mostly with vectors in \(\mathbb R^n\text{.}\) The same algebraic rules also make sense for matrices and polynomials. This section gives the abstract definition so that later courses can use the same language in many settings: vectors may be columns, matrices, polynomials, or other objects.
🔗

Definition 7.1.1.

A real vector space consists of a nonempty set \(V\) of objects (which we will call vectors) that can be added together, and multiplied by a real number (called a scalar in this context) and for which the ten axioms below hold. If \(\mathbf{v}\) and \(\mathbf{w}\) are two vectors in \(V\text{,}\) their sum is denoted \(\mathbf{v} + \mathbf{w}\text{,}\) and the scalar product of \(\mathbf{v}\) by a real number \(a\) is denoted by \(a \mathbf{v}\text{.}\) The axioms of a vector space are described below:
(A1) Closure Under Addition:
🔗
If \(\mathbf{u}\) and \(\mathbf{v}\) are any elements in \(V\text{,}\) then \(\mathbf{u} + \mathbf{v}\) is in \(V\text{.}\)
(A2) Commutativity of Addition:
🔗
For any \(\mathbf{u},\mathbf{v} \in V\text{,}\) \(\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}\text{.}\)
(A3) Associativity of Addition:
🔗
For any \(\mathbf{u},\mathbf{v},\mathbf{w} \in V\text{,}\) \(\mathbf{u} + (\mathbf{v} + \mathbf{w}) = (\mathbf{u} + \mathbf{v}) + \mathbf{w}\text{.}\)
(A4) Existence of a Zero Vector:
🔗
There exists a zero vector \(\mathbf{0}\) in \(V\text{,}\) such that \(\mathbf{u} + \mathbf{0} = \mathbf{0} + \mathbf{u} = \mathbf{u}\) for any \(\mathbf{u} \in V\text{.}\)
(A5) Existence of Additive Inverses
🔗
For each \(\mathbf{v} \in V\) there exists \(-\mathbf{v}\) in \(V\) such that
\begin{equation*} \mathbf{v} + (-\mathbf{v}) = -\mathbf{v} + \mathbf{v} = \mathbf{0}\text{.} \end{equation*}
The vector \(- \mathbf{v}\) is the negative or additive inverse of \(\mathbf{v}\text{.}\)
(S1) Closure Under Scalar Multiplication
🔗
If \(\mathbf{v}\) is any element of \(V\) and \(a\) is any real number, then \(a \mathbf{v}\) is in \(V\text{.}\)
(S2) Distributive Law 1
🔗
For any \(\mathbf{u},\mathbf{v} \in V\) and any \(a \in \R\text{,}\) \(a (\mathbf{u} + \mathbf{v}) = a\mathbf{u} + a \mathbf{v}\text{.}\)
(S3) Distributive Law 2
🔗
For any \(\mathbf{v} \in V\) and any \(a,b \in \R\text{,}\) \((a + b) \mathbf{v} = a \mathbf{v} + b \mathbf{v}\text{.}\)
(S4) Associativity of Scalar Multiplication
🔗
For any \(\mathbf{v} \in V\) and \(a,b \in \R\text{,}\) \(a(b\mathbf{v}) = (ab) \mathbf{v}\text{.}\)
(S5)
🔗
\(1 \mathbf{v} = \mathbf{v}\) for any \(\mathbf{v} \in V\text{.}\)
The operation \(\mathbf{v} + \mathbf{w}\) is called vector addition and the operation \(a\mathbf{v}\) is called scalar multiplication.
🔗
🔗

Note 7.1.2.

In order to be clear about the operations used, we will often denote newly defined addition and scalar multiplication operations using the symbols \(\oplus\) and \(\odot\text{,}\) respectively.
🔗
🔗

Note 7.1.3.

We often use the phrase “vector space”, dropping the “real” and assuming our scalars are real numbers in this course. However, it is possible to define complex vector spaces, rational vector spaces, and more, using different systems of scalars than the real numbers.
🔗
🔗

Note 7.1.4.

From now on, a “vector” will mean an element of a vector space, not necessarily a column vector or line segment.
🔗
🔗
The main examples in this unit are
\begin{equation*} \mathbb R^n,\qquad M_{m,n},\qquad \mathcal P_{\le d}. \end{equation*}
Here \(M_{m,n}\) is the set of \(m\times n\) matrices, and
\begin{equation*} \mathcal P_{\le d} = \{a_0+a_1x+\cdots+a_dx^d:a_0,\ldots,a_d\in\mathbb R\}. \end{equation*}
In \(\mathbb R^n\text{,}\) addition and scalar multiplication are entrywise. In \(M_{m,n}\text{,}\) they are entrywise. In \(\mathcal P_{\le d}\text{,}\) they are polynomial addition and scalar multiplication. The set \(\mathcal P\) of all polynomials is also a vector space, but it is infinite-dimensional and will not be our main example.
🔗

Activity 7.1.1.

Consider the vector space \(\mathbf{M}_{22}\) of all \(2\times2\) matrices.
🔗

(a)

Prove that \(\mathbf{M}_{22}\) is closed under addition and scalar multiplication. What is the zero vector in this vector space?
🔗
Solution.
For two matrices
\begin{equation*} \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix} \quad\text{and}\quad \begin{bmatrix} e \amp f \\ g \amp h \end{bmatrix} \end{equation*}
in \(\mathbf{M}_{22}\text{,}\) their sum is
\begin{equation*} \begin{bmatrix} a + e \amp b + f \\ c + g \amp d + h \end{bmatrix}\text{,} \end{equation*}
which is an element of \(\mathbf{M}_{22}\text{.}\) The product of a scalar \(k\) with a matrix
\begin{equation*} \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix} \end{equation*}
is
\begin{equation*} k \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix} = \begin{bmatrix} ka \amp kb \\ kc \amp kd \end{bmatrix}\text{,} \end{equation*}
which is an element of \(\mathbf{M}_{22}\text{.}\)
🔗
🔗
🔗

(b)

What is the zero vector in \(\mathbf{M}_{22}\text{?}\)
🔗
Solution.
The zero vector in \(\mathbf{M}_{22}\) is the zero matrix
\begin{equation*} \mathbf{0} = \begin{bmatrix} 0 \amp 0 \\ 0 \amp 0 \end{bmatrix}\text{,} \end{equation*}
because \(X + \mathbf{0} = \mathbf{0} + X\) for any \(X \in \mathbf{M}_{22}\text{.}\)
🔗
🔗
🔗
🔗

Activity 7.1.2.

Let’s walk through the proof that \(\mathcal P_{\le2}\) is a vector space.
🔗
Solution.
We go through each axiom one by one:
🔗
Let us start by verifying Axiom (A1). We must verify the sum of two polynomials with degree at most two has degree at most two. Let \(u(x) = a_0 + a_1 x + a_2 x^2\) and \(v(x) = b_0 + b_1 x + b_2 x^2\) be two arbitrary polynomials in \(\mathcal P_{\le2}\text{.}\) Then
\begin{equation*} (u + v)(x) = (a_0 + b_0) + (a_1 + b_1) x + (a_2 + b_2) x^2 \end{equation*}
has degree at most \(2\text{,}\) so lies in \(\mathcal P_{\le2}\text{.}\) Thus Axiom (A1) holds.
🔗
Next, we verify Axiom (A2). We must verify that for any two polynomials, \(u,v \in \mathcal P_{\le2}\text{,}\) \(u + v = v + u\text{.}\) But using the commutativity of real numbers, for any real number \(x\text{,}\) if \(u(x) = a_0 + a_1 x + a_2 x^2\) and \(v(x) = b_0 + b_1 x + b_2 x^2\text{,}\) then
\begin{align*} (u + v)(x) \amp = (a_0 + a_1 x + a_2 x^2) + (b_0 + b_1 x + b_2 x^2)\\ \amp = (b_0 + b_1 x + b_2 x^2) + (a_0 + a_1 x + a_2 x^2)\\ \amp = (v + u)(x)\text{.} \end{align*}
Thus Axiom (A2) holds.
🔗
Next, we verify Axiom (A3). We must verify that for any two polynomials \(u,v,w \in \mathcal P_{\le2}\text{,}\) \(u + (v + w) = (u + v) + w\text{.}\) But this follows, as above, by using the associativity of real numbers, i.e., that if \(u(x) = a_0 + a_1 x + a_2 x^2\text{,}\) \(v(x) = b_0 + b_1 x + b_2 x^2\text{,}\) and \(w(x) = c_0 + c_1 x + c_2x^2\text{,}\) then for any \(x \in \R\text{,}\)
\begin{align*} (u + (v + w))(x) \amp = u(x) + (v(x) + w(x))\\ \amp = (a_0 + a_1 x + a_2 x^2)\\ \amp \quad + ( (b_0 + b_1 x + b_2 x^2) + (c_0 + c_1 x + c_2x^2) )\\ \amp = (a_0 + b_0 + c_0) + (a_1 + b_1 + c_1) x\\ \amp \quad + (a_2 + b_2 + c_2) x^2\\ \amp = ( (a_0 + a_1 x + a_2 x^2) + (b_0 + b_1 x + b_2 x^2) )\\ \amp \quad + (c_0 + c_1 x + c_2x^2)\\ \amp = ((u + v) + w)(x)\text{.} \end{align*}
Thus Axiom (A3) holds.
🔗
Next, we verify Axiom (A4). We must check that if \(\mathbf{0}\) is the zero polynomial, then \(\mathbf{0} + v = v\) for any \(v \in \mathcal P_{\le2}\text{.}\) Now if \(v = a_0 + a_1 x + a_2 x^2\text{,}\) then for any \(x \in \R\text{,}\)
\begin{align*} \mathbf{0}(x) + v(x) \amp = (0 + 0x + 0x^2) + (a_0 + a_1 x + a_2 x^2)\\ \amp = (0 + a_0) + (0 + a_1) x+ (0 + a_2)x^2\\ \amp = a_0 + a_1 x + a_2 x^2\\ \amp = v(x) \end{align*}
Thus Axiom (A4) holds.
🔗
Next, we verify Axiom (A5). For each polynomial \(u \in \mathcal P_{\le2}\) given by \(u(x) = a_0 + a_1 x + a_2 x^2\text{,}\) if we define \((-u)(x) = (-a_0) + (-a_1) x + (-a_2) x^2\text{,}\) then \(u + (-u) = 0\text{.}\) Indeed
\begin{align*} (u + (-u))(x) \amp = (a_0 + a_1 x + a_2 x^2) + ( (-a_0) + (-a_1) x + (-a_2) x^2)\\ \amp = ((a_0 - a_0) + (a_1 - a_1) x + (a_2 - a_2) x^2)\\ \amp = 0\text{.} \end{align*}
Thus \(u + (-u) = 0\text{,}\) verifying Axiom (A5).
🔗
Next, we verify Axiom (S1). We must verify that for \(\mathbf{v} \in \mathcal P_{\le2}\) and \(r \in \R\text{,}\) \(r \mathbf{v} \in \mathcal P_{\le2}\text{.}\) If \(u(x) = a_0 + a_1 x + a_2x^2\text{,}\) and a scalar \(r\text{,}\)
\begin{align*} (ru)(x) \amp = r(a_0 + a_1x + a_2x^2)\\ \amp = (ra_0) + (ra_1) x + (ra_2) x^2\text{.} \end{align*}
The polynomial \(ru\) still has degree at most \(2\text{,}\) so \(ru \in \mathcal P_{\le2}\text{.}\) Thus Axiom (S1) holds.
🔗
Next, we verify Axiom (S2). We must verify that for any \(u,v \in \mathcal P_{\le2}\) and \(r \in \R\text{,}\) then \(r(u + v) = ru + rv\text{.}\) If \(u(x) = a_0 + a_1x + a_2x^2\) and \(v(x) = b_0 + b_1x + b_2x^2\text{,}\) then
\begin{align*} (r(u + v))(x) \amp = r( (a_0 + a_1x + a_2x^2) + (b_0 + b_1x + b_2x^2) )\\ \amp = (ra_0 + rb_0) + (ra_1 + rb_1) x + (ra_2 + rb_2) x^2\\ \amp = (ru + rv)(x)\text{.} \end{align*}
Thus Axiom (S2) holds.
🔗
Next, we verify Axiom (S3). We must verify that for any \(u \in \mathcal P_{\le2}\) and \(r,s \in \R\text{,}\) \((r + s) u = ru + su\text{.}\) If \(u(x) = a_0 + a_1 x + a_2x^2\text{,}\) then
\begin{align*} ((r + s) u)(x) \amp = (r + s)( a_0 + a_1 x + a_2x^2 )\\ \amp = ( (ra_0 + sa_0) + (ra_1 + sa_1) x + (ra_2 + sa_2) x^2 )\\ \amp = (ru + su)(x)\text{.} \end{align*}
Thus Axiom (S3) holds.
🔗
Next, we verify Axiom (S4). We must verify that for any \(r,s \in \mathbb R\) and \(u \in \mathcal P_{\le2}\text{,}\) \(r(s u) = (rs) u\text{.}\) If \(u(x) = a_0 + a_1 x + a_2 x^2\text{,}\) then
\begin{align*} (r(s u))(x) \amp = r(s(a_0 + a_1 x + a_2 x^2))\\ \amp = (rs)(a_0 + a_1 x + a_2 x^2)\\ \amp = ((rs) u)(x)\text{.} \end{align*}
Thus Axiom (S4) holds.
🔗
Finally, we verify Axiom (S5). We must verify that for any \(u \in \mathcal P_{\le2}\text{,}\) \(1 u = u\text{.}\) If \(u(x) = a_0 + a_1x + a_2x^2\text{,}\) then
\begin{equation*} (1 u)(x) = 1 (a_0 + a_1 x + a_2x^2) = u(x)\text{.} \end{equation*}
Thus Axiom (S5) holds.
🔗
🔗
🔗

Activity 7.1.3.

Let \(V\) be the set of all ordered pairs of real numbers \((x,y)\) with the operations
\begin{equation*} (x_1,y_1) \oplus (x_2,y_2) = (x_1,0) \quad\text{and}\quad c \odot (x,y) = (cx,cy)\text{.} \end{equation*}
Show that \(V\) is not a vector space.
🔗
Solution.
The set \(V\text{,}\) and the operations defined upon it, fail to satisfy Axiom (A4), i.e., no zero vector exists in \(V\text{.}\) Since \((x_1,y_1) \oplus (x_2,y_2) = (x_1,0)\text{,}\) it is impossible for there to exist a vector \((x_0,y_0)\) such that \((x_0,y_0) \oplus (x,y) = (x,y)\) for all ordered pairs \((x,y)\text{.}\)
🔗
Note that this is not the only axiom that \(V\) fails to satisfy. \(V\) also fails to satisfy axioms (A2), (A5), and (S3). Verifying that any one of these axioms fail to hold would also consitute a valid argument that \(V\) is not a vector space. But a valid argument only needs to check that one of the axioms fails to hold.
🔗
🔗
🔗

Activity 7.1.4.

Let \(V\) be the set of all ordered pairs of real numbers \((x,y)\) with the operations
\begin{equation*} (x_1,y_1) \oplus (x_2,y_2) = (x_1 + x_2, y_1 + y_2) \quad\text{and}\quad c \odot (x,y) = (cx,y)\text{.} \end{equation*}
Show that \(V\) is not a vector space.
🔗
Solution.
Under this definition of addition and multiplication, \(V\) fails to satisfy Axiom (S3). We calculate that
\begin{equation*} (a + b) \odot (x,y) = ((a + b)x, y)\text{,} \end{equation*}
and that
\begin{equation*} \left(a \odot (x,y) \right) \oplus \left( b \odot (x,y) \right) = (ax,y) \oplus (bx,y) = (ax + bx, 2y)\text{.} \end{equation*}
If \(y \neq 0\text{,}\) then the two tuples will not be equal to one another. Thus a particular counterexample that shows that Axiom (S3) fails is obtained by taking \(a = b = 1\text{,}\) and \((x,y) = (0,1)\text{.}\) Then
\begin{equation*} (1 + 1) \odot (0,1) = (0,1) \end{equation*}
whereas
\begin{equation*} (1 \odot (0,1)) \oplus (1 \odot (0,1)) = (0,2)\text{.} \end{equation*}
These two quantities are evidently not equal to one another, which shows Axiom (S3) fails, and so \(V\) is not a vector space.
🔗
🔗
🔗

Activity 7.1.5.

Let \(V\) be the set of all non-negative real numbers, and define vector addition as
\begin{equation*} \mathbf{v} \oplus \mathbf{u} = |\mathbf{v} - \mathbf{u}|\text{,} \end{equation*}
and scalar multiplication as
\begin{equation*} a \odot \mathbf{v} = |a| \mathbf{v}\text{.} \end{equation*}
Does this structure form a vector space?
🔗
Solution.
With this definition, addition is not associative, i.e., Axiom (A3) fails. To find a counterexample, we must find three elements of \(V\) such that
\begin{equation*} |x - |y - z|| \neq ||x - y| - z|\text{.} \end{equation*}
We might as well start simply by guessing some values for which this equation fails to hold, because often if an algebraic identity fails to hold, it fails to hold for most inputs. Choose \(x = 3\text{,}\) \(y = 1\text{,}\) and \(z = 2\text{.}\) Then
\begin{equation*} (x \oplus y) \oplus z = ||3 - 1| - 2| = |2 - 2| = 0\text{,} \end{equation*}
whereas
\begin{equation*} x \oplus (y \oplus z) = |3 - |1 - 2|| = |3 - 1| = 2\text{.} \end{equation*}
Thus we have found a particular counterexample that shows that addition is not associative.
🔗
🔗
🔗
Below we list several facts about the interaction of addition and multiplication that hold in any vector space.
🔗
The point is not that every vector looks like an arrow. The point is that the same algebraic rules let us reuse span, independence, bases, coordinates, dimension, and linear maps in settings where the objects are matrices or polynomials.
🔗
🔗
PrevTopNext