Skip to main content

MATH 345: Linear Algebra and Optimization

Section 5.1 Eigenvalues and eigenvectors

Let \(A\) be a square matrix. An eigenvector is a nonzero vector whose line is preserved by \(A\text{:}\) applying \(A\) may stretch, reverse, or collapse the vector, but it does not turn the vector away from that line. Eigenvectors are general tools for understanding matrix actions, independently of any particular application.

Subsection Basic definitions

Definition 5.1.1.

Let \(A\) be an \(n \times n\) matrix. Suppose \(\mathbf{v}\) is a nonzero vector in \(\R^n\) and \(\lambda\) is a scalar (which could be \(0\)) such that
\begin{equation*} A \mathbf{v} = \lambda \mathbf{v}\text{.} \end{equation*}
Then \(\lambda\) is an eigenvalue of \(A\text{,}\) and \(\mathbf{v}\) an eigenvector of \(A\) associated with the eigenvalue \(\lambda\text{.}\)

Activity 5.1.2. Checking eigenvectors and their eigenvalues (U5-LO1).

Check that
\begin{equation*} \mathbf{v}_1=\begin{bmatrix} 1 \\ 1 \end{bmatrix}\quad\text{and}\quad \mathbf{v}_2=\begin{bmatrix} 1 \\ 6 \end{bmatrix} \end{equation*}
are eigenvectors for the matrix
\begin{equation*} A= \begin{bmatrix} 3 \amp -1\\ 6 \amp -4 \end{bmatrix}\text{,} \end{equation*}
and find the corresponding eigenvalues.
Solution.
We plug in \(\mathbf{v}_1\) into the matrix \(A\text{,}\) and calculate that
\begin{align*} A\mathbf{v}_1 \amp = \begin{bmatrix} 3 \amp -1 \\ 6 \amp -4 \end{bmatrix}\begin{bmatrix} 1 \\ 1 \end{bmatrix}\\ \amp = \begin{bmatrix} 3(1) + (-1)(1) \\ 6(1) + (-4)(1) \end{bmatrix}\\ \amp = \begin{bmatrix} 2 \\ 2 \end{bmatrix}\\ \amp = 2\begin{bmatrix} 1 \\ 1 \end{bmatrix} = 2\mathbf{v}_1\text{.} \end{align*}
Since \(A\mathbf{v}_1 = 2 \mathbf{v}_1\text{,}\) the vector \(\mathbf{v}_1\) is an eigenvector of \(A\) associated with the eigenvalue \(2\text{.}\)
Next, we plug in \(\mathbf{v}_2\text{,}\) and check that
\begin{align*} A\mathbf{v}_2 \amp = \begin{bmatrix} 3 \amp -1 \\ 6 \amp -4 \end{bmatrix}\begin{bmatrix} 1 \\ 6 \end{bmatrix}\\ \amp = \begin{bmatrix} 3(1) + (-1)(6) \\ 6(1) + (-4)(6) \end{bmatrix}\\ \amp = \begin{bmatrix} -3 \\ -18 \end{bmatrix}\\ \amp = -3\begin{bmatrix} 1 \\ 6 \end{bmatrix} = -3\mathbf{v}_2\text{.} \end{align*}
Since \(A\mathbf{v}_2 = -3 \mathbf{v}_2\text{,}\) \(\mathbf{v}_2\) is an eigenvector of \(A\) associated with the eigenvalue \(-3\text{.}\)

Activity 5.1.3. Finding eigenvalues and eigenvectors (U5-LO1).

Let
\begin{equation*} A=\begin{bmatrix} 3 \amp 0 \\ -1 \amp -4 \end{bmatrix} \end{equation*}
be a matrix. Find the eigenvalues and eigenvectors of \(A\text{.}\)
Solution.
We need to solve the equation \(A\mathbf{v} = \lambda \mathbf{v}\text{,}\) where both \(\lambda\) and \(\mathbf{v} \neq 0\) are unknown.
The equation is equivalent to the equation \((\lambda I - A) \mathbf{v} = \mathbf{0}\text{.}\) For each \(\lambda\text{,}\) in order for this equation to have a nontrivial solution \(\mathbf{v}\text{,}\) the matrix \(\lambda I - A\) must be singular, i.e., \(\det(\lambda I - A) = 0\text{.}\) This allows us to temporarily eliminate \(\mathbf{v}\) from the problem, determining the values of \(\lambda\) that have solutions \(\mathbf{v}\text{.}\)
We calculate that
\begin{align*} \det(\lambda I - A) \amp = \det\begin{bmatrix} \lambda - 3 \amp 0 \\ 1 \amp 4 + \lambda \end{bmatrix}\\ \amp = (\lambda - 3)(\lambda + 4) \end{align*}
Thus \(\det(\lambda I - A) = 0\) holds precisely when \((\lambda - 3)(\lambda + 4) = 0\text{,}\) i.e., when \(\lambda = 3\) or \(\lambda = -4\text{,}\) and so these two values are the two eigenvalues of \(A\text{.}\) It remains to find the eigenvectors for each of the eigenvalues.
We now find the nontrivial solutions to the equation \((A - \lambda I) \mathbf{v} = \mathbf{0}\) for \(\lambda = 3\) and \(\lambda = -4\text{.}\) Each of these is a linear system that can be solved, either using the methods of UnitΒ 2, or more ad hoc methods, which may be faster since the matrices we are dealing with are small.
For \(\lambda = 3\text{,}\) we calculate that
\begin{equation*} A - 3I = \begin{bmatrix} 0 \amp 0 \\ -1 \amp -7 \end{bmatrix}\text{.} \end{equation*}
Thus the solutions to \((A - 3I) \mathbf{v} = 0\) are given by \(-v_1 - 7v_2 = 0\text{,}\) or \(v_1 = -7v_2\text{.}\) Thus the solutions to this equation are given by nonzero multiples of the basic solution
\begin{equation*} \begin{bmatrix} -7 \\ 1 \end{bmatrix}\text{.} \end{equation*}
These are all eigenvectors of \(A\) with eigenvalue \(3\text{.}\)
For \(\lambda = -4\text{,}\) we calculate that
\begin{equation*} A + 4 I = \begin{bmatrix} 7 \amp 0 \\ -1 \amp 0 \end{bmatrix}\text{.} \end{equation*}
Thus the solutions to \((A + 4I) \mathbf{v} = 0\) are those vectors \(\mathbf{v}\) such that \(v_1 = 0\text{,}\) and so the eigenvectors for \(A\) with eigenvalue \(-4\) are given by nonzero multiples of the basic solution
\begin{equation*} \begin{bmatrix} 0 \\ 1 \end{bmatrix}\text{.} \end{equation*}
To conclude, the two eigenvalues of the matrix \(A\) are \(\lambda = 3\) and \(\lambda = -4\text{.}\) The set of all eigenvectors of \(A\) associated with the eigenvalue \(\lambda = 3\) is given by the expression
\begin{equation*} \left\{ \begin{bmatrix} -7t \\ t \end{bmatrix} : t \neq 0 \right\}\text{,} \end{equation*}
and the set of eigenvectors of \(A\) associated with the eigenvalue \(\lambda = -4\) is given by
\begin{equation*} \left\{ \begin{bmatrix} 0 \\ t \end{bmatrix} : t \neq 0 \right\}\text{.} \end{equation*}

Definition 5.1.4.

Let \(A = [a_{ij}]\) be an \(n \times n\) matrix. Then
\begin{equation*} \det(\lambda I_n-A) = \det\begin{bmatrix} \lambda- a_{11} \amp -a_{12} \amp \cdots \amp -a_{1n} \\ -a_{21} \amp \lambda-a_{22} \amp \cdots \amp -a_{2n} \\ \vdots \amp \vdots \amp \ddots \amp \vdots \\ -a_{n1} \amp -a_{n2} \amp \cdots \amp \lambda-a_{nn} \end{bmatrix} \end{equation*}
is called the characteristic polynomial of \(A\) and is sometimes denoted \(c_A(\lambda)\text{.}\) The equation
\begin{equation*} \det \left( \lambda I_n- A \right)=0 \end{equation*}
is called the characteristic equation of \(A\text{.}\)

Remark 5.1.6.

Some authors define the characteristic polynomial to be \(\det(A - \lambda I_n)\) rather than \(\det(\lambda I_n - A)\text{.}\) The advantage of the definition chosen in these notes is that the characteristic polynomial is always monic, i.e., the coefficient associated with the highest power of \(\lambda\) is always equal to one. The zeros of the two polynomials are both the same, and as we saw in ActivityΒ 5.1.3, they are the eigenvalues of \(A\text{.}\)

Activity 5.1.7. Computing a characteristic polynomial (U5-LO1).

Find the characteristic polynomial of \(A\) if
\begin{equation*} A = \begin{bmatrix} 1 \amp 2 \amp -1 \\ 1 \amp 0 \amp 1 \\ 4 \amp -4 \amp 5 \end{bmatrix}\text{.} \end{equation*}
Solution.
We calculate that
\begin{align*} \lambda I_3 - A\amp = \begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda-0 \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix}\\ \amp = \begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix} \end{align*}
The characteristic polynomial is therefore
\begin{align*} \det(\lambda I_3 - A) \amp = \det\begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix}\\ \amp = (\lambda-1)\det\begin{bmatrix} \lambda \amp -1 \\ 4 \amp \lambda-5 \end{bmatrix}\\ \amp\quad\quad - (-2)\det\begin{bmatrix} -1 \amp -1 \\ -4 \amp \lambda-5 \end{bmatrix}\\ \amp\quad\quad + (1)\det\begin{bmatrix} -1 \amp \lambda \\ -4 \amp 4 \end{bmatrix}\\ \amp = (\lambda-1)[\lambda(\lambda-5) - (-1)(4)]\\ \amp\quad\quad - (-2)[(-1)(\lambda-5) - (-1)(-4)]\\ \amp\quad\quad + (1)[(-1)(4) - (\lambda)(-4)]\\ \amp = (\lambda-1)[\lambda^2-5\lambda + 4]\\ \amp\quad\quad - (-2)[-\lambda+1] + (1)[4\lambda-4]\\ \amp = (\lambda-1)[\lambda^2-5\lambda + 4]\\ \amp\quad\quad + (\lambda-1) [-2] + (\lambda-1) [4]\\ \amp = (\lambda-1) [\lambda^2 - 5 \lambda +6]\\ \amp = \lambda^3-6\lambda^2+11\lambda-6 \end{align*}

Note 5.1.8. Finding eigenvalues and eigenvectors of a matrix \(A\).

  1. Determine the roots of the characteristic polynomial \(c_A(\lambda)= \det \left( \lambda I_n - A \right)\text{.}\) These are the eigenvalues of \(A\text{.}\)
  2. For each eigenvalue \(\lambda\text{,}\) find all nontrivial solutions to the homogeneous system \((\lambda I_n - A) \mathbf{x} = \mathbf{0}\text{.}\) These are the eigenvectors of \(A\) associated with \(\lambda\text{.}\)

Definition 5.1.9.

A basic solution (recall DefinitionΒ 2.2.63) to the homogeneous equation \((\lambda I_n - A) \mathbf{x} = \mathbf{0}\) will be called a basic eigenvector associated with the eigenvalue \(\lambda\text{.}\)

Warning 5.1.10.

Do not make the common mistake of first row-reducing \(A\) to RREF and then finding the eigenvalues and eigenvectors of the RREF matrix. Row operations on the matrix \(A\) do not preserve the set of basic solutions. They only preserve solutions to the homogeneous equation \(A\mathbf{v} = \mathbf{0}\text{.}\)
If \(\mathbf{v}\) is an eigenvector of a matrix \(A\) with eigenvalue \(\lambda\text{,}\) then any nonzero scalar multiple of \(\mathbf{v}\) is also an eigenvector with the same eigenvalue \(\lambda\text{.}\) Indeed, if \(\mathbf{w} = c\mathbf{v}\) for some scalar \(c \neq 0\text{,}\) then:
\begin{align*} A\mathbf{w} \amp = A(c\mathbf{v})\\ \amp = c(A\mathbf{v}) \quad \text{(by linearity of matrix multiplication)}\\ \amp = c(\lambda\mathbf{v}) \quad \text{(since } \mathbf{v} \text{ is an eigenvector with eigenvalue } \lambda\text{)}\\ \amp = \lambda(c\mathbf{v})\\ \amp = \lambda\mathbf{w} \end{align*}
Put differently, the entire line \(\text{span}(\mathbf{v})\) is β€œinvariant” under the map defined by \(A\text{,}\) with every vector in this line being stretched (or compressed) by the factor \(\lambda\text{.}\)

Activity 5.1.12. Eigenvalues and eigenspace bases (U5-LO1).

Find the eigenvalues and basic eigenvectors of the matrix
\begin{equation*} A=\begin{bmatrix} 2 \amp 1 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix}\text{.} \end{equation*}
Solution.
We begin by calculating the characteristic polynomial of \(A\text{,}\) i.e.,
\begin{equation*} \det(\lambda I_3 - A) = \det\begin{bmatrix} \lambda-2 \amp -1 \amp 0 \\ 0 \amp \lambda-2 \amp 0 \\ 0 \amp 0 \amp \lambda-2 \end{bmatrix} \end{equation*}
Since this matrix is upper triangular, the determinant is the product of the diagonal entries, and so the characteristic polynomial is \(c_A(\lambda) = (\lambda - 2)^3\text{.}\) The polynomial has only a single zero, i.e., \(\lambda = 2\text{,}\) and so \(\lambda = 2\) is the only eigenvalue of \(A\text{.}\)
To find the basic eigenvectors of \(A\) with eigenvalue \(2\text{,}\) we solve the system \((2 I - A) \mathbf{x} = \mathbf{0}\text{,}\) i.e.,
\begin{equation*} \begin{bmatrix} 0 \amp -1 \amp 0 \\ 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \end{equation*}
Thus the solutions are given by \(x_2 = 0\text{,}\) with \(x_1\) and \(x_3\) as free variables. The set of basic eigenvectors is thus given by
\begin{equation*} \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \right\}\text{.} \end{equation*}