Skip to main content

MATH 345: Linear Algebra and Optimization

Sebastien Roch with Jacob Denson and Kaitlyn Phillipson

Section 5.3 Eigenvalues, eigenvectors, and diagonalization

Subsection Basic definitions

Definition 5.3.1.

Let \(A\) be an \(n \times n\) matrix. Suppose \(\mathbf{v}\) is a nonzero vector in \(\R^n\) and \(\lambda\) is a scalar (which could be \(0\)) such that
\begin{equation*} A \mathbf{v} = \lambda \mathbf{v}\text{.} \end{equation*}
Then \(\lambda\) is an eigenvalue of \(A\text{,}\) and \(\mathbf{v}\) an eigenvector of \(A\) associated with the eigenvalue \(\lambda\text{.}\)
🔗
🔗

Activity 5.3.1.

Check that
\begin{equation*} \mathbf{v}_1=\begin{bmatrix} 1 \\ 1 \end{bmatrix}\quad\text{and}\quad \mathbf{v}_2=\begin{bmatrix} 1 \\ 6 \end{bmatrix} \end{equation*}
are eigenvectors for the matrix
\begin{equation*} A= \begin{bmatrix} 3 \amp -1\\ 6 \amp -4 \end{bmatrix}\text{,} \end{equation*}
and find the corresponding eigenvalues.
🔗
Solution.
We plug in \(\mathbf{v}_1\) into the matrix \(A\text{,}\) and calculate that
\begin{align*} A\mathbf{v}_1 \amp = \begin{bmatrix} 3 \amp -1 \\ 6 \amp -4 \end{bmatrix}\begin{bmatrix} 1 \\ 1 \end{bmatrix}\\ \amp = \begin{bmatrix} 3(1) + (-1)(1) \\ 6(1) + (-4)(1) \end{bmatrix}\\ \amp = \begin{bmatrix} 2 \\ 2 \end{bmatrix}\\ \amp = 2\begin{bmatrix} 1 \\ 1 \end{bmatrix} = 2\mathbf{v}_1\text{.} \end{align*}
Since \(A\mathbf{v}_1 = 2 \mathbf{v}_1\text{,}\) the vector \(\mathbf{v}_1\) is an eigenvector of \(A\) associated with the eigenvalue \(2\text{.}\)
🔗
Next, we plug in \(\mathbf{v}_2\text{,}\) and check that
\begin{align*} A\mathbf{v}_2 \amp = \begin{bmatrix} 3 \amp -1 \\ 6 \amp -4 \end{bmatrix}\begin{bmatrix} 1 \\ 6 \end{bmatrix}\\ \amp = \begin{bmatrix} 3(1) + (-1)(6) \\ 6(1) + (-4)(6) \end{bmatrix}\\ \amp = \begin{bmatrix} -3 \\ -18 \end{bmatrix}\\ \amp = -3\begin{bmatrix} 1 \\ 6 \end{bmatrix} = -3\mathbf{v}_2\text{.} \end{align*}
Since \(A\mathbf{v}_2 = -3 \mathbf{v}_2\text{,}\) \(\mathbf{v}_2\) is an eigenvector of \(A\) associated with the eigenvalue \(-3\text{.}\)
🔗
🔗
🔗

Activity 5.3.2.

Let
\begin{equation*} A=\begin{bmatrix} 3 \amp 0 \\ -1 \amp -4 \end{bmatrix} \end{equation*}
be a matrix. Find the eigenvalues and eigenvectors of \(A\text{.}\)
🔗
Solution.
We need to solve the equation \(A\mathbf{v} = \lambda \mathbf{v}\text{,}\) where both \(\lambda\) and \(\mathbf{v} \neq 0\) are unknown.
🔗
The equation is equivalent to the equation \((\lambda I - A) \mathbf{v} = \mathbf{0}\text{.}\) For each \(\lambda\text{,}\) in order for this equation to have a nontrivial solution \(\mathbf{v}\text{,}\) the matrix \(\lambda I - A\) must be singular, i.e., \(\det(\lambda I - A) = 0\text{.}\) This allows us to temporarily eliminate \(\mathbf{v}\) from the problem, determining the values of \(\lambda\) that have solutions \(\mathbf{v}\text{.}\)
🔗
We calculate that
\begin{align*} \det(\lambda I - A) \amp = \det\begin{bmatrix} \lambda - 3 \amp 0 \\ 1 \amp 4 + \lambda \end{bmatrix}\\ \amp = (\lambda - 3)(\lambda + 4) \end{align*}
Thus \(\det(\lambda I - A) = 0\) holds precisely when \((\lambda - 3)(\lambda + 4) = 0\text{,}\) i.e., when \(\lambda = 3\) or \(\lambda = -4\text{,}\) and so these two values are the two eigenvalues of \(A\text{.}\) It remains to find the eigenvectors for each of the eigenvalues.
🔗
We now find the nontrivial solutions to the equation \((A - \lambda I) \mathbf{v} = \mathbf{0}\) for \(\lambda = 3\) and \(\lambda = -4\text{.}\) Each of these is a linear system that can be solved, either using the methods of Unit 2, or more ad hoc methods, which may be faster since the matrices we are dealing with are small.
🔗
For \(\lambda = 3\text{,}\) we calculate that
\begin{equation*} A - 3I = \begin{bmatrix} 0 \amp 0 \\ -1 \amp -7 \end{bmatrix}\text{.} \end{equation*}
Thus the solutions to \((A - 3I) \mathbf{v} = 0\) are given by \(-v_1 - 7v_2 = 0\text{,}\) or \(v_1 = -7v_2\text{.}\) Thus the solutions to this equation are given by non-zero multiples of the basic solution
\begin{equation*} \begin{bmatrix} -7 \\ 1 \end{bmatrix}\text{.} \end{equation*}
These are all eigenvectors of \(A\) with eigenvalue \(3\text{.}\)
🔗
For \(\lambda = -4\text{,}\) we calculate that
\begin{equation*} A + 4 I = \begin{bmatrix} 7 \amp 0 \\ -1 \amp 0 \end{bmatrix}\text{.} \end{equation*}
Thus the solutions to \((A + 4I) \mathbf{v} = 0\) are those vectors \(\mathbf{v}\) such that \(v_1 = 0\text{,}\) and so the eigenvectors for \(A\) with eigenvalue \(-4\) are given by non-zero multiples of the basic solution
\begin{equation*} \begin{bmatrix} 0 \\ 1 \end{bmatrix}\text{.} \end{equation*}
🔗
To conclude, the two eigenvalues of the matrix \(A\) are \(\lambda = 3\) and \(\lambda = -4\text{.}\) The set of all eigenvectors of \(A\) associated with the eigenvalue \(\lambda = 3\) is given by the expression
\begin{equation*} \left\{ \begin{bmatrix} -7t \\ t \end{bmatrix} : t \neq 0 \right\}\text{,} \end{equation*}
and the set of eigenvectors of \(A\) associated with the eigenvalue \(\lambda = -4\) is given by
\begin{equation*} \left\{ \begin{bmatrix} 0 \\ t \end{bmatrix} : t \neq 0 \right\}\text{.} \end{equation*}
🔗
🔗
🔗

Definition 5.3.2.

Let \(A = [a_{ij}]\) be an \(n \times n\) matrix. Then
\begin{equation*} \det(\lambda I_n-A) = \det\begin{bmatrix} \lambda- a_{11} \amp -a_{12} \amp \cdots \amp -a_{1n} \\ -a_{21} \amp \lambda-a_{22} \amp \cdots \amp -a_{2n} \\ \vdots \amp \vdots \amp \ddots \amp \vdots \\ -a_{n1} \amp -a_{n2} \amp \cdots \amp \lambda-a_{nn} \end{bmatrix} \end{equation*}
is called the characteristic polynomial of \(A\) and is sometimes denoted \(c_A(\lambda)\text{.}\) The equation
\begin{equation*} \det \left( \lambda I_n- A \right)=0 \end{equation*}
is called the characteristic equation of \(A\text{.}\)
🔗
🔗

Remark 5.3.4.

Some authors define the characteristic polynomial to be \(\det(A - \lambda I_n)\) rather than \(\det(\lambda I_n - A)\text{.}\) The advantage of the definition chosen in these notes is that the characteristic polynomial is always monic, i.e., the coefficient associated with the highest power of \(\lambda\) is always equal to one. The zeros of the two polynomials are both the same, and as we saw in Activity 5.3.2, they are the eigenvalues of \(A\text{.}\)
🔗
🔗

Activity 5.3.3.

Find the characteristic polynomial of \(A\) if
\begin{equation*} A = \begin{bmatrix} 1 \amp 2 \amp -1 \\ 1 \amp 0 \amp 1 \\ 4 \amp -4 \amp 5 \end{bmatrix}\text{.} \end{equation*}
🔗
Solution.
We calculate that
\begin{align*} \lambda I_3 - A\amp = \begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda-0 \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix}\\ \amp = \begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix} \end{align*}
The characteristic polynomial is therefore
\begin{align*} \det(\lambda I_3 - A) \amp = \det\begin{bmatrix} \lambda-1 \amp -2 \amp 1 \\ -1 \amp \lambda \amp -1 \\ -4 \amp 4 \amp \lambda-5 \end{bmatrix}\\ \amp = (\lambda-1)\det\begin{bmatrix} \lambda \amp -1 \\ 4 \amp \lambda-5 \end{bmatrix}\\ \amp\quad\quad - (-2)\det\begin{bmatrix} -1 \amp -1 \\ -4 \amp \lambda-5 \end{bmatrix}\\ \amp\quad\quad + (1)\det\begin{bmatrix} -1 \amp \lambda \\ -4 \amp 4 \end{bmatrix}\\ \amp = (\lambda-1)[\lambda(\lambda-5) - (-1)(4)]\\ \amp\quad\quad - (-2)[(-1)(\lambda-5) - (-1)(-4)]\\ \amp\quad\quad + (1)[(-1)(4) - (\lambda)(-4)]\\ \amp = (\lambda-1)[\lambda^2-5\lambda + 4]\\ \amp\quad\quad - (-2)[-\lambda+1] + (1)[4\lambda-4]\\ \amp = (\lambda-1)[\lambda^2-5\lambda + 4]\\ \amp\quad\quad + (\lambda-1) [-2] + (\lambda-1) [4]\\ \amp = (\lambda-1) [\lambda^2 - 5 \lambda +6]\\ \amp = \lambda^3-6\lambda^2+11\lambda-6 \end{align*}
🔗
🔗
🔗

Note 5.3.5. Finding eigenvalues and eigenvectors of a matrix \(A\).

  1. Determine the roots of the characteristic polynomial \(c_A(\lambda)= \det \left( \lambda I_n - A \right)\text{.}\) These are the eigenvalues of \(A\text{.}\)
    🔗
    🔗
  2. For each eigenvalue \(\lambda\text{,}\) find all nontrivial solutions to the homogeneous system \((\lambda I_n - A) \mathbf{x} = \mathbf{0}\text{.}\) These are the eigenvectors of \(A\) associated with \(\lambda\text{.}\)
    🔗
    🔗
🔗
🔗

Definition 5.3.6.

A basic solution (recall Definition 2.2.36) to the homogeneous equation \((\lambda I_n - A) \mathbf{x} = \mathbf{0}\) will be called a basic eigenvector associated with the eigenvalue \(\lambda\text{.}\)
🔗
🔗

Warning 5.3.7.

Do not make the common mistake of first row-reducing \(A\) to RREF and then finding the eigenvalues and eigenvectors of the RREF matrix. Row operations on the matrix \(A\) do not preserve the set of basic solutions. They only preserve solutions to the homogeneous equation \(A\mathbf{v} = \mathbf{0}\text{.}\)
🔗
🔗
If \(\mathbf{v}\) is an eigenvector of a matrix \(A\) with eigenvalue \(\lambda\text{,}\) then any nonzero scalar multiple of \(\mathbf{v}\) is also an eigenvector with the same eigenvalue \(\lambda\text{.}\) Indeed, if \(\mathbf{w} = c\mathbf{v}\) for some scalar \(c \neq 0\text{,}\) then:
\begin{align*} A\mathbf{w} \amp = A(c\mathbf{v})\\ \amp = c(A\mathbf{v}) \quad \text{(by linearity of matrix multiplication)}\\ \amp = c(\lambda\mathbf{v}) \quad \text{(since } \mathbf{v} \text{ is an eigenvector with eigenvalue } \lambda\text{)}\\ \amp = \lambda(c\mathbf{v})\\ \amp = \lambda\mathbf{w} \end{align*}
🔗
Put differently, the entire line \(\text{span}(\mathbf{v})\) is “invariant” under the map defined by \(A\text{,}\) with every vector in this line being stretched (or compressed) by the factor \(\lambda\text{.}\)
🔗

Activity 5.3.4.

Find the eigenvalues and basic eigenvectors of the matrix
\begin{equation*} A=\begin{bmatrix} 2 \amp 1 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 2 \end{bmatrix}\text{.} \end{equation*}
🔗
Solution.
We begin by calculating the characteristic polynomial of \(A\text{,}\) i.e.,
\begin{equation*} \det(\lambda I_3 - A) = \det\begin{bmatrix} \lambda-2 \amp -1 \amp 0 \\ 0 \amp \lambda-2 \amp 0 \\ 0 \amp 0 \amp \lambda-2 \end{bmatrix} \end{equation*}
Since this matrix is upper triangular, the determinant is the product of the diagonal entries, and so the characteristic polynomial is \(c_A(\lambda) = (\lambda - 2)^3\text{.}\) The polynomial has only a single zero, i.e., \(\lambda = 2\text{,}\) and so \(\lambda = 2\) is the only eigenvalue of \(A\text{.}\)
🔗
To find the basic eigenvectors of \(A\) with eigenvalue \(2\text{,}\) we solve the system \((2 I - A) \mathbf{x} = \mathbf{0}\text{,}\) i.e.,
\begin{equation*} \begin{bmatrix} 0 \amp -1 \amp 0 \\ 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \end{equation*}
Thus the solutions are given by \(x_2 = 0\text{,}\) with \(x_1\) and \(x_3\) as free variables. The set of basic eigenvectors is thus given by
\begin{equation*} \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \right\}\text{.} \end{equation*}
🔗
🔗
🔗
🔗

Subsection Diagonalization

Definition 5.3.9.

An \(n\times n\) matrix \(A\) is called diagonalizable if there exists an invertible \(n \times n\) matrix \(P\) so that \(P^{-1}AP\) is a diagonal matrix (recall Definition 2.5.22). The matrix \(P\) is called a diagonalizing matrix for \(A\text{.}\)
🔗
🔗
Note that if \(P^{-1}AP\) is a diagonal matrix with diagonal entries \(\lambda_1,\dots,\lambda_n\text{,}\) then for each standard basis vector (recall Definition 1.3.8) \(P^{-1}AP \mathbf{e}_i = \lambda_i \mathbf{e}_i\text{.}\) Multiplying this equation by \(P\) on the left hand side and setting \(\mathbf{v}_i = P \mathbf{e}_i\) we obtain that
\begin{equation*} A \mathbf{v} = P(P^{-1}AP \mathbf{e}_i) = P(\lambda_i \mathbf{e}_i) = \lambda_i \mathbf{v}_i\text{.} \end{equation*}
Thus \(\mathbf{v}_i\) is an eigenvector of the matrix \(A\) associated with the eigenvalue \(\lambda_i\text{.}\) Conversely,
🔗

Activity 5.3.5.

\begin{equation*} A=\begin{bmatrix} 3 \amp 1 \\ -2 \amp 0 \end{bmatrix}\text{.} \end{equation*}
(a)
Compute \(P^{-1}AP\) where
\begin{equation*} P=\begin{bmatrix} -1/2 \amp -1 \\ 1 \amp 1 \end{bmatrix}\text{.} \end{equation*}
You may use the fact that
\begin{equation*} P^{-1}=\begin{bmatrix} 2 \amp 2 \\ -2 \amp -1 \end{bmatrix}\text{.} \end{equation*}
What does this computation say about the matrix \(A\text{?}\)
🔗
Solution.
We start by calculating that
\begin{align*} AP \amp = \begin{bmatrix} (3)(-1/2) + (1)(1) \amp (3)(-1) + (1)(1) \\ (-2)(-1/2) + (0)(1) \amp (-2)(-1) + (0)(1) \end{bmatrix}\\ \amp = \begin{bmatrix} -1/2 \amp -2 \\ 1 \amp 2 \end{bmatrix} \end{align*}
and thus that
\begin{align*} P^{-1}AP \amp = \begin{bmatrix} 2 \amp 2 \\ -2 \amp -1 \end{bmatrix} \begin{bmatrix} -1/2 \amp -2 \\ 1 \amp 2 \end{bmatrix}\\ \amp = \begin{bmatrix} (2)(-1/2) + (2)(1) \amp (2)(-2) + (2)(2) \\ (-2)(-1/2) + (-1)(1) \amp (-2)(-2) + (-1)(2) \end{bmatrix}\\ \amp = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 2 \end{bmatrix} \end{align*}
Since \(P^{-1}AP\) is a diagonal matrix, the matrix \(A\) is diagonalizable.
🔗
🔗
🔗
(b)
Use the previous computation to determine the eigenvalues of \(A\text{.}\)
🔗
Solution.
We calculate that
\begin{align*} c_A(\lambda) \amp = \det( \lambda I - A )\\ \amp = \det( P( \lambda I - P^{-1} AP ) P^{-1} )\\ \amp = \det( P ) \det( \lambda I - P^{-1} A P ) \det(P)^{-1}\\ \amp = \det( \lambda I - P^{-1} A P )\\ \amp = \det \begin{bmatrix} \lambda - 1 \amp 0 \\ 0 \amp \lambda - 2 \end{bmatrix}\\ \amp = (\lambda - 1)(\lambda - 2)\text{.} \end{align*}
Thus \(\{ 1, 2 \}\) is the set of eigenvalues of \(A\text{.}\)
🔗
🔗
🔗
(c)
Use the previous computation to calculate \(A^5\text{.}\)
🔗
Solution.
Let’s start by computing smaller powers of \(A\text{.}\) We find that
\begin{align*} A^2 \amp = A \cdot A = (PDP^{-1})(PDP^{-1})\\ \amp = PD(P^{-1}P)DP^{-1} = PD^2P^{-1} \end{align*}
and
\begin{align*} A^3 \amp = A \cdot A^2 = (PDP^{-1})(PD^2P^{-1})\\ \amp = PD(P^{-1}P)D^2P^{-1} = PD^3P^{-1} \end{align*}
Continuing this calculation, noticing the pattern, we conclude that \(A^5 = PD^5P^{-1}\text{.}\)
🔗
Multiplying diagonal matrices is easy, i.e.,
\begin{equation*} \begin{bmatrix} a_1 \amp 0 \\ 0 \amp b_1 \end{bmatrix} \begin{bmatrix} a_2 \amp 0 \\ 0 \amp b_2 \end{bmatrix} \cdots \begin{bmatrix} a_m \amp 0 \\ 0 \amp b_m \end{bmatrix} = \begin{bmatrix} a_1 \cdots a_m \amp 0 \\ 0 \amp b_1 \cdots b_m \end{bmatrix}\text{.} \end{equation*}
Thus
\begin{equation*} D^5 = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 2 \end{bmatrix}^5 = \begin{bmatrix} 1^5 \amp 0 \\ 0 \amp 2^5 \end{bmatrix} = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 32 \end{bmatrix}\text{.} \end{equation*}
We use this computation to compute \(A^5\text{,}\) i.e., we calculate that
\begin{equation*} PD^5 = \begin{bmatrix} -1/2 \amp -1 \\ 1 \amp 1 \end{bmatrix} \begin{bmatrix} 1 \amp 0 \\ 0 \amp 32 \end{bmatrix} = \begin{bmatrix} -1/2 \amp -32 \\ 1 \amp 32 \end{bmatrix} \end{equation*}
and thus that
\begin{align*} PD^5P^{-1} \amp = \begin{bmatrix} -1/2 \amp -32 \\ 1 \amp 32 \end{bmatrix} \begin{bmatrix} 2 \amp 2 \\ -2 \amp -1 \end{bmatrix}\\ \amp = \begin{bmatrix} 63 \amp 31 \\ -62 \amp -30 \end{bmatrix}\text{.} \end{align*}
🔗
🔗
🔗
🔗
The previous activity shows why diagonalizing a matrix may be useful. But given a matrix \(A\text{,}\) how do we potentially find \(P\) such that \(P^{-1}AP\) is a diagonal matrix?
🔗

Activity 5.3.6.

Suppose \(P^{-1}AP = D\) for some diagonal matrix \(D = \diag(\lambda_1,\dots,\lambda_n)\text{.}\) Let \(\mathbf{v}_1, \dots, \mathbf{v}_n\) be the columns of \(P\text{.}\) Note that the equation \(P^{-1} A P = D\) holds if and only if \(AP = PD\text{.}\)
🔗
(a)
What happens when I take \(PD\text{?}\)
🔗
Solution.
\begin{align*} PD \amp = [\mathbf{v}_1 \mathbf{v}_2 \cdots \mathbf{v}_n] \begin{bmatrix} \lambda_1 \amp 0 \amp \cdots \amp 0 \\ 0 \amp \lambda_2 \amp \cdots \amp 0 \\ \vdots \amp \vdots \amp \ddots \amp \vdots \\ 0 \amp 0 \amp \cdots \amp \lambda_n \end{bmatrix} \end{align*}
Matrix multiplication gives us:
\begin{equation*} PD = [\lambda_1\mathbf{v}_1 \; \lambda_2\mathbf{v}_2 \; \cdots \; \lambda_n\mathbf{v}_n] \end{equation*}
So \(PD\) is the matrix whose columns are the vectors \(\mathbf{v}_i\) scaled by their corresponding \(\lambda_i\) values.
🔗
🔗
🔗
(b)
Calculate \(AP\) and \(PD\) in terms of the vectors \(\mathbf{v}_1,\dots,\mathbf{v}_n\text{,}\) the scalars \(\lambda_1,\dots,\lambda_n\) and the matrix \(A\text{.}\)
🔗
Solution.
When we take \(AP\text{,}\) we compute that
\begin{equation*} AP = A \begin{bmatrix} \mathbf{v}_1 \amp \mathbf{v}_2 \amp \cdots \amp \mathbf{v}_n \end{bmatrix} = \begin{bmatrix} A\mathbf{v}_1 \amp A\mathbf{v}_2 \amp \cdots \amp A\mathbf{v}_n \end{bmatrix}\text{.} \end{equation*}
🔗
When we calculate \(PD\text{,}\) we compute that
\begin{align*} PD \amp = \begin{bmatrix} \mathbf{v}_1 \amp \mathbf{v}_2 \amp \cdots \amp \mathbf{v}_n \end{bmatrix} \begin{bmatrix} \lambda_1 \amp 0 \amp \cdots \amp 0 \\ 0 \amp \lambda_2 \amp \cdots \amp 0 \\ \vdots \amp \vdots \amp \ddots \amp \vdots \\ 0 \amp 0 \amp \cdots \amp \lambda_n \end{bmatrix}\\ \amp = \begin{bmatrix} \lambda_1 \mathbf{v}_1 \amp \cdots \amp \lambda_n \mathbf{v}_n \end{bmatrix}\text{.} \end{align*}
🔗
🔗
🔗
(c)
So what conditions on \(P\) and \(D\) are necessary and sufficient for the equatino \(AP = PD\) to hold?
🔗
Solution.
For the equation \(AP = PD\) to hold, we need \(A \mathbf{v}_i = \lambda_i \mathbf{v}_i\) for each \(i\text{.}\) Thus we need each column of \(P\) to be an eigenvector of \(A\text{,}\) whose eigenvalue is the corresponding entry of the diagonal matrix \(D\text{.}\)
🔗
🔗
🔗
🔗

Remark 5.3.11.

Recalling Fact 2.5.10, the matrix \(P\) in Theorem 5.3.10 is invertible if and only if the vectors \(\mathbf{v}_1,\dots,\mathbf{v}_n\) form a basis for \(\R^n\text{.}\)
🔗
🔗
If \(\R^n\) has a basis of eigenvectors \(\mathbf{v}_1,\ldots,\mathbf{v}_n\) with corresponding eigenvalues \(\lambda_1,\ldots,\lambda_n\text{,}\) then every vector \(\mathbf{v} \in \R^n\) can be written as a linear combination of these eigenvectors, i.e., there exists constants \(a_1,\dots,a_n\) such that
\begin{equation*} \mathbf{v} = a_1 \mathbf{v}_1 + a_2 \mathbf{v}_2 + \cdots + a_n \mathbf{v}_n\text{.} \end{equation*}
Thus
\begin{align*} T_A (\mathbf{v}) \amp = A \mathbf{v}\\ \amp = A(a_1 \mathbf{v}_1 + a_2 \mathbf{v}_2 + \cdots + a_n \mathbf{v}_n)\\ \amp = a_1 A \mathbf{v}_1 + a_2 A\mathbf{v}_2 + \cdots + a_n A \mathbf{v}_n\\ \amp = a_1 \lambda_1 \mathbf{v}_1 + a_2 \lambda_2 \mathbf{v}_2 + \cdots + a_n \lambda_n \mathbf{v}_n\\ \amp = \lambda_1 (a_1 \mathbf{v}_1) + \lambda_2 (a_2 \mathbf{v}_2) + \cdots + \lambda_n (a_n \mathbf{v}_n) \end{align*}
In other words, \(T_A\) ‘stretches’ the components of \(\mathbf{v}\) along each eigenvector by the corresponding eigenvalue.
🔗

Activity 5.3.7.

Consider the matrix
\begin{equation*} M = \begin{bmatrix} 5/3 \amp -2/3 \\ -1/3 \amp 4/3 \end{bmatrix}\text{.} \end{equation*}
Verify that
\begin{equation*} \mathbf{u} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} \quad\text{and}\quad \mathbf{v} = \begin{bmatrix} 1 \\ -1/2 \end{bmatrix} \end{equation*}
are eigenvectors of \(M\) corresponding to different eigenvalues. What are the eigenvalues?
🔗
Solution.
We compute that
\begin{align*} M\mathbf{u} \amp = \begin{bmatrix} 5/3 \amp -2/3 \\ -1/3 \amp 4/3 \end{bmatrix}\begin{bmatrix} 1 \\ 1 \end{bmatrix}\\ \amp = \begin{bmatrix} 5/3 - 2/3 \\ -1/3 + 4/3 \end{bmatrix}\\ \amp = \begin{bmatrix} 1 \\ 1 \end{bmatrix}\\ \amp = 1 \cdot \mathbf{u} \end{align*}
and
\begin{align*} M\mathbf{v} \amp = \begin{bmatrix} 5/3 \amp -2/3 \\ -1/3 \amp 4/3 \end{bmatrix}\begin{bmatrix} 1 \\ -1/2 \end{bmatrix}\\ \amp = \begin{bmatrix} 5/3 - 2/3 \cdot (-1/2) \\ -1/3 + 4/3 \cdot (-1/2) \end{bmatrix}\\ \amp = \begin{bmatrix} 5/3 + 1/3 \\ -1/3 - 2/3 \end{bmatrix}\\ \amp = \begin{bmatrix} 2 \\ -1 \end{bmatrix}\\ \amp = 2 \cdot \begin{bmatrix} 1 \\ -1/2 \end{bmatrix}\\ \amp = 2 \cdot \mathbf{v}\text{.} \end{align*}
Therefore, \(\mathbf{u}\) is an eigenvector with eigenvalue \(\lambda_1 = 1\) and \(\mathbf{v}\) is an eigenvector with eigenvalue \(\lambda_2 = 2\text{.}\)
🔗
🔗
🔗
Figure 5.3.12 illustrates the effect of the matrix \(M\) from Activity 5.3.7 on a grid in \(\R^2\text{.}\) In particular, it highlights an important geometric interpretation of eigenvectors and eigenvalues; when we apply the linear map defined by the matrix \(M\) to any vector in \(\R^2\text{,}\) the matrix map stretches eigenvector directions by their eigenvalues.
🔗
Before-and-after grid diagram for a matrix with two highlighted eigenvector directions.
The left panel shows a square coordinate grid with a red line in the direction \(\mathbf{u}=(1,1)\) and a blue line in the direction \(\mathbf{v}=(1,-1/2)\text{.}\) The right panel shows the image of the grid under \(M\text{.}\) The red direction is unchanged, while the blue direction is stretched to twice its original length.
🔗
Figure 5.3.12. Visualization of the action of the matrix \(M\) from Activity 5.3.7 on a grid. The red line along the eigenvector \(\mathbf{u}\) is unchanged, because \(\mathbf{u}\) has eigenvalue \(1\text{.}\) The blue line along the eigenvector \(\mathbf{v}\) is stretched by a factor of \(2\text{,}\) because \(\mathbf{v}\) has eigenvalue \(2\text{.}\) Adapted from: Stanford’s MATH 51 textbook.
🔗

Remark 5.3.13. Non-diagonalizable matrices.

Consider the rotation matrix
\begin{equation*} A_\theta = \begin{bmatrix} \cos\theta \amp -\sin\theta \\ \sin\theta \amp \cos\theta \end{bmatrix} \end{equation*}
where \(\theta\) is not a multiple of \(180°\text{.}\) For any nonzero vector \(\mathbf{v} \in \R^2\text{,}\) the vector \(A_\theta\mathbf{v}\) is obtained by rotating \(\mathbf{v}\) counter-clockwise by an angle \(\theta\text{,}\) and so \(A \mathbf{v}\) is never on the line spanned by \(\mathbf{v}\text{,}\) i.e., it is never true that the equation \(A \mathbf{v} = \lambda \mathbf{v}\) for any \(\lambda\text{,}\) and any non-zero \(\mathbf{v}\text{.}\)
🔗
Thus the matrix \(A_\theta\) has no eigenvectors or eigenvalues. It therefore is a matrix that is not diagonalizable. This makes geometric sense: we cannot find special directions in \(\R^2\) so that the action of rotating vectors in the plane is obtained by stretching in those special directions.
🔗
🔗
Remark 5.3.13 illustrates why not every matrix is diagonalizable. A matrix is diagonalizable precisely when it has a basis of eigenvectors, but this is not always possible.
🔗

Activity 5.3.8.

Let
\begin{equation*} A=\begin{bmatrix} 3 \amp 0 \\ -1 \amp -4 \end{bmatrix}\text{.} \end{equation*}
Is \(A\) diagonalizable?
🔗
Solution.
We need to check if \(A\) has a basis of eigenvectors. In Activity 5.3.2, we showed that \(A\) has eigenvectors
\begin{equation*} \mathbf{v}_1 = \begin{bmatrix} -7 \\ 1 \end{bmatrix} \quad\text{and}\quad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}\text{.} \end{equation*}
The two vectors are linearly independent as they are not parallel, and thus form a basis for \(\R^2\text{.}\) Therefore, \(A\) is diagonalizable.
🔗
🔗
🔗

Activity 5.3.9.

Let
\begin{equation*} A = \begin{bmatrix} 4 \amp 1 \\ 0 \amp 4 \end{bmatrix}\text{.} \end{equation*}
Is \(A\) diagonalizable?
🔗
Solution.
We start by calculating the characteristic polynomial of \(A\text{,}\) that
\begin{equation*} \det(\lambda I - A) = \det\begin{bmatrix} \lambda-4 \amp -1 \\ 0 \amp \lambda-4 \end{bmatrix} = (\lambda-4)^2\text{.} \end{equation*}
Thus \(\lambda = 4\) is the only eigenvalue of \(A\text{.}\)
🔗
To find the eigenvectors of \(A\text{,}\) we solve the equation \((4I - A)\mathbf{v} = \mathbf{0}\text{,}\) which we expand as
\begin{equation*} \begin{bmatrix} 0 \amp 1 \\ 0 \amp 0 \end{bmatrix}\begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}\text{.} \end{equation*}
The equation has only a single basic solution, i.e.,
\begin{equation*} \begin{bmatrix} 1 \\ 0 \end{bmatrix}\text{.} \end{equation*}
Recalling Fact 5.3.8, every eigenvector of \(A\) is a scalar multiple of this vector. So \(A\) cannot have a basis of eigenvectors, and so \(A\) is not diagonalizable.
🔗
🔗
🔗
🔗
🔗
PrevTopNext