Skip to main content

Understanding Linear Algebra: Data Science Edition

Randall Pruim

Section 5.2 Finding eigenvalues and eigenvectors

The last section introduced eigenvalues and eigenvectors, presented the underlying geometric intuition behind their definition, and demonstrated their use in understanding the long-term behavior of certain systems. We will now develop a more algebraic understanding of eigenvalues and eigenvectors. In particular, we will find an algebraic method for determining the eigenvalues and eigenvectors of a square matrix.

Preview Activity 5.2.1.

Let’s begin by reviewing some important ideas that we have seen previously.
  1. Suppose that \(A\) is a square matrix and that the nonzero vector \(\xvec\) is a solution to the homogeneous equation \(A\xvec = \zerovec\text{.}\) What can we conclude about the invertibility of \(A\text{?}\)
  2. How does the determinant \(\det(A)\) tell us if there is a nonzero solution to the homogeneous equation \(A\xvec = \zerovec\text{?}\)
  3. Suppose that
    \begin{equation*} A = \left[\begin{array}{rrr} 3 \amp -1 \amp 1 \\ 0 \amp 2 \amp 4 \\ 1 \amp 1 \amp 3 \\ \end{array}\right]\text{.} \end{equation*}
    Find the determinant \(\det(A)\text{.}\) What does this tell us about the solution space to the homogeneous equation \(A\xvec = \zerovec\text{?}\)
  4. Find a basis for \(\nul(A)\text{.}\)
  5. What is the relationship between the rank of a matrix and the dimension of its null space?
Solution.
  1. The matrix cannot have a pivot position in every column so it is not invertible.
  2. If there is a nonzero solution to the homogeneous equation \(A\xvec=\zerovec\text{,}\) then \(A\) is not invertible so \(\det(A) = 0\text{.}\)
  3. We find that \(\det(A) = 0\) so there is a nonzero solution to the homogeneous equation.
  4. The reduced row echelon form of \(A\) is
    \begin{equation*} \left[\begin{array}{rrr} 1 \amp 0 \amp 1 \\ 0 \amp 1 \amp 2 \\ 0 \amp 0 \amp 0 \\ \end{array}\right] \end{equation*}
    so the solution space to the homogeneous equation may be described parametrically as \(x=x_3\threevec{-1}{-2}{1}\text{.}\) A basis for \(\nul(A)\) is therefore \(\threevec{-1}{-2}{1}\text{.}\)
  5. If \(A\) is an \(n\by n\) matrix, then \(\dim~\nul(A) = n-\rank(A)\text{.}\)

Subsection 5.2.1 The characteristic polynomial

We will first see that the eigenvalues of a square matrix appear as the roots of a particular polynomial. To begin, notice that we originally defined an eigenvector as a nonzero vector \(\vvec\) that satisfies the equation \(A\vvec = \lambda\vvec\text{.}\) We will rewrite this as
\begin{equation*} \begin{aligned} A\vvec \amp {}={} \lambda\vvec \\ A\vvec - \lambda\vvec \amp {}={} \zerovec \\ A\vvec - \lambda I\vvec \amp {}={} \zerovec \\ (A-\lambda I)\vvec \amp {}={} \zerovec\text{.} \\ \end{aligned} \end{equation*}
In other words, an eigenvector \(\vvec\) is a solution of the homogeneous equation \((A-\lambda I)\vvec=\zerovec\text{.}\) This puts us in the familiar territory explored in the next activity.

Activity 5.2.2.

The eigenvalues of a square matrix are defined by the condition that there be a nonzero solution to the homogeneous equation \((A-\lambda I)\vvec=\zerovec\text{.}\)
  1. If there is a nonzero solution to the homogeneous equation \((A-\lambda I)\vvec = \zerovec\text{,}\) what can we conclude about the invertibility of the matrix \(A-\lambda I\text{?}\)
  2. If there is a nonzero solution to the homogeneous equation \((A-\lambda I)\vvec = \zerovec\text{,}\) what can we conclude about the determinant \(\det(A-\lambda I)\text{?}\)
  3. Let’s consider the matrix
    \begin{equation*} A = \left[\begin{array}{rr} 1 \amp 2 \\ 2 \amp 1 \\ \end{array}\right] \end{equation*}
    from which we construct
    \begin{equation*} A-\lambda I = \left[\begin{array}{rr} 1 \amp 2 \\ 2 \amp 1 \\ \end{array}\right] - \lambda \left[\begin{array}{rr} 1 \amp 0 \\ 0 \amp 1 \\ \end{array}\right] = \left[\begin{array}{rr} 1-\lambda \amp 2 \\ 2 \amp 1-\lambda \\ \end{array}\right]\text{.} \end{equation*}
    Find the determinant \(\det(A-\lambda I)\text{.}\) What kind of equation do you obtain when we set this determinant to zero to obtain \(\det(A-\lambda I) = 0\text{?}\)
  4. Use the determinant you found in the previous part to find the eigenvalues \(\lambda\) by solving the equation \(\det(A-\lambda I) = 0\text{.}\) We considered this matrix in Activity 5.1.2 so we should find the same eigenvalues for \(A\) that we found by reasoning geometrically there.
  5. Consider the matrix \(A = \left[\begin{array}{rr} 2 \amp 1 \\ 0 \amp 2 \\ \end{array}\right]\) and find its eigenvalues by solving the equation \(\det(A-\lambda I) = 0\text{.}\)
  6. Consider the matrix \(A = \left[\begin{array}{rr} 0 \amp -1 \\ 1 \amp 0 \\ \end{array}\right]\) and find its eigenvalues by solving the equation \(\det(A-\lambda I) = 0\text{.}\)
  7. Find the eigenvalues of the triangular matrix \(\left[\begin{array}{rrr} 3 \amp -1 \amp 4 \\ 0 \amp -2 \amp 3 \\ 0 \amp 0 \amp 1 \\ \end{array}\right] \text{.}\) What is generally true about the eigenvalues of a triangular matrix?
Solution.
  1. The matrix \(A-\lambda I\) cannot be invertible.
  2. It must be the case that \(\det(A-\lambda I) = 0\text{.}\)
  3. We find that \(\det(A-\lambda I) = \lambda^2 - 2\lambda - 3 = 0\text{.}\)
  4. \(\lambda^2 - 2\lambda - 3= (\lambda-3)(\lambda+1) = 0\) so we find eigenvalues \(\lambda = 3\) and \(\lambda = -1\text{.}\)
  5. For this matrix, we have \(\det(A-\lambda I) = (\lambda-2)^2=0\) so there is one eigenvalue, \(\lambda = 2\text{.}\)
  6. \(\det(A-\lambda I) = \lambda^2 + 1 = 0\) so there are complex eigenvalues, \(\lambda = i\) and \(\lambda = -i\text{.}\)
  7. Because the determinant of a triangular matrix equals the product of its diagonal entries, The eigenvalues are equal to the entries on the diagonal.
This activity demonstrates a technique that enables us to find the eigenvalues of a square matrix \(A\text{.}\) Since an eigenvalue \(\lambda\) is a scalar for which the equation \((A-\lambda I)\vvec = \zerovec\) has a nonzero solution, it must be the case that \(A-\lambda I\) is not invertible. Therefore, its determinant is zero. This gives us the equation
\begin{equation*} \det(A-\lambda I) = 0 \end{equation*}
whose solutions are the eigenvalues of \(A\text{.}\) This equation is called the characteristic equation of \(A\text{.}\)

Example 5.2.1.

If we write the characteristic equation for the matrix \(A = \left[\begin{array}{rr} -4 \amp 4 \\ -12 \amp 10 \\ \end{array}\right] \text{,}\) we see that
\begin{equation*} \begin{aligned} \det(A-\lambda I) \amp {}={} 0 \\ \\ \det \left[\begin{array}{rr} -4 - \lambda \amp 4 \\ -12 \amp 10 - \lambda \\ \end{array}\right] \amp {}={} 0 \\ \\ (-4-\lambda)(10-\lambda) + 48 \amp {}={} 0 \\ \lambda^2-6\lambda + 8 {}={} 0 \\ (\lambda-4)(\lambda-2) {}={} 0\text{.} \\ \end{aligned} \end{equation*}
This shows us that the eigenvalues are \(\lambda = 4\) and \(\lambda=2\text{.}\)
In general, the expression \(\det(A-\lambda I)\) is a polynomial in \(\lambda\text{,}\) which is called the characteristic polynomial of \(A\text{.}\) If \(A\) is an \(n\by n\) matrix, the degree of the characteristic polynomial is \(n\text{.}\) For instance, if \(A\) is a \(2\by2\) matrix, then \(\det(A-\lambda I)\) is a quadratic polynomial; if \(A\) is a \(3\by3\) matrix, then \(\det(A-\lambda I)\) is a cubic polynomial.
The matrix in Example 5.2.1 has a characteristic polynomial with two real and distinct roots. This will not always be the case, as demonstrated in the next two examples.

Example 5.2.2.

Consider the matrix \(A=\begin{bmatrix} 5 \amp -1\\ 4 \amp 1 \\ \end{bmatrix} \text{,}\) whose characteristic equation is
\begin{equation*} \lambda^2 - 6\lambda + 9 = (\lambda-3)^2 = 0. \end{equation*}
In this case, the characteristic polynomial has one real root, which means that this matrix has a single real eigenvalue, \(\lambda = 3\text{.}\)

Example 5.2.3.

To find the eigenvalues of a triangular matrix, we remember that the determinant of a triangular matrix is the product of the entries on the diagonal. For instance, the following triangular matrix has the characteristic equation
\begin{equation*} \begin{aligned} \det\left( \left[\begin{array}{rrr} 4 \amp 2 \amp 3 \\ 0 \amp -2 \amp -1 \\ 0 \amp 0 \amp 3 \\ \end{array}\right] -\lambda I\right) \amp {}={} \det \left[\begin{array}{rrr} 4-\lambda \amp 2 \amp 3 \\ 0 \amp -2-\lambda \amp -1 \\ 0 \amp 0 \amp 3-\lambda \\ \end{array}\right] \\ \\ \amp {}={}(4-\lambda)(-2-\lambda)(3-\lambda) = 0\text{,} \end{aligned} \end{equation*}
showing that the eigenvalues are the diagonal entries \(\lambda = 4,-2,3\text{.}\)
Example 5.2.3 is important enough to present as the following proposition.

Subsection 5.2.2 Finding eigenvectors

Now that we can find the eigenvalues of a square matrix \(A\) by solving the characteristic equation \(\det(A-\lambda I) = 0\text{,}\) we will turn to the question of finding the eigenvectors associated to an eigenvalue \(\lambda\text{.}\) The key, as before, is to note that an eigenvector is a nonzero solution to the homogeneous equation \((A-\lambda I)\vvec = \zerovec\text{.}\) In other words, the eigenvectors associated to an eigenvalue \(\lambda\) form the null space \(\nul(A-\lambda I)\text{.}\)
This shows that the eigenvectors associated to an eigenvalue form a subspace of \(\real^n\text{.}\) We will denote the subspace of eigenvectors of a matrix \(A\) associated to the eigenvalue \(\lambda\) by \(E_\lambda\) and note that
\begin{equation*} E_\lambda = \nul(A-\lambda I)\text{.} \end{equation*}
We say that \(E_\lambda\) is the eigenspace of \(A\) associated to the eigenvalue \(\lambda\text{.}\)

Activity 5.2.3.

In this activity, we will find the eigenvectors of a matrix as the null space of the matrix \(A-\lambda I\text{.}\)
  1. Let’s begin with the matrix \(A = \left[\begin{array}{rr} 1 \amp 2 \\ 2 \amp 1 \\ \end{array}\right] \text{.}\) We have seen that \(\lambda = 3\) is an eigenvalue. Form the matrix \(A-3I\) and find a basis for the eigenspace \(E_3 = \nul(A-3I)\text{.}\) What is the dimension of this eigenspace? For each of the basis vectors \(\vvec\text{,}\) verify that \(A\vvec = 3\vvec\text{.}\)
  2. We also saw that \(\lambda = -1\) is an eigenvalue. Form the matrix \(A-(-1)I\) and find a basis for the eigenspace \(E_{-1}\text{.}\) What is the dimension of this eigenspace? For each of the basis vectors \(\vvec\text{,}\) verify that \(A\vvec = -\vvec\text{.}\)
  3. Is it possible to form a basis of \(\real^2\) consisting of eigenvectors of \(A\text{?}\)
  4. Now consider the matrix \(A = \left[\begin{array}{rr} 3 \amp 0 \\ 0 \amp 3 \\ \end{array}\right] \text{.}\) Write the characteristic equation for \(A\) and use it to find the eigenvalues of \(A\text{.}\) For each eigenvalue, find a basis for its eigenspace \(E_\lambda\text{.}\) Is it possible to form a basis of \(\real^2\) consisting of eigenvectors of \(A\text{?}\)
  5. Next, consider the matrix \(A = \left[\begin{array}{rr} 2 \amp 1 \\ 0 \amp 2 \\ \end{array}\right] \text{.}\) Write the characteristic equation for \(A\) and use it to find the eigenvalues of \(A\text{.}\) For each eigenvalue, find a basis for its eigenspace \(E_\lambda\text{.}\) Is it possible to form a basis of \(\real^2\) consisting of eigenvectors of \(A\text{?}\)
  6. Finally, find the eigenvalues and eigenvectors of the diagonal matrix \(A = \left[\begin{array}{rr} 4 \amp 0 \\ 0 \amp -1 \\ \end{array}\right] \text{.}\) Explain your result by considering the geometric effect of the matrix transformation defined by \(A\text{.}\)
Solution.
  1. We have
    \begin{equation*} A - 3I = \mattwo{-2}{2}{2}{-2} \sim \mattwo{1}{-1}{0}{0}\text{.} \end{equation*}
    The null space \(\nul(A-3I)\) is one-dimensional with basis \(\twovec{1}{1}\text{.}\)
  2. We have
    \begin{equation*} A - (-1)I = \mattwo{2}{2}{2}{2} \sim \mattwo{1}{1}{0}{0}\text{.} \end{equation*}
    The null space \(\nul(A+I)\) is one-dimensional with basis \(\twovec{-1}{1}\text{.}\)
  3. We can form a basis for \(\real^2\) consisting of eigenvectors of \(A\) by taking \(\left\{\twovec{1}{1}, \twovec{-1}{1}\right\}\text{.}\)
  4. The characteristic equation is \((3-\lambda)^2 = 0\text{,}\) which means that there is a single eigenvalue \(\lambda=3\text{.}\) This eigenspace is two-dimensional with basis \(\left\{\twovec{1}{0},\twovec{0}{1}\right\}\text{.}\) In this case, we can form a basis for \(\real^2\) consisting of eigenvectors of \(A\text{.}\)
  5. The characteristic equation is \((2-\lambda)^2 = 0\) so there is again a single eigenvalue \(\lambda=2\text{.}\) In this case, the eigenspace is one-dimensional with basis vector \(\twovec{1}{0}\text{.}\) It is not possible to form a basis for \(\real^2\) consisting of eigenvectors.
  6. We have eigenvectors \(\twovec10\) with associated eigenvector \(\lambda=4\) and \(\twovec01\) with associated eigenvector \(\lambda=-1\text{.}\)
Once we find an eigenvalue of a matrix \(A\text{,}\) describing the associated eigenspace \(E_\lambda\) amounts to the familiar task of describing the null space \(\nul(A-\lambda I)\text{.}\)

Example 5.2.5.

Revisiting the matrix \(A = \left[\begin{array}{rr} -4 \amp 4 \\ -12 \amp 10 \\ \end{array}\right]\) from Example 5.2.1, we recall that we found eigenvalues \(\lambda = 4\) and \(\lambda = 2\text{.}\)
Considering the eigenvalue \(\lambda = 4\text{,}\) we have
\begin{equation*} A -4I = \begin{bmatrix} -8 \amp 4 \\ -12 \amp 6 \\ \end{bmatrix} \sim \begin{bmatrix} 1 \amp -1/2 \\ 0 \amp 0 \\ \end{bmatrix}. \end{equation*}
Since the eigenvectors \(\vvec = \twovec{v_1}{v_2}\) are the solutions of the equation \((A-4I)\vvec=\zerovec\text{,}\) we see that they are determined by the single equation \(v_1-\frac12 v_2 = 0\) or \(v_1 = \frac12v_2\text{.}\) Therefore the eigenvectors in \(E_4\) have the form
\begin{equation*} \vvec=\twovec{v_1}{v_2} = \ctwovec{\frac12 v_2}{v_2} = v_2\ctwovec{1/2}{1}. \end{equation*}
In other words, \(E_4\) is a one-dimensional subspace of \(\real^2\) with basis vector \(\ctwovec{1/2}{1}\) or basis vector \(\twovec12\text{.}\) In the same way, we find that a basis for the eigenspace \(E_2\) is \(\twovec23\text{.}\)
We note that, for this matrix, it is possible to construct a basis of \(\real^2\) consisting of eigenvectors, namely,
\begin{equation*} \bcal = \left\{\twovec12, \twovec23\right\}. \end{equation*}

Example 5.2.6.

Consider the matrix \(A=\begin{bmatrix} 1 \amp 1 \\ -1 \amp 3 \\ \end{bmatrix}\) whose characteristic equation is
\begin{equation*} \det(A-\lambda I) = \lambda^2 - 4\lambda + 4 = (\lambda-2)^2 =0. \end{equation*}
There is a single eigenvalue \(\lambda= 2\text{,}\) and we find that
\begin{equation*} A - 2\lambda = \begin{bmatrix} -1 \amp 1 \\ -1 \amp 1 \\ \end{bmatrix} \sim \begin{bmatrix} 1 \amp -1 \\ 0 \amp 0 \\ \end{bmatrix}. \end{equation*}
Therefore, the eigenspace \(E_2 = \nul(A-2I)\) is one-dimensional with a basis vector \(\twovec11\text{.}\)

Example 5.2.7.

If \(A=\begin{bmatrix} -1 \amp 0 \\ 0 \amp -1 \\ \end{bmatrix} \text{,}\) then
\begin{equation*} \det(A - \lambda I) = (\lambda+1)^2 = 0, \end{equation*}
which implies that there is a single eigenvalue \(\lambda=-1\text{.}\) We find that
\begin{equation*} A-(-1)I = \begin{bmatrix} 0 \amp 0 \\ 0 \amp 0 \\ \end{bmatrix}, \end{equation*}
which says that every two-dimensional vector \(\vvec\) satisfies \((A-(-1)I)\vvec = \zerovec\text{.}\) Therefore, every vector is an eigenvector and so \(E_{-1} = \real^2\text{.}\) This eigenspace is two-dimensional.
We can see this in another way. The matrix transformation defined by \(A\) rotates vectors by \(180^\circ\text{,}\) which says that \(A\xvec = -\xvec\) for every vector \(\xvec\text{.}\) In other words, every two-dimensional vector is an eigenvector with associated eigenvalue \(\lambda=-1\text{.}\)
These last two examples illustrate two types of behavior when there is a single eigenvalue. In one case, we are able to construct a basis of \(\real^2\) using eigenvectors; in the other, we are not. We will explore this behavior more in the next subsection.

A check on our work.

When finding eigenvalues and their associated eigenvectors in this way, we first find eigenvalues \(\lambda\) by solving the characteristic equation. If \(\lambda\) is a solution to the characteristic equation, then \(A-\lambda I\) is not invertible and, consequently, \(A-\lambda I\) must contain a row without a pivot position.
This serves as a check on our work. If we row reduce \(A-\lambda I\) and find the identity matrix, then we have made an error either in solving the characteristic equation or in finding \(\nul(A-\lambda I)\text{.}\)

Subsection 5.2.3 The characteristic polynomial and the dimension of eigenspaces

Given a square \(n\by n\) matrix \(A\text{,}\) we saw in the previous section the value of being able to express any vector in \(\real^n\) as a linear combination of eigenvectors of \(A\text{.}\) For this reason, Question 5.1.8 asks when we can construct a basis of \(\real^n\) consisting of eigenvectors. We will explore this question more fully now.
As we saw above, the eigenvalues of \(A\) are the solutions of the characteristic equation \(\det(A-\lambda I) = 0\text{.}\) The examples we have considered demonstrate some different types of behavior. For instance, we have seen the characteristic equations
  • \((4-\lambda)(-2-\lambda)(3-\lambda) = 0\text{,}\) which has real and distinct roots,
  • \((2-\lambda)^2 = 0\text{,}\) which has repeated roots, and
  • \(\lambda^2 +1 = (i-\lambda)(-i-\lambda) = 0\text{,}\) which has complex roots.
If \(A\) is an \(n\by n\) matrix, then the characteristic polynomial is a degree \(n\) polynomial, and this means that it has \(n\) roots. Therefore, the characteristic equation can be written as
\begin{equation*} \det(A-\lambda I) = (\lambda_1 - \lambda)(\lambda_2-\lambda)\ldots(\lambda_n-\lambda) = 0 \end{equation*}
giving eigenvalues \(\lambda_1, \lambda_2,\ldots,\lambda_n\text{.}\) As we have seen, some of the eigenvalues may be complex. Moreover, some of the eigenvalues may appear in this list more than once. However, we can always write the characteristic equation in the form
\begin{equation*} (\lambda_1-\lambda)^{m_1} (\lambda_2-\lambda)^{m_2} \ldots (\lambda_p-\lambda)^{m_p} = 0. \end{equation*}
The number of times that \(\lambda_j - \lambda\) appears as a factor in the characteristic polynomial, is called the multiplicity of the eigenvalue \(\lambda_j\text{.}\)

Example 5.2.8.

We have seen that the matrix \(A = \left[\begin{array}{rr} 1 \amp 1 \\ -1 \amp 3 \\ \end{array}\right]\) has the characteristic equation \((2-\lambda)^2 = 0\text{.}\) This matrix has a single eigenvalue \(\lambda = 2\text{,}\) which has multiplicity \(2\text{.}\)

Example 5.2.9.

If a matrix has the characteristic equation
\begin{equation*} (4-\lambda)^2(-5-\lambda)(1-\lambda)^7(3-\lambda)^2 = 0\text{,} \end{equation*}
then that matrix has four eigenvalues: \(\lambda=4\) having multiplicity 2; \(\lambda=-5\) having multiplicity 1; \(\lambda=1\) having multiplicity 7; and \(\lambda=3\) having multiplicity 2. The degree of the characteristic polynomial is the sum of the multiplicities \(2+1+7+2 = 12\) so this matrix must be a \(12\by12\) matrix.
The multiplicities of the eigenvalues are important because they reflect the dimension of the eigenspaces. We know that the dimension of an eigenspace must be at least one; the following proposition also tells us the dimension of an eigenspace can be no larger than the multiplicity of its associated eigenvalue.

Example 5.2.11.

The diagonal matrix \(\left[\begin{array}{rr} -1 \amp 0 \\ 0 \amp -1 \\ \end{array}\right]\) has the characteristic equation \((-1-\lambda)^2 = 0\text{.}\) There is a single eigenvalue \(\lambda = -1\) having multiplicity \(m = 2\text{,}\) and we saw earlier that \(\dim E_{-1} = 2 \leq m = 2\text{.}\)

Example 5.2.12.

The matrix \(\left[\begin{array}{rr} 1 \amp 1 \\ -1 \amp 3 \\ \end{array}\right]\) has the characteristic equation \((2-\lambda)^2 = 0\text{.}\) This tells us that there is a single eigenvalue \(\lambda = 2\) having multiplicity \(m = 2\text{.}\) In contrast with the previous example, we have \(\dim~ E_2 = 1 \leq m = 2\text{.}\)

Example 5.2.13.

We saw earlier that the matrix \(\left[\begin{array}{rrr} 4 \amp 2 \amp 3 \\ 0 \amp -2 \amp -1 \\ 0 \amp 0 \amp 3 \\ \end{array}\right]\) has the characteristic equation
\begin{equation*} (4-\lambda)(-2-\lambda)(3-\lambda)=0\text{.} \end{equation*}
There are three eigenvalues \(\lambda=3,-2,1\) each having multiplicity \(1\text{.}\) By the proposition, we are guaranteed that the dimension of each eigenspace is \(1\text{;}\) that is,
\begin{equation*} \dim E_3 = \dim E_{-2} = \dim E_1 = 1\text{.} \end{equation*}
It turns out that this is enough to guarantee that there is a basis of \(\real^3\) consisting of eigenvectors.

Example 5.2.14.

If a \(12\by12\) matrix has the characteristic equation
\begin{equation*} (4-\lambda)^2(-5-\lambda)(1-\lambda)^7(3-\lambda)^2 = 0\text{,} \end{equation*}
we know there are four eigenvalues \(\lambda=4,-5,1,3\text{.}\) Without more information, all we can say about the dimensions of the eigenspaces is
\begin{equation*} \begin{aligned} 1 \leq \dim E_4 \amp {}\leq{} 2 \\ 1 \leq \dim E_{-5} \amp {}\leq{} 1 \\ 1 \leq \dim E_1 \amp {}\leq{} 7 \\ 1 \leq \dim E_3 \amp {}\leq{} 2\text{.} \\ \end{aligned} \end{equation*}
We can guarantee that \(\dim E_{-5} = 1\text{,}\) but we cannot be more specific about the dimensions of the other eigenspaces.
Fortunately, if we have an \(n\by n\) matrix, it frequently happens that the characteristic equation has the form
\begin{equation*} (\lambda_1-\lambda) (\lambda_2-\lambda) \ldots (\lambda_n-\lambda) = 0 \end{equation*}
where there are \(n\) distinct real eigenvalues, each of which has multiplicity \(1\text{.}\) In this case, the dimension of each of the eigenspaces \(\dim E_{\lambda_j} = 1\text{.}\) With a little work, it can be seen that choosing a basis vector \(\vvec_j\) for each of the eigenspaces produces a basis for \(\real^n\text{.}\) We therefore have the following proposition.
This proposition provides one answer to our Question 5.1.8. The next activity explores this question further.

Activity 5.2.4.

  1. Identify the eigenvalues, and their multiplicities, of an \(n\by n\) matrix whose characteristic polynomial is \((2-\lambda)^3(-3-\lambda)^{10}(5-\lambda)\text{.}\) What can you conclude about the dimensions of the eigenspaces? What is the shape of the matrix? Do you have enough information to guarantee that there is a basis of \(\real^n\) consisting of eigenvectors?
  2. Find the eigenvalues of \(\left[\begin{array}{rr} 0 \amp -1 \\ 4 \amp -4 \\ \end{array}\right]\) and state their multiplicities. Can you find a basis of \(\real^2\) consisting of eigenvectors of this matrix?
  3. Consider the matrix \(A = \left[\begin{array}{rrr} -1 \amp 0 \amp 2 \\ -2 \amp -2 \amp -4 \\ 0 \amp 0 \amp -2 \\ \end{array}\right]\) whose characteristic equation is
    \begin{equation*} (-2-\lambda)^2(-1-\lambda) = 0\text{.} \end{equation*}
    1. Identify the eigenvalues and their multiplicities.
    2. For each eigenvalue \(\lambda\text{,}\) find a basis of the eigenspace \(E_\lambda\) and state its dimension.
    3. Is there a basis of \(\real^3\) consisting of eigenvectors of \(A\text{?}\)
  4. Now consider the matrix \(A = \left[\begin{array}{rrr} -5 \amp -2 \amp -6 \\ -2 \amp -2 \amp -4 \\ 2 \amp 1 \amp 2 \\ \end{array}\right]\) whose characteristic equation is also
    \begin{equation*} (-2-\lambda)^2(-1-\lambda) = 0\text{.} \end{equation*}
    1. Identify the eigenvalues and their multiplicities.
    2. For each eigenvalue \(\lambda\text{,}\) find a basis of the eigenspace \(E_\lambda\) and state its dimension.
    3. Is there a basis of \(\real^3\) consisting of eigenvectors of \(A\text{?}\)
  5. Consider the matrix \(A = \left[\begin{array}{rrr} -5 \amp -2 \amp -6 \\ 4 \amp 1 \amp 8 \\ 2 \amp 1 \amp 2 \\ \end{array}\right]\) whose characteristic equation is
    \begin{equation*} (-2-\lambda)(1-\lambda)(-1-\lambda) = 0\text{.} \end{equation*}
    1. Identify the eigenvalues and their multiplicities.
    2. For each eigenvalue \(\lambda\text{,}\) find a basis of the eigenspace \(E_\lambda\) and state its dimension.
    3. Is there a basis of \(\real^3\) consisting of eigenvectors of \(A\text{?}\)
Solution.
  1. There are three eigenvalues, \(\lambda=2\) has multiplicity \(3\text{,}\) \(\lambda=-3\) has multiplicity \(10\text{,}\) and \(\lambda=5\) has multiplicity \(1\text{.}\) We know that
    \begin{equation*} \begin{aligned} 1 \leq \dim E_2 \amp {}\leq{} 3 \\ 1 \leq \dim E_{-3} \amp {}\leq{} 10 \\ 1 \leq \dim E_5 \amp {}\leq{} 1\text{.} \\ \end{aligned} \end{equation*}
    We can guarantee that \(\dim E_{5} = 1\text{,}\) but we can say nothing further about the other two eigenspaces.
    The dimension of the matrix is \(14\by14\) since the degree of the characteristic polynomial is \(14\text{.}\) We cannot guarantee that we can form a basis for \(\real^{14}\) consisting of eigenvectors, however.
  2. There is one eigenvalue \(\lambda=-2\) having multiplicity two. Because the eigenspace \(E_{-2}\) is one-dimensional, however, we cannot find a basis for \(\real^2\) consisting of eigenvectors of \(A\text{.}\)
  3. For the \(3\by3\) matrix \(A\text{,}\)
    1. We have eigenvalues \(\lambda=-2\) with multiplicity \(2\) and \(\lambda=-1\) with multiplicity \(1\text{.}\)
    2. The eigenspace \(E_{-2}\) is two-dimensional with basis \(\left\{\threevec{-2}{0}{1},\threevec010\right\}\text{.}\) The eigenspace \(E_{-1}\) is one-dimensional with basis vector \(\threevec{-1}20\text{.}\)
    3. We are able to form a basis for \(\real^3\) consisting of eigenvectors of \(A\text{.}\)
  4. For the \(3\by3\) matrix \(A\text{,}\)
    1. We have eigenvalues \(\lambda=-2\) with multiplicity \(2\) and \(\lambda=-1\) with multiplicity \(1\text{.}\)
    2. The eigenspace \(E_{-2}\) is one-dimensional with basis vector \(\threevec{-2}01\text{.}\) The eigenspace \(E_{-1}\) is also one-dimensional with basis vector \(\threevec{-1}{2}{0}\text{.}\)
    3. It is not possible to form a basis for \(\real^3\) consisting of eigenvectors of \(A\text{.}\)
  5. For this matrix,
    1. There are three eigenvalues \(\lambda=-2\text{,}\) \(-1\text{,}\) and \(1\text{,}\) each having multiplicity \(1\text{.}\)
    2. A basis vector for the eigenspace \(E_{-2}\) is \(\threevec{-2}01\text{.}\) A basis vector for the eigenspace \(E_{-1}\) is \(\threevec{1}{-2}0\text{.}\) A basis vector for the eigenspace \(E_{1}\) is \(\threevec{-2}{3}{1}\text{.}\)
    3. We can form a basis for \(\real^3\) consisting of eigenvectors of \(A\text{.}\)

Subsection 5.2.4 Using Python to find eigenvalues and eigenvectors

We can use Python to find the characteristic polynomial, eigenvalues, and eigenvectors of a matrix. As we will see, however, some care is required when dealing with matrices whose entries include decimals.

Activity 5.2.5.

We will use Python to find the eigenvalues and eigenvectors of a matrix. Let’s begin with the matrix \(A = \left[\begin{array}{rr} -3 \amp 1 \\ 0 \amp -3 \\ \end{array}\right] \text{.}\)
  1. We can find the characteristic polynomial of a sympy.Matrix A by writing A.charpoly('lambda'). Notice that we have to give Python a variable in which to write the polynomial; here, we use lambda though x works just as well.
    The factored form of the characteristic polynomial may be more useful since it will tell us the eigenvalues and their multiplicities. The factored characteristic polynomial is found with sympy.factor().
  2. If we only want the eigenvalues, we can use numpy.linalg.eigvals() or scipy.linalg.eigvals().
    Notice that the multiplicity of an eigenvalue is the number of times it is repeated in the list of eigenvalues.
  3. Finally, we can find both eigenvalues and (right) eigenvectors using numpy.linalg.eig(A) or scipy.linalg.eig(A).
    The first item in the tuple returned gives the eigenvalues, the second the eigenvectors.
  4. When working with decimal entries, which are called floating point numbers in computer science, we must remember that computers perform only approximate arithmetic. This can be a problem when we wish to find the eigenvectors of a matrix. To illustrate, consider the matrix \(A=\left[\begin{array}{rr} 0.4 \amp 0.3 \\ 0.6 \amp 0.7 \\ \end{array}\right] \text{.}\)
    1. Without using Python, find the eigenvalues of this matrix.
    2. What do you find for the reduced row echelon form of \(A-I\text{?}\)
    3. Let’s now use Python to determine the reduced row echelon form of \(A-I\text{:}\)
      What result does Python report for the reduced row echelon form? Why is this result not correct? (Hint: compute row and column sums.)
    4. Despite the error in finding the reduce row echelon form, NumPy and SciPy can compute these eigenvalues and eigenvectors for us.
      As you might expect, this implies that the method being used does not rely on using RREF along the way. RREF is not numerically stable, this is why it is not included in numpy and scipy.
    5. If we provide sympy with rational values instead of floating point, then it can compute the RREF of our matrix exactly and the error from above goes away.
      In Chapter 6 we will see other methods for computing eigevectors that do not rely on RREF.
Solution.
  1. The characteristic polynomial is \(\lambda^2 + 6\lambda + 9\text{.}\)
  2. linalg.eig(A) returns [-3, -3]. This means we have one eigenvalue (\(\lambda = -3\)) with multiplicity 2.
  3. The eigenvectors are \(\twovec{1}{0}\) and \(\twovec{-1}{0}\text{.}\) Notice that \(\twovec{-1}{0} = -1 \twovec{1}{0}\text{,}\) so we really only have one independent eigevector here and the eigenspace is 1-dimensional (the x-axis).
  4. If we begin with the matrix \(A\text{,}\) we find
    1. The eigenvalues are \(\lambda=1\) and \(\lambda=0.1\text{.}\)
    2. The reduced row echelon form is \(\mattwo{-2}{1}{0}{0}\text{,}\) which shows that \(A-I\) is not invertible, as expected.
    3. Python returns \(\mattwo{1}{0}{0}{1}\text{,}\) which is not correct because \(A-I\) cannot be invertible if \(\lambda=1\) is an eigenvalue of \(A\text{.}\) The issue is round-off errors that make things that should be exactly zero slightly different from 0.
    4. Here we find the correct eigenvalues, \(\lambda=1\) with basis vector \(\twovec{1}{2}\) for \(E_1\) and \(\lambda=0.1\) with basis vector \(\twovec{1}{-1}\) for \(E_{0.1}\text{.}\)

Subsection 5.2.5 Summary

In this section, we developed a technique for finding the eigenvalues and eigenvectors of an \(n\by n\) matrix \(A\text{.}\)
  • The expression \(\det(A-\lambda I)\) is a degree \(n\) polynomial, known as the characteristic polynomial of \(A\text{.}\) The eigenvalues of \(A\) are the roots of the characteristic polynomial found by solving the characteristic equation \(\det(A-\lambda I) = 0\text{.}\)
  • The set of eigenvectors associated to the eigenvalue \(\lambda\) forms a subspace of \(\real^n\text{,}\) the eigenspace \(E_\lambda = \nul(A-\lambda I)\text{.}\)
  • If the factor \((\lambda_j - \lambda)\) appears \(m_j\) times in the characteristic polynomial, we say that the eigenvalue \(\lambda_j\) has multiplicity \(m_j\) and note that
    \begin{equation*} 1 \leq \dim E_{\lambda_j} \leq m_j\text{.} \end{equation*}
  • If each of the eigenvalues is real and has multiplicity \(1\text{,}\) then we can form a basis of \(\real^n\) consisting of eigenvectors of \(A\text{.}\)
  • We can use Python to find the eigenvalues and eigenvalues of matrices. However, we need to be careful working with floating point numbers since floating point arithmetic is only an approximation.

Exercises 5.2.6 Exercises

1.

For each of the following matrices, find its characteristic polynomial, its eigenvalues, and the multiplicity of each eigenvalue.
  1. \(A=\left[\begin{array}{rr} 4 \amp -1 \\ 4 \amp 0 \\ \end{array}\right] \text{.}\)
  2. \(A=\left[\begin{array}{rrr} 3 \amp -1 \amp 3 \\ 0 \amp 4 \amp 0 \\ 0 \amp 0 \amp -6 \end{array}\right] \text{.}\)
  3. \(A = \left[\begin{array}{rr} -2 \amp 0 \\ 0 \amp -2 \\ \end{array}\right] \text{.}\)
  4. \(A=\left[\begin{array}{rr} -1 \amp 2 \\ 2 \amp 2 \\ \end{array}\right] \text{.}\)

2.

Given an \(n\by n\) matrix \(A\text{,}\) an important question, Question 5.1.8, asks whether we can find a basis of \(\real^n\) consisting of eigenvectors of \(A\text{.}\) For each of the matrices in the previous exercise, find a basis of \(\real^n\) consisting of eigenvectors or state why such a basis does not exist.

3.

Determine whether the following statements are true or false and provide a justification for your response.
  1. The eigenvalues of a matrix \(A\) are the entries on the diagonal of \(A\text{.}\)
  2. If \(\lambda\) is an eigenvalue of multiplicity \(1\text{,}\) then \(E_\lambda\) is one-dimensional.
  3. If \(\lambda = 0\) is an eigenvalue for a matrix \(A\text{,}\) then \(A\) is not invertible.
  4. If \(A\) is a \(13\by 13\) matrix, the characteristic polynomial has degree less than \(13\text{.}\)
  5. The eigenspace \(E_\lambda\) of \(A\) is the same as the null space \(\nul(A-\lambda I)\text{.}\)

4.

Provide a justification for your response to the following questions.
  1. Suppose that \(A\) is a \(3\by 3\) matrix having eigenvalues \(\lambda = -3,3,-5\text{.}\) What are the eigenvalues of \(2A\text{?}\)
  2. Suppose that \(D\) is a diagonal \(3\by 3\) matrix. Why can you guarantee that there is a basis of \(\real^3\) consisting of eigenvectors of \(D\text{?}\)
  3. If \(A\) is a \(3\by 3\) matrix whose eigenvalues are \(\lambda = -1,3,5\text{,}\) can you guarantee that there is a basis of \(\real^3\) consisting of eigenvectors of \(A\text{?}\)
  4. Suppose that the characteristic polynomial of a matrix \(A\) is
    \begin{equation*} \det(A-\lambda I) = -\lambda^3 + 4\lambda\text{.} \end{equation*}
    What are the eigenvalues of \(A\text{?}\) Is \(A\) invertible? Is there a basis of \(\real^n\) consisting of eigenvectors of \(A\text{?}\)
  5. If the characteristic polynomial of \(A\) is
    \begin{equation*} \det(A-\lambda I) = (4 -\lambda)(-2-\lambda)(1-\lambda)\text{,} \end{equation*}
    what is the characteristic polynomial of \(A^2\text{?}\) what is the characteristic polynomial of \(A^{-1}\text{?}\)

5.

For each of the following matrices, use Python to determine its eigenvalues, their multiplicities, and a basis for each eigenspace. For which matrices is it possible to construct a basis for \(\real^3\) consisting of eigenvectors?
  1. \(\displaystyle A = \left[\begin{array}{rrr} -4 \amp 12 \amp -6 \\ 4 \amp -5 \amp 4 \\ 11 \amp -20 \amp 13 \\ \end{array}\right]\)
  2. \(\displaystyle A = \left[\begin{array}{rrr} 1 \amp -3 \amp 1 \\ -4 \amp 8 \amp -5 \\ -8 \amp 17 \amp -10 \\ \end{array}\right]\)
  3. \(\displaystyle A = \left[\begin{array}{rrr} 3 \amp -8 \amp 4 \\ -2 \amp 3 \amp -2 \\ -6 \amp 12 \amp -7 \\ \end{array}\right]\)

6.

There is a relationship between the determinant of a matrix and the product of its eigenvalues.
  1. We have seen that the eigenvalues of the matrix \(A = \left[\begin{array}{rr} 1 \amp 2 \\ 2 \amp 1 \\ \end{array}\right]\) are \(\lambda = 3,-1\text{.}\) What is \(\det A\text{?}\) What is the product of the eigenvalues of \(A\text{?}\)
  2. Consider the triangular matrix \(A = \left[\begin{array}{rrr} 2 \amp 0 \amp 0 \\ -1 \amp -3 \amp 0 \\ 3 \amp 1 \amp -2 \\ \end{array}\right] \text{.}\) What are the eigenvalues of \(A\text{?}\) What is \(\det A\text{?}\) What is the product of the eigenvalues of \(A\text{?}\)
  3. Based on these examples, what do you think is the relationship between the determinant of a matrix and the product of its eigenvalues?
  4. Suppose the characteristic polynomial is written as
    \begin{equation*} \det(A-\lambda I) = (\lambda_1-\lambda)(\lambda_2-\lambda) \ldots (\lambda_n-\lambda)\text{.} \end{equation*}
    By substituting \(\lambda = 0\) into this equation, explain why the determinant of a matrix equals the product of its eigenvalues.

7.

Consider the matrix \(A=\left[\begin{array}{rr} 0.5 \amp 0.6 \\ -0.3 \amp 1.4 \\ \end{array}\right] \text{.}\)
  1. Find the eigenvalues of \(A\) and a basis for their associated eigenspaces.
  2. Suppose that \(\xvec_0=\twovec{11}{6}\text{.}\) Express \(\xvec_0\) as a linear combination of eigenvectors of \(A\text{.}\)
  3. Define the vectors
    \begin{equation*} \begin{aligned} \xvec_1 \amp {}={} A\xvec_0 \\ \xvec_2 \amp {}={} A\xvec_1 = A^2\xvec_0 \\ \xvec_3 \amp {}={} A\xvec_2 = A^3\xvec_0 \\ \vdots \amp {}={} \vdots \end{aligned}\text{.} \end{equation*}
    Write \(\xvec_1\text{,}\) \(\xvec_2\text{,}\) and \(\xvec_3\) as a linear combination of eigenvectors of \(A\text{.}\)
  4. What happens to \(\xvec_k\) as \(k\) grows larger and larger?

8.

Consider the matrix \(A=\left[\begin{array}{rr} 0.4 \amp 0.3 \\ 0.6 \amp 0.7 \\ \end{array}\right]\)
  1. Find the eigenvalues of \(A\) and a basis for their associated eigenspaces.
  2. Suppose that \(\xvec_0=\twovec{0}{1}\text{.}\) Express \(\xvec_0\) as a linear combination of eigenvectors of \(A\text{.}\)
  3. Define the vectors
    \begin{equation*} \begin{aligned} \xvec_1 \amp {}={} A\xvec_0 \\ \xvec_2 \amp {}={} A\xvec_1 = A^2\xvec_0 \\ \xvec_3 \amp {}={} A\xvec_2 = A^3\xvec_0 \\ \vdots \amp {}={} \vdots \end{aligned}\text{.} \end{equation*}
    Write \(\xvec_1\text{,}\) \(\xvec_2\text{,}\) and \(\xvec_3\) as a linear combination of eigenvectors of \(A\text{.}\)
  4. What happens to \(\xvec_k\) as \(k\) grows larger and larger?
<Prev^TopNext>