Let us first recall some basics from linear algebra. Let be a matrix. The kernel of (frequently called the nullspace of ) is the set
In other words, the kernel of consists of vectors that get mapped to the zero vector when multiplied by . Clearly, the zero vector is in since and in fact is a subspace of because it is closed under scalar multiplication and scalar addition (verify this!). We say that has a trivial kernel if , that is, if the only vector in is the zero vector. In a first course in linear algebra you proved that has a trivial kernel if and only if is invertible (if and only if ).
Now let be a matrix with real entries. Given a non-zero vector if it holds that
for some number then we say that is an eigenvector of with eigenvalue. We will say that is an eigenpair of . Notice that we require to be non-zero, and that and also . We are restricting our considerations only to real eigenvectors and real eigenvalues although it is easy to construct matrices with real entries that do not have real eigenvectors/eigenvalues, such as
However, because the matrices we will study have only real eigenvectors and eigenvalues, this lost of generality will not be too restrictive.
If is an eigenpair of , then from we can write that or equivalently by factoring on the right we have
Hence, will have an eigenvector associated to if the matrix
has a non-trivial kernel. Now, as a non-trivial kernel if and only if is not invertible if and only if . Hence, the only way that is an eigenvalue of is if
If we replace by a variable then to find the eigenvalues of we must therefore find the roots of the polynomial
that is, we must find numbers such that . The polynomial is called the characteristic polynomial of and we have just showed that the roots of are exactly the eigenvalues of . Notice that to compute the polynomial we do not need any information about the eigenvectors of and is only used to find the eigenvalues. However, if is known to be a root of then any vector in is an eigenvector of with corresponding eigenvalue . Now since is a subspace of it has a basis, say , consisting of eigenvectors of with eigenvalue . The dimension of the subspace is called the geometric multiplicity of and is sometimes called the eigenspace associated to because it is where all the eigenvectors associated to live.
If is an eigenvalue of , the algebraic multiplicity of is the number of times that appears as a root of the characteristic polynomial . An eigenvalue is said to be simple if its algebraic multiplicity is one and said to be repeated otherwise. The geometric multiplicity is always less than or equal to the algebraic multiplicity.
Suppose that is a matrix with characteristic polynomial
By inspection we can factor :
Therefore, the eigenvalues of are , and , and thus has only three distinct eigenvalues (even though it is a matrix). The algebraic multiplicity of is and it is thus repeated, while and are both simple eigenvalues. Thus, as a set, the eigenvalues of are , whereas if we want to list all the eigenvalues of in say increasing order we obtain
The latter is sometimes called the ``set of eigenvalues listed with multiplicities'' or the ``list of eigenvalues with multiplicities''.
If is a matrix and has simple eigenvalues (i.e., all have algebraic multiplicity ) then each eigenspace is one-dimensional (i.e., it is a line in through the origin). Therefore, if and are two eigenvectors associated to the same eigenvalue then and are scalar multiplies of each other, that is, for some non-zero .
Let us now focus on the case that is a symmetric matrix, that is, . One of the most important results in linear algebra is that the eigenvalues of are all real numbers and moreover there exists an orthonormal basis of consisting of eigenvectors of . Hence, if denote the eigenvalues of (some of which may be repeated) then we are guaranteed the existence of an orthonormal basis of such that for all . If we set to be the diagonal matrix with th diagonal entry equal to and set then the condition for all can be written as the matrix equation
and therefore
However, since (because is an orthogonal matrix) we have that
In Python, one can compute the eigenvalues and eigenvectors of a symmetric matrix by using the function eigh which is contained in the module numpy.linalg or scipy.linalg. The function eigh returns a 2-tuple where the first element is an array of the eigenvalues and the second element is an orthogonal matrix consisting of the eigenvectors. For example, a typical call to eigh is
E, X = numpy.linalg.eigh(M)
and E is a array that stores the eigenvalues and X is a numpy array whose columns consist of orthonormal eigenvectors of . For example, to confirm that the 3rd column of X is an eigenvector whose eigenvalue is the 3rd entry of E we type
M @ X[:, 2] - E[2] * X[:, 2]
and the returned value should be the zero vector of length . To verify that X is an orthogonal matrix (i.e., that ) type:
X.T @ X
Automorphisms and eigenpairs
Recall that if is the permutation matrix of the permutation then if and only if or equivalently . Our first result describes how eigenvectors behave under the action of an automorphism of .
Let be a graph with adjacency matrix and suppose that is the permutation matrix representation of an automorphism . If is an eigenpair of then so is .
Let be an eigenvector of with eigenvalue , that is, . Since is an automorphism of we have that and therefore
Thus, is an eigenvector of with the same eigenvalue as .
Consider the graph shown in Figure 6.1 with adjacency matrix
Graph for Example 6.4
The spectrum of and corresponding eigenvectors written as the columns of the matrix are
One can verify that for instance is an automorphism of . If denotes the permutation matrix of , one can verify that which is an eigenvector of with same eigenvalue as . As another example, one can verify that and that . Hence, in some cases is a scalar multiple of and in some cases is a non-scalar multiple of ; the latter case can occur if the eigenvalue associated to is repeated as it occurs with and (in this case ).
Our next result relates the algebraic multiplicities of the eigenvalues of with the order of the automorphisms of .
Let be a graph with adjacency matrix . If has simple eigenvalues then every non-identity automorphism of has order . In particular, the automorphism group is an abelian group.
Since has simple eigenvalues then is one-dimensional, and therefore for some scalar (see Example 6.2). Therefore, multiplying the equation by on the left we obtain
Now since is an orthogonal matrix, , and thus which implies that . Therefore,
Hence, the matrix has as an eigenvector with eigenvalue . Since was an arbitrary eigenvector of , if is a basis of consisting of orthonormal eigenvectors of then consists of eigenvectors of all of which have eigenvalue . Therefore, if then
and therefore since is invertible we have
Thus is the identity permutation and consequently has order . That is abelian then follows from Example 1.28.
Proposition 6.2.2 does not say that will necessarily contain non-trivial automorphisms (of order 2). In fact, most graphs will have distinct eigenvalues and a trivial automorphism group, that is, . What Proposition 6.2.2 does say is that if there is any non-trivial automorphism then has order whenever has distinct eigenvalues.
Before we move on, we need to recall the notion of a partition of a set . A partition of is a collection of subsets such that and for all . The subsets are called the cells of the partition . If has -cells then it is called a -partition. For example, if then is a partition of and it has cells. The unit partition of is the partition that contains only one cell, namely, and the discrete partition of is the partition that contains cells, namely, . We will refer to these partitions as the trivial partitions of .
Given a set , how many partitions of are there? The total number of partitions of an -element set is the Bell number. The first several Bell numbers are , , , , , , and . The Bell numbers satisfy the recursion
See the Wiki page on \href{https://en.wikipedia.org/wiki/Bell_number}{Bell numbers} for more interesting facts.
Every permutation induces a partition of as follows. Suppose that has cycles in its cycle decomposition:
The sets formed from the integers within each cycle of and the singleton sets formed from the remaining integers fixed by forms a partition of the vertex set . In other words, if are the integers fixed by , and we set , then if we set
then is a partition of . The notation is messy but an example will make the above clear.
As an example, consider given by
Hence, fixes the integer . Thus, if we set , , , , , then is a partition of the vertex set . We say that the partition is induced by the permutation .
Suppose now that and let be the partition of induced by the cycle decomposition of . Pick a cell and pick a vertex and suppose that has neighbors in cell , say that they are . Since sends a vertex in one cell to a vertex in the same cell (recall that each cell consists of the integers in a cycle) then necessarily and for . Now since , it follows that and therefore the neighbors of in are . Hence, the number of vertices in adjacent to is equal to the number of vertices in adjacent to , in this case . Since cycles through all the vertices in , it follows that all the vertices in have the same number of neighbors in . Surprisingly, this observation turns out to be an important one.
We introduce some notation to capture what we have just discussed. Given a partition of , define for any vertex the degree of in by
Formally, if denotes all the vertices adjacent to then
Using this notation, what we showed above is that for any two cells and contained in a partition induced by an automorphism of , the number
is the same for all . Now, could be zero and it is certainly less than . We can reverse the role of the cells and and conclude that is the same for all . However, it is not necessarily the case that will equal . Lastly, we could also consider the case that and thus for the number is the number of vertices in adjacent to . We summarize our discussion with the following lemma.
Suppose that and let be the partition induced by the cycle decomposition of . Then for any pair of cells and it holds that for all .
Consider the graph shown in Figure 6.2, which is called the Petersen graph.
The Petersen graph
One can verify that is an automorphism of . Consider the induced partition , and let be the cells of . For any vertex , one can verify that , , and . For any vertex , one can verify that , , and . Lastly, for any vertex , one can verify that , , and . We summarize our results using a matrix which we denote by
As another example, is an automorphism of and the induced partition is . One can verify that for any it holds that , , , and . Also, for any it holds that , , , and . Similar verifications can be made for vertices in and , and in this case
In the next section, we will see how the degree conditions that exist for a partition induced by an automorphism exist for more general partitions not necessarily arising from an automorphism.
Equitable partitions of graphs
In this section, we will introduce the notion of an equitable partition of a graph and how we can use such partitions to define a notion of a quotient graph. We begin with two examples.
Consider again the Petersen graph in Figure 6.2. One can verify that is not the partition induced by any automorphism of . However, one can verify that for any it holds that , , and , that for any it holds that , , and ; and finally that for any it holds that , , and . Hence, even though is not induced by any automorphism, it still satisfies the degree conditions that are satisfied by a partition induced by an automorphism. In this case,
Consider the graph shown in Figure 6.3, which is called the Frucht graph. This graph has a trivial automorphism group, i.e., . However, it contains the type of partitions that satisfy the degree conditions of the partitions induced by an automorphism. For example, for , one can verify that for any it holds that , , and ; for any it holds that , , and ; and finally for any it holds that , , and . Hence, in this case
A graph with trivial automorphism group but with non-trivial equitable partitions
Let us now define the vertex partitions that will be of interest to us and that generalize the partitions induced by automorphisms of a graph.
Let be graph and let be a partition of . If for each pair of cells (not necessarily distinct) it holds that for every then we say that is an equitable partition of .
By Lemma 6.2.3, every partition induced by an automorphism is an equitable partition, however, Examples 6.7- 6.8 show that the converse does not hold, that is, not every equitable partition is induced by an automorphism. In fact, numerical evidence indicates that the proportion of equitable partitions induced by an automorphism tends to zero as .
There is an elegant linear-algebraic way to characterize an equitable partition that is very useful and which gives insight into the structure of the eigenvectors of a graph. We first need to introduce some notation and review more linear algebra.
Let be a partition of . For any cell let denote the vector whose entries are equal to on the integer indices in and zero elsewhere. For example, if and then , i.e., has a 1 in positions and zero elsewhere. The vector is called the characteristic vector (or indicator vector) of the cell . We define the characteristic matrix of as the matrix whose columns are the characteristic vectors of the cells of :
Notice that if has cells then is a matrix. Also, and more importantly, since is a partition of , the columns of are orthogonal, and thus .
If then is a partition of with cells , , , , and is a -partition. The characteristic matrix of is
Now, since has orthogonal columns then we have
Notice that the diagonals are just the cardinalities of the cells, i.e., .
For any matrix recall that the image of (frequently called the range of ) is
The image of is a subspace and more concretely, it is the subspace spanned by the columns of (frequently called the column space of ). To see this, if then for we have
and thus is a linear combination of the columns of , i.e., . We now introduce the following important notion.
Let be a matrix and let be a subspace. If for every it holds that then we say that is -invariant.
Hence, a subspace is -invariant if maps any element in back to an element in . There is no reason to expect that this will be true for an arbitrary subspace and so that is why such subspaces are singled out. Suppose that is -invariant and let be a basis for and consider the matrix . Now, since is -invariant then and therefore can be written as a linear combination of the basis vectors . Therefore, there is some vector such that
This holds for each and therefore if we set then
Suppose that has eigenvalue and let be a set of linearly independent eigenvectors of with eigenvalue ( is necessarily the geometric multiplicity of ). Consider the subspace . Let so that there exists constants such that . Then,
from which we see that , that is, . Hence, is -invariant.
Although not true for a general matrix , if is symmetric then every -invariant subspace has a basis of eigenvectors of . This fact is so important that we state it as a theorem.
If is a symmetric matrix and is -invariant then there exists a basis of consisting of eigenvectors of .
Suppose that is -dimensional and let be an orthonormal basis of . Since is -invariant then for each there exists constants such that
If we let be the matrix with entries and we put then
Now since is an orthonormal basis we have that and therefore multiplying by on the left we obtain that
Now since is symmetric it follows that is symmetric and thus has linearly independent eigenvectors, say , with associated eigenvalues . We claim that is an eigenvector of with eigenvalue . We compute
The eigenvectors are clearly contained in and they are linearly independent since are linearly independent and the matrix has rank .
We are now finally ready to characterize an equitable partition in terms of the invariant subspaces of the adjaceny matrix.
Let be a graph with adjacency matrix . Let be a -partition of with characteristic matrix . Then is an equitable partition of if and only if is -invariant. Equivalently, is equitable if and only if there exists a matrix such that . In this case, .
Let and let , where is the characteristic vector of cell . Let , in other words, is the diagonal matrix containing a 1 in diagonal entry if and zero other wise. Hence, . For any cell let
Then it is not hard to see that , and therefore we can write
For each we have
Therefore, if for all then for any . Hence, if is equitable then
where , and thus . Conversely, if then our computations above show that necessarily for all , and thus is equitable.
The matrix in Theorem 6.3.4 is actually that appeared in Examples 6.7- 6.8. To be precise, if is an equitable partition then is the same for all . We can therefore unambiguously define
for some . Then one can show that
and as discussed before in general , i.e., is not necessarily symmetric. The proof of Theorem 6.3.4 shows that
that is, .
Let us now discuss how the existence of an equitable partition imposes constraints on the eigenvectors of . Suppose that is a partition of and is its characteristic matrix. Then consists of vectors that have the same numerical value on the entries of each cell. For instance, if
then if then
Hence, a vector has the same values on the entries , the same values on entries , the same values on entries , and the same values on entries (this is trivial because is a singleton cell). Now, if is an equitable partition then is -invariant and therefore by Theorem 6.3.3 since is symmetric there is a basis of consisting of eigenvectors of . These eigenvectors therefore have entries that are equal on each of the cells of and there will be of these eigenvectors (linearly independent) if is a -partition.
We consider a specific example. Consider again the Frucht graph which we reproduce in Figure 6.4. The Frucht graph has three non-trivial equitable partitions, one of which is
Hence, since is a partition then there exists linearly independent eigenvectors of whose entries on the cells of are equal.
The Frucht graph
The eigenvectors (rounded to two decimal places) of are (as columns):
Observe that the eigenvectors in columns 2, 7, 12 all have entries that are constant on the cells of . What can we say about the eigenvectors of not contained in ? It turns out that because is symmetric then the orthogonal complement of the subspace is also -invariant, where the orthogonal complement of is the set of vectors orthogonal to vectors in . The orthogonal complement of a subspace is denoted by and thus
Since is -invariant, by Theorem 6.3.3, there is a basis of consisting of eigenvectors . One can show that for it holds that
It is not hard to see that if and only if the sum of the entries of on each cell sum to zero. Hence, the remaining eigenvectors of not contained in have the property that the sum of their entries on each cell sum to zero. We summarize with the following.
Suppose that is an equitable partition of . Then the eigenvectors of can be split into two groups, namely, those that are constant on the cells of (i.e., contained in and those that sum to zero on the cells of (i.e., contained in .
Another partition of the Frucht graph is
so that is a partition. One can verify that has linearly independent eigenvectors that are constant on the cells of , namely, the 5th eigenvector and the last. The remaining eigenvectors sum to zero on the cells of . For example, for the first eigenvector the sum of the entries on cell is (due to rounding errors) and the sum of the entires on cell is (again due to rounding errors).