Eigenvalue and Eigenvector

Yun-Hao Lee, Cheng-Han Huang

Credit: Burak Kebapci from Pexels

1. Eigenvalue and Eigenvector

Definition 1.1. Let $A \in ℝ^{n \times n}$ , then a nonzero vector $v \in ℝ^{n}$ is called an eigenvector if there exist a $λ \in ℂ$ , which is called eigenvalue, such that

Av = λv .

In general, real-valued matrices may have complex eigenvalues, but for symmetric real-valued matrices, the eigenvalues are also real-valued. The following theorem will give more information about it.

Theorem 1.2 (Spectral Decomposition Theorem). Let $A \in ℝ^{n \times n}$ be symmetric. Then there exists an orthogonal matrix $U \in ℝ^{n \times n}$ and a diagonal matrix $D = diag [λ_{1}, λ_{2}, . . ., λ_{n}]$ such that

U^{T} AU = D

Theorem 1.3. Trace and determinant are invariant values under linear transformation. For any $A \in ℝ^{n \times n}$ , trace and determinants of $A$ can be expressed as

tr (A) = \sum_{i = 1}^{n} λ_{i} and \det (A) = \prod_{i = 1}^{n} λ_{i}

Definition 1.4. For a symmetric matrix $A \in ℝ^{n \times n}$ , the Rayleigh quotient is defined by

R_{A} (x) = \frac{x^{T} Ax}{x^{T} x} for any x \neq 0

Theorem 1.5. Let $A \in ℝ^{n \times n}$ , then

λ_{min} (A) \leq R_{A} (x) \leq λ_{max} (A) for any x \neq 0

Proof. By spectral decomposition theorem, there exist an orthogonal matrix $U \in ℝ^{n \times n}$ and a diagonal matrix $D = diag [λ_{1}, λ_{2}, . . ., λ_{n}]$ such that $U^{T} AU = D$ . Without loss of generality, may assume that $λ_{1} \geq λ_{2} \geq . . . \geq λ_{n}$ , then $λ_{1} = λ_{max}$ and $λ_{n} = λ_{min}$ . Now let $x = Uy$ , $y = {[y_{1}, y_{2}, . ., y_{n}]}^{T}$ and note that $U$ is nonsingular since $U$ is orthogonal, then

\frac{x^{T} Ax}{x^{T} x} = \frac{y^{T} U^{T} AUy}{y^{T} U^{T} Uy} = \frac{y^{T} Dy}{y^{T} y} = \frac{\sum_{i = 1}^{n} λ_{i} y_{i}^{2}}{\sum_{i = 1}^{n} y_{i}^{2}}

Hence

λ_{n} = \frac{λ_{n} \sum_{i = 1}^{n} y_{i}^{2}}{\sum_{i = 1}^{n} y_{i}^{2}} = \min_{x \neq 0} \frac{x^{T} Ax}{x^{T} x} \leq \frac{x^{T} Ax}{x^{T} x} \leq \max_{x \neq 0} \frac{x^{T} Ax}{x^{T} x} = \frac{λ_{1} \sum_{i = 1}^{n} y_{i}^{2}}{\sum_{i = 1}^{n} y_{i}^{2}} = λ_{1}

Lemma 1.6. Let $A \in ℝ^{n \times n}$ be symmetric. Then

$\min_{x \neq 0} R_{A} (x) = λ_{min} (A)$ and eigenvectors of $A$ corresponding to the minimal eigenvalue minimize $R_{A} (x)$ .
$\max_{x \neq 0} R_{A} (x) = λ_{max} (A)$ and eigenvectors of $A$ corresponding to the maximal eigenvalue minimize $R_{A} (x)$ .

Reference:

Beck, A. (2014). Introduction to nonlinear optimization: Theory, algorithms, and applications with MATLAB. Society for Industrial and Applied Mathematics.