The spectral theorem

Statement

If $A = A^T$ (symmetric), then its eigenvalues are real and its eigenvectors can be chosen orthonormal:

$$ A = Q\Lambda Q^T, \quad Q^TQ = I \ (\text{so } Q^{-1} = Q^T). $$

Shapes

$$ \underbrace{A}_{n\times n}= \underbrace{\begin{pmatrix}\vert&&\vert\\q_1&\cdots&q_n\\\vert&&\vert\end{pmatrix}}_{Q,\ n\times n} \underbrace{\begin{pmatrix}\lambda_1&&\\&\ddots&\\&&\lambda_n\end{pmatrix}}_{\Lambda,\ n\times n} \underbrace{\begin{pmatrix}-\,q_1^T-\\\vdots\\-\,q_n^T-\end{pmatrix}}_{Q^T,\ n\times n}. $$

The columns of $Q$ are the eigenvectors; the rows of $Q^T$ are the same vectors, laid flat. Orthonormality is the statement that every row of $Q^T$ dotted with every column of $Q$ gives $1$ on the diagonal and $0$ elsewhere, i.e. $Q^TQ=I$.

Intuition: symmetric matrices are a rotation into perpendicular axes, a pure scaling along them, and a rotation back; nothing else can happen.

Eigenvalues are real

For $Av = \lambda v$ (allowing complex $\lambda, v$ for now), conjugate-transpose both sides and use $A^H = A^T = A$:

$$ v^H A v = \lambda\, v^Hv, \qquad v^H A v = (Av)^Hv = (\lambda v)^Hv = \bar\lambda\, v^Hv. $$

Since $v^Hv > 0$, this forces $\lambda = \bar\lambda$, i.e. $\lambda \in \mathbb{R}$.

Intuition: symmetry leaves no room for the complex (rotational) eigenvalues a generic matrix can have.

Eigenvectors are orthogonal

For distinct eigenvalues $\lambda_1 \neq \lambda_2$ with eigenvectors $v_1, v_2$:

$$ \lambda_1 v_1^Tv_2 = (Av_1)^Tv_2 = v_1^TA^Tv_2 = v_1^TAv_2 = v_1^T(\lambda_2 v_2) = \lambda_2 v_1^Tv_2. $$

$(\lambda_1 - \lambda_2)v_1^Tv_2 = 0$ and $\lambda_1 \neq \lambda_2$, so $v_1^Tv_2 = 0$.

Intuition: symmetry ($A^T=A$) is exactly what lets the eigenvalue "slide across" the dot product like this.

Example

Reusing $A = \begin{pmatrix}2&1\\1&2\end{pmatrix}$, eigenvalues $3,1$, eigenvectors $(1,1),(1,-1)$ (from Diagonalization):

$$ q_1 = \tfrac{1}{\sqrt2}\begin{pmatrix}1\\1\end{pmatrix}, \quad q_2 = \tfrac{1}{\sqrt2}\begin{pmatrix}1\\-1\end{pmatrix}, \quad q_1 \cdot q_2 = \tfrac12(1-1) = 0. $$

$$ Q = \tfrac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}, \ \Lambda = \begin{pmatrix}3&0\\0&1\end{pmatrix}, \quad Q\Lambda Q^T = \begin{pmatrix}2&1\\1&2\end{pmatrix} = A. \ \checkmark $$

What if an eigenvalue repeats?

The argument above only orthogonalizes eigenvectors from different eigenvalues: two eigenvectors sharing one eigenvalue aren't automatically orthogonal. But a repeated eigenvalue's eigenspace is just a subspace, so pick any orthonormal basis for it directly. Combined with the other eigenvalues' (automatically orthogonal) eigenvectors, that's a full orthonormal eigenbasis, repeats or not.

So symmetric $\Rightarrow$ always diagonalizable, $A = Q\Lambda Q^T$, never defective. That's not the same as full rank: eigenvalues can still be $0$.

Intuition: a repeated eigenvalue means more freedom in choosing eigenvectors, never less.

Example

$A = \begin{pmatrix}1&1&1\\1&1&1\\1&1&1\end{pmatrix}$ (symmetric): eigenvalues $3,0,0$; $0$ repeats with $\operatorname{AM}=2$.

$$ N(A) = \{x : x_1+x_2+x_3=0\}, \ \dim = 2 = \operatorname{AM}(0) \;\Rightarrow\; \text{not defective.} $$

Still, $\operatorname{rank}(A) = 1$: diagonalizable and low-rank are unrelated.

Pick any orthonormal basis of that plane, e.g. $q_2 = \tfrac{1}{\sqrt2}(1,-1,0)$, $q_3 = \tfrac{1}{\sqrt6}(1,1,-2)$, and $q_1 = \tfrac{1}{\sqrt3}(1,1,1)$ for $\lambda=3$: all three pairwise orthogonal.

$A$ as a sum of simple pieces

Multiplying out $Q\Lambda Q^T$ column by column gives a rank-1 sum:

$$ A = \sum_i \lambda_i\, q_iq_i^T. $$

Intuition: $A$ is a weighted sum of simple directions $q_iq_i^T$, one per eigenvector, weighted by how much $A$ stretches that direction.

Shapes

Each piece is a column times a row, an outer product, so it is a full $n\times n$ matrix of rank 1:

$$ \underbrace{q_i}_{n\times1}\underbrace{q_i^T}_{1\times n}=\underbrace{\begin{pmatrix}q_{i1}q_{i1}&\cdots&q_{i1}q_{in}\\\vdots&\ddots&\vdots\\q_{in}q_{i1}&\cdots&q_{in}q_{in}\end{pmatrix}}_{n\times n,\ \text{rank }1}, \qquad A=\lambda_1\underbrace{q_1q_1^T}_{n\times n}+\cdots+\lambda_n\underbrace{q_nq_n^T}_{n\times n}. $$

Each $q_iq_i^T$ is also the projection onto the line through $q_i$ (the line formula from Rank, subspaces, and projections, with $q_i^Tq_i=1$): $A$ projects onto each axis, scales by $\lambda_i$, and adds the results.

Example

$$ q_1q_1^T = \tfrac12\begin{pmatrix}1&1\\1&1\end{pmatrix}, \quad q_2q_2^T = \tfrac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix}. $$

$$ 3\,q_1q_1^T + 1\,q_2q_2^T = \begin{pmatrix}1.5&1.5\\1.5&1.5\end{pmatrix} + \begin{pmatrix}0.5&-0.5\\-0.5&0.5\end{pmatrix} = \begin{pmatrix}2&1\\1&2\end{pmatrix} = A. \ \checkmark $$