# 7.8. Multivariate Gaussian Distribution#

Definition 7.32

A random vector $$X = [X_1, \dots, X_n]^T$$ is called Gaussian random vector if

$\langle t , X \rangle = X^T t = \sum_{i = 1}^n t_i X_i = t_1 X_1 + \dots + t_n X_n$

follows a normal distribution for all $$t = [t_1, \dots, t_n ]^T \in \RR^n$$. The components $$X_1, \dots, X_n$$ are called jointly Gaussian. It is denoted by $$X \sim \NNN_n (\mu, \Sigma)$$ where $$\mu$$ is its mean vector and $$\Sigma$$ is its covariance matrix.

Let $$X \sim \NNN_n (\mu, \Sigma)$$ be a Gaussian random vector. The subscript $$n$$ denotes that it takes values over the space $$\RR^n$$. We assume that $$\Sigma$$ is invertible. Its PDF is given by

$f_X (x) = \frac{1}{(2\pi)^{n / 2} \det (\Sigma)^{1/2} } \exp \left \{- \frac{1}{2} (x - \mu)^T \Sigma^{-1} (x - \mu) \right\}.$

Moments:

$\EE [X] = \mu \in \RR^n.$
$\EE[XX^T] = \Sigma + \mu \mu^T.$
$\Cov[X] = \EE[XX^T] - \EE[X]\EE[X]^T = \Sigma.$

Let $$Y = A X + b$$ where $$A \in \RR^{n \times n}$$ is an invertible matrix and $$b \in \RR^n$$. Then

$Y \sim \NNN_n (A \mu + b , A \Sigma A^T).$

$$Y$$ is also a Gaussian random vector with the mean vector being $$A \mu + b$$ and the covariance matrix being $$A \Sigma A^T$$. This essentially is a change in basis in $$\RR^n$$.

The CF is given by

$\Psi_X(j \omega) \exp \left ( j \omega^T x - \frac{1}{2} \omega^T \Sigma \omega \right ), \quad \omega \in \RR^n.$

## 7.8.1. Whitening#

Usually we are interested in making the components of $$X$$ uncorrelated. This process is known as whitening. We are looking for a linear transformation $$Y = A X + b$$ such that the components of $$Y$$ are uncorrelated. i.e. we start with

$X \sim \NNN_n (\mu, \Sigma)$

and transform $$Y = A X + b$$ such that

$Y \sim \NNN_n (0, I_n)$

where $$I_n$$ is the $$n$$-dimensional identity matrix.

### 7.8.1.1. Whitening by Eigen Value Decomposition#

Let

$\Sigma = E \Lambda E^T$

be the eigen value decomposition of $$\Sigma$$ with $$\Lambda$$ being a diagonal matrix and $$E$$ being an orthonormal basis.

Let

$\Lambda^{\frac{1}{2}} = \Diag (\lambda_1^{\frac{1}{2}}, \dots, \lambda_n^{\frac{1}{2}}).$

Choose $$B = E \Lambda^{\frac{1}{2}}$$ and $$A = B^{-1} = \Lambda^{-\frac{1}{2}} E^T$$.
Then

$\Cov (B^{-1} X) = \Cov (A X) = \Lambda^{-\frac{1}{2}} E^T \Sigma E \Lambda^{-\frac{1}{2}} = I.$
$\EE [B^{-1} X] = B^{-1} \mu \iff \EE [B^{-1} (X - \mu)] = 0.$

Thus the random vector $$Y = [B^{-1} (X - \mu)$$ is a whitened vector of uncorrelated components.

### 7.8.1.2. Causal Whitening#

We want that the transformation be causal, i.e. $$A$$ should be a lower triangular matrix. We start with

$\Sigma = L D L^T = (L D^{\frac{1}{2}} ) (D^{\frac{1}{2}} L^T).$

Choose $$B = L D^{\frac{1}{2}}$$ and $$A = B^{-1} = D^{-\frac{1}{2}} L^{-1}$$. Clearly, $$A$$ is lower triangular.

The transformation is $$Y = [B^{-1} (X - \mu)$$.