Guassian Distributions

Resources

• Products of Guassian Distributions - lecture notes with very detailed proofs of the product/convolution formulas for guassian distributions.
• Guassian Processes for Machine Learning - this book has some useful notes on guassian distributions in appendix A.

Definitions

•  : pdf for a guassian distribution with mean  and variance .
•  or  :  has a guassian distribution with mean  and variance .
• A guassian distribution  has the functional form:

(1)

Properties

Closures

The  linear transformation of a guassian is also a guassian (i.e. it is closed under linear (affine) transformations).

(2)

Products

Something that is immediately intuitive is that the product of two guassians of the one random variable is not guassian, but it is proportional to a guassian distribution, with the scalar itself a guassian like expression of the means.

(3)

Detailed proofs can be found in Products of Guassian Distributions and hints for such a proof in appendix A of Guassian Processes for Machine Learning.

Marginal & Conditional Distributions

Given a multivariate guassian, we can break it up in the following way. Let  be normally distributed such that

Note that here  is the non-symmetric cross-covariance matrix that would be computed from the appropriate quadrant of (?).

Then the marginal distributions are:

and the conditional distributions are:

(4)

The proof involves plugging the terms back into the exponential function and playing around (gets quite messy). For conditioning it uses (?). It is done in these lecture notes in all their glory, but a rather simpler to follow concept of the proof with zero means is in this blog.

MultiVariate Guassian Generators

There are several ways to generate random vectors for a multivariate guassian. This is also referred to as sampling the multivariate guassian. All three methods - rotational, conditional and triangular factorisation are based on univariate guassian samples.

To generate a random vector with mean  and covariance  for a multivariate guassian using triangular factorisation:

• Generate a  vector  where each element is independently sampled from .
• Decompose , for example with svd we have

(5)

where .

• Compute the random vector 

To see this, note that the covariance of  is by definition:

The mean is a trivial calculation. Thus the multivariate random vector .

Matlab Code

x = (0:.005:1);
n = length(x);
u = randn(n,1);
C = zeros(n,n);
for i = 1:n
  for j = 1:n
    C(i,j) = x(i)*x(j);
  end
end
[A, S, B]  = svd(C);
z = A*sqrt(S)*u;
figure(2); hold on; clf
plot(x,z,'.-')
axis([0,1,-2,2])