Definitions

•  : random variables or a random vector 
•  : constant vectors.
•  : probability distribution function for a random variable .

Conditional/Marginal Distributions

The conditional distribution is defined as:

(1)

When referring to more than one random variable, then the individual probabilities such as  and  are referred to as the marginal distributions for  and  respectively. These represent the probabilities not contingent on each other.

(2)

We say the variable  has been marginalized out.

Expected Value

The expected value of a random variable  or function of a random variable  is given by:

Some properties:

Covariance

The covariance between two random variables is defined as:

(3)

 Some properties:

(4)

Variance is a special case when the two variables are identical.

(5)

Independent Variables

Independent variables have the following characteristics:

(6)

The sum of the variance of two independent variables follows from the linear combination rule in (4) and  :

(7)

Jointly Distributed Random Variables

If  is a vector of random variables, then the variance takes the form:

(8)