Definitions

: random variables or a random vector
: constant vectors.
: probability distribution function for a random variable .

Conditional/Marginal Distributions

The conditional distribution is defined as:

(1)

When referring to more than one random variable, then the individual probabilities such as and are referred to as the marginal distributions for and respectively. These represent the probabilities not contingent on each other.

(2)

We say the variable has been marginalized out.

Expected Value

The expected value of a random variable or function of a random variable is given by:

Some properties:

Covariance

The covariance between two random variables is defined as:

(3)

Some properties:

(4)

Variance is a special case when the two variables are identical.

(5)

Independent Variables

Independent variables have the following characteristics:

(6)

The sum of the variance of two independent variables follows from the linear combination rule in (4) and :

(7)

Jointly Distributed Random Variables

If is a vector of random variables, then the variance takes the form:

(8)