Probability Distibutions

Contents

Notation

We use a notation that applies equally to discrete and continuous distributions

Properties of Distributions

Characteristic Function

The characteristic function of a distribution is the conjugate of the fourier transform of its pdf: g(t) = Integral ( exp(jtx) dF, x=-inf...+inf). For discrete distributions it is g(t) = Sum ( p(xk) exp(jtxk)) over all values xk.

The usefulness of characteristic functions arises because the characteristic function of the sum of two independent random variables equals the product of the two characteristic functions concerned.

Moments

The moments of a distribution (about the origin) are given by m0i = Integral (xi dF, x=-inf...+inf) = (-j)i (dig/dti)|t=0= (di(-jg)/dti)|t=0.

The moments about the mean are given by mi = Integral((x-m)i dP, x=-inf...+inf).

The moments about an arbitrary point, x, may be obtained by formally exanding mxi = (m* + (m-x))i and then replacing m*i by mi.

Cumulants

The r'th cumulant of a distribution, kr, is the coefficient of (jt)r/r! in the power series expansion of loge of the characteristic function, i.e. of ln(Integral ( exp(jtx) dF, x=-inf...+inf). It may be obtained from the characteristic function as kr=(-j)i (diln(g)/dti)|t=0=(diln(-jg)/dti)|t=0

The cumulants are related to the moments as follows:

The formula for mr contains all terms of the form kaA * kbB * kcC * ... where aA+bB+cC + ... = r and A,B,C,... are all >= 1 and 2 <= a,b,c,... <= r. The coefficient for a general term is r!/(A! * a!A * B! * b!B * C! * c!C * ...).

The inverse relationships are

The formula for kr contains all terms of the form maA * mbB * mcC * ... where aA+bB+cC + ... = r and A,B,C,... are all >= 1 and 1 < a,b,c,... <= r. The coefficient for a general term is (-1)k-1(k-1)! r!/(A! * a!A * B! * b!B * C! * c!C * ...) where k=A+B+C+.... The two sets of coefficients are the same when k=1 and the same but for their sign when k=2.

We can also define the normalised cumulants gr = kr/sr:

Bounds

Chebyshev Inequality: Pr(|X-m|>=d) = 1 - F(m+d) + F(m-d) <= (s/d)2 gives a rather weak bound on the sum of the two tail probabilities.


Transforming Distributions

Linear Transformation: Suppose Y=aX+b where X has a pdf f(x)=dF(x)/dx with mean m and standard deviation s and a characteristic function g(t), then:


Probability Identities

The identities below are expressed in terms of discrete distributions. The also work for continuous distributions with sums replaced by integrals. Sx() denotes the sum over all values of x.


Discrete Distributions


In these distributions, the variable r takes integer values.

Binomial Distribution

Poisson Distribution


Continuous Distributions


Beta Distribution

Cauchy Distribution

Chi-Squared Distribution

This is the distribution of the sum of the squares of n independent standard gaussian random variables. If Y=X, then Y has a gamma distribution with parameter p=n.

Non-central Chi-squared Distribution

This is the distribution of the sum of the squares of n independent gaussian random variables with unit variances non-zero means. The non-centrality parameter d is the sum of the squares of the means [some people call this d2].

Exponential Distribution

Fisher's F Distribution

Fisher's z Distribution

Gamma Distribution (Pearson Type III Distribution)

Gaussian or Normal Distribution

Laplace Distribution

Lognormal Distribution

This is a distribution such that ln(x) has a gaussian distribution with mean a and standard deviation b.

Nakagami Distribution

Rayleigh Distribution

This distribution arises in communications theory as the magnitude of a component of the fourier transform of white noise.

Rectangular Distribution

Rician Distribution

This distribution arises in communications theory as the magnitude of the fourier transform of a cosine wave (of amplitude A) corrupted by additive white noise.

Students t Distribution


Multivariate Gaussian


If x is an n-dimensional multivariate gaussian with mean m and covariance matrix S then