Random Variables and Probability Distributions

Random Variable

• P(X = x) is the probability that the random variable X takes the value x

• When the range of a random variable can be counted → discrete random variable

• When the range of a random variable cannot be counted → continuous random variable

Probability Mass Function

• The exact probability that a discrete random variable is equal to a specific value

• Probability of any individual outcome cannot be less than zero

• The sum of all the probabilities of the outcomes in a sample space equals 1

Cumulative Distribution Function

• The output values of a CDF always falls between 0 and 1

• As x gets larger the CDF can increase or stay flat, but never decreases

The Expected Value/Theoretical Mean

• Computes the sum of whatever X is, weighted by the corresponding probability

Properties of Expectation

Theoretical Variance

• Measures the spread about the mean

• Var[X] = E(X²) - E(X)²

• Property : Var(aX + b) = a²E[(X - E(X))²] = a²Var(X)

Discrete Uniform Distribution

Requirements for binomial distribution

Number of distinct arrangements of x successes and (n-x) failures

Binomial Distribution

• Expected value of the binomial distribution: E(X) = np

• Variance of the binomial distribution: var[X] = np(1 - p)

Geometric distribution

• Memoryless: past failures don’t change the probability of success on future trials

• Number of trials can theoretically go on forever

• Expected value of geometric distribution : E(X) = 1/p

• Variance of geometric distribution : Var(X) = (1 - p)/p²

Poisson Distribution

• A Poisson distribution is a discrete probability distribution that predicts the exact number of times an independent event will occur within a fixed interval of time

• The expected value of the Poisson distribution is equal to the variance

• E(X) = Var(X) = λ

• When n is large and p is small enough to make np small, the Poisson distribution can approximate a binomial random variable such that the number of successes is approximately Poisson

Probability Density Function

• PDF: describes the likelihood for a continuous random variable to take on a given value

• P(X=x) = 0 for any real value x

• If b is sufficiently close to a: P(a < X <= b) = fX(a)(b - a )

CDF of a continuous random variable

• the PDF is the derivative of the CDF

Theoretical mean of a continuous random variable

Theoretical Variance of a continuous random variable

• Must be non-negative

Continuous Uniform Distribution

• E(X) : (a + b)/2

• Var(X) : (b - a)²/12

Exponential Distribution

• The exponential is the only continuous distribution which is memoryless

• E(X) = 1 / λ

• Var(X) = 1 / λ²

Change of Variables; making one random variable a function of another

Normal Distribution

• A standard normal random variable has mean E(Z) = 0 and variance Var(Z) = 1 → note E(Z²) = 1

• The distribution is symmetric about the mean

PDF of the Normal distribution

Normal distribution probability expressions

Standardising the normal distribution

The Chi-Square Distribution

• a continuous probability distribution that represents the sum of the squares of independent standard normal variables

Log-Normal distribution

Log-Normal PDF vs Normal PDF

Chebyshev’s Inequality

• Provides an upper bound on the probability of deviation of a random variable from its mean

Random variable with variance 0

Chebyshev’s inequality - variance