Mathematical Statistics - 7th Edition - Wackerly - Ch 3

Chapter 3: Discrete Random Variables and Their Probability Distributions

3.1 Basic Definition

• A random variable is a real-valued function defined over a sample space, quantifying events of interest in an experiment.

• Example: In an opinion poll, the random variable Y could represent the number of voters favoring a candidate.

• The possible observed values of Y range from 0 to the sample size, and Y is considered discrete if it can take a finite or countably infinite number of distinct values.

3.2 The Probability Distribution for a Discrete Random Variable

• A discrete random variable Y is characterized by a probability distribution, which assigns probabilities to each possible value.

• Probability distribution must satisfy:

  • 0 ≤ p(y) ≤ 1 for all values of y.

  • The sum of probabilities of all distinct values is 1.

• Notationally, use an uppercase letter (Y) for the random variable and lowercase letter (y) for the specific value observed.

• Example: For a die roll, Y could take values 1, 2, 3, 4, 5, or 6.

3.3 The Expected Value of a Random Variable

• The expected value (mean) of a discrete random variable Y, E(Y), is calculated as:

  • [ E(Y) = \sum_{y} y p(y) ]

• The variance of Y is defined as:

  • [ V(Y) = E[(Y - \mu)^{2}] ]

  • Where \mu = E(Y) is the mean.

3.4 The Binomial Probability Distribution

• A binomial experiment meets the following criteria:

  1. A fixed number of trials, n.

  2. Each trial results in one of two outcomes (success or failure).

  3. The probability of success (p) remains constant across trials.

  4. Trials are independent.

• The probability of obtaining y successes in n trials is given by:

  • [ P(Y = y) = (\binom{n}{y}) p^{y} (1-p)^{n-y} ]

3.5 The Geometric Probability Distribution

• The geometric distribution models the number of trials until the first success occurs.

• Probability function is:

  • [ P(Y = y) = (1-p)^{y-1} p ]

• This implies waiting until a success over repeated trials.

3.6 The Negative Binomial Probability Distribution

• Models the number of trials until the r-th success.

• Probability function:

  • [ P(Y = y) = \binom{y-1}{r-1} p^{r} (1-p)^{y-r} ]

  • Valid for r ≤ y.

3.7 The Hypergeometric Probability Distribution

• Used for sampling without replacement from a finite population.

• Probability function:

  • [ P(Y = y) = \frac{\binom{r}{y} \binom{N-r}{n-y}}{\binom{N}{n}} ]

  • Ensures trials depend on the previous ones as draws are made.

3.8 The Poisson Probability Distribution

• The Poisson distribution is used for modeling the number of events occurring in a fixed interval of time or space.

• Probability function:

  • [ P(Y = y) = \frac{\lambda^{y} e^{-\lambda}}{y!} ]

  • Where \lambda is the average rate of occurrences.

3.9 Moments and Moment-Generating Functions

• Moments provide measures such as mean and variance, which describe the distribution's shape and spread.

• The moment-generating function (MGF) allows derivation of moments:

  • [ M(t) = E(e^{tY}) ]

• MGFs can be used to identify the distribution of random variables.

3.10 Probability-Generating Functions

• Useful for integer-valued random variables, providing a method for deriving their distributions.

• The probability-generating function for a random variable Y:

  • [ P(t) = E(t^{Y}) = \sum_{i=0}^{\infty} P(Y = i) t^{i} ]

3.11 Tchebysheff's Theorem

• Provides a lower bound for the probability that a random variable Y falls within k standard deviations from the mean.

• For any k > 0:

  • [ P(|Y - \mu| < k\sigma) \geq 1 - \frac{1}{k^{2}} ]

3.12 Summary

• Reviewed discrete random variables, their probability distributions, expected values, and variances.

• Discussed moment and probability-generating functions useful for analyzing discrete distributions and deriving relationships between them.

Chapter 3: Discrete Random Variables and Their Probability Distributions

3.1 Basic Definition

A random variable is a function that assigns a real number to each outcome in a sample space, allowing us to quantify outcomes of interest in experiments. The character of a random variable can be either discrete or continuous, but this chapter focuses on discrete random variables, which can assume either a finite or a countably infinite number of values.

Example:

In an opinion poll, let the random variable Y represent the number of voters favoring a specific candidate. The possible observed values of Y would range from 0 (none) to the sample size (all voters favoring that candidate).

3.2 The Probability Distribution for a Discrete Random Variable

A discrete random variable Y is fundamentally characterized by its probability distribution, which assigns a probability to each possible outcome of Y.

• The probability distribution must satisfy two conditions:

  1. Probability Range: 0 ≤ P(Y = y) ≤ 1 for all values y.

  2. Total Probability Sum: The sum of the probabilities of all distinct values must equal 1, i.e., [ \sum p(y) = 1 ].

• Notation: An uppercase letter (Y) represents the random variable, while a lowercase letter (y) indicates a specific observed value.

Example:

For a standard die roll, Y could take values 1, 2, 3, 4, 5, or 6, each with equal probability of 1/6.

3.3 The Expected Value of a Random Variable

The expected value (or mean) of a discrete random variable Y, denoted as E(Y), is a key measure of the center of the probability distribution, calculated as follows: [ E(Y) = \sum_{y} y \cdot P(Y=y) ]The variance of Y, represented as V(Y), quantifies the dispersion of the random variable around its mean, calculated with: [ V(Y) = E[(Y - \mu)^{2}] ] Where ( \mu = E(Y) ) is the expected value.

3.4 The Binomial Probability Distribution

A binomial experiment is defined by the following criteria:

1. A fixed number of trials, denoted as n.

2. Each trial results in one of two outcomes: success or failure.

3. The probability of success, p, remains constant across all trials.

4. The trials are independent, meaning the outcome of one trial does not affect another.

Probability Formula:

The probability of obtaining exactly y successes in n trials is given by: [ P(Y = y) = \binom{n}{y} p^{y} (1-p)^{n-y} ] Where ( \binom{n}{y} ) is the binomial coefficient, representing the number of ways to choose y successes from n trials.

3.5 The Geometric Probability Distribution

The geometric distribution models the number of trials needed to achieve the first success in a series of Bernoulli trials (trials with two outcomes).

Probability Function:

[ P(Y = y) = (1-p)^{y-1} p ]This indicates that the probability of first success occurs on the y-th trial where y is a positive integer.

3.6 The Negative Binomial Probability Distribution

This distribution extends the geometric distribution, modeling the number of trials needed to achieve the r-th success.

Probability Function:

[ P(Y = y) = \binom{y-1}{r-1} p^{r} (1-p)^{y-r} ]This formula is valid only for y ≥ r, ensuring r successes can occur before y total trials are reached.

3.7 The Hypergeometric Probability Distribution

The hypergeometric distribution applies when sampling without replacement from a finite population. This distribution is crucial in scenarios where each observation affects subsequent observations.

Probability Function:

[ P(Y = y) = \frac{\binom{r}{y} \binom{N-r}{n-y}}{\binom{N}{n}} ]Where N is the total population size, r is the total number of successes in the population, and n is the number of draws.

3.8 The Poisson Probability Distribution

The Poisson distribution is utilized for modeling the number of events occurring within a fixed interval of time or space, particularly useful for rare events.

Probability Function:

[ P(Y = y) = \frac{\lambda^{y} e^{-\lambda}}{y!} ]Where ( \lambda ) is the average rate of occurrences within the specified interval.

3.9 Moments and Moment-Generating Functions

Moments are statistics that provide various measures, such as mean and variance, that describe a distribution’s shape and spread. The moment-generating function (MGF) is a powerful tool for finding moments of a probability distribution: [ M(t) = E(e^{tY}) ]The MGF can also help identify the distribution type by its generating properties.

3.10 Probability-Generating Functions

Probability-generating functions (PGF) are particularly useful for integer-valued random variables and can facilitate the derivation of their distributions. The PGF for a random variable Y is defined as: [ P(t) = E(t^{Y}) = \sum_{i=0}^{\infty} P(Y = i) t^{i} ]

3.11 Tchebysheff's Theorem

Tchebysheff's Theorem provides a useful lower bound for the probability that a random variable Y will be found within k standard deviations from the mean. For any k > 0: [ P(|Y - \mu| < k\sigma) \geq 1 - \frac{1}{k^{2}} ]This theorem is essential for understanding the spread of probability distributions and the likelihood of observing values within a specific range.

3.12 Summary

This chapter has reviewed critical concepts surrounding discrete random variables, their associated probability distributions, the calculation of expected values, variances, and the significance of moment-generating and probability-generating functions. These concepts are pivotal for analyzing discrete distributions and understanding their relationships.