Distributions of Random Variables

Definition and Properties of Random Variables

• A random variable maps a random process into a numerical outcome.

• Notation standards designate capital letters such as $X$ or $Y$ to represent the random variable itself, while lower case letters such as $x$ or $y$ denote specific realized outcomes.

• The concept of randomness is the core defining element of these variables.

• The law of large numbers serves as the bridging principle that allows concepts calculated from sample data to be transferred to the population level.

• In a population study regarding $100$ enrolled students and book purchases:

  • Data reveals $20\%$ buy no books, $55\%$ buy the textbook only, and $25\%$ buy both the textbook and its study guide.

  • At the population level, the bookstore expects around $20$ students to buy $0$ books, about $55$ students to buy $1$ book, and approximately $25$ students to buy $2$ books.

  • The total expected book sales for such a class is calculated as: $0 \times 20 + 1 \times 55 + 2 \times 25 = 105$ books.

• It is critical to recognize sampling variability; selling slightly more or less than the exact expected value (e.g., $104$ or $106$ books) is natural and statistically normal, comparable to how flipping a coin $100$ times rarely results in exactly $50$ heads but usually stays close to that figure.

Revenue Variables and Numerical Coding

• Analysis of bookstore revenue ($Y$) for a class of $100$ students:

  • The textbook costs $\text{\$137}$ and the study guide costs $\text{\$33}$.

  • Expected revenue from textbook-only buyers: \text{\137} \times 55 = \text{\7535}.

  • Expected revenue from dual-book buyers: (\text{\137} + \text{\33}) \times 25 = \text{\170} \times 25 = \text{\4250}.

  • Total expected class revenue: \text{\7535} + \text{\4250} = \text{\$11785}.

• The random variable $X$ represents revenue from a single student with outcomes $m_1 = \text{\$0}$, $m_2 = \text{\$137}$, and $m_3 = \text{\$170}$ occurring with probabilities $0.20$, $0.55$, and $0.25$ respectively.

• Caveat regarding Numerical Coding:

  • Non-numeric results, such as academic disciplines, are often coded numerically: Sociology ($1$), Psychology ($2$), and Pedagogy ($3$).

  • Coded numbers must not be treated as regular numbers; operations such as calculating the average of codes are invalid because the numerical value does not represent a magnitude (e.g., $3$ is not "three times as much" as $1$).

• Variable Classification:

  • Discrete Random Variables: The set of possible outcomes is a discrete set (finite or countably infinite).

  • Continuous Random Variables: The set of outcomes consists of intervals. While transaction outcomes may appear discrete, revenue as a concept is continuous in nature.

Discrete Probability Distributions

• A probability distribution for a discrete variable lists all possible disjoint outcomes and their associated probabilities.

• It serves as the population-level equivalent of a sample frequency table.

• Three Mandatory Rules for Probability Distributions:

  1. The outcomes listed must be disjoint.

  2. Each probability must be between $0$ and $1$.

  3. The total sum of all probabilities must be exactly $1$.

• Example: Sum of Two Dice:

  • Disjoint outcomes range from $2$ to $12$.

  • Probabilities follow the rule of Laplace: $P(2) = \frac{1}{36}$, $P(3) = \frac{2}{36}$, $P(4) = \frac{3}{36}$, $P(5) = \frac{4}{36}$, $P(6) = \frac{5}{36}$, $P(7) = \frac{6}{36}$, $P(8) = \frac{5}{36}$, $P(9) = \frac{4}{36}$, $P(10) = \frac{3}{36}$, $P(11) = \frac{2}{36}$, $P(12) = \frac{1}{36}$ .

• As observations increase, sample proportions converge to these theoretical probabilities due to the law of large numbers.

• Visual summaries are provided by bar plots where heights represent the probabilities of specific outcomes.

Cumulative Distribution and Quantile Functions

• Cumulative Distribution Function ($F(x)$):

  • Defined for a random variable $X$ as $F(x) = P(X \le x)$ for all real numbers $x$.

  • It is the population counterpart to the empirical cumulative distribution function ($\hat{F}(x)$).

• Quantile Function ($Q(p)$):

  • The inverse of the cumulative distribution function.

  • Defined as $Q(p) = x$, where $x$ is the smallest outcome such that $F(x) \ge p$.

  • This function is used to define medians, quartiles, and other specific distribution markers.

• Calculations for the sum of two dice ($X$):

  • To find $F(3)$, calculate $P(X \le 3) = P(X = 2) + P(X = 3) = \frac{1}{36} + \frac{2}{36} = \frac{3}{36}$.

  • To find the median $Q(0.5)$, identify the smallest $x$ where $F(x) \ge 0.5$. Given $P(X \le 6) = \frac{15}{36}$ ($0.417$) and $P(X \le 7) = \frac{21}{36}$ ($0.583$), the result is $Q(0.5) = 7$.

Expected Value and Variability

• Expected Value ($E(X)$ or $\mu$):

  • This is the weighted mean of all possible outcomes.

  • Formula: $E(X) = \sum_{i=1}^{k} m_i P(X = m_i)$.

  • In physics, the expectation represents the center of gravity; the distribution balances on a triangle placed at $\mu$.

  • For the single student revenue model: $E(X) = 0(0.20) + 137(0.55) + 170(0.25) = \text{\$117.85}$.

• Population Variance ($Var(X)$ or $\sigma^2$):

  • Describes the volatility or variability of a random variable.

  • Defined as the weighted mean of squared deviations from the population mean.

  • Formula: $\sigma^2 = \sum_{j=1}^{k} (m_j - \mu)^2 P(X = m_j)$.

  • General formula: $\sigma^2 = E[(X - E(X))^2] = E(X^2) - (E(X))^2$.

• Standard Deviation ($\sigma$):

  • The square root of the variance ($\sigma = \sqrt{Var(X)}$).

  • For the student revenue model: $Var(X) = (0 - 117.85)^2(0.20) + (137 - 117.85)^2(0.55) + (170 - 117.85)^2(0.25) = 3659.3$.

  • Standard Deviation: $\sqrt{3659.3} = \text{\$60.49}$.

The Bernoulli Distribution

• A Bernoulli random variable studies a single trial with only two possible outcomes: "success" and "failure".

• Success is numerically coded as $1$ and failure as $0$.

• The success probability is denoted by $p$, and the failure probability by $q = 1 - p$.

• Descriptive stats for Bernoulli variables:

  • Mean: $\mu = p$.

  • Standard Deviation: $\sigma = \sqrt{p(1 - p)}$.

• Term Utility: The label "success" does not carry a positive connotation but is a neutral mathematical label (e.g., success could be labeled as refusing to give a shock).

The Binomial Distribution

• The Binomial distribution counts the number of successes ($k$) in a fixed number ($n$) of independent and identically distributed (iid) Bernoulli trials.

• Mathematical Framework:

  • The probability of exactly $k$ successes in $n$ trials with success probability $p$ is: $P(X = k) = \binom{n}{k} p^k (1 - p)^{n-k}$.

  • Combination formula: $\binom{n}{k} = \frac{n!}{k!(n - k)!}$.

• Descriptive Figures for Binomial variables:

  • Mean: $\mu = np$.

  • Variance: $\sigma^2 = np(1 - p)$.

  • Standard Deviation: $\sigma = \sqrt{np(1 - p)}$.

• Four Required Conditions for the Binomial Model:

  1. The trials are independent.

  2. The number of trials $n$ is fixed.

  3. Each outcome must be classified as a success or failure.

  4. The probability of success $p$ remains constant across every trial.

Computational Implementation and Tips

• Binomial probability calculations often involve factorial cancellation for efficiency.

• Excel Functionality:

  • Individual probability $P(X = k)$: \text{binom.dist}(k; n; p; \text{false}).

  • Cumulative probability $P(X \le k)$: \text{binom.dist}(k; n; p; \text{true}).

• Rule of Thumb: Approximately $95\%$ of observations are expected to fall within $2$ standard deviations of the mean.

Questions & Discussion

• US Household Income Distributions Check

  • Three distributions were proposed for US household income ranges ($0-25$, $25-50$, $50-100$, $100+$ in thousands).

  • (a) $0.18, 0.39, 0.33, 0.16$: Sum is $1.06$ (Incorrect, must be $1.00$).

  • (b) $0.38, -0.27, 0.52, 0.37$: Contains a negative probability (Invalid).

  • (c) $0.28, 0.27, 0.29, 0.16$: Correct distribution (Sum is $1.00$, all values valid).

• The Milgram Experiment Analysis

  • The study found that about $65\%$ of individuals administered the worst shock, meaning the probability of "success" (refusal) is $p = 0.35$.

  • For a sample of $4$ people, the probability exactly one refuses is: $4 \times (0.35)^1 \times (0.65)^3 = 0.38$.

  • For a randomized sample of $40$ students, the expected number of refusers is $14$ ($40 \times 0.35$) with a standard deviation of $3.02$ ($\sqrt{40 \times 0.35 \times 0.65}$).

• Smoker and Lung Condition Scenarios

  • A random smoker has a $0.3$ probability of a severe lung condition.

  • Check for $4$ friends: Binomial model applies if friends are independent and do not share habits.

  • Probability none develop condition: $\binom{4}{0} (0.3)^0 (0.7)^4 = 0.2401$.

  • Probability exactly one develops condition: $\binom{4}{1} (0.3)^1 (0.7)^3 = 0.4116$.

  • Probability at most one develops condition: $0.2401 + 0.4116 = 0.6517$.

  • Probability at least two develop condition: $1 - 0.6517 = 0.3483$.

• Chemistry Book Revenue Practice

  • Costs: textbook $\text{\$159}$, supplement $\text{\$41}$. History: $25\%$ buy textbook, $60\%$ buy both.

  • (a) Percent buying nothing: $100\% - 25\% - 60\% = 15\%$.

  • (b) Expected Revenue ($Y$) per student: $0(0.15) + 159(0.25) + 200(0.60) = \text{\$159.75}$.

  • (c) Variance and Volatility: $Var(Y) = 4800$; $\sigma = \sqrt{4800} = \text{\$69.28}$.

• Combinatorial Logic

  • Question: Why is $\binom{n}{0} = 1$ and $\binom{n}{n} = 1$?

  • Answer: There is only one way to have zero successes (all failures) and only one way to have all successes in $n$ trials.

  • Question: Ways to arrange $1$ success or $n - 1$ successes?

  • Answer: Both are equal to $n$, as there are exactly $n$ unique slots to place a single disparate outcome (one success among failures or one failure among successes).