Distributions of Random Variables: Continuous Distributions, Linear Transformations, and the Normal Model

Continuous Distribution

  • For discrete random variables, the probability distribution is the equivalent of a frequency table based on sample observations, often represented in a bar plot.

  • Analogously, the distribution for a continuous random variable can be linked to the density histogram.

  • In a density histogram, the proportion of a sample falling within a specific range is determined by adding the areas of the bins within that range.

Example: Adult Height Analysis (USDA Database)

  • Consider the height of 33 million US adults from the mid-90's (sourced from the USDA Food Commodity Intake Database). This sample functions as a simple random sample of the US population.

  • To estimate the proportion of the sample between 180 cm180\,cm and 185 cm185\,cm tall using a density histogram with a bin width of 2.5 cm2.5\,cm:

    • The two bins in the 180−185 cm180-185\,cm range have counts of 195307195307 and 156239156239 individuals.

    • The heights of these bins in the density histogram are calculated as:

      • 12.5×1953073000000\frac{1}{2.5} \times \frac{195307}{3000000}

      • 12.5×1562393000000\frac{1}{2.5} \times \frac{156239}{3000000}

    • The resulting probability estimate for the range [180,185][180, 185] is summarized as:

      • 195307+1562393000000=0.1172\frac{195307 + 156239}{3000000} = 0.1172

  • This fraction represents the proportion of the histogram's total area falling within the specified range.

Density Functions and Refinement

  • Adding more bins to a density histogram provides greater detail. Larger sample sizes allow for smaller bin widths; small sample sizes with many bins lead to volatile bin heights due to low counts per bin.

  • As bin widths become extremely slim, the hollow histogram begins to resemble a smooth curve. This smooth curve represents the probability density function ff, also known as the density or distribution.

  • This curve serves as the best explanation for a population characteristic when the variable is a continuous numerical variable.

Rules for Density Functions

  • The density ff for a continuous random variable XX must satisfy two primary conditions:

    1. f(x)≥0f(x) \ge 0 for any real xx.

    2. The total area under the density's curve is exactly 11.

  • Well-known densities include the normal density and the uniform density.

The Uniform Distribution

  • Let XX be a random number between 00 and 11. In a histogram using bins of width 0.10.1 ([0,0.1],[0.1,0.2],…,[0.9,1.0][0, 0.1], [0.1, 0.2], \dots, [0.9, 1.0]), each bin contains roughly 10%10\% of the data.

  • The height of the density histogram would be constant: 0.10.1=1\frac{0.1}{0.1} = 1.

  • The density function for a variable XX following a uniform distribution on the interval [0,1][0, 1] (denoted X∼U[0,1]X \sim U[0, 1]) is:

    • f(x)=1f(x) = 1 for x∈[0,1]x \in [0, 1]

    • f(x)=0f(x) = 0 otherwise.

  • General Uniform Distribution: For XX on the interval [a,b][a, b], denoted X∼U[a,b]X \sim U[a, b]:

    • f(x)={1b−ax∈[a,b]0otherwisef(x) = \begin{cases} \frac{1}{b-a} & x \in [a, b] \\ 0 & \text{otherwise} \end{cases}

Probabilities for Continuous Random Variables

  • Probabilities for a continuous random variable XX are determined as the area under the density curve ff over a specific interval.

  • Calculation for the probability of XX being between aa and bb:

    • P(a≤X≤b)=∫abf(x)dxP(a \le X \le b) = \int_a^b f(x) dx

  • Population Proportion: The sample proportion of a bin tends toward the probability of the bin as sample size nn increases (Law of Large Numbers). The density histogram simultaneously converges into the continuous density.

  • Example calculation: For US adult heights, P(height between 180 and 185)=0.1157P(\text{height between 180 and 185}) = 0.1157. This is similar to the histogram estimate of 0.11720.1172.

  • Zero Probability for Exact Values: The probability of a randomly selected person being exactly a specific value (e.g., exactly 180.0000… cm180.0000\dots\,cm) is zero. This is because there is no area captured between a point and itself (P(X=x)=0P(X=x) = 0). Area represents probability, and a single point has no width.

Cumulative Distribution Function (CDF) and Quantile Function

  • CDF Definition: The cumulative distribution function evaluated at xx, denoted F(x)F(x), is the probability that XX takes a value lower than or equal to xx:

    • F(x)=P(X≤x)F(x) = P(X \le x)

  • For continuous variables, F(x)F(x) is a smooth continuous function starting at 00 (as xx approaches −∞-\infty) and rising to 11 (as xx approaches +∞+\infty).

  • Quantile Function Definition: The quantile function QQ is the inverse of the CDF:

    • Q(p)=xQ(p) = x if F(x)=pF(x) = p

  • Specific Quantiles:

    • Q(0.5)Q(0.5): Population median.

    • Q(0.25)Q(0.25): First quartile.

    • Q(0.75)Q(0.75): Third quartile.

    • Q(p)Q(p): The pp-quantile or 100p100p percentile.

  • Interquartile Range (IQR): Calculated as Q(0.75)−Q(0.25)Q(0.75) - Q(0.25).

Expectation and Variability

  • Expected Value (Population Mean): Denoted by μ\mu or E(X)E(X). It is the counterpart to the sample mean and represents the balancing point (center of gravity) of the continuous probability distribution.

  • Mean Formula: The expected value is the area under the function xf(x)xf(x), expressed via the integral:

    • μ=∫−∞∞xf(x)dx\mu = \int_{-\infty}^{\infty} x f(x) dx

  • Symmetrical Distributions: For distributions like the normal or uniform, the mean lies exactly on the symmetry axis. In these cases, the population mean equals the population median.

  • Variance (Variability): The formula for variance (σ2\sigma^2) holds for continuous variables using integrals instead of sums:

    • σ2=Var(X)=E[(X−E(X))2]=∫−∞∞(x−μ)2f(x)dx\sigma^2 = \text{Var}(X) = E[(X - E(X))^2] = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) dx

Linear Transformations

  • Adding a constant (aa): All outcomes increase by aa. Mean and median increase by aa. Standard deviation and IQR remain unchanged.

  • Multiplying by a constant (bb): All outcomes are multiplied by bb. Mean and median are multiplied by bb. Standard deviation and IQR are multiplied by bb. Variance is multiplied by b2b^2.

  • Investment Example: Leonard invests $4000\$4000 in Google Inc. with a $50\$50 fee. Monthly growth XX has mean 2.1%2.1\% (0.0210.021 in decimal) and standard deviation 8.486%8.486\% (0.084860.08486).

    • Gain Formula: G=4000X−50G = 4000X - 50

    • Expected Gain: E(G)=4000(0.021)−50=$34E(G) = 4000(0.021) - 50 = \$34

    • Standard Deviation of Gain: σG=4000(0.08486)=$339.44\sigma_G = 4000(0.08486) = \$339.44

    • Variance of Gain: Var(G)=40002×Var(X)=115219.51\text{Var}(G) = 4000^2 \times \text{Var}(X) = 115219.51

The Normal Distribution Model

  • Nature: Symmetric, unimodal, bell-shaped density curve. It is also called the Gaussian distribution (after Frederic Gauss).

  • Parameters: The distribution is described exactly by the mean (μ\mu) and the standard deviation (σ\sigma).

    • Changing μ\mu shifts the curve left/right.

    • Changing σ\sigma stretches/constricts the curve.

  • Notation: N(μ,σ)N(\mu, \sigma).

  • Standard Normal Distribution: Specifically N(0,1)N(0, 1).

  • Application Examples: SAT scores (X∼N(1500,300)X \sim N(1500, 300)) and ACT scores (Y∼N(21,5)Y \sim N(21, 5)).

Z-scores (Standardization)

  • Definition: A z-score specifies the number of standard deviations an observation falls above or below the mean.

  • Formula: z=x−μσz = \frac{x - \mu}{\sigma}

  • Intersubject Comparison: Comparisons between different distributions (like SAT vs. ACT) are made using z-scores.

    • Example: Ann (SAT 18001800) has z=1800−1500300=1z = \frac{1800 - 1500}{300} = 1. Tom (ACT 2424) has z=24−215=0.6z = \frac{24 - 21}{5} = 0.6. Ann performed better relative to her group.

  • Random Variable Z: If XX is normally distributed, the z-score random variable Z=X−μσZ = \frac{X - \mu}{\sigma} follows the standard normal distribution N(0,1)N(0, 1).

  • Inverse Transformation: If Z∼N(0,1)Z \sim N(0, 1), then X=a+bZX = a + bZ follows a normal distribution with mean aa and standard deviation bb.

Calculating Normal Probabilities

  • Area Interpretation: The area to the left of an outcome xx represents the percentile linked to xx (P(X≤x)P(X \le x)).

  • Software Integration (Excel Commands):

    • Cumulative probability (P(X≤x)P(X \le x)): norm.dist(x; mean; sd; TRUE)

    • Complement probability (P(X≥x)P(X \ge x)): 1 - norm.dist(x; mean; sd; TRUE)

    • Interval probability (P(a≤X≤b)P(a \le X \le b)): norm.dist(b; mean; sd; TRUE) - norm.dist(a; mean; sd; TRUE)

  • Standard Normal Command: To find P(Z≤z)P(Z \le z), use norm.dist(z; 0; 1; TRUE).

  • Normal Probability Table: If software is unavailable, a table lists z-scores and corresponding probabilities P(Z≤z)P(Z \le z).

Normal Quantiles and Percentiles

  • Inverse Probability: To find the observation xx corresponding to a known percentile pp:

    • Example: The 40th40^{th} percentile of male heights (N(70,3.3)N(70, 3.3)) is found using norm.inv(0.4; 70; 3.3), yielding 69.16 inches69.16\,inches.

    • Specific thresholds: The 95th95^{th} percentile for SAT scores (N(1500,300)N(1500, 300)) is approximately 1993.51993.5.

The 68-95-99.7 Rule

  • Rule of Thumb for Normal Distributions:

    • ~68%68\% of observations fall within 11 standard deviation of the mean (μ±σ\mu \pm \sigma).

    • ~95%95\% of observations fall within 22 standard deviations of the mean (μ±2σ\mu \pm 2\sigma).

    • ~99.7%99.7\% of observations fall within 33 standard deviations of the mean (μ±3σ\mu \pm 3\sigma).

  • Extreme Outliers: Probability of falling further than 44 SDs is 11-in-1500015000; 55 SDs is 11-in-22 million; 66 SDs is 11-in-500500 million.

Evaluating Normality

  • Histogram Method: Overlay the best-fitting normal curve (using sample mean and SD) on a histogram of the data. Close alignment suggests normality.

  • Normal Quantile Plot (Q-Q Plot): Data are plotted against theoretical standard normal quantiles.

    • If the data follow the normal model, points should form a straight line.

    • Linear Properties: The line has an intercept of −μσ-\frac{\mu}{\sigma} and a slope of 1σ\frac{1}{\sigma}.

  • Structural Departures:

    • Left Skew: Histogram is shifted; Q-Q plot shows a "wave" or consistent curvature.

    • Right Skew: Strong deviations on the upper right of the Q-Q plot.

    • Sample Size Influence: Smaller data sets (n=40n=40) show more volatility in plots; larger data sets (n=400n=400) show much clearer structure and tighter adherence to the model.

Normal Approximation to the Binomial

  • Use Case: Computing binomial probabilities is cumbersome for large nn. The normal distribution can estimate these probabilities.

  • Success-Failure Conditions: The approximation is valid if:

    • np≥10np \ge 10

    • n(1−p)≥10n(1 - p) \ge 10

  • Approximation Parameters:

    • μ=np\mu = np

    • σ=np(1−p)\sigma = \sqrt{np(1 - p)}

  • Continuity Correction: The normal distribution is continuous, while the binomial is discrete. To improve accuracy for specific intervals:

    • Lower cutoff: Decrease by 0.50.5.

    • Upper cutoff: Increase by 0.50.5.

    • Example: For smokers (n=400,p=0.20n=400, p=0.20), the probability of finding 69,70, or 7169, 70, \text{ or } 71 smokers using normal approximation requires calculating P(68.5≤X≤71.5)P(68.5 \le X \le 71.5), which equals 0.06870.0687. The exact binomial result is 0.07030.0703.

Questions & Discussion

  • Question on Sampling: What is the source of the adult height data?

    • Response: It relies on the USDA Food Commodity Intake Database and is considered a simple random sample.

  • Question on Exact Measurements: Is there a chance a person's measured height is exactly 180 cm180\,cm?

    • Response: In practice, yes, because height is often rounded to the nearest centimeter. This means anyone between 179.5 cm179.5\,cm and 180.5 cm180.5\,cm would be recorded as 180 cm180\,cm, which has a positive probability (P(179.5≤X≤180.5)P(179.5 \le X \le 180.5)).

  • Practice Problem (Guided Practice 4.36):

    • Given P(one adult is between 180 and 185)=0.1157P(\text{one adult is between 180 and 185}) = 0.1157.

    • (a) Probability all three randomly selected adults are in this range: 0.11573=0.00150.1157^3 = 0.0015.

    • (b) Probability none are in this range: (1−0.1157)3=0.692(1 - 0.1157)^3 = 0.692.

  • Practice Problem (Guided Practice 4.48): Compute Tom's z-score.

    • Response: zTom=24−215=0.6z_{Tom} = \frac{24 - 21}{5} = 0.6. Ann's z-score of 1.01.0 exceeds Tom's, meaning she performed better within her SAT group than Tom did in his ACT group.

  • Practice Problem (Guided Practice 4.61): Percentile for SAT scores.

    • (a) Percent scoring 900900 to 21002100: Since These are 22 SDs from the mean (1500±2(300)1500 \pm 2(300)), it is about 95%95\%.

    • (b) Percent scoring 15001500 to 21002100: Since the model is symmetric, this is half of the 95%95\% range, or 47.5%47.5\%.