Distributions of Random Variables: Continuous Distributions, Linear Transformations, and the Normal Model
Continuous Distribution
For discrete random variables, the probability distribution is the equivalent of a frequency table based on sample observations, often represented in a bar plot.
Analogously, the distribution for a continuous random variable can be linked to the density histogram.
In a density histogram, the proportion of a sample falling within a specific range is determined by adding the areas of the bins within that range.
Example: Adult Height Analysis (USDA Database)
Consider the height of million US adults from the mid-90's (sourced from the USDA Food Commodity Intake Database). This sample functions as a simple random sample of the US population.
To estimate the proportion of the sample between and tall using a density histogram with a bin width of :
The two bins in the range have counts of and individuals.
The heights of these bins in the density histogram are calculated as:
The resulting probability estimate for the range is summarized as:
This fraction represents the proportion of the histogram's total area falling within the specified range.
Density Functions and Refinement
Adding more bins to a density histogram provides greater detail. Larger sample sizes allow for smaller bin widths; small sample sizes with many bins lead to volatile bin heights due to low counts per bin.
As bin widths become extremely slim, the hollow histogram begins to resemble a smooth curve. This smooth curve represents the probability density function , also known as the density or distribution.
This curve serves as the best explanation for a population characteristic when the variable is a continuous numerical variable.
Rules for Density Functions
The density for a continuous random variable must satisfy two primary conditions:
for any real .
The total area under the density's curve is exactly .
Well-known densities include the normal density and the uniform density.
The Uniform Distribution
Let be a random number between and . In a histogram using bins of width (), each bin contains roughly of the data.
The height of the density histogram would be constant: .
The density function for a variable following a uniform distribution on the interval (denoted ) is:
for
otherwise.
General Uniform Distribution: For on the interval , denoted :
Probabilities for Continuous Random Variables
Probabilities for a continuous random variable are determined as the area under the density curve over a specific interval.
Calculation for the probability of being between and :
Population Proportion: The sample proportion of a bin tends toward the probability of the bin as sample size increases (Law of Large Numbers). The density histogram simultaneously converges into the continuous density.
Example calculation: For US adult heights, . This is similar to the histogram estimate of .
Zero Probability for Exact Values: The probability of a randomly selected person being exactly a specific value (e.g., exactly ) is zero. This is because there is no area captured between a point and itself (). Area represents probability, and a single point has no width.
Cumulative Distribution Function (CDF) and Quantile Function
CDF Definition: The cumulative distribution function evaluated at , denoted , is the probability that takes a value lower than or equal to :
For continuous variables, is a smooth continuous function starting at (as approaches ) and rising to (as approaches ).
Quantile Function Definition: The quantile function is the inverse of the CDF:
if
Specific Quantiles:
: Population median.
: First quartile.
: Third quartile.
: The -quantile or percentile.
Interquartile Range (IQR): Calculated as .
Expectation and Variability
Expected Value (Population Mean): Denoted by or . It is the counterpart to the sample mean and represents the balancing point (center of gravity) of the continuous probability distribution.
Mean Formula: The expected value is the area under the function , expressed via the integral:
Symmetrical Distributions: For distributions like the normal or uniform, the mean lies exactly on the symmetry axis. In these cases, the population mean equals the population median.
Variance (Variability): The formula for variance () holds for continuous variables using integrals instead of sums:
Linear Transformations
Adding a constant (): All outcomes increase by . Mean and median increase by . Standard deviation and IQR remain unchanged.
Multiplying by a constant (): All outcomes are multiplied by . Mean and median are multiplied by . Standard deviation and IQR are multiplied by . Variance is multiplied by .
Investment Example: Leonard invests in Google Inc. with a fee. Monthly growth has mean ( in decimal) and standard deviation ().
Gain Formula:
Expected Gain:
Standard Deviation of Gain:
Variance of Gain:
The Normal Distribution Model
Nature: Symmetric, unimodal, bell-shaped density curve. It is also called the Gaussian distribution (after Frederic Gauss).
Parameters: The distribution is described exactly by the mean () and the standard deviation ().
Changing shifts the curve left/right.
Changing stretches/constricts the curve.
Notation: .
Standard Normal Distribution: Specifically .
Application Examples: SAT scores () and ACT scores ().
Z-scores (Standardization)
Definition: A z-score specifies the number of standard deviations an observation falls above or below the mean.
Formula:
Intersubject Comparison: Comparisons between different distributions (like SAT vs. ACT) are made using z-scores.
Example: Ann (SAT ) has . Tom (ACT ) has . Ann performed better relative to her group.
Random Variable Z: If is normally distributed, the z-score random variable follows the standard normal distribution .
Inverse Transformation: If , then follows a normal distribution with mean and standard deviation .
Calculating Normal Probabilities
Area Interpretation: The area to the left of an outcome represents the percentile linked to ().
Software Integration (Excel Commands):
Cumulative probability ():
norm.dist(x; mean; sd; TRUE)Complement probability ():
1 - norm.dist(x; mean; sd; TRUE)Interval probability ():
norm.dist(b; mean; sd; TRUE) - norm.dist(a; mean; sd; TRUE)
Standard Normal Command: To find , use
norm.dist(z; 0; 1; TRUE).Normal Probability Table: If software is unavailable, a table lists z-scores and corresponding probabilities .
Normal Quantiles and Percentiles
Inverse Probability: To find the observation corresponding to a known percentile :
Example: The percentile of male heights () is found using
norm.inv(0.4; 70; 3.3), yielding .Specific thresholds: The percentile for SAT scores () is approximately .
The 68-95-99.7 Rule
Rule of Thumb for Normal Distributions:
~ of observations fall within standard deviation of the mean ().
~ of observations fall within standard deviations of the mean ().
~ of observations fall within standard deviations of the mean ().
Extreme Outliers: Probability of falling further than SDs is -in-; SDs is -in- million; SDs is -in- million.
Evaluating Normality
Histogram Method: Overlay the best-fitting normal curve (using sample mean and SD) on a histogram of the data. Close alignment suggests normality.
Normal Quantile Plot (Q-Q Plot): Data are plotted against theoretical standard normal quantiles.
If the data follow the normal model, points should form a straight line.
Linear Properties: The line has an intercept of and a slope of .
Structural Departures:
Left Skew: Histogram is shifted; Q-Q plot shows a "wave" or consistent curvature.
Right Skew: Strong deviations on the upper right of the Q-Q plot.
Sample Size Influence: Smaller data sets () show more volatility in plots; larger data sets () show much clearer structure and tighter adherence to the model.
Normal Approximation to the Binomial
Use Case: Computing binomial probabilities is cumbersome for large . The normal distribution can estimate these probabilities.
Success-Failure Conditions: The approximation is valid if:
Approximation Parameters:
Continuity Correction: The normal distribution is continuous, while the binomial is discrete. To improve accuracy for specific intervals:
Lower cutoff: Decrease by .
Upper cutoff: Increase by .
Example: For smokers (), the probability of finding smokers using normal approximation requires calculating , which equals . The exact binomial result is .
Questions & Discussion
Question on Sampling: What is the source of the adult height data?
Response: It relies on the USDA Food Commodity Intake Database and is considered a simple random sample.
Question on Exact Measurements: Is there a chance a person's measured height is exactly ?
Response: In practice, yes, because height is often rounded to the nearest centimeter. This means anyone between and would be recorded as , which has a positive probability ().
Practice Problem (Guided Practice 4.36):
Given .
(a) Probability all three randomly selected adults are in this range: .
(b) Probability none are in this range: .
Practice Problem (Guided Practice 4.48): Compute Tom's z-score.
Response: . Ann's z-score of exceeds Tom's, meaning she performed better within her SAT group than Tom did in his ACT group.
Practice Problem (Guided Practice 4.61): Percentile for SAT scores.
(a) Percent scoring to : Since These are SDs from the mean (), it is about .
(b) Percent scoring to : Since the model is symmetric, this is half of the range, or .