Comprehensive Study Notes on Z-Scores, Standard Deviation, and the Normal Model

Standardizing Data and Z-Scores

• Standardized Testing Comparison Context:

  • Comparing raw scores from disparate assessment systems (such as an SAT score of $1500$ versus an ACT score of $21$) presents an inherent measurement challenge.

  • Raw values exist on separate, non-comparable scales. Standardized testing reports resolve this by converting raw scores into percentiles (e.g., the $99\text{th}$ percentile), measuring relative standing among all test-takers regardless of the scale.

• Distance from the Mean:

  • Evaluating raw distance from the sample mean is expressed as:     $\text{Distance} = y - \bar{y}$     where $y$ is the individual data value and $\bar{y}$ is the sample mean.

  • This distance preserves original measurement units (e.g., points, meters, or seconds).

• Multi-Sport / Track & Field Analogy:

  • In multi-event athletics (e.g., decathlons featuring the long jump and the $100\,\text{m}$ dash), performance metrics use incompatible units ($\text{meters}$ vs. $\text{seconds}$ or $\text{m/s}$).

  • Comparing an athlete who places $1\text{st}$ in the long jump and $2\text{nd}$ in the race against another who places $2\text{nd}$ in the long jump and $1\text{st}$ in the race requires standardizing data to evaluate overall athletic performance directly.

• Standardization to Dimensionless Quantities (Z-Scores):

  • Standardizing removes all measurement units to produce a direct scale evaluating how unusual or exceptional a data point is relative to its distribution.

  • Standard deviation ($s$) functions as the default standard ruler for distributions centered at the mean.

  • Dividing deviation from the mean by the standard deviation cancels units out completely:     $\frac{\text{units}}{\text{units}} = \text{dimensionless quantity}$

• The Z-Score Formula & Definition:

  • The Z-score formula is defined as:     $Z = \frac{y - \bar{y}}{s}$

  • Key variables:

  • $Z$: The Z-score (dimensionless count of standard deviations a data point lies from the mean).

  • $y$: Individual raw data point.

  • $\bar{y}$: Sample mean.

  • $s$: Sample standard deviation.

• Properties of Z-Scores:

  • Dimensionless: Z-scores possess no physical units.

  • Sign of $Z$:

  • Positive Z-score (Z > 0): Data value $y$ is strictly greater than the mean (y > \bar{y}).

  • Negative Z-score (Z < 0): Data value $y$ is strictly less than the mean (y < \bar{y}).

  • Magnitude ($|Z|$) & Unusualness:

  • Higher absolute values of $Z$ signify greater distance from the mean, indicating higher exceptionality or unusualness.

  • A Z-score of $Z = 2$ is more unusual than $Z = 1$.

  • A Z-score of $Z = -3$ is more unusual than $Z = 1$ because |-3| = 3 > 1

Worked Examples: Standard Deviation and Z-Score Calculations

• Rulers of Spread:

  • For skewed distributions centered at the median, the Interquartile Range ($IQR$) serves as the standard ruler for typical spread.

  • For symmetric distributions centered at the mean, the standard deviation ($s$) serves as the standard ruler.

• Problem 1: Statistics Exam Performance (Gregor):

  • Given parameters: Mean score $\bar{y} = 75\,\text{points}$, Standard deviation $s = 5\,\text{points}$.

  • Gregor's Z-score: $Z_g = -2$

  • Goal: Calculate Gregor's raw exam score ($y$).

  • Step-by-step calculation:     $Z = \frac{y - \bar{y}}{s}$     $-2 = \frac{y - 75}{5}$     $-2 \times 5 = y - 75$     $-10 = y - 75$     $y = -10 + 75$     $y = 65\,\text{points}$

  • Interpretation: A negative Z-score indicates performance below the class mean; Gregor scored $65\,\text{points}$.

• Problem 2: IQ Test Genius Threshold:

  • Genius classification criteria: $Z \ge 2.5$

  • Given parameters: Mean IQ $\bar{y} = 100\,\text{points}$, Standard deviation $s = 15\,\text{points}$.

  • Goal: Calculate the raw IQ score ($y$) required for genius status.

  • Step-by-step calculation:     $Z = \frac{y - \bar{y}}{s}$     $2.5 = \frac{y - 100}{15}$     $2.5 \times 15 = y - 100$     $37.5 = y - 100$     $y = 37.5 + 100$     $y = 137.5\,\text{points}$

  • Interpretation: An IQ score of $137.5\,\text{points}$ or higher places an individual $2.5$ standard deviations above average.

• Problem 3: Weather Extremes (January vs. July High Temperatures):

  • Given climate parameters:

  • January: Mean $\bar{y}_{Jan} = 36^\circ\text{F}$, Standard deviation $s_{Jan} = 10^\circ\text{F}$

  • July: Mean $\bar{y}_{Jul} = 74^\circ\text{F}$, Standard deviation $s_{Jul} = 8^\circ\text{F}$

  • Target temperature to evaluate: $y = 55^\circ\text{F}$

  • Goal: Determine which month experiences a high temperature of $55^\circ\text{F}$ as more unusual.

  • January Z-score calculation:     $Z_{Jan} = \frac{55 - 36}{10} = \frac{19}{10} = 1.9$

  • July Z-score calculation:     $Z_{Jul} = \frac{55 - 74}{8} = \frac{-19}{8} = -2.375$

  • Comparative Analysis:

  • Both temperatures differ from their monthly means by an absolute value of $19^\circ\text{F}$ ($|55 - 36| = 19$ and $|55 - 74| = 19$).

  • Absolute Z-score magnitudes: $|Z_{Jul}| = 2.375$ while $|Z_{Jan}| = 1.9$

  • Because 2.375 > 1.9, a high temperature of $55^\circ\text{F}$ is significantly more unusual in July than in January.

Center, Spread, and the Theoretical Foundation of the Normal Model

• Battery Life Consistency (Problem 28k Context):

  • Comparing two battery models:

  • Model 1: Mean life $\bar{y}_1 = 11\,\text{hours}$

  • Model 2: Mean life $\bar{y}_2 = 12\,\text{hours}$

  • Evaluating performance by mean alone is insufficient without evaluating spread (consistency).

  • If standard deviation $s = 5\,\text{hours}$, battery life varies wildly (dropping as low as $6\,\text{hours}$).

  • If standard deviation $s = 1\,\text{hour}$, battery life stays tightly bound between $10\,\text{hours}$ and $12\,\text{hours}$.

  • Center and spread must always be presented together.

• The Normal Model Definition:

  • Applicable to quantitative distributions that are unimodal and symmetric.

  • Mathematical Abstraction: The Normal Model smooths histogram bars into a continuous theoretical bell curve. No real-world raw dataset is perfectly unimodal and symmetric.

  • Functions like a street map: An abstraction that simplifies raw reality to enable analysis.

  • Notation: Population/model parameters use Greek letters—mean $\mu$ (or model center) and standard deviation $\sigma$ (or model scale).

• The 68-95-99.7 Empirical Rule:

  • For unimodal, symmetric distributions, data divides into standardized intervals:

  • 68% of data falls within $1$ standard deviation of the mean ($-1 \le Z \le 1$). This interval represents expected, non-unusual behavior.

  • 95% of data falls within $2$ standard deviations of the mean ($-2 \le Z \le 2$). Data between $1$ and $2$ standard deviations approaches unusual thresholds.

  • 99.7% of data falls within $3$ standard deviations of the mean ($-3 \le Z \le 3$). Data values falling beyond $3$ standard deviations (|Z| > 3) are extreme outliers.

Visualizing and Drawing the Normal Model

• Guidelines for Drawing the Normal Model:

  • Draw a horizontal axis and a symmetric, bell-shaped curve centered at $Z = 0$ (or raw mean $\bar{y}$).

  • Inflection Points: Identify points where the curve changes concavity (switches from concave downward at the peak to concave upward along the tails).

  • Inflection points occur precisely at $Z = 1$ and $Z = -1$ ($1$ standard deviation from the mean).

  • Mark tick marks symmetrically at equal distances for $1$, $2$, and $3$ standard deviations ($+1\sigma, +2\sigma, +3\sigma$ and $-1\sigma, -2\sigma, -3\sigma$).

  • The curve terminates visually around $3$ standard deviations on either side.

• Illustrative SAT Score Normal Distribution:

  • Given parameters: Mean $\bar{y} = 500\,\text{points}$, Standard deviation $s = 100\,\text{points}$.

  • Standard deviation scale values:

  • Center ($Z = 0$): $500$

  • $+1\sigma$ ($Z = 1$): $600$

  • $+2\sigma$ ($Z = 2$): $700$

  • $+3\sigma$ ($Z = 3$): $800$

  • $-1\sigma$ ($Z = -1$): $400$

  • $-2\sigma$ ($Z = -2$): $300$

  • $-3\sigma$ ($Z = -3$): $200$

  • Interval coverage:

  • $68\%$ of scores lie between $400$ and $600$

  • $95\%$ of scores lie between $300$ and $700$

  • $99.7\%$ of scores lie between $200$ and $800$

Empirical Rule Applications and Percentile Calculations

• Problem 9: Automotive Fuel Economy (MPG):

  • Given parameters: Mean fuel economy $\bar{y} = 24.8\,\text{mpg}$, Standard deviation $s = 6.2\,\text{mpg}$.

  • Standard deviation tick mark values:

  • Center ($Z = 0$): $24.8\,\text{mpg}$

  • $+1s$ ($Z = 1$): $24.8 + 6.2 = 31.0\,\text{mpg}$

  • $+2s$ ($Z = 2$): $31.0 + 6.2 = 37.2\,\text{mpg}$

  • $+3s$ ($Z = 3$): $37.2 + 6.2 = 43.4\,\text{mpg}$

  • $-1s$ ($Z = -1$): $24.8 - 6.2 = 18.6\,\text{mpg}$

  • $-2s$ ($Z = -2$): $18.6 - 6.2 = 12.4\,\text{mpg}$

  • $-3s$ ($Z = -3$): $12.4 - 6.2 = 6.2\,\text{mpg}$

• Step-by-Step Probability & Area Calculations (Problem 9):

  • Central 68% Interval: $68\%$ of vehicles achieve fuel economy between $18.6\,\text{mpg}$ and $31.0\,\text{mpg}$.

  • Percentage above $31.0\,\text{mpg}$ (+1 SD):

  • Total area outside middle $68\%$: $100\% - 68\% = 32\%$

  • Dividing by symmetry across upper and lower tails: $\frac{32\%}{2} = 16\%$

  • Result: $16\%$ of vehicles achieve greater than $31.0\,\text{mpg}$ (and $16\%$ get below $18.6\,\text{mpg}$).

  • Central 95% Interval: $95\%$ of vehicles achieve fuel economy between $12.4\,\text{mpg}$ and $37.2\,\text{mpg}$.

  • Percentage above $37.2\,\text{mpg}$ (+2 SD):

  • Total area outside middle $95\%$: $100\% - 95\% = 5\%$

  • Single upper tail area: $\frac{5\%}{2} = 2.5\%$

  • Result: $2.5\%$ of vehicles achieve greater than $37.2\,\text{mpg}$ (and $2.5\%$ get below $12.4\,\text{mpg}$).

  • Central 99.7% Interval: $99.7\%$ of vehicles achieve fuel economy between $6.2\,\text{mpg}$ and $43.4\,\text{mpg}$.

  • Percentage above $43.4\,\text{mpg}$ (+3 SD):

  • Total area outside middle $99.7\%$: $100\% - 99.7\% = 0.3\%$

  • Single upper tail area: $\frac{0.3\%}{2} = 0.15\%$

  • Result: $0.15\%$ of vehicles achieve greater than $43.4\,\text{mpg}$.

  • Percentage between $31.0\,\text{mpg}$ and $37.2\,\text{mpg}$ (+1 SD to +2 SD):

  • Method 1 (Tail Subtraction): Area above $31.0\,\text{mpg}$ (16\%$) minus area above 37.2\,\text{mpg}$($2.5\%):\n      16\% - 2.5\% = 13.5\%\n - *Method 2 (Empirical Range Subtraction):* Area between 95\%$and$68\%$is$95\% - 68\% = 27\%. Divide symmetric sides:\n      \frac{27\%}{2} = 13.5\%\n - **Percentage less than 37.2\,\text{mpg}$($< +2\text{ SD}):**\n - Area up to +2$standard deviations:$100\% - 2.5\% = 97.5\%\n\n- **Problem 10: Exam Score Distribution & Percentiles:**\n - Given parameters: Mean \bar{y} = 100\,\text{points}$, Standard deviation$s = 15\,\text{points}.\n - Scale breakdown:\n - Center (Z = 0$):$100\,\text{points}\n - \pm 1s$($Z = \pm 1$):$85$to$115\,\text{points}$($68\% of scores)\n - \pm 2s$($Z = \pm 2$):$70$to$130\,\text{points}$($95\% of scores)\n - \pm 3s$($Z = \pm 3$):$55$to$145\,\text{points}$($99.7\% of scores)\n - Calculations:\n - 68\%$of exam scores fall between$85$and$115\,\text{points}.\n - Scores exceeding 130\,\text{points}$represent the top$2.5\% of test-takers.\n - **Percentile Definition:** A percentile represents the cumulative percentage of data falling at or below a given value from left to right.\n - A test score of 130\,\text{points}$($Z = 2$) corresponds to the$97.5\text{th}$percentile ($100\% - 2.5\% = 97.5\%$$).