Z-Scores and the Normal Distribution (Lecture Notes)
Z-Scores and the Normal Distribution (Lecture Notes)
Acknowledgment of country: recognize the traditional owners of the lands where we meet, their ancestors and descendants, and their cultural and spiritual connections to country, acknowledging their contributions to Australian and global society and that these lands have been sites of education and research for millennia.
Week-to-week building: this week builds on last week's topic (distributions) and sets up next week's focus (correlations), which rely on z-scores. Correlations use z-scores in their calculation and are a core method in research and the basis of the upcoming assignment.
Recap: distributions have three key characteristics to describe them:
Shape
Measure of central tendency (where the center sits; the middle score)
Measure of spread (how spread out the scores are)
When a distribution is symmetric around its central tendency (the mean for a normal distribution), it tends to form a normal distribution (bell-shaped curve, Gaussian).
Core concept for this week: z-scores are normal scores expressed in units of standard deviations (SD). They are a standardization of scores from any distribution, enabling comparisons across different scales.
Connection to standard deviation (SD): we already learned how to compute SD last week. Z-scores convert raw scores into standard deviation units using that SD. This is a unit conversion, not a change in the relative position of scores.
Why standardization matters: converting to z-scores allows comparisons across apples and oranges (different scales), and enables precise probability calculations under the standard normal curve.
What you’ll learn and how it fits into the course:
This week builds on SDs and normality; next week covers correlations (which use z-scores) and form a basis for the assignment.
The normal distribution is a mathematical construct that can be described with a formula, allowing calculations beyond direct empirical data, and enabling a uniform framework across many variables.
What is a normal distribution? a family of symmetric, bell-shaped curves where:
The mean, median, and mode coincide (in a perfectly normal distribution).
The spread is characterized by SD; different distributions can have different means and SDs.
With enough data, many real-world variables (e.g., height, IQ) approach this ideal shape.
Why can we rely on a normal shape? The central limit theorem underpins parametric tests (e.g., t-tests). If the data are roughly normally distributed or the sample size is large, many statistical procedures work well and yield inferences about populations.
The standard deviation as a central concept:
It reflects the typical distance of scores from the mean.
It is the unit used to express dispersion; converting data to SD units yields z-scores.
Units and conversions (intuition):
Converting to SD units is a unit change, akin to converting height between centimeters and inches. The actual value doesn’t change, only the label changes.
Example intuition: Phil’s height (in inches vs. cm) is the same measurement represented differently; the same goes for standard deviation when converting to z-scores.
What is a z-score? The deviation of a score from the mean, expressed in units of SD.
Positive z-scores: above the mean; negative z-scores: below the mean; z ≈ 0: around the mean.
Z-score is a standard score: it converts raw scores into standard units, enabling comparisons across distributions.
Notation for mean and SD (population vs. sample):
Population mean:
μ (mu)
Population SD: σ (sigma)
- Sample mean:
m (often denoted as b5 or sometimes bc; here referenced as m)
Sample SD: s.d. (often denoted as s or s_d)
In practice, if a formula uses μ and σ you’re dealing with population parameters; if it uses m and s (or s.d.) you’re dealing with a sample.
The z-score formula (transformation to standard normal):
For a score x from a distribution with mean μ and SD σ:
Conversely, given a z-score and the distribution parameters, the original score can be recovered by:
For a sample, replace μ with the sample mean m and σ with the sample SD s:
and
Why z-scores are useful:
They put different distributions on a common scale (standard normal with mean 0 and SD 1).
They allow meaningful comparisons across different measures (e.g., test scores from different subjects).
They enable precise probability statements about where a score lies in its distribution, using the standard normal curve.
Three distributions: a demonstration of standardization across different spreads:
Case A: mean = 100, SD = 10, score x = 110
Deviation = 110 - 100 = 10; z = 10/10 = 1.
Case B: mean = 100, SD = 15, score x = 110
Deviation = 10; z = 10/15 ≈ 0.67.
Case C: mean = 100, SD = 25, score x = 110
Deviation = 10; z = 10/25 = 0.40.
Observation: same raw score (110) is more unusual in Case A (higher SD means more spread) than in Case C, when viewed in SD units. Z-scores reveal equivalent relative positions across different distributions.
Why the same score can be equally 'unusual' across distributions:
If two distributions have the same relative position (e.g., 1 SD above the mean) but different spreads, the z-score places them at the same relative location in standard units. This allows fair comparison of scores from different contexts.
Practical example: comparing two courses with different grading schemes
If calculus has mean 60 and SD 10, and another subject has mean 90 and SD 15, a raw score of 70 vs 105 cannot be judged by raw scores alone. Converting to z-scores lets you compare who performed better relative to their course’s distribution.
Real-world examples (illustrative):
Don Bradman (cricket) vs. Ted Williams (baseball):
By converting their sport-specific scores to z-scores, you can compare dominance within their peers despite different scales and sports.
Example approximations mentioned in the lecture: Bradman’s z-score was extremely high (described as around 4 to 5 SDs above the mean in the example). Williams also scored highly in his own distribution, but the z-score difference highlights relative outperformance within each sport.
Einstein’s IQ (reported around 180):
IQ tests are designed to have μ = 100 and σ = 15, so z = (180 - 100)/15 ≈ 5.33. Such a score is extraordinarily rare under the normal assumption.
Worked examples (practice with z-scores and back-conversion):
Example 1 (reverse from z to raw score):
Given mean μ = 55, SD σ = 3, and desired z = 1.5
Convert to raw score:
Example 2 (reverse for another subject):
Given μ = 60, σ = 10, z = -0.4
These show that converting to z-scores and back is straightforward and consistent.
The standard normal distribution and its properties:
If we standardize any normal distribution, we get the standard normal distribution with:
The shape is preserved in relative positions; only the axis labels change.
The normal curve is symmetric; probabilities on one side mirror the other.
The area under the curve from -∞ to ∞ is 1 (i.e., 100% of data).
The point of inflection occurs at one SD from the mean (in a standard normal, at Z = ±1).
Reading and using z-tables (the practical tool):
A z-table provides, for a given Z,:
The area between the mean (0) and Z: P(0 ≤ Z ≤ z)
The area beyond Z: P(Z ≥ z) or P(Z ≤ -z) depending on tail orientation
Because the standard normal is symmetric, you can obtain negative-side probabilities from positive-side values by symmetry.
Common reference values:
For Z = 1, the area between the mean and Z is 34.13% (0.3413).
For Z = 1.96, the cumulative tail beyond Z is about 2.5% on one side (hence 5% two-tailed).
The 68-95-99.7 rule (empirical rule):
Approximately 68% of data lie within ±1 SD
Approximately 95% lie within ±2 SDs
Approximately 99.7% lie within ±3 SDs
These probabilities apply to any normally distributed variable after standardization.
Percentiles and z-tables in practice:
Percentile rank of a score is the percentage of scores below that value.
To find a percentile from a z-score, use the z-table to get the area between the mean and the z-score, and add 50% (to account for the below-mean half).
Example: IQ = 115 with μ = 100, σ = 15
z = (115 - 100)/15 = 1
Area between mean and Z = 34.13% → percentile = 50% + 34.13% = 84.13%
For very high percentiles (e.g., 98th), the z-table may not go that far; interpolate or use the approximate z-value (e.g., Z ≈ 2.05 for the 98th percentile in many tables).
Worked percentile examples and applications:
Mensa cutoff: 98th percentile on IQ tests. Steps illustrated:
98% area corresponds to the upper tail area of 0.02 (two-tailed includes the lower tail as well for a two-tailed test; for the upper tail, use the 0.48 area beyond the 0.50 marker).
The corresponding z-score is about Z ≈ 2.05.
Convert back to raw score:
Between 100 and 115 on IQ (μ = 100, σ = 15):
z = (115 - 100)/15 = 1
Percentile between mean and 115 is 34.13%
Therefore, percentile from leftmost to 115 is 50% + 34.13% = 84.13%
Top 5% scores on IQ:
Find z such that P(Z ≥ z) = 0.05 (tail probability)
z ≈ 1.64 (upper-tail cutoff)
Convert back to raw IQ score:
Two-tailed 5% significance example:
For a two-tailed test with α = 0.05, each tail is 2.5%.
Critical Z values: Z = ±1.96.
Corresponding IQ cutoffs: upper ≈ 100 + 1.96×15 = 129.4; lower ≈ 100 - 1.96×15 = 70.6.
Practical workflow using z-scores in research:
Step 1: Check if the data are approximately normally distributed (or use a large enough sample via the Central Limit Theorem).
Step 2: Standardize scores to z-scores using .
Step 3: Use the standard normal curve to determine probabilities, percentiles, and relative standing via z-tables (or calculators).
Step 4: If needed, convert back to raw scores using to report concrete values.
Step 5: For comparisons across different measures, compare z-scores rather than raw scores.
Important practical implications and uses:
Z-scores provide a precise, standardized way to understand how far a score is from the mean, in SD units.
They enable meaningful comparisons across different measures and scales (e.g., cross-subject performance, different sports, or different tests).
They underpin probabilities and expectations about populations, enabling precise predictions and clinical cutoffs (e.g., thresholds like prosopagnosia ~5% cutoff).
They form the backbone of the standard normal curve, so results about probabilities and percentiles generalize across all normally distributed variables.
Real-world connections and philosophical notes:
The normal distribution is pervasive in psychology and natural phenomena; many variables converge to normality with sufficient data collection due to sampling processes and the central limit theorem.
Statistical reasoning using z-scores is a key skill for making inferences about populations from samples.
The practice has ethical and practical implications when used for clinical cutoffs or decisions (e.g., diagnosing conditions, eligibility for programs). Cutoffs are based on percentile/tail criteria, and the choice of α (e.g., 0.05) reflects conventions about balancing false positives and false negatives.
Quick connections to next topics:
Next week: correlations (which rely on z-scores for computation and interpretation).
Correlation analysis uses standardized scores to measure relationships between variables on different scales.
Summary takeaways:
Z-scores convert raw scores into SD units, enabling unit-free comparisons and precise probability calculations under the normal curve.
The standard normal distribution (mean 0, SD 1) provides a universal reference for all normal distributions.
Use z-tables (or calculators) to obtain percentile ranks and tail probabilities, and convert back to raw scores when needed.
The 68-95-99.7 rule gives quick intuition about where most data lie relative to the mean in a normal distribution.
Readings and prep for next lecture:
Reading: Chapter 3 of the textbook (this week).
Preparation for next week's lecture: Chapter 11 (Correlations).
Practice problems in UQ Extend Module 6.
Exam reminder (brief): details about permitted items and logistics for the upcoming exam are provided in class materials; bring pencils, ID, and any approved resources as specified.
Final note: the aim of this content is not just to pass an exam but to understand how distributions work in the world and how we can make precise inferences about them using z-scores and the normal distribution.