Exhaustive Study Guide: Variance, Standard Deviation, and Z-Score Standardization

Fundamentals of Variance and Standard Deviation

• Dataset Overview:

  • Dataset consisting of four population values ($N = 4$): $4, 8, 2, 6$

  • This dataset represents a complete population of four individuals.

• Summation Notation ($\Sigma$):

  • The symbol $\Sigma$ (sigma) denotes summation.

  • In statistical notation, $\sum_{i=1}^{N} x_i$ indicates that every individual score ($x_i$) in the dataset from index $1$ to $N$ must be summed together.

  • Calculating the sum of raw scores:         $\sum x = 4 + 8 + 2 + 6 = 20$

• Calculating the Mean ($\bar{x}$ or $\mu$):

  • The mean is calculated by dividing the sum of all scores by the total number of scores ($N$):         $\bar{x} = \frac{\sum x}{N} = \frac{20}{4} = 5$

  • The mean for this dataset is $5$

Step-by-Step Calculation of Variance and Standard Deviation

• Step 1: Calculate Deviation Scores ($x - \bar{x}$):

  • A deviation score measures the distance and direction of a raw score from the mean.

  • Formula: $\text{Deviation} = x - \bar{x}$

  • Calculations for each score:

    • $4 - 5 = -1$

    • $8 - 5 = 3$

    • $2 - 5 = -3$

    • $6 - 5 = 1$

  • The deviation scores are $-1, 3, -3, 1$

  • Note: Summing raw deviation scores always yields $0$ ($\sum (x - \bar{x}) = 0$), which necessitates squaring them.

• Step 2: Calculate Squared Deviation Scores ($(x - \bar{x})^2$):

  • Squaring eliminates negative values so they do not cancel out during summation:

    • $(-1)^2 = 1$

    • $(3)^2 = 9$

    • $(-3)^2 = 9$

    • $(1)^2 = 1$

• Step 3: Calculate the Sum of Squares ($SS$):

  • The Sum of Squares ($SS$) is the sum of all squared deviation scores:         $SS = \sum (x_i - \bar{x})^2 = 1 + 9 + 9 + 1 = 20$

  • Note: The sum of squares equaling $20$ in this specific example is purely coincidental and does not always match the sum of raw scores.

• Step 4: Calculate the Variance ($\sigma^2$ or $s^2$):

  • Variance is the average squared distance of scores from the mean.

  • For a population, divide $SS$ by $N$:         $\sigma^2 = \frac{SS}{N} = \frac{20}{4} = 5$

  • The variance for this dataset is $5$

• Step 5: Calculate the Standard Deviation ($\sigma$ or $s$):

  • Standard deviation is the square root of the variance, converting the metric back to the original units of measurement:         $\sigma = \sqrt{\sigma^2} = \sqrt{5} \approx 2.237$

  • Interpretation: Scores in this dataset deviate from the mean by an average of approximately $2.237\,\text{units}$.

  • Adding $2.237\,\text{units}$ to the mean ($5 + 2.237 = 7.237$) establishes a boundary one standard deviation above the mean.

• Visualizing Variance and Distance from the Mean:

  • When plotting raw data points (e.g., $97.88, 107.91, 88.26, 115.21, 87.38$), deviation can be envisioned as a linear distance from the mean line.

  • Squaring each deviation score visually constructs physical square areas attached to each data point.

  • Variance represents the average surface area of these constructed squares across all data points.

Population versus Sample Variance Correction

• Formula Difference for Samples:

  • Random samples tend to cluster around the center and are less likely to contain extreme values compared to the full population.

  • Using the population formula ($N$ in the denominator) on sample data creates bias, underestimating the true population variance.

  • To correct for this bias, the denominator is adjusted to $n - 1$ for samples:         $s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1}$

  • Reducing the denominator slightly increases the overall calculated variance and standard deviation, providing an unbiased estimate of the population parameters.

Concept and Calculation of Z-Scores

• Purpose of Standardization:

  • Raw scores convey absolute magnitude but do not specify where a score sits relative to the overall distribution.

  • Standardizing transforms a raw score into a $z$-score, identifying its exact relative position within a distribution.

• Notation Standards:

  • Sample Standard Deviation: $s$ or $SD$

  • Sample Variance: $s^2$ or $SD^2$

  • Population Mean: $\mu$

  • Population Standard Deviation: $\sigma$

  • Sample Mean: $\bar{x}$

• Required Components for Computing a Z-Score:

  1. Raw score ($x$)

  2. Distribution mean ($\mu$ or $\bar{x}$)

  3. Distribution standard deviation ($\sigma$ or $s$)

• Z-Score Formulas:

  • Population Formula:         $z = \frac{x - \mu}{\sigma}$

  • Sample Formula:         $z = \frac{x - \bar{x}}{s}$

  • Note: The mathematical computation is identical for both populations and samples because the $z$-score formula is inherently unbiased within its own distribution.

• Interpreting Z-Scores:

  • Sign:

    • Positive $z$ (z > 0): Score lies above the mean.

    • Negative $z$ (z < 0): Score lies below the mean.

    • Zero ($z = 0$): Score is exactly equal to the mean.

  • Magnitude: Represents the precise number of standard deviations a raw score lies away from the mean.

  • Note: Standard deviations and variances can never be negative, but $z$-scores frequently contain negative values.

• Z-Score Calculation Examples (Target Score $x = 77$):

  • Scenario A: $\bar{x} = 75$, $s = 2$         $z = \frac{77 - 75}{2} = \frac{2}{2} = 1$

  • Scenario B: $\bar{x} = 70$, $s = 7$         $z = \frac{77 - 70}{7} = \frac{7}{7} = 1$

  • Scenario C: $\bar{x} = 83$, $s = 2$         $z = \frac{77 - 83}{2} = \frac{-6}{2} = -3$

Converting Z-Scores Back to Raw Scores

• Transformation Formulas:

  • Population Context:         $x = \mu + (z \times \sigma)$

  • Sample Context:         $x = \bar{x} + (z \times s)$

• Step-by-Step Conversion Example:

  • Given parameters: $z = -2$, $\mu = 17$, $\sigma = 4$

  • Step 1: Multiply $z$-score by standard deviation:         $-2 \times 4 = -8$

  • Step 2: Add population mean:         $x = 17 + (-8) = 9$

  • The resulting raw score ($x$) is $9$

Properties and Characteristics of Standardized Distributions

• Core Properties of any Z-Distribution:

  1. Distribution Shape: Preserved identically. Plotting $z$-scores produces a distribution with the exact shape as the raw score distribution.

  2. Mean ($\mu_z$): Always equal to $0$         $\mu_z = 0$

  3. Standard Deviation ($\sigma_z$): Always equal to $1$         $\sigma_z = 1$

• Unitless Property:

  • $z$-scores are unitless numbers representing distance in standard deviation units.

  • Transforming measurements (e.g., body weight measured in kilograms or pounds) into $z$-scores strips away physical units.

• Utility of Standardized Distributions:

  • Enables direct comparison of performance across different measures, scales, or distinct samples (e.g., comparing student performance across two separate statistics exam sections with different averages and standard deviations).

Data Transformations: Shifting and Rescaling Data

• Shifting Data (Addition / Subtraction):

  • Adding or subtracting a constant from every score alters the mean by that exact amount while keeping distribution shape and variability unchanged.

  • Example: Adult male weight in kilograms ($kg$).

    • Raw weight values: $100\,kg, 50\,kg, 150\,kg$

    • Subtracting a recommended health benchmark of $74\,kg$ shifts scores to $26\,kg, -24\,kg, 76\,kg$.

    • This recontextualizes data to display distance relative to a health threshold without changing relative distance between scores.

• Rescaling Data (Multiplication / Division / Logarithmic):

  • Multiplying or dividing every score by a constant (or applying non-linear functions such as log transformations) changes the scale of measurement.

  • Example: Measuring cotinine levels (a tobacco byproduct) in $\mu g/mL$

    • Raw values for secondhand smoke exposure are truncated near zero when plotted next to active smokers.

    • Applying a logarithmic transformation rescales the axis, expanding truncated lower-range values for visual clarity without altering fundamental statistical relationships.

• Standardization as a Combined Operation:

  • Standardizing data to $z$-scores combines shifting (subtracting $\bar{x}$) and rescaling (dividing by $s$).

Software Implementation and Practical Application in JASP and Excel

• Standardizing Scores in JASP:

  • Formula entry using the drag-and-drop interface:         $z_{\text{Age}} = \frac{\text{Age} - \text{mean}(\text{Age})}{\text{stdDev}(\text{Age})}$

  • Dataset empirical parameters:

    • Original Age variable: Mean $\bar{x} = 29.7\,\text{years}$, Standard Deviation $s = 14.53\,\text{years}$

    • Standardized Age variable ($z_{\text{Age}}$): Mean $\bar{x} = 0$ (displayed in scientific notation as $\approx 10^{-17}$ due to floating-point calculations), Standard Deviation $s = 1$

  • Comparing visual distribution outputs:

    • Histograms vary slightly based on automatic bin width selection algorithms.

    • Density plots display identical distribution curves.

• Dark Triad Excel Practice Analysis:

  • Sample size $n = 17$ cases

  • Degrees of freedom for sample variance: $n - 1 = 16$

  • Data processing sequence:

    1. Compute mean for each psychological trait (e.g., Narcissism, Psychopathy).

    2. Compute deviation scores ($x - \bar{x}$) across all rows.

    3. Compute squared deviations ($(x - \bar{x})^2$).

    4. Sum squared deviations to find Sum of Squares ($SS$).

    5. Divide $SS$ by $16$ to calculate sample variance ($s^2 \approx 0.1649$).

    6. Calculate square root of variance for sample standard deviation ($s \approx 0.4061$).

    7. Compute individual $z$-scores ($z = \frac{x - \bar{x}}{s}$).

• Case Study Interpretation (Participant 5):

  • Participant 5 achieved identical raw scores across two personality subscales.

  • Standardized $z$-scores revealed that relative to the sample distribution, Participant 5 scored significantly higher in Psychopathy than in Narcissism.

• Course Logistics and Announcements:

  • Homework assignment focus: Calculating descriptive statistics and generating histograms in JASP.

  • Homework due date: One week from next Wednesday.

  • Exit ticket password: standardized

Questions and Interactive Discussion

• Question: Why do we subtract the mean when finding variance?

  • Answer: Subtracting the mean yields the distance of every score from the center of the distribution. Summing these distances allows the calculation of the average distance of all scores from the mean.

• Question: Clarification on standardizing step sequence.

  • Answer: Step 1 calculates the mean, then subtracts the mean from each raw score. Step 2 squares each resulting deviation score before summing and dividing by the sample or population denominator.

• Question: Explanation of $\sum_{i=1}^{n} x_i$ summation notation variables.

  • Answer: The variable $i = 1$ at the bottom denotes the starting case/observation, and $n$ at the top denotes the total number of observations. $x_i$ represents the raw score for case number $i$.

• Question: What does the bar over the letter $x$ ($\bar{x}$) represent?

  • Answer: The notation $\bar{x}$ ("x-bar") represents the sample mean.

• Question: Will formulas be provided on exams and homeworks?

  • Answer: Homeworks do not explicitly list formulas, but they are accessible in reference materials. Exam formula provisions will be clarified prior to test dates.

• Question: Why does the $z$-score formula remain unchanged between samples and populations?

  • Answer: The $z$-score formula is unbiased because it evaluates relative positioning strictly within its own existing distribution, requiring no sample size correction factor.