Comprehensive Notes on Variability and Standard Deviation

Foundations of Statistics: Chapter Four - Variability

• Chapter Four focuses on variability, which, along with central tendency (Chapter Three), constitutes the two most foundational topics in statistics.
• The first four chapters of the course provide the essential building blocks for all subsequent material.
• Starting in Chapter Five (Z-scores), students will begin combining previously learned concepts, specifically scores and means from Chapter Three with the measures of variability learned in Chapter Four.
• Understanding the underlying concept of variability is critical before proceeding to calculations. A student must be able to interpret whether a value, such as a standard deviation of $9$, represents a large or small amount of spread within a specific context.
• Just as there are three ways to define central tendency (mean, median, and mode), there are multiple ways to approach and measure variability.

The Range

• The range is the simplest measure of variability and can be defined as the difference between the highest score and the lowest score in a dataset.
• In mathematics, the word "difference" always signifies the operation of subtraction.
• Range Formula:$Range = X_{max} - X_{min}$
• $X_{max}$ represents the maximum score (highest value) and $X_{min}$ represents the minimum score (lowest value) for the variable $X$.

Example Calculation of Range

• Given a dataset where $N = 8$ scores are: $5, 4, 6, 5, 3, 8, 4, 5$.
• Step 1: Identify the highest score ($X_{max}$). In this set, $X_{max} = 8$.
• Step 2: Identify the lowest score ($X_{min}$). In this set, $X_{min} = 3$.
• Step 3: Apply the formula: $8 - 3 = 5$.
• The range for this dataset is $5$.

Limitations of the Range

• While the range is easy to calculate, it is a limited and often poor measure of variability.
• The primary benefit is speed: one merely scans for the highest and lowest points and subtracts.
• The Two-Score Problem: Regardless of the sample size ($N = 8, 80, 8000$, or even a census of $300,000,000$ Americans), the range only takes two scores into account.
• Example (U.S. Census): If the oldest person is $121$ and the youngest is $0$, the range is $121$. This single number fails to represent the spread of ages for the other hundreds of millions of people.
• Sensitivity to Outliers: Outliers are, by definition, unrepresentative scores. Because the range uses only the maximum and minimum, it is extremely sensitive to these outliers.
• Hypothetical Scenario (Bank Accounts): In a class of $30$ people with modest bank accounts (e.g., between $0$ and $1000$ dollars), adding an outlier like Bill Gates (worth approximately $40,000,000,000$) would change the range from $1000$ to $40,000,000,000$. This massive number does not accurately describe the spread of the rest of the class.

Variance and Standard Deviation

• Variance and standard deviation are superior measures of variability because they take every score in the dataset into account.
• These two measures are a "pair" with a direct mathematical relationship:   - To get the standard deviation from the variance, you take the square root.   - To get the variance from the standard deviation, you square it.
• Symbol Notation Warning:   - Variance is represented as standard deviation squared ($\sigma^2$ or $s^2$).   - Unlike in algebra, where a square is an operation to be solved for, in statistics, the squared symbol is part of the name for variance. You do not "solve" for the base unless you are looking for the standard deviation.

Sum of Squares (SS)

• The bulk of calculations in Chapter Four involves finding the Sum of Squares ($SS$), which is shorthand for the Sum of Squared Deviations.
• To understand $SS$, one must first define a Deviation.
• Deviation Definition: The difference between an individual score ($X$) and the mean ($\mu$ or $M$).   - Deviation Formula: $X - \text{Mean}$

Interpreting Deviations

• A positive deviation indicates the score is above the mean.
• A negative deviation indicates the score is below the mean.
• Exam Example: If the class average is $75$:   - Maria scores $78$: Deviation = $78 - 75 = +3$ (3 points above the mean).   - Betty scores $70$: Deviation = $70 - 75 = -5$ (5 points below the mean).
• Even if you do not know the actual score or the mean, knowing a deviation (e.g., $+3$) tells you exactly how the individual performed relative to the average.
• Note: Positive is not always "good" and negative is not always "bad." For a psychopath scale or a race time, a negative deviation (scoring lower or faster than average) might be preferable.

The Definitional Formula for SS

• $SS = \sum (X - M)^2$
• Order of Operations (PEMDAS) for SS:   1. Parentheses: Calculate every individual deviation ($X - M$).   2. Exponents: Square each deviation individually ($(X - M)^2$). Squaring ensures all values are non-negative.   3. Summation: Add all the squared deviations together ($\sum$).

The Computational Formula for SS

• An alternative formula that is often faster for hand calculations:   $SS = \sum X^2 - \frac{(\sum X)^2}{N}$
• Terms in the Computational Formula:   - $\sum X^2$: The sum of the squared scores. Each score is squared first, then summed.   - $(\sum X)^2$: The sum of the scores, squared. All scores are added first, then the total is squared.   - $N$: The number of scores.

Calculation Check: SS must never be negative.

• Because $SS$ is a sum of squared values, it must be zero or positive. If you calculate a negative $SS$, there is a calculation error.
• The Zero-Sum Rule: In any dataset, the sum of the raw deviations (not squared) must always equal zero ($\sum (X - M) = 0$). This is a crucial check before squaring for $SS$.

Step-by-Step SS Example (Definitional)

• Dataset ($X$): $5, 4, 6, 5, 3, 8, 4, 5$
• $N = 8$; $\sum X = 40$; Mean ($M$) = $40 / 8 = 5$.
• Deviations ($X - 5$):   - $5 - 5 = 0$   - $4 - 5 = -1$   - $6 - 5 = 1$   - $5 - 5 = 0$   - $3 - 5 = -2$   - $8 - 5 = 3$   - $4 - 5 = -1$   - $5 - 5 = 0$   - Check: $0 + (-1) + 1 + 0 + (-2) + 3 + (-1) + 0 = 0$. (Correct).
• Squared Deviations:   - $0^2 = 0$   - $(-1)^2 = 1$   - $1^2 = 1$   - $0^2 = 0$   - $(-2)^2 = 4$   - $3^2 = 9$   - $(-1)^2 = 1$   - $0^2 = 0$
• Summing Squared Deviations: $0 + 1 + 1 + 0 + 4 + 9 + 1 + 0 = 16$.
• $SS = 16$.

Final Formulas for Variance and Standard Deviation

• The formula used depends entirely on whether the data is a population or a sample.

Population Formulas

• Population Variance: $\sigma^2 = \frac{SS}{N}$
• Population Standard Deviation: $\sigma = \sqrt{\frac{SS}{N}}$
• Population variance is literally the "mean squared deviation."

Sample Formulas

• Sample Variance: $s^2 = \frac{SS}{n - 1}$
• Sample Standard Deviation: $s = √(\frac{SS}{n - 1})$
• We divide by $n - 1$ for samples to adjust for bias (making the sample variability a better estimate of the population).

Practical Example (Treatment as Sample)

• Using the same dataset ($SS = 16, n = 8$):
• $s^2 = \frac{16}{8 - 1} = \frac{16}{7} \approx 2.29$
• $s = \sqrt{2.29} \approx 1.51$

Interpreting Standard Deviation

• Standard deviation is the "standard" (average) deviation (distance) from the mean.
• It is highly useful for understanding the spread of scores.

Guesstimating Standard Deviation

• A rough rule of thumb: Standard deviation is often halfway between the smallest possible deviation ($0$) and the largest deviation in the set.
• Test Case: In the example with $M = 5$, the closest score was $5$ (distance $0$) and the furthest was $8$ (distance $3$). Halfway between $0$ and $3$ is $1.5$. Our calculated $s = 1.51$ matches this guess almost perfectly.
• Salary Example: For typists with a mean salary of $\$40,000$, a standard deviation of $\$50$ is too small, and $\$30,000$ is too high. A reasonable spread might be between $\$2,000$ and $\$8,000$ based on experience and company differences.

Data Transformations

• Data transformations occur when the same operation is applied to every score in a dataset.

Adding/Subtracting a Constant

• Adding a constant to every score does not change the variance or the standard deviation.
• Reasoning: Adding a constant shifts the entire distribution (and the mean) on the number line, but the distances between the scores (the "spread") remain identical. The shape does not change.

Multiplying/Dividing by a Constant

• Multiplying every score by a constant does change variability.
• Standard Deviation: Is multiplied by the same constant as the scores.
• Variance: Is multiplied by the square of the constant.
• Example: If scores are multiplied by $5$:   - The new standard deviation = $\text{Original } s \times 5$.   - The new variance = $\text{Original } s^2 \times 25$.
• Reasoning: Multiplication stretches the distribution. Scores that were $1$ unit apart become $5$ units apart, increasing the spread dramatically.

Requirements for Calculation

• Variance and standard deviation can only be calculated for Interval and Ratio variables.
• They cannot be found for Nominal or Ordinal variables because these measures are built upon the mean, and the mean requires numerical values representing magnitude and equal intervals.

Questions & Discussion

• Q: Why is the range usually a bad measure?
• A: It only uses two scores ($X_{max}$ and $X_{min}$) and ignores the rest. It is also completely compromised by single outliers (unrepresentative scores).
• Q: Is standard deviation squared ever removed from the symbol in algebra?
• A: In algebra, one is trained to isolate the variable, but in statistics, $\sigma^2$ is the permanent name for variance. The square indicates its relationship to standard deviation; it is not a part of the calculation to be "removed" through algebra unless one specifically seeks the standard deviation.
• Q: Does adding 5 points to every test score affect how spread out the grades are?
• A: No. While the average grade will increase by 5 points (the mean shifts), the variability (distances between scores) remains the same. If there was a 30-point gap between the A and the F before the curve, there is still a 30-point gap after the curve.
• Q: What is the significance of standard deviation being the "average deviation"?
• A: It provides a gut-check. If you calculate an SD of $40$ for a test out of $100$ points with a mean of $75$, the number is likely too high, as it implies the average person was $40$ points away from the mean.