Comprehensive Study Notes: Skewed Distributions, Quartiles, and Summary Statistics

Measures of Center and Spread for Symmetric and Skewed Distributions

Symmetric Distributions:
- Center: Described using the mean.
- Spread: Described using the standard deviation.
- Data Distribution: The empirical rule is applied to determine where the majority of the dataset exists.
Skewed Distributions:
- Center: Described using the median.
- Spread: Described using the Interquartile Range ( $\text{IQR}$ ).
- Data Distribution: The empirical rule, mean, and standard deviation are not appropriate or useful for skewed distributions or skewed histograms.
Determining Distribution Shape by Comparing Mean and Median:
- Rather than relying strictly on visual estimation of histograms, compare the numerical values of the mean and median.
- Perfectly Symmetric: The mean is exactly equal to the median ( $\text{Mean} = \text{Median}$ ).
- Basically Symmetric: The mean is almost equal to the median ( $\text{Mean} \approx \text{Median}$ ).
- Skewed: The mean is several units away from the median ( $\text{Mean} \neq \text{Median}$ ).
- Skewed Left: The mean is significantly smaller than the median ( $\text{Mean} < \text{Median}$ ).
- Skewed Right: The mean is significantly larger than the median ( $\text{Mean} > \text{Median}$ ).
Numerical Example of Skewness:
- A histogram dataset has a mean of $250$ and a median of $226$ .
- Because $250 \neq 226$ , the distribution is not symmetric.
- Because the mean ( $250$ ) is larger than the median ( $226$ ), the distribution is skewed right, meaning data points are more spread out to the right of the tallest peak than to the left.

Questions & Discussion: Z-Scores for US Birth Weights

Question: When evaluating a birth weight measurement relative to all US births within a symmetric distribution, which mean and standard deviation parameters should be selected?
Response:
- Mean: Use the overall US birth mean of $3562$ \, $\text{g}$ (or relative unit), rather than the mean specifically for infants born one month early.
- Standard Deviation: Use the population standard deviation of $500$ .
- Z-Score Application: The z-score formula calculates how many standard deviations a specific data point is away from the mean, allowing one to classify the observation as usual or unusual.

Calculating the Median and Summary Statistics

Procedure for Finding the Median by Hand:
1. Arrange the data values in order from smallest to largest.
2. Identify the direct middle of the ordered list.
3. If the dataset contains an odd number of observations ( $n$ ), the median is the single value located exactly in the center at position $\frac{n+1}{2}$ .
4. If the dataset contains an even number of observations ( $n$ ), the median is the arithmetic average of the two middle numbers.
Resistance Property:
- The median is resistant to extreme values and outliers, whereas the mean is sensitive to outliers.
Class A Dataset Analysis:
- Dataset size: $n = 9$ data points.
- Median: The $5\text{th}$ ordered data point, which equals $77$ .
- Mean: Approximately $79$ .
- Comparison: The mean ( $\approx 79$ ) and median ( $77$ ) are not equal, indicating the distribution is skewed.
Class B Dataset Analysis:
- Dataset size: $n = 10$ data points.
- Median: The average of the two middle observations ( $5\text{th}$ and $6\text{th}$ values), which equals $83.5$
- Mean: $81.4$ .
- Comparison: The mean ( $81.4$ ) and median ( $83.5$ ) are not equal, indicating the distribution is skewed.
StatCrunch Software Workflow for Summary Statistics:
1. Open a blank spreadsheet or table.
2. Enter raw data into labeled columns (e.g., Class A, Class B). Data does not need to be sorted manually; the software sorts automatically.
3. Navigate to Summary Stats $\rightarrow$ Columns.
4. Hold the Ctrl key on the keyboard to select multiple columns simultaneously.
5. Click Compute to generate summary statistics (mean, median, range, quartiles).

Assessing Skewness and Range in Datasets

Evaluating Degree of Skewness:
- When comparing two distributions, evaluate which is more dramatically skewed or whether one is sufficiently close to symmetric to permit the use of symmetric rules.
New York Income Case Example:
- Histograms representing income in New York (scaled in thousands of dollars per year) show dramatic right-skewness.
- The mean and median lie in significantly different positions on the histogram.
- Because the mean and median are vastly different, the visual skewness is confirmed numerically. The median must be used to represent a typical income value.
The Range:
- Definition: The total distance spanned by the dataset.
- Formula: $\text{Range} = \text{Maximum Value} - \text{Minimum Value}$
- Class A Range Calculation: For Class A data with a maximum value of $98$ and a minimum value of $56$ : $\text{Range} = 98 - 56 = 42$
- Properties: Range defines a window $42$ units wide containing all data points. Range does not indicate whether a distribution is symmetric or skewed and is severely affected by outliers.

Quartiles and Interquartile Range (IQR)

Quartiles:
- Quartiles partition an ordered dataset into four equal segments, each containing $25\%$ of the data.
- Median ( $Q_2$ ): Splits the total dataset in half ( $50\%$ below, $50\%$ above).
- First Quartile ( $Q_1$ ): The median of the lower half of the dataset (left side of the overall median).
- Third Quartile ( $Q_3$ ): The median of the upper half of the dataset (right side of the overall median).
Manual Quartile Step-by-Step Calculation (Class A Example):
- Class A has $n = 9$ data values with overall median $77$ .
- Finding $Q_1$ :
  - Look at the $4$ data points to the left of $77$ .
  - Average the middle two values of these $4$ observations to get Q_1 = 71$.\n * 71 $splits the lower half:$ 2 $data points sit below$ 71 $, and$ 2 $data points sit between$ 71 $and$ 77$.
- Finding $Q_3$ :
  - Look at the $4$ data points to the right of $77$ (values: $81$ , $87$ , $95$ , $98$ ).
  - Average the middle two values ( $87$ and $95$ ) to get Q_3 = 91$.\n * 91 $splits the upper half:$ 2 $data points sit between$ 77 $and$ 91 $, and$ 2 $data points sit above$ 91$.
- Central Spread: Between $Q_1 = 71$ and $Q_3 = 91$ lies approximately $50\%$ (or slightly more for odd $n$ ) of the dataset. Values falling between $Q_1$ and $Q_3$ represent usual or typical values.
Class B Quartiles and IQR:
- $Q_1 = 75$
- $Q_3 = 86$
- Interquartile Range Formula: $\text{IQR} = Q_3 - Q_1$
- Class B IQR Calculation: $\text{IQR} = 86 - 75 = 11$
Generating IQR in StatCrunch:
1. Navigate to Summary Stats $\rightarrow$ Columns.
2. Select the target column.
3. Under the Statistics selection box, scroll down and click IQR.
4. Hold the Ctrl key while clicking to keep default summary statistics selected alongside IQR.
5. Click Compute.

Comparing Center Spread: Empirical Rule versus IQR

Graphical Representation of Quartile Spacing:
- Symmetric Distributions: Standard deviation intervals are evenly spaced on both sides of the mean.
- Skewed Distributions: Distance between quartiles is uneven.
  - For example, the distance from the median to $Q_1$ may be significantly narrower than the distance from the median to $Q_3$ .
- Quartile Proportions: Regardless of symmetry or skewness, the data is strictly divided into fixed $25\%$ intervals:
  - $25\%$ of data values $\le Q_1$
  - $25\%$ of data values between $Q_1$ and Median
  - $25\%$ of data values between Median and $Q_3$
  - $25\%$ of data values $\ge Q_3$
Comprehensive Sample Dataset Comparison:
- Dataset size: $n = 8$ observations.
- Calculated summary metrics:
  - $\text{Mean} \neq \text{Median}$
  - $\text{Median} = 53.75$
  - $Q_1 = 50.5$
  - $Q_3 = 61$
  - $\text{IQR} = 61 - 50.5 = 10.5$
  - $\text{Standard Deviation} = 7.78$
- Methodology Comparison:
  - Empirical Rule Approach (Incorrect for skewed data): Uses standard deviation ( $7.78$ ) to establish a spread window of roughly $16$ total units ( $\approx 8$ units above the mean and $\approx 8$ units below the mean).
  - IQR Approach (Correct for skewed data): Uses central interquartile range ( $Q_3 - Q_1$ ) to establish a central spread window of $10.5$ units around the median.