Descriptive Statistics: Outliers and Boxplots

Learning Objectives & Overview of Descriptive Statistics

• Descriptive statistics involves summarizing and describing quantitative data using four central concepts:

  • Center: Describes the middle or most typical value of a dataset.

  • Variation: Describes the spread or extent to which data points vary.

  • Distribution: Describes the shape and pattern of data values.

  • Outliers: Identifies highly unusual observations that lie far outside the range of typical data.

• Quantitative data can be comprehensively summarized and visualized using quartiles, fences, and boxplots.

The 5-Number Summary

• Definition: The 5-number summary of a quantitative data set provides a concise description of its spread and center. It consists of:

  • Minimum value ($\text{Min}$)

  • First quartile ($Q_1$)

  • Second quartile or Median ($Q_2 = M$

  • Third quartile ($Q_3$)

  • Maximum value ($\text{Max}$)

• Notation: The 5-number summary is written in sequential order as: $\text{Min}, Q_1, \text{Median} = Q_2, Q_3, \text{Max}$.

• Example 1: Finding the 5-number Summary for Quiz Scores

  • Dataset ($n = 11$): $8, 5, 7, 5, 9, 4, 9, 4, 10, 3, 9$

  • Step 1: Arrange data in ascending order:     $3, 4, 4, 5, 5, 7, 8, 9, 9, 9, 10$

  • Step 2: Identify summary statistics:

  • Minimum: $\text{Min} = 3$

  • First quartile: $Q_1 = 4$

  • Second quartile (Median): $Q_2 = 7$

  • Third quartile: $Q_3 = 9$

  • Maximum: $\text{Max} = 10$

  • 5-number summary: $3, 4, 7, 9, 10$

  • StatCrunch Procedure: Select Stat $\rightarrow$ Summary Stats $\rightarrow$ Columns, choose the column variable, and click Compute!.

• Example 2: Finding the 5-number Summary from Poverty Rate Data

  • Dataset ($n = 51$): Average poverty rates for all 50 US states and the District of Columbia (2016–2018).

  • Complete Data Values:

  • Alabama: $16.0$

  • Alaska: $12.2$

  • Arizona: $14.4$

  • Arkansas: $15.5$

  • California: $12.5$

  • Colorado: $8.9$

  • Connecticut: $9.9$

  • Delaware: $9.6$

  • District of Columbia: $14.9$

  • Florida: $13.6$

  • Georgia: $14.7$

  • Hawaii: $9.5$

  • Idaho: $11.3$

  • Illinois: $10.9$

  • Indiana: $11.8$

  • Iowa: $9.2$

  • Kansas: $10.8$

  • Kentucky: $15.3$

  • Louisiana: $19.8$

  • Maine: $12.6$

  • Maryland: $7.1$

  • Massachusetts: $9.8$

  • Michigan: $11.0$

  • Minnesota: $8.7$

  • Mississippi: $19.8$

  • Missouri: $11.9$

  • Montana: $10.4$

  • Nebraska: $10.4$

  • Nevada: $12.2$

  • New Hampshire: $6.4$

  • New Jersey: $9.1$

  • New Mexico: $18.7$

  • New York: $11.8$

  • North Carolina: $14.1$

  • North Dakota: $10.9$

  • Ohio: $12.8$

  • Oklahoma: $14.0$

  • Oregon: $10.6$

  • Pennsylvania: $11.5$

  • Rhode Island: $10.3$

  • South Carolina: $14.1$

  • South Dakota: $11.8$

  • Tennessee: $12.6$

  • Texas: $13.7$

  • Utah: $7.9$

  • Vermont: $9.6$

  • Virginia: $10.7$

  • Washington: $10.3$

  • West Virginia: $17.2$

  • Wisconsin: $9.5$

  • Wyoming: $11.4$

  • Summary Statistics Output:

  • $\text{Min} = 6.4$

  • $Q_1 = 9.9$

  • $Q_2 = 11.5$

  • $Q_3 = 14$

  • $\text{Max} = 19.8$

  • Practical Interpretation:

  • Approximately $25\%$ of states have average poverty rates between $6.4\%$ and $9.9\%$.

  • Approximately $25\%$ of states have average poverty rates between $9.9\%$ and $11.5\%$.

  • Approximately $25\%$ of states have average poverty rates between $11.5\%$ and $14\%$.

  • Approximately $25\%$ of states have average poverty rates between $14\%$ and $19.8\%$.

Boxplots without Outliers

• Definition: A boxplot (also referred to as a "whisker plot") is a visual diagram constructed from the 5-number summary. It displays the quartiles, interquartile range ($IQR$), and total range of a dataset.

• Anatomical Components of a Standard Boxplot:

  • Middle Box: Represents the middle $50\%$ of the dataset. The left edge is drawn at $Q_1$, and the right edge is drawn at $Q_3$. The width of the box corresponds exactly to the Interquartile Range ($IQR = Q_3 - Q_1$).

  • Median Line: A vertical line segment drawn inside the box at $Q_2$

  • Whiskers: Horizontal line segments extending from the box edges:

  • Left whisker: Extends from $\text{Min}$ to $Q_1$.

  • Right whisker: Extends from $Q_3$ to $\text{Max}$.

  • Percentile Breakdown: Each section (left whisker, left box side, right box side, right whisker) represents roughly $25\%$ of the data observations.

• Step-by-Step Procedure to Construct a Boxplot by Hand:

  • Step 1: Draw a horizontal axis scaled with equal spacing long enough to contain the minimum and maximum data values. Draw vertical line segments above the axis at $Q_1$, $Q_2$, and $Q_3$.

  • Step 2: Draw horizontal line segments connecting the tops and bottoms of the vertical segments at $Q_1$ and $Q_3$ to form the middle box.

  • Step 3: Place a point/dot at the minimum value and at the maximum value.

  • Step 4: Draw whisker line segments from $Q_1$ to $\text{Min}$ and from $Q_3$ to $\text{Max}$.

• Critical Rule for Axis Scaling:

  • When drawing a boxplot, tick marks on the horizontal axis must be labeled using equally spaced increments. Incorrect or non-uniform spacing results in a visually distorted and misleading boxplot.

• Example 3: Hand Construction using Poverty Rate Data

  • Given 5-number summary: $\text{Min} = 6.4$, $Q_1 = 9.9$, $Q_2 = 11.5$, $Q_3 = 14$, $\text{Max} = 19.8$

  • Axis marked evenly from $4$ to $20$

  • Vertical lines at $9.9$, $11.5$, and $14.0$

  • Box formed between $9.9$ and $14.0$

  • Dots at $6.4$ and $19.8$

  • Whiskers extending from $9.9$ down to $6.4$ and from $14.0$ up to $19.8$

• StatCrunch Procedure for Making Boxplots:

  1. Input data into a column in StatCrunch.

  2. Select Graph $\rightarrow$ Boxplot.

  3. Select the desired column variable name.

  4. Check the option Draw boxes horizontally.

  5. Click Compute!.

Identifying Outliers using Fences

• Definition of Outlier: An outlier is an observation that lies far outside the normal range of values for a data set.

• Mathematical Definition of Fences:

  • Fences serve as statistical boundaries to identify potential outliers.

• Left Fence (Lower Fence): $LF = Q_1 - 1.5 \times IQR$

• Right Fence (Upper Fence): $RF = Q_3 + 1.5 \times IQR$

• Interquartile Range formula: $IQR = Q_3 - Q_1$

  • Outlier Criterion:

• Observations falling between the fences ($LF \le x \le RF$) are considered typical data values.

• Observations falling outside the fences ($x < LF$ or $x > RF$) are defined as outliers.

• Procedure to Identify Outliers in a Dataset:

  1. Determine quartiles $Q_1$, $Q_2$ (Median), and $Q_3$

  2. Compute the Interquartile Range: $IQR = Q_3 - Q_1$

  3. Calculate the boundaries:      $LF = Q_1 - 1.5 \times IQR$      $RF = Q_3 + 1.5 \times IQR$

  4. Compare all data values to $LF$ and $RF$. Any value strictly less than $LF$ or strictly greater than $RF$ is classified as an outlier.

• Order of Operations Tip:

  • When calculating fences, always execute multiplication before addition or subtraction:

  • Multiply $1.5 \times IQR$ first.

  • Subtract that product from $Q_1$ for $\text{LF}$.

  • Add that product to $Q_3$ for $\text{RF}$.

• Example 5: Checking Poverty Rates for Outliers

  • Summary Stats: $n = 51$, $Q_1 = 9.9$, $Q_3 = 14.0$, $\text{Min} = 6.4$, $\text{Max} = 19.8$

  • Calculations:     $IQR = 14.0 - 9.9 = 4.1$     $LF = 9.9 - (1.5 \times 4.1) = 9.9 - 6.15 = 3.75$     $RF = 14.0 + (1.5 \times 4.1) = 14.0 + 6.15 = 20.15$

  • Comparison:

  • Lowest data value is $6.4 \ge 3.75$

  • Highest data value is $19.8 \le 20.15$

  • Conclusion: All values fall strictly between $3.75$ and $20.15$. Therefore, there are no outliers in the poverty rate data set.

• Example 6: Checking Presidential Inauguration Ages for Outliers

  • Dataset ($n = 46$): Ages of US Presidents at inauguration.

  • Data Values: $57, 68, 56, 54, 60, 64, 61, 51, 46, 42, 61, 46, 57, 49, 54, 51, 43, 54, 57, 64, 49, 56, 55, 47, 58, 50, 51, 55, 56, 70, 57, 48, 47, 51, 61, 78, 61, 65, 55, 54, 52, 54, 52, 55, 51, 69$

  • Summary Output:

  • $n = 46$

  • $\text{Mean} = 55.478261$

  • $\text{Variance} = 53.766184$

  • $\text{Std. dev.} = 7.3325428$

  • $\text{Std. err.} = 1.0811245$

  • $\text{Median} = 55$

  • $\text{Range} = 36$

  • $\text{Min} = 42$

  • $\text{Max} = 78$

  • $Q_1 = 51$

  • $Q_3 = 60$

  • Outlier Calculations:     $IQR = Q_3 - Q_1 = 60 - 51 = 9$     $LF = 51 - (1.5 \times 9) = 51 - 13.5 = 37.5$     $RF = 60 + (1.5 \times 9) = 60 + 13.5 = 73.5$

  • Evaluation:

  • $\text{Min} = 42 \ge 37.5$

  • $\text{Max} = 78 > 73.5$

  • Conclusion: The age $78$ is greater than the upper fence ($73.5$). Thus, $78$ is identified as an outlier.

Modified Boxplots (Boxplots with Outliers)

• Definition: A modified boxplot explicitly denotes outliers rather than extending whiskers out to the extreme values.

• Key Visual Indicator: A dot or asterisk displayed on a modified boxplot represents an individual outlier.

• Procedure to Create a Modified Boxplot:

  1. Find quartiles $Q_1$, $Q_2$ (Median), and $Q_3$

  2. Compute $IQR = Q_3 - Q_1$

  3. Compute fences: $LF = Q_1 - 1.5 \times IQR$ and $RF = Q_3 + 1.5 \times IQR$

  4. Scale a horizontal number line to encompass all dataset values (from minimum to maximum).

  5. Draw vertical lines at $Q_1$, $Q_2$, and $Q_3$ and construct the middle box.

  6. Identify fence positions internally.

  7. Draw the left whisker from $Q_1$ down to the smallest data value that lies inside the fences.

  8. Draw the right whisker from $Q_3$ up to the largest data value that lies inside the fences.

  9. Mark any data values outside the fences individually using a dot or asterisk.

• Application to Presidential Age Data:

  • $Q_1 = 51$, $Q_2 = 55$, $Q_3 = 60$

  • $LF = 37.5$, $RF = 73.5$

  • Smallest value within fences: $42$ (left whisker extends from $51$ down to $42$

  • Largest value within fences: $70$ (right whisker extends from $60$ up to $70$

  • Value outside fences: $78$ (plotted as an isolated dot at $78$

Comparative Boxplot Analysis

• Comparative boxplots display two or more boxplots on the same scaled axis to allow direct comparison between groups.

• Example 7: Comparing Statistics Exam Grades (10 AM vs. 11 AM Sections)

  • Background: A professor taught two back-to-back statistics classes (10 AM and 11 AM) and was concerned that 10 AM students shared exam answers with 11 AM students.

  • Questions & Answers:

  • Question a: Which class had a higher median grade?

    • Answer: The 10 AM class. The median line for the 10 AM section is near $80$, whereas the median for the 11 AM section is below $70$

  • Question b: Which class had a larger $IQR$?

    • Answer: The 10 AM class. Interquartile range corresponds to box width. The 10 AM box is slightly wider than the 11 AM box, indicating that the middle $50\%$ of scores were spread further apart in the 10 AM class.

  • Question c: Which class had more scores higher than $70$?

    • Answer: The 10 AM class. Over $75\%$ of 10 AM students scored above $70$ ($Q_1 > 70$), while fewer than $50\%$ of 11 AM students scored above $70$ (Median $< 70$

  • Question d: Which class had an outlier?

    • Answer: The 11 AM class had an outlier, shown as a dot near $31$ points.

  • Question e: Based on the boxplots, was the professor's concern justified?

    • Answer: No. The 10 AM class performed significantly better overall. If answer leakages had occurred, the 11 AM class would be expected to score higher, which was not the case.

Boxplots and Distribution Shapes

• Boxplots reflect the shape of the underlying distribution:

• Shape Classifications:

  • Symmetric / Bell-Shaped:

  • The middle box is centered between the whiskers.

  • The left and right whiskers are approximately equal in length.

  • The median line divides the box into two roughly equal halves.

  • Skewed Left:

  • The left whisker is significantly longer than the right whisker.

  • The middle box is shifted toward the right side of the diagram.

  • The longer left whisker corresponds to the long left tail seen in a histogram.

  • Skewed Right:

  • The right whisker is significantly longer than the left whisker.

  • The middle box is shifted toward the left side of the diagram.

  • The longer right whisker corresponds to the long right tail seen in a histogram.

Comprehensive Analysis Framework: Putting It All Together

• Complete Data Description Requirements: Describing a quantitative dataset requires addressing four distinct factors:

  1. Center (Median or Mean)

  2. Variation ($IQR$, Range, or Standard Deviation)

  3. Shape / Distribution (Symmetric, Skewed Left, Skewed Right)

  4. Outliers (Presence or absence of extreme values beyond fences)

• Example 8: Comparing Rotten Tomatoes Movie Ratings (Marvel vs. DC through 2019)

  • Center Analysis: Marvel has a significantly higher median rating (near $80$) than DC (near $52$). Marvel's middle box is positioned farther to the right.

  • Variation Analysis: Both companies display roughly equal box widths ($IQR$). Marvel has a slightly larger total range. Overall variability is approximately equivalent.

  • Shape Analysis:

  • Marvel: Skewed left due to a longer left whisker and a box shifted toward higher values.

  • DC: Symmetric / approximately bell-shaped because the left and right whiskers are nearly equal in length and the box is centered.

  • Outlier Analysis: No dots or asterisks appear on either boxplot; therefore, neither company has outliers.

  • Overall Summary: Marvel's median rating roughly equals DC's third quartile ($Q_3$

  • $50\%$ of Marvel movies received scores of $80$ or higher.

  • Only $25\%$ of DC movies received scores of $80$ or higher.

  • Marvel movies demonstrate consistently higher critical acclaim while exhibiting similar variation to DC movies.