Comprehensive Guide to Descriptive Statistics: Deciles, Percentiles, Interquartile Range, Box Plots, and Outlier Analysis

Measures of Position: Deciles and Percentiles

• Deciles ($D_k$):

  • Deciles divide an ordered dataset into ten equal parts.
  • Position formula for deciles:     $D_k = \frac{k \times N}{10}$     where $k$ represents the decile number ($1, 2, \dots, 9$) and $N$ represents the total number of items in the dataset.

• Percentiles ($P_k$):

  • Percentiles divide an ordered dataset into one hundred equal parts.
  • Position formula for percentiles:     $P_k = \frac{k \times N}{100}$     where $k$ represents the percentile rank ($1, 2, \dots, 99$) and $N$ represents the total number of items in the dataset.

• Example Problem: Grade 10 Math Quiz Scores:

  • Dataset: Scores of $12$ Grade 10 students in a $15$-item Math quiz:     $13, 20, 16, 14, 19, 15, 19, 11, 10, 18, 9$
  • Step 1: Order the Dataset ($N = 12$):
  • $1\text{st}$ position: $9$
  • $2\text{nd}$ position: $10$
  • $3\text{rd}$ position: $11$
  • $4\text{th}$ position: $13$
  • $5\text{th}$ position: $14$
  • $6\text{th}$ position: $15$
  • $7\text{th}$ position: $15$
  • $8\text{th}$ position: $16$
  • $9\text{th}$ position: $18$
  • $10\text{th}$ position: $19$
  • $11\text{th}$ position: $19$
  • $12\text{th}$ position: $20$
  • Part A: Calculating Deciles ($D_1$, $D_4$, and $D_9$):
  • Calculating $D_1$:       $D_1 = \frac{1 \times N}{10}$$D_1 = \frac{1 \times 12}{10} = \frac{12}{10} = 1.2 \rightarrow 2\text{nd}\text{ position}$$D_1 = 10$
  • Calculating $D_4$:       $D_4 = \frac{4 \times N}{10}$$D_4 = \frac{4 \times 12}{10} = \frac{48}{10} = 4.8 \rightarrow 5\text{th}\text{ position}$$D_4 = 14$
  • Calculating $D_9$:       $D_9 = \frac{9 \times N}{10}$$D_9 = \frac{9 \times 12}{10} = \frac{108}{10} = 10.8 \rightarrow 11\text{th}\text{ position}$$D_9 = 19$
  • Part B: Proving $D_5 = Q_2$:
  • Position of $D_5$:       $D_5 = \frac{5 \times N}{10}$$D_5 = \frac{5 \times 12}{10} = \frac{60}{10} = 6\text{th}\text{ position}$
  • Value of $D_5$ (averaging $6\text{th}$ and $7\text{th}$ positions):       $D_5 = \frac{6\text{th} + 7\text{th}}{2}$$D_5 = \frac{15 + 15}{2} = \frac{30}{2} = 15$
  • Value of $Q_2$ (Second Quartile / Median):       $Q_2 = \frac{15 + 15}{2} = \frac{30}{2} = 15$
  • Equivalence:       $D_5 = Q_2$
  • Part C: Calculating Percentiles ($P_{43}$ and $P_{60}$):
  • Calculating $P_{43}$:       $P_{43} = \frac{43 \times N}{100}$$P_{43} = \frac{43 \times 12}{100} = \frac{516}{100} = 5.16 \rightarrow 6\text{th}\text{ position} \quad (\text{round up})$$P_{43} = 15$
  • Calculating $P_{60}$:       $P_{60} = \frac{60 \times N}{100}$$P_{60} = \frac{60 \times 12}{100} = \frac{720}{100} = 7.20 \rightarrow 8\text{th}\text{ position} \quad (\text{round up})$$P_{60} = 16$
  • Part D: Proving $Q_3 = P_{75}$:
  • Position of $P_{75}$:       $P_{75} = \frac{75 \times N}{100}$$P_{75} = \frac{75 \times 12}{100} = \frac{900}{100} = 9 \rightarrow 9\text{th}\text{ and } 10\text{th}\text{ positions}$
  • Value of $P_{75}$:       $P_{75} = \frac{18 + 19}{2} = \frac{37}{2} = 18.5$
  • Calculating $Q_3$ from upper half ($15, 16, 18, 19, 19, 20$):       $Q_3 = \frac{18 + 19}{2} = \frac{37}{2} = 18.5$
  • Equivalence:       $Q_3 = P_{75}$

Interquartile Range (IQR) and Spread

• Interquartile Range (IQR) Definition:

  • The Interquartile Range measures the spread or dispersion of the middle $50\%$ of a dataset.
  • It quantifies the distance between the upper quartile ($Q_3$) and lower quartile ($Q_1$).

• Formula:   $\text{IQR} = Q_3 - Q_1$

• Example Problem: Finding IQR:

  • Dataset: $8, 10, 12, 14, 16, 18, 20$
  • Step 1: Arrange data in ascending order (already arranged):     $8, 10, 12, 14, 16, 18, 20$
  • Step 2: Identify $Q_1$ and $Q_3$:
  • $Q_1 = 10$
  • $Q_2 = 14$
  • $Q_3 = 18$
  • Step 3: Apply IQR formula:     $\text{IQR} = Q_3 - Q_1$$\text{IQR} = 18 - 10$$\text{IQR} = 8$
  • Multiple choice question evaluation:
  • Options: A. $4$, B. $6$, C. $8$, D. $10$
  • Correct Answer: C. $8$

Graphical Interpretation: Box-and-Whisker Plot

• Definition:

  • A Box-and-Whisker Plot is a statistical graph displaying data distribution by showing spread, median, quartiles, and potential outliers.

• Structural Components:

  • Box:
  • Represents the interquartile range (IQR).
  • Spans the middle $50\%$ of data between $Q_1$ ($P_{25}$) and $Q_3$ ($P_{75}$).
  • Median Line:
  • A line drawn inside the box indicating the dataset's median ($50\text{th}$ percentile).
  • Whiskers:
  • Lines extending outward from the box to the minimum and maximum values located within $1.5 \times \text{IQR}$ from $Q_1$ and $Q_3$
  • Outliers:
  • Individual points plotted distinctly beyond the whiskers, representing unusually high or low values.

• Example Problem: Assessing Outliers with Box Plot Boundaries:

  • Dataset of exam scores: $\{55, 60, 65, 70, 75, 80, 85, 90, 100\}$
  • Step 1: Determine $Q_1$, Median, and $Q_3$:     $Q_1 = \frac{60 + 65}{2} = 62.5$$\text{Median} = 75$$Q_3 = \frac{85 + 90}{2} = 87.5$
  • Step 2: Compute IQR:     $\text{IQR} = Q_3 - Q_1$$\text{IQR} = 87.5 - 62.5 = 25$
  • Step 3: Compute Whisker Limits:
  • Limit below $Q_1$:       $Q_1 - 1.5 \times \text{IQR}$$62.5 - (1.5 \times 25) = 62.5 - 37.5 = 25$
  • Limit above $Q_3$:       $Q_3 + 1.5 \times \text{IQR}$$87.5 + (1.5 \times 25) = 87.5 + 37.5 = 125$
  • Step 4: Evaluate for Outliers:
  • Dataset minimum value = $55$
  • Dataset maximum value = $100$
  • Since all scores fall strictly within the limits $25$ and $125$, there are no outliers present in this dataset.

Outlier Detection and Analysis

• Definition of Outliers:

  • An outlier is a value that is significantly higher or lower than the rest of the dataset.

• Illustrative Examples:

  • In an exam dataset with typical scores ranging from $60$ to $90$, a score of $12$ or a score of $99$ would be considered an outlier.

• Statistical Impact of Outliers:

  • Strongly skews and affects the mean.
  • Inflates standard deviation.
  • Biases linear regression models.
  • The median is resistant to outliers compared to the mean.

• IQR Method Formulas for Outlier Detection:

  • Lower bound threshold:     $\text{Below } Q_1 = Q_1 - 1.5 \times \text{IQR}$
  • Upper bound threshold:     $\text{Above } Q_3 = Q_3 + 1.5 \times \text{IQR}$
  • Rule: Any data value falling outside these lower and upper bounds is classified as an outlier.

• Practice Question on Outlier Criteria:

  • Question: Outliers are values that fall:
  • A. Beyond $1.5 \times \text{IQR}$ from $Q_1$ or $Q_3$
  • B. Within $Q_1$ and $Q_3$
  • C. Equal to median
  • D. Equal to mean
  • Answer: A. Beyond $1.5 \times \text{IQR}$ from $Q_1$ or $Q_3$