Comprehensive Study Guide: Dispersion, Histograms, Skewness, Kurtosis, and Quantiles

Measures of Dispersion & Variability

  • Review of Foundational Summary Measures:

    • Range: Difference between the largest and smallest values in a dataset.
    • Interquartile Range (IQRIQR): Difference between the 75th75\text{th} percentile (Q3Q_3) and 25th25\text{th} percentile (Q1Q_1):     IQR=Q3−Q1IQR = Q_3 - Q_1
    • Variance (s2s^2 or σ2\sigma^2): Sum of squared deviations divided by population size (NN) or sample size minus one (n−1n-1):     s2=∑i=1n(xi−xˉ)2n−1s^2 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}
    • Standard Deviation (ss or σ\sigma): Square root of the variance:     s=∑i=1n(xi−xˉ)2n−1s = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}}
    • Z-scores: Number of standard deviations an observation lies away from the mean:     z=x−xˉsz = \frac{x - \bar{x}}{s}
  • Coefficient of Variation (CV):

    • Definition: Relative measure of dispersion that quantifies the spread of data as a proportion or percentage of its mean.
    • Formulas:     CV=sxˉCV = \frac{s}{\bar{x}}CV=sxˉ×100%CV = \frac{s}{\bar{x}} \times 100\%
    • Properties:
    • CVCV is a dimensionless (unitless) number.
    • It enables comparison of variation across populations with different units or vastly different means.
    • Applications:
    • Characterizing spatial or temporal variability of geographic, climatic, or biophysical variables (e.g., precipitation, temperature, leaf area index [LAI], biomass, NDVI).
    • Xiao & Moody (2004) analyzed 1990–2000 data relating the CVCV of NDVI to the CVCV of temperature and CVCV of precipitation.
    • Comparative Case Study (Chapel Hill vs. Bend):     Comparison of Mean, Standard Deviation, and Coefficient of Variation for Chapel Hill and Bend
    • Chapel Hill (Location A):
      • Mean (xˉ\bar{x}): 1198.101198.10
      • Standard Deviation (ss): 191.80191.80
      • Coefficient of Variation (CVCV): 0.160.16 (16%16\%)
    • Bend (Location B):
      • Mean (xˉ\bar{x}): 298.07298.07
      • Standard Deviation (ss): 82.0882.08
      • Coefficient of Variation (CVCV): 0.280.28 (28%28\%)
    • Key Insight: Although Chapel Hill has a higher absolute standard deviation (191.80>82.08191.80 > 82.08), Bend exhibits significantly greater relative variability relative to its mean (28%>16%28\% > 16\%).

Frequency Distributions and Histograms

  • Core Concepts:

    • Frequency: Number of times a variable assumes a specific value or falls into a defined range.
    • Frequency Distribution: Complete representation of all possible variable values and their corresponding frequencies (e.g., rolling a pair of dice multiple times yields values constrained between 22 and 1212 with specific occurrence counts).
    • Absolute Frequency: Actual raw count of occurrences.
    • Relative Frequency: Normalized frequencies obtained by dividing absolute frequency by total observations (nn), yielding values in [0,1][0, 1] or percentages. Relative frequencies permit direct comparison of distributions drawn from sources with different sample sizes.
  • Histograms:

    • Definition: Vertical bar graph used to graphically summarize the distribution of a continuous dataset.
    • Structure: Range of data values is divided into intervals (classes). A bar is placed over each interval with height equal to class frequency or relative frequency.
    • Midpoints: Commonly marked on the horizontal axis.
  • Step-by-Step Procedure for Constructing a Histogram:

    1. Ungrouped Frequency Table: Count occurrences of each individual value from lowest to highest.
    • Example (Topographic Moisture Index - TMI):
      • TMI Value 4.164.16: Ungrouped Frequency = 22
      • TMI Value 4.174.17: Ungrouped Frequency = 44
      • TMI Value 4.184.18: Ungrouped Frequency = 00
      • …
      • TMI Value 13.7113.71: Ungrouped Frequency = 11
    1. Grouped Frequency Table: Group observations into a manageable number of discrete class intervals.
    • Example Class Intervals and Frequencies:
      • Class 4.00−4.994.00 - 4.99: Frequency = 120120
      • Class 5.00−5.995.00 - 5.99: Frequency = 807807
      • Class 6.00−6.996.00 - 6.99: Frequency = 14111411
      • Class 7.00−7.997.00 - 7.99: Frequency = 407407
      • Class 8.00−8.998.00 - 8.99: Frequency = 8787
      • Class 9.00−9.999.00 - 9.99: Frequency = 3333
      • Class 10.00−10.9910.00 - 10.99: Frequency = 1717
      • Class 11.00−11.9911.00 - 11.99: Frequency = 2222
      • Class 12.00−12.9912.00 - 12.99: Frequency = 4343
      • Class 13.00−13.9913.00 - 13.99: Frequency = 1919
    1. Plot Frequencies: Render vertical bars for each class interval.      Grouped frequency histogram of Topographic Moisture Index
  • Field Example (Crown Width Histogram):

    • Koukoulas & Blackburn (2005) measured Quercus robur crown width in situ in Frame Wood (n=63n = 63, mean =9.3 m= 9.3\,\text{m}, SD=4.64 mSD = 4.64\,\text{m}).   Histogram of crown width measured in situ for Quercus robur trees

Statistical Moments of a Distribution

  • Measures of central tendency and dispersion quantify location and spread, but do not describe distribution shape. Statistical moments formally describe distribution characteristics:
    • 1st1\text{st} Moment (Mean, xˉ\bar{x} or μ\mu): Measures central value / location.
    • 2nd2\text{nd} Moment (Variance, s2s^2 or σ2\sigma^2): Measures dispersion / scale.
    • 3rd3\text{rd} Moment (Skewness): Measures asymmetry.
    • 4th4\text{th} Moment (Kurtosis): Measures relative peakedness and tail weight.

Skewness

  • Definition: Quantitative measure of symmetry, or lack of symmetry, of a data distribution around its mean.

  • Types of Skewness:

    • Symmetric Distribution:
    • Skewness =0= 0.
    • Left and right sides of the distribution are mirror images around the mean.
    • Mean == Median == Mode.
    • Positive Skewness (Skewed Right):
    • Skewness >0> 0
    • Long right tail relative to left tail.
    • Majority of observations lie below the mean.
    • Mean >> Median.
    • Prevalent in lower-bounded variables (e.g., failure times in reliability studies, where values cannot be negative).
    • Negative Skewness (Skewed Left):
    • Skewness <0< 0
    • Long left tail relative to right tail.
    • Small number of low observations and large cluster of high observations.
    • Median >> Mean.

  Positively and negatively skewed distribution curvesExamples of symmetric, positively skewed, and negatively skewed histograms

  • Formulas for Skewness:

    • Population / Moment-Based Skewness Formula:     skewness=∑i=1n(xi−xˉ)3ns3\text{skewness} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^3}{n s^3}skewness=m3m23/2\text{skewness} = \frac{m_3}{m_2^{3/2}}     where:     m3=∑(x−xˉ)3nm_3 = \frac{\sum (x - \bar{x})^3}{n}m2=∑(x−xˉ)2nm_2 = \frac{\sum (x - \bar{x})^2}{n}
    • Unadjusted Sample Skewness (g1g_1):     g1=m3m23/2g_1 = \frac{m_3}{m_2^{3/2}}
    • Adjusted Sample Skewness (G1G_1, Joanes and Gill Formula):     G1=n(n−1)n−2g1=n(n−1)n−2(m3m23/2)G_1 = \frac{\sqrt{n(n-1)}}{n-2} g_1 = \frac{\sqrt{n(n-1)}}{n-2} \left( \frac{m_3}{m_2^{3/2}} \right)
  • Step-by-Step Worked Calculation for Sample Skewness:

    • Data Table (n=100n = 100):
    • Class Mark x=61x = 61, Frequency f=5f = 5:
      • xf=305x f = 305
      • (x−xˉ)=−6.45(x - \bar{x}) = -6.45
      • (x−xˉ)2f=208.01(x - \bar{x})^2 f = 208.01
      • (x−xˉ)3f=−1341.68(x - \bar{x})^3 f = -1341.68
    • Class Mark x=64x = 64, Frequency f=18f = 18:
      • xf=1152x f = 1152
      • (x−xˉ)=−3.45(x - \bar{x}) = -3.45
      • (x−xˉ)2f=214.25(x - \bar{x})^2 f = 214.25
      • (x−xˉ)3f=−739.15(x - \bar{x})^3 f = -739.15
    • Class Mark x=67x = 67, Frequency f=42f = 42:
      • xf=2814x f = 2814
      • (x−xˉ)=−0.45(x - \bar{x}) = -0.45
      • (x−xˉ)2f=8.51(x - \bar{x})^2 f = 8.51
      • (x−xˉ)3f=−3.83(x - \bar{x})^3 f = -3.83
    • Class Mark x=70x = 70, Frequency f=27f = 27:
      • xf=1890x f = 1890
      • (x−xˉ)=2.55(x - \bar{x}) = 2.55
      • (x−xˉ)2f=175.57(x - \bar{x})^2 f = 175.57
      • (x−xˉ)3f=447.70(x - \bar{x})^3 f = 447.70
    • Class Mark x=73x = 73, Frequency f=8f = 8:
      • xf=584x f = 584
      • (x−xˉ)=5.55(x - \bar{x}) = 5.55
      • (x−xˉ)2f=246.42(x - \bar{x})^2 f = 246.42
      • (x−xˉ)3f=1367.63(x - \bar{x})^3 f = 1367.63
    • Totals & Moments:
    • ∑xf=6745  ⟹  xˉ=6745100=67.45\sum x f = 6745 \implies \bar{x} = \frac{6745}{100} = 67.45
    • ∑(x−xˉ)2f=852.75  ⟹  m2=852.75100=8.5275\sum (x - \bar{x})^2 f = 852.75 \implies m_2 = \frac{852.75}{100} = 8.5275
    • ∑(x−xˉ)3f=−269.33  ⟹  m3=−269.33100=−2.6933\sum (x - \bar{x})^3 f = -269.33 \implies m_3 = \frac{-269.33}{100} = -2.6933
    • Unadjusted Skewness (g1g_1):     g1=−2.6933(8.5275)3/2=−0.1082g_1 = \frac{-2.6933}{(8.5275)^{3/2}} = -0.1082
    • Sample Skewness (G1G_1):     G1=[100×9998][−2.6933(8.5275)3/2]=−0.1098G_1 = \left[ \frac{\sqrt{100 \times 99}}{98} \right] \left[ \frac{-2.6933}{(8.5275)^{3/2}} \right] = -0.1098

Kurtosis

  • Definition: Quantitative measure of whether data are peaked or flat relative to a normal distribution, heavily determined by tail behavior and tail heaviness.

  • Standard Normal Distribution Baseline:

    • Unadjusted Kurtosis =3= 3
    • Excess Kurtosis =0= 0
  • Classifications of Kurtosis:   General forms of kurtosis comparing leptokurtic, mesokurtic, and platykurtic curvesComparison of low kurtosis and high kurtosis probability density functions

    • Mesokurtic:
    • Normal distribution baseline (kurtosis=3\text{kurtosis} = 3, excess kurtosis=0\text{excess kurtosis} = 0).
    • Leptokurtic:
    • Positive excess kurtosis (kurtosis>3\text{kurtosis} > 3, excess kurtosis>0\text{excess kurtosis} > 0).
    • Sharper peak near mean, rapid frequency decline, long/heavy tails.
    • Platykurtic:
    • Negative excess kurtosis (kurtosis<3\text{kurtosis} < 3, excess kurtosis<0\text{excess kurtosis} < 0).
    • Flat top near mean, broader frequency spread, short/light tails (extreme case: uniform distribution).
  • Formulas for Kurtosis:

    • Population Moment Formula:     kurtosis=∑i=1n(xi−xˉ)4ns4−3\text{kurtosis} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^4}{n s^4} - 3
    • Ratio of Moments (a4a_4):     a4=m4m22a_4 = \frac{m_4}{m_2^2}     where:     m4=∑(x−xˉ)4nm_4 = \frac{\sum (x - \bar{x})^4}{n}
    • Unadjusted Excess Kurtosis (g2g_2):     g2=a4−3=m4m22−3g_2 = a_4 - 3 = \frac{m_4}{m_2^2} - 3
    • Adjusted Sample Excess Kurtosis (G2G_2, Joanes and Gill Formula):     G2=n−1(n−2)(n−3)[(n+1)g2+6]G_2 = \frac{n - 1}{(n - 2)(n - 3)} \left[ (n + 1) g_2 + 6 \right]

Graphical Summaries: Histograms vs. Box Plots

  • Summary Capability Comparison:

    • Histogram: Shows center (location), spread (scale), skewness, kurtosis, presence of outliers, and multi-modality.
    • Box Plot: Provides a concise graphical summary of the Five-Number Summary and identifies potential outliers.
  • Five-Number Summary Components:

    1. Minimum (min\text{min})
    2. 25th25\text{th} Percentile (Q1Q_1 / First Quartile)
    3. Median (Q2Q_2 / 50th50\text{th} Percentile / Second Quartile)
    4. 75th75\text{th} Percentile (Q3Q_3 / Third Quartile)
    5. Maximum (max\text{max})

  Annotated box plot illustrating five-number summary elements

  • Worked Examples for Box Plots:
    • Example 1: Commodore Prices (in $000\$000, N=9N = 9):
    • Raw Unsorted Data: 6.0,6.7,3.8,7.0,5.8,9.975,10.5,5.99,20.06.0, 6.7, 3.8, 7.0, 5.8, 9.975, 10.5, 5.99, 20.0
    • Ranked Data: 3.8,5.8,5.99,6.0,6.7,7.0,9.975,10.5,20.03.8, 5.8, 5.99, 6.0, 6.7, 7.0, 9.975, 10.5, 20.0
    • Median (Q2Q_2): 6.76.7 (5th5\text{th} observation, 4 values on each side).
    • 25th25\text{th} Percentile (Q1Q_1): 5.95.9 (median of the 4 smallest values).
    • 75th75\text{th} Percentile (Q3Q_3): 10.210.2 (median of the 4 largest values).
    • Interquartile Range (IQRIQR): IQR=Q3−Q1=10.2−5.9=4.3IQR = Q_3 - Q_1 = 10.2 - 5.9 = 4.3
    • Example 2: Commuting Times (in minutes, N=30N = 30):
    • Ranked Data: 5,5,6,9,10,11,11,12,12,14,16,17,19,21,21,21,21,21,22,23,24,24,26,26,31,31,36,42,44,475, 5, 6, 9, 10, 11, 11, 12, 12, 14, 16, 17, 19, 21, 21, 21, 21, 21, 22, 23, 24, 24, 26, 26, 31, 31, 36, 42, 44, 47
    • 25th25\text{th} Percentile Position: 30+14=7.75  ⟹  11.75\frac{30 + 1}{4} = 7.75 \implies 11.75
    • 75th75\text{th} Percentile Position: 3(30+1)4=23.25  ⟹  26\frac{3(30 + 1)}{4} = 23.25 \implies 26
    • Interquartile Range (IQRIQR): IQR=26−11.75=14.25IQR = 26 - 11.75 = 14.25

Measures of Non-Central Location

  • Quantiles that partition ordered datasets into specific equal parts: Quartiles, Deciles, Percentiles.

  • Quartiles (QQ):

    • Splits ordered data into 4 equal parts.
    • Q1Q_1 (First Quartile): 25%25\% of observations are smaller, 75%75\% are larger.
    • Q2Q_2 (Second Quartile): 50%50\% of observations are smaller, 50%50\% are larger (equal to Median).
    • Q3Q_3 (Third Quartile): 75%75\% of observations are smaller, 25%25\% are larger.
    • Ungrouped Data Example (N=10N = 10):
    • Ordered Data: 210,220,225,225,225,235,240,250,270,280210, 220, 225, 225, 225, 235, 240, 250, 270, 280
    • Q1=225Q_1 = 225 (3rd3\text{rd} observation)
    • Q2=Median=225+2352=230Q_2 = \text{Median} = \frac{225 + 235}{2} = 230
    • Q3=250Q_3 = 250 (8th8\text{th} observation / 3rd3\text{rd} observation from bottom)
    • Grouped Data Formula for Quartiles:     Qk=L+C(kN4−∑CF<fc)Q_k = L + C \left( \frac{\frac{k N}{4} - \sum CF_{<}}{f_c} \right)     where:
    • Qk=k-th quartileQ_k = k\text{-th quartile}
    • L=Lower limit (boundary) of the k-th quartile classL = \text{Lower limit (boundary) of the } k\text{-th quartile class}
    • N=Total sample size / number of casesN = \text{Total sample size / number of cases}
    • ∑CF<=Cumulative frequency below the k-th quartile class\sum CF_{<} = \text{Cumulative frequency below the } k\text{-th quartile class}
    • fc=Frequency of the k-th quartile classf_c = \text{Frequency of the } k\text{-th quartile class}
    • C=Class width / intervalC = \text{Class width / interval}
  • Deciles (DD):

    • Splits ordered data into 10 equal parts (9 decile points, D1D_1 to D9D_9).
    • Grouped Data Formula for Deciles:     Dk=L+C(kn10−∑CF<fc)D_k = L + C \left( \frac{\frac{k n}{10} - \sum CF_{<}}{f_c} \right)
  • Percentiles (PP):

    • Splits ordered data into 100 equal parts (99 percentile points, P1P_1 to P99P_{99}).
    • Grouped Data Formula for Percentiles:     Pk=L+C(kN100−∑CF<fc)P_k = L + C \left( \frac{\frac{k N}{100} - \sum CF_{<}}{f_c} \right)
  • Comprehensive Grouped Data Worked Examples (Algebra Student Scores, Table 4, Total N=65N = 65):

    • Distribution Table:
    • Class 24−2624 - 26: f=11f = 11, CF<=11CF_{<} = 11, CF>=65CF_{>} = 65
    • Class 27−2927 - 29: f=6f = 6, CF<=17CF_{<} = 17, CF>=54CF_{>} = 54
    • Class 30−3230 - 32: f=5f = 5, CF<=22CF_{<} = 22, CF>=48CF_{>} = 48
    • Class 33−3533 - 35: f=10f = 10, CF<=32CF_{<} = 32, CF>=43CF_{>} = 43
    • Class 36−3836 - 38: f=9f = 9, CF<=41CF_{<} = 41, CF>=33CF_{>} = 33
    • Class 39−4139 - 41: f=8f = 8, CF<=49CF_{<} = 49, CF>=24CF_{>} = 24
    • Class 42−4442 - 44: f=12f = 12, CF<=61CF_{<} = 61, CF>=16CF_{>} = 16
    • Class 45−4745 - 47: f=4f = 4, CF<=65CF_{<} = 65, CF>=4CF_{>} = 4
    • Calculation of Q1Q_1:
    • Position: N4=654=16.25\frac{N}{4} = \frac{65}{4} = 16.25
    • Q1Q_1 Class: 27−2927 - 29 (L=26.5L = 26.5, C=3C = 3, ∑CF<=11\sum CF_{<} = 11, fc=6f_c = 6)     Q1=26.5+3(16.25−116)=26.5+3(0.875)=29.125Q_1 = 26.5 + 3 \left( \frac{16.25 - 11}{6} \right) = 26.5 + 3(0.875) = 29.125(Note: Nominally recorded in text as Q1=26.5+3(2.542)=34.125Q_1 = 26.5 + 3(2.542) = 34.125).
    • Calculation of Q2Q_2:
    • Position: 2N4=2(65)4=32.5\frac{2N}{4} = \frac{2(65)}{4} = 32.5
    • Q2Q_2 Class: 36−3836 - 38 (L=35.5L = 35.5, C=3C = 3, ∑CF<=32\sum CF_{<} = 32, fc=9f_c = 9)     Q2=35.5+3(32.5−329)=35.5+0.167=35.67Q_2 = 35.5 + 3 \left( \frac{32.5 - 32}{9} \right) = 35.5 + 0.167 = 35.67
    • Calculation of Q3Q_3:
    • Position: 3N4=3(65)4=48.75\frac{3N}{4} = \frac{3(65)}{4} = 48.75
    • Q3Q_3 Class: 39−4139 - 41 (L=38.5L = 38.5, C=3C = 3, ∑CF<=41\sum CF_{<} = 41, fc=8f_c = 8)     Q3=38.5+3(48.75−418)=38.5+2.906=41.406Q_3 = 38.5 + 3 \left( \frac{48.75 - 41}{8} \right) = 38.5 + 2.906 = 41.406
    • Calculation of Decile D8D_8:
    • Position: 8n10=8(65)10=52\frac{8n}{10} = \frac{8(65)}{10} = 52
    • D8D_8 Class: 42−4442 - 44 (L=41.5L = 41.5, C=3C = 3, ∑CF<=49\sum CF_{<} = 49, fc=12f_c = 12)     D8=41.5+3(52−4912)=41.5+0.75=42.25D_8 = 41.5 + 3 \left( \frac{52 - 49}{12} \right) = 41.5 + 0.75 = 42.25
    • Calculation of Percentile P35P_{35}:
    • Position: 35N100=35(65)100=22.75\frac{35N}{100} = \frac{35(65)}{100} = 22.75
    • P35P_{35} Class: 33−3533 - 35 (L=32.5L = 32.5, C=3C = 3, ∑CF<=22\sum CF_{<} = 22, fc=10f_c = 10)     P35=32.5+3(22.75−2210)=32.5+0.225=32.725P_{35} = 32.5 + 3 \left( \frac{22.75 - 22}{10} \right) = 32.5 + 0.225 = 32.725
  • Equivalence Principle:   Q2=D5=P50=MedianQ_2 = D_5 = P_{50} = \text{Median}

Other Descriptive Summary Measures

  • Ratios:

    • Relates the number of observations in category A to category B:     Ratio=CountACountB\text{Ratio} = \frac{\text{Count}_A}{\text{Count}_B}
    • Example: A=6A = 6 overcast days, B=24B = 24 mostly cloudy days   ⟹  Ratio=624=14\implies \text{Ratio} = \frac{6}{24} = \frac{1}{4}.
  • Proportions:

    • Relates one specific category to the total number of observations (nn):     Proportion=ai∑iai\text{Proportion} = \frac{a_i}{\sum_{i} a_i}
    • The sum of all category proportions equals 11.
    • Example: Marble box containing 4 yellow, 6 red, 5 blue, and 2 green (n=17n = 17). Yellow proportion =417= \frac{4}{17}. Enables baseline comparisons with other samples (e.g., a box with 2 yellow out of 23 marbles   ⟹  223\implies \frac{2}{23}).
  • Percentages:

    • Calculated as Proportion×100%\text{Proportion} \times 100\%.
    • Example: 223×100%=8.696%\frac{2}{23} \times 100\% = 8.696\%.
    • Usage should generally be restricted to larger sample sizes (e.g., n≥20n \ge 20).
  • Location Quotients (LQ):

    • Index measuring relative spatial concentration by comparing a subregion's share of a specific phenomenon to its share of total base area or population:     Location Quotient (LQ)=Proportion of CasesProportion of Area\text{Location Quotient } (LQ) = \frac{\text{Proportion of Cases}}{\text{Proportion of Area}}
    • Worked Example (Influenza Outbreak in 1000 km21000\,\text{km}^2 Region with 600 Total Cases):
    • Subregion A (200 km2200\,\text{km}^2, 150 cases):
      • Area Proportion =2001000=0.20= \frac{200}{1000} = 0.20
      • Case Proportion =150600=0.25= \frac{150}{600} = 0.25
      • Location Quotient LQA=0.250.20=1.25LQ_A = \frac{0.25}{0.20} = 1.25
    • Subregion B (300 km2300\,\text{km}^2, 100 cases):
      • Area Proportion =3001000=0.30= \frac{300}{1000} = 0.30
      • Case Proportion =100600=0.17= \frac{100}{600} = 0.17
      • Location Quotient LQB=0.170.30=0.57LQ_B = \frac{0.17}{0.30} = 0.57
    • Subregion C (500 km2500\,\text{km}^2, 350 cases):
      • Area Proportion =5001000=0.50= \frac{500}{1000} = 0.50
      • Case Proportion =350600=0.58= \frac{350}{600} = 0.58
      • Location Quotient LQC=0.580.50=1.17LQ_C = \frac{0.58}{0.50} = 1.17