Module 2 - Reading. Descriptive Statistics: Frequency Distributions, Measures of Location, Variability, Distribution Analysis, and Association

Relative Frequency and Percent Frequency Distributions

• Frequency distributions summarize categorical or quantitative data by showing the number (frequency) of items in each of several non-overlapping bins.

• Relative frequency represents the fraction or proportion of total items belonging to a specific bin.

• The formula for calculating relative frequency is:

$\text{Relative Frequency of a Bin} = \frac{\text{Frequency of the Bin}}{n}$

  where $n$ equals the total number of observations in the dataset.

• Relative frequency distribution is a tabular summary of data displaying the relative frequency for each bin.

• Percent frequency distribution is a tabular summary showing the percent frequency for each bin, computed by multiplying the relative frequency by $100\%$.

• Soft Drink Purchases Example ($n = 50$ observations):

  • Coca Cola: Frequency = $19$, Relative Frequency = $\frac{19}{50} = 0.38$, Percent Frequency = $38\%$

  • Diet Coke: Frequency = $8$, Relative Frequency = $\frac{8}{50} = 0.16$, Percent Frequency = $16\%$

  • Doctor Pepper: Frequency = $5$, Relative Frequency = $\frac{5}{50} = 0.10$, Percent Frequency = $10\%$

  • Pepsi: Frequency = $13$, Relative Frequency = $\frac{13}{50} = 0.26$, Percent Frequency = $26\%$

  • Sprite: Frequency = $5$, Relative Frequency = $\frac{5}{50} = 0.10$, Percent Frequency = $10\%$

  • Total Relative Frequency = $1.00$, Total Percent Frequency = $100\%$

  • Top 3 Soft Drinks Combined: Coca Cola (38\%$) + Pepsi (26\%$) + Diet Coke (16\%$) = 80\% of all purchases.\n- Estimating Probability Distributions:\n - A percent frequency distribution derived from observations of a random variable provides empirical estimates of the relative likelihoods of different outcomes.\n - Example application: A concession stand manager at an upcoming concert can use soft drink historical percent frequency distributions to estimate future demand probabilities and accurately order supply volumes.\n\n# Constructing Frequency Distributions for Quantitative Data\n\n- Defining Non-Overlapping Bins:\n - Quantitative data require careful specification of non-overlapping bins so that each observation falls into exactly one bin.\n- Actuarial Case Study (n = 700 individual ages at death from file `Agedith`):\n - Data values range from 0$to$109 years.\n- Three Steps to Define Bins for Quantitative Data:\n 1. Determine the Number of Non-Overlapping Bins:\n - General guideline: Select between 5$and$20 bins.\n - Using too many bins results in bins with very few observations, obscuring generalizable patterns.\n - Using too few bins aggregates data across too wide a range, failing to capture actual variation.\n - Selecting bin count is subjective and depends on sample size and objective. For n = 700$,$16 bins are selected.\n 2. Determine the Width of Each Bin:\n - General guideline: Keep bin width identical for every bin.\n - Bin width and bin count are inversely related.\n - Equation for approximate bin width:\n\n\text{Approximate Bin Width} = \frac{\text{Largest Data Value} - \text{Smallest Data Value}}{\text{Number of Bins}}\n\n - Calculation for age data:\n\n\text{Approximate Bin Width} = \frac{109 - 0}{16} = 6.8125\n\n - Round up to a convenient value: Bin width of 7\,\text{years}$. Rounding up ensures$16$bins of width$7$cover the full range of$109\,\text{years}.\n 3. Determine the Range Spanned by the Bins:\n - Range spanned by 16$bins of width$7$equals$16 \times 7 = 112\,\text{years}.\n - Since data range is 109\,\text{years}$, the spanning range exceeds data range by$3\,\text{years}.\n - Four possible starting values for the lower limit of the first bin yield valid sets: [-3, 4]$,$[-2, 5]$,$[-1, 6]$, or$[0, 7].\n - Choosing 0\,\text{years}$as lower limit gives the first bin$[0, 7].\n- Bin Limit Notation and Inclusivity Rules:\n - First bin [0, 7]$: Inclusive of both lower limit$0$and upper limit$7$($0 \le x \le 7$). Sorted values in dataset:$0, 0, 0, 0, 0, 1, 5$(Frequency =$7).\n - Second bin (7, 14]$: Excludes lower limit$7$($x > 7$) and includes upper limit$14$($x \le 14$). Value in dataset:$10$(Frequency =$1).\n - Standard Rule: First bin includes both limits; all subsequent bins exclude lower limit and include upper limit. This guarantees non-overlapping bins.\n - Bin Sequence: [0, 7]$,$(7, 14]$,$(14, 21]$,$(21, 28]$,$(28, 35]$,$(35, 42]$,$(42, 49]$,$(49, 56]$,$(56, 63]$,$(63, 70]$,$(70, 77]$,$(77, 84]$,$(84, 91]$,$(91, 98]$,$(98, 105]$,$(105, 112]$,$112+.\n - Overflow Bin 112+$: Contains all values strictly greater than$112\,\text{years}. Used for large datasets or handling extreme outliers.\n- Complete Age at Death Distribution (n = 700):\n - [0, 7]$: Frequency =$7$, Relative Frequency =$0.010$, Percent Frequency =$1.0\%\n - (7, 14]$: Frequency =$1$, Relative Frequency =$0.001$, Percent Frequency =$0.1\%\n - (14, 21]$: Frequency =$6$, Relative Frequency =$0.009$, Percent Frequency =$0.9\%\n - (21, 28]$: Frequency =$7$, Relative Frequency =$0.010$, Percent Frequency =$1.0\%\n - (28, 35]$: Frequency =$10$, Relative Frequency =$0.014$, Percent Frequency =$1.4\%\n - (35, 42]$: Frequency =$15$, Relative Frequency =$0.021$, Percent Frequency =$2.1\%\n - (42, 49]$: Frequency =$16$, Relative Frequency =$0.023$, Percent Frequency =$2.3\%\n - (49, 56]$: Frequency =$36$, Relative Frequency =$0.051$, Percent Frequency =$5.1\%\n - (56, 63]$: Frequency =$56$, Relative Frequency =$0.080$, Percent Frequency =$8.0\%\n - (63, 70]$: Frequency =$70$, Relative Frequency =$0.100$, Percent Frequency =$10.0\%\n - (70, 77]$: Frequency =$115$, Relative Frequency =$0.164$, Percent Frequency =$16.4\%\n - (77, 84]$: Frequency =$148$, Relative Frequency =$0.211$, Percent Frequency =$21.1\%\n - (84, 91]$: Frequency =$142$, Relative Frequency =$0.203$, Percent Frequency =$20.3\%\n - (91, 98]$: Frequency =$58$, Relative Frequency =$0.083$, Percent Frequency =$8.3\%\n - (98, 105]$: Frequency =$3$, Relative Frequency =$0.004$, Percent Frequency =$0.4\%\n - (105, 112]$: Frequency =$2$, Relative Frequency =$0.003$, Percent Frequency =$0.3\%$(values:$109, 109)\n - 112+$: Frequency =$0$, Relative Frequency =$0.000$, Percent Frequency =$0.0\%\n - Total: Frequency = 700$, Relative Frequency =$1.000$, Percent Frequency =$100.0\%\n- Primary Findings:\n - Most common age at death: (77, 84]$years ($148$individuals, or$21.1\%).\n - Only 7$individuals ($1.0\%$) died at age $\le 7\,\text{years}$.

  • Zero individuals lived beyond $112\,\text{years}$.

• Excel Implementation via FREQUENCY Function:

  • Column C contains lower limits, Column D contains upper limits (D2:D17).

  • Range A2:A701 contains raw age data.

  • Enter formula =FREQUENCY(A2:A701, D2:D17) in cell E2 to populate bin frequencies in E2:E18.

  • Legacy Excel method: Highlight cells E2:E17, type formula, press CTRL + SHIFT + ENTER (array formula).

Histograms, Frequency Polygons, and Distribution Shapes

• Histograms:

  • A graphical display of quantitative data constructed from frequency, relative frequency, or percent frequency distributions.

  • Formed by adjacent columns with zero gap width between them to display variable continuity.

• Creating Automatic Histograms in Excel:

  1. Select data range A1:A701.

  2. Click Insert tab on ribbon.

  3. Click Insert Statistic Chart button in Charts group $\rightarrow$ Select Histogram.

  • Excel automatically generates $16$ bins of width $7\,\text{years}$. Tallest bar corresponds to $(77, 84]$.

• Creating Manual Histograms in Excel (Custom Control):

  • Uses FREQUENCY function and CONCAT (or CONCATENATE in older Excel versions) for axis labels.

  1. Select cells D2:E17.

  2. Click Insert tab $\rightarrow$ Insert Column or Bar Chart $\rightarrow$ Select Clustered Column.

  3. Right-click chart $\rightarrow$ Change Chart Type $\rightarrow$ Select single variable clustered column.

  4. Right-click data series column $\rightarrow$ Format Data Series $\rightarrow$ Set Gap Width to $0\%$

  5. Right-click chart $\rightarrow$ Select Data $\rightarrow$ Under Horizontal Category Axis Labels click Edit $\rightarrow$ Enter =Data!$F$2:$F$17 $\rightarrow$ Click OK.

• Bin Parameter Sensitivity ($n = 30$ subset in Agedith30):

  • $5$ bins (width $12$): Indicates oldest bin is most likely.

  • $8$ bins (width $8$): Indicates second-oldest bin is most likely; oldest is third most likely.

  • $10$ bins (width $6$): Indicates oldest bin is most likely.

  • Explanation: $6$ out of $30$ observations equal exactly $87\,\text{years}$, making visual shape highly sensitive to bin boundary placement.

• Distribution Shape and Skewness:

  • Skewness reflects lack of symmetry in a distribution.

  • Left-Skewed Distribution: Left tail extends farther than right tail (e.g., age at death, exam scores where most scores exceed $70\%$ and none exceed $100\%$).

  • Right-Skewed Distribution: Right tail extends farther than left tail.

  • Symmetric Distribution: Left and right sides are mirror images.

• Frequency Polygons:

  • Uses continuous line segments connecting bin frequency counts rather than column bars.

  • Superior tool for comparing distributions across multiple groups without cluttering.

  • Requires relative frequency distributions when sample sizes differ ($n_{\text{female}} = 327$, $n_{\text{male}} = 373$ in AgedithSex).

• Excel Formulas for Comparative Polygons:

  • Total females in F3: `=COUNTIF(B2:B701,