1: Statistical Reasoning

Module 3 Notes: Statistical Reasoning in Everyday Life

  • Overview

    • Statistical literacy means understanding statistics and what they actually mean in real life.

    • There is widespread confusion around COVID risks and vaccine effectiveness; people may overestimate or underestimate risk depending on vaccination status.

    • National surveys show unvaccinated individuals are generally less likely to fear the virus, while many vaccinated individuals still fear it. This fear can influence health decisions (e.g., seeking vaccination, preventive behaviors).

    • Casual estimates and “top-of-the-head” numbers (e.g., 80%, 70%, or claims like “10% of your brain”) are often misleading without context.

    • The goal is accurate statistical understanding and greater transparency in presenting statistical information.

  • Vaccine literacy and base rates

    • Base-rate concept: understanding how base proportions affect interpretation is crucial.

    • Example setup: Town population is 100 people (dots).

    • Media claim: among people dying of a pandemic virus, half were vaccinated.

    • Important missing context: vaccination base rate is 90% (i.e., 90 vaccinated, 10 unvaccinated).

    • Given data:

    • Vaccinated deaths: 5 out of 90 vaccinated → death rate among vaccinated: ext{Rate}_V = rac{5}{90}
      \approx 0.0556 \ ( ext{5.56%})

    • Unvaccinated deaths: 5 out of 10 unvaccinated → death rate among unvaccinated: ext{Rate}_U = rac{5}{10} = 0.50 \text{(50%)}

    • Takeaway: Although 5 vaccinated deaths and 5 unvaccinated deaths exist, the death rate among the vaccinated is much lower due to the base rate of vaccination in the population. The correct interpretation is about rates, not raw counts.

    • The base-rate pitfall in reporting:

    • Saying “half of the deaths were vaccinated” can be misleading if you don’t include the base rates and the population composition.

    • Isolated percentages can misrepresent risk unless connected to base rates and denominators.

    • Additional example: common misinterpretation about votes within a union:

    • A claim may say: “85% vote for a strike amongst a union.”

    • But if only 30% of union workers voted, and among those voters 80% voted for the strike, the overall proportion of the entire union that voted for the strike is:0.30×0.80=0.24=24%.0.30 \times 0.80 = 0.24 = 24\%.

    • If someone reports “80%” or “85%” without clarifying whether it’s conditional on voting or unconditional across all members, misinterpretation easily occurs.

  • Descriptive statistics: central tendency and variability

    • Descriptive statistics provide simple summaries of data, usually numeric or pictorial (e.g., bar graphs).

    • Central tendency measures:

    • Mode: most frequently occurring value in the data distribution.

    • Mean: arithmetic average; xˉ=1n∑<em>i=1nx</em>i.\bar{x} = \frac{1}{n} \sum<em>{i=1}^{n} x</em>i.

    • Median: middle value when data are ordered; for an even number of observations, the median is the average of the two middle values.

    • Example to illustrate skew and how it affects mean vs median vs mode:

    • Village income distribution:

      • Mode = $40,000{\$40{,}000} (most common income).

      • Mean = $140,000{\$140{,}000} (the average income across all households).

      • Median = $60,000{\$60{,}000} (the middle value when incomes are ordered).

      • Interpretation: There are a few very high earners pulling the mean up; the majority earn much less, so the mean is not representative of a typical household.

    • Measures of dispersion (variation):

    • Range: difference between highest and lowest values. Range=x<em>max⁡−x</em>min⁡.\text{Range} = x<em>{\max} - x</em>{\min}. The range is sensitive to outliers.

    • Standard deviation: a measure of how much scores differ from the mean. s=1n−1∑<em>i=1n(x</em>i−xˉ)2.s = \sqrt{\frac{1}{n-1}\sum<em>{i=1}^{n}(x</em>i - \bar{x})^2}. It accounts for all deviations from the mean and is typically smaller than the range.

    • Normal distribution (the “bell curve”):

    • A symmetric distribution where most scores cluster near the mean; about 68%68\% fall within one standard deviation of the mean.

    • Example (Wechsler scales): mean = 100, standard deviation = 15. So, about 68%≈P(85≤X≤115)68\%\approx P(85 \le X \le 115) have an IQ between 85 and 115.

  • Graphs, scales, and the presentation of data

    • Descriptive graphs can be designed to exaggerate or minimize differences by choice of scale.

    • Example: three truck brands with different durability data presented in two graphs:

    • Graph A uses a y-axis from 65% to 100%, which can make small differences look large.

    • Graph B uses a full 0% to 100% axis, and the apparent differences between brands become much smaller.

    • Lesson: The Y-axis scale, labeling, and range matter for interpretation. Literate statisticians examine scales and ranges to assess what a graph truly shows.

  • Inferential statistics and the role of sampling

    • Descriptive vs inferential statistics:

    • Descriptive: summarize data we have.

    • Inferential: use sample data to draw conclusions about a population.

    • Why inferential statistics matter:

    • People’s behavior and scores can vary from moment to moment; observed differences between groups may reflect sampling variation, not true population differences.

    • Statistical significance:

    • A statistical statement about how likely it is that an observed difference occurred by chance, assuming no real difference exists in the populations.

    • Formal concept: if the difference between sample averages is reliable and large enough, it is statistically significant.

    • The common threshold in psychology (and many fields): if the probability of obtaining such a result by chance is less than 5%, the result is deemed statistically significant.

    • Notation: the p-value represents the probability of the observed data (or more extreme) under the null hypothesis. If p < 0.05, the result is considered statistically significant.

    • Null hypothesis (H0): there is no difference between groups in the population.

    • The logic of hypothesis testing:

    • Start with H0 (no difference).

    • Collect data and compute a test statistic.

    • If the p-value is below the pre-specified alpha level (e.g., α=0.05\alpha = 0.05), reject H0.

    • Practical vs statistical significance:

    • A result can be statistically significant but have a small practical effect size; i.e., the real-world importance may be negligible.

    • General principles for reliable inference:

    • Representative samples are better than biased or unrepresentative samples.

    • Larger samples are better than smaller ones.

    • More estimates (replications) are better than fewer estimates.

    • Case studies and generalization:

    • Generalizing from a few unrepresentative cases is unreliable; caution is needed when extending findings beyond the studied cases.

    • Hypothesis testing basics (summary):

    • H0: no difference.

    • Compute a test statistic and p-value.

    • If p < 0.05, reject H0; otherwise, fail to reject H0.

    • Statistical significance depends on both the size of the effect and the precision of the estimate (sample size).

  • Practical implications and ethical considerations

    • Statistical misinformation is common and can be amplified by presenting vague numbers (e.g., “80%,” “70%,” or sweeping claims) without context.

    • Transparent reporting includes base rates, denominators, and the actual population context.

    • When communicating statistical findings, it is important to distinguish between:

    • Percentages (proportions conditional on a subgroup) vs. rates (risk within a population segment).

    • Absolute risk reduction vs. relative risk, etc. (not explicitly in the transcript but a key part of literacy).

    • For students and the public: Always ask

    • What is the base rate?

    • What is the denominator?

    • Is the percentage describing a conditional probability (e.g., among those who voted) or an unconditional proportion (e.g., among all members)?

    • Is the figure emphasizing practical significance or merely statistical significance?

  • Quick recap of key formulas and concepts (encapsulated in the transcript)

    • Mean: xˉ=1n∑<em>i=1nx</em>i.\bar{x} = \frac{1}{n}\sum<em>{i=1}^{n} x</em>i.

    • Median: the middle value when data are ordered; for even n, the average of the two middle values.

    • Mode: most frequent value in the data set.

    • Range: Range=x<em>max⁡−x</em>min⁡.\text{Range} = x<em>{\max} - x</em>{\min}.

    • Standard deviation: s=1n−1∑<em>i=1n(x</em>i−xˉ)2.s = \sqrt{\frac{1}{n-1}\sum<em>{i=1}^{n}(x</em>i - \bar{x})^2}.

    • Normal distribution (68-95-99.7 rule): approximately 68% within μ±σ\mu \pm \sigma, ~95% within μ±2σ\mu \pm 2\sigma, ~99.7% within μ±3σ\mu \pm 3\sigma (noted here specifically as the 68% example with IQ: IQ∼N(100,152)\text{IQ} \sim N(100, 15^2) giving 68% in [85, 115]).

    • Inferential statistics goal: determine if observed differences generalize beyond the sample; interpret p-values and null hypothesis.

    • p-value threshold: p < 0.05 often used as cutoff for statistical significance.

    • Base-rate reasoning: distinguish numbers that refer to groups who were exposed to a factor from those that were not; adjust interpretation accordingly.

  • Final takeaway from Module 3

    • Statistical reasoning helps avoid misinterpretation in everyday life and public health decisions.

    • Transparency in presenting data (denominators, bases, and scales) is essential to avoid misinformation.

    • A strong foundation in descriptive and inferential statistics supports better judgment, policy understanding, and ethical communication of findings.

  • Tip from the module

    • Take breaks during study sessions to maintain attention and retention: e.g., “Make sure you take a break” to prevent fatigue and consolidate learning.

  • End of Module 3