Statistical Thinking

Components of a Statistical Investigation

• Planning the study: Formulate a testable research question and establish a protocol for data collection.
• Examining the data: Use appropriate graphs and descriptive statistics to identify patterns, central tendencies, variability, and deviations.
• Inferring from the data: Utilize formal statistical methods to draw conclusions extending beyond the collected sample.
• Drawing conclusions: Determine population generalizability and assess whether causal claims are warranted.

Distributional Thinking

• Fundamental Principle: Data vary, and analyzing the pattern of variation (the distribution) yields key insights.
• Comparing Distributions: Evaluating full distributions provides a complete picture, whereas relying solely on central tendency measures (such as medians or means) hides variability and key discrepancies.

Statistical Significance and P-Values

• Purpose: Determines whether an observed pattern represents a true systematic effect or simple random variation.
• P-Value: The probability of obtaining a sample result at least as extreme as the observed data, assuming random chance is the sole operating factor.
• Significance Threshold: When a p-value falls below the significance level (typically $\alpha = 0.05$), the hypothesis of chance alone is rejected, confirming statistical significance.

Generalizability and Random Sampling

• Random Sampling: Selects individuals from a population using probability-based methods where every member has an equal selection chance, ensuring sample representativeness.
• Margin of Error: Quantitative estimate of expected sampling variation, approximately calculated as $\frac{1}{\sqrt{n}}$ for a sample size $n$.
• Confidence Interval: Plausible range for an unknown population parameter, combining the sample statistic and margin of error.
• Sampling Bias: Occurs in non-random sampling when specific population segments are systematically over- or under-represented.

Cause-and-Effect Conclusions and Random Assignment

• Random Assignment: Assigns participants to treatment conditions using probability methods, balancing known and unknown confounding variables across groups.
• Causal Inference: Validated through random assignment when statistically significant differences emerge between treatment groups.
• Observational Limitations: Observational studies lack random assignment, preventing definitive cause-and-effect conclusions due to potential unadjusted confounding variables.

Questions & Discussion

• Research Design Critique: What was the primary research question, selection method, and design of a recent study in your field?
• Sample Size Inferences: Is it reasonable to use a random sample of $1\,000$ individuals to draw conclusions about all U.S. adults? Explain why or why not.