Hypothesis Testing and Statistical Inference

Introduction

  • Lecture on hypothesis testing considered the most important of the semester.

  • Foundation for understanding scientific inquiry, decision-making, and statistical inference.

Importance of the Lecture

  • Builds on previous lectures and serves as a basis for future implementation.

  • Focus on null hypothesis significance testing (NHST).

Overview of Hypothesis Testing

  • NHST helps answer questions despite uncertainty.

  • Key takeaway: Statistical tests only indicate how likely results are due to chance, not the correctness of theories or experiments.

  • Points out limitations of conclusions drawn from NHST.

Structure of the Lecture

  • Topics to be covered:

    • Statistical inference overview.

    • Definitions of null and alternative hypothesis.

    • Meaning of statistical significance.

    • Sampling distributions.

    • Decision errors in statistical testing.

Acknowledgment of Land

  • Recognition of traditional custodians of the land and appreciation for their contributions.

Statistical Inference

  • Aim to make conclusions beyond a specific sample.

  • Importance of sampling in statistical inference.

Null and Alternative Hypotheses

  • Definitions and importance of both hypotheses:

    • Null Hypothesis (H0): Assumes no effect or difference; serves as a baseline.

    • Alternative Hypothesis (H1): Assumes an effect or difference; what researchers seek evidence for.

  • Examples:

    • Asking if smoking is harmful, or if distraction impairs memory.

Statistical Significance

  • Definition of statistical significance relates to the probability of an observed result occurring by chance.

  • P-value: A measure used to determine significance in hypothesis testing.

    • Common threshold is p < 0.05.

  • Importance of understanding that a significant result does not prove a hypothesis true.

Sampling Distributions

  • To generalize findings from samples to populations:

    • Characterize distributions using mean, median, mode, range, variance, and standard deviation.

  • Example: Flipping a coin 200 times to test if it is fair:

    • Expected heads: 100.

    • Variability based on chance leads to different results (e.g., getting 198 heads).

Illustrative Example of Decision Making

  • Hypothetical scenario involving the fairness of a coin:

    • Repeated experiments generate a distribution of outcomes.

    • Differences in heads obtained lead to questioning the coin’s fairness.

Populations vs. Samples

  • Importance of differentiating between population and sample:

    • Population: All possible measurements.

    • Sample: A subset of the population used for analysis.

  • Example: Polling a representative sample for elections.

  • Different types of populations (e.g., all Australians, all coin flips).

Random Sampling

  • Definition: Selecting participants randomly from the population.

  • Importance of validity in statistical testing:

    • Results from biased samples (e.g., convenience sampling) are less reliable.

Sampling Error and Variability

  • Sampling Error: The difference between the sample mean and the true population mean.

  • Sampling Variability: Variability observed among different samples drawn from the same population.

  • Illustration: Continuous variation when taking multiple samples of varying sizes.

Central Limit Theorem (CLT)

  • Definition: The distribution of sample means approaches a normal distribution as sample size increases.

  • Key points:

    • Applies to any population distribution.

    • For effective application, sample sizes generally need to exceed 25.

  • Visualization through demonstrations and simulations in MATLAB:

    • Demonstrated the normal distribution of sample means regardless of the original population distribution.

Standard Error of the Mean (SEM)

  • Definition: The standard deviation of the sampling distribution of the mean.

  • Calculation:

    • SEM = rac{ ext{Population Standard Deviation}}{ ext{Square Root of Sample Size}}

  • Importance: SEM quantifies the precision of sample means as an estimate of the population mean.

Z-scores and Probabilities

  • Z-scores used to relate observed means back to the sampling distribution.

  • Application to hypothesis testing by determining how surprising a sample mean is relative to the null hypothesis.

  • Implication: The probability of observing this mean can indicate whether it is from the null population, ultimately influencing hypothesis acceptance or rejection.

Conclusion on NHST

  • NHST evaluates how likely it is that an observed result derives from the null distribution.

  • It does not provide confidence in the alternative hypothesis’s truth, which can lead to misconceptions about results' implications.

  • Emphasizes the importance of a critical view towards statistical testing and its interpretation.

Key Takeaways

  • P-value threshold is arbitrary and should be treated with caution in scientific research.

  • Different fields may adopt different thresholds for significance (e.g., 5 sigma in physics).

  • Importance of robust experimental design, including random sampling to minimize bias.

  • Decision-making under uncertainty can lead to potential decision errors, which need addressing.

Final Reminder

  • Emphasized the importance of understanding these concepts thoroughly as they will recur throughout the semester in future applications and tests.