Frequentist Probability and Statistical Testing
Core Concepts of Frequentism
Frequentism:
- The philosophical interpretation of probability holding that the probability of an event is defined as its relative frequency within a specified reference class.
Relative Frequency:
- The mathematical proportion calculated as:
- Expressed equivalently as the number of times an event happened the number of times it could have happened.
Major Variants of Frequentism
Actual Frequentism:
- Defines the probability of an event strictly as its actual relative frequency within an actual, concrete reference class.
- Example: If a real coin is flipped times and lands on heads times, the actual relative frequency—and thus the probability—is:
Hypothetical Frequentism:
- Defines the probability of an event as its limiting relative frequency within a hypothetical infinite reference class.
- Conceptualized as the frequency that would be obtained if the trial or experiment were repeated forever into infinity.
The Reference Class and Its Theoretical Limitations
Reference Class:
- The designated group or set of trials within which an event's relative frequency is observed and measured.
- Example: A specific empirical set such as "these flips of this coin."
Reference Class Problem:
- A foundational dilemma in frequentist theory arising because any single event belongs simultaneously to multiple distinct reference classes.
- Because different reference classes yield different relative frequencies, they assign different probabilities to the exact same event, and there is no objective principle or criterion to select the single "true" reference class.
- Hypothetical Frequentism Complication: In infinite hypothetical reference classes, varying the orderings of the identical infinite trials results in different mathematical limits, making the limiting relative frequency unstable.
Asymptotic Properties: The Law of Large Numbers
- Law of Large Numbers:
- A fundamental theorem stating that if the theoretical probability of an outcome is , then as the total number of experimental trials approaches infinity (), the observed relative frequency of can be expected to approach .
Statistical Hypothesis Testing and the Chi-Square Statistic
Null Hypothesis:
- The initial starting assumption or baseline model utilized in hypothesis testing that allows for the derivation of expected values.
- Example: The assumption that "the coin is fair."
(Chi-Square) Statistic:
- A quantitative metric that measures the magnitude of divergence between observed experimental counts and expected theoretical counts.
- Defined by the summation formula:
- Alternatively formatted as:
- Component variables:
- : The observed count for each individual outcome.
- : The expected count for each individual outcome based on the assumed model.
-Value:
- The probability, calculated under the strict assumption that the null hypothesis is true, of obtaining the results actually observed or any result that is even less likely (more extreme).
- Critical Distinction: The -value is NOT the probability that the null hypothesis is true.