Frequentist Probability and Statistical Testing

Core Concepts of Frequentism

  • Frequentism:

    • The philosophical interpretation of probability holding that the probability of an event is defined as its relative frequency within a specified reference class.
  • Relative Frequency:

    • The mathematical proportion calculated as:     Relative Frequency=Number of times an event happenedNumber of times it could have happened\text{Relative Frequency} = \frac{\text{Number of times an event happened}}{\text{Number of times it could have happened}}
    • Expressed equivalently as the number of times an event happened ÷\div the number of times it could have happened.

Major Variants of Frequentism

  • Actual Frequentism:

    • Defines the probability of an event strictly as its actual relative frequency within an actual, concrete reference class.
    • Example: If a real coin is flipped 10001000 times and lands on heads 499499 times, the actual relative frequency—and thus the probability—is:     4991000=0.499\frac{499}{1000} = 0.499
  • Hypothetical Frequentism:

    • Defines the probability of an event as its limiting relative frequency within a hypothetical infinite reference class.
    • Conceptualized as the frequency that would be obtained if the trial or experiment were repeated forever into infinity.

The Reference Class and Its Theoretical Limitations

  • Reference Class:

    • The designated group or set of trials within which an event's relative frequency is observed and measured.
    • Example: A specific empirical set such as "these 10001000 flips of this coin."
  • Reference Class Problem:

    • A foundational dilemma in frequentist theory arising because any single event belongs simultaneously to multiple distinct reference classes.
    • Because different reference classes yield different relative frequencies, they assign different probabilities to the exact same event, and there is no objective principle or criterion to select the single "true" reference class.
    • Hypothetical Frequentism Complication: In infinite hypothetical reference classes, varying the orderings of the identical infinite trials results in different mathematical limits, making the limiting relative frequency unstable.

Asymptotic Properties: The Law of Large Numbers

  • Law of Large Numbers:
    • A fundamental theorem stating that if the theoretical probability of an outcome XX is p(X)=xp(X) = x, then as the total number of experimental trials approaches infinity (n→∞n \rightarrow \infty), the observed relative frequency of XX can be expected to approach xx.

Statistical Hypothesis Testing and the Chi-Square Statistic

  • Null Hypothesis:

    • The initial starting assumption or baseline model utilized in hypothesis testing that allows for the derivation of expected values.
    • Example: The assumption that "the coin is fair."
  • χ2\chi^2 (Chi-Square) Statistic:

    • A quantitative metric that measures the magnitude of divergence between observed experimental counts and expected theoretical counts.
    • Defined by the summation formula:     χ2=∑(O−E)2E\chi^2 = \sum \frac{(O - E)^2}{E}
    • Alternatively formatted as:     χ2=∑(O−E)2÷E\chi^2 = \sum (O - E)^2 \div E
    • Component variables:
    • OO: The observed count for each individual outcome.
    • EE: The expected count for each individual outcome based on the assumed model.
  • pp-Value:

    • The probability, calculated under the strict assumption that the null hypothesis is true, of obtaining the results actually observed or any result that is even less likely (more extreme).
    • Critical Distinction: The pp-value is NOT the probability that the null hypothesis is true.