Philosophy Introductory Bayes 2016 (W10)

Philosophy of Statistics

• Focuses on the comparison between Bayesian and orthodox (frequentist) approaches.

Key Quotes

• "The statistician cannot excuse himself from the duty of getting his head clear on the principles of scientific inference, but equally no other thinking person can avoid a like obligation" - Fisher 1951

Historical Context

• Prior to 1930s:

  • Multiple statistical procedures available but lacking coherent frameworks.

  • Neyman and Pearson formalized statistics into an orthodox structure.

  • Ongoing debates about the robustness of this logical framework.

Understanding Probability

• Probability: Relative frequency interpretation requires specification of a collective (e.g., coin tosses).

  • In the long run, the probability of heads in fair coin tosses is 1/2.

Hypothesis Probability

• Cannot assign probability to hypotheses (e.g., "this cancer drug is more effective than placebo").

• Hypotheses are not true for certain proportions; they are simply true or false.

  • Subjective probability arises when discussing hypotheses.

Neyman-Pearson Framework

• Defined standard statistics philosophy:

  • Probabilities are long-run relative frequencies and not subjective.

  • Statistics do not provide the probability of specific hypotheses being true.

Data and Hypotheses

• Example Statement: For data D and hypothesis H (e.g., drug is ineffective), we can consider conditional probabilities like p(D|H).

• Reference classes play a critical role in proper probabilistic reasoning.

  • Hypothetical scenarios can determine probabilities based on countless experiments.

Limitations of Conditional Probability

• Cannot discuss p(H|D); a hypothesis is simply true or false.

• Inversion of probabilities (p(H|D) vs p(D|H)) significantly affects understanding.

  • Example: p(dying | head bitten off by shark) vs p(head bitten off by shark | died).

Decision Procedures

• Statistical decision-making relies on setting rules for accepting or rejecting hypotheses to minimize long-term errors.

• Example: Run 40 subjects and reject the null hypothesis if the t-value exceeds a critical value.

Non-significant Results Analysis

• Non-significant results offer no direct inference about specific hypotheses.

  • Non-significance could imply a fair coin or lack of sufficient data.

  • Fisher's view suggests suspending judgment on non-significant outcomes.

Bayesian Relevance

• Bayes factors allow the distinction of evidence for or against hypotheses, contrasting with mere p-value reliance.

  • P-values require establishing arbitrary significance thresholds, affecting scientific interpretations.

Common Misinterpretations in Research

• Non-significant results often incorrectly interpreted as support for null hypotheses.

• Critical evaluation of studies with non-significant findings is essential to avoid erroneous conclusions.

Statistical Significance Testing Critiques

• Ongoing criticism of the conventional p-value approach:

  • Cohen (1994): Significance testing does not serve the desired research outcomes, yet reliance persists.

  • Meehl (1978): Reliance on null hypothesis testing is detrimental to psychological research.

Subjective Probabilities

• Defined as personal convictions about hypotheses, tied to the axioms of probability and dependent on data revision.

Bayesian vs Orthodoxy Objectives

• Bayesian Statistics: Focus on changing confidence levels based on evidence.

• Orthodox Statistics: Control error proportions in long-term hypothesis testing.

Conclusion

• Understanding the philosophical differences between Bayesian and frequentist statistics is crucial in scientific inquiry and hypothesis testing.