Unit 10: Some Problems for NHST and Introduction to Bayesian Statistics

Flashcards covering the problems with NHST, the Replication Crisis, the components of Bayes’ Theorem, and the application of Bayesian inference including Parameter Estimation and Bayes Factors.

Replication Crisis

The realization that many published findings in psychology and other fields simply cannot be replicated, leading to an era of self-examination in research practices.

p-hacking

A poor research practice that involves changing analytical procedures, such as removing outliers, until a significant result is found.

Null Hypothesis Significance Testing (NHST)

A statistical framework criticized for forcing binary decisions based on a critical value of $.05$, ignoring degrees of belief, and failing to provide direct evidence for the null hypothesis.

Psi

A term used by Daryl J. Bem to denote anomalous processes of information or energy transfer, including precognition and premonition, which are currently unexplained by known physical mechanisms.

Daryl J. Bem

The author of the 2011 study 'Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect' which published statistically significant results for phenomena like precognition using 9 experiments.

Thomas Bayes

An English theologian and mathematician (1701-1761) whose work forms the basis of Bayesian statistics and probabilistic belief updating.

Bayesian Statistics

An approach to inference where beliefs are treated as probabilistic rather than all-or-none, and updated correctly based on new evidence and prior information.

Bayes’ Theorem

A mathematical formula that describes how the probability of a hypothesis changes with new evidence: $P(h|d) = \frac{P(d|h) \times P(h)}{P(d|h) \times P(h) + P(d| \sim h) \times P(\sim h)}$.

Posterior probability

The probability assigned to a hypothesis after encountering data, denoted as $P(h|d)$.

Prior probability

The probability assigned to a hypothesis before encountering new data, denoted as $P(h)$. It represents the initial state of belief.

Likelihood

The probability of the data occurring if a specific hypothesis is true, denoted as $P(d|h)$. In Bayesian inference, it can also be evaluated assuming the hypothesis is false, denoted as $P(d| \sim h)$, where $\sim$ means 'not'.

Parameter Estimation

A use of Bayesian inference to determine the probability distribution for a population parameter (like a mean or correlation) based on a prior distribution and a likelihood function.

Prior Distribution

A mathematically formal way of stating assumptions about the likelihood of different parameter values before an experiment is conducted.

Likelihood Function

A function showing the relative likelihood of observed data under different possible values of a parameter.

Posterior Distribution

The probability distribution resulting from the combination (multiplication) of the prior distribution and the likelihood function, showing the probability of various values after evidence is considered.

Bayes Factor

The ratio of the likelihood of the data under the alternative hypothesis to the likelihood under the null hypothesis, calculated as $\frac{P(d|h_1)}{P(d|h_0)}$.

BF10 > 1

A Bayes Factor result indicating that the data are more likely under the alternative hypothesis ($h_1$) than the null hypothesis ($h_0$).

BF10 < 1

A Bayes Factor result indicating that the data are more likely under the null hypothesis ($h_0$) than the alternative hypothesis ($h_1$).

Extreme Evidence

A qualitative category for a Bayes Factor ($BF_{10}$) that is either greater than $100$ (for the alternative) or less than $1/100$ (for the null).