Effect Size & Power

These flashcards cover key vocabulary and definitions regarding effect size, power, and null-hypothesis significance testing from the lecture by Dr. Mingyuan Chu.

Null-hypothesis significance testing (NHST)

A statistical method used to determine whether there is enough evidence to reject a null hypothesis.

Effect size

A quantitative measure of the magnitude of a phenomenon or the size of group differences.

Power

The probability that a statistical test will correctly reject a false null hypothesis.

Statistical significance vs. Practical significance

Statistical significance indicates that a result is unlikely due to chance, while practical significance assesses the real-world relevance of the result.

Cohen's d

An effect size measure that expresses the difference between two means in standard deviation units.

Glass' delta

An effect size measure that uses the standard deviation from the control group. It is useful when comparing multiple treatments to a control.

Hedge’s g

An effect size measure similar to Cohen's d, adjusted for small sample sizes.

Arbitrary alpha level

A predetermined threshold (e.g., α=0.05) for defining statistical significance, often criticized for being arbitrary.

Practical significance example

The scenario where statistically significant results have little to no real-world impact, illustrated by an insignificant group mean difference.

All-or-nothing thinking

A flawed reasoning pattern in NHST where p-value thresholds dictate significance, leading to potentially misleading conclusions.

Sample size estimation

Using effect size to determine the number of participants needed to achieve sufficient statistical power.

Meta-analysis

A statistical technique that combines the results of multiple studies to determine overall trends.

Difference in means

A method to calculate effect size by subtracting the means of two groups.

Standardized mean difference

An effect size measure that compares the means of two groups in terms of their variability.

Group difference indices

Various methods for calculating effect sizes that express differences between groups.

Risk estimates

Measures used in statistics to quantify the risk of an event occurring in one group compared to another.

p-value

The probability of obtaining results at least as extreme as the observed results, assuming that the null hypothesis is correct.

Sample means vs. Population means

Sample means are calculated from a subset of a larger population, which may have unknown population parameters.

Type I error

The incorrect rejection of a true null hypothesis (false positive).

Type II error

Failing to reject a false null hypothesis (false negative).

Statistical tests types

Methods like t-tests and ANOVAs used to assess differences between groups.

Variance explained

A measure of how much of the variability in a dependent variable is explained by an independent variable.

Limitations of NHST

Concerns regarding the reliance on p-values, such as the difficulty in proving the null hypothesis.

Sample representation

The degree to which a sample reflects the characteristics of the population.

Effect size interpretation

Understanding effect size within the context of the study and its practical implications.

Confidence interval (CI)

A range of values derived from sample statistics that is likely to contain the true population parameter.

Meta-analysis significance

The importance of combining data from various studies and how it influences the interpretation of effect sizes.

Study design influence

How the structure and methods of a study can affect its results and the derived effect sizes.

Causal inference

The process of drawing a conclusion about a causal connection based on the conditions of the occurrence of an effect.

Null hypothesis

A general statement or default position that there is no relationship between two measured phenomena.

Alternative hypothesis

The hypothesis that there is an effect or a difference; it opposes the null hypothesis.

Sample power

The likelihood that a study will find an effect when there is an effect to be found.

Significance threshold

The criterion set for deciding whether a p-value indicates statistical significance.