3* Hypothesis Testing, z-Scores, Effect Size, and Statistical Power

Vocabulary practice flashcards covering z-scores, hypothesis testing, sampling distributions, error types, effect size, and statistical power.

Degrees of Freedom ($df$)

The number of scores free to assume any value in a set of data; because the sum of deviation scores must always equal $0$, only $N - 1$ out of $N$ deviation scores are free to vary.

$z$-Distribution

A hypothetical, symmetrical distribution that uses $z$-scores as the unit of measurement, with a mean, median, and mode of $0$, a standard deviation of $1$, asymptotic tails, inflection points at $-1\text{σ}$ and $1\text{σ}$, and a total area (probability) under the curve equal to $1$.

$z$-Score

A standardized score expressed in standard deviation units that accounts for distribution variability and allows comparison of scores across different distributions and metrics.

Sampling Distribution

A frequency distribution composed not of individual raw scores, but of statistics (such as sample means) calculated from separate samples.

Inferential Statistics

A branch of statistics where a sample is used to make inferences about a population, inherently accepting some uncertainty about the results.

Confidence Interval (CI)

A range of scores derived from sample data that is expected to capture a population parameter with a specified amount of confidence (typically $95\%$ or $99\%$).

Null Hypothesis Significance Testing (NHST)

The standard hypothesis testing framework that begins by assuming a null hypothesis of no difference or relationship, computes a test statistic (such as a $p$-value), compares it to a critical threshold, and decides whether to reject or retain the null hypothesis.

Alpha ($\alpha$)

The acceptable probability threshold set by the researcher (typically $.05$ or $.01$) representing the risk of concluding a statistically significant difference exists when the observed difference is actually due to chance alone.

$p$-Value

The probability that the observed sample data came from the null (original) population.

Standard Error of the Mean (SEM)

The measure of variability of sample means across repeated samples, calculated using the formula $\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{N}}$.

Standard Deviation (SD) vs. SEM

Standard deviation represents the variability of individual data points, whereas the standard error of the mean represents the variability of sample means.

Critical Value

The threshold value located in a distribution table (such as a $z$-table) corresponding to a designated alpha level that separates the rejection region from the retention region.

Two-Tailed (Nondirectional) Test

A conservative hypothesis test that divides alpha into both tails of the distribution to detect differences in either direction.

One-Tailed (Directional) Test

A hypothesis test sensitive to smaller effects in a specified direction based on an a priori directional prediction, carrying the penalty of retaining $H_0$ if the prediction is incorrect.

Effect Size

A measure of the magnitude of the difference or the strength of a relationship between variables that considers the spread and distance between means to determine if an observed pattern is meaningful.

Cohen's $d$

A standardized metric of effect size calculated as $d = \frac{\bar{X} - \mu}{\hat{s}}$, comparing the sample mean to the population mean divided by the standard deviation.

Cohen's $d$ Benchmarks

Established effect size thresholds where $d = .2$ is a small effect ($83\%$ overlap), $d = .5$ is a medium effect ($67\%$ overlap), and $d = .8$ is a large effect ($53\%$ overlap).

Statistical Power

The probability of correctly rejecting the null hypothesis when $H_0$ is false ($1 - \beta$), indicating the likelihood of detecting a real effect if it exists.

Type I Error ($\alpha$)

A false-positive conclusion that occurs when a researcher rejects a true null hypothesis.

Type II Error ($\beta$)

A false-negative conclusion that occurs when a researcher fails to reject (retains) a false null hypothesis.

Power Analysis

A planning procedure used to determine the minimum required sample size by balancing effect size, sample size, significance level ($\alpha$), and power ($1 - \beta$).