Lectures 9–12: Probability, Uncertainty, Hypothesis Testing, and Regression

Comprehensive vocabulary flashcards defining core terms, rules, and mathematical concepts across probability, sampling uncertainty, hypothesis testing, and regression analysis.

Experiment

An action producing random outcomes.

Sample Space ($\Omega$)

The set of all possible outcomes of an experiment.

Event

A subset of possible outcomes.

Probability

The likelihood of an event occurring, bounded between $0$ and $1$.

Mutually Exclusive

A property describing two events that cannot occur together.

Independent Events

Events where knowing that one event occurred does not change the probability of the other occurring, satisfying $P(A \cap B) = P(A)P(B)$.

Addition Rule

The rule used to calculate the probability of $A$ OR $B$, given by $P(A \cup B) = P(A) + P(B) - P(A \cap B)$, where subtraction prevents double-counting overlap.

Permutations

Arrangements of objects where order matters, calculated as $nP_k = \frac{n!}{(n-k)!}$.

Combinations

Selections of objects where order does not matter, calculated as $nC_k = \frac{n!}{k!(n-k)!}$.

Joint Probability

The probability that event $A$ AND event $B$ both occur, denoted $P(A \cap B)$.

Conditional Probability

The probability that event $A$ occurs given that event $B$ has occurred, calculated as $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$.

Marginal Probability

The probability of event $A$ occurring regardless of $B$, denoted $P(A)$.

Discrete Variable

A random variable that takes on countable values, such as the number of cases.

Continuous Variable

A random variable that takes on values along a continuum, such as BMI.

Bernoulli Variable

A random variable with exactly two possible outcomes: success and failure.

Binomial Distribution

The probability distribution of the number of successes in repeated independent Bernoulli trials.

Probability Mass Function (PMF)

A mathematical function giving the probability of an exact value for a discrete random variable, denoted $P(X = x)$.

Probability Density Function (PDF)

A mathematical function representing the density of a continuous variable, where probabilities correspond to areas under the curve.

Cumulative Distribution Function (CDF)

A mathematical function that gives the cumulative probability that a random variable is less than or equal to a specified value, denoted $P(X \le x)$.

Z-score

A standardized value indicating how many standard deviations an observation is from the population mean, given by $Z = \frac{X - \mu}{\sigma}$.

Population

The entire group of interest in an analysis.

Sample

A set of observations drawn from the population.

Parameter

A numerical characteristic that describes a population.

Estimand

The true population quantity that researchers want to know.

Estimator

The specific procedure or rule used to estimate the estimand.

Estimate

The actual numerical result produced by applying an estimator to data.

Sampling Distribution

The probability distribution of an estimator over repeated random samples from a population.

Statistical Inference

The process of using sample results to make formal statements or generalizations about a population.

Bias

Systematic deviation of an estimator or estimate from the true population parameter.

Noise

Random variation in an estimate, particularly arising from sampling variation.

Standard Deviation (SD)

A statistic that measures variability among individual observations, describes the spread of data, and does not necessarily decrease as sample size $n$ increases.

Standard Error (SE)

A statistic that measures the variability of an estimator across samples, describes uncertainty in an estimate, and generally decreases as sample size increases, calculated for a mean as $SE = \frac{s}{\sqrt{n}}$.

Degrees of Freedom ($df$)

The number of independent values free to vary in a calculation, defined as $df = n - 1$ for sample variance because estimating the sample mean imposes one constraint.

Central Limit Theorem (CLT)

The theorem stating that as sample size becomes sufficiently large, the sampling distribution of the sample mean approaches a normal distribution, regardless of the population distribution shape.

Law of Large Numbers (LLN)

The principle stating that as sample size $n$ increases, the sample mean converges toward the population expectation, focusing on accuracy and convergence rather than distributional shape.

Confidence Interval

A range of plausible values for a population parameter constructed using a procedure that possesses a specified long-run coverage rate (e.g., $95\%\text{ CI} = \text{Estimate} \pm 1.96(SE)$).

Student's t-distribution

A symmetric distribution that resembles the normal distribution but features heavier tails; used for inference about means when the population standard deviation is unknown.

Null Hypothesis ($H_0$)

A statement of no relationship or difference in the population, typically represented as $H_0: \theta = 0$.

Alternative Hypothesis ($H_1$)

A statement that a relationship or difference exists in the population, typically represented as $H_1: \theta \ne 0$.

Type I Error

A false positive error occurring when a true null hypothesis is incorrectly rejected.

Type II Error

A false negative error occurring when researchers fail to reject a false null hypothesis.

Alpha ($\alpha$)

The probability of committing a Type I error under a hypothesis testing procedure.

Beta ($\beta$)

The probability of committing a Type II error for a specified alternative value.

Statistical Power

The probability of correctly rejecting a false null hypothesis, defined as $1 - \beta$.

P-value

The probability, assuming the null hypothesis is true, of observing a test statistic at least as extreme as the one obtained.

Two-Sample t-test

A hypothesis test used to evaluate whether the means of two groups differ significantly, utilizing the test statistic $t = \frac{(\bar{Y}_1 - \bar{Y}_2) - 0}{SE_{\text{difference}}}$.

Chi-Square Test ($\chi^2$)

A statistical test used to evaluate whether two categorical variables are associated, calculated as $\chi^2 = \sum \frac{(O - E)^2}{E}$.

BLUE

An acronym describing the OLS estimator under Gauss–Markov assumptions as Best (minimum variance among linear unbiased estimators), Linear, Unbiased, and Estimator.

Omitted Variable Bias (OVB)

Bias that arises when a regression model excludes a variable that is both a cause of the outcome and correlated with an included predictor.

Multicollinearity

A condition in multiple regression where predictor variables are strongly correlated, causing inflated standard errors and reduced precision.

Variance Inflation Factor (VIF)

A diagnostic metric used to detect multicollinearity, calculated as $VIF_i = \frac{1}{1 - R_i^2}$, where values above $5$ indicate substantial shared variation.

Heteroskedasticity

A regression violation where error variance changes across values of the predictor variables, rendering conventional OLS standard errors unreliable.

Autocorrelation

A condition where regression error terms are correlated across observations, invalidating conventional standard errors and reducing efficiency.

Confounding

A threat to causal inference occurring when an outside variable influences both the predictor and the outcome.

Reverse Causality

A threat to causal inference where the presumed outcome variable actually influences the predictor variable.

What is a Monte Carlo simulation?

Repeated random simulations used to approximate probabilities or statistical results

Sampling with vs. without replacement

With: an item can be selected again

Without: each selected item is removed

What defines a normal distribution?

A symmetric bell-shaped distribution defined by mean (mu) and standard deviation

What is an unbiased estimator?

An estimator whose expected value equals the true population parameter

What is a consistent estimator?

An estimator that converges to the true parameter as sample size increases

How does higher confidence affect CI width?

A higher confidence requires a wider interval when other factors stay constant

How does increasing sample size affect standard error?

SE generally decreases; quadrupling n halves the SE of a sample mean

What is a critical value?

A cutoff on a reference distribution used to determine rejection of the null hypothesis

What is a reference distribution?

The distribution of a test statistic under the null hypothesis

What is statistical significance?

A result is significant when its p-value is at or below the chosen alpha

Does statistical significance establish causality?

No. Confounding, reverse causality, and other biases may explain an association

What is a difference-of-proportions test?

A test comparing the proportions of an outcome between two groups

What is Welch’s two-sample t-tes?

A test comparing two means without assuming equal population variances

What is a two-tailed hypothesis test?

Tests for differences in either direction

What is a one-tailed hypotehsis test?

Tests a directional alternative

What is an expected frequency in chi-squared?

Expected count=(row total * column total) / grand total

What are chi-square degrees of freedom?

df=(number of rows - 1) * (number of columns - 1)

How do you test an OLS coefficient

Test null: slope=0 using t=(B^-0)/SE(B^)

When does a 95% CI indicate significance?

For a two-sided test of a difference, when the 95% CI excludes zero

What is the zero conditional error mean assumption?

E(E | X) = 0: The error’s mean is zero at each predictor combination

What is homoskedasticity?

Constant error variacne across combitions of predictor values

What is perfect multicollinearity?

One predictor is an exact linear combination of other model predictors

What is measurement error?

A difference between a variable’s recorded value and its true value

What is the direction of omitted variable bias?

Same-sign confounder relationships: upward bias

Opposite signs: downward bias

What happens to OLS standard errors under heteroskedasticity?

Conventional OLS standard errors may be incorrect, affecting inference.

What is the dummy variable trap?

Including every category dummy and an intercept creates perfect mutlicollinearity

What is the difference between association and causation?

Association means variables vary together; causation mean changing X affects Y

What is the main purpose of regression diagnostics?

To assess model assumptions and identify problems affecting estimates or inference