Introductory Statistics Explained: Comprehensive Review

Comprehensive review flashcards covering study design, sampling techniques, descriptive statistics, probability rules, continuous distributions, and sampling distributions from Introductory Statistics Explained.

In statistical inference, what is the fundamental distinction between a parameter and a statistic?

A parameter is a fixed, numerical characteristic of a population (such as the population mean $\mu$ or population variance $\sigma^2$), whose true value is almost always unknown. A statistic is a numerical characteristic calculated from an observed sample (such as the sample mean $\bar{x}$ or sample variance $s^2$), which is used to estimate or draw inferences about the unknown population parameter.

What is the formal definition of a simple random sample (SRS) of size $n$ drawn from a finite population of size $N$?

A simple random sample of size $n$ is a sample selected in such a manner that every possible sample combination of size $n$ has the exact same probability of being chosen. Consequently, every individual in the population has an equal probability, $\frac{n}{N}$, of being included in the sample.

How do stratified random sampling and cluster sampling differ in both design structure and objective?

In stratified random sampling, the population is divided into non-overlapping, homogeneous subgroups (strata) based on known characteristics, and an SRS is drawn from each stratum to guarantee representation and reduce estimator variance. In cluster sampling, the population is naturally divided into heterogeneous clusters; researchers randomly select entire clusters and measure individuals within them primarily to reduce logistical difficulty and cost.

What is systematic sampling (sampling every $k\text{th}$ unit), and under what condition can it yield severe bias?

Systematic sampling consists of randomly choosing a starting unit and subsequently selecting every $k\text{th}$ item in an ordered sequence. While it behaves like simple random sampling if units are randomly ordered, it yields severe systematic bias if there is an underlying cyclical or periodic pattern in the sequence that matches the sampling interval $k$.

Why can a well-designed randomized experiment establish a cause-and-effect relationship, whereas an observational study generally cannot?

In a randomized experiment, researchers actively impose treatments on experimental units that have been randomly assigned to treatment groups, ensuring that any pre-existing differences between groups are due to chance alone. In an observational study, researchers merely observe pre-existing conditions without manipulating treatments, meaning observed associations may be caused by unmeasured lurking variables.

What is the distinction between a lurking variable and confounding in a statistical study?

A lurking variable is an unmeasured variable associated with both the explanatory and response variables that may account for the observed relationship. Confounding occurs when two variables (which can be measured explanatory variables or unmeasured variables) are related in such a way that their separate effects on the response variable cannot be distinguished.

When assessing the precision and reliability of an estimator, why does the absolute sample size ($n$) matter far more than the proportion of the population sampled ($\frac{n}{N}$)?

The precision of estimators depends directly on the absolute sample size $n$ (for instance, the standard deviation of the sample mean is $\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$). Provided the sample constitutes a small fraction of a large population, the total population size $N$ has virtually no influence on estimation variability.

How are nominal and ordinal categorical variables distinguished from one another?

Nominal variables consist of qualitative categories that have no natural hierarchy or ordering (such as mode of transportation: pedestrian, cyclist, motorist, or public transit). Ordinal variables consist of categories that possess a natural, meaningful sequence or rank (such as handgun calibre: small, medium, large; or Likert survey response scales).

How does distribution skewness influence the relative positions of the arithmetic mean and the median?

In a right-skewed (positively skewed) distribution, extreme values in the right tail pull the mean upward, resulting in $\text{mean} > \text{median}$. In a left-skewed (negatively skewed) distribution, extreme values in the lower tail pull the mean downward, resulting in $\text{mean} < \text{median}$. In a perfectly symmetric distribution, the mean and median are equal.

Why is the denominator $n - 1$ used instead of $n$ in the sample variance formula $s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1}$?

Calculating squared deviations around the sample mean $\bar{x}$ rather than the unknown population mean $\mu$ mathematically minimizes the sum of squared deviations, causing $\frac{\sum (x_i - \bar{x})^2}{n}$ to systematically underestimate $\sigma^2$. Dividing by the degrees of freedom $n - 1$ compensates for this bias, ensuring that the sample variance $s^2$ is an unbiased estimator ($E(s^2) = \sigma^2$).

What is Chebyshev's inequality, and how does its scope compare to that of the Empirical Rule?

Chebyshev's inequality guarantees that for any distribution and any constant $k > 1$, the proportion of observations lying within $k$ standard deviations of the mean is at least $1 - \frac{1}{k^2}$ (e.g., at least $75\%$ for $k = 2$). Unlike the Empirical Rule ($\approx 68\%$, $\approx 95\%$, and almost all within $1$, $2$, and $3$ standard deviations), which applies strictly to mound-shaped, approximately normal distributions, Chebyshev's inequality holds for any distribution shape.

If every observation in a sample undergoes a linear transformation $x^* = a + bx$, what are the resulting measures of centre and variability?

Measures of centre undergo the full linear shift and scale: $\bar{x}^* = a + b\bar{x}$ and $\text{median}^* = a + b\,\text{median}$. Measures of variability are completely unaffected by the additive constant $a$: the new standard deviation is $s_{x^*} = |b|s_x$, the new variance is $s^2_{x^*} = b^2 s^2_x$, and the new interquartile range is $\text{IQR}_{x^*} = |b|\text{IQR}_x$.

What is the definition of a sample $z$-score, and what are the mean and standard deviation of all $z$-scores within any dataset?

A $z$-score measures the signed distance an observation falls from the sample mean in units of standard deviations: $z_i = \frac{x_i - \bar{x}}{s}$. Because $z$ is a linear transformation with constants $a = -\frac{\bar{x}}{s}$ and $b = \frac{1}{s}$, the mean of all $z$-scores in any dataset is always $0$, and their standard deviation is always $1$.

How are the box, whiskers, and potential outliers constructed in a standard modified boxplot?

The central box spans from the first quartile ($Q_1$) to the third quartile ($Q_3$), with an interior line marking the median. The upper whisker extends to the largest observed value that does not exceed $Q_3 + 1.5 \times \text{IQR}$, and the lower whisker extends to the smallest observed value not less than $Q_1 - 1.5 \times \text{IQR}$. Any observations falling outside these boundary limits are plotted individually as outliers.

What is the critical distinction between mutually exclusive (disjoint) events and independent events?

Mutually exclusive events cannot occur simultaneously on the same trial, meaning $P(A \cap B) = 0$. Independent events are events where the occurrence of one does not alter the probability of the other, meaning $P(A \cap B) = P(A)P(B)$. If two events with non-zero probabilities are mutually exclusive, they cannot be independent because knowing one occurred means the other cannot occur ($P(A \mid B) = 0 \ne P(A)$).

What is the general addition rule of probability for the union of two events $A$ and $B$, and why must the intersection be subtracted?

The general addition rule is $P(A \cup B) = P(A) + P(B) - P(A \cap B)$. Subtracting the intersection $P(A \cap B)$ is necessary because simply summing $P(A)$ and $P(B)$ double-counts the probability of outcomes that satisfy both events simultaneously.

What is the formal definition of conditional probability $P(A \mid B)$, and how is it interpreted through a reduced sample space?

The conditional probability formula is $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$, provided $P(B) > 0$. Conceptually, conditioning on event $B$ reduces the original sample space strictly to the outcomes belonging to $B$, and $P(A \mid B)$ represents the proportion of that reduced sample space that also belongs to event $A$.

What is the general multiplication rule for finding the probability of an intersection $P(A \cap B)$, and how does it simplify under independence?

The general multiplication rule states that $P(A \cap B) = P(A)P(B \mid A) = P(B)P(A \mid B)$. If events $A$ and $B$ are independent, then $P(B \mid A) = P(B)$, simplifying the formula to $P(A \cap B) = P(A)P(B)$.

What does the Law of Total Probability state for a set of mutually exclusive and exhaustive events $B_1, B_2, \dots, B_k$ and an arbitrary event $A$?

It states that the total probability of event $A$ is the weighted sum of its conditional probabilities across all partitions of the sample space: $P(A) = \sum_{i=1}^k P(B_i)P(A \mid B_i)$.

What is Bayes' Theorem, and what is its role in probability calculations?

Bayes' Theorem states that for mutually exclusive and exhaustive events $B_1, B_2, \dots, B_k$ and an event $A$ with $P(A) > 0$: $P(B_j \mid A) = \frac{P(B_j)P(A \mid B_j)}{\sum_{i=1}^k P(B_i)P(A \mid B_i)}$. It provides a mathematical rule to update the prior probability of a state or cause $P(B_j)$ into a posterior probability $P(B_j \mid A)$ upon observing new evidence $A$.

What is the mathematical difference between permutations and combinations when selecting $x$ items from $n$ distinct items?

Permutations determine the number of ordered arrangements: $P_x^n = \frac{n!}{(n - x)!}$. Combinations determine the number of subsets when order does not matter: $\binom{n}{x} = \frac{n!}{x!(n - x)!}$. Because each combination of $x$ items can be ordered in $x!$ ways, the number of combinations equals the number of permutations divided by $x!$.

What does the Law of Large Numbers dictate regarding the long-run behaviour of repeated trials, and what does it NOT guarantee?

It dictates that as the number of independent trials approaches infinity, the empirical proportion of times an event occurs must tend toward its theoretical probability. It does NOT guarantee that the absolute discrepancy between counts (e.g., $|\text{Heads} - \text{Tails}|$) will diminish; in fact, the absolute difference between counts often increases over time.

Why is the probability of a continuous random variable equaling any single point constant $P(X = c) = 0$?

For continuous random variables, probabilities correspond to areas under the probability density function (PDF): $P(a \le X \le b) = \int_a^b f(x)\,dx$. A single constant $c$ represents a line of zero width, and the definite integral over a single point has zero area: $\int_c^c f(x)\,dx = 0$. Consequently, non-zero probabilities exist only for intervals.

What two conditions must any legitimate continuous probability density function $f(x)$ satisfy?

1. The function cannot take negative values anywhere: $f(x) \ge 0$ for all $-\infty < x < \infty$. 2. The total area underneath the entire curve over its domain must equal exactly one: $\int_{-\infty}^\infty f(x)\,dx = 1$.

How are the continuous probability density function $f(x)$ and cumulative distribution function $F(x)$ related mathematically?

The cumulative distribution function represents the integrated area to the left of $x$: $F(x) = P(X \le x) = \int_{-\infty}^x f(t)\,dt$. Taking the derivative of the CDF recovers the PDF: $f(x) = \frac{d}{dx}F(x)$ wherever the derivative exists, meaning $f(x)$ is the rate of change of the accumulated area.

What are the probability density function, mean, and variance of a continuous uniform random variable $X \sim \text{Unif}(L, U)$?

The PDF is constant over the interval: $f(x) = \frac{1}{U - L}$ for $L \le x \le U$ (and $0$ elsewhere). The mean is $\mu = \frac{L + U}{2}$, and the variance is $\sigma^2 = \frac{1}{12}(U - L)^2$.

How does the Chi-square ($\chi^2$) distribution originate from the standard normal distribution, and what are its mean and variance?

If $Z_1, Z_2, \dots, Z_\nu$ are independent standard normal variables ($Z_i \sim N(0, 1)$), then the sum of their squares $Z_1^2 + Z_2^2 + \dots + Z_\nu^2$ follows a $\chi^2$ distribution with $\nu$ degrees of freedom. Its mean is $\mu = \nu$ and its variance is $\sigma^2 = 2\nu$.

What is the structural definition of Student's $t$ distribution in terms of independent standard normal and $\chi^2$ random variables?

If $Z \sim N(0, 1)$ and $U \sim \chi^2(\nu)$ are independent random variables, then the quantity $T = \frac{Z}{\sqrt{U / \nu}}$ follows a $t$ distribution with $\nu$ degrees of freedom. It is symmetric about $0$ with heavier tails than the standard normal, converging to $N(0, 1)$ as $\nu \to \infty$.

How is the $F$ distribution constructed from independent $\chi^2$ random variables, and what are its parameters?

If $U_1 \sim \chi^2(\nu_1)$ and $U_2 \sim \chi^2(\nu_2)$ are independent random variables, then the ratio of each variable divided by its respective degrees of freedom, $F = \frac{U_1 / \nu_1}{U_2 / \nu_2}$, follows an $F$ distribution with $\nu_1$ numerator degrees of freedom and $\nu_2$ denominator degrees of freedom.

What is the primary function of a normal quantile-quantile (QQ) plot, and how do departures from linearity indicate non-normality?

A normal QQ plot plots observed ordered sample values against corresponding theoretical quantiles from a standard normal distribution. If the data is approximately normal, points fall roughly along a straight line; a downward or upward curve indicates skewness, while an S-shaped pattern indicates tails that are heavier or lighter than those of a normal distribution.

What is the conceptual definition of the sampling distribution of a statistic?

The sampling distribution of a statistic is the theoretical probability distribution that describes the values the statistic would take across repeated independent samples of the exact same size $n$ drawn from the same population.

What are the expected value and variance of the sample mean $\bar{X}$ when drawn from a population with mean $\mu$ and variance $\sigma^2$?

The expected value is $E(\bar{X}) = \mu$, proving that the sample mean is an unbiased estimator of the population mean. The variance of the sample mean is $\text{Var}(\bar{X}) = \frac{\sigma^2}{n}$, giving a standard deviation of $\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$.

What does the Central Limit Theorem (CLT) state regarding the sampling distribution of the sample mean $\bar{X}$?

The CLT states that when drawing random samples of size $n$ from any distribution with a finite mean $\mu$ and finite variance $\sigma^2$, the sampling distribution of $\bar{X}$ approaches a normal distribution $N(\mu, \frac{\sigma^2}{n})$ as the sample size $n$ increases. In practice, $n > 30$ is a common rough guideline for approximate normality.

What is the distinction between the true standard deviation of the sample mean $\sigma_{\bar{X}}$ and the standard error of the sample mean $\text{SE}(\bar{X})$?

The true standard deviation $\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$ is the theoretical spread of the sampling distribution of $\bar{X}$, which is typically unknown because the population parameter $\sigma$ is unknown. The standard error $\text{SE}(\bar{X}) = \frac{s}{\sqrt{n}}$ is the empirical estimate of that standard deviation, computed using the sample standard deviation $s$.

When sampling from a normally distributed population, why is the sample mean preferred over the sample median as an estimator of $\mu$?

Although both the sample mean and the sample median are unbiased estimators of $\mu$ for a symmetric normal distribution, the sample mean has a smaller variance. According to the Cramér-Rao Lower Bound, $\bar{X}$ achieves the minimum possible variance among all unbiased estimators of $\mu$, making it more efficient and precise.