AP Statistics Test 2

Random Number Generator (RNG) / Digit Table / Hat Method

Tools for randomly selecting individuals without human bias. RNG: calculator/computer-generated numbers. Digit table: a printed table of random digits, read off in groups and discarding out-of-range values and repeats. Hat method: drawing labeled slips from a container. All three give every individual a chance to be picked, just with different tools.

Bias (Sampling)

A flaw in a sampling method that systematically favors certain outcomes, causing consistent overestimation or underestimation of the true population value (not just random chance variation).

Voluntary Response Sample vs. Convenience Sample

Both are biased, non-random methods. Voluntary: people choose to participate themselves (e.g., call-in polls), overrepresenting those with strong opinions. Convenience: the easiest-to-reach individuals are sampled (e.g., first 200 trees along a road), overrepresenting whichever group is most accessible rather than the population as a whole.

Census

A study that collects data from every individual in an entire population, not just a sample.

Simple Random Sample (SRS)

A sample where every individual (or group of individuals) has an equal chance of being selected. Method: (1) label all individuals with numbers, (2) generate random numbers with an RNG, (3) select unique individuals until reaching the desired sample size, ignoring repeats and out-of-range numbers.

Sample Size Tradeoff

Larger samples are generally more accurate (less variable) but also more costly, time-consuming, or impractical. This tradeoff is why stratified, cluster, and systematic sampling exist as alternatives to a pure SRS.

Stratified Random Sample

Divide the population into subgroups (strata) that share a characteristic relevant to the study, take a separate SRS from each stratum, then combine them. Strata can be based on rows or columns of a dataset, depending on which categories matter. Strata should be homogeneous (similar) within each group but different from each other. Example: stratifying students by grade level when asking about internet trust, since experience and skepticism may vary by grade.

Benefit of Stratified Sampling

Mainly reduces variability: homogeneous strata give more precise estimates than an SRS of the same size, and every key subgroup is guaranteed to be represented. It doesn’t fix bias; the random selection is what keeps bias low.

Good Estimator

A sample statistic with both low bias (centers near the true population value) and low variability (consistent from sample to sample).

Cluster Sample

Divide the population into naturally occurring groups (clusters), randomly select a few entire clusters with an SRS, then survey every individual within the selected clusters. Useful when no full list of individuals exists but a list of natural groups does (e.g., sampling a few city blocks and surveying everyone on them).

Benefit of Cluster Sampling

Mainly cost and practicality: no full list of individuals is needed, and surveying whole nearby groups saves travel and time. It does not reduce bias, and it usually has higher variability than an SRS of the same size because people within a cluster tend to be similar.

Clusters vs. Strata

Clusters are “different within” (heterogeneous) and should each resemble the whole population; strata are “same within” (homogeneous) but differ from each other. Use strata when you want guaranteed subgroup representation and lower variability; use clusters when a full individual list is unavailable or impractical and cost matters more.

Systematic Random Sample

Choose a random starting point, then select every kth individual at a fixed interval, where k = population size ÷ sample size. Example: to sample 120 rooms from a 1200-room hotel, k = 1200/120 = 10, so pick a random start between 1-10, then survey every 10th room. Practical for populations already arranged in a sequence, since it doesn’t need a full numbered list like an SRS.

Undercoverage

Some members of the population are left out of the sampling frame (or are much less likely to be included) before sampling even begins. It is a flaw in the frame, not in people’s behavior (e.g., a phone survey missing people without phones).

Non-Response

An individual was properly and randomly selected for the sample but chooses not to participate. This differs from voluntary response, because these individuals were specifically chosen rather than self-selected.

Response Bias

A systematic pattern of inaccurate answers, caused by: (1) the respondent lying, misremembering, or making up answers; (2) the interviewer, whose age, gender, or tone makes respondents uncomfortable, especially on sensitive topics; (3) question wording (e.g., asking if a town’s population is above/below 20,000 vs. above/below 200,000 anchors very different answers).

Bias and Variability by Method

SRS, stratified, cluster, and systematic samples are random, so all have low bias; they differ in variability and practicality (stratified has the lowest variability, cluster usually the highest). Voluntary response and convenience samples have high bias because they are not random.

Experimental Unit

The individual object, person, or subject that a treatment is applied to and whose response is measured (called a “subject” when it’s a person).

Observational Study

A study that observes individuals and measures variables WITHOUT imposing any treatment. It can show association but cannot prove cause and effect.

Experiment

A study in which researchers impose a treatment on the experimental units and then measure the response.

Treatment

What is done to the experimental units, which may be a specific condition or no action at all (e.g., a control).

Explanatory Variable

The variable that helps explain or predict changes in the response variable; in an experiment, it’s the treatment being imposed.

Response Variable

The outcome being measured; it’s what we think is affected by the explanatory variable.

Confounding Variable

A variable related to both the explanatory and response variables, so its effect on the response can’t be separated from the explanatory variable’s effect. Example: ice cream sales (explanatory) and drownings (response) are both driven by hot weather (confounder).

Why Observational Studies Can’t Show Causation

Confounding variables may be unevenly spread across groups, so we can’t tell whether the explanatory variable or a confounder caused the difference in response.

Random Assignment

Using chance to assign experimental units to treatment groups, which spreads confounding variables roughly equally across groups and makes the groups roughly equivalent before treatment.

4 Key Principles of Experimental Design

(1) Comparison: use two or more treatments (one can be no treatment). (2) Random Assignment: use chance to assign units to treatments. (3) Control: keep all other variables (besides the treatment) consistent across groups. (4) Replication: use enough experimental units to distinguish a real difference from chance.

Control Group

A group that receives no treatment (or a placebo), used as a baseline for comparison.

Placebo Effect

When a fake treatment produces a real response because the subject expects it to work.

Single-Blind

The experimental units do not know which treatment they are receiving.

Double-Blind

Neither the experimental units nor the people administering the treatment know who is getting which treatment. This reduces bias from both sides.

Random Assignment Steps (Be SPECIFIC on the test)

(1) Label each experimental unit with a unique number. (2) Randomize using an RNG, generating unique numbers within the range and ignoring repeats. (3) Assign the first group of units selected to Treatment 1, the next group to Treatment 2, etc.

Factor

A variable that is manipulated and may cause a change in the response variable.

Levels

The different values or settings of a factor (e.g., altitude: low, medium, high).

Treatment (Multiple Factors)

A specific combination of one level from each factor. The number of treatments = (levels of factor 1) × (levels of factor 2) × …

Block

A group of experimental units that are similar in a way that might affect the response (e.g., grade level, age, gender).

Randomized Block Design

Separate units into blocks of similar individuals, then randomly assign treatments WITHIN each block, so treatments are compared fairly inside each block.

Why Use Blocking

It removes the variability caused by the blocking variable, so differences between treatments are easier to detect and results are more precise. Use it when you know a variable strongly affects the response (e.g., grade level affecting SAT scores).

Matched Pairs Design

Units are paired by similarity and randomly assigned (one gets each treatment), or one subject receives both treatments in a randomly assigned order. It is effectively a block design with blocks of size 2.

Perfect Match (Matched Pairs)

A matched pairs design where each subject is paired with themselves and receives BOTH treatments in random order (e.g., the Pepsi challenge: drink Coke and Pepsi, rate each).

Why Use Matched Pairs

Each pair (or each person) is as similar as possible, so differences in response are more likely caused by the treatment, not by differences between individuals. It is especially useful when only a small number of subjects is available.

Randomized Block Example (SAT Prep)

50 students are blocked by grade level. Seniors (20): random assignment → Group 1a gets the online course, Group 2a gets the teacher course → compare SAT scores. Juniors (30): random assignment → Group 1b gets the online course, Group 2b gets the teacher course → compare SAT scores. Finally, combine the results and compare online vs. teacher course.

Sampling Variability

Different samples from the same population give different estimates. Larger samples produce more accurate (less variable) estimates.

Margin of Error

Creates an interval of plausible values around the sample estimate: estimate ± margin of error.

Statistically Significant

A result that is too unusual to have happened by chance alone if the treatment had no effect.

Using a Simulation to Check Significance

Simulate many trials assuming no treatment effect (e.g., 100 trials on a dotplot) and find the proportion of trials as extreme as the actual result. If the proportion is ≤ 5%, the result is statistically significant; if it’s more than 5%, it is not. Example: 2 of 100 trials past the sample result = 2%, so it is significant.

Statistically Significant Conclusion

We have convincing evidence that the treatment caused the difference in response, as long as the study used random assignment.

Resume Experiment Example

Identical resumes were sent out with a male name vs. a female name. A statistically significant difference in callbacks gives convincing evidence of gender bias in who gets chosen, since the resumes were the same.

Random Sampling and Inference

Random selection of individuals from a population allows us to GENERALIZE results to the population that was sampled.

Random Assignment and Inference

Random assignment of units to treatments allows us to conclude that the TREATMENT CAUSED the change in response.