AP Stats Unit 2: Sampling & Experimental Design

Population

the entire group of individuals we want information about

Census

a complete count of the population/when you gather other info about the population

Why wouldn’t we do a census all the time?

1. Not always accurate

2. Very expensive/time consuming

3. Perhaps impossible

4. If using destructive sampling it would destroy the population (ex: lifetime of batteries)

Sample

a part of the population we actually examine in order to gather information; used to make inferences about the population

Sampling Design

the method used to choose the sample from the population

Sampling Frame

a list of every individual in the population

Simple Random Sample (SRS)

every individual (or set of individuals) has an equal chance at being chosen

pros: unbiased, easy to design

cons: more variability (than stratified), must have sampling frame (list of every individual)

Stratified Random Sample

population is divided into homogenous groups called strata (groups that are alike based on a common characterisitc and then are pulled from each strata

pros: unbiased, less variability, easy if strata already exists

cons: difficult if you must divide stratum, must have sampling frame

Systematic Random Sample

sample members are selected according to a random starting point (between 1 and x) and a fixed, periodic interval (x) between sucessive sampling units. basically randomly pick a number between 1-10 and chose the every 10th sample member from that

pros: unbiased, don’t need sampling frame, easier/more efficient

cons: more variability (than stratified)

Cluster Random Sampling

randomly pick a location (or a few locations) and sample all from those locations; selecting multiple locations reduces variability

pros: unbiased, don’t need sampling frame, more efficient

cons: more variability, clusters may not be representative of population

Hat method (when describing sampling designs)

Remember to “mix” and “randomly select” and “without replacement”. Number the populatoin when necessary, but use names if possible

Random number generator method (when describing sampling designs)

“number the population _-_” , give range of values for the generator, select “unique” numbers or “ignore repeating numbers", and then “survey the people with the corresponding numbers”

Bias

a systematic error in the sampling procedure that results in a statistic being consistently smaller than the parameter the statistic is used to estimate

• might be attributed to the reserchers, respondents, or sampling method

• cannot do anything with bad data (except lie to ppl and manipulat them)

Voluntary Response Bias

people “select” themselves to participate in the study; they are not randomly selected; they are VOLUNTEERS

• usually only people with very strong opinions respond (this is a side effect, not the definition)

• ex: online poll

Nonresponse Bias

occurs when individuals randomly for the sample can’t be contacted or refuse to cooperate

• the respondents and nonrespondents could differ significantly in ways that are important for the study

• one way to help with this issue is to make follow up contact w/ people who don’t answer the first time

• easily confused w/ voluntary response but for nonresponse ppl r randomly selected while for voluntary they r self selected. nonresponse and voluntary response CANNOT both happen at the same time

• ex: telephone survey

Convenience Sampling

when you ask people who are easy to ask; when it’s convenient but not random and produces biased results

• nonrandom sampling methods (ex: samples chosen by convenience or voluntary response) introduce potential bias b/c they don’t use random chance to select the individuals

• stopping friendly ppl @ the mall; magazine surveys

Under Coverage Bias

may occur when the sampling meethod fails to include part of the population or a part of the population is less likely to be selected based on the sampling method

• ex: sampling ppl on Allen’s facebook group (gen z, ppl w/out facebook left out)

Response Bias

may occur when responses to a survey or measurements of observational units tend to differ from the “true” value in one direction

• response bias examples include questions that are confusing or leading (question wording bias) or self-reported responses

• ex: having the principal ask high schoolers if they cheat on exams

Wording of the Question Bias

occurs when the wording of the question influence the answers that are given; a type of response bias; questions should be neutral to avoid influencing the responses, and the level of vocabulary should be appropriate for the level you are surveying

• ex: using SAT vocab w/ elementary students

Statistical Study

a study in which data are collected from a sample to answer an investigative question about a larger population. Statistical studies are necessary when the population is too larrge or it is too difficult to collect data from every item or individual in the population

Observational Study

observe outcomes w/out imposing any treatment. The researcher records the values of the variables of interest in order to explore an investigative question of interest. Includes: prospective study, retrospected study, survey

Prospective Study

observational units of study are selected at a point in time, and data are gathered both at the time and into the future

Retrospected study

one in which the observational units of study are selected at a point in time and then data from the past are gathered

Survey

an observational study in which the data are collected from humans using a standard set of questions

Observational Unit

an item or individual from which a datum is collected

Experiment

a statistical study where the researcher actively imposes a randomly assigned treatment in order to observe the response so that they can explore an investigative question

Experimental Unit

the single individual (person, animal, plant, etc.) to which the different treatments are randomly assigned (a type of observational unit)

Factor (Explanatory Variable)

what we test/what we change/what we give to the observational units

Level

a specific value or types for the factor

Treatment

a specific experimental condition applied to the units; same as levels when there is one factor; a combination of the different levels when there are multiple factors

Response Variable

what you measure or recrod at the end of the statistical study

Control Group

a group that is used to compare the factor against; can be a placebo or the old/current item; counts as one of the levels. Not all experiements need a control gropu as long as there are at least two treatments to compare

Placebo

a “dummy” treatment that can have no physical effect; not required in every experiment

Placebo effect

the difference between the average response to a placebo and the average response to no treatment

Single blind

method used so that units OR evaulators do not know which treatments the units are getting

Double Blind

neither the units nor the evauluations know which treatment a subject recieved

A well designed experiment has the following 4:

1. Comparisions of at least 2 treatments groups (one of which could be a control group)

2. Random Assignment

3. Replciation of the experiment on many subjects to quantify the natural variation (at least 30 ish units)

4. Direct control of potential extraneous sources of variation

Extraneous variable

a variable that is known (or believed) to affect the response but isn’t an explanatory variable being studied

Confounding Variable

provides an alternative explanation for the observed relationship between an explanatory and response variable, thereby preventing the researcher from proving a causal relationship

• MUST be associated with the explanatory and response variable

• It’s the pre-existing condition that makes the subject choose the factor and influences the response

• You must connect the confounding variable to the explanatory variable AND the response variable

• Random assignment are used to reduce the effect of extranous variables → experiments have no confounding variable → can be used to show causation

Completely Randomized Experimental Design

experimental units are assigned completely at random to treatments. often the number of experimental units assigned to each treatments will be the same (not required though)

Randomized Block Experiment Design

experimental units are first blocked into homogenous groups and then randomly assigned to treatments (units should be blocked based on a varriable that affects the response)

• purpose: seperate the variation in the response caused by the blocking variable from the rest of the extraneous variation in the experiment

• should chose a blocking variable that would have a significant effect on the response variable

Matched Pairs Design

a special type of block design (2 methods)

1. match up experimental units according to similar characteristics and randomly assign one to treatment A and the other automatically gets treatment B

2. have each experimental unit do both treatments in a random order

• the assignment of treatments is dependent

What do larger samples do?

produce statistics with less variablility but don’t affect bias