Lecture 10: Sampling and Generalizability

what is sampling

concerned with the selection of a subset of individuals from a population to estimate characteristics of the whole population

what is a population

entire set of people or products in which you are interested

what is a census

a set of observations that contains all members of the population of interest

what type of valididty relates to sampling and how

external validity: how well this claim would generalize

determined by how the sample was selected; is it representative of the larger population?

Case Study: “Half of Canadians spending all or more of their paycheques”

What population does this claim describe?

What kind of sample would you need to select to make a claim like this?

pop.: canadians

sample: must look at people across Canada and not just populated areas such as Toronto; must disregard social class

Case Study: “Goal: investigate ON high school student’s typical transportation methods

A: What is the population of interest?

B: Methods- surveyed students from a primarily white, privileged high school in Waterloo

In what way(s) is this sample biased?

How could this bias impact the conclusions of the study?

A: pop- HS students

B: bias- only looks at white students

bias impact conclusions- may disregard the unprivileged students in Waterloo

what is sampling bias?

when the sample does not accurately represent the population

some members of the population have a higher probability of being included than others

examples of sampling biases

convenience sample

self-selection bias

what is convenience sample

when a certain method of recruitment is easy; ie. when certain individuals are readily available or easier to contact

ex: SONA using PSYCH students

Case Study: “Study Aim: to investigate the functional consequences of living with depression;

Method: go to an inpatient depression clinic and interview willing clients.”

What makes this a convenience sample?

Why might this sample be problematic in answering your research question?

How would this impact the generalizability of the study?

no random selection of all people who have depression; not representative of the entire population that has depression; only selecting patients that are easier to contact (ie. all in that inpatient clinic)

what is self-selection bias+ example

sampling those who “invite themselves”; can bias final conclusion

this occurs the most with internet-based studies; to gain something ($)

ie. A survey titled “opinions on LGBTQ issues”: bias want more random selection

five strategies for getting a representative sample (probability sampling)

1. simple random/ probability sampling

2. systematic sampling

3. cluster sampling & multistage sampling

4. stratified random sampling

5. oversampling

downsides of simple random sampling

you are unaware of who your large sample is

e systematic sampling

random number generator to select systematically from longer list (ie. starting on number 3, select every 7th person)

e cluster sampling and multistage sampling

taking randomized groups from each HS then evaluating them; only do if population is already in clusters (ie. HS students)

two random samples are selected: a random sample of clusters and then a random sample of people within those clusters; ie. list of high schools in the state, then select 100 random schools, then select random sample of students from the selected schools

e stratified random sampling

to retain important charactersitics of population in sample (ie. male: female ratio)

e oversampling+ example

when interested in an important but low frequency category; instead of maintaining correct ratio, change ratio to get certain group

ie. dementia in the canadian household population

5% of individuals 80+ have a dementia diagnosis; only 25 participants in a sample of 500 if maintaining proportion of population

random sampling vs random assignment

random sampling (of participant) improves external validity

random assignment (of conditions) improves internal validity; only improves experiment

e other sampling strategies (validity)

depending on the goals of your study, external validity might not be your top priority

if your focus is a causal claim, external validity is not as important

if this is true, nonrandom sampling is acceptable when you simply want to focus a small causal claim

four types of non-probability sample+ what non-probability sample is

some types of people are systemically left out

1. convenience sample

2. purposive sample

3. snowball sampling

4. quota sampling

e purposive sampling

targeting a specific groups for recruitment (ie. only choosing women in a small group of people)

e snowball sampling

relying on participants to help you find more participants

occurs in rare population; ie. AA groups

e quota sampling

set target number for each category; non-randomly sample until quota in each category is met (ie. 3 males, 3 females)

sample size; what is best?

large samples are genrally bestter; more representative of population

Case 1: Dr. Barr is conducting a study that investigates whether laptop use in undergraduate lectures affects recall of course content. She looks at the course offerings for the term and selects 4 courses to sample from. She then samples 30 students from each of the 4 courses for participation in his study.

What sampling techniques were used

cluster and multistage sampling

Case 2: Dr. Barr is conducting a study that investigates whether laptop use in undergraduate lectures affects recall of course content. She looks at the course offerings for the term and selects 4 courses to sample from. She then samples 30 students from each of the 4 courses for participation in his study.

What sampling techniques were used

stratifeid since it focuses on the ratio of men and women

Case 3: Dr. Barr is conducting a study that investigates whether laptop use in undergraduate lectures affects recall of course content. She looks at the course offerings for the term and selects 4 courses to sample from. She then samples 30 students from each of the 4 courses for participation in his study.

What sampling technique was used

oversampling (focus is low frequency category)

Exercise: Imagine that you are planning to estimate the price of the average book at your university bookstore. The bookstore carries 13,000 titles, but you plan to sample only 200 books. You will select a sample of 200 books, record the price of each book, and use the average of the 200 books to estimate the average price of the 13,000 books in the bookstore.

A.What is the sample in this study, and what is the population?

B.How might you collect a simple random sample of books?

C.How might you collect a stratified random sample?

D.How might you collect a convenience sample?

E.How might you collect a systematic random sample?

F.How might you collect a cluster sample of 200 books?

G.How might you collect a quota sample?

A: 200 books; all 13,000 books at uni bookstore

B: assign number to book, randomly select

C: genre ratios still randomly remain

D: people volunteer to say how much their books cost

E: choose every 222nd book

F: identify genre, randomly select sample

G: pick # if you want, choose until its met (ie. 50 psych, 20 soc, 10 math books, etc)