Lecture 8: selecting research participants

define “population”

a group sharing some common characteristics

define “sample”

subset of the population

define “sampling”

process of selecting participants for a research project

define “sampling error”

naturally occurring differences between the population and a sample

define “representativeness” of a sample

how closely your sample resembles the population

define “representative sample”

a sample with the same characteristics as the population

define “biased sample”

sample with different characteristics from those of the population

define “selection bias” or “sampling bias”

when participants are selected in a way that increases the probability of obtaining a biased sample

[small/big] samples have more chances of being unrepresentative of the population

small

why do we want to select samples that represent the population?

so that the study can be generalized to the population

what’s the difference between the target population, the accessible population and the sample?

• target: all the individuals with the characteristic you want to study

• accessible: portion of the target population, people who are accessible to be recruited

• sample: people who are selected to participate

why don’t we include everyone in the population for our study? (6)

• not possible

• population too large

• too costly

• time consuming

• hard

• not necessary

what are some types of population records? (8)

• census, residential population

• census, working population

• birth records

• hospitalization records

• education records

• mobile phones

• crime records

why don’t we use population records to help us sample? (3)

• some population members are unknown

• it can be hard to reach all members with equal probability

• can be expensive and time consuming to enrol all members

true or false: the sampling is still considered random if you favour the inclusion of certain people over others

false

define “law of large numbers”

the bigger your sample size, the more accurately it will represent the population

according to the law of large numbers, how many participants should we try to recruit and why?

• we try to recruit 30 participants ish

• you want the sample mean to share characteristics with the population mean

• as the sample size increases (x), you start to get more accuracy between the sample and the population mean

• after 16 participants, it starts to plateau

ethically speaking, why shouldn’t your sample be too small or too large?

• too small: not enough participants, won’t be successful → waste of resources

• too large: use too many people when not needed

define “statistical power”

formula that identifies the minimum number of participants needed to detect an expected effect in a study

why do we only use statistical power in lab studies and not in surveys?

because surveys usually require more participants to get an accurate sample that reflects the population

define “probability sampling”

based on random sampling: each member of a population has equal chances of being selected

define “nonprobability sampling”

not randomly selected: each member of a population does not have equal chances of being selected

what’s the difference between probability and nonprobability sampling? (definition, randomness/bias, knowledge of population size and chances of selecting someone)

probability: everyone has equal chances of being sampled

• random, less risk of a biased sample

• we know the population size

• we know the chances of selecting a certain participant

nonprobability: not everyone has equal chances of being sampled

• not random, more chances of a biased sample

• we don’t know the population size

• we don’t know the chances of selecting a certain participant

for psychology, we usually us [probability/nonprobability] sampling

nonprobability as we don’t have access to population data

what are the probability sampling methods? (5) (description, strength and weakness)

simple random:

• description: random process to select participants from the population

• _{^{strength: fair and unbiased}}

• _{^{weakness: not sure if sample is representative}}

systematic random:

• description: selecting the nth participant from a list after a random start

• _{^{strength: easy, representative}}

• _{^{weakness: not independent or random}}

stratified random:

• description: dividing population into strata and randomly selecting equal numbers from each strata

• _{^{strength: all subgroups are represented}}

• _{^{weakness: not representative because the size strata don’t represent the population}}

proportionate stratified random:

• description: same as stratified, but each strata represents the proportion of the population

• _{strength: good representation}

• _{weakness: some strata are harder to be represented}

cluster sampling:

• description: instead of sampling one by one, we sample groups that already exist

• _{^{strength: quick to get large sample, easy}}

• _{^{weakness: not much independence of score}}

define “cluster sampling” (description, strength, weakness)

• description: instead of sampling one by one, we sample groups that already exist

• strength: quick to get large sample, easy

• weakness: not much independence of score

what’s the difference between stratified random sampling and proportionate stratified random sampling?

• stratified: we select equal numbers from each strata for the sample

• proportionate stratified: we select proportionate numbers to the population from each strata for the sample

define “combined-strategy sampling”

combining two or more sampling strategies to have a better representation of the population

explain “simple random sampling” (description, strength, weakness)

• description: random process to select participants from the population, everyone has equal chances to be included

• strength: fair and unbiased

• weakness: not sure if sample is representative

how can simple random sampling be unrepresentative of the population?

it is unlikely, but you can have a distorted sample, especially if you have a small sample

what are the methods of random sampling? (2)

• with replacement: we select and record you as a sample member and return you to the population before the next selection

• without replacement: we select you and remove you from the population before the next selection

sampling with replacement is considered as [independent/non-independent] because […] while sampling without replacement is considered as [independent/non-independent] because […]

• with: independent because the odds of the previous choice don’t affect the odds of the current choice (1/6 → 1/6 → 1/6)

• without: non-independent because the outcome of the previous choice affects the odds of the current choices (1/6 → 1/5 → 1/4)

sampling [with/without] replacement is considered biased because […]

• without

• it bases you towards the next person (you had less chances in the beginning)

if we have a small population, we prefer sampling [with/without] replacement because […]

• with

• you want the chances of choosing someone to be consistent

why is it okay to use sampling without replacement if the population is large?

the outcome of 1÷(N-1) is pretty similar to 1÷N

bootstrapping datasets are used for sampling [with/without] replacement while cross-validation studies are used for sampling [with/without] replacement

• bootstrapping: with replacement

• cross-validation: without replacement

explain “systematic random sampling” (description, strength, weakness)

• description: selecting the nth participant from a list after a random start have do intervals

• strength: easy, representative

• weakness: not independent or random

why does systematic random sampling violate independence?

because once you’ve chosen the interval and who you start with, then the entire sample is already chosen

why does systematic random sampling have a high degree of representativeness?

because it’s ordered

[simple random/systematic] sampling has higher degree of representativeness

systematic

true or false: systematic sampling is a type of simple random sampling

true: it just has a few extra rules

why is systematic sampling less random than simple random sampling?

because it violates the principle of independence (meaning that it’s less random)

explain “stratified random sampling” (description, strength, weakness)

• description: dividing population into strata and randomly selecting equal numbers from each strata

• strength: all subgroups are represented

• weakness: not representative because the size strata don’t represent the population

explain how we do stratified sampling

• identify the key strata from the population

• participants are chosen by taking a random sample in each strata

• we select equal numbers for each strata

• we combine all the strata into one sample

explain “proportionate stratified random sampling” (description, strength, weakness)

• description: same as stratified, but each strata represents the proportion of the population

• strength: good representation

• weakness: some strata are harder to be represented

simple random, systematic random, stratified random or proportionate stratified random: researchers test which of 3 teaching methods best increases 5th graders’ vocabulary. They identify the number of classrooms teaching each method (Method1 = 40%; Method2 = 30%; Method 3 = 30%) and sample a proportionate number of students from classes offering each method.

proportionate stratified

simple random, systematic random, stratified random or proportionate stratified random: researchers identify age-related changes in memory. They sample every 7th person from a census list of residents after choosing a random starting point until they reach 100 participants.

systematic random

simple random, systematic random, stratified random or proportionate stratified random: researchers identify age-related changes in memory. They divide a population into groups based on age (0-10; 11-20; 21-30; etc) and then choose a random sample from each of those groups.

stratified random

simple random, systematic random, stratified random or proportionate stratified random: there is a new mosquito-carried disease harming babies. Researchers have developed a new vaccine to test with babies, who are 15% of the local population. The researchers sample 100 babies from local communities to administer the vaccine.

simple random

what are the nonprobability sampling methods? (3) (description, strength and weakness)

convenience:

• description: sample by selecting participants who are easy to get

• strength: easy

• weakness: biased

quota:

• description: sample by identifying subgroups to be included and then establishing quotas each individual to be selected through convenience from each subgroup

• strength: more control on the composition of a convenience sample

• weakness: biased

snowball:

• description: participants refer other people who are willing to participate

• strength: easy

• weakness: biased and less representative

define “convenience sampling” (description, strength, weakness)

• description: sample by selecting participants who are easy to get

• strength: easy

• weakness: biased

how can convenience sampling be biased?

because it’s only composed of people who came in contact with the experimenter

how can you limit the loss of validity during convenience sampling? (2)

• use a large sample with diverse people

• in your report, have a detailed description of your sample so that people can make inferences about generalizability

define “quota sampling” (description, strength, weakness)

• description: sample by identifying subgroups/strict numbers to be included and then establishing quotas each individual to be selected through convenience from each subgroup

• strength: more control on the composition of a convenience sample

• weakness: biased

explain how quota sampling is done

• we identify the relevant category of people

• we select a sample size for each category based on the predetermined number of participants

• we establish quotas for each subgroup to reflect the population or for each subgroup to be equally represented

• we recruit the participants out of convenience

what’s the difference between proportionate stratified random sampling and quota sampling?

• proportionate strata: we want to perfectly represent the proportion of the population, uses random sampling

• quota: we want to be as close as possible, but not perfectly, sampling isn’t random

→ both want to be representative of the population, but strata is more scientific

why isn’t quota sampling random?

because the odds of the next person being chosen depends on the quota and the proportion (still selecting by convenience)

why is quota sampling still biased?

because not everyone has the same chances of being in the sample

explain “snowball sampling” (description, strength, weakness)

• description: participants refer other people who are willing to participate

• strength: easy

• weakness: biased and less representative

why is snowball sampling biased and not representative?

because people know each other

convenience, quota, snowball: researchers want to know whether university students are reporting higher rates of math anxiety if they are pursuing a Bachelor of Arts or a Bachelor of Science major. They offer participation for extra credit in a psychology course that enrolls students from both programs, and students sign up to participate.

convenience

convenience, quota or snowball: researchers want to know about personality differences among people who drive premium luxury cars and those who drive economical (less expensive) cars. The premium luxury car owners are harder to sample, so the researchers ask the participants in that category to refer their friends who also own luxury cars.

snowball

convenience, quota, snowball: researchers studied the prevalence of discrimination toward immigrant students in a university. In order to recruit more broadly from participants who may remain hidden to avoid discrimination, the researchers recruited family members of each participant.

snowball

convenience, quota, snowball: A researcher wants to survey individuals with different employment status on how likely they are to purchase smartphone brands. The known employment rates in the city are 90% employed and 10% unemployed. The researcher recruits 90% of their sample from employed participants and 10% from unemployed participants.

quota