EBP EXAM 2: SAMPLING

sampling

a Rigorous process involoving selecting representative units of a population

where would you find sampling in a research article

methods section

sampling is an essential what

element to solvingresearch problems

being careful when sampling make sure to not

select too small of a sample= Nonrepresentative

• BIG deal in research

sampling concept: Popularion

A well-defined “set” with specific properties

sampling concept: target population

The entire set of cases to generalize about

sampling concept: accessible/available population

Must meet the target population criteria

sampling strategy steps

step1: identify target population

step 2: delineate the accessible population

step 3 develop a sampling plan

step 4: obtain approval from institutional review board

sampling is a process of

electing a portion/subset of the defined population

sample

a set of elements that make up the population

element

the most basic unit about which information is collected

inclusion (eligible) and exclusion (ineligible) criteria

◦ Defining criteria to include or exclude a subject from a study

◦ Rationale MUST be explained by the literature

what is the goal of inclusion and exclusion criteria

homogenous sample

what form of control is Inclusion (eligible) and exclusion (ineligible) criteria

a form of control to minimize extraneous variables or bias

how does Inclusion (eligible) and exclusion (ineligible) criteria minimize ectraneous variables or biases

◦ Limit the strength of evidence depending on study design

◦ Affects findings accuracy

◦ Limits generalizability

Representativeness

key characteristics closely match those of the entire population

Generalizability

making sure the sample can apply t the larger unstudied sample

what is the purpose of sampling and samples

efficiency

what do the samples have to be

representativness and geralizations

smapling and sample types

1. non probability

2. probability

probability sampling

Uses a form of random selection

nonprobability sampling

Participants chosen in a nonrandom way

in nonprobability sampling group inclusion is

NOT obtained randomly

• Less generalizable

• Less representative

nonprobability sample types

1. Convenience

2. Quota

3. Purposive

probabilty sampling uses what type of selection

random selection which gives equal and independent chance

what is the stringest sampling strategy

probability sampling

what is probability sampling used is

experimental (RCT & quasi-experimental)

convience sampling

most readily accessible persons/objects,

easy to obtain subjects

convience sampling: self selecting volunteer

representativeness is questionable

conveninece nonprobability sampling usage

QUANT nonexperimental or QUAL

which samples have the greatest risk of bias

convenience sampling

convenience sampling is the NP

Weakest sampling method unless then randomly assigned

what must researchers do when using convenience sampling NP

cautiously interpret data and applicability of findings

quota sampling NP

Eligible subjects are consecutively enrolled until the targeted sample size is met.

what is used to build sample representativeness in quota sampling NP

Characteristics of the population of interest & review of literature

how to enroll people into the sample NP

Strata of the population is identified and then, that proportion is matched in the sample

inclusion criteria of quota sampling should be NP

related to variables of interest that Potentially affect the outcomes

pro of quota sampling NP

Increases representativeness & decreases over -OR- underrepresenting

purposive sampling NP

Deliberately select participants based on specific criteria relevant to the study

purposive sampling difficulty NP

Difficult or impossible to locate in other ways (social networks/work/friends): “referral”

Stratified Random OR Random Quota (probability sample)

Divide population into homogeneous strata/ subgroups THEN randomize to meet the QUOTA/ STRATA

Stratified Random OR Random Quota (probability sample) use

Used in experimental (RCT & quasi-experimental)

Stratified Random OR Random Quota (probability sample) vs simple random

More representative than simple random

Stratified Random OR Random Quota advantages

◦ Not subject to conscious biases

◦ Maximizes representativeness of sample

◦ CHANCE is reason for characteristic differences of sample and population

◦ INVERSE relationship between the sample size and the probability of choosing a NON-representative sample

◦ If you have a disproportionately SMALL stratum, oversampling adjusts for underrepresentation

Stratified Random OR Random Quota disadvantages

◦ Enrollment is challenging

◦ Can be HARD to obtain a FULL population list with critical variables (e.g., type

of degrees)

◦ Time consuming

◦ Costly

Multistage (cluster) Sampling (probability sample)

A successive random sampling of units (clusters) that progress from large to small meeting the criteria

Multistage (cluster) Sampling (probability sample) advantages

More ECONOMICAL: TIME & MONEY

Multistage (cluster) Sampling (probability sample) disadvantages

More sampling errors than a. simple random OR b. stratified random/random quota

Statistical data from cluster samples is COMPLEX.

what is a power analysis

Estimation of sample size

what happens without power analysis

have LESS confidence in the findings

what happens if a power analysis isnt done

Study findings are WEAKENED and less GENERALIZABLE if it is not done.

how you statistically validate a sample size.

power analysis

why is a power analysis usually done

to obtain a 95% confidence interval

• (p < .05)

• Need to account for 25% (add 15% in power analysis p.248)

attrition

the gradual loss of participants, data points, or study units over the course of a study. It is especially common in longitudinal studies, clinical trials, and multi-wave surveys where participants drop out, move away, lose interest, or fail to complete follow-up measurements

• subject mortality or loss to follow-up

What do researchers do to account for the potential loss of patients throughout the study?

Over-sampling (Recruiting extra participants) and Statistical adjustments: If patients do drop out, researchers use specific statistical methods (like Intent-to-Treat analysis or data imputation) to analyze the data fairly without biasing the final results

Does this change the power analysis numbers?

attrition changes power analysis numbers by reducing the effective sample size, which lowers your statistical power or increases your Minimum Detectable Effect (MDE) if left unaccounted for.

How does high attrition impact the study's findings?

reducing statistical power and introducing attrition bias and affects internal and external validity

selection bias

Selection bias occurs when we do not have a representative sample in the subjects we choose and do not represent the community well. Best way to avoid selection bia is randomly assigning subjects to groups

What is the purpose of describing samples at baseline (after enrolling eligible subjects but before the study is started)

• Establish starting points: Measure participant characteristics, demographics, and clinical outcomes before any intervention begins to track change over time.

• Characterize the study population: Provide readers and regulators with a clear picture of who actually enrolled in the study (generalizability and transparency).

• Account for variability: Control for individual differences and improve the precision of statistical analyses (e.g., using pre-specified covariates or ANCOVA).

What is the concern if describing samples at baseline is not done?

• Inability to measure change: Without knowing where participants started, researchers cannot prove that an intervention—rather than natural variation or external factors—caused the final outcome.

• Weakened credibility: Lack of baseline documentation diminishes the validity of study claims for retailers, consumers, or regulatory bodies.

• Unaccounted confounding: In non-randomized or observational studies, skipping baseline measurements hides pre-existing group differences that could completely skew the results.

What does it mean if the groups are statistically significantly different at baseline? in RCT’s

• It is usually just random chance: If proper randomization and allocation concealment were used, baseline differences are expected to occur occasionally by random variation.

• Significance testing at baseline is discouraged: Major guidelines (such as the CONSORT statement) discourage running p-tests on baseline characteristics to see if randomization "worked". A non-significant p-value (p > 0.05) does not prove groups are equal, and a significant one (p < 0.05) does not mean randomization failed or that you must automatically adjust for the variable.

• What you should do instead: Look at the substantive size of the difference (e.g., using normalized differences) rather than a p-value, and rely on pre-specified prognostic covariates outlined in your protocol rather than reacting to post-hoc baseline imbalances.

What does it mean if the groups are statistically significantly different at baseline? in non-randomized studies

• It signals selection bias or confounding: Because participants were not randomly assigned, a baseline difference often means the groups were inherently different to begin with (e.g., one group was older, sicker, or had higher baseline severity).

• It threatens internal validity: Any difference you see at the end of the study might be caused by these pre-existing baseline gaps rather than your intervention.

• What you should do instead: Use appropriate analytical strategies—such as multivariable regression adjustment, ANCOVA, propensity-score matching, or weighting—to account for measured confounders, while keeping in mind that unmeasured confounding may still remain.

statistical significance

When a researcher tests a hypothesis and finds that it is statistically significant, it means that the finding is unlikely to have happened by chance.

terminology for a statistical significant before study

Significance level or Alpha level (alpha)

• the probability of making a type I error (the probability of rejecting a true null hypothesis).

terminology for statistical significance after the study

The P value represents the probability that the results of a statistical test were obtained by chance alone.

The minimum level of significance acceptable for most research

is 0.05. If the researcher sets alpha at 0.05, the researcher is willing to accept the fact that if the study were done 100 times, the decision to reject the null hypothesis would be wrong 5 times out of those 100 trials.

if the researcher wants to have a smaller risk for rejecting a true null hypothesis, the level of significance may be

0.01. In this case, the researcher is willing to be wrong only once in 100 trials.

what is the p value often set at

P<.05 (the result is considered statistically significant).

effect size

establish a large enough sample size to ensure that there are enough subjects in each study group to statistically detect differences between those who receive the intervention and those who do not. This is called the ability to statistically detect the treatment effect or effect size—that is, the impact of the independent variable/intervention on the dependent variable

sample size considerations

◦ Type of research design

◦ Sampling procedure

◦ Formula used for estimating optimum sample size

◦ Degree of precision required

◦ Heterogeneity of the attributes being studied

◦ How often the phenomenon of interest occurs in the population (common vs. rare)

◦ Projected cost of strategy

when should a sample size be determined

BEFORE the study isconducted

what type of sample size should you use for probability and nonprobability sampling

Always to use the largest sample possible

chance for type 2 error increases when

sample size is too small

sampling threats: internal validity

Selection effect or the representativeness

a) bias in subject self-selection

b) researcher bias in selecting

• Attrition and mortality: must check to see if the subjects who remain in the study are different from those who drop out

sampling threats: external validity

Generalizability to other populations or settings