Significance levels and statistical power

What is the term relating to a= 0.05 meaning 5% chance of finding a significant difference when none actually exists (FALSE POSITIVE) - does NOT mean 95% chance the effect is real?

Significance level - a

What does the term relating to 1 - beta mean, which tells us the chance of detecting a real effect of a specified size e.g., 80% power means study has 80% chance of detecting a real effect and 20% Beta chance of missing it?

Statistical power

What is the term to describe a study showing an effect which doesnt exist?

False positive

What is the term to describe a study where an effect was present but the study missed it?

False negative

What is a table showing type 1/false positive and type 2/false negative errors?

What happens to confidence intervals with larger samples?

Confidence interval narrower

What happens to confidence intervals if data is more scattered?

Confidence interval widens

What factors are needed to choose a survey sample framework?

• Determine population of interest

• Estimate number of individuals in the population

• Determine how confident you need to be that results are representative and accurate

• Determine how many surveys need to be distributed, divide the required sample size by anticipated response rate

What is a small effect size?

There is a real effect - something is really happens in the world but can only see through careful study

What is a large effect size?

Effect which is big enough that you may be able to see with the naked eye

What is the term to describe the family of indices that measures the magnitude of treatment effect - key concept is the overlap between two conditions e.g., treatment vs placebo?

Effect size (Cohen’s D)

What will sample size in qualitative studies depend on?

What you want to know, purpose of inquiry, what is at stake, what will be useful, what will have credibility and what can be done with available time and resources

What can you do for qualitative studies with fixed resources?

Choose to study 1 specific phenomenon in depth with smaller samples size or bigger sample size for breath

What is the term to describe collecting data continues until no new themes emerge?

Saturation

What is the name of the error which occurs when some members of the population are excluded or underrepresented e.g., missing rural participants because it uses online questionnaires?

Coverage error

What is the name of the error that happens when individuals who do not respond differ significantly from those who do e.g., people with strong opinions are more likely to respond, creating bias?

Nonresponse error

What is the name of the error which results from inaccurate responses due to poorly worded questions, misunderstanding or interview influence e.g., leading questions or complex wording?

Measurement error

What is the name of the error occurring during data entry, coding or analysis e.g., typing mistakes or misclassifying responses?

Processing error

What is the name of the error where respondents give socially desirable answers rather than truthful ones e.g., underreporting drug use or overreporting healthy habits?

Response bias

What is the name of the error where the interviewers behaviour or characteristics influence responses e.g., respondents alter answers based on perceived expectations?

Interviewer bias

What can be someone’s of bias in epidemiological studies?

• Measurement error - exposure, outcome, confounders

• Loss to follow up in longitudinal studies and selection bias in case control studies

When does selection bias occur?

When there is selective referral or enrolment of cases or controls and selection factor relates to probability of being expoed

What is significance level related to?

How likely we are to claim an effect exists when it doesnt

What is power/sample size related to?

How likely are we to detect a real effect when it truly exists

What is effect size related to?

How big the difference is - is it important to patients?

What does random error affect?

Precision

What does bias affect?

Accuracy