Lecture 5 Sample Size and Confounding

1. Research question and hypothesis

2. How many people do you need to be able to test this hypothesis?

3. Write protocol and get approvals to do study

4. Enroll subjects

5. Collect data

6. Get results

7. Draw conclusions

Steps of a Study

Type 1 (alpha) error

Study power

Effect size you wish to detect - magnitude of difference

Sample size determination

Type I error (alpha)

Typically set at alpha=0.05; there is a 5% chance that we will say there is an association, when there is not

Type II error (beta)

Typically set at beta=0.2; there is a 20% chance that we will say there is no association, when there actually is.

First everyone believed there was an effect, when there wasn’t. Next they believed there was no effect, when there was

Boy Who Cried Wolf caused Type I and II errors, in that order

Study design

Methodology errors can be minimized by attention to…

Chance

errors due to ___ can never be completely eliminated;

Can be estimated by type I and II errors; must be determined during the planning phase of study

How much error can we accept?

Type I error

the observed difference between groups is not a true difference but is due to change instead

false positive

we really don’t want to make this error

usually set at 0.05 —> the researcher is willing to accept a 5% risk of committing this error (falsely concluding that the groups differ when in reality they do not)

Type II error

there is no observed difference between groups, when there is a true difference

usually set at 0.20 —> the researcher is willing to accept a 1 in 5 chance of missing a true difference between groups

Statistical Power

the ability of a study to detect a true difference between groups

probability of NOT making a type II error

= 1 - beta level

Study has an 80% chance of detecting a specified difference in outcome between the treatment groups

Magnitude of Difference

specify before determining the sample size

What difference in outcome would be important for treating patients? What difference would be meaningful to the patient? What difference would justify use of more expensive treatment?

CI or 1-alpha

probability that if 2 samples differ, this reflects a true difference in the 2 populations

power or 1-beta

probability that if 2 populations differ, the 2 samples will show a significant difference

decrease sample size

Type 1 error: the probability of finding a difference between two groups that is not true

Increase alpha —>

increase sample size

Type II error: probability of not finding a difference between the two groups when there is

decrease beta —>

increase sample size

Power (1-beta): the ability to detect a true difference between groups

Increase power —>

increase sample size

decrease sample size

Magnitude of difference

small difference —> 

large difference —> 

Random Error

Type of Error

Influenced by sample size

Quantify using statistical tests

Systematic Error

Type of error

Selection bias, information bias, confounding

Not influenced by sample size

Not assessed using statistical tests

Confounding

Occurs when the measure of association is distorted because it is mixed with the effect of an extraneous factor that is not balanced between comparison groups

Associated with exposure, and independent of exposure, be a risk factor for disease

predictive of disease but not a causal factor

cannot be an intermediate step in the causal pathway

Characteristics of confounding variables

1. confounder must be a risk factor for outcome

2. confounder must be associated with exposure

3. confounder cannot be an intervening variable

A priori criteria method to assess confounding

Must be done before the start of the study to determine potential confounders that need to be measured (review literature)

Data-based method

assessing confounding:

must be completed after data collection

compare measures of association when adjusting for potential confounders and without adjusting for potential confounder

see if estimates differ