Week 2a: Research Design Concerns

Vocabulary flashcards covering core concepts of research design, statistical inference, probability distributions, errors, power analysis, and study design formats.

what is the basis for much, if not most, statistical testing of data?

the assumption of a normal distribution (the significance of results is based on the probability distribution of “normal” curve)

Sampling Error

The difference between the mean and standard deviation of a random sample and those of the population from which it was drawn.

Central Limit Theorem

A theorem stating that if random samples of size $n$ are taken from a population with mean = μ and SD = σ, as n increases, the mean of the sampling distribution approaches normality with the population mean, regardless of the population distribution ie even if not normally distributed (important for inference)

based on the central limit theory, what assumptions can be made about how likely a sample is to reflext the population based on the size of the sample?

as size of sample increases, they are more likely to reflect the population (limits to reasonable collection)

if we assume a normal distribution of them sample, what is the probability that an individual from the population will have a value of parameter of interest between plus or minus 1 SD?

68% of the time

what is the confidence interval based on convention?

95%

if the CI difference contains 0, what does that tell you?

we cannot be sure enough that there is a big difference, can’t really say there is really a difference

if there is a risk ratio contains 1, what does that mean?

there is no significant difference in risk between groups.

are you more prone to finding a false positive when sampling in smaller sample sizes or large samples?

smaller sample sizes

Control

The need to minimize bias, variability, or confounding in a study.

What are common areas of control in a study?

• subject recruitment methods

• validity of measures

• collection and recording of data

• communication among investigators and subjects

what is important for study assignment?

assignment should be randomized- subjects should have an equal chance of going into either group

what is important for minimizing error due to communication among investigators and subjects?

blinding

t-test

A statistical test used to compare the means of two groups to determine if they are significantly different from each other (using t-distribution). CI is also calculated slightly differently

Type I Error

Concluding that a difference exists when in fact there is no difference (a false positive), represented by the probability $α$. (I assume dif, I was wrong)

Type II Error

Concluding that no difference exists when in fact there is a difference (a false negative), represented by the probability $β$. Tends to be inversely related to alpha

Alpha Level ($α$)

The predetermined probability that a difference or relationship occurred due to chance, most commonly set to $0.05$ ($95$) or $0.01$ ($99$) to establish statistical significance.

Power

The likelihood that one will detect a difference or relationship when one exists, mathematically defined as $1 - β$. Is mostly considered a problem if you don’t see an effect. (beta is often made to be 0.2 which implies we are less concerned about type II vs type I error)

what are some methods for increasing power in design?

increasing sample size, control extraneous variables and apply methods consistently, use homogenous groups (usually these factors alter effect size)

Effect Size

A statistical expression of the size of the difference between sample means. Is calculated as actual differences or standardized differences

Cohen's d

A specific standardized effect size calculated for the difference between two sample means.

equation: (mean score of group 1 - mean score of group 2)/ pooled standard deviation ((which equals signal/noise))

what does a Cohen’s value of greater than 0.8 mean?

big treatment effect

what does a Cohen’s value of between 0.5 and 0.8 mean?

big enough treatment effect that we can see it “with the naked eye”

what does a Cohen’s value of 0.2-0.49 mean?

treatment effect is small enough that we can’t see it with the naked eye

what does a Cohen’s value of less than 0.2 mean?

treatment effect is negligible or trivial

although you can use standardized effect sizes to plan a study and perform sample size estimate, what is a better method?

use effect sizes generated from pilot data

A Priori Power Analysis

A power analysis conducted during study planning to determine the required sample size based on an estimate of effect size, pre-established $α$, number of groups, and $N$.

A Posteriori Power Analysis

A power analysis conducted after a study to evaluate observed power for tests of significance and assess whether a Type II error occurred if no effect was observed.

Experimental Design

A study design format characterized by high control, randomization, manipulation of an intervention, and the inclusion of a control group. Tend to be the most restrictive

Quasi-Experimental Design

A study design format that purposefully manipulates an intervention but lacks control of subject randomization or a control group. overall a lower level of control

what are design formats possible for quasi experimental studies?

• time series format

• nonequivalent control group format

• single system (subject) design

Non-Experimental Design

A descriptive study design format that reports findings, group differences, correlations, or variance without manipulating an intervention.

Cohort Design

A study design where a group sharing a common characteristic is followed over time, often used for questions regarding prognostic factors. (can be prospective which offers more control and retrospective with assures that the outcome has occurred)

Case-Control Design

A retrospective study design where subjects with a specific outcome are compared to a control group free of that outcome to identify risk factors.

Randomized Controlled Trial (RCT)

A study design providing the highest levels of control, featuring two or more groups to which subjects are randomly assigned, with outcomes measured before assignment and after intervention. (one group gets intervention and one does not/gets standard intervention and the results are compared at the end for statistical significance of differences)