Validity in experimental design

What is the purpose of experiments

establish cause and effect relationships but manipulating and controlling certain variables an limiting influence of extraneous/cofounding variables

What type of variable is room temperature

extraneous

What type of variable is ice vs pain medication

confounding

Define extraneous variable

any variable other than the independent variable that may influence the dependent variable

Define cofounding variable

a variable that influences both the independent variable and dependent variable and leads to a false correlation between them

What are some risks and benefits with attempting to make groups homogenous

blocking and matching

Why do we deliberately manipulate variables

to set predetermined conditions

Define an active variable

manipulated by the experimenter

Define an attribute variable

not able to assign, so must observe in natural group

example of active variable

meds vs placebo, TDN vs control

example of attribute variable

age, sport, gender

What is significant about random assignment

each subject has equal chance of being assigned to any group

• limits bias

• does not guarantee equivalence

What are the options that the control group can recieve

nothing, standard of car, placebo

What is significant about a research protocol

helps eliminate extraneous variables, or at least make groups equal across groups

helps minimize the effect of lost data

What are the 5 reasons we would loose data

1. drop out

2. cross over to another treatment

3. refusal of assigned treatment after allocation

4. may not be compliant with assigned treatments

5. may be excluded after randomization

define the intention to treat approach

analyze data based on how we intend to treat the subjects

Pros of using intention to treat analysis

preserve randomization

guards against bias from dropouts

considered reflective of routine clinical situations

Con of using intention to treat analysis

often underestimates treatment effect

Analysis strategies for handling missing data

computer, noncomputer, LOCF

What is the best way to blind

Using a double blind: protection against observation bias

What determines is the study will be a single or double blind

necessity and feasibility is based on the nature of the experiment

What is an example of a variable we would block

height

example of matching variables

using identical twins

If you believe that a specific trait in a subject may interfere with your dependent variable, what is the simplest way to eliminate it?

What is the compromise for that decision?

Choose subjects that are homogenous (healthy males 18-24 y/o)

however loose generalizability

Example of a repeated measurements design for taping

tape shoulder, measure strength every 7 days for 3 weeks (time is repeated factor

What are the 4 threats to design validity

statistical conclusion validity

internal validity

construct validity

external validity

What is statistical conclusion validity

is there a relationship between the IV and DV

Were the appropriate statistical procedures utilized for analyzing data

What is internal validity

is there evidence of a causal relationship between IV and DV

What is construct validity

to what theoretical construct can results be generalized

External validity

Can the results be generalized to other persons, settings, or times

What does statistical conclusion validity look at

low statistical power

violated assumptions of statistical tests

reliability and variance

failure to use intervention to treat analysis

how can low power affect data analysis

concerns the ability to reject the null (document a real change between IV and DV)

How can violated assumptions of statistical tests affect data analysis

can lead to erroneous inferences when assumptions (like randomization) are violated because statistical tests are based on them

How can violated reliability and variance affect data analysis

increased variability and unreliable measurements threaten validity

What does internal validity entail

given a statistical relationship between the IV and DV, is there evidence that one causes the other

What does external validity entail

can the results be generalized to persons, setting, and times that are different from those employed in the experimental group

What are the 3 components of internal validity

temporal precedence

covariation of cause and effect (using a control group to compare)

no plausible alternative explanations

What are 2 groups of factors that affect internal validity

participant associated and measurement associated

What are the participant associated threats

selection

maturation

attrition

history

What are the measurement associated affects

regression

instrumentation

testing effect

Who affect social threats

researchers and subjects

What is a synonym for social threats

performance bias

How can researchers cause social threats

compensatory equalization of treatments

How can subjects cause social threats

diffusion or imitation of treatments

What are 2 main ways subjects threaten validity

compensatory rivalry

resentful demoralization

How do we rule out threats to internal validity

they will always be present to some degree, but randomization and blinding can address most of them

What are the 3 concepts of external validity

interaction of treatment and selection

interaction of treatment and setting

interaction of treatment and history

How does selection affect external validity

generalize beyond the people in the study?

How does setting affect external validity

generalize beyond the setting of a rehab hospital?

What is ecological validity

does this apply to other cultural or societal norms

How does history affect external validity

can you generalize to different periods in time?

major shifts in treatment paradigms create bigger problems

What are the parts of construct validity of causes and effects

use operational definitions, comprehensive measurements, multiple treatment interaction, timeframe, and experimental bias

Define experimental bias

active and passive behavior on the part of the researchers; subjects want to react positively and do what the researchers want