experimental design

principles of good research

reliability, validity, cumulative, parsimony, public

public

open to criticism and scrutiny

reliable (key)

-not just a fluke, will happen again and again

-consistent confidence (a given finding will be reproduced again and again)

valid (key)

-confident our results mean what we think

-given finding a given finding shows what we believe it to show

-internal validity

-external validity

-construct validity

internal validity

-does the outcome really reflect the experimental manipulation

-no or little extraneous variables

external validity

extent to which we can generalize findings to real-world settings

construct validity

the extent to which variables measure what they are supposed to measure

parsimonious

theory is a simple as we can make it while still explaining what it means

cumulative

builds on prior research

how do you design an experiment

1. identifying the research question

2, defining the IV (what will you manipulate?) will it be a between-subjects or winthin-subjects design?

3. define the dv (what outcomes are you interested in?) (consider the relevance-sensitivity trade-off)

4. choose a sample

5. how will the results be interpreted?

research question example

does social support improve symptoms of depression

iv and dv example (from question: does social support improve symptoms of depression)

IV = social support levels

DV = symptoms of depression

Operationalism

-'translating' concept of interest into something observable and measurable

operationalism example (from question: does social support improve symptoms of depression)

social support → number of friends on facebook? hours spent at social events?

between-subjects design

2 groups (control and experimental) and manipulation occurs between these groups

within-subjects design

a research design that uses each participant as his or her own control; for example, the behavior of an experimental participant before receiving treatment might be compared to his or her behavior after receiving treatment

within-subjects design strengths

same people, so changes can be probably linked to IV

within-subjects design weakness

people cant be measured twice at the same time (time might be extraneous variable)

IV and DV need to be a balance of

-what we want to measure → as close as possible to interested construct

-what can we measure → needs to be measurable

-what should we measure → must be ethical to measure

relevance-sensitivity trade-off

the more sensitive a DV is to changes in the IV, the less relevant it may become to the real-world phenomena in which one is interested

threats to internal validity

time, experimental situation, chosen sample

time and internal validity

-esp an issue in within-subjects design

- Practice effects

-Fatigue effects

-Maturation effects

-History effects

threats to external validity

-experiment is too artificial (study of time in nature is conducted in sterile lab)

-experiment itself introduces confounds (demand characteristics, experimenter bias)

-sample doesnt reflect target population (4 yr old reading intervention tests with uni students)