Ch4: Control

control

ability to control for threats to research validity; ability to rule out alternative explanations

• perhaps most important element in/feature of science

experimental control

reduce variability in everything but IV

• control groups/conditions are qual on everything but IV

• random experiment and large sample sizes are also forms of experimental control

  • variability (of individual differences across groups/conditions and of statistics across samples) reduces

control experiment

provide standard of comparison

• control group (absence of manipulation) not always necessary

  • ex. low level of IV vs high level of IV

  • control condition (not group) when using within-subjects design; “group” implies between-subjects design

strategies to achieve/enhance control

• random assignment to groups

• matching

• subject as own control (within-subject designs)

• manipulation checks

• instrumentation of response

• building nuisance variables into the experiment

• statistical control

random assignment to groups

equates groups of participants by ensuring that every member of the sample has an equal chance of being assigned to any group

• used in experimental designs

• controls for both known and unknown variables

• increasing sample size makes it more likely that groups will be more nearly equal

matching

participants are matched on some variable or characteristic of interest. pair participants by similarity, then randomly assign each participant in pair to groups.

• controls only known variables

• control and treatment groups should be more alike on matched variable than if randomly assigned

• control and treatment groups should be less alike on unmatched variables than if randomly assigned

• only use when you are sure the matched variable is related to the DV

subject as own control (within-subjects design)

participants complete all/every experimental condition(s)

• potential problems?

  • irreversibility of treatment effects - participants permanently changed as a result of first/prior condition

  • dependability of treatment effects - condition operates differently as functino of prior condition

    • contrast/order effects: second condition perceptually contrasted w/ first condition

    • carry-over/sequence effects: participants “carry” with them some “psychological baggage” from first into second condition

how to control for some within-subjects design threats

• practice effects → increase time intervals

• irreversibility of treatment effects → cannot use within-design; switch to between-subjects design

• dependability of treatment effects → counterbalancing

  • run analyses on full sample and dependability effects will “wash away/even out”

  • intrasubjects (all participants complete each order) vs. intersubjects (some complete one order, others complete other order)

  • reverse → ABCD, DCBA

  • complete (all possible combinations) vs. incomplete (not all possible combinations)

    • 3-condition complete counterbalancing → ABC, BCA, CAB, BAC, ACB, BCA

manipulation checks

using wording to influence a participant’s thinking

• assess efficacy of manipulation of IV

• “did manipulation work as intended?”

• usually accomplished vis t-test and/or ANOVa

• different from attention checks (eg. bogus items, instructed response items) → do not tell you about manipulation

instrumentation of response

review: measures should be objective, standardizes, reliable, and valid

measures should also be sensitive (ex. fineness, precision)

• artificially discrete/coarse, measures should be avoided → use continuous levels of measurement where possible

• measures should be able to discriminate between two individuals with slightly different construct levels

• we want to allow the DV to vary as much as possible between groups/conditions

  • more variability in DV → more variability than can be explained by variability in IV

building nuisance variables into the study

treating extraneous variables as moderator variables

• moderator variable: moderates/influences the relationship between the IV and DV

• controls for only the nuisance/moderate variable

• nuisance/moderator variable cannot be confounded with IV

  • cannot vary systematically with IV

  • must have variability within all experimental conditions

statistical control

use correct test to statistically equate individuals on some control variable

• correlation design (continuous IV) w/ control variables(S) → regression

• experimental design (categorical IV) w/ control variables (s) → ANCOVA

• statistical power of the study

• measurement error

• use statistical tests to confirm the successfulness of manipulation check and random assignment