Research Methods Exam #2

Interviews

• Directly ask participant questions/items

• Lack of standardization can introduce error

Likert (Type) Items

Individuals rate agreement/disagreement with statements

Likert item: “Gorditas are the BEST food every created”

Likert-type item: “Better pecan ice cream is the WORST Ince cream flavor”

Semantic Differential

Biopolar adjective scales

• Opposing adjectives placed at opposite ends of scale

• Bad (numbers) Good

• Pleasant (numbers) Unpleasant

Crafting a Questionnaire

• Open-ended

• Likert

• Wording and order are important (affect construct validity)

Questions/error

Do you eat healthfully and exercise regularly

• Fix: Ask question individually

How short was Napoleon? vs. How would you describe Napoleon’s height?

• Fix: Word questions neutrally, ask question in more than one way

Negative language increases cognitive effort

• Fix: ask question both ways (negative and positive)

Question Order

Context of question can affect questions

Response Sets

Answering all item in one direction/consistently

Acquiescence or Nay-saying

Response Sets: Acquiescence or Nay-saying

Agree/disagree to all items (neutral tendency, lack motivation to answer accurately)

Response Sets Acquiescence or Nay-saying Solutions

• Reverse-coded items

• Attention check within survey

Reverse Coding

Scale: Social Dominance Orientation scale (16 items)

• High scores = high SDO

• Some questions may ask the opposite of SDO

  • High scores on #2 & #3 = Low SDO

  • High scores on #1 and #4 = High SDO

• Reverse, add items and then divide by total number of items

Response Sets: Fence-sitting

• Neutral answer when if they have an opinion

• More common with controversial item

Response Sets Fence-sitting Solution

• Remove neutral/middle option

• Forced-choice (give response)

Response Sets: Social desirability - “faking good”

• Answering in a way that makes one “look good” (what researchers may want)

Response Sets Social desirability Solution

• Remind responses are anonymous

• Administer the Social Desirability Scale or something similar

• Check agreement with other responses

Problems: Measurement Design

Double-barreled / Leading questions / Negative wording / Order

Problems: Response Sets

Acquiescence / Nay-saying / Fence-sitting / Social Desirability

Alternatives to Self-Report

• Observational research record a behavior or code content

• Indirect measures: less straightforward

• Physiological/Neurological measures

Observational Measure

Archival / content analyses

• Code behavior / characteristics from previously recorded sources

Coding nonverbal behaviors

• “Strange situation”, attachment-related behaviors

Indirect

• Complete the word

• Implicit Association test: reaction times

Physiological & Neurological Measures

• Heart Rate / Blood Pressure / EKG

• Cortisol (saliva or blood sample)

• Skin Conductance (measure sweating)

• Facial Electromyography (EMG)

• Electroencephalogram (EEG)

Neurological: fMRI

• Differences/changes in brain region activation

• Activity in response to stimuli

Population

All the people interested in

Sample (N)

A subset of the population

Participants to recruit

Estimate N through power analysis:

• Effect size

• Type of test

• Power to detect effect

• General rules

  • Group differences: 30 per condition

  • Correlation/regressions: 50 participants

Random Sampling

Everyone in the population has an equal chance of being selected for your study (using 10 random numbers)

Technique: Simple Random Sample

1. List of all subjects in the population

2. Assign a number to each subject in the list

3. Randomly select participants (power analysis)

4. Use the random number generator for number of participants (skip repeat numbers

Technique: Cluster Sampling

1. Identify all clusters in the population

2. Assign a number to each of the clusters (not numbering each person)

3. Randomly select naturally occurring clusters

4. Use the random number generator (sample everyone in each clusters)

Technique: Multistage Sampling; Cluster Sampling + 1 extra step

1. Identify and number all the clusters in the population

2. Randomly select occurring clusters

3. randomly select participants from each cluster

4. Number each person across the cutters

5. Randomly select N numbers

Technique: Stratified Random Sample

1. Break population into groups

2. Randomly elect from each group based on population

   1. Determine % of each group in population

   2. Randomly sample based on %

The percentage of your population should have the same amount of the number of people in your sample

Oversampling

• Variation of stratified random sampling

• Oversample from some groups (instead of selecting % to represent)

Non-Random Sampling

Purposive

• Seek out group in a non-random way

• Snowball- type of purposive

  • Participants recruit other participants

Study (experiment)

1. At least one variable is directly manipulated (IV)

2. Other variable are controlled (outcome)

3. Random assignment (different from sampling): participants are assigned to condition in a way that ensures the equivalence of groups

Experiments

• To examine cause & effect

  • Causality requires: covariance, temporal procedure (action has to happen when you do it), internal validity

• More control over the situation & relevant variables

Types of Variables

Independent variable

• Manipulated by experimenter (levels/conditions)

• Randomly assign participants to level

Dependent variable

• Measured by experimenter

• Affected by IV

Example: Note taking (I IV with 2 levels) → By hand and laptop (both IV) → Scores on essay questions (DV)

Random Assignment

• Equally distributes people across conditions

Between vs. within subjects designs

Between subject manipulation

• Participants assigned to only one level

  • Ex. either by hand or laptop

Within subject manipulation

• Participants assigned to all levels of IV

  • Ex. by hand & laptop + by hand & laptop

Between-Subjects Designs

Post-test only

• Randomly assign to condition

• Measure DV once

  • Ex. Randomly assigned participants -→ one group nutrition class (IV), other group mindfulness class (IV) → Verbal GRE Score for both classes (DV)

Pretest/Post-test

• Randomly assign to condition

• Measure DV twice: before and after manipulation

  • Ex. Randomly assigned participants → Take the Verbal GRE Score (DV), one group nutrition class (IV), other group mindfulness class (IV) → Retake the erbal GRE Score for both classes (DV)

Types of Within-Subjects Designs

1. Concurrent Measures Design

• Exposed to all levels at about there same

• Measure choice /preference

  • Ex. One group -→ Female face, male face IV) -→ Looking time for each face (DV)

2. Repeated Measure Designs

• Measure DV after every IV level

  • Ex. One group → Taste chocolate with confederate (shared experience) (IV) → Rate chocolate (DV) → Taste chocolate alone (IV) → Rate chocolate (DV)

APA’s 5 General Ethical Principles

1. Respect for people’s rights and dignities

2. Beneficence & nonmaleficence

3. Justice

4. Integrity

5. Fidelity & responsibility

Respect for people’s rights and dignities (autonomy)

• Informed consent

• Cannot mislead of risks/benefits

• Cannot coerce (don’t force them to do anything)

• No undue influence

• Protection for groups (Children, pregnant women, prisoners)

Beneficence & nonmaleficence

• Benefits vs. risks

• Minimize pain and suffering

  • Participant can leave experiment at any time

  • Everyday situations, stress environement

Justice

Treat all group fairly, don’t target a certain group

Integrity

Strive to be accurate, truthful and honest

Fidelity & responsibility

Should build as educators, researches and clinical professionals (build appropriate trusting relationships)

Informed Consent

• Risks and benefits

• Autonomy

• Free to end participants with fear of consequence

• Confidentiality: when some personally identifiable information is collected

Omission

Withhold information (committed deception by not telling the truth)

Commission

Lie (Can be okay at times based the the results they need for the study)

Research Misconduct

Data fabrication & falsification

• Invert or knowingly change data

Review: Random vs. Systematic Error

Ex. Does positive mood increase stereotyping

Random error: variability in personality type in both conditions

Systematic error: the happy move is a different length than sad movie (confound)

IV: happy vs. sad movie

DV: stereotyping of job applicant

Confounds: Maturation

Naturally occurring change over time

• Ex. Spontaneous remission and depression

  • Measure depression (pre-intervention) → Intervention given → Measure depression (post-intervention)

Confounds: Regression to the mean

Extreme scores become less extreme

• Ex. Severely depressed people seeking treatment (improving due to therapy? Pre/Post therapy)

Confounds: History

Event that coincide with experiment treatment

• Ex. Strat a “Go Green” campaign over time to reduce energy consumption

  • Storm is history event → use more energy during storm vs. not having a storm

  • Winder storm: measure electric energy consumption → Put up flyers to promote “Go Green” campaign (intervention) → Winter (no storm): measure electric energy consumption→ Results: decrease in energy use pre to post flyers

Confounds: Attraction/mortality

Dropout of participants from study

Systematic problem

• Drop outs -→ Treatment group

• Drop out → after treatment (different sample pre-treatment)

Confounds: Order effect

Order of condition presentation affects DV

• Ex. Study lecture like normal (control) → Take 150 question test → Study with flashcards (experimental) → Take 150 question test

  • Practice effect: get better due to practice

  • Fatigue effects: tired/bored towards end of task

  • Carryover effects: earlier manipulation affects next manipulation

Confounds Over Time

Time 1 → Maturation, History, Attrition

Time 2 → Internal chance, External change, Different sample

Solutions: Maturation, Regression to the mean & History

Comparison group

Maturation

• Getting better naturally or due to treatment?

History

• Did event affect both groups?

Regression to the mean

• Do both groups start at the same level?

Solutions: Subject loss (Attrition)

• Remove pre-tests scores or dropouts

• See If it is equal across conditions

• If subjects differ on key variables between conditions

Solution: Order Effects

Counterbalancing: present conditions in random order for each participant

1. Full counterbalancing: all possible combinations used

2. Partial counterbalancing: some possible combination are used

Between-Groups Design Confounds

Selection: different participants between conditions

• Participants choose condition

• researcher assigns one type of participant to condition

Instrumentation: changes in measurement/procedure

Solutions: Selection

Random assignment

• Equal opportunity for any level of IV

Matching

• Helpful with small design

• Groups formed:

  • Matching participants on variables

  • Randomly assigning matched participants to conditions

Issues of Within & Between Designs

Confounds more problematic for within-subjects designs:

• Maturation, history, attrition/mortality and order effects

Confounds more problematic for between-subjects designs:

• Election and instrumentation

Null Effect (between groups) Demand Characteristics

Study contains cues to hypotheses → participants changes response

“Sensory deprivation”

Implied effects: asked hx of dizziness, fainting; emergency tray in view and control → Isolation chamber

Observer (Experimenter) Bias

• Researchers expectations influence interpretation of results

  • Code based on expectations of condition

  • Subtly convey different information or instructions

Solutions: Demand Characteristics & Observer (Experimenter) Bias

• Double-Blind Procedure

  • Neither experimenter nor participant know condition assignment

• Masked/blind Procedure

  • Experimenter does not know condition assignment

Placebo Effects

Given placebo → believe they receive treatment → show actual improvement

Manipulation effectiveness

No different between groups?

• Maybe there is no effect

• Maybe out DV lacked reliability or validity

• Maybe manipulation did not work

Ceiling & Floor Effects

All scores are either high or low

• Ex. Stereotype threat and math performance

Solution: Pilot Testing

Examine effectiveness of a manipulation/measures prior to the experiment

Solution: Manipulation (IV) Checks

Use another measure that will “check” whether your manipulation is doing what designed to do (given to participants during the actual experiment

Solution: Internal Analyses

• Manipulation check data can be used as a predictor variable

• Useful if manipulation was weak or there were too many individual differences

• Results are correlational

Conceptualizing the IV

1. Environmental

• Mood

• Used of confederates

2. Instructions

3. Invasive

• Ex. Invasive- Epinephrine shot or placebo, Instructions- Accurate or no information about epinephrine shot, Environment- Euphoric or angry confederate

Null Effects (within groups), Non-systematic error

• Measurement error

• Individual differences

• Situation nouse

Power

Likelihood of correctly rejecting the null hypotheses

Determined by:

• Effect size: smaller sample

• Sample size (N): larger sample

• Alpha level

• Reducing nouse = increase power