Advanced Research Methods – Week 1 Essentials

Module Overview

  • Focus: Advanced Research Methods & statistics refresher for PSY5007M (Week 1)
  • Core themes this week:
    • Principles in experimental & survey research
    • Research questions, hypothesis generation & testing
    • Effect sizes & power analysis
    • Foundations of Open Science
  • Assessments:
    • Assessment 1: Pre-registration (due 2 May 2025)
    • Assessment 2: End-semester MCQ exam

Key Success Tips

  • Attend both lectures & workshops (hands-on SPSS practice)
  • Engage on Moodle/Padlet; ask questions early
  • Install SPSS (PsychResources > AppsAnywhere) & Mentimeter

Experimental vs Survey Research

  • Survey: measures naturally occurring variables; relies on sampling for generalisation
  • Experiment: manipulates IVs, uses randomisation, can be within-, between- or mixed-design; aims to establish causality

Hypothetico-Deductive Cycle (Popper)

  • Theory → Hypothesis → Operationalisation → Data collection → Data analysis → Findings → Theory revision

Test Statistics Refresher

  • Core ratio:
    Test statistic=Variance explained (Effect)Variance unexplained (Error)\text{Test statistic}=\frac{\text{Variance explained (Effect)}}{\text{Variance unexplained (Error)}}
  • Larger ratio ⇒ better model fit

Hypothesis Testing Essentials

  • Hypotheses can be one-tailed or two-tailed
  • Decision rule: reject H0H_0 when p<.050
  • Errors:
    • Type I (false positive, α\alpha)
    • Type II (false negative, β\beta)

Alpha & Beta Benchmarks

  • Typical values: α=.05\alpha=.05, β=.20\beta=.20 (power =1−β=.80=1-\beta=.80)

Effect Sizes (address Type I focus)

  • Indicate magnitude; compare across studies
  • Cohen’s conventional cut-offs:
    • dd small .20 | medium .50 | large .80
    • rr small .10 | medium .30 | large .50
    • η2\eta^2 small .01 | medium .059 | large .138

Power Analysis (address Type II risk)

  • A-priori: determines required sample size before data collection
  • Post-hoc: estimates achieved power after analysis
  • Aim for power ≥.80\ge .80

Descriptive Statistics Equations

  • Mean: xˉ=∑xN\bar{x}=\frac{\sum x}{N}
  • Variance: s2=∑d2N−1s^2=\frac{\sum d^2}{N-1}
  • Standard deviation: s=∑d2N−1s=\sqrt{\frac{\sum d^2}{N-1}}

Parametric vs Non-Parametric Tests

  • Parametric: assume normality & homogeneity; ratio/interval data
  • Non-parametric: fewer assumptions; ordinal or skewed data / small samples

Open Science Principles

  • Transparency across the research cycle (data, code, materials, preregistration)
  • Motivated by replication crisis & trust in science
  • Core integrity values (Singapore Statement): Honesty, Accountability, Fairness, Stewardship

Questionable Research Practices to Avoid

  • P-hacking: unplanned data fishing
  • HARKing: Hypothesising After Results Known
  • Low power studies: insufficient sample size

Pre-Registration Basics (Assessment 1)

  • Provide background & theory (≤500 words)
  • State up to 3 directional hypotheses
  • Detail design, variables, sample, power justification, exclusion criteria
  • Outline full analysis plan (e.g., multiple regression or two-way ANOVA)
  • Upload plan to repository (e.g., OSF) for transparency

Workflow Summary for Robust Research

  1. Formulate hypotheses
  2. Conduct a-priori power analysis → set sample size
  3. Collect data & check assumptions
  4. Run inferential test; report pp, effect size, 95 % CIs
  5. Interpret results; reject/fail to reject H0H_0
  6. Share data/materials; consider replication