RM2 Lecture 2

formulating research question (RQ)

1. Select topic
   
   

2. Review literature

   1. Distinguish with claims and findings

      • internal validity: does the cause-effect relationship hold? are there alternative explanations?

      • construct validity: do the operationalisation capture the construct?

      • statistical validity: do the statistics support the claim?

      • external validity: are the results generalisable?

3. Formulate problem/question
   

4. Review literature
   

5. Formulate hypothesis
   → Research question → Hypothesis

   1. In confirmatory research, hypotheses must always be stated beforehand. Not doing this is called HARKING.

how to go from RQ to hypothesis

Constructs must be operationalized (defined in measurable terms)

HARKING

Hypothesising after Results are Known

pretending you had this hypothesis all along

• can lead to confirmation bias

• produces type 1 error

• misleads readers; undermines scientific integrity

P-hacking

could be:

• Running many analyses, but only reporting the ones with significant results

• Dropping/adding variables until significance is achieved. 

• Excluding outliers selectively to change results

• Collecting more data and adding it in the data set to reach significance 

• Trying multiple statistical tests but only reporting the ones that work (i.e. are significant)

• Optional stopping: the practice of peeking at data and then, based on the results, deciding whether or not to continue an experiment

publication bias

When studies with statistically significant/ positive results are more likely to be published than null results or negative findings.

• Why does it occur? People aren’t interested in knowing that something isn't there, they only are interested in what is there.

• related to both HARKING and p-hacking
  

how to solve HARKING, p-hacking, publication bias?

preregistration: Publish hypotheses, experimental design, and analysis plan before executing research and data collection

• especially important in confirmatory research

• helps avoid HARKING (you need to already give your hypotheses)

• p-hacking (you need to show your analysis plan beforehand)

• publication bias (helps avoid selective reporting, they don’t have a choice but to publish it, even if it’s null)

why is preregistration not always feasible?

if someone has never used the technique they are planning on using before, or there’s other unforeseen things, they may need to change stuff in the middle of the process

• But hypotheses always need to be specified before you execute planned research. 
  

Kaplan & Irvin, 2015

Before 2000, 17/30 studies showed an effect. After pre-registration became required, only 2/25 showed an effect.

purpose of confirmatory vs. explanatory research

confirmatory: test hypothesis

exploratory: generate hypothesis

type 1 or 2 focus for confirmatory vs exploratory research?

confirmatory: type 1, minimise false positives

exploratory: type 2, minimise false negatives

what validity do we need in C vs E?

confirmatory: high statistical/ external

exploratory: low validity

what inferences do we want to make with C vs E?

confirmatory: inferences may be drawn to wider population

exploratory: not useful for making inferences to any wider population

statistical value of C vs E?

confirmatory: p-values regain diagnostic value

exploratory: p-values lose diagnostic value

questionnaires & surveys

• Descriptive (e.g. census, polls)
  

• Testing:
  

  • Without manipulation: association claims
    
    

  • With manipulation: causal claims
    
    

e.g. Hypothesis: "Stress increases smoking and impairs sleep"

• Limitation: Cannot confirm causality with self-report data
  
  

• Need experimental design to establish causality (Lecture 4)
  

question types

• Open-ended – flexible, rich, difficult to score

• Closed-ended

  • Forced-choice

  • Multiple selection; can be tricky to score

  • Ranking

There’s also different formatting for question types, so be careful that formatting is reflective of question type.

phrasing guidelines

avoid:

• Complex/unclear wording

• Making assumptions about what your readers know

• Negatives/double-negatives

• Double-barreled questions (combining two questions into one)

• Suggestive or loaded phrasing (like emotional questions), ask them very directly, don’t use words with negative connotations.

include

• option to select “Not applicable”

• optional open comment at end, for participants to indicate difficulties they might have had

question-related response bias

• Scale ambiguity; e.g. “a lot” doesn’t quantify to the same amount for different people

• Suggestive/  negative phrasing

• Context effects (previous questions influence later ones); people may answer differently depending on the questions that came before 

person- related response bias

• Social desirability (answer to “look good”); this is why anonymity is important
  
  

• Acquiescence (agreeing or disagreeing automatically); this is solved by including a negatively and positively phrases question about the same subject
  
  

• Extreme/middle option bias; where people tend to choose only the extreme answers or only the middle answers 
  

measurement levels

Nominal	Categories	Gender, ethnicity, postcode
Ordinal	Ranked, uneven intervals	Education level, happiness
Interval	Even intervals, no true zero	Celsius, IQ
Ratio	True zero	Age, weight, time

choosing the right analysis

Quant + Quant	Quant	Multiple regression / ANCOVA
Cat + Cat	Quant	Factorial ANOVA
Quant + Quant	Cat (multi)	Multinomial logistic reg.
Quant + Quant	Cat (binary)	Logistic regression
Cat + Cat	Cat	Chi-square / log-linear

key principles of ethics in research

1. Respect for persons – autonomy, voluntary, gender-inclusive
   

2. Beneficence – protect from harm, data confidentiality, compensate for participation
   

3. Justice – fair distribution of research benefits, also provide treatment for those in control groups

ethical practice- before

• Info brochure, informed consent

• Get approval from ethical committee

ethical practice- during

• No personal data collection e.g. address, name

• Be respectful and clear, be aware of your role as a researcher

after

• Debrief participants (especially if deception used)

• Store data anonymously (not OneDrive in real studies)
  
  

what to put in info brochure

• Purpose of study

• Procedure and duration

• Risks/benefits

• Data handling

• Compensation

• Withdrawal rights

• Ethics contact details

informed consent

• Must be clear and understandable

• Consent to:

  • Participate voluntarily

  • Data collection and storage

  • Anonymity/confidentiality
    
    

Ethical Framework (NL/EU)

Key Bodies:

• VCWE – Local ethical committee (non-invasive psych research)
  
  

• METC – Medical ethical review board (invasive research)
  
  

• WMO – Dutch law for human research
  
  

• GDPR – EU privacy law
  

Key Declarations

Declaration of Helsinki:

VCWE

A local ethics committee at Dutch universities (like VU) responsible for reviewing non-invasive psychological and behavioral research.

What it covers:

• Studies involving surveys, EEG, fMRI, skin conductance, heartbeat, etc.

• You submit your study proposal here before collecting data (for official research, not class assignments).

METC

What it is:
A medical-level ethics board for invasive studies.

What it covers:

• Studies involving blood samples, DNA collection, injections, drugs, or withholding medication.

• Required for medical or high-risk research – more strict and detailed than VCWE.

WMO-

What it is:
A Dutch national law ("Medical Research Involving Human Subjects Act").

What it covers:

• Protects participants in medical and scientific research.

• If a study involves invasive procedures or serious risk, it must be approved under WMO and reviewed by METC.

GDPR

A strict EU law about data privacy and protection.

What it means for researchers:

• You cannot collect, use, or share personal data without clear consent.

• Must protect participants' data (e.g., anonymize, encrypt).

• Participants have the right to see, correct, or delete their data.What it is:
  A strict EU law about data privacy and protection.

  What it means for researchers:

  • You cannot collect, use, or share personal data without clear consent.

  • Must protect participants' data (e.g., anonymize, encrypt).

  • Participants have the right to see, correct, or delete their data.

Declaration of Helsinki

A global ethical standard for human research, developed by the World Medical Association in 1964.

What it emphasizes:

• Respect for individuals

• Informed consent

• Scientific integrity

• Risk-benefit balance

• Right to withdraw

It serves as a foundation for research ethics worldwide, influencing laws like WMO and review boards like VCWE and METC.