PhD Viva Flashcards: Critical Appraisal, Bias & Examiner Traps

Critical Appraisal

The systematic evaluation of research to determine how much confidence can be placed in study findings.

Primary Purpose of Critical Appraisal

To examine whether study design, conduct, analysis and interpretation minimise bias and support valid conclusions.

Most Important Principle of Critical Appraisal

A study's conclusions are only as trustworthy as the methods used to generate them.

Greenhalgh Question 1

Was the study original?

Greenhalgh Question 2

Whom was the study about?

Greenhalgh Question 3

Was the design sensible?

Greenhalgh Question 4

Was bias minimised?

Greenhalgh Question 5

Was it large enough and followed long enough?

Methodological Quality

The extent to which a study's design, conduct, analysis and reporting minimise threats to validity and bias.

Poor Quality Study

A study whose findings should be interpreted cautiously but may still contribute useful evidence.

Validity

Whether a measure assesses what it claims to assess.

Reliability

Whether a measure produces consistent results when repeated.

Reliable but Invalid Measurement

A measure that consistently gives the same incorrect result.

Systematic Bias

Any process that consistently distorts results away from the truth.

Recruitment Bias

Bias introduced when participants do not adequately represent the intended population.

Performance Bias

Differences in care, treatment, expectations or behaviour that influence outcomes beyond the intervention itself.

Detection Bias

Bias caused when outcome assessment is influenced by knowledge of group allocation.

Blinding

Concealing group allocation to prevent assessors, participants or clinicians from influencing outcome measurements.

Confounder

A third variable associated with both exposure and outcome that creates a misleading relationship.

Confounding Example

Ice cream sales and drowning appear related, but hot weather increases both.

Confounding in Observational Studies

Can usually be reduced but rarely completely eliminated.

Congruity

Alignment between philosophical position, methodology, data collection, analysis and interpretation.

Importance of Congruity

Mismatch between methodology and methods threatens credibility.

Reflexivity

Explicit acknowledgement of how researchers' backgrounds, assumptions and interactions influence research.

Purpose of Reflexivity

To make researcher influence transparent rather than eliminate it.

Participant Voice

Findings supported by participants' quotations, accounts, experiences or observations.

Cross-Sectional Study

A study in which exposure and outcome are measured at the same point in time.

Major Limitation of Cross-Sectional Studies

Temporal sequence between exposure and outcome is unclear.

What Cross-Sectional Studies Demonstrate

Associations.

What Cross-Sectional Studies Cannot Demonstrate

Causation with confidence.

JBI Cross-Sectional Domain 1

Inclusion criteria.

JBI Cross-Sectional Domain 2

Condition measurement.

JBI Cross-Sectional Domain 3

Exposure validity.

JBI Cross-Sectional Domain 4

Outcome validity.

JBI Cross-Sectional Domain 5

Confounders identified.

JBI Cross-Sectional Domain 6

Confounders managed.

JBI Cross-Sectional Domain 7

Statistical analysis.

JBI Cross-Sectional Domain 8

Detailed reporting of sample and setting.

Surrogate Outcome

An indirect measure used instead of the outcome of real interest.

Problem with Surrogate Outcomes

Changes in surrogate measures may not translate into meaningful patient outcomes.

Statistical Power

The probability of detecting a true effect when one genuinely exists.

Type I Error

False positive; concluding an effect exists when it does not.

Type II Error

False negative; concluding no effect exists when one actually does.

Underpowered Study

A study that may fail to detect a true effect because of insufficient sample size.

Statistical Significance

Whether a result is unlikely to have occurred by chance.

Clinical Significance

Whether a result is meaningful in practice.

Why Choose JBI Over CASP

JBI aligns with study designs included, provides structured risk-of-bias assessment and supports evidence synthesis.

Critical Appraisal Influence on Synthesis

Influences weighting of evidence, interpretation strategy and confidence in findings.

Biggest Threat to Qualitative Credibility

Lack of congruity, poor reflexivity and weak participant representation.

Biggest Threat to Observational Studies

Confounding, selection bias and lack of temporal certainty.

Ultimate Viva Answer

Critical appraisal examines whether study methods justify confidence in findings by assessing methodological quality, bias, validity, confounding and the strength of conclusions.