Critical Appraisal of Qualitative Studies

Nature of Qualitative Research

  • Qualitative research collects and analyzes non-numerical data such as written text, audio, or video files.
  • Aims to understand participants' experiences, opinions, and attitudes, or to explore a concept.
  • Provides in-depth insight and can generate new research questions.
  • Considers contextual factors like social, legal, and resource constraints to inform best practices.

Qualitative Study Designs

  • Grounded Theory:
    • Constructs theory from observations about people's lived experiences.
    • Applies inductive reasoning to identify emerging themes from data.
    • Codes are applied based on ideas arising from the data.
    • Questions address what people are doing, saying, and taking for granted, as well as how context influences actions and statements.
  • Discourse Analysis:
    • Analysis of language beyond the sentence level, examining larger chunks of language.
    • Data can be written, spoken, or non-verbal interactions.
    • Context is crucial for understanding meaning.
    • Focuses on social aspects of communication and how language is used to achieve specific effects (e.g., building trust, evoking emotions).
  • Phenomenology:
    • Study of experience to uncover and describe the meaning of lived experience.
    • Explores what it was like to experience something, influenced by individual beliefs, values, morals, culture, and religion.
  • Ethnography:
    • Study of cultures and subcultures to uncover and describe the meaning of rituals, symbols, and customs.
    • Research questions are specific and relevant to the group in question.
  • Case Study:
    • In-depth study of a person, family, group, community, or institution.
    • Examines a complex issue or object connected to political, social, historical, and personal issues.
    • Data collected through observation, interviews, etc.
    • Answers how or why questions.

Inductive vs. Deductive Approaches

  • Quantitative research is deductive, aiming to deduce new knowledge from known facts or test existing theory.
  • Qualitative research often develops a theory inductively, moving from specific observations to broader generalizations.
  • Qualitative research may use deductive, inductive, or combined approaches.
    • Deductive: Uses an a priori expectation model with themes for coding.
    • Inductive: Works exclusively from data to drive analysis, with detailed re-readings to derive concepts and themes.

Appraisal of Qualitative Research

  • Necessary to assess trustworthiness before implementing findings into practice.
  • Evaluates methods (data collection, analysis) and research design appropriateness.
  • Rigor is assessed by transferability, credibility, reflexivity, and transparency, rather than reliability, validity, and generalizability.

Key Concepts for Assessing Rigor

  • Transferability: Extent to which readers can connect study data to wider community settings.
  • Credibility: Extent to which the research account is believable and appropriate.
  • Reflexivity: Researchers' account of their engagement, examining how they influenced the research.
  • Transparency: Making explicit the entire research process and rationale behind decisions.

Checklists for Appraisal

  • SRQR (Standards for Reporting Qualitative Research): A standard for researchers when reporting qualitative studies, which has also been adapted into a critical appraisal tool.
  • CASP (Critical Appraisal Skills Programme) Checklists:
    • Developed for various study types, including qualitative studies.
    • Asks questions to help make sense of a study with yes, no, or can't tell answers.
    • No scoring system; items may be inapplicable.
    • Two screening questions: clear statement of aims and appropriateness of qualitative methodology.
    • Sections:
      • A: Validity of Results
      • B: What are the Results?
      • C: Will These Results Help Locally?

CASP Questions and Considerations

  • Screening Questions:
    • Was there a clear statement of the aims of the research? (Goal, importance, relevance)
    • Is qualitative methodology appropriate? (Interpret actions/experiences, right methodology for the research goal)
  • Section A: Are the results of the study valid?
    • Was the research design appropriate to address the aims of the research? (Justification, rationale for method)
    • Was the recruitment strategy appropriate to the aims of the research? (Participant selection, rationale, discussion around recruitment)
      • Sampling is purposeful, not probabilistic.
      • Techniques: maximum variation, convenient, snowball, stratified, homogenous.
    • Were data collected in a way that addressed the research issue?
      • Justification of setting/data collection
      • Qualitative data from interviews (structured, semi-structured, face-to-face, telephone, video platform), written responses, focus groups.
      • Field notes: record social phenomena, interactions, and behaviors directly.
      • Audio/video recordings: less intrusive, but may miss cues or information.
      • Data saturation: point where more data sampling yields little new information.
      • Iterative process: loops of data collection and analysis.
  • Section B: What are the results?
    • Ethical considerations: ethics committee approval, details of research explanation to participants, discussion of issues around consent/confidentiality/effects on participants.
    • Was the data analysis sufficiently rigorous?
      • In-depth description of analysis process, explanation of how categories/themes were derived.
      • Presentation of sufficient data to support findings, consideration of contradictory data.
      • Qualitative analysis is interpretive, ranging from quasi-statistical to immersion and crystallization.
      • Role of researchers: reflection and critical appraisal of their own role and potential biases.
      • Bracketing: separating own experiences from what's being studied (used in phenomenology).
    • Is there a clear statement of the findings?
      • Explicit presentation, adequate discussion of evidence, relationship to research question.
      • Discussion of credibility: triangulation, respondent validation, multiple coders, inter-rater agreement.
  • Section C: Will these results help locally?
    • How valuable is this research?
      • Contribution to knowledge, consideration of current practice/policy/literature.
      • Identification of new research areas.
      • Discussion of transferability to other populations.
    • Key questions: Does this study help me understand the context of my own practice? Does the study help me understand my relationships (e.g., with patients and their families)?