Critical Appraisal of Research Papers Study Notes

CRITICAL APPRAISAL OF RESEARCH PAPERS

  • Presented by Aseel Sharaireh, DDS, MFDS RCS-Ire, PhD

Seminar Outline

  • Summary of critical appraisal
      1. What is critical appraisal?
      2. The scientific mindset
      3. ABC’s of research papers
      4. Types of research
      5. Critical checklists for evaluation

Scientific Research

  • Definition: The creative and systematic work undertaken to increase the stock of knowledge.

  • Research article: A journal article in which authors report on research findings; always considered primary sources, representing the final product of a scientific study.

The Process of Research

  • It begins with formulating a theory based on previous knowledge before conducting experiments to gather results that validate or invalidate the theory.

  • Importance of reliable findings over questionable or lacking research designs.

Critical Appraisal Defined

  • Definition: "Critical appraisal is the process of systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision." – Hill, A. Spittlehouse.

  • Critically Appraised Topic (CAT): A short summary of evidence focused around a clinical question, serving as a concise reference to assist in clinical decision-making.

Aristotle Quote

  • "It is the mark of an educated mind to be able to entertain a thought without accepting it."

Fact vs. Truth

  • Fact: A statement that can be proven true or false based on empirical evidence or objective reality.

  • Truth: A broader, more philosophical concept referring to the state or quality of being in accord with fact or reality, often subjective or intersubjective based on perception or belief systems.

  • The goal of science is to reveal truths, but it relies on facts which can be tested and verified.

The Publish or Perish Culture

  • Initially coined by Coolidge in 1932, emphasizes the pressure on academics to publish frequently, potentially leading to unethical practices and wasted research resources. (S. Rawat and S. Meena, 2014)

Critical Thinking

  • The importance of nurturing critical thinking throughout the evaluation process.

  • Critical thinking involves being critical of one's own biases and thought processes.

Components of a Research Article

  • Title

  • Abstract

  • Materials and Methods

  • Results

  • Discussion (and sometimes a Conclusion)

  • Reference list

  • Supplemental data

Types of Published Articles

  • Review Articles

  • Case Reports

  • Research Articles

  • Clinical Trials

  • Randomized Controlled Trials (RCT)

  • Systematic Reviews

  • Book Chapters

  • Perspectives/Commentaries

Types of Research Methodology

  • Quantitative Research: Focuses on quantifying data and generalizing results across populations.

  • Qualitative Research: Involves exploring phenomena through subjective interpretation.

  • Descriptive and Analytical Research: Descriptive is observational (not intervening), while analytical includes comparisons.

  • Applied vs. Fundamental Research: Applied research addresses real-world problems; fundamental research seeks to increase basic knowledge.

  • Exploratory and Conclusive Research: Exploratory seeks to investigate new areas while conclusive aims to provide answers to specific questions.

  • Surveys and Case Studies: Collect data for research purposes.

Types of Scientific Research Classification

  • By Data Collection Techniques:
      1. Observational: Just observing and collecting data.
      2. Experimental: Involves intervention with subjects (e.g., treatments).

  • By Causality Relationships:
      1. Descriptive (with case reports/series and surveillance studies).
      2. Analytical (with comparisons, use of clinical data).

  • By Relationships with Time:
      1. Retrospective: Looks back at past events.
      2. Prospective: Follows subjects into the future.
      3. Cross-sectional: Examines a snapshot at a single point in time.

Cohort Studies

  • Prospective Cohort Study: Following a group of individuals over time.

  • Retrospective Cohort Study: Looking back at a previous cohort.

  • Measure of Effect: Relative risk calculation to assess strength of exposure.

Interventional Research

  • Quasi-Experimental Research: Evaluates interventions without randomization.

  • Clinical Research: Requires a control group for effect evaluation.

Systematic Reviews and Meta-Analysis

  • Classifications of research ranging from RCTs down to case reports.

  • Systematic reviews and analysis of RCTs are considered the highest strength of evidence in clinical decisions.

Evidence-Based Practice (EBP)

  • Definition: A systematic process where decisions are informed by the best available evidence, aiming to reduce bias and unfounded beliefs in organizational decisions.

  • Informed Decision Making: Evaluating options based on relevant information and potential outcomes.

Evidence-Informed Decision-Making (EIDM)

  • Emphasizes integrating evidence from research along with other factors such as public opinion and feasibility.

  • Applies structured methods in decision-making processes.

Sources of Evidence

  • Tertiary Research: Evidence briefs for policy, health technology assessments, etc.

  • Secondary Research: Systematic reviews, synthesis of evidence, and gap maps.

  • Primary Research: Observational studies, qualitative studies, and public health research.

Contextual Decision Making

  • Achieving sustained impact, understanding priority issues, and continuous adaptation for lasting change in healthcare practices.

  • Engagement in evidence dissemination and educational initiatives to empower practitioners and patients.

Critique Strategies for Research Papers

Evaluate Sources

  • Journal Quality: Consider the journal’s reputation and standards.

  • Methods: Evaluate the methodology employed in the study.

  • Results: Examine the significance and presentation of results.

Checklist for Evaluation

  • Currentness of information

  • Clarity of research question

  • Appropriateness of study eligibility criteria

  • Literature search strategy

  • Risk of bias assessment

  • Results precision and relevance to practice.

The Importance of Citations

  • A citation is a reference to the source of information; citation counts can indicate research impact but do not necessarily measure quality.

Understanding Journal Impact Factors

  • Impact Factor: Based on the number of citations within a 2-year period.

  • CiteScore: Uses Scopus data within a 4-year window.

  • H-Index: Evaluates an author's scientific productivity based on published papers and citations.

Predatory Journals

  • Given the rise of predatory journals, caution is advised. Seek guidance from supervisors to avoid pitfalls associated with low-quality or deceptive publications.

Reading Papers Effectively

  • Key elements: Introduction, Methods, Results, and Discussion.

  • Strategic Reading: Focus on relevance and method before diving into specifics.

Critical Approach to Reading

  • Skim read for relevance.

  • Re-read and evaluate the methods for appropriateness.

  • Examine results critically, focusing on data interpretation rather than captions.

Conclusion

  • The effective critique of research papers involves a thorough understanding of various methodologies, a strategic reading approach, and awareness of the nuances in research quality and impact.

  • Thank you for attending the seminar!