Chapter 9 (Evaluating the Merits of Qualitative Research Studies in Kinesiology)

Can we use the same approaches we use to evaluate quantitative research to determine whether qualitative research is appropriate?

No, in qualitative we have no measures or instruments to assess (no construct or internal validity), and no intervention or treatment to examine for threats to internal validity

Rigour and validation

Synonymous with trustworthiness

Qualitative Evaluation Approaches

Trustworthiness, methodological coherence, consolidated criteria for reporting qualitative research, relativistic approach to characterizing traits, and ethics

Trustworthiness

convincing an audience that a study is worth paying attention to and worth taking account of (merit of the study)

Four Aspects of Trustworthiness

Truth Value, applicability, consistency, and neutrality

Truth Value

Credibility of a study, confidence in the “truth” of study findings for participants (how does it reflect their experiences and meaning)

Applicability

Transferability of a study, forming understandings that may be relevant to other contexts or participants

Consistency

dependability of a study, would similar findings emerge if a study were replicated in similar circumstances

What aspects of trustworthiness seeks to understand variability of study findings or unique experiences that stem from the multiple realities assumption

Consistency

Neutrality

findings are based on participants’ meanings and experiences, there is researcher reflexivity, and the findings are not a mere function of researchers’ biases, interests, and perspectives

Researcher Reflexivity

A researcher’s positionality, reflect on biases, experiences, and backgrounds to consider how to shape research

What is the starting place when evaluating qualitative research?

Trustworthiness

How many strategies are available to enhance trustworthiness, rigour, and validation?

9 strategies, that target different or multiple phases of the research study

Strategies to enhance trustworthiness, rigour, and validation

audit trail, member check, peer debrief, present negative or discrepant information, prolonged engagement, purposeful sampling, researcher reflexivity, triangulation, and rich, thick description

Audit Trail

Researchers maintain detailed descriptions of the entire research process, someone external to the study examines various components of the study, the events, influences, and actions are all recorded

Detailed recollection of their analytic process, including transcribing of interviews, reading and coding the data, and developing categories and themes

Audit Trail

True or False: the audit trail is more difficult to incorporate because it requires more timing and thinking

True

Member Check

the participants of the study review the data and/or study themes and findings, they are then given the opportunity to add, alter, or delete information

Which strategy is very easy to implement and increases accuracy of the data?

Member Check

Peer Debrief

Researchers pushed by professional “peer” to critically reflect on the study, they discuss and debate the themes generated

Present Negative or Discrepant Information

presenting information that counters the main study findings and highlights opposing views and unique experiences

Prolonged Engagement

very important in ethnography, sustained time spent with participants in the field, and it enhances the quality of the research

Purposeful Sampling

recruiting information-rich participants who can best inform research questions

Reflexivity

Reflecting on one’s experiences with the phenomenon being explored; and considering how one’s experiences shape the research process

Rich, thick descriptions

-generating thorough descriptive data

-presenting findings in a rich manner

-use quotes to bring participants’ voices forward

-ask probing, open-ended questions

Triangulation

-crosscheck study findings and interpretations

-use a variety of data sources, perspectives, and methods

True or False: a certain number of strategies does not guarantee a strong study

True

Coherence

alignment between various components of a research study, between the researchers’ philosophical assumptions, research questions, study design, methods of data generation, data analysis, and interpretation

What is deemed to be reflective of strong research?

crosschecking study with comprehensive “checklist” of criteria

Methodological Coherence

-indicator of quality research

-requires alignment within a research design

-coherence among philosophical assumptions, research questions, study design, data generation, data analysis, and interpretation

-researchers need to critically think about, and carefully plan, their qualitative study

What evaluation approach uses the armchair walkthrough method?

Methodological Coherence

Plan all aspects of your study and reflect on what you’re doing and what changes you may need to make

Armchair Walkthrough

Armchair Walkthrough

-reflect on all aspects of the qualitative study

-consider alternative approaches

-provides a road map

COREQ

Consolidated Criteria for Reporting Qualitative Research

Consolidated Criteria for Reporting Qualitative Research

a comprehensive protocol to assess qualitative research, three categories with a total of 32 items

Who developed the COREQ?

Tong, Sainsbury, and Craig (2007)

Three categories of the COREQ

Research team and reflexivity, study design, and analysis and findings

Purposes of the COREQ

-a guide to inform researchers of important aspects to include in their research (can add transparency when reporting qualitative research)

-identifies a variety of details that qualitative research can be evaluated on

Taking a Relativistic Approach to Evaluating Research

-no one-size-fits-all approach

-qualitative research can include a variety of methods and generate various types of data

-not a pre-determined set of criteria

-it is up to researchers (and the research audience) to evaluate and determine the merits of a study

Who introduced the relativistic approach to evaluating qualitative research?

Andrew Sparkes, Brett Smith, and colleagues

Who often uses the relativistic approach for evaluating qualitative research?

Experts in the field

Characterizing traits

Criteria that may allude to the quality of research, fluid and dynamic, the context of the study is very important, and what is deemed to be of high quality in one study may not be of high quality in another study

Ethics

Should be the foundation of all research in kinesiology, should be ongoing within the research process, and researchers need to ensure respect for the peoples, concern for welfare, and equity

What is integral to the way researchers think about, approach, and do their research?

Ethics