Reliability and Validity

Dealing with Reliability and Validity

Validity

• Definition: Validity is the extent that something measures what it is intended to measure. This is also referred to as construct validity.

Validity in Qualitative Research

• Validity is discussed differently by qualitative researchers compared to quantitative researchers.

• Key differences:

  1. Validity is often interpreted as the extent to which the analysis fits the data (usually text). The focus is on the validity of the analysis, not the objective validity of a scale or measure. A valid analysis fits the data well.

  2. There is a tendency to assume that qualitative research is intrinsically more valid than quantitative research.

  3. The fidelity of the transcription (e.g., of a conversation) to the original source can be regarded as an indication of the validity of the transcription.

Validity Criteria in Qualitative Studies (Mays and Pope, 2000)

• According to Mays and Pope (2000), validity in qualitative studies involves several criteria:

  1. Triangulation

  2. Respondent validation or member checking

  3. Clear description/explication of the methods of data collection and analysis involved

  4. Reflexivity

  5. Attention to negative or deviant instances

  6. Richness of detail in the data and analysis

  7. Using quantitative techniques where appropriate

  8. Openness to evaluation

  9. Participant’s own understandings within the data

Reliability

• Definition:

  • The quality or state of being reliable.

  • The extent to which an experiment, test, or measuring procedure yields the same results on repeated trials.

Reliability in Qualitative Research

• In quantitative research, the concept of reliability is applied to the measures (of variables) which the researcher is using.

• Two main usages of reliability in quantitative research:

  1. Test-retest reliability: Indicates an assessment of how stable or consistent ‘scores’ on a measure are at different points in time.

  2. Internal consistency: Refers to the internal consistency of a measure.

• Neither of these has much bearing on most qualitative research, because:

  1. Qualitative researchers eschew the use of scales for various reasons.

  2. The notion of reliability may be relevant to just a few aspects of qualitative research.