Measurement Foundations & Validity

Research Methods

Measurement Foundations & Validity

Role of Conceptual and Operational Variables and Scales of Measurement
  • Conceptual variables: The theoretical definitions of variables as understood at an abstract level.
  • Operational variables: Specific decisions about how to measure or manipulate conceptual variables in research.
  • Scales of measurement: The various ways in which variables can be quantified and categorized.
  • Importance in Psychological Science: Understanding both types of variables aids in correctly interpreting study findings and enhances the rigor of psychological research.
Validity and Reliability of Measured Variables
  • Validity: The degree to which an operational definition measures what it claims to measure.
  • Reliability: The consistency of a measure across time, evaluators, or items.
  • Evaluation Techniques:
    • Strategies to assess reliability:
    • Test-retest reliability: Consistency of scores over time.
    • Inter-rater reliability: Agreement between different observers measuring the same variable.
    • Internal reliability: Consistency within items of a questionnaire.
    • Strategies to assess validity:
    • Face validity: Appearance of measuring what it should measure.
    • Content validity: Inclusion of all relevant parts of a construct.
    • Criterion validity: Correlation with a relevant outcome.
    • Convergent validity: Correlation with similar constructs.
    • Discriminant validity: Lack of correlation with dissimilar constructs.

Conceptual vs. Operational Definitions

Happiness

  • Conceptual Definition: Subjective well-being.
  • Operational Definition: Measured through a 5-item questionnaire assessing life satisfaction. Respondents rate satisfaction on a 7-point scale (1 = strongly disagree, 7 = strongly agree).
Scoring for Happiness Measurement
  • Lowest possible score = 5 (unhappiest).
  • Highest possible score = 35 (happiest).
  • Findings (Pavot & Diener, 1993): Most individuals reported happiness, with 63% of high school and college students scoring above 20.
Ladder of Life Satisfaction Example (Cantril, 1965)
  • Imagery used:
    • A ladder with steps numbered from 0 (worst life) to 10 (best life); where respondents indicate their current life satisfaction.

Types of Measures in Psychological Research

Self-report Measures

  • Involves recording responses to self-descriptive questions.
    • Can include parent or teacher reports for children.
    • Examples of self-report measures:
    • 5-item happiness measure
    • Ladder of Life Satisfaction
    • Mood ratings
    • Count of stressful events.

Observational Measures

  • Behavioral measures focus on recording observable behaviors or physical traces of behavior.
    • Examples:
    • Frequency of smiles
    • Duration of gaze at stimuli
    • Public legal records documenting stressful events.

Physiological Measures

  • Recording physiological data necessitating specialized equipment (e.g., fMRI, heart rate monitors).
    • Examples:
    • Brain activity
    • Hormone levels
    • Heart rate
    • Facial muscle movements (EMG)
    • Skin conductance.

Scales of Measurement

Requirements for Variables

  • All variables must have at least two levels.
  • Levels of operational variables can be coded based on the type of scale used:
    • Categorical (nominal) variables: Race/ethnicity, gender.
    • Quantitative variables: Coded with meaningful numbers.
    • Continuous variables: Height, weight, salivary cortisol, happiness scale.

Types of Quantitative Variables

  1. Ordinal: Numbers indicate a ranked order (e.g., top 10 best-selling books).
    • Note on intervals: Differences between ranks may be unequal (gold medalist vs. silver/bronze).
  2. Interval: Numerals represent equal intervals between levels and lack a true zero (e.g., temperature).
  3. Ratio: Numerals have equal intervals and true zero indicates absence (e.g., number of correct answers).

Reliability & Validity

Definitions

  • Reliability: Consistency of the results of a measure.
    • Can be categorized into:
    • Test-retest reliability.
    • Inter-rater reliability.
    • Internal reliability.
  • Validity: Operationalization reflects what it's intended to measure.
Types of Reliability
  • Test-retest reliability: Score consistency over multiple measurements.
  • Inter-rater reliability: Consistency of scores by different observers.
  • Internal reliability: Patterns of consistent responses across a questionnaire.

Correlation Coefficient (r) & Reliability

Defining Correlation Coefficient (r)

  • Describes both the strength and direction of the relationship between two variables, irrespective of measurement scale.
  • Interpretation:
    • r near 1.0 or -1.0 indicates stronger correlation.
    • Positive slope = positive r; negative slope = negative r.
Visual Representation
  1. Scatter diagrams demonstrate strength (closeness of data points).
  2. Perfect positive correlation: r = 1, perfect negative correlation: r = -1, no correlation: r = 0.

Internal Reliability and Measurements

  • Inter-item consistency and average inter-item correlation provide insight into reliability.
  • Cronbach’s alpha quantifies reliability over multiple items in a scale, higher values (closer to 1.0) indicate better reliability.

Validity Assessment

Subjective and Empirical Measures of Validity

Subjective Measures
  1. Face Validity: Subjective assessment that a measure appears to reflect the construct.
    • Example: Number of pets as a proxy for animal affection.
  2. Content Validity: A measure must encompass all relevant aspects of the construct.
    • Example: Operationalizations of intelligence should include components like reasoning and problem-solving.
Empirical Measures
  1. Criterion Validity: Association with real-world outcomes.
  2. Convergent Validity: Correlation with similar constructs.
  3. Discriminant Validity: Lack of correlation with dissimilar constructs.

Known-Groups Paradigm

  • Evaluates if a measure can differentiate between groups with known behavioral differences (e.g., GAD vs. non-GAD) through physiological measures.

Conclusion on Validity & Reliability

  • Reliability is a prerequisite for validity, yet not the sole determinant of it.
  • Understanding the measurements' operational definitions and their evaluations is crucial for sound psychological research and establishing construct validity.