Measurement Foundations & Validity
Research Methods
Measurement Foundations & Validity
Role of Conceptual and Operational Variables and Scales of Measurement
- Conceptual variables: The theoretical definitions of variables as understood at an abstract level.
- Operational variables: Specific decisions about how to measure or manipulate conceptual variables in research.
- Scales of measurement: The various ways in which variables can be quantified and categorized.
- Importance in Psychological Science: Understanding both types of variables aids in correctly interpreting study findings and enhances the rigor of psychological research.
Validity and Reliability of Measured Variables
- Validity: The degree to which an operational definition measures what it claims to measure.
- Reliability: The consistency of a measure across time, evaluators, or items.
- Evaluation Techniques:
- Strategies to assess reliability:
- Test-retest reliability: Consistency of scores over time.
- Inter-rater reliability: Agreement between different observers measuring the same variable.
- Internal reliability: Consistency within items of a questionnaire.
- Strategies to assess validity:
- Face validity: Appearance of measuring what it should measure.
- Content validity: Inclusion of all relevant parts of a construct.
- Criterion validity: Correlation with a relevant outcome.
- Convergent validity: Correlation with similar constructs.
- Discriminant validity: Lack of correlation with dissimilar constructs.
Conceptual vs. Operational Definitions
Happiness
- Conceptual Definition: Subjective well-being.
- Operational Definition: Measured through a 5-item questionnaire assessing life satisfaction. Respondents rate satisfaction on a 7-point scale (1 = strongly disagree, 7 = strongly agree).
Scoring for Happiness Measurement
- Lowest possible score = 5 (unhappiest).
- Highest possible score = 35 (happiest).
- Findings (Pavot & Diener, 1993): Most individuals reported happiness, with 63% of high school and college students scoring above 20.
Ladder of Life Satisfaction Example (Cantril, 1965)
- Imagery used:
- A ladder with steps numbered from 0 (worst life) to 10 (best life); where respondents indicate their current life satisfaction.
Types of Measures in Psychological Research
Self-report Measures
- Involves recording responses to self-descriptive questions.
- Can include parent or teacher reports for children.
- Examples of self-report measures:
- 5-item happiness measure
- Ladder of Life Satisfaction
- Mood ratings
- Count of stressful events.
Observational Measures
- Behavioral measures focus on recording observable behaviors or physical traces of behavior.
- Examples:
- Frequency of smiles
- Duration of gaze at stimuli
- Public legal records documenting stressful events.
Physiological Measures
- Recording physiological data necessitating specialized equipment (e.g., fMRI, heart rate monitors).
- Examples:
- Brain activity
- Hormone levels
- Heart rate
- Facial muscle movements (EMG)
- Skin conductance.
Scales of Measurement
Requirements for Variables
- All variables must have at least two levels.
- Levels of operational variables can be coded based on the type of scale used:
- Categorical (nominal) variables: Race/ethnicity, gender.
- Quantitative variables: Coded with meaningful numbers.
- Continuous variables: Height, weight, salivary cortisol, happiness scale.
Types of Quantitative Variables
- Ordinal: Numbers indicate a ranked order (e.g., top 10 best-selling books).
- Note on intervals: Differences between ranks may be unequal (gold medalist vs. silver/bronze).
- Interval: Numerals represent equal intervals between levels and lack a true zero (e.g., temperature).
- Ratio: Numerals have equal intervals and true zero indicates absence (e.g., number of correct answers).
Reliability & Validity
Definitions
- Reliability: Consistency of the results of a measure.
- Can be categorized into:
- Test-retest reliability.
- Inter-rater reliability.
- Internal reliability.
- Validity: Operationalization reflects what it's intended to measure.
Types of Reliability
- Test-retest reliability: Score consistency over multiple measurements.
- Inter-rater reliability: Consistency of scores by different observers.
- Internal reliability: Patterns of consistent responses across a questionnaire.
Correlation Coefficient (r) & Reliability
Defining Correlation Coefficient (r)
- Describes both the strength and direction of the relationship between two variables, irrespective of measurement scale.
- Interpretation:
- r near 1.0 or -1.0 indicates stronger correlation.
- Positive slope = positive r; negative slope = negative r.
Visual Representation
- Scatter diagrams demonstrate strength (closeness of data points).
- Perfect positive correlation: r = 1, perfect negative correlation: r = -1, no correlation: r = 0.
Internal Reliability and Measurements
- Inter-item consistency and average inter-item correlation provide insight into reliability.
- Cronbach’s alpha quantifies reliability over multiple items in a scale, higher values (closer to 1.0) indicate better reliability.
Validity Assessment
Subjective and Empirical Measures of Validity
Subjective Measures
- Face Validity: Subjective assessment that a measure appears to reflect the construct.
- Example: Number of pets as a proxy for animal affection.
- Content Validity: A measure must encompass all relevant aspects of the construct.
- Example: Operationalizations of intelligence should include components like reasoning and problem-solving.
Empirical Measures
- Criterion Validity: Association with real-world outcomes.
- Convergent Validity: Correlation with similar constructs.
- Discriminant Validity: Lack of correlation with dissimilar constructs.
Known-Groups Paradigm
- Evaluates if a measure can differentiate between groups with known behavioral differences (e.g., GAD vs. non-GAD) through physiological measures.
Conclusion on Validity & Reliability
- Reliability is a prerequisite for validity, yet not the sole determinant of it.
- Understanding the measurements' operational definitions and their evaluations is crucial for sound psychological research and establishing construct validity.