PY2501 Items and Scales 2024 - Tagged
Page 1: Introduction
Course Title: PY2501 Research Methods and Data Analysis - Questionnaire Design: Items and Scales
Instructor: Prof Andrew Schofield, based on slides by Prof Adrian Burgess
Page 2: Lecture Schedule
Week 3: Theory of measurement
Week 8: From items to scales
Week 9: Assessing quality of scales and their meaning
Page 3: Scale Development Principles
Don’t reinvent the wheel: Utilize existing reliable and valid scales when possible.
Reliability and Validity: Critical yet challenging to achieve
Page 4: Psychometrics Overview
Topics Covered:
Transition from items to scales
Creation of items
Scaling responses
Understanding multivariate scales
Page 5: Multidimensional Psychological Traits
Psychological traits are multidimensional; fear consists of three components:
Physiological: Example indicators include GSR (Galvanic Skin Response), ECG (Electrocardiogram)
Behavioral: Fight, flight, freeze responses
Cognitive/Affective: Subjective emotional states
Some phenomena require subjective inquiry through direct questions.
Page 6: Item Generation Techniques
Sources for Item Development:
Psychological Theory: Start with robust theoretical frameworks.
Systematic Search: Includes literature review and expert consultation
Page 7: Deriving Items from Theory
Define a psychological trait. For example, extroversion:
Questions might include:
"Do you enjoy going to parties?" (Yes/No)
Reversed statements such as "I like to sit quietly and read".
Ensure all questions align with the trait definition, avoiding irrelevant questions about transient states.
Page 8: Systematic Search for Item Generation
Semantic Links:
Explore related terms in dictionaries or web resources.
Expert Consultation: Engage focus groups and informants for input.
Page 9: Ensuring Content Validity
Items should encompass the entire spectrum of the domain (e.g., fear of spiders includes behavioral, physiological, cognitive/affective dimensions).
Example items:
“I feel an urge to run away at the sight of a spider.”
Page 10: Item Design Considerations
Single Focus: Avoid composite questions (e.g., "I like parties and dancing").
Readability: Strive for an 8-year reading age.
Cultural Sensitivity: Consider potential translation or cultural misunderstandings.
Page 11: Scaling Overview
Definition: Transform qualitative attributes into quantitative measures.
Two Approaches to Scaling:
Comparative Scaling
Direct Estimation
Page 12: Comparative Scaling Techniques
Techniques include:
Paired Comparison
Thurstone’s Method
Steven’s Scaling
Guttman Scaling
Page 13: Paired Comparison Technique
Identifies the key aspects that determine lecturer quality based on preferences.
Page 14: Theoretical Basis for Paired Comparison
Assumptions about latent psychological measurement scales and normally distributed weighting estimates.
Page 15: Algorithm for Paired Comparison
Pick a small set of relevant items.
Have judges compare each pair.
Convert preferences into z-scores.
Average z-scores for final scaling.
Page 16: Step-by-Step Paired Comparison
Assess pairs of lecturer attributes for importance.
Page 17: Converting Preferences to z-scores
Example: 60% preference yields a z-score of 0.26.
Page 18: Weight Conversion from z-scores
Final scores convert z-scores to a scaled weight reflecting importance.
Page 19: Direct Estimation Method
Participants provide direct ratings of their experiences (e.g., fear levels).
Page 20: Response Formats
Various ways to quantify fear:
Dichotomous
Rating scales
Visual analogue scales
Page 21: Likert Scale Application
Common formats to gauge responses to fear levels through various comparative phrases.
Page 22: Frequency, Intensity, Duration
Considerations for how to ask about fear across different timeframes.
Page 23: Scaling Considerations and FAQ
Balance between too few and too many response categories.
Importance of clarity and context for adjectives used.
Page 24: Response Formats Continued
Understand implications of number choices and consistency.
Page 25: Direct Estimation vs. Comparative Scaling
Efficiency of direct estimation versus the reliability of comparative scaling.
Page 26: Quiz 1
Topics covered in the first quiz.
Page 27: Advantages of Scales
Improves accuracy and reliability, achieving interval level measurement.
Page 28: Classical Test Theory Overview
Factors affecting observations and reliability calculations.
Page 29: Reliability and Validity Through Multiple Measurements
Errors can cancel out over repeated measurements.
Page 30: Repeated Measurements Insight
Challenges of repeated questions in psychological contexts due to memory effects.
Page 31: Classical Test Theory on Errors
Discusses random error distributions in psychological measurement.
Page 32: Interval Level Assumptions
Rating scales often viewed as ordinal but can be treated as interval under certain conditions.
Page 33: Justifications for Interval Level Data
Theory supporting the treatment of ordinal data as interval data.
Page 34: Benefits of Using Interval Level Data
Enables precise statistical analysis and inferences.
Page 35: Scale Correlation and Justification
Importance of item correlation for scale validity and internal consistency.
Page 36: Correlation and Shared Variance Concept
Correlation as a measure of variance and reliability of item sets.
Page 37: Correlation Categories
Ranges of correlation coefficients indicating strength and shared variance.
Page 38: Internal Consistency Assessment
Homogeneity in items influences scale reliability.
Page 39: Internal Consistency Reliability Criteria
Correlated items validate the formation of a scale.
Page 40: Measuring Internal Consistency
Methods such as item-total correlation, split-half reliability, and Cronbach’s Alpha.
Page 41: Cronbach's Alpha Interpretation
Thresholds for assessing internal consistency of scales.
Page 42: Quiz 2
Topics covered in the second quiz.
Page 43: Multivariate Psychological Constructs
Understanding constructs like personality and intelligence, which consist of multiple factors.
Page 44: Analyzing Correlations in Multivariate Data
Identifying potential multiple latent factors through correlation inspection.
Page 45: Factor Analysis Methodology
Statistical approach to explore latent variable representation in scale measurements.