PY2501 Items and Scales 2024 - Tagged

PY2501 Research Methods and Data Analysis

Course Overview

• Course Title: PY2501 Research Methods and Data Analysis

• Focus: Questionnaire Design: Items and Scales

• Instructor: Prof Andrew Schofield

• Content Based On: Slides by Prof Adrian Burgess

Lecture Schedule

• Week 3: Theory of Measurement

• Week 8: From Items to Scales

• Week 9: Evaluation of Scale Quality and Interpretation

Key Principles of Scale Design

• Avoid Redundancy: Do not create new scales if reliable ones already exist.

• Essentials: Good scales must be both reliable and valid.

• Challenges: Achieving these attributes in scale creation can be difficult.

Psychometrics Overview

• Main Topics:

  • Transition from items to scales

  • Development of items for measurement

  • Scaling and response methods

Measurement of Psychological Traits

• Psychological traits are often multidimensional, shown through:

  • Physiological (e.g., GSR, ECG responses)

  • Behavioral (flight/fight/freeze response)

  • Cognitive/Affective (subjective feelings)

• Subjective phenomena require direct inquiry (asking individuals).

Generating Measurement Items

Approaches

1. From Psychological Theory: Start with hypotheses around characteristics (e.g., extroversion).

2. Systematic Search for Items: Utilize resources like dictionaries and expert consultations.

Example of Item Creation

• A well-defined theory leads to specific questions:

  • Example (Extroversion):

    • "Do you enjoy going to parties? (Y/N)"

    • Include reverse-scaling items (e.g., "I prefer quiet activities.")

Ensuring Content Validity

• Items must reflect all dimensions of a construct (e.g., fear of spiders). Example questions:

  • Behavioral: "I feel the urge to flee when I see a spider."

  • Physiological: "Seeing a spider makes my heart race."

  • Cognitive/Affective: "I often worry about encountering spiders."

Item Construction Guidelines

• Simplicity: Questions must only ask one thing (avoid compound questions).

• Readability: Aim for clarity understandable to an average 8-year-old.

• Cultural Appropriateness: Ensure language is suitable across different cultures.

Scaling Methods Overview

• Scaling Definition: Transform qualitative attributes into quantitative variables.

• Types of Scaling:

  • Comparative Scaling (e.g., paired comparison)

  • Direct Estimation (e.g., rating scales)

Comparative Scaling Techniques

• Formulate criteria for evaluating attributes (e.g., lecturer quality).

• Paired Comparison Method: Estimates item differences using judges’ preferences.

Steps in Paired Comparison

1. Select relevant items to compare.

2. Engage judges for preferences on pairs.

3. Conversion of preferences to statistical measures like z-scores.

Direct Estimation Techniques

• Eliciting direct responses quantified (e.g., assessing fear levels).

• Response Formats: Various scales such as Likert, visual analogue scales, etc.

Considerations in Scale Length and Format

• The number of response steps must balance information richness and statistical reliability.

• Labeling of Categories: Important for clarity; endpoints must always be labeled.

Justifications for Interval Level Data

• Statistical Operations: Assumption allows for parametric testing (means, standard deviations).

• Limitations: Be cautious about claims regarding values (e.g., IQ scales).

Reliability and Validity Considerations

• Measurement Goals: Use multiple items to reduce error and increase reliability (errors are random).

• Analyze correlations between items to ensure internal consistency (e.g., Cronbach's Alpha).

Tools for Assessing Internal Consistency

• Item-Total Correlation: Correlation between single items and total scores.

• Split-Half Reliability: Comparing results from two halves of the items.

• Cronbach’s Alpha: Statistical measure for overall consistency across items.

Addressing Multivariate Constructs

• Recognize that some psychological scales measure multiple underlying variables (e.g., personality traits).

• Use Factor Analysis to identify latent factors and correlations among items.