Week 7 - Psychometrics

Relevance and Sources of Help

  • Lectures:

    • Combine conceptual frameworks of psychometrics with applicable statistical knowledge.

    • Essential for final group research study and lab report.

    • Knowledge of psychometrics is valued by employers according to BPS.

  • Sources of Help:

    • Recommended reading list provided.

    • Discussion board available on Blackboard for peer and instructor engagement.

    • Pre-recorded analysis walkthroughs demonstrate SPSS analyses through example datasets.

    • Worksheets available for step-by-step analysis guidance.

Overview of Course Structure (Weeks 7-12)

  • Week 7: Introduction to Psychometrics

  • Week 8: Basics of Validity and Subscales

  • Week 9: Analysing Questionnaire Data in Validation Research; Filtering and Item Discrimination

  • Easter Break: Three weeks

  • Week 10: Principal Components Analysis

  • Week 11: Reliability and Validity Testing; Interpreting Questionnaire Scores

Session Focus

  • Defining Psychometrics

  • Understanding Psychometric Tests and their Use

  • Developing Quality Psychometric Tests

  • Selecting Suitable Psychometric Tests

  • Introduction to Reliability as a key feature of good tests

Definition of Psychometrics

  • Psychometrics: A branch of psychology dedicated to the measurement of variables.

  • Definition from Rust & Golombok (2009): "Branch of psychology dealing with measurable factors."

Importance of Psychometrics

  • Provides critical knowledge for psychologists in various assessments.

  • Functions as an evidence-based tool for:

    • Diagnosing psychological disorders.

    • Assessing cognitive abilities.

    • Identifying learning disabilities.

  • Enhances rigorous methodology critique in research.

  • Validating research improves data quality by ensuring accurate measurement of constructs.

  • Importance of aligning measurement with theory to facilitate better theoretical intervention and evolution.

Methods of Psychological Assessment

  1. Interviews

    • Pros: Direct interaction.

    • Cons: Prone to biases like Social Desirability and Interviewer Bias.

  2. Behavioral Observations

    • Observing behaviors directly in context.

  3. Psychological Tests

    1. Psychological Measurements

      • Numerical scores quantifying psychological constructs (e.g., traits, abilities).

    2. Psychological Profiles

      • Qualitative descriptions encompassing psychological characteristics and traits.

Comparison of Interviews and Psychometric Tests

  • Interviews:

    • Subjective results influenced by interviewer perceptions.

  • Psychometric Tests:

    • More objective and standardized assessments, yet still subject to some errors (e.g., cultural bias).

    • Should not solely dictate conclusions about individuals.

Example: South Oaks Gambling Screen (SOGS; Lesieur and Blume, 1987)

  • Overview:

    • 20-item self-report assessing gambling behavior.

    • Scoring scale:

    • 0 = No gambling problem

    • 1-4 = Some gambling problems

    • 5+ = Probable pathological gambling

  • Critiques:

    • Issues of construct validity.

    • Highlighted by Battersby et al. (2002).

  • Recent Findings:

    • SOGS meets criterion of validity generalization as per Gambino and Lesieur (2006).

Measurement of Psychological Constructs

  • Psychological constructs are abstract and lack physical forms.

  • Measurement involves observing the effects of constructs rather than direct observation (e.g., questionnaire responses).

Definition of a Psychometric Test

  • An objective and standardized measure of psychological attributes, abilities, or traits.

  • Typologies of Psychometric Tests (MAPIN):

    • Mental Ability and Aptitude: Assess cognitive skills and potential.

    • Achievement Tests: Evaluate knowledge proficiency in specific domains.

    • Personality Questionnaires: E.g., Big 5 Personality Test.

    • Interest Questionnaires: Determine preferences for various work and study areas.

    • Neurological Tests: Assess cognitive and emotional functioning, diagnose disorders.

  • Distinction:

    • Maximum Performance Tests: E.g., IQ tests (assess peak performance).

    • Typical Performance Tests: E.g., personality assessments (measure typical behaviors).

Uses of Psychometric Tests

  1. Clinical Psychology:

    • For diagnosis, treatment assessment, etc.

  2. Educational Psychology:

    • Assessing educational success and progress tracking.

  3. Careers Guidance:

    • Assessing motivation and interests.

  4. Occupational Psychology:

    • Evaluating competencies and training needs.

  5. Research:

    • Integral for final year projects.

Differentiating Psychometric Tests from Online or Magazine Tests

  • Key Attributes:

    • Rigour: Tests are well-researched and validated.

    • Standardization: Consistent administration and procedure.

    • Professional Oversight: Conducted under ethical standards.

    • Purpose and Validity: Establishes clear objectives and measures.

Practical Activity: Write Down Words

  • Task Overview:

    • Generate a list of words starting with the letter B in 30 seconds.

    • Discuss the implications of results for measuring intelligence.

Results Evaluation

  • Questions for Reflection:

    • Was administration uniform?

    • Was scoring fair?

    • Validity of fluency measure?

    • Can results predict future performance?

    • Can results be cross-tested with other measures?

Test Development Process

  • Essential Stages:

    • Rationale: Define constructs being measured and aim of the test.

    • Sources of Development:

    • Theory, prior research, and expert consultations.

    • Collaborate with populations relevant to the constructs.

Camouflaging Autistic Traits Questionnaire (CAT-Q)

  • Aim: Investigate autism-spectrum conditions and camouflage strategies used by individuals.

    • Camouflage: Strategies to hide or compensate for autism characteristics.

  • Importance: Addressing the consequences of camouflaging (e.g., mental health impacts).

  • Example Questions:

    • "In social situations, I feel like I'm 'performing' rather than being myself."

    • Reference: Full scale available in supplementary materials.

Importance of Subscales in Questionnaires

  • Subscales measure dimensions within overarching constructs.

    • Example: Big-5 Personality

    • O: Openness

    • C: Conscientiousness

    • E: Extraversion

    • A: Agreeableness

    • N: Neuroticism

  • Using a single questionnaire format that measures all facets.

Test Selection Criteria

  • Identifying the Best Fit for Testing Needs:

    • Consistency: Reliability of tests reported in manuals.

    • Validity: Ensuring the test measures its intended construct.

    • Consider appropriateness—age, clinical vs. non-clinical samples.

    • Standardized administration and scoring methods aid in reliability.

    • Normative data interprets test results relative to established standards.

Reliability in Psychometrics

  • Definition: The consistency of a test over repeated administrations.

  • Illustration: Analogy of consistently hitting a target—reliable measures cluster together.

Types of Reliability

  1. Internal Consistency:

    • Measurement of consistency among test items.

    • Measured commonly using Cronbach’s Alpha.

  2. Temporal Stability:

    • Measures stability of results over time.

    • For example, measuring height on different days should yield consistent results.

Application of Reliability

  • Importance for:

    • Developers when creating measures.

    • Users (researchers/practitioners) when selecting tests for study.

  • Ensure familiarization with reliability when using tests in academic work.

Validity in Psychometrics

  • Definition: The degree to which a test measures what it claims.

  • Types of Validity:

    1. Face Validity: Perceived by respondents.

    2. Content Validity: Representation of items.

    3. Criterion-Related Validity:

    • Predictive validity = Future performance predictions.

    • Concurrent validity = Relationship with other test scores.

    1. Construct Validity: Alignment with theoretical understanding.

Considerations for Validity

  • Test developers need to ensure robust constructs are measured.

  • Users should select tests that are validated and suitable for their research purpose.

  • Test validation is a continuous process that starts at development and persists during application.

Standardization in Psychometrics

  • Defined Procedures:

    • Consistent methods for test administration and scoring to minimize errors.

    • Normative data assists in interpretation of test scores.

Professional and Ethical Standards

  • Restricted access to tests ensures qualified administration.

  • Adhering to ethical standards maintains the integrity of psychological assessments.

Normative Data and Interpretation

  • Norms: Summary of typical test results for a representative sample.

  • Comparison of individual scores to norms allows interpretation of achievement levels.

Final Thoughts and Summary

  • Psychometrics emphasizes the measurement of individual differences through varied test types.

  • Key aspects include reliability, validity, and standardization in testing.

  • Ethical considerations are paramount in maintaining the quality and integrity of psychological assessments.

Preview of Upcoming Week 8 Topics

  • Key components of validity and subscale analysis.

  • Exploration into the nature and significance of validity in psychometrics.