Chapter 5 - Identifying Good Measurement

Ways of Measurment

1.) Self Report

2.) Observation

• Naturalistic Observation

• Systematic Observation

3.) Physiological Measures

Self Report

A method where individuals evaluate and provide insights about their own thoughts, feelings, or behaviors, often through surveys or questionnaires.

Naturalistic Observation

A research method that involves observing subjects in their natural environment without any manipulation or control by the observer, allowing for the collection of data on behavior in real-world settings.

Systematic Observation

A research method that involves observing and recording behavior in a structured manner, often using predefined criteria and systematic protocols to ensure consistency and objectivity.

Physiological Measures

Methods that assess biological data through techniques such as heart rate monitoring, brain imaging, or hormone analysis, providing objective evidence of physiological responses related to thoughts, feelings, or behaviors.

Reliability

The degree to which a measurement is consistent across different instances, ensuring that the results are stable and repeatable over time. Reliability indicates the extent to which a measurement tool yields the same results under consistent conditions. The degree to which a measurement is consistent

Includes:

• Test-Retest Reliability

• Internal Reliability

• Inter-Rater Reliability

Test-Retest Reliability

A measure of reliability that examines the stability of a test over time by comparing scores from the same individuals across multiple testing instances (ex. doing the same personality test a week apart).

Internal Reliability

• Internal consistency of a test, assessing how well items on a test measure the same construct by examining correlations among them. This ensures that all parts of the test yield similar results. Only relevant to self report scales with multiple items.

• Cronbach’s Alpha (0-1). The closer to 1, the better.

• Average Inter-Item Correlation (AIC) is a measure of the average correlation between items in a test, indicating how uniformly the items assess the same underlying construct.

Inter-Rater Reliability

A measure of reliability that assesses the degree to which different raters or observers give consistent estimates of the same phenomenon. It is crucial in studies where subjective judgments are used.

Ways to Improve Reliability

• Increase the precision of measurement (defining variables)

• Increase the number of indicators (ex. asking multiple questions about different aspects of one life vs just asking “how satisfied are you with your life?).

Subjective Measures of Validity

• Face Validity

• Content Validity

Face Validity

The extent to which a measure appears to assess the construct it is intended to measure, based on subjective judgment.

Content Validity

The degree to which a measurement instrument comprehensively covers the content it aims to measure, ensuring that all aspects are adequately represented.

Evidence-Based Types of Construct Validity

• Criterion Validity - Predicts behavior well (good criterion validity allows you to predict behavior)

• Convergent validity - The degree to which a measure correlates with other measures of the same or similar constructs; it should show significant relationships with related constructs (high r).

• Discriminant Validity - The degree to which a measure does not correlate with measures of different constructs, ensuring distinctiveness between different constructs (low r).

Reliability + Validity

• A measure must be reliable for it to be valid, but reliability alone is not sufficient enough for establishing validity (ex. a clock being 5 minuets ahead is reliable, but not valid).

The Rosenberg Scale of Self Esteem (RSE)

A widely used self-report instrument that assesses self-esteem through a 10-item scale, measuring global feelings of self-worth or self-acceptance.