PT7400- Appraising Self-Report Outcome Measures

Outcome measures

Measures taken to assess the impact of a disease or disorder on the patient. Ex: Impairment (ROM, pain, strength), function/disability (observed include gait and ADLs, self-report include health status and quality of life).

Survey; difficulty; assistance; psychological

Self-report measures of function and disability are ______________________ instruments that may:

- Assess the level of ______________________ a patient has when performing a task.

- Assess the need for _______________________ a patient has with a task.

- Assess the _______________________ (or other) impact of the disease or disorder on the patient's job, activities, family, and/or social life (i.e., quality of life).

Condition; region

Types of Self-Report Measures

- Generic (SF-36)

- Disease or _______________________ specific (KOOS)

- _______________________ specific (LEFS)

- Single questions → Global rating (GRC) and patient satisfaction

SF-36

Measure of overall health and quality of life, used to evaluate other outcomes.

KOOS

Knee and osteoarthritis outcomes, specific to the knee.

LEFS

Lower extremity.

Improved/declined; treatment

Single question self-report measures can be used to measure how patient feels they've _______________________ over time, with _______________________, etc.

Ratio or interval data

Cumulative scores from self-reported outcome measures can be used as _________________________ (Pearson correlation coefficient, ICC) even though questions are ordinal.

Relevant condition; shorten; translate; development; clinimetric

Evidence about Self-Report Measures:

- New measure development → Are all these questions going to ask about the ______________________ (i.e., knee OA)? What if I _____________________ the outcome measure (i.e. Quick DASH)?

- Established measures under new conditions → Is this outcome measure used for knee OA also applicable to chronic ankle sprains? What if I move it from pen and paper to online? What if I _________________________ it to Spanish?

- Usefulness of evidence depends upon extent to which: Instrument ______________________ is described and _______________________ properties are evaluated.

Patients; content experts; how many; phenomena

Is there an adequate description of the survey development process, including identification of participants and methods for item development and testing?

Addresses:

- Content: Topics to cover → Input from ___________________, ___________________

- Items: Specific questions asked → ____________________ questions are sufficient? What ____________________ are they intended to reflect? (Theoretical approach, Statistical approach/factor analysis)

Resource demands; interpretability

Administration and scoring requirements? How easily interpretable are the scores both from clinician and patient? → Addresses ______________________ for patient (or caregiver) and clinician.

Also need _____________________ of the scale (how to interpret the final total of points, is higher score good or bad, etc.).

Reproducibility; internal consistency

Is the instrument reliable? → Addresses _______________________.

Test-retest

Reproducibility (stability) of a score when a measure is repeated under the same conditions at the same point in time.

Internal consistency

Relationship between items in a questionnaire → Are all the items measures of the same construct? (Ex: Measuring LE function with standing balance, walking, and jumping → All measures of LE function but different.)

Correlation coefficients (ICC, kappa)

Reliability of test-retest is measured by ________________________.

0.75; 0.51

ICC >___________________ and kappa >____________________ for test-retest reliability.

0.00-0.25

Kappa score of "little to no agree".

0.26-0.5

Kappa score of "fair".

0.51-0.75

Kappa score of "moderate to good".

0.76-1.0

Kappa score of "good to excellent".

Cronbach's alpha (α)

Reliability of internal consistency is measured by ______________________.

0.7-0.9

α represents correlation of like items and should be ____________________.

Distinct areas

Items should all measure the same construct overall, but they should measure ______________________ of construct (i.e., walking vs. sitting LEFS).

Redundancy

α >0.9 indicates ___________________ of questions.

Face; content; construct; criterion

Is the instrument valid? → Address __________________ validity, __________________ validity, ___________________ validity, ____________________ validity

Face validity

Does the measurement appear, on the face of it, to assess what is intended?

Content validity

The extent to which a measurement is judged to reflect the meaningful elements of a variable.

Construct validity

Are the things you chose to measure a construct actually good for measuring the thing?

Convergent validity and discriminant validity

What are the two forms of construct validity?

Convergent validity

Does it predict change in two similar measures?

Divergent validity

Does it differentiate between people with high and low levels of the thing you're measuring?

Criterion validity

How the measure compares to the "gold standard".

Predictive validity and concurrent validity

What are the two forms of criterion validity?

SF-36

_______________________ may be used in research as "gold standard".

Instrument scores

Correlation coefficients → _______________________ vs. scores from gold standard

Regression equations

Predictive validity.

Measurement error; floor; ceiling

Is the instrument responsive? → Addresses:

- Ability to detect change beyond _____________________ → Clinically meaningful?

- ______________________ effects

- ______________________ effects

Higher; appropriate

Ceiling effects → May have ceiling effect for people with _____________________ levels of function so there would be a different, more ______________________ outcome measure.

Measurement error

Error in the measurement technique.

Person; environment

Sources of error:

- Instrument

- _____________________ collecting measure

- ____________________

- Patient

Single; change

Forms of error:

- Error in a ______________________ score

- Error in the _______________________ score

Standard error of measurement (SEM)

Error at a single point in time; repeated measures (test-retest). → If I repeat LEFS on you, there will be some error.

Minimal detectable change (MDC)

Test-retest: Error in the change score (2 points in time). → Change in LEFS score should be beyond error.

Standardized response mean (SRM)

Mean of change scores/standard deviation of change scores.

1.0

Criteria for "good" SRM: >___________________

Effect size (ES)

Mean of change scores/standard deviation of initial scores.

0.8

Criteria for "large" effect: >_____________________

Scores; performance

Can survey responses be interpreted in a meaningful way? → Addresses the relationship between _____________________and the influences on patient ______________________.

Reexamine; different

If there is more than one method for administering the self-report instrument of interest, did the authors _____________________ its measurement properties for each mode? → Paper vs. online vs. verbal → Addresses the extent to which a measure has been evaluated for use in circumstances ___________________ from those in which it was developed.

Cultures or languages; different

If the self-report instrument of interest is being considered for use in other _____________________, did the authors reexamine its measurement properties under these new conditions? → Addresses the extent to which a measure has been evaluated for use in circumstances ___________________ from those in which it was developed.