Quiz 2 - DPT 653 USM

What is the concept of reliability defined as?

How can you reproduce something with consistency

What is the concept of validity defined as?

How accurate or correct something is

What example is not valid and not reliable?

marks on a target spread out and not close to the bullseye

What example is not valid but reliable?

Marks on a target close together but not close to the bullseye

What example is both valid and reliable?

When the marks are close together and on the bullseye

What is the definition of reliability?

The extent to which a measurement is consistent and free from error

Can a reliable measure be expected to repeat the same score on 2 or more different occasions?

yes it can be, as long as the characteristic of interest doesn't change

How is reliability tied to the concept of "measurement error"

in the way that it is the inverse of measurement error

- when 1 goes up, the other goes down

Examples of measurement error

bathroom scale: +/- 1lbs

range of motion: +/- 5 degrees

temp outside: +/- 1 degree C

body temp: +/- 0.2 degree C

oswestry disability sale: +/- 4 points

measurement error equation

observed score = true score +/- error

In the example of "Dr clark gives an exam," what would the observed score be?

How well you did

In the example of "Dr clark gives an exam," what would the true score be?

the actually score

In the example of "Dr clark gives an exam," what would the error be?

sleep, coffee, etc

What are the 2 types of measurement errors?

systematic error and random error

What is a systematic error?

an error that always either overestimates or underestimates

What is random error?

a variable due to chance or is unpredictable

What does the concept of regression to the mean state?

That when you repeat the same test, and nothing changes, your values should get closer to the mean value

What are the three sources of measurement error?

1. Rater or human error

2. Instrument error (faulty function)

3. variability of the characteristic being observed

What is an example of the variability of a characteristic being observed

blood glucose

- it changes over time

What is relative reliability?

The ratio of total variability between scores to variability within scores is measured as a unitless coefficient

What is the equation for relative reliability?

Rxx = true score variability/ true score variability + error variability

At the top of the equation, "true score variability" is what?

the difference BETWEEN individuals

At the bottom of the equation, "true score variability + error variability" is what?

the difference WITHIN individuals

If Rxx = 0? What does this mean?

that there is no reliability

If Rxx = 1? What does this mean?

1 is perfect reliability

If true variance is larger, what will that tell us about Rxx?

that Rxx will be larger

What is absolute reliability?

How much of a measured value is likely due to error, and the standard error of the measurement (SEM) used

How can you improve reliability?

1. standardize measurement methods (same steps every time)

2. Take multiple measurements (cancels the effects of random error)

3. train and test observers (reduces human error)

4. calibrate and improve instruments

3 types of reliability most relevant for clinicians?

test-retest

inter-rater

intra-rater

3 types of reliability of questionnaires and surveys, or for comparing different forms of testing?

internal consistency

split-hald

alternate forms

What is the test-retest reliability

Any instrument or tool that is capable of measuring a variable consistently

- usually 2 or more separate occasions, and keep conditions as constant as possible

Where do we see test-retest reliability?

used most for self-report questionnaires, and measures with mechanical/digital readouts

What is INTRA-rater reliability?

the within-rater

the degree that the examiner agrees with themself

What is rater bias?

when 1 rater takes 2 measurements and is influenced by the memory of the first score

How to avoid rater bias?

Use blinding and make assessments as objective as possible

What is INTER-rater reliability?

think "between" two different raters

- 2 or more raters who measure the same group of people

What is internal consistency?

it estimates how well the items that reflect the same construct yield similar results

What is internal consistency used for?

used to construct and evaluate scores and/or questionnaires

How do we measure internal consistency?

Cronback's alphs

- types of average inter-item correlation

What is split-half reliability (a branch of internal consistency)?

When you take all the questions, divide them, and then compare the halves

What is the purpose of alternate forms of reliability?

to see if 2 different versions of the same instrument are equivalent

ex: ROM

What are the issues affecting the validity of change?

levels of measurement

reliability (is it change or just measurement error)

stability (are there meaningless natural fluctuations)

baseline score

What is minimal detectable change (MDC)?

When the change is real but not clinically meaningful

MDC def

value at which the observed change is NOT measurement error

What is the minimal clinically important difference (MCID)

when the change is real and clinically significant

MCID def

The ability of an instrument to detect minimally important change

Measurement validity def

The extent to which an instrument measures what it is intended to measure

What are the 3 things you have to take into consideration during measurement validity?

can the text...

1. discriminate among individuals

2. evaluate change in magnitude or quality from one time to another

3. Predict an outcome

What comes first? reliability or validity?

Reliability is a prerequisite for validity

difference between reliability and validity?

Reliability = consistency of a measurement

validity = alignment of the measurement with a targeted construct

Are validity and reliability an all-or-none concept?

no they are evaluated on a continuum

What is face validity?

When an instrument appears to test what it is intended to test

- It is judged by the users of a test after the test is developed

Face validity is judged by those who _____ the instrument

USE

What are the 3 C's of validity?

contect

construct

criterion-related

What does content validity look at?

If measurements adequately represent the concept

What is content validity used for?

They are typically used in questionnaire development

What are the 3 requirements for content validity?

1. Items must represent the full scope of the construct being studied

2. The number of items is proportionate to the importance of that component

3. No irrelevant items

Content validity is judged by those who ____ the instrument

DEVELOP

What is the similarity between face validity and content validity?

There are no statistics to measure either type of validity

Criterion-related validity def

Can the outcomes of the instrument or index test be substituted for an established gold standard

What is criterion-related validity measured by?

correlation coefficients

sensitivity/specificity analyses

What are the 2 types of criterion-related validity?

concurrent and predictive validity

concurrent validity def

Scores from a new measure correlate with scores from a well-established measure administered AT THE SAME TIME

predictive validity def

establishes that the outcome of the target test can be used to predict a future criterion score or outcome

Construct validity def

reflects the ability of an instrument to measure the theoretical dimensions of a construct

What are the 3 types of construct validity?

known-group method

convergence

divergence

What does the known-group method test?

tests if the results differ between 2 groups that are known to be different

What does convergence show

If there is a correlation between 2 constructs

if R = 1, what does this mean?

1 is a perfect correlation

very strong correlation

What does divergence show?

show that there isn't a correlation between 2 constructs

If R = 0, what does this mean?

If R=0, there is no correlation