PSYC 223 - Substance Use Meets Measurement (ONE)

Measurement

the process of assessing the size, amount of degree of an attribute using specific rules for transforming the attribute into numbers

Classical test theory

true score (T) + error (E) = observed test score (X)

Effect of random error on measurement

doesn't affect the average, only the variability around the average

Effect of systemic error

affects the average (bias)

Psychological test

all require a person to perform a behavior (observable and measurable action)

Psychological tests method

• Behavior used to make inferences about some psychological construct

2 differences between psychological tests and surveys

1. Psychological tests focus on individual outcomes while surveys focus on group outcomes

2. Psychological test results are reported as overall derived score or scaled scores while survey results are reported at the question level

Estimates of ____ - ____ are usually good enough for most purposes

0.7-0.8

Reliability of the test scores is high → standard error of the measurement is ________

low

Test-retest reliability

administer the same test to the same people at two points in time

Test-retest reliability formula

Pearson product-moment correlation

Alternative/parallel forms reliability

 administer two forms of the test to the same people

Alternative/forms/parallel forms reliability formula

Pearson product-moment correlation

Internal consistency (method 1)

give the test in one administration and then split the test into two halves for scoring

Internal consistency (method 1) formula

Pearson product-moment correlation corrected for length by the Spearman-Brown formula

Internal consistency (method 2)

give the test in one administration and then compare all possible split halves

Internal consistency (method 2) formula

coefficient alpha or KR-20

Interrater reliability

give the test once and have it scored (interval/ratio level) by two scorers or two methods

Interrater reliability formula

Pearson product-moment correlation

Interrater agreement (method 1)

Gve a rating instrument and have it completed by two or more judges

Interrater agreement (method 1) formula

Cohen's Kappa or intraclass correlations

Interrater agreement (method 2)

calculate the consistency of scores for one scorer across multiple tests

Interrater agreement (method 2) formula

 KR-20 or coefficient alpha

Content validity

extent to which the items on a test are representative of the construct the test measures

Criterion-related validity

whether a test indeed predicts what it claims to predict

2 types of criterion-related reliability

1. Predictive

2. Concurrent

Predictive validity

relationship between test scores and a future behavior correlate scores on a test with scores on future performance or behavior

Concurrent validity

determining whether scores on a specific test are systematically related to a criterion method collected at the same time as the test

Construct validity

gradual accumulation of evidence that the scores on the test are related to observable behaviors in the was predicted by the theory underlying the construct

2 types of construct validity

1. Convergent validity

2. Discriminant validity

Convergent validity

shows that constructs that should be theoretically related to the test scores are indeed related

Discriminant validity

shows that constructs that should not be related to the test scores are in fact not related

Face validity

concerned with how test takers perceive the attractiveness and appropriateness of a test

8 ways to operationalize substance use

1. Substance type

2. Initiation

3. Dependency

4. Intake (frequency and quantity)

5. Liability

6. Problematic use

7. Abstinence

8. Withdrawal

Sampling errors

snowball and respondent-driven sampling have been used

Snowball sampling

non-probability recruitment method where current research participants refer new subjects

Respondent-driven sampling

chain-referral method for surveying hard-to-reach groups, like drug users or sex workers, by having initial participants recruit peers from their social networks

Nonresponse errors

When selected sample members don't provide data, making the results unrepresentative of the whole population

3 measurement errors

1. Mode effects

2. Type of questions

3. Memory retrieval

Mode effects

Self-administration vs interviewer

Type of questions error

respondents are more likely to report binge drinking when answering close-ended questions than when answering open-ended questions

DUSI-R:

drug use screening inventory (self-report questionnaire) developed by Tarter in 1990

DUSI-R phasing

“ordinarily, how many times each month have you used each of the following drugs in the past year?”

DUSI-R format

Measures severity of problem in 10 domains: substance abuse, psychiatric disorder, health status, behavior problems, school adjustment, work adjustment, peer relations, social competency, family adjustment, and leisure/recreation

How does the DUSI-R ensure honesty?

Contains lie scale

When is the DUSI-R used?

Used in both clinical and research settings for both adults and adolescents

3 uses of the DUSI-R

1. Measuring current status

2. identifying areas in need of prevention

3. Evaluating the magnitude of change after a treatment intervention

Reference periods

Ued to restrict and specify the time intervals for which respondents are asked to report

Short reference frame approximate duration

About 7 days

Advantages of short reference frame

Exact recall

Disadvantages of short reference frame

Not suited for infrequent users

Long reference frame approximate duration

About 12 months

Long reference feeling advantage

Less likely to be affected by seasonal variations in substance use

Long reference frame disadvantage

Susceptible to recall concerns