CH07- Psych402

What affects test scores?

Simple View / Classical Test theory - X = T + E •Observed Score = True Score + Random Error Modern View - Error - is not always random

Test protocol

Simplistic - What: the test items & scoring.

More Realistic - the entire situation: Set: Why? /Setting: Where/When /Examiner: Who? /Method of administration: How?

Stereotype Threat

Subject’s own beliefs about group performance affects individual performance

Theory: self-defeating cognitions increase load on Working Memory, lower engagement, motivation, etc.

Expectancy Effects

(A) selection bias in collecting data -- ignore data that seems wrong, accept data that fits your theory

(B) actually changing the environment -- encourage desired behavior by subtle or overt prompting

assessment

Test Design

Test Administration

Test Scoring

Test Interpretation

Automated Testing Issues

Boundaries of competence?

Scientific Basis?

Delegation of Work:

Use of Assessments:

Assessment by Unqualified Persons

subject variables

•Motivation •Anxiety • Illness •Medications •Hormones • Sleep • etc...

Behavioral Assessment

aka “Functional Assessment”

Work samples, on-the-job testing, “in situ” / “in vivo”

More active role of psychologist / observer / rater can lead to bigger problems with accurate measurement

Reactivity

Reliability of observers is highest when the observers are being observed

Methods: random sampling, covert sampling

Drift

Observers can be trained to certain level of accuracy, but their performance tends to change slowly over time.

Drift can happen on individual or group basis.

Group drift especially hard to counteract, since the group members tend to support each other’s ratings.

Drift is frequently ignored

Deception

People are very poor at detecting deception

Polygraph Examination

Lie detector tests: poor reliability /poor validity

prohibited by employee polygraph protection act of 1988 (EPPA)

False Positives

NAS concluded that if 10,000 employees (of whom 10 were spies) were given a polygraph:

Roughly 99.6% of those failing the test would be False Positives

false negatives

Notorious people not being caught by polygraphs

Detection of Malingering

Sometimes there are benefits to performing poorly on a test (disability, forensic, military, etc.) Often called “faking bad”

On some tests, an untrained person can’t know what “normal” performance is.

true positive

test results: +

real world: +

false positive

test results: +

real world: -

false negative

test results: -

real world: +

true negative

test results: -

real world: -