Performance Appraisal Chapter 11: Evaluating Performance Ratings

What are the 4 rating data class measures?

• Measures of agreement and reliability

• Rater error measures

• Ratings accuracy measures

• Assessments of validity and usefulness

What is generalizability theory?

Analysis of diff sources of variability in rating

Studies on source variability concluded:

• Differences between ratees are usually larger than difference within ratees

• Differences between raters are as big as differences between ratees

• Ratings are not as unreliable as inter-rater correlations suggest

• Ratings included multiple sources of variability outside of employee performance

What are the distributional errors?

• Leniency

• Severity

• Range restriction

What is leniency?

Giving ratings that are uniformly high

What is severity?

Giving ratings that are uniformly low

What is range restriction?

Giving ratings that are pretty much the same from one person to another

What is halo error?

When raters tend to give similar evaluations across separate aspects of a person’s performance. Inflation of the intercorrelation among dimensions

Why have rater error measures fallen out of favor?

• Difficult to know true distribution of performance

• Separating role of rate behavior from role of potential rater errors is difficult

What are two types of accuracy measures?

• Behavior-based

• Judgmental

What are classes of evidence uses in evaluating construct validity?

• content of the scales

• Evidence of convergence validity

• Criterion-related validity

• Bias in performance ratings

There are assumptions that performance ratings contain biases about?

• Race

• Gender

• Age

Data suggest this bias is very small

What is the usefulness of ratings influenced by?

Reliability and validity