[4] Evaluating Selection Techniques and Decisions

Industrial/Organizational Psychology: An Applied Approach - Michael G. Aamodt

Reliability

The extent to which a score from a test or from an evaluation is consistent and free from error.

Test-retest reliability

The extent to which repeated administration of the same test will achieve similar results.

Temporal stability

The consistency of test scores across time.

Alternate-forms reliability

The extent to which two forms of the same test are similar.

Counterbalancing

A method of controlling for order effects by giving half of a sample Test A first, followed by Test B, and giving the other half of the sample Test B first, followed by Test A.

Form stability

The extent to which the scores on two forms of a test are similar.

Item stability

The extent to which responses to the same test items are consistent.

Item homogeneity

The extent to which test items measure the same construct.

Kuder-Richardson Formula 20 (K-R 20)

A statistic used to determine internal reliability of tests that use items with dichotomous answers (yes/no, true/false).

Split-half method

A form of internal reliability in which the consistency of item responses is determined by comparing scores on half of the items with scores on the other half of the items.

Spearman-Brown prophecy formula

Used to correct reliability coefficients resulting from the split-half method.

Coefficient alpha

A statistic used to determine internal reliability of tests that use interval or ratio scales.

Scorer reliability

The extent to which two people scoring a test agree on the test score, or the extent to which a test is scored correctly.

Validity

The degree to which inferences from test scores are justified by the evidence.

Content validity

The extent to which tests or test items sample the content that they are supposed to measure.

Criterion validity

The extent to which a test score is related to some measure of job performance.

Criterion

A measure of job performance, such as attendance, productivity, or a supervisor rating.

Concurrent validity

A form of criterion validity that correlates test scores with measures of job performance for employees currently working for an organization.

Predictive validity

A form of criterion validity in which test scores of applicants are compared at a later date with a measure of job performance.

Restricted range

A narrow range of performance scores that makes it difficult to obtain a significant validity coefficient.

Validity generalization (VG)

The extent to which inferences from test scores from one organization can be applied to another organization.

Synthetic validity

A form of validity generalization in which validity is inferred on the basis of a match between job components and tests previously found valid for those job components.

Construct validity

The extent to which a test actually measures the construct that it purports to measure.

Known-group validity

A form of validity in which test scores from two contrasting groups “known” to differ on a construct are compared.

Face validity

The extent to which a test appears to be valid.

Barnum statements

Statements, such as those used in astrological forecasts, that are so general that they can be true of almost anyone.

Mental Measurements Yearbook (MMY)

A book containing information about the reliability and validity of various psychological tests.

Computer-adaptive testing (CAT)

A type of test taken on a computer in which the computer adapts the difficulty level of questions asked to the test taker’s success in answering previous questions.

Taylor-Russell tables

A series of tables based on the selection ratio, base rate, and test validity that yield information about the percentage of future employees who will be successful if a particular test is used.

Selection ratio

The percentage of applicants an organization hires.

Base rate

Percentage of current employees who are considered successful.

Proportion of correct decisions

A utility method that compares the percentage of times a selection decision was accurate with the percentage of successful employees.

Lawshe tables

Tables that use the base rate, test validity, and applicant percentile on a test to determine the probability of future success for that applicant.

Utility formula

Method of ascertaining the extent to which an organization will benefit from the use of a particular selection system.

Tenure

The length of time an employee has been with an organization.

Measurement bias

Group differences in test scores that are unrelated to the construct being measured.

Adverse impact

An employment practice that results in members of a protected class being negatively affected at a higher rate than members of the majority class. it is usually determined by the four fifths rule.

Predictive bias

A situation in which the predicted level of job success falsely favors one group over another.

Single-group validity

The characteristic of a test that significantly predicts a criterion for one class of people but not for another.

Differential validity

The characteristic of a test that significantly predicts a criterion for two groups, such as both minorities and nonminorities, but predicts significantly better for one of the two groups.

Multiple regression

A statistical procedure in which the scores from more than one criterion-valid test are weighted according to how well each test score predicts the criterion.

Top-down selection

Selecting applicants in straight rank order of their test scores.

Compensatory approach

A method of making selection decisions in which a high score on one test can compensate for a low score on another test. For example, a high GPA might compensate for a low GRE score.

Rule of three

A variation on top-down selection in which the names of the top three applicants are given to a hiring authority who can then select any of the three.

Passing score

The minimum test score that an applicant must achieve to be considered for hire.

Multiple-cutoff approach

A selection strategy in which applicants must meet or exceed the passing score on more than one selection test.

Multiple-hurdle approach

Selection practice of administering one test at a time so that applicants must pass that test before being allowed to take the next test.

Banding

A statistical technique based on the standard error of measurement that allows similar test scores to be grouped.

Standard error of measurement (SEM)

The number of points that a test score could be off due to test unreliability.