MGT 4322 CH 8 Measurement

Measurement

Assigning numbers based on conventions to aspects of people, jobs, success, or staffing system

The measures relevant to staffing are those that assess:

The characteristics of the job (job requirements), The characteristics of job candidates- (KSAO), Staffing outcomes (performance or turnover)

Why is proper measure important?

Improperly assessing and measuring candidate characteristics can lead to systematically hiring the wrong people, offending and losing good candidates, exposing your company to legal action

What is data?

The numerical outcomes of measurement

What are the 2 types of data?

Predictive data and Criterion data

What is predictive data?

s information about measures used to make projections about outcomes (Independent variable)

What is criterion data?

s information about important outcomes of the staffing process (Dependent variable)

Scoring

The process of assigning numerical values during measurement

Raw Scores

The unadjusted scores on a measure

Criterion referenced measures

Measures in which the scores have meaning in and of themselves E.g., Number of errors on a typing test

Norm-referenced measures

Measures in which the scores have meaning only in comparison to the scores of other respondents E.g., Quality of structured interview responses

Standard Scores

Converted raw scores that indicate where a person’s score lies in comparison to a referent group.

What’s an example of a standard score?

The Z score: a z score indicates how many units of standard deviations the individual’s score is above or below the mean of the referent group

When is a Z score negative?

When person’s raw score is below the referent group’s mean

When is a Z score positive?

When the target individual’s raw score is above the referent group’s mean

Z score formula

(Individuals raw score - referent group mean) / referent group standard deviation

True or False: most assumptions show that the data will be normally distributed

False

True or False: The candidate pool is not normally distributed

True

What is a correlation coefficient?

Ranges from -1 to +1 and reflects the direction (positive or negative) of the relationship between two variables. Also reflects the magnitude (strength) of the relationship between two variables.

True or false: A correlation coefficient of zero indicates that two variables are perfectly correlated

False

True or false: If the relationship between candidates who scored highly on the “openness to experience” dimension of a job interview test and turnover is -.60, candidates who scored highly are less likely to leave the organization.

True

What is a scatterplot?

Graphical illustration of the relationship between two variables. Each point on the chart corresponds to how a person scored on a measure and how he or she performed on the job

What are examples of Correlation Coefficient uses?

Relating store size with staffing levels, relating seniority in a firm with job performance, relating the time to fill a job with new hire quality, relating quality of new hires with business performance and customer satisfaction

Sampling error

the variability in sample correlations due to chance. You can address this through statistical significance testing procedures.

Statistical significance

The relationship is not likely due to sampling error. A minimum requirement for establishing a meaningful relationship.

Practical Significance

The relationship is valuable in a practical sense e.g., in large samples almost anything will “predict”, e.g., An inexpensive assessment system may be useful even if the correlation is small.

Multiple Regression

Predicts an outcome using one or more predictor variables; identifies the ideal weights of predictors to maximize the validity of those predictors. Based on predictor’s correlation with the outcome and the degree to which the predictors are themselves intercorrelated. Examines the effect of each predictor variable after statistically controlling for the effects of other predictors in the equation

Reliability

Refers to how dependably or consistently a measure assesses a particular characteristic. Things can influence reliability

What factors can influence reliability?

Temporary physical or psychological state, Environmental factors, Versions of measure, Forms of measure, Different evaluators

Random Error

Error that is not due to any consistent cause

Systematic Error

Error that occurs because of consistent and predictable factors

True or False: The proper interpretation of reliability coefficients depends on the type of reliability being assessed and the purpose of the measure.

True

Test-retest reliability

reflects the repeatability of scores over time and the stability of the underlying construct being measured

Alternate or parallel form reliability

indicates how consistent scores are likely to be if a person completes two or more forms of the same measure

Internal Consistency Reliability

Indicates the extent to which items on a given measure assess the same construct

Inter-rater reliability

Indicates how consistent scores are likely to be if the responses are scored by two or more raters using the same item, scale, or instrument

Joan asked Sara and Alex to score candidates’ interviews using a scoring guide. She calculated a reliability coefficient to determine the correlation between Sara and Alex’s evaluations. What form of reliability is this?

Inter-rater reliability

Joan creates two forms of a job-knowledge test to discourage any cheating during test administration. To test how reliable they are, she asks Duane to take both forms of the test and she correlates his scores. What form of reliability is this?

Parallel form of reliability

Validity

How useful a measure is for a particular situation: (a) assesses a given construct and (b) the degree to which you can make predictions.

Reliability

how consistent scores from that measure will be.

Do you need both validity and reliability?

Yes: You might be able to measure a person’s shoe size reliably but it may not be useful as a predictor of job performance.

Face validity

Subjective assessment of how well items seem to be related to the requirements of the job. Job applicants tend to react negatively to assessment methods if they perceive them to be unrelated to the job or not face valid.

True or False: Even if a measure seems face valid, if it does not predict job performance, it should not be used

True

Validity coefficient

Number of magnitude of the relationship between predictor and criterion and rarely exceed .40 in staffing contexts.

Applicants

A valid assessment system can result in adverse impact for applicants

Organization’s time and cost

A valid assessment system can have an unacceptably long time to fill or cost per hire, result in the identification of high-quality candidates who demand high salaries, resulting in increasing payroll costs; and be cumbersome, difficult, or complex to use.

Future Recruits

A system can be valid but if the system is too long or onerous then applicants, particularly high-quality applicants, are more likely to drop out of consideration

Current employees

A valid assessment system may favor external applicants or not give all qualified employees an equal chance of applying for an internal position; employees may question its fairness.

Validity Coefficients: above .35

Very beneficial

Validity Coefficients: .21 - .35

Potential to be useful

Validity Coefficients: .11 - .20

Useful in certain circumstances

Validity Coefficients: below .11

Unlikely to be useful

Selection Errors

False Positive or False Negative

False Negative

Reject someone who would have succeeded

False Positive

Hire someone who fails on the job

Variability

The degree of difference or spread among scores in a data set.

Shifting the Normal Curve

This concept explains how improving the quality of hiring or assessments can move the distribution of talent to the right (higher performance). The applicant pool is rarely perfectly “normal.” Effective staffing systems help “shift the curve” by identifying top talent and increasing the proportion of high performers.

Correlation

(r) measures the relationship between two variables — how changes in one relate to changes in the other.

• Ranges from −1.0 to +1.0

• Positive = both increase together

• Negative = one increases while the other decreases

• 0 = no relationship

True or False: If r = .70 between test scores and job performance, this means higher test scores predict higher performance.

True

Correlation Strengths (±.70 or higher)

Strong

Correlation Strengths: ±.30 to ±.69

Moderate

Correlation Strengths: Below ±.30

Weak

Things Affecting Reliability

• Temporary physical or psychological states (e.g., fatigue, stress)

• Environmental conditions (e.g., noise, distractions)

• Different test forms or versions

• Different raters/scorers

Deficiency Error

Fails to measure an important part of the intended attribute. Example: A performance test that ignores teamwork when teamwork is crucial.

Contamination Error

Measures factors unrelated to the attribute you want. Example: A performance score affected by favoritism or poor equipment.

Validity Coefficient

The correlation between a predictor (e.g., test score) and a criterion (e.g., job performance). It shows how well a measure predicts job success.

Content Validation

Demonstrates that the test items represent important job tasks or KSAOs. (ex. A typing test for an admin role)

Criterion-Related Validation

Shows that scores correlate with job performance (uses statistics). (ex. correlating sales test results with actual sales)

Construct Validity

Demonstrates that the test truly measures the theoretical trait it claims to (like intelligence or integrity). (ex. A leadership assessment shown to measure “influence,” not just personality.)

Predictor Variable

A measure used to forecast or predict an individual’s future job performance or success. It is collected before hiring. (ex. A candidate’s interview score, cognitive ability test, or personality assessment.)

Outcome Variable

The result or performance measure you want to predict. It reflects job success and is collected after hiring. (ex. Job performance ratings, sales figures, or turnover data.)