I&O Psych Exam 2

What is a Performance Measurement?

Evaluating them on job related score

Reps job perf (construct) -> performance measure (variable)

Use scores to eval job perf

What are the Three Types of Performance Measurement?

Objective

Personnel

Judgmental

What is Objective (Hard Criteria) Performance Measurement? Adv/Disadv?

quantitative counts of the results of work (sales volume, number of complaint letters)

ex: Number of sales, tips made etc

Advantages

Exact, not subjective

Disadvantages

Narrow in scope (one number rarely captures full picture of someone's perf)

Can be affected by external factors (similar to effectiveness)

May still entail many judgmental contents and not necessarily more reliable

What is Personnel Performance Measurement?

HR records (absences, disciplinary actions)

What is Judgmental (Soft Criteria) Performance Measurement? Adv/Disadv?

subjective evaluations of work behavior (supervisory ratings, self ratings, peer)

Subjective

Soft Criteria

Advantages:

Can capture more aspects of performance, less influenced by external factors

Disadvantages

Subjectivity of the rater

Correlation with objective perf: about .20 around .39

Special Cases: Electronic Performance Monitoring...

Monitoring work processes with electronic devices

Cost effective, detailed and accurate work logs

Privacy and fairness issues (can lead to bad work attitudes, self esteem, and communication)

Also positively associated with performance at work (under some circumstances)

Highly skilled students/workers were monitored

The student/worker was able to delay or prevent monitoring at particular times

Lower-skilled were not monitored (performed better without EPM)

When will EPMs be welcomed and effective? When people

Job relevance (its actually relevant to their job)

Participation (able to be involved in the process)

Control (more control (delay/prevent)

Advance warnings (when and how monitoring will occur)

What is Performance Management?

Does not equal performance appraisal/evaluation (is broader than performance appraisal)

System that emphasizes the link between individual behavior and organizational strategies, goals, and feedback

1. By defining performance in the context of these goals (Goal customer service = clients solving problems)

What are the 3 Components of Performance Management?

Define performance (task job analysis and strategic analysis)

Measurement of performance

Communication between supervisor and subordinate (feedback)

Goals of Performance Management?

1. Evaluate proficiency of employees

Make ratings

Evaluate efficiency

2. Provide feedback

Set goals

Develop to improve performance

Purpose of Performance Management?

Administration purposes

Promotions, rewards, transfer, layoffs, fires

Development and feedback

See where and how employees can improve

Criterion Data

How well does something predict performance

In this case you need a measure of performance

What are Overall Performance Ratings?

High-level and simple; Usually for administrative purposes

No "real" or conceptual meaning

Affected by (Rotundo & Sackett, 2002)

Task performance (+)

OCB (+)

CWB (-)

Campbell (1999) suggests looking inside the "boxcar" of overall job performance

What are the Specific Performance Ratings?

Trait ratings

Task Based ratings

Critical Incidents

What are Trait Ratings (Specific Peformance)?

(don't use these)

Persistence, concentration, personality

Based on traits and not actions or behaviors

Unfair

What are Task Based Ratings (Specific Performance rating)

(defensible in court and accepted by incumbents)

Extensions of job analysis

Effectiveness on groups of similar and critical tasks (often called duties)

What are Critical Incidents (Specific Performance Rating)?

Effective vs ineffective versions of important aspects of performance

What are the 3 Fundamental Characteristics of Ratings?

Behavioral Definitions

Response Categories Defined

Response Unambiguous

What are Behavioral Definitions?

Is it clear what behavior you are measuring

Customer service quality rate from 1 to 5

Vs

Customers service quality (prompt response, clear communication, timely follow up) rate from 1 to 5

What is Response Categories Defined?

what does a certain response option (satisfactory or 4) mean?

Customer service quality, rate from 1 (receives neg customer feedback weekly) to 5 (receives pos customer feedback weekly)

These benchmarks defining the scale points are called anchors (explain what beh are expected for each response category)

What is Response Unambiguous?

can we interpret as the rater intended

If the rater intended to rate 4 on the scale,can we identify that based on their rating

5 4 3 2 1 VS. 5—------1

[] [] [] [] []

First one has response unambiguous

What is a Graphic Rating Scale?

scores running from high on one end and low on the other end

Most widely used judgmental measure

First types of scales used for performance evaluation

Can be very effective if well designed

What are Checklists?

Lists of behaviors presented to a rater, who needs to select items best (least) describe the ratee

May be weighted by SMEs

Variation: forced choice checklist

Rater need to check only one behavior that describes the ratee the best

Not particular conductive to providing feedback

Evaluation not day to day perf management

Assessment > Management

Avoiding bias, halo effect (bias)

Examples of Checklists

The instructor created a classroom environment that encouraged questions and discussion (1.2).

The instructor presented material clearly (4.2).

Lectures were adequately organized (3.7).

The instructor was enthusiastic and friendly (2.7).

The instructor used examples from his/her experience or research

^ example of a weighted checklist (includes items that have value or weights assigned to them, derived from expert judgments of incumbents)

Example 2 Forced Choice

Choose one of the following options that best describes the employee:

Takes up challenges eagerly

Always comes up with a new idea

Leads and shows proper direction to each of its team member

Understand the problem and provides valuable alternative solutions to it

What are Behaviorally Anchored Rating Scales (BARS)?

Scales included behavioral anchors describing work behaviors

Choose the beh that matched the number the best

Rater choose what a target employee has done or might do

High inter rater reliability

Great way to make shit more objective and behavioral focused

exceeds/expected behavior

Pros and Cons of BARS?

Pros

Clearly defined anchors

Measures behaviors, not stereotypes

High involvement with employees and supervisors (extensives SME interaction)

Cons

Time consuming and costly to develop

Job and task dependent

Not all critical incidents or tasks may be listed or observed

Beh didnt beh, would imagine beh, then provide ratings based on imagined beh

What are Behavioral Observation Scales (BOS)

Scales asking raters to consider the frequency of an employee acting in a particular way

Gets around BARS issue of using anchors that may not be observed

Easier to develop because more closely tied to job analysis

Beh dont show up = no imagination, will just be a zero

What are Computer Adaptive Rating Scales?

Uses forced choice format

Uses computer adaptive testing (CAT) format

Raters response to an item

Next item is chosen so that it matches the current estimate of performance

Until certain criteria is reached

Increases efficiency and accuracy of ratings

What are Employee Comparison Methods?

Performance rating = compare employee behavior with given standards (anchors)

Compare between employees..

Rank order

Paired comparison

forced distribution

What is Rank Order?

Rank all employees on some dimensions from low to high (worst to best)

For mult dimensions can also use average ranks of each dimensions PRO

Hard with large number of employees (100 vs 10 employees) CON

What is Paired Comparison?

employees compared against each other

Individuals score = number of times chosen over others

Total number to compare: N(N-1)/2

Forcing raters to make direct judgment of employees

Clear decision of who performs better

Good for layoff decisions, but time consuming

But what about 1000..

What is Forced Distribution?

force raters to distribute employees in some distributions

Normal distribution, Top 20% = A performers

Sometimes arbitrary, forces arbitrary distribution

Can prevent rating inflation sometimes BUT arbitrary and may not be fair

Forces to put employees to be put in a distribution (fairness issue)

Not legally defensible either

Types of Rating Sources?

Supervisor

Peers

Subordinates’

Clients/customers

Yourself

All rating sources for judgmental

What are Supervisor Ratings?

Typically see results of your performance

Good for measuring maximum performance

What are Peer Ratings?

May know more about typical performance and more likely to observe OCB

But better for developmental purpose rather than administrative (giving feedback, coaching needs, team growth rather than promotion of firing)

Conflicts of interest

Friendships

What are Subordinate Ratings?

Best for evaluating leadership behaviors and their effect on subordinates

Needs to be kept anonymous to prevent supervisor retaliation

Better for developmental purposes than administrative

What are Client/Customer Ratings?

Good choice when supervisors have limited observation

Can be biased by:

External factors (angry customer, products quality = might provide low rating despite not actually reflecting employee)

Feeling towards the company

Limited to customer oriented behaviors

What are Self Ratings?

Potential for leniency bias (perform better than others, provide higher ratings for self)

Perceive self to perform better than others]

When discussing with supervisors, can be minimized

Better for developmental purpose rather than administrative purpose

Why used Self Ratings?

Important role in performance management

Better understands performance

Reflect on behs in the past

Helps employees themselves

CWB: self ratings report more CWB than supervisor/peer ratings

OCB: difference between self and supervisor/peer ratings is small

What is 360 Degree Feedback?

Collective ratings and feedback from many sources

Best fro developmental purposes

Are Humans Good at Ratings? Intentional/Systematic Distortions of this

Not always, there can be errors, or intentional/systematic distortions

Central tendency: raters pick middle point to be safe (avoiding extreme response options)

Avoid extra justification for extreme choices

Leniency/Severity Bias: Raters are unusually easy/harsh in the ratings

Due to being demanding, avoid friction

Can be reduced by providing well-defined behavioral anchors

Halo: raters give all high ratings, based on one positive aspect of the employee

Examples of Rating Training (for minimizing distortions)?

Administrative Training

Psychometric Training

Frame of Reference

What is Administrative Training?

Discussion for meaning of behaviors and anchors (particularly for BARS and BOS)

But always more direct to design better scales

What is Psychometric Training?

Train raters aware of distortions

Does really workL at the cost of harming accuracy

What is Frame od Reference (FOR) Training?

Give rater a context for ratings

Facets of performance

Meaning of anchors

Make practice ratings

Provide immediate feedback on practice

Effective in reducing rating errors

Reliability in Performance Ratings...

Should inter-rater reliability be high

Different sources = differences perspective

In this case, don't expect high IRR. Otherwise info is redundant (not really capturing diff persp of ones performance)

Typically .5 - .6 (overlap moderately, should be)

Same source = expect high IRR (same aspect of performance)

Validity in Performance Ratings?

How can rating scales be valid

Based on job analysis

Represents more aspects of performance

Rater training

Embrace three fundamental characteristics of rating

What Makes Feedback Effective?

Backed by data (numbers, critical incidents)

Specific and informative: what's wrong and how to it can be fixed

Not simply a documentation of poor performance

Focus on behavior not outcomes

Avoid praise criticism praise sandwich

Start with praise, then give criticism, then praise again

Then praise signals bad news coming

Confusing to do this

Negative feedback must always be constructive

Destructive feedback: Cruel, sarcastic, and offensive; Usually general than specific; Directed towards personal characteristics than job-relevant behaviors

Legal Contexts..

Courts care more about fairness than theory or validity

Care about fairness

For example, forced distribution has lota of legal issues

Forced distributions (10-80-10) rating system for seniors managers

Many lawsuits based on discrimination

How can it be legally sound

Substance

Behavior based not trait based

Process

System

Procedure of perf measurement

What are Staffing Decisions? Purpose?

associated with recruiting, selecting, promoting, and separating (layoff or termination) employees

Purpose: hire and keep people who will be successful

Why do we Care about Staffing Decisions?

Staffing practices do have a positive association with firm performance when they follow high performance work practices

Use of job analysis (help guide staffing decisions, carefully define task required for job)

Select from existing employees for key positions (internal selection)

Adjustment timeline typically shorter (current employees alr know the culture, less time for them to adjust)

Motivates current employees as well

Merit-based promotions

Amount of training

Use of formal assessment devices for selection decisions

Decreases turnover, increases sales and overall profits

What is Recruitment in Staffing Process?

Build right applicant pool

1st step in staffing

To attract people with right qualifications to apply (not yet selecting)

Based on job analysis

Internal vs external (within or outside members of organization)

What is Internal Recruitment? Pros? Cons?

recruiting from within organization

(Pros):

know company culture, adjustment time is shorter, company already knows how they work (past performance), can see opportunity for promotion = higher motivation for the future

Cons:

Got a smaller pool, homogeneous applicants (hard to bring new ideas/policies), comp between employees (politics)

What is external recruitment?

recruiting outside members of the organization

Pros:

new ideas/perspectives, larger applicant pool (more diverse)

Cons:

more expensive (higher costs), riskier (more uncertainty)

Common methods: job posting, referrals, school recruiters

What is Selection of the Staffing Process? Types?

Needs to answer these questions

What KSAOs (job analysis)

How to measure KSAOs

How to score measures and combine info

How to make a decision

How to evaluate outcomes

Compare with recruitment

Types:

Compensatory

Non-Compensatory

What is the Compensatory Selection System?

candidates can make up for weaknesses in one area by strength si another area

Ex: candidate for assistant prof does not have much exp in instruction, but has many publications

What is the Non-Compensatory Selection System?

candidates wont be hired if they lack certain KSAO

Ex: legally blind individual cannot become a bus driver

Nurses, truck drivers, specific requirements (lawyers/attorneys)

Promotion and Termination/Deselection...

Both based on performance measurement

Two types of deselection

Layoffs: lose job due to downsizing

Termination for cause: lose job due to underperformance

Decisions to make at these stages:

How to make decisions

How to evaluate decisions

How do you combine information from different methods to make decisions

Methods in an assessment center

Clinical vs statistical decisions

Clinical (intuitive): combine info based on human judgment (unreliable based)

Statistical (actuarial): use a math formula to combine info (reliable, fair)

Compensatory Methods for Combining Info?

Total Mean Score (ranking/cut score)

Multiple Regression (Weighted Score)

Score Banding

What is Total Mean Score? Ranking? Cut Score?

Based on cut score; ranking

total/mean score approach (Take average of all their score)

Ranking

Rank total score or mean and rank them in order and take however many we want going down the line highest to lowest

Cut score

Higher score than 23 can be selected

What is Multiple Regression (Weighred Score)?

Determine diff task have diff weight

Weighted score = 0.5g + 0.2C + 0.3 Intervie

What is Score Banding?

Close scores as ties/band

Same band = essentially equal

A = total > 23

B = total > 21

Reason:

Sometimes not care about actual score differences

Tests not perfectly reliable

What is the Hurdle System (NON-Compensatory)?

Cut score for each assessment

Applicants must pass one stage to move onto the next

Effective in narrowing down large applicant pool

More costly overall (mult assessments)

usually mult hurdle systems in non compensatory

What is Subgroup Norming?

develop separate lists of individuals in different demographic groups, then rank within groups

White employees vs black employees; women vs men then ranking them within the groups

Can actually lead to more bias despite intention to reduce subgroup differences

For example, may introduce reverse discrimination (not using same standard to all applicants) people who actually score higher overall can be passed over just bc they are in a certain group

Illegal under the Civil Rights Act of 1991

What Does Good Decision Mean?

Identify good performers - validity

Cost efficient - utility

Is it cost effective

Save or earn more on the whole process

Unbiased across various groups - fairness

Is our decision fair

Applying the same process/rules to everyone?

What is Validity in Good Decisions?

Translate the question does the selection system work

Actually end up being a good performer..?

Criterion related validity: correlation between selection measure scores and job performance scores

r= 0, useless

0 < r < 1, typical

R = 1, perfect (impossible)

What is Selection Ratio (SR)

Proportion of applicants we hire

Proportion of applicants that are selected into positions

10 positions, 100 applicants -> SR = 10%

validity

What is Base rate (BR)

How common something is in the overall population

Proportion of applicants that would be successful performers

If you hire all 100 applicants, 40 will have good performance -> BR = 40%

Don't know BR at the time of hire

Estimate BR from historical performance data

Can use last year BR to estimate

What is a Cut Score?

Cutoff for hiring employees

Higher cut score = low selection ratio

E.g., 0-100 scale, 100 applicants:

Cut score 70 → 40 people pass → SR= 40%

Cut score 85 → 15 people pass → SR = 15%

BR and SR....

N = 30

1. Identify performance standard

Its 4 = considered good performers, below = bad performers

Now have a base rate

BR = # above the line/N

2. Set a Cut Score

People who score higher than cut score = hired (right = hired, left = reject)

SR = # right to the line/N

Example of Incorrect (False) Decisions

What are Positive Decisions?

You decide to hire this candidate

If indeed a good performer then its a correct (true) decision -> true positive

instead a bad performer, then it's an incorrect (False) decision → False Positive

What are Negative Decisions?

You decide not to hire this candidate

If indeed a bad performer then its a correct (true Decision) -> true negative

If instead a good performer, then it's an incorrect (False) decision → False Negative

goal = less false positive and false negatives

Decrease the Performance Standard..

More successful performers in the pool, BR increases

Less false positives, more false negatives

More true positives

Less true negatives

Increase the Cutoff and Decrease the Selection Ratio...

Choosing the Best SR...

Lower SR is better

Higher cut score, lower selection ration

Reduce cost of false positives

SR = # hired/ # applicants

Hire less = decrease SR

Get more applicants = decrease SR

What is Utility? ROI?

assessment of return on investment (ROI)

In the case of staffing

ROI

Benefit of hiring good performers/cost of staffing strategy

Utility Analysis Considers?

Costs of hiring and FALSE POSITIVES (hiring bad performers)

Base rate (proportion of good performers among applicant pool)

High BR = utility of good assessment would be low

Low BR = need a really good assessment to identity good performers (utility high)

Decrease false positives

What is Fairness? If unfair...

Is our decision fair

Applying the same process/rules to everyone?

If staffing are unfair, employees may

Engage in CWB

File formal complaint

Sue the company

What is Adverse Treatment?

employer knowingly and willfully treats minority applicants differently (than majority) intentional

Ex: diff cut score for diff groups, diff groups diff questions

Avoid hiring older workers

What is Adverse Impact?

employer incidentally treats minority applicants differently unintentional

The company requires to complete a 5 mile team building run during the hiring process

Adverse Impact Ratio

What is the 4/5th rule?

potential adverse impact when: minority group received less than 80% (4/5th) of desirable outcomes received by majority group

Not necessarily adverse treatment or adverse impact. Why

Determining Adverse Impact?

4/5th rule is only a guideline

Final judgment made in court

In response to evidence of AI, employers must demonstrate that this practice is job related (therefore valid)

I/O psychologists as expert witnesses

Discrimination cases typically violate

Title VII rights act of 1964

Americans with disabilities Act

Age discrimination in employment act

Very expensive for organization and hurts public opinion

What Is Training?

Training = systematic acquisition of

Concepts (K)

Skills (S)

Attitudes (O)

Not ability (A)- because ability is more resistance to change and music less malleable

Foundation for Training Progrmas..

learning

Learning outcomes

Cognitive (e.g., declarative knowledge (facts, procedural)

Skill based (motor and technical)

Affective (attitudes and beliefs)

Training does NOT...

Does not guarantee learning

Planned or informal experience intended to lead to learning

How much is learned is influenced by many factors

What is Learning?

Relatively permanent change in behavior and capabilities

Cannot be observed:

Assumed to be taken place when observing performance

Infer learning from performance

What is Performance?

Actions or behaviors relevant to organization goals

Can be directly observed

Learning often leads to better performance in training and job tasks

Why do we care about training?

Big cost: 102.8 billion spent on training in 2025

Assessment of return of investment (ROI) is good if training is useful

Attract and refrain good employees

Combined with selection to build strong workforce

Select on certain KSOs

Enforce and Enhance these KSOs in training

Some Conceptual Issues of Training?

Individual differences assumes KSAOs are relatively stable, but training and development assumes K and S can be changed and enhanced

Rank order stability (Individual Differences KSAOs)

The extent to which individuals maintain their relative positions (rankings) on a trait compared to others over time

Mean level stability (change in training and development)

The extent to which the average score of a group remains the same over time

The average motivation score on increases from 80 to 90 after training

If K and S can be improved, then what's the role of personnel selection

Schmidt and Hunter 1992

Training Needs Analysis...

necessary step before actually designing a training system

Training needs analysis

Three-step process to identify where, what, and who of training

Where is training needed? (ORG)

What needs to be trained? (What)

Who will be trained? (Who)

What is Organizational Analysis (Where)?

Where is training needed

Identifies the training needs of different departments/units (or company-wide goal and problems)

Determine through examined organizational goals, available resources and the organizational environment

Helps to build appropriate support for training

What is Task Analysis (What)?

What needs to be trained

Identifies: what does the employee need to do in order to perform properly

Examines the actual content of training can be consisting of

Developing task statements

Identifying KSAOS required for the job

Based on Job analysis

Can also assess competencies

What is Person Analysis (Who)?

Who needs training and what instructions

Identifies which individuals in the organization should be trained and what instruction they need to improve

Can be identified by performance appraisal critical incidents and 360 degree feedback

What is Trainee Readiness?

whether employees have the personal characteristics necessary to (1) acquire KSOs from training and (2) apply those to the job

"g"- predictive performance, r=.56, according to schmit and hunter (1998) can be used for grouping employees in training)

Goal orientation: performance vs Mastery oriented

What is Performance and Mastery (Goal Orientation of Trainee Readiness)?

Performance: cares more about doing well in training and to be evaluated positively

Mastery: concerned with increasing competence for task at hand

More flexible and adaptable in learning situations (care about process not end results)

Experience (Trainee Readiness)?

More experienced: shorter, less structured training programs

Less experience: longer, more structured training programs

What is Trainee Motivation?

interest in attending training, learning from training, and KSO transferring

Trainee motivation and learning goal orientation contribute most to positive outcomes

What affects trainee motivation

Expectancy: Belief of the association between effort and performance

Learning Style: preferred method of learning (auditory, visual, kinesthetic)

What is Transfer of Training?

degree to which trainees apply KSOs gained in training their jobs

Affected by TOT climate: workplace characteristics that inhibit or facilitate TOT

Examples: emphasis on training importance, the supervisor and peer support

Climate also affects reactions to training and motivation to learn

Activities before/after training (supervisor involvement, supervisor support) strongly related to TOT than activities during training (rewards, feedback)