LEC09: Test Bias and Test Utility

Test Bias - possible reasons behind it

• norm - references psychological tests are designed to measure differences between people

• problems when different groups score differently on psychological tests

Test Bias - what can be done

• Deal with construct-irrelevant issues

• Modify problematic test content

• Use appropriate norms/ref groups

• Use or develop a more appropriate test

• reconsider how the test is used

Conceptualising test bias

• one way of modeling test bias is to look at regression lines for criterion validity scatterplots separately for different groups

• for eg. look at the relationship between how well people will do a job (prediction) and measure how well they actually do and look at the relationship on a scatterplot

• then we get in trouble that the test is against group B (groups can;t be compared.

• The slope of regression lines of the groups are the same but they intercept the vertical axis at different places intercept bias.

• Test is biased against Group B and is also les predictive of job performance for them, which makes the test differentially valid for the test groups.

• The slope of regression lines of the groups are different, which is a slope bias.

The Pygmalion Effect

• Basically a stereotyping effect where teachers expectation of students boost their performance(self-fulfilling prophecy).

Self-stereotypes

• US-based experiment, where Group 1 were told the teat measured intellectual ability while Group 2 were told the test was about problem-solving not intellectual abilities.

• African-Americans did worse than White Americans when in groups 1 but not in Group 2 (where they did not differ)

• Self-stereotyping affecting test scores

The law and psychological testing

• In Australia, psychological tests based on discrimination in employment, education etc. is banned such as disabilities.

• Only aptitude and personality tests related directly to genuinely job requirements can be developed.

• ‘‘ Inconsistencies in court decisions are commonplace “

• ‘‘ The validity and reliability of many psychological tests are currently being decided by the courts “

Test Utility

• It is the practical usefulness of a test (in terms of finance, savings in time etc.) and Does it lead to better decisions about something ?

• a good reliability + validity doers not equal to good utility

• A test has to yield worthwhile benefit to outweigh its costs and has to better than its alt options.

Utility Analysis

• Used to answer questions like what test to use and is adding a new test to an existing battery worthwhile

• Expectancy table: False positive (test incorrectly identifies a person as being good when its not), False negative (test incorrectly identifies person as being no good when they’re good), Selection ratio (the ratio between available job positions and number of applicants), Cut off (the minimum test score needed in test to be hired)

• Limitations: assumes linear relationship between job performance and test score. doesn’t take into other factors into account such as minority status, physical health etc.

Utility Analysis: More sophisticated options

• Analysis of financial cost/benefit

•