Assessment of Individual Differences Exam #3

The body of knowledge that the test represents

What is a Testing universe

helps focus the test/makes sure all questions relevant/ cover what you want to measure

Why define a testing universe prior to making a test

specifies the characteristics of the test

Test Plan

operational definition of construct

content to be measured ( testing universe)

format for questions

3 things part of test plan

Objective 1 correct answer

Objective vs. Subjective test format

Stimulus to which test takers respond

mechanism for response

2 thing to consider when choosing test format

Sampling ( provide opportunity to sample universe)

Scoring (simple and can be done by computer)

Two things easier for objective testing process

Patterns of responding that result in false/ misleading info

Response style/set

Social Desirability: choose answers that are socially acceptable or present them in favorable light

Acquiescence: select agree for everything

2 different response sets

Administrator (how to set up/run)

Test taker (knowledge needed to complete it)

Scorers (accurately scoring)

3 types of instructions that should be written for new test

helps test developers identify problems/assess how test items perform/ ensure test is valid/reliable before

Why develop a pilot test

item difficulty

Clarity of instructions

Timing

What gained from pilot test

To examine test item performance/ identify what to keep/ modify/ delete

Why conduct test analyses ?

Distractor Analysis

revising item difficulty

2 things that can improve test analysis

Frequency with which an incorrect answer is chosen

if good, incorrect answers will be randomly spread across all alternatives

What info is involved in conducting a distractor analysis

if incorrect answer partially true or obviously incorrect

Writing issue that makes an item poor in distractor analysis

policy/ placement/ counseling/ guidance

3 types of decisions educators make using tests

administrators/ teachers/ testing professionals

Who in educational process makes these decisions

w/o training educators may misinterpret test scores/leads to inaccurate conclusions about student abilities

Why Educators should be trained in psychological testing

improper grade assignment/ placement

2 thing that happen if educators not trained in testing

Unfairly advantage some groups

only measure memorization

2 myths about cognitive ability test

Yes, it suggest that some bias exists but still remains valid predictors of performance as they do measure reasoning, problem solving, and critical thinking

Does data refute the myths?

SES does affect SAT scores but still strong predictor of GPA

SES explain why SAT score predict college GPA?

Norm: score compared with scores of other test takers

Criterion: score compared with an objectively stated standard of achievement

Norm-referenced vs. criterion-referenced tests

Norm identify student standing with others

criterion determines whether student meets learning objectives

Norm-referenced vs. criterion-referenced tests in educational decisions

Assesses student's ability to perform in real-world

Authentic Assessment in educational decision-making

impractical for school systems that need to perform large-scale testing

Why Authentic Assessment rarely used in most educational decision?

Credentials/ training required

Required equipment

audience to which test is administered

requirements of location

What elements should administrator instructions include

simple/ concise/ low reading level

first appear in writing

encouragement/context for answering

What elements should test taker instructions include

written scoring guide

How to interpret score

What elements should test scorer instructions include

The looked at Affirmative Action

What did Fisher v. Texas (for) and SFFA vs Harvard (against) case examine about the selection system?

Structured/semi- structured interviews Clinical interview

Comprehensive Self- report tests

Performance-based/ projective personality tests

Cognitive/ memory testing

Psychological test used in clinical settings

4 categories of psychological test and their use

Clinical interview; ASD parent interview for autism spectrum disorder

Self-report; MMPI objective evaluation personality traits

Projective; Inkblot test reveals unconscious aspects of person's personality projected onto inkblots

Cognitive; WAIS-5 assessment of adult's cognitive abilities

Projective - stimuli elicits responses that reflect unconscious thoughts/ emotions

Objective - structured/ standardized questions w/ clear scoring systems

Projective vs. Objective techniques in clinical assessments

Objective

Projective vs. Objective test used for other than psychological measurement

Ensures clinical methods/ tools/ interventions evidence-based/ effective

Scientific research and clinical professionalism

Clinical professionalism in terms of test use/ administration

Properly trained in administering/ interpreting test evidence must be up-to-date

Must be ethical in use

Reliability and Validity

what form of evidence is necessary for supporting the usage of a psychological test to diagnose/ screen for clinical diagnosis

To examine the reliability/ dependability of perspective employee

Why use credit scores to hire?

Contains more information then just reliability/ dependability

(i.e not being able to build it due to circumstance)

Is it fair to use credit scores in hiring

2 myths about pre-employment testing that prevent businesses from using test

Too expensive

Too difficult/ time-consuming

Structured/standardized interview

Key to interview being a valid predictor

the interviewers are trained on it which increases reliability and validity

Why it increases validity of interview

increase predictive accuracy by standardizing questions/ evaluation criteria

Describe 1 method for accomplishing validity of interview

High - Realistically replicate job setting

Low- simulate task visually/verbally/ with written description

High fidelity vs. low fidelity

High-work sample (simulate environment have them solve a case to choose which is better)

Low- Situational Judgement Test ( tell me what you would do?)

High fidelity vs. low fidelity examples

knowledge, skills, abilities, and other characteristics

KSAO

Psychotherapist

KSAO required for job you want

O - Ability to listen emphatically and empathetically

How to test for one of the KSAOs

Halo effect

Leniency/ severity errors

Recency error

Central tendency errors

Errors in Performance Appraisal

Halo effect- giving better scores to people who are liked more

Describe 1 rating error

Test internal validity ( testing what its supposed to)

Item discrimination ( can discriminate between high and low performers)

item-test correlation

Fairness (judgment call): ethical side of testing

Bias: statistical feature of test

Fairness vs. Bias