2.0 Test Conceptualization & Test Construction

Test conceptualization, Test construction, Test tryout, Item Analysis, & Item Revision

Steps of Test Development

Test Conceptualization

Process of conceptualizing the construct, items included, and design of the test being developed

Norm-referenced

Comparisons typically are insufficient and inappropriate when knowledge of mastery is required

Criterion-referenced

At least two groups of the test takers, one group known to have mastered the knowledge or skill being measured and another group known not have mastered such knowledge or skill

Test Construction

Stage in the test development process process that entails writing test items (or rewriting or revising existing test items), as well as formatting items, setting scoring rules, and otherwise designing and building a test

Item Pool

The reservoir or well from which items will or will not be drawn for the final version of the test. Generally advised to have twice the number of items that the final version of the test will contain

Item Banks

Relatively large and easily accessible collection of test questions

Computerized Adaptive Testing (CAT)

Items presented to test takers are based in the part on the test taker’s performance on previous items

Floor Effect

Low end of ability, trait, or other measurable attribute

Ceiling Effect

High end of ability, trait, or other measurable attribute

Item Branching

The ability of the computer to tailor the content and order of presentation of items on the basis of responses to previous items

Item Format

Form, plan, structure, arrangement, and layout of individual test items

Category Format

A format where respondents are asked to rate a construct

Checklist

Format where the subject receives a long list of adjectives and indicate whether each one is characteristic of himself/herself

Guttman Scale

Format where answers are arranged sequentially from weaker to stronger expressions of attitudes, beliefs, or feelings being measured

Selected-Response Format

Format where test takers select a response from a set of alternatives

Multiple choice, matching item, binary choice

Types of selected-response format

Stem, correct answer, & incorrect alternatives such as distractors or foils

Three elements of multiple choice formats

Effective Distractors

Alternative answers chosen by both high- and low-performing groups, which enhances the consistency of a test result

Ineffective Distractors

Alternative answers that may hurt the reliability of the test because they are time-consuming to read and can limit the number of good items

Cute Distractors

Alternative answers that are less likely to be chosen, may affect the reliability of the test takers, who may guess from the remaining options

Constructed-Response Format

Format where test takers supply or create the correct answer

Scaling

Process of setting rules for assigning numbers in measurement

Notion of Absolute Scaling

Procedure that involves obtaining a measure of item difficulty across samples of test takers who vary in ability