M3: TEST DEVELOPMENT AND ADMINISTRATION

The 5 Stages of the Test Development Process

Test Conceptualization, Test Construction, Test Tryout, Item Analysis, and Test Revision.

Test Conceptualization

The beginnings of any published test can probably be traced to thoughts—self-talk, in behavioral terms. The test developer says to himself or herself something like: “There ought to be a test designed to measure [fill in the blank] in [such and such] way.”

Pilot Work

• It is the preliminary research surrounding the creation of a prototype of the test.

• The test developer typically attempts to determine how best to measure a targeted construct.

Pilot Work

• The process may entail the creation, revision, and deletion of many test items in addition to literature reviews, experimentation, and related activities.

• Once it has been completed, the process of test construction begins.

Scaling

•   It was previously defined a measurement as the assignment of numbers according to rules.

• May be defined as the process of setting rules for assigning numbers in measurement.

Louis Leon Thurstone

• He is credited for being at the forefront of efforts to develop methodologically sound scaling methods.

• He adapted psychophysical scaling methods to the study of psychological variables such as attitudes and values.

MDBS-R

• This is an example of rating scale.

• Defined as a grouping of words, statements, or symbols on which judgments of the strength of a particular trait, attitude, or emotion are indicated by the test taker.

• Rating scales can be used to record judgments of oneself, others, experiences, or objects, and they can take several forms.

Likert-scale

Usually reliable, which may account for their widespread popularity.

The Many Faces of Rating Scales

•   Rating scales can take many forms. “Smiley” faces, such as those illustrated here as Item A, have been used in social-psychological research with young children and adults with limited language skills.

•   The faces are used in lieu of words such as positive, neutral, and negative.

Method of Paired Comparisons

Test takers are presented with pairs of stimuli (two photographs, two objects, two statements), which they are asked to compare. They must select one of the stimuli according to some rule.

Guttman Scale

• Another scaling method that yields ordinal level measures.

• Items on it sequentially from weaker to stronger expressions of the attitude, belief, or feeling being measured.

• A feature of Guttman scales is that all respondents who agree with the stronger statements of the attitude will also agree with milder statements.

Item Format

Variables such as the form, plan, structure, arrangement, and layout of individual test items.

Two kinds of Format

Selected-response Format and Constructed-response Format.

Selected-response Format

Requires test takers to select a response from a set of alternative responses.

Three types of Selected-response Format

Multiple-choice, Matching, and True–false.

Constructed-response Format

Requires test takers to supply or to create the correct answer, not merely to select it.

Cumulative Model

•   It is the most commonly used model in scoring the items, owing, in part, to its simplicity and logic.

• The rule in a cumulatively scored test is that the higher the score on the test, the higher the test taker is on the ability, trait, or other characteristic that the test purports to measure.

Test Tryout

Having created a pool of items from which the final version of the test will be developed, the test developer will try out the test. The test should be tried out on people who are similar in critical respects to the people for whom the test was designed.

Cross Validation

Refers to the revalidation of a test on a sample of testtakers other than those on whom test performance was originally found to be a valid predictor of some criterion.