Phase 4: Test Revision

Choosing the Final Items

• requires weighing each items content validity, item difficulty, and discrimination, item correlation and bias also test length and face validity

• When new items need to be added or items need to be revised the items must again be pilot tested to ensure that the changes produced desired results

• constructing a matrix makes choosing the best items easier

  • best items have higher positive correlation coefficents

Validation

• process of obtaining evidence that the test effectively measures what it is supposed to measure

• first part of establishing content validity is carried out as the test is developed

  • measures the constructs (construct validity)

  • predicts an outside criterion determined in subsequent data collection

• Validation study needs to be done in multiple places so theres evidence that your test can be generalized

• The sole purpose of the validation study is to affirm the test’s ability to yield meaningful results.

Cross Validation

• When final revision of a test yields scores with sufficent evidence of reliability and validity

  • test developers then conduct cross-validation - final round of test administration to another sample (Target)

• cross-validation breaks the original sample used in the original validation study into two parts.

  • One part is called the screening sample and consists of the majority of the test takers often about 70%.

  • The remaining 30% of the test takers are called the validation sample

  • A regression equation is first created using only the screening sample, where original test scores predict the original criterion scores.

  • This new equation is then used to estimate criterion scores for the people in the validation sample.

Plan Implementation

• Reevaluate the test content and structure

• Be willing to make changes dictated by the data or the feedback recieved during tryouts

Prepare the Manual

• Include an adequate description of the test development process so others can replicate what was accomplished and for user to evaluate the usefulness of the test for their purposes

• The specific contents will vary according to the type of test and its applications

  • some measures may have special legal requirements

Submit a Test Proposal

• If you want to have your test published, you will need to submit a test proposal to an appropriate publisher.

• The major test publishers have slightly different guidelines for developing and submitting a test proposal or prospectus.

• This information is usually available at their websites.

• Self-publishing?

• Revisions