the constructs or knowledge domains that the test will assess
the type of population with which the test will be used
the objectives of the items to be developed, within the framework of the test’s purpose
the concrete means through which the behavior samples will be gathered and scored
the last point includes decisions about the method of administration, the format of the test item stimuli and responses, and the scoring procedures to be used
after these issues are decided and a preliminary plan for the test is made, the process of test development usually involves the following steps:
- Generating the item pool by writing or otherwise creating the test items, as well as the administration and scoring procedures to be used
- Submitting the item pool to reviewers for qualitative item analysis, and revising or replacing items as needed
- Trying out the items that have been generated and reviewed on samples that are representative of the population for whom the test is intended
- Evaluating the results of trial administrations of the item pool through quantitative item analysis and additional quantitative analysis
- Adding, deleting, and/or modifying items as needed, on the basis of both qualitative and quantitative item analysis
- Conducting additional trial administrations for the purpose of checking whether item statistics remain stable across different groups -- cross-validation -- until a satisfactory set of items is obtained
- Standardizing or fixing the length of the test and the sequencing of items, as well as the administration and scoring procedures to be used in the final form of the test, on the basis of the foregoing analyses
- Administering the test to an new sample of individuals -- carefully selected to represent the population of test takers for whom the test is intended -- in order to develop normative data or performance criteria, indexes of test score reliability and validity, as well as item-level statistics for the final version of the test
- Publishing the test in its final form, along with an administration and scoring manual, accompanying documentation of standardization data, reliability and validity studies, and the materials needed for test administration and scoring