Phase 1: Test Conceptualization

What are the 5 main reasons we develop a new test?

1. meet the needs of a special group of test takers

• New group needs to be assessed that we want to learn more about and understand

• Such as people who are in newly developed fields

  • E.g., the youth, children's, adults, intelligence tests

  • These weren't around before

 

2. To sample behaviours from a newly defined test domain

• There are some clinical disorders that have not existed before

  • E.g., adult ADHD, internet or gambling addiction

 

3. To improve the accuracy of test scores for their intended purpose – low quality

• Improve reliability and validity

• Complete tests of similar constructs, assess reliability

 

4. tests need to be revised/modified (e.g., old items, old norms)

• Remake tests, how do they need to be changed? Has vocabulary changed over time?

• Items would need to be changed for the groups now taking it. Have the norms changed?

  • e.g., you received criticism on a test you made, now it must be revised

 

5. tests may assess clinically useful constructs, but may be impractical for real-world clinical applications.

• New tests can me remade to have some practical use

  • E.g., MSCEIT. They struggled to make a test that is difficult to use in what setting, despite being a really good instrument.

What are the Four Phases of developing a test? MUST KNOW THE ORDER

(1) Test Conceptualization

(2) Test Structure and Format

(3) Standardization

(4) Plan Implementation (Revisions)

Two questions must be answer (Hint: Practicality and Knowledge base of the construct)

Will the test improve practice/research & will it improve our knowledge of human behaviour

What are the Four Steps in Phase 1?

1. Defining the test universe.

2. Defining the target audience.

3. Defining the purpose.

4. Defining the construct.

What happens in Phase 1, STEP 1: Defining the testing universe? (I.e., what is the testing universe?)

• The testing universe is the body of knowledge or behaviors that the test represents

• prepare a working definition of the construct (more conceptual in nature)

  • the developer prepares a working definition of the construct that the test will measure

    • E.g., What is ___ in general?

• the developer conducts A thorough review of psychological literature to locate studies that explain the construct and any current tests that measure it

• We need to define constructs operationally in terms of behaviors that are associated with the construct

  • The actual definition of something? This is the operational definition

What happens in Phase 1, STEP 2: Defining the target audience? (Hint: what is the target audience?)

• The target audience is the group of individuals who will take the test

• tests should be designed with specific users in mind.

  • make a list of characteristics of persons who will take the test--particularly those characteristics that will affect how test takers will respond to the test questions

    • These characteristics affect how test takers will respond to these questions

      • E.g., you cannot simply develop a test for children; you must indicate the characteristics of the children, such as age and reading level. A test for 6th graders is significantly different from those of first graders

    • (e.g., age range, reading level, disabilities, honesty, language)

  • who is able to access the assessment/who can use it and why – Level A, B, or C (may be different from state/province or test publisher

 

What happens in Phase 1, STEP 3: Defining the purpose of the test? (Hint: what is the purpose of the test?)

• The purpose of the test is the information that the test will provide to the test user

• includes not only what the test will measure, but also how scores will be used

  • e.g., will scores be used to compare test takers (normative approach) or to indicate achievement (criterion approach)?

• Remember: what the test measures and how the scores will be used

• We may simply test just to test.

  • We want to know how much of a construct someone has

Information about the testing universe, target audience, and test purpose provides the basis for making other decisions about the test

Phase 1, STEP 3: Defining the purpose of the test – What is a normative approach?

Normative: comparative - MCAT,LSAT, GRE…

• Normative approach e.g.,: An employment test in which the applicant who achieves the highest score will receive the job offer

Phase 1, STEP 3: Defining the purpose of the test – What is a criterion approach?

Criterion: must meet minimum to pass/benchmark

• E.g., The way we interpret achievement test results… individuals must achieve a certain score to qualify as passing or excellent (this is an absolute decision)

 

What happens in Phase 1, STEP 4: Defining the construct? (Hint: what is construct explication?)

• After reviewing the literature about the construct and any available tests, the test developer is ready to write a concise definition of the construct

  • This includes operationalizing the construct in terms of observable and measurable behaviors

  • Specify approximate number of items needed, how much time needed

• Provides boundaries for the test domain (what should and shouldn’t be included) – construct explication

  • The process of providing a detailed description of the relationship between specific behaviors and abstract constructs.

  • Identify:

    • behaviours that relate to the construct

    • Other related constructs

      • Behaviours related to other construct

Phase 1, STEP 4: Defining the construct – What is Awareness?

• Having insight to understand the link between mind and body in response to change

Phase 1, STEP 4: Defining the construct – What is Regulation? (Hint: literally just emotion regulation)

• The process by which one alters and controls emotion and behaviour in order to succeed as a specific task

Phase 1, STEP 4: Defining the construct – What is Resilience? ( (Hint: literally just the definition)

• The ability to effectively cope, adjust, and recover from adversity in order to enhance performance