Assumptions about Psychological Testing

Psychological Traits and States Exist

Trait

• any distinguishable, relatively enduring way in which one individual varies from another

• thousands of traits terms can be found in the English language

• psychological traits exists as constructs -- an informed, scientific concept developed or constructed to describe or explain behavior

  • we can’t see, hear, or touch constructs, but we can infer their existence from overt behavior, such as test scores

• relatively stable

  • may change over time, yet there are often high correlations between trait scores at different time points

• nature of situation influences how traits will be manifested

• refer to ways in which one individual varies, or differs, from another

State

• distinguish one person from another but are relatively less enduring

Traits and States can be Quantified and Measured

• different test developers may define and measure constructs in different ways

• once a construct is defined, test developers turn to item content and item weighting

• a scoring system and a way to interpret results need to be devised

Test-Related Behavior Predicts Non-Test-Related Behavior

• Responses on tests are though to predict real-world behavior. The obtained sample of behavior is expected to predict future behavior.

Tests have Strengths and Weaknesses

• Competent test users understand and appreciate the limitations of the tests they use as well as how those limitations might be compensated for by data from other sources.

Various Sources of Error are Part of Assessment

• Error: refers to a long-standing assumption that factors other than what a test attempts to measure will influence performance on the test

• Error Variance

  • the component of a test score attributable to sources other than the trait or ability measured

  • both the assessee and assessor are sources of error variance

Testing and Assessment can be Conducted in a Fair Manner

• all major test publishers strive to develop instruments that are fair when used in strict accordance with guidelines in the test manual

• problems arise if the test is used with people for whom it was not intended

Testing and Assessment Benefit Society

• there is a great need for tests, especially good tests, considering the many areas of our lives that they benefit

“A Good Test”

• Reliability: the consistency of the measuring tool: the precision with which the test measures and the extent to which error is present in measurements

• Validity: the test measures what it purports what it measures

• Other Considerations: Administration, scoring, interpretation should be straightforward for trained examiners. A good test is a useful test that will ultimately benefit individual test takers or society at large

Norms

• Norm-Referenced Testing and Assessment

  • a method of evaluation and a way of deriving meaning from test scores by evaluating an individual test taker’s score and comparing it to scores of a group of test takers

• the meaning of an individual test score is understood relative to other scores on the same test

• Norms: the test performance data of a particular group of test takers that are designed for use as a reference when evaluating or interpreting individual test scores

• a normative sample is the reference group to which test takers are compared

Sampling to Develop Norms

• Standardization: the process of administering test to a representative sample of test takers for the purpose of establishing norms

• Sampling: test developers select a population, for which the test is intended, that has at least one common, observable characteristic

• Stratified Sampling: sampling that includes different subgroups, or strata, from the population

• Stratified Random Sampling: every member of the population has an equal opportunity of being included in a sample

• Purposive Sampling: arbitrarily selecting a sample that is believed to be representative of the population

• Incidental or Convenience Sample: a sample that is convenient or available for use

• generalization of findings from convenience samples must be made with caution

Developing Norms

• having obtained a sample test developers

  • administer the test with standard test of instructions

  • recommend a setting for test administration

  • collect and analyze data

  • summarize data using descriptive statistics including measures of central tendency and variability

  • provide a detailed description of the standardization sample itself

Types of Norms

• Percentile

  • the percentage of people whose score on a test or measure falls below a particular raw score

  • a popular method for organizing test-related data because they are easily calculated

• Age Norms: average performance of different samples of test takers who were at various ages when the test was administered

• Grade Norms: the average test performance of test takers in a given school grade

• National Norms: derived from a normative sample that was nationally representative of the population at the time the norming study was conducted

• Local Norms: provide normative information with respect to the local population’s performance on some test

Norm-Referenced vs Criterion-Referenced

• Normative-Referenced Tests: involve comparing individuals to the normative group

• Criterion-Referenced Tests: evaluated as to whether they meet a set standard

Culture and Inference

• In selecting a test for use, responsible test users should research the test’s available norms to check how appropriate they are to use with the targeted test taker population

• When interpreting test results it helps to know about the culture and era of the test taker

• It is important to conduct culturally informed assessment