ch 7 kaplan
Introduction to Test Administration
- Overview of the last six chapters focusing on test construction and the importance of test administration.
- Mention of various factors that can influence test scores, emphasizing the difference between observed score and true ability/trait.
Reliability and Measurement Error
- Review of Chapter 4's concept of reliability which includes:
- Measurement error defined as the difference between the true score and the observed score.
- Primary focus of reliability theory is on random sources of error.
- Importance of considering other potential sources of error in test applications:
- Testing situation,
- Tester characteristics,
- Test-taker characteristics.
Rethinking Test Score Beliefs
- Evidence that biases arising from test administration must be reconsidered:
- Initial beliefs postulated that test scores were unaffected by testing environments.
- New studies indicate significant influence of testing circumstances on results.
- Stereotyping can cause discomfort that negatively impacts test performance.
The Role of the Examiner
Relationship Between Examiner and Test Taker
- Relationship dynamics can directly affect test scores based on examiner behavior.
- Study Example:
- Children given the Wechsler Intelligence Scale for Children (WISC) under two conditions:
- Enhanced rapport (friendly conversation) vs. neutral rapport (no interaction).
- Children aged 5th-9th grade scored significantly higher (Mean IQ=122) under enhanced rapport than neutral (Mean IQ=109).
- Additional study revealed disapproving comments from examiners led to lower scores compared to neutral or approving comments (Witmer et al., 1971).
- Familiarity Factor:
- Familiar proctors improved reading test scores in children (DeRosa & Patalano, 1991).
- Systematic procedures developed to mitigate bias effects from familiar vs. unfamiliar administrators.
Racial Bias and Test Administration
Race of the Tester
- Research indicates some communities feel children should be tested only by examiners of their own race, particularly African Americans.
- Sattler's reviews indicate little scientific evidence that the race of the examiner affects IQ scores.
- The belief that white examiners negatively impact African American scores is labeled a myth.
- Previous assumptions asserted race should not affect locked, standardized testing procedures.
- Insight into existing biases shows examiner discretion may skew results:
- Example: White examiners received higher scores from white children than African American children (Abramson, 1969).
- Expanded findings indicated African American children performed better when items were presented with context and themes (Fagundes et al., 1998).
The Nature of Stereotype Threat
- Definition: Evaluation anxiety can be amplified for individuals from negatively stereotyped groups.
- Additional pressure to disprove negative stereotypes exists, particularly relevant in gender contexts (e.g., women in mathematics).
- Performance differentials noted on SAT Math attributed to stereotype threat.
- Performance of groups affected:
- 50% to 80% of gender SAT Math differences,
- 25% to 41% of differences between white non-Hispanic and Hispanic students,
- 17% to 29% of differences between white non-Hispanic and African American students attributed to stereotype threat (Walton et al., 2013).
Addressing Stereotype Threat
- Interventions aimed at reshaping beliefs about intelligence have shown to assist students underperforming due to stereotype threat.
- Providing evidence of intelligence as a malleable trait improved academic performance (Aronson et al., 2002).
- Growth mindset interventions demonstrated applications to other groups facing stereotype threats (Froehlich et al., 2016).
Language and Testing
- Linguistic proficiency significantly affects outcomes for non-English speakers.
- Validity and reliability of translations must be validated against original tests.
Training of Test Administrators
- Various training requirements for different assessments (Oakland & Wechsler, 2016).
- Importance of conducting adequate practice sessions to minimize scoring errors (Patterson et al., 1995).
Expectancy Effects in Assessment
- Expectancies of the examiner can subconsciously shape test scores (Rosenthal effects).
- Studies by Robert Rosenthal showcased effect where subjects confirmed the experimenter's expectations (Rosenthal, 1966).
- Found in both humankind and animal behavior, reinforcing that perception influences judgment (Sattler et al., 1970).
The Impact of Reinforcing Responses
- Reinforcement has significant effects on behavior during testing, necessitating strict control under standardized conditions.
- Positive reinforcement results in improved performance, particularly for targeted subgroups of children.
Testing Standardization
- Importance of standardization in procedures for valid outcomes (AERA et al., 2014).
- Recommendations for alteration to administration conditions based on demographic questions.
Computer-Assisted Test Administration
- Overview of advantages of computerized testing:
- Improved standardization, individualization, and bias reduction.
- Objective scoring and reporting mechanisms.
Mode of Test Administration
- Analysis showing little impact of mode of administration (self-administered, interviewer-led) on results in specific contexts (Vernon et al., 2008).
Subject Variables and Test Scores
- Factors such as health, test anxiety, and overall motivation influence performance (Spielberger et al., 2015).
Summary of Test Administration
- Emphasis on the necessity of standardized administration for accurate reflection of abilities and traits.
- Recognition of how situational factors and subjective experiences can influence testing outcomes significantly.