ch 7 kaplan

Introduction to Test Administration

  • Overview of the last six chapters focusing on test construction and the importance of test administration.
  • Mention of various factors that can influence test scores, emphasizing the difference between observed score and true ability/trait.

Reliability and Measurement Error

  • Review of Chapter 4's concept of reliability which includes:
    • Measurement error defined as the difference between the true score and the observed score.
    • Primary focus of reliability theory is on random sources of error.
    • Importance of considering other potential sources of error in test applications:
    • Testing situation,
    • Tester characteristics,
    • Test-taker characteristics.

Rethinking Test Score Beliefs

  • Evidence that biases arising from test administration must be reconsidered:
    • Initial beliefs postulated that test scores were unaffected by testing environments.
    • New studies indicate significant influence of testing circumstances on results.
    • Stereotyping can cause discomfort that negatively impacts test performance.

The Role of the Examiner

Relationship Between Examiner and Test Taker

  • Relationship dynamics can directly affect test scores based on examiner behavior.
    • Study Example:
    • Children given the Wechsler Intelligence Scale for Children (WISC) under two conditions:
      • Enhanced rapport (friendly conversation) vs. neutral rapport (no interaction).
      • Children aged 5th-9th grade scored significantly higher (Mean IQ=122) under enhanced rapport than neutral (Mean IQ=109).
    • Additional study revealed disapproving comments from examiners led to lower scores compared to neutral or approving comments (Witmer et al., 1971).
    • Familiarity Factor:
    • Familiar proctors improved reading test scores in children (DeRosa & Patalano, 1991).
    • Systematic procedures developed to mitigate bias effects from familiar vs. unfamiliar administrators.

Racial Bias and Test Administration

Race of the Tester

  • Research indicates some communities feel children should be tested only by examiners of their own race, particularly African Americans.
    • Sattler's reviews indicate little scientific evidence that the race of the examiner affects IQ scores.
    • The belief that white examiners negatively impact African American scores is labeled a myth.
    • Previous assumptions asserted race should not affect locked, standardized testing procedures.
    • Insight into existing biases shows examiner discretion may skew results:
    • Example: White examiners received higher scores from white children than African American children (Abramson, 1969).
    • Expanded findings indicated African American children performed better when items were presented with context and themes (Fagundes et al., 1998).

Stereotype Threat and Test Performance

The Nature of Stereotype Threat

  • Definition: Evaluation anxiety can be amplified for individuals from negatively stereotyped groups.
    • Additional pressure to disprove negative stereotypes exists, particularly relevant in gender contexts (e.g., women in mathematics).
    • Performance differentials noted on SAT Math attributed to stereotype threat.
    • Performance of groups affected:
    • 50% to 80% of gender SAT Math differences,
    • 25% to 41% of differences between white non-Hispanic and Hispanic students,
    • 17% to 29% of differences between white non-Hispanic and African American students attributed to stereotype threat (Walton et al., 2013).

Addressing Stereotype Threat

  • Interventions aimed at reshaping beliefs about intelligence have shown to assist students underperforming due to stereotype threat.
  • Providing evidence of intelligence as a malleable trait improved academic performance (Aronson et al., 2002).
  • Growth mindset interventions demonstrated applications to other groups facing stereotype threats (Froehlich et al., 2016).

Language and Testing

  • Linguistic proficiency significantly affects outcomes for non-English speakers.
    • Validity and reliability of translations must be validated against original tests.

Training of Test Administrators

  • Various training requirements for different assessments (Oakland & Wechsler, 2016).
  • Importance of conducting adequate practice sessions to minimize scoring errors (Patterson et al., 1995).

Expectancy Effects in Assessment

  • Expectancies of the examiner can subconsciously shape test scores (Rosenthal effects).
    • Studies by Robert Rosenthal showcased effect where subjects confirmed the experimenter's expectations (Rosenthal, 1966).
    • Found in both humankind and animal behavior, reinforcing that perception influences judgment (Sattler et al., 1970).

The Impact of Reinforcing Responses

  • Reinforcement has significant effects on behavior during testing, necessitating strict control under standardized conditions.
    • Positive reinforcement results in improved performance, particularly for targeted subgroups of children.

Testing Standardization

  • Importance of standardization in procedures for valid outcomes (AERA et al., 2014).
    • Recommendations for alteration to administration conditions based on demographic questions.

Computer-Assisted Test Administration

  • Overview of advantages of computerized testing:
    • Improved standardization, individualization, and bias reduction.
    • Objective scoring and reporting mechanisms.

Mode of Test Administration

  • Analysis showing little impact of mode of administration (self-administered, interviewer-led) on results in specific contexts (Vernon et al., 2008).

Subject Variables and Test Scores

  • Factors such as health, test anxiety, and overall motivation influence performance (Spielberger et al., 2015).

Summary of Test Administration

  • Emphasis on the necessity of standardized administration for accurate reflection of abilities and traits.
  • Recognition of how situational factors and subjective experiences can influence testing outcomes significantly.