230pm-tech shenanigans

Education Protocols and High Stakes Testing

Overview

  • High stakes testing and education protocols are complex and wide-ranging topics.

  • Multiple vendors are involved in administering various types of exams.

  • Speaker denotes a departure from typical presentation style to provide a structured overview.

Exam Administration Prior to COVID

  • Different methods of delivering exams exist, which cater to varied needs.

  • Types of High-Stakes Exams:

    • Insurance licensing exams administered primarily at company-owned or third-party brick-and-mortar test centers.

  • Mid-Stakes Exams:

    • IT certification exams administered at brick-and-mortar test centers and proctored at remote locations (e.g., education expos).

    • Combination of on-site proctored exams and remote proctored exams.

  • Statistics Pre-COVID:

    • Approximately 200,000 to 250,000 IT exams were conducted remotely.

Exam Administration During COVID

  • By 2021, remote proctored exams increased to over 2,000,000 due to the need for alternative testing options when in-person testing centers were unavailable.

Spectrum of Exam Administration Options

  • Exam delivery options range from highly controlled access to wider access with lower control.

  • A Venn diagram used to illustrate the interplay between different methods and their associated characteristics.

Components of Exam Security

  1. Administrative:

    • Refers to the processes related to exam registration, payment, and delivery.

    • Discusses the technical and administrative measures taken but does not delve deeply into specific security features.

  2. Forensic & Data:

    • Involves data collected by testing vendors to identify suspicious behavior in applicants or during testing.

    • Information is shared between vendors and regulators to investigate potential fraud.

  3. Enforcement:

    • Primarily the responsibility of regulators, which can include administrative investigations or law enforcement if necessary.

  • All components are interrelated and should continuously inform one another.

Rising Incidents of Cheating

  • Notable increase in misconduct observed at testing centers, necessitating involvement of law enforcement.

  • Increasing behaviors of cheating have been reported since the onset of COVID.

Forms of Misconduct Observed

  • Proxy Testing:

    • When third parties attempt to take exams on behalf of candidates.

    • May lead to licensing prohibitions, bans, or investigations if found to be part of organized fraud.

  • Evidence of systemic fraud observed in investigations; collaboration with law enforcement aids in prosecutions related to widespread fraudulent activities.

Cheating Patterns

  • Graph of Cheating Attempts:

    • Vertical axis: Degree of Organization (low to high).

    • Horizontal axis: Sophistication of cheating attempts (low to high).

  • Low Organization: Individual cheating behaviors, e.g., using crib notes, stashing materials.

  • Higher Organization: Assisted cheating attempts through organized teams.

  • Advanced Techniques:

    • Use of micro-cameras and audio aids in remote testing scenarios, complicating detection of cheating.

    • Proxy rings attempting to exploit remotely proctored exams often consist of non-US residents with fraudulent documentation.

Implications of Systemic Fraud

  • Attempting to create fraudulent licenses has financial implications, benefiting scammers substantially.

  • Examples of multi-state proxy testing rooms observed, with rampant questionable activities in both domestic and international testing.

  • Regulatory action based on evidence collected through technological means, focusing on suspicious IP usage and testing patterns.

Forensic Investigations

  • Discussion of the limitations and challenges in forensic investigations, highlighting the constrained resources available for detection and prosecution of fraudulent behavior.

  • Presenting actionable findings to regulators who can then undertake administrative or criminal investigations.

Observations from Remote Testing Infrastructure

  • Introduction of second camera for enhanced surveillance, requiring candidates to use their phone in addition to their computer's camera during exams.

  • Setup and Monitoring Process:

    • Candidates guided on camera setup allowing comprehensive monitoring of the testing environment to prevent cheating.

  • Concerns raised regarding potential invasions of privacy, necessitating clear communication and consent from candidates beforehand.

Conclusion and Future Considerations

  • There is a need for robust inter-vendor communications and regulations to address the challenges posed by proxy testing and other vulnerabilities in the examination process.

  • Shifts in testing modalities highlight the ongoing evolution and need for effective solutions in maintaining exam integrity and security, especially in a post-COVID landscape.

  • Various states exhibit differing needs for testing centers depending on geographic considerations, underscoring the necessity for individualized approaches in testing policies.