Sathi Global Assessment – Expert Consultation Notes

Comparative Landscape of Existing Exams

Global Exams Mentioned

  • SAT (College Board) – adaptive design, multi-date, recognised internationally.

  • GMAT – computer-adaptive, multiple attempts, accepted by B-schools.

Indian High-stakes Exams

  • JEE & NEET – single annual sitting; syllabus-locked; heavy coaching culture.

  • CUET, CET variants – multi-batch; exposed cases of batch-wise advantage despite normalisation.

  • CBSE board – now allows “improvement attempt”.

  • State boards – sometimes mis-aligned with coaching demands.

  • LSAT-India – adoption fell because only Jindal University demanded it.

Key Critiques of Current Systems (Mahesh’s Observations)

  • Batch inequity: 2021 NEET & 2024 NEET showed city-wise advantage; paper leakage via coaching-staff overlap.

  • Small item banks (~1,0001{,}000–2,0002{,}000 items) lead to ≈60%\approx 60\% repetition between mock papers and actual test.

  • Single-day fate: annual, non-repeatable tests punish “one bad day”.

  • Coaching bias: expensive preparatory ecosystems favour higher socio-economic classes (SEC). Example metaphor: driver’s daughter scores 7575 vs my coached son 9595 – who’s truly brighter?

  • Language bias: judging fluency in English is unfair; language can be acquired within months once immersed.

  • Outcome mis-alignment: GRE & even IIT-JEE scores not strongly predictive of graduation performance; causes trust erosion.

Desired Design Principles for Sathi

  • Multi-attempt, year-round → mitigates single-day risk.

  • Equitable item design:

    • Should not require external coaching.

    • Should test foundational understanding & reasoning over rote content.

  • Large & dynamic item bank:

    • Substantial concept-level permutations, not just value swaps.

    • Secure author pools with firewalls against coaching industry.

  • Blend of Modalities:

    • Predominantly MCQ for scalability.

    • Include subjective / constructed-response items to tap expression & higher-order thinking.

    • Possibility of spoken / listening components (IELTS analogy) but avoid penalising accent.

  • Language Offering Strategy:

    • UG-level ⇒ allow regional languages so rural / government-school students compete fairly.

    • PG-level ⇒ English-only acceptable once learners have exposure.

  • Psychometric Reporting:

    • Produce multi-dimensional indices (e.g., Logical Reasoning, Data Familiarity, Verbal Analysis, Creative Expression).

    • Enable prescriptive guidance for students, parents, colleges, and employers.

Testing Format & Delivery Options

  • Computer-Based Testing (CBT) strongly preferred; still evaluate:

    • Pure Internet-based vs local-server models.

    • Hybrid: secure browser + proctored centre.

  • Paper-Pen fallback for deep-rural areas if infrastructure unreliable.

  • “Adaptive vs Linear” still open; SAT & GMAT cited as adaptivity benchmarks.

Equity & Socio-economic Considerations

  • Exam must internally normalise for SES advantage – difficult but aspirational.

  • Ideas:

    • Differential item functioning (DIF) analysis to ensure neutrality.

    • Contextualised scoring—compare students to similar opportunity groups.

  • Language inclusion again framed as an equity lever rather than convenience.

University / Employer Adoption Strategy

  • Must secure exclusive or primary adoption by 15–20 marquee universities (Ashoka, Krea, FLAME, Shiv Nadar, etc.) to generate student pull.

  • Offer to replace each college’s own in-house test, reducing their administrative burden.

  • Provide rich analytics to Deans (trait buckets ≈ Finance-fit, Data-fit, Product-fit, Communication-fit).

Credibility & Trust-Building Measures

  • Transparent reliability & validity studies; publish psychometric white papers.

  • Demonstrate longitudinal outcome correlation (student performance through graduation).

  • Robust proctoring, encryption, auditing to avoid leaks; large rotating writer pool.

Risk Factors Identified

  1. Normalization errors → batch advantage complaints.

  2. Item leak / coaching collusion.

  3. Infrastructure outages in CBT mode.

  4. Low institutional uptake = low candidate volume (LSAT-India cautionary tale).

  5. Language expansion complexity – cost & consistency across versions.

Open Questions & Future Research

  • Which global/Indian assessments have not spawned coaching cultures? (Research task).

  • Single composite paper vs two-tier structure? (General + specialised modules).

  • Optimal weight of subjective items vs scalability.

  • How to operationalise SES-adjusted interpretation.

Immediate Action Points (as distilled by Durbin & Balu)

  • Map core construct blueprint before item writing.

  • Engage prospective universities early for requirements & MoUs.

  • Draft risk-mitigation plan (item bank size, secure author contracts, tech redundancy).

  • Circulate meeting transcript & let Mahesh append further notes.

Illustrative Numerical & Statistical Mentions (for reference)

  • NEET 2021 & 2024 anomalies.

  • 60%60\% overlap between coaching mock and real exam (Narayan 2022 example).

  • Board candidate population ≈ 17,00,00017{,}00{,}000.

  • Ideal item bank >> 2,0002{,}000 questions to prevent exposure.

Ethical & Philosophical Undercurrents

  • Education assessment should discover talent, not manufacture rank.

  • Language, wealth, geography must not decide destiny.

  • Over-objective (MCQ-only) systems risk “dumbing down”; need space for creativity & nuance.

Concluding Metaphor Recap

“Driver’s daughter scoring 7575 with no coaching vs my privileged son scoring 9595 after coaching” – Sathi must recognise true potential, not purchased performance.