IOA & Non-Sampling Errors – Quick Reference

Interactive Oral Assessment (IOA) – Key points

• IOA definition and purpose

  • Two-way free-flowing unscripted conversation based on a real-world scenario; not a formal oral exam or presentation.
  • Authentic, personalised learning: prompts adapt to what you say; helps showcase understanding while preserving academic integrity.
  • Aims: develop oral communication, promote engagement, and foster higher-order thinking (synthesis and analysis).

• Why we use IOA

  • Aligns with graduate attributes (communication in multiple formats).
  • Encourages authentic dialogue and demonstrates learning in real-time.
  • Past student feedback (last year): generally positive; IOA seen as enjoyable and engaging.

• Logistics and setup

  • Appointment length: $10$ minutes total; conversation portion about $5-6$ minutes; remaining time for marking.
  • Mode: in person or Zoom. Last year: about $frac{2}{3}$ Zoom and $frac{1}{3}$ in person; plan blocks for each mode this year.
  • Scheduling: via Canvas Scheduler; materials can be used as a reference (notes allowed).
  • Notes: bring a one-paper reference (preferably one-sided, A4); not a full script.
  • Week six window for collection of notes and marking; tutorials will provide practice sessions.

• Preparation and materials

  • Reread the Doctor’s Appearance case study; used as a model for IOA conversations.
  • Materials converted to Canvas page, Google Doc, and Word Doc for easy access.
  • Tutorials two instructions to be released this afternoon; announcements will follow.
  • Practice session: Friday tutorial to try IOA in peers.

• Marking and rubric

  • Rubric is broad; practice with example videos to understand expectations.
  • Assessors: Leela and Steph (two-way evaluation).
  • If uncertain between marks, lean toward the higher mark.
  • Focus on conversational nature; responding to prompts rather than a rehearsed monologue.

• Preparation time and expectations

  • Time varies by individual; still requires reading materials and forming a judgement prior to conversation.
  • Compared to a long written assignment, IOA presents the judgment in a 5-minute conversation.
  • You will still complete guideline and worry-question analysis to inform your judgement.

• Materials and practice opportunities

  • Survey materials available: the Doctor’s Appearance case study, survey report, and a NZ Herald article for assignment one.
  • Friday: two example videos will be reviewed and marked against the rubric to illustrate expectations.

• Accessibility and wellbeing

  • If anxiety or other accommodations are needed, inform the course staff to arrange support.

• Past feedback and integrity considerations

  • 2023: some GenAI use in a written assignment; IOA helps ensure critical analysis and synthesis are performed by the student.
  • It will be possible to use AI tools for preparation, but you must demonstrate your own engagement and analysis in the conversation.

• Practical tips for success

  • Bring a concise reference (one A4 page) to keep thoughts organized.
  • Engage with prompts; demonstrate your understanding through discussion and interpretation of the material.
  • Attend Friday practice to familiarize yourself with the flow and rubric.

• Quick takeaway

  • IOA is a structured, authentic conversational assessment designed to showcase your understanding in real-time while maintaining integrity and supporting your communication skills.

Non-sampling errors in polling – Overview

• Core idea

  • Non-sampling errors are biases not due to the act of sampling; they cannot be reliably estimated or corrected after data collection.
  • They include how the sample is collected, who responds, wording, format, and other procedural factors.

• Sampling error vs non-sampling error

  • Sampling error decreases with larger sample sizes; standard error roughly scales as $SE \, \propto \, \frac{1}{\sqrt{n}}$.
  • Non-sampling errors require design, weighting, and methodological controls to mitigate.

• Nonresponse bias

  • When the response rate is low (e.g., about $35\%$), the 65% nonrespondents may differ systematically.
  • Causes: phone screening, caller recognition, time of day, online panels with voluntary participation.

• Question (item) effects

  • Wording can dramatically change responses; e.g.,
  • "increased benefits for poor people" vs "increasing welfare benefits" yields different support levels.
  • Easiest vs allowed methods (e.g., obtaining vs purchasing beer) can alter reported behaviors.
  • Best test of a poll question is the respondent’s intuitive reaction to it.

• Survey format effects

  • Format (online, phone, in-person) affects answers; e.g., online panels may yield higher nonresponse for dont know responses after 2020 changes.
  • Layout, color, print, length, and order of questions influence responses.
  • Example: a German study comparing face-to-face, telephone, online quota, and online snowball showed substantial format-driven differences in reported behavior.

• Interviewer effects

  • Differences in interviewer characteristics (e.g., gender, ethnicity) can influence responses even with fixed questions.
  • Monitoring and standardized procedures aim to minimize these effects.

• Self-selection bias

  • Online polls advertised to self-selecting participants attract those with stronger opinions; middle-ground respondents are underrepresented.
  • Common in online newspapers polls and opinion pieces.

• Behavioral considerations (social desirability bias)

  • People overreport socially desirable behaviors (e.g., handwashing) and underreport undesirable ones; effects vary by mode (interview vs computer).
  • Across studies, modes (computer vs interviewer) influence reported behaviors and honesty levels.

• Age reporting bias

  • Respondents tend to round ages to convenient numbers (e.g., 21, 29, 39, etc.), creating digit bias in age data.

• Transferring findings to other populations

  • Generalizing from one population to another can be problematic (e.g., NHANES vs hospital ER samples).
  • Historical underrepresentation of women in trials led to dosing and safety issues; modern trials aim to address this bias.

• Poll format and ordering effects

  • Question order can bias results (e.g., early questions framing later responses).
  • Consistency in wording and order is crucial for comparability over time (e.g., polls since 1997 in NZ).

• Weighting and representativeness

  • After data collection, samples are often weighted to census benchmarks (age, gender, ethnicity) to improve representativeness.
  • Pre-2021, phone sampling relied on urban/rural quotas; mobile sampling uses probability sampling to reflect service mix.

• Practical takeaway for polling practice

  • Be aware of format, wording, and order effects; ensure representative sampling and appropriate weighting; report limitations clearly.

• Summary formulae and concepts

  • Larger sample size reduces sampling error: $SE \propto \frac{1}{\sqrt{n}}$
  • Non-sampling errors require design and post-survey adjustments; cannot be estimated from data alone.