ELTAD - (Bailey) - Learning about language asssessment

ELTAD reading list.

Definition of Assessment

• Systematic process of collecting information about learners’ language ability.

• Broader than “testing” — includes informal and formal methods.

• Used to inform teaching, learning, and decision-making.

Assessment vs Testing vs Evaluation

• Assessment: ongoing information gathering.

• Testing: specific, structured measurement tool.

• Evaluation: interpreting results to make judgments or decisions.

Purposes of Assessment

• Placement, diagnosis, monitoring progress, measuring achievement, certifying proficiency.

• One assessment can serve multiple purposes.

• Purpose influences design and interpretation.

Formative vs Summative

• Formative: “assessment for learning” — supports ongoing improvement.

• Summative: “assessment of learning” — measures outcomes at a point in time.

• Both can be formal or informal.

Dilemmas of Conflicting Purposes

• A single test may be used for incompatible goals.

• Conflicts between teacher, learner, and institutional needs.

• Risk of reduced validity or fairness.

Stakeholder Perspectives

• Teachers: want diagnostic, instructional feedback.

• Learners: want fair, motivating assessments.

• Institutions: need accountability and standardization.

Principles of Good Assessment

• Validity: measures what it claims to measure.

• Reliability: consistent results across time and raters.

• Practicality: feasible in time, cost, and resources.

Impact and Washback

• Assessments influence teaching and learning (“washback”).

• Positive washback: encourages desired learning behaviors.

• Negative washback: narrows curriculum or promotes test-driven teaching.

Purpose of the Framework

• Helps teachers systematically evaluate language tests.

• Encourages looking beyond surface features to underlying qualities.

• Supports informed decisions about test use or adaptation.

Key Test Qualities

• Validity: Does the test measure what it claims to measure?

• Reliability: Are results consistent across time, tasks, and raters?

• Practicality: Is it feasible in terms of time, cost, and resources?

Additional Considerations

• Authenticity: Does the test reflect real-world language use?

• Impact/Washback: How does it influence teaching and learning?

• Fairness: Is it equitable for all test-takers?

“Entering a new Culture” Analogy

• An unfamiliar test is like entering a new culture — requires observation and analysis.

• Avoid quick judgments; learn the “rules” before deciding on its value.

• Consider the test’s context, purpose, and stakeholders.

Using the Framework

• Apply criteria systematically to any assessment tool.

• Identify strengths, weaknesses, and areas for improvement.

• Use findings to adapt, replace, or justify the test.

Purpose of the Chapter

• Introduces key contrasting pairs in language assessment.

• Shows how each pair represents a design choice with trade‑offs.

• Helps teachers recognise tensions when selecting or interpreting tests.

Norm‑Referenced vs. Criterion‑Referenced

• Norm‑Referenced: compares test‑takers to each other.

• Criterion‑Referenced: measures performance against fixed standards.

• Choice affects score meaning and use.

Direct vs. Indirect Testing

• Direct: assesses the skill itself (e.g., speaking test for speaking ability).

• Indirect: measures related abilities or subskills (e.g., grammar test for writing).

• Direct often more authentic; indirect may be easier to score.

Discrete‑Point vs. Integrative Testing

• Discrete‑Point: tests one element at a time (e.g., single grammar item).

• Integrative: combines multiple skills/knowledge in one task (e.g., cloze test).

• Integrative can reflect real‑world use better.

Objective vs. Subjective Scoring

• Objective: clear right/wrong answers; minimal scorer judgment.

• Subjective: requires human judgment (e.g., essay rating).

• Reliability depends on scoring method and rater training.

Formative vs. Summative (Revisited)

• Formative: ongoing feedback to improve learning.

• Summative: final judgment at course or program end.

• Both can be formal or informal.

High‑Stakes vs. Low‑Stakes

• High‑Stakes: major consequences for learners (e.g., graduation, visas).

• Low‑Stakes: minimal consequences; often for practice or feedback.

• Stakes influence preparation, anxiety, and washback.

Trade-Off Awareness

• No single “best” choice — depends on purpose, context, and constraints.

• Understanding contrasts helps avoid mismatches between goals and tools.

• Teachers should balance validity, reliability, practicality, and impact.

Receptive Skills Defined

• Listening and reading are receptive skills — learners receive and interpret language.

• Success depends on both language ability and background knowledge.

• Background knowledge = prior experiences, cultural context, topic familiarity.

Role of Background Knowledge

• Influences comprehension and test performance.

• Can help fill gaps when language input is unclear.

• Lack of relevant knowledge can disadvantage otherwise capable learners.

Assessing Listening

• Involves processing spoken input in real time.

• Tasks may include note‑taking, answering questions, summarising.

• Must consider accent, speed, and clarity of recordings.

Assessing Reading

• Involves decoding written text and constructing meaning.

• Tasks may include multiple‑choice, short answers, matching, ordering.

• Text choice should match learners’ proficiency and avoid cultural bias.

Test-Design Considerations

• Choose topics accessible to all test‑takers to reduce bias.

• Provide necessary context or pre‑teaching for unfamiliar topics.

• Balance authenticity with fairness.

Validity and Fariness

• Ensure tests measure language ability, not just topic knowledge.

• Use a variety of topics to avoid advantaging specific groups.

• Consider piloting tasks to detect unintended bias.

Strategies for Teachers

• Activate learners’ background knowledge before assessment.

• Teach strategies for dealing with unfamiliar content.

• Reflect on how cultural assumptions shape test materials.

Cloze-Tests Defined

• Text with regular word deletions (e.g., every 5th–7th word) replaced by blanks.

• Test‑takers fill in missing words from context.

• Measures integrative language ability, especially reading comprehension.

Rationale for Cloze

• Requires use of grammar, vocabulary, and discourse knowledge together.

• Encourages processing of overall meaning, not just isolated items.

• Can reveal both linguistic and contextual competence.

Variations of Cloze

• Fixed‑ratio deletion: every nth word removed.

• Rational deletion: specific words removed based on test purpose.

• C‑tests: second half of every second word deleted.

Scoring Methods

• Exact‑word scoring: only original word accepted.

• Acceptable‑word scoring: synonyms/contextually correct words allowed.

• Choice affects reliability and fairness.

Advantages

• Easy to construct from authentic texts.

• Tests multiple skills in one task.

• Can be adapted for different proficiency levels.

Limitations

• Performance influenced by topic familiarity.

• Scoring can be subjective if using acceptable‑word method.

• May disadvantage learners unused to the format.

Design Type

• Choose texts appropriate for learners’ level and background.

• Give clear instructions and examples.

• Pilot test to check difficulty and scoring consistency.

Communicative Language Teaching Defined

• Focuses on assessing ability to use language for real‑life communication.

• Goes beyond grammar/vocabulary to include meaning, context, and purpose.

• Rooted in communicative language teaching principles.

Key Features

• Tasks simulate authentic language use (e.g., role‑plays, problem‑solving).

• Integrates multiple skills (listening, speaking, reading, writing).

• Emphasises appropriateness, fluency, and effectiveness of communication.

Rationale

• Language ability = knowledge + ability to use it in context.

• Tests should reflect how language is actually used outside the classroom.

• Encourages positive washback on teaching and learning.

Task Types

• Information‑gap activities, interviews, group discussions.

• Writing tasks with real‑world purposes (letters, reports, emails).

• Listening/reading tasks linked to follow‑up speaking or writing.

Scoring Considerations

• Use analytic or holistic rating scales for performance tasks.

• Criteria may include accuracy, fluency, range, coherence, appropriacy.

• Rater training essential for reliability.

Advantages

• High authenticity and face validity.

• Encourages integrated skill development.

• Can motivate learners by showing practical relevance.

Challenges

• More complex and time‑consuming to design and score.

• Reliability can be harder to ensure than with discrete‑point tests.

• Requires careful planning to match tasks with test purposes.

Teacher Tips

• Align tasks with learners’ real‑world needs.

• Pilot tasks to check clarity, difficulty, and scoring consistency.

• Balance authenticity with practicality.

Productive Skills Defined

• Speaking and writing involve producing language rather than receiving it.

• Require control of grammar, vocabulary, and discourse.

• Also demand consideration of audience, purpose, and context.

Assessing Speaking

• Can be tested via interviews, role‑plays, discussions, presentations.

• Criteria often include fluency, accuracy, pronunciation, and appropriacy.

• Rater training essential for consistent scoring.

Assessing Writing

• Tasks may include essays, letters, reports, summaries.

• Criteria often include content, organisation, language use, mechanics.

• Rubrics can be holistic (overall impression) or analytic (separate scores for components).

Task Design Considerations

• Match tasks to learners’ proficiency and real‑world needs.

• Provide clear instructions and time limits.

• Avoid cultural bias in topics or prompts.

Reliability Challenges

• Subjective scoring can reduce reliability.

• Use multiple raters or double‑marking to improve consistency.

• Detailed rubrics help standardise judgments.

Validity Considerations

• Tasks should reflect authentic language use.

• Ensure alignment between test content and intended skills.

• Avoid overemphasis on minor errors that don’t affect communication.

Feedback and Washback

• Detailed feedback supports skill development.

• Positive washback: encourages practice in meaningful contexts.

• Negative washback: leads to “teaching to the test” without real skill growth.

Performance-Based Assessment Defined

• Assesses learners through real‑world tasks that require active language use.

• Focuses on process and product, not just discrete items.

• Often integrated into regular classroom activities.

Portfolios

• Collection of learner work over time showing progress and achievement.

• Can include drafts, final pieces, self‑reflections, teacher feedback.

• Encourages learner autonomy and self‑assessment.

Projects

• Extended tasks with a tangible outcome (e.g., report, presentation).

• Promote integration of skills and application of knowledge.

• Allow for creativity and learner choice.

Conferences

• One‑to‑one or small‑group discussions between teacher and learner(s).

• Used to review progress, set goals, and give feedback.

• Provide opportunities for personalised assessment.

Advantages

• High authenticity and positive washback.

• Encourages deeper learning and skill integration.

• Builds learner responsibility and engagement.

Challenges

• Time‑consuming to plan, conduct, and assess.

• Scoring can be subjective; requires clear rubrics.

• May be affected by unequal access to resources.

Design Tips

• Align tasks with learning objectives and real‑life relevance.

• Use transparent criteria and share them with learners.

• Combine with other assessment types for a balanced approach.

Self-Assessment Defined

• Learners evaluate their own language ability or performance.

• Encourages reflection, goal‑setting, and learner autonomy.

• Can be formal (checklists, rating scales) or informal (journals, reflections).

Peer Assessment Defined

• Learners assess each other’s work or performance.

• Promotes collaborative learning and critical thinking.

• Can use rubrics, checklists, or guided feedback forms.

Benefits of Self‑ and Peer Assessment

• Develops awareness of language strengths and weaknesses.

• Encourages responsibility for learning.

• Provides additional feedback beyond the teacher’s.

Challenges

• Risk of inaccuracy due to bias or lack of assessment skill.

• Some learners may feel uncomfortable judging peers.

• Requires training to ensure fairness and usefulness.

Teacher’s Role

• Provide clear criteria and examples of good performance.

• Train learners in constructive feedback techniques.

• Monitor and guide the process to maintain reliability.

Self‑Assessment Defined

• Learners judge their own language ability or performance.

• Builds reflection, self‑awareness, and autonomy.

• Can be structured (checklists, rating scales) or informal (journals, notes).

Peer Assessment Defined

• Learners evaluate each other’s work or performance.

• Encourages collaboration and critical thinking.

• Uses rubrics, checklists, or guided feedback prompts.

Benefits

• Promotes learner responsibility and engagement.

• Provides more feedback sources than teacher alone.

• Develops evaluative and analytical skills.

Challenges

• Risk of bias, inaccuracy, or over‑leniency/harshness.

• Some learners may feel uncomfortable judging peers.

• Requires training to ensure fairness and reliability.

Teacher’s Role

• Provide clear criteria and performance examples.

• Train learners in constructive feedback techniques.

• Monitor and guide to maintain quality and consistency.

Practical Tips

• Start with low‑stakes tasks to build confidence.

• Combine with teacher assessment for balance.

• Use results to inform teaching and learner goal‑setting.

Definition of LSP Assessment

• LSP = Language for Specific Purposes (e.g., business, aviation, medicine).

• Focuses on language skills needed in a particular professional or academic domain.

• Tailored to real‑world tasks and contexts of that field.

Needs Analysis

• Identifies the specific language demands of the target context.

• Involves input from stakeholders (employers, trainers, learners).

• Guides test design to ensure relevance and authenticity.

Authenticity

• Tasks mirror real‑life communication in the target domain.

• Uses authentic materials (e.g., workplace documents, recordings).

• Increases face validity and learner motivation.

Task Types

• Role‑plays simulating workplace interactions.

• Reading/writing tasks based on job‑related texts.

• Listening tasks using domain‑specific audio.

Scoring Considerations

• Criteria reflect both language accuracy and task effectiveness.

• May require raters with domain knowledge.

• Balance between linguistic and professional performance.

Advantages

• High relevance to learners’ goals.

• Strong positive washback on training programs.

• Can improve job readiness and performance.

Challenges

• Time‑consuming to design and validate.

• Requires specialist input and authentic materials.

• Risk of over‑narrow focus limiting transferability of skills.

Teacher Tips

• Collaborate with domain experts for authenticity.

• Pilot tasks with target‑group learners.

• Keep a balance between specific and general language skills.

Definition of Content-Based Assessment

• Assesses language ability through subject‑matter content (e.g., science, history).

• Integrates language learning with academic or vocational topics.

• Reflects real‑world contexts where language is used to learn content.

Rationale

• Mirrors how language is used in academic/professional settings.

• Encourages meaningful communication and deeper learning.

• Supports both language and content objectives.

Task Types

• Reading subject‑specific texts and answering comprehension questions.

• Writing summaries, reports, or essays on content topics.

• Oral presentations or discussions on subject matter.

Design Considerations

• Align tasks with both language and content learning goals.

• Ensure content is accessible to learners’ background knowledge.

• Avoid bias by selecting culturally and contextually fair materials.

Scoring

• Use rubrics that assess language skills and content understanding.

• Balance weight between language accuracy and content accuracy.

• Provide clear descriptors for each criterion

Advantages

• High authenticity and positive washback.

• Promotes integrated skill development.

• Motivates learners by showing practical relevance.

Challenges

• Requires collaboration between language and subject‑matter teachers.

• Risk of content overshadowing language assessment.

• More complex to design and score than single‑focus tests.

Teacher Tips

• Pilot tasks to check difficulty and clarity.

• Train raters to evaluate both language and content fairly.

• Use a variety of topics to maintain engagement and fairness.

Purpose of Statistics in Assessment

• Help interpret test results objectively.

• Identify patterns, strengths, and weaknesses in performance.

• Support decisions about test quality and fairness.

Descriptive Statistics

• Mean: average score.

• Median: middle score in ordered list.

• Mode: most frequent score.

Measures of Spread

• Range: highest minus lowest score.

• Standard deviation: how far scores vary from the mean.

• Smaller spread = more consistent performance.

Reliability Coefficients

• Indicate consistency of test scores.

• Values range from 0 (no reliability) to 1 (perfect reliability).

• Higher values = more dependable results.

Item Analysis

• Item facility: proportion of test‑takers answering correctly.

• Item discrimination: how well an item distinguishes between high and low performers.

• Used to improve test quality.

Correlation

• Shows relationship between two sets of scores.

• Positive correlation: both increase together.

• Negative correlation: one increases as the other decreases.

Practical Tips

• Use statistics to guide, not replace, professional judgment.

• Combine quantitative data with qualitative insights.

• Regularly review test data for improvement.

Purpose of the Chapter

• Encourages teachers to continue developing assessment literacy.

• Emphasises assessment as an evolving professional skill.

• Links assessment competence to better teaching and learning outcomes.

Lifelong Learning in Assessment

• Stay updated on new theories, tools, and practices.

• Reflect regularly on your own assessment methods.

• Adapt to changes in curriculum, technology, and learner needs.

Professional Development Strategies

Attend workshops, conferences, and training courses.

• Read research and practitioner literature on assessment.

• Join professional networks or communities of practice.

Collaboration

• Share assessment ideas and materials with colleagues.

• Observe and discuss each other’s assessment practices.

• Engage in peer review of test design and scoring.

Self-Reflection and Action Plans

• Analyse strengths and weaknesses in your assessment skills.

• Set specific, measurable goals for improvement.

• Monitor progress and adjust strategies as needed.

Ethical and Fair Practice

• Commit to fairness, transparency, and respect for learners.

• Consider the impact of assessments on teaching and learning.

• Strive for positive washback in all assessment activities.

Looking Ahead

• Expect continued integration of technology in assessment.

• Anticipate more emphasis on authentic, performance‑based tasks.

• Recognise the growing importance of assessment literacy in all teaching contexts.