[LA] Terms

language assessment

an ongoing process of collecting information about a given performance according to systematic and substantively grounded procedures

teaching

sets up the practice games of language learning: the opportunities for learners to listen, think, take risks, set goals, and process feedback from the 'coach' and then recycle through the skills that they are trying to master

strategic competence

the ability to use communicative strategies to compensate for breakdowns as well as to enhance the rhetorical effect of utterances

construct

what a test measures

practicality

the extent to which resources and time available to design, develop, and administer a test are manageable and feasible

reliability

the extent to which a test yields consistent and dependable results

inter-rater reliability

condition in which two or more scorers yield consistent scores for the same test

intra-rater reliability

condition in which the same scorer yields consistent scores across all tests

student-related reliability

a learner-related issue such as fatigue, anxiety, or physical or psychological factors that may make an "observed" score deviate from one's "true" score

test reliability

consistency of different facets of a test (e.g., instructions, item types, organization) during each test administration

validity

• whether a test 'measures accurately what it is intended to measure'

• 'the extent to which inferences made from assessment results are appropriate, meaningful, and useful in terms of the purpose of the assessment'

content-related/ content validity

the extent to which a test actually samples the subject matter about which conclusions are to be drawn

criterion-related validity

the extent to which the linguistic criteria of the test (e.g., specified classroom objectives) are measured and implied predetermined levels of performance are actually reached

concurrent validity

the extent to which results of a test are supported by other relatively recent performance beyond the test itself

predictive validity

when the test results are used to predict future performance, such as academic success (e.g. placement tests)

construct validity

to what extent a test is able to measure its claims

consequential validity

a test's impact, including such considerations as its accuracy in measuring intended criteria, its effect on preparation by test-takers, and the (intended and unintended) social consequences of the test's interpretation and use

face validity

• the extent to which 'students view the assessment as fair, relevant, and useful for improving learning'

• 'the degree to which a test looks right and appears to measure the knowledge or ability it claims to measure, based on the subjective judgement of the examinees who take it, the administrative personnel who decide on its use, and other psychometrically unsophisticated observers'

authenticity

the relationship between test task characteristics, and the characteristics of tasks in the real world

washback

the effect of assessments on classroom teaching and learning

test specifications / specs

provide the official statement of what the test tests and how it tests it, can be a simple and practical outline of the test

assessment plan

an overall guide for how we will assess students' achievement of the learning goals and outcomes relevant to instruction

item facility (IF)

the extent to which an item is easy or difficult for the proposed group of test-takers

item discrimination (ID)

the extent to which an item differentiates between high- and low-ability test-takers

distractor efficiency

the extent to which: The distractors 'lure' a sufficient number of test-takers, especially lower-ability ones. Those responses are somewhat evenly distributed across all distractors.

proficiency test

assess general ability in a second language

achievement test

evaluate how much a learner knows from a defined amount of course or class work

diagnostic test

a test for identifying a student's strengths or weaknesses in specific areas of language

placement test

determine which would be the most appropriate class, stream or level in which to place a student so that subsequent language teaching is appropriate to their needs

performance-based assessment

involves oral production, written production, open-ended responses, integrated performance (across skill areas), group performance, and other interactive tasks → more direct testing, real-world tasks >< time consuming, expensive

dynamic assessment

promoting learner development in the abilities being assessed through feedback or other forms of support

measurement

the process of quantifying the observed performance of classroom learners (test scores)

evaluation

is involved when the results of a test (or other assessment procedure) are used to make decisions (meaning of the score; interpretation of performance)

learning goals/outcomes

• what does the T want Ss to learn explicit

• statements of expectations describing the skills, knowledge, attitudes and capabilities that students should achieve as a result of our work with them during a course

classroom activity

what will T do; what will Ss do

assessment

(in an assessment plan) what will Ss do to show what they have learned

horizontal plan

- Begins with the intended learning outcomes and the assessment tasks (e.g., quizzes, reports, presentations, portfolios, projects, group interactions), which we will use to collect evidence of our students’ learning.

- Through defined learning outcomes and backward design, we move from the outcomes of the course and their assessment to the day-by-day activity in the course.

vertical plan

- Consists of the day-by-day activity in a course.

- Maps assessment tasks, activity, events and resources in time (and over time) on to the intended learning outcomes of a course.

construct definition

the process of defining what it is we intend to measure

criterion-referenced assessment

an assessment that measures what students know and can do

norm-referenced assessment

an assessment that compares students' performance with that of other students to rank them

domain-referenced assessment / assessment of language for specific purposes

an assessment that focuses on a specific context by testing the language use and typical tasks required in that domain

target language use domain

the domain of interest on which the assessment is based, representing the real-life context in which the target language is used

rating scale

a description of the behaviours, features, or characteristics of a performance, which are referenced to point values that form a scale

scale

typically a numerical range from low to high

test techniques

means of eliciting behaviour from candidates → candidates' language abilities

multiple-choice items

all receptive, or selective, responsive in that the test taker chooses from a set of responses rather than creating a response.

gap-filling items

items in which test-takers have to fill a gap with a word

matching items

• assess Ss knowledge regarding characteristics associated with homogenous sets of ideas, terms, concepts, events,…

• two lists of related words, phrases, pictures, or symbols - each item in one list is paired with at least one item in the other list

short-answer items

items in which the test-taker has to provide a short answer, particularly in listening and reading tests

stem

presents a stimulus, the text of the questions

options/alternatives

the choices provided after the stem (these include the key and distractors)

key

the correct answer in the list of options

distractors

the incorrect answers in the list of options

functioning items

show acceptable IF, ID, choice distribution patterns

nonfunctioning items

violate one or more than one of the criteria set for a good item and should be modified, such as changing a distractor or revising the stem.

malfunctioning items

items with negative ID and should be discarded

conventional vocabulary test

assess knowledge of the meaning of particular content words (nouns, verbs, adjectives, and adverbs)

recognition test

asks learners to identify the correct meaning of an L2 word from a set of options rather than produce the word themselves

recall test

asks learners to retrieve and produce the appropriate L2 word based on a given meaning, context, or L1 equivalent

organisational knowledge

knowledge of how language and structures are produced to form grammatically correct utterances and sentences

grammatical forms

both morphology or how words are formed, and syntax, how words are strung together

grammatical meaning

the literal and intended meaning of utterances and sentences

pragmatic meaning

pragmatic or implied meaning results from the appropriate language choices a learner makes in a given communicative event

textual knowledge

language at the discourse level

grammatical ability

an individual's capacity to utilise mental representations of language knowledge built up through practice or experience in order to convey meaning

test method effect

the effect that the choice of format have on the cognitive processing that the task will elicit

discrimination task

(task type for assessing grammar) asks the test-takers to attend to input that can be either language or non-language and to respond in the form of a choice between or among contrasts or opposites, such as true/false

noticing/consciousness-raising task

test-takers are asked to indicate (underline or circle) that they have identified a specific feature in the language sample. (grammatical meaning) e.g.,

dialogue completion task

the input = short conversation or dialogue in which a part of the exchange or the entire exchange is left blank and the expected response meant to be grammatically correct

tokens

all words in the paragraph

types

words that are different forms (do not count words that are repeated)

alternative assessment

an alternative to standardised testing and all the problems found in such testing e.g., portfolios, journals, observations, self-assessment, peer-assessment

performance assessment

Is not completely synonymous with the concept of alternative assessment → one of the primary traits of the many available alternatives to assessment.

triangulate

consider at least two or more performances and/or contexts before drawing a conclusion

intensive listening

listening for perception of the components of a larger stretch of language

responsive listening

listening to a relatively short stretch of language in order to make an equally short response

selective listening

processing stretches of discourse such as short monologues for several minutes → 'scan' for certain information → to comprehend designated information in a context of longer stretches of spoken language

extensive listening

listening to develop a top down, global understanding of spoken language

perceptive reading

involving attending to the components of larger stretches of discourse when reading: letters, words, punctuation, and other graphemic symbols. bottom-up processing is implied

selective reading

• to ascertain learners'reading recognition of lexical, grammatical, or discourse features of very short stretch of language

• a combination of bottom-up and top-down processing may be used

interactive reading

• including stretches of language of several paragraphs to one page or more in which the reader must interact with the text

• top-down processing is typical (and sometimes bottom up performance)

extensive reading

applies to texts of more than a page, up to and including professional articles, essays, technical reports, short stories, and books → learner's global understanding Top-down processing is assumed for most extensive tasks.

speaking

a productive skill that can be directly and empirically observed

imitative speaking

the ability to parrot back (imitate) a word or phrase or possibly a sentence

intensive speaking

The production of short stretches of oral language designed to demonstrate competence in a narrow band of grammatical, phrasal, lexical, or phonological relationships. e.g.,: directed response tasks, reading aloud, sentence and dialogue completion; limited pictured cued tasks including simple sentences; and translation up to the simple sentence level.

responsive speaking

Interaction and test comprehension - limited level of very short conversations, standard greetings and small talk, simple requests and comments.

interactive speaking

• longer and more complex interaction than responsive speaking, sometimes include multiple exchanges and multiple participants.

• can take two forms: → Transactional language: purpose: exchanging specific information. → Interpersonal exchanges: purpose: maintaining social relationships.

extensive (monologue)

including speeches, oral presentations, story-telling language style: often more deliberative (involving planning) and formal, but sometimes including informal speeches

imitative writing

including the ability to spell correctly and to perceive phoneme-grapheme correspondences in the English spelling system form is the primary focus, context and meaning are of secondary concern

intensive writing

producing appropriate vocabulary within the context, collocation and idioms, and correct grammatical features up to the length of a sentence. meaning and context are→ correctness and appropriateness, but most assessment tasks a more concerned with a focus on form and are strictly controlled by the test design

responsive writing

• tasks at a limited discourse level, connecting sentences into a paragraph and creating a logically connected sequence of two or three paragraphs.

• form-focused attention is mostly at the discourse level, with a strong emphasis on context and meaning

extensive writing

• management of all the processes and strategies of writing for all purposes focusing on achieving a purpose, organizing and developing ideas logically, using details to support or illustrate ideas, demonstrating syntactic and lexical variety, and in many cases, engaging in the process of multiple drafts to achieve a final product.

• focus on grammatical form is limited to occasional editing or proofreading of a draft

wind-down (step 4 in interview)

feelings about the interview, information on results, further questions

rubric

an assessment tool often shaped like a matrix, which describes levels of achievement in a specific area of performance, understanding, or behavior

analytic scoring

provides a separate score for each characteristic or major element of a performance (e.g., grammar, vocabulary, organization)

assessment for learning

formative assessment which aims at bringing assessment and learning closer together

assessment of learning

summative assessment in which the product of learning is assessed

bottom-up listening strategies

listening strategies focusing on details and processing language at a sound or word level

c-test

a gap-filling variation where the second half of every other word is obliterated and must be restored