EDPY 303 Educational Assessment

Past Teaching cycle

Teaching - Learning - Assessment

Old assessment practices

Emphasis on outcomes

Isolated Facts

Paper and pencil tasks

after instruction

little feedback

assessment of learning

summative

Contemporary assessment practices

Assessing process

Application of knowledge/skills

authentic performance tasks

before, during, and after instructions

detailed feedback

assessment for learning

formative and summative

Classroom assessment

The collection, evaluation, and use of information to help teachers make decisions regarding student learning

Goals of classroom assessment

Improving student learning

Improving instruction and planning

Measuring and evaluating student achievement

Providing information for student placement.

Communicating understandable and meaningful results

Assessing the effectiveness of instructional and assessments practices.

Questions when preparing assessments

Preparing: Why am I administering this assessment?

Measuring: What methods should I use to gather the required information?

Summarising: How will I interpret the assessment?

Applying: How will I utilise assessment results?

Teacher responsibility

Legal Obligation: The Education Act

196 A teacher while providing instruction or supervision must:

regularly assess students and periodically report the results of the assessment to the students, the students parents and the board.

Alberta Teaching Quality Standards

Standard 3 A Teacher applies a current and comprehensive repertoire of effective planning, instruction, and assessment practices to meet the learning needs of every student. Achievement of this competency is demonstrated by indicators such as applying student assessment and evaluation practices that:

- accurately reflect the learner outcomes within the program of study;

- Generate evidence of student learning to inform teaching practice through a balance of formative and summative assessment experiences;

- Provide a variety of methods through which students can demonstrate their achievement of the learning outcomes;

- Provide accurate, constructive and timely feedback on student learning; and

- support the use of reasoned judgement about the evidence used to determine and report the level of student learning.

Challenges for New Teachers - What are the potential challenges for new teachers regarding assessment practices?

- Understand program of studies standards

- Choosing the most suitable assessments

- Identifying the fairest way to assess each student.

- Communicating assessment results to students, parents, and other consumers.

Factors that can affect assessment processes - outside teacher control

- unclear/ambiguous provincial outcomes and/or standards

- Student demographics

- Student "issues"

- School/ district/ provincial policies

- The quality of assessment training experience: university, professional development.

- The inevitability of measurement error.

factors that can affect assessment processes - within teacher control

- Alignment of instruction and assessment with Program outcomes and standards

- Quality of assessments (fair, valid, and reliable)

- Variety of Assessments

- Frequency of Assessments.

Three P's of Assessment

Pedagogical

Political

Practical

Effective Teaching and Assessment: AISI Study

1 Know the Program of Studies thoroughly

2 Make learning outcomes known to students

3 use of rubrics, exemplars and criteria

4 I CAN statements

5 Continuous informal feedback

6 Questioning skills

7 Use of portfolios

8 Student self assessment

Big C Curriculum PAIR

P program of studies

A assessment

I instruction

R resources

P - What are students required to learn?

A - How can I best determine what students have learned?

I - Which teaching methods will be most effective?

R - What resources will animate the program of the benefit of my students?

Who is responsible for each letter of PAIR?

P - Alberta Education

A - Primarily teachers

I - Primarily teachers

R - Alberta Education/ teachers

T or F It is legally required that teacher must teach the outcomes provided in Programs of Study.

True (with some exceptions

What are ways to tell student knowing

- Fill in the blank

- true / false

- conversation / questioning

- matching items

- recall multiple choice

- Observation

What are ways to tell student understanding?

Restricted - constructed response

source based m/c items

Conversation / Questioning

Observation

What are student ways of doing

- Observation

- Extended Constructed response

0 Performante assessments

What are the principles for fair assessment?

1 Plan and develop assessments

2 Collect Assessment information

3 assess evidence of performance

4 summarise and interpret results

5 communicate assessments information

When developing assessments ensure ..

- assessments and instructional strategies are prepared concurrently

- assessments align with program of studies expectations

- assessment methods are compatible with the instructional approaches used

- assessments are based on valid measurements of the knowledge and skills learned.

- A variety of forms of assessments are clearly defined as either formative of summative

- Care is taken to ensure that expectations are clear, concise, and unambiguous

- Assessments focus on purposeful knowledge and appropriate skills

- Assessments are appropriate to student backgrounds, experiences, and learning needs.

- Stereotypes /offending language is avoided

- Reliable and valid information is generated about the achievement of each student.

T or F ensure assessments align with program of studies expectations (fidelity)

true

T or F ensure a variety of forms of assessment are provided to ensure comprehensive measurement of student achievement

true

T or F provide a variety of forms of assessment to ensure comprehensive measurement of student achievement.

True

True of False A variety of forms of assessment are provided to ensure comprehensive measurement of student achievement.

true

T or F Care is taken to ensure that expectations are clear, concise, and unambiguous

true

Collect Assessment Information

- Students clearly understand both why and how they will be assessed

- Assessments are completed under optimum conditions (proper lighting, minimal noise, etc.)

- Students have sufficient time to complete assigned tasks

- When observational assessment are used, limited number of components are assessed at one time.

- Students are encouraged to complete all assigned tasks

- When collecting assessment information, the support provided for each student is equitable.

- Circumstances that interfere with assessment are noted and considered when results are interpreted.

- Unless explicit mandated by a program of studies, group work is used for formative purposes and not as a part of the calculation of an individual student's assessment result.

- Self reflection and / or peer feedback are used for formative purposes and not as part of the calculation of assessment results.

- Alternative assessments are developed in accordance with policy and provided when professionally determined to be appropriate.

Assess evidence of performance

- Procedures and criteria for assessing a task are in place prior to its administration.

- Students reflecting upon their own work or providing feedback to peers are provided clear directions regarding what they are to do and how to maintain objectivity.

- Student self reflection and peer feedback are used judiciously and appropriately.

- Peer feedback benefits the recipient and maintains student dignity.

- An appropriate weighting is assigned to each summative task.

- When assessing students, favourable or unfavourable personal biases do not affect the assessment process.

- Comments on student performance are designed to improve learning and directly linked to program of studies expectations.

- Assessment processes are adapted to account for unanticipated responses.

- Provincial standards are applied.

Summarise and interpret results

- Students and parents know how assessments results are determined.

- Policies that explain how results are summarised and interpreted are readily available to students and parents,

- Summative comments or assessment results should include an explanation of how they were derived from assessment processes.

- Non - gradable factors such as effort, participation, behaviour, and attendance are reported, but not used in the calculation of results.

- Summative results and comments are based on more than one assessment.

- When calculating a single summary result careful consideration is given to the weighting assigned to the contributing elements.

- The basis for interpretation of assessments results is described and justifable.

- Interpretation of results takes into consideration of the backgrounds of students.

- Student scored are based on performance relative to provincial standards and not performance relative to other students in the same class.

- In determining assessment results factors that contribute to measurement error are considered.

Communiate Assessment Information

- Communication of assessment results is timely

- Communication of assessment results is meaningful and in language that is readily understood.

- Communication of achievement links student performance to program of studies expectations and standards.

- Conferences among parents, students,, and teachers occur with the goal of improving student learning and achievement.

- Parents and students are informed, at the beginning of the course or school year as to the process for appealing assessment results.

- The confidentiality of individual student assessment results is ensured,

- The maintenance and transfer of assessment information from school to school are guided by policy and ensure security and confidentiality.

Measures of reliability & Validity

R - Consistency of an assessment V - Appropriateness of an assessment

Questions asked in reliability and validity

R - is error of measurement minimised? V - Are we assessing what we should be assessing?

Strategies to increase reliability and validity

Reliability - Use of a variety of assessment methods, consistent administration, clear directions, avoid ambiguously worded questions, work with other teachers.

Validity Clearly specify the intended learning, identify criteria, select relevant and representative tasks

Relationship between reliability and validity

- Reliability provides the consistency of results that makes valid inferences possible. Reliability = valid inferences.

- Reliability is a necessary, but not a sufficient condition for making valid inferences , reliability inside validity.

Reliability

- Measured by a reliability coefficient, typically denoted as the Greek letter rho (p)

- p is a number between 0-1 , 0.0 the scored are completely inconsistent (unrelated)

- 1.00 - the scores would be completely consistent (identical)

- error of measurement causes reliability to be less than 1.00

Assessment reliability

Reliability ranges from 0-1, where numbers closer to 1.0 indicate higher reliability.

- The lower the reliability number the greater the concern regarding the fairness of examination. - Unreliable

- Reasonable reliability or a classroom exam .70 to .80. - acceptable reliability

- Typical reliability for a diploma or exam .88-.95. - highly reliable

Contributions to error within a student

- lack of reading skill

- language barriers

- emotional upset

- poor health

- physical handicap

- lack of motivation / concentration

- test wiseness

- luck in guessing

- lack of personal confidence leading to test anxiety

- carelessness in recording answers

contributions to error within the environment

- noise distractions

- poor lighting

- cultural insensitivity in assess or the assessment.

contributions to error within the test and scoring

- ambiguous items

- tricky questions

- poor directions

Scoring

- insufficient scoring guidelines

- carelessness

- computational errors

Collecting Valid assessment evidence - suggestions for improving assessment validity

Prepare an examination blueprint

• Does the exam cover content as different cognitive levels?

Compare results from different assessment methods measuring the same knowledge/understanding/ skills

• Are the results from different assessments similar?

• Can we use current results to predict future results?

Ensure a wide range of difficulty in assessments

• is the exam / assignment too difficult or too easy?

Provide enough time to complete assignments/assessments

• Does the assessment test the targeted knowledge and/or skills or a student's performance speed?

Ensure appropriate vocabulary, sentence structure, and clarity in the language.

• Is the assessment sensitive of linguistic and cultural differences?

Potential assessment methods: selected response items

- multiple choies

- Binary choice

- Matching

Assessment methods: constructed response items

- Brief constructed response items

- Performance tasks

- Restricted response and extended response

- Oral questioning

Assessment methods

Performance assessments, teacher observations and student self reflection.

Pre instructional assessments

Differentiated needs, current ability needs, readiness to acquire the knowledge and skills outlined in program of studies.

Purposes of pre instructional assessments include:

identifying students' relative strengths and weakness

establishing realistic expectations for instruction

modifying instructional objectives

establishing initial student grouping

differentiating instruction based on the needs of students

identifying problems that might affect the mastery of an objective

Diagnostic assessment identification

Students with learning difficulties

Students with special needs

The literacy and numeracy levels of students

Gifted and talented students

Student learning styles

Formative assessment

A planned process in which evidence of students learning is gathered and used by teachers to inform their instruction and by students to improve their learning.

Formative assessment strategies

Assessed by the extent to which they are successful at ensuring that all students learn to the best of their abilities.

Key formative assessment goal

monitor learning progress - assessment FOR learning

What formative assessment is NOT

- Assessments that fail to inform instructional adaptation.

- Assessments that do not promote student learning but focus on measuring achievement.

- Short assessments given at the end of a lesson / chapter; unless each serves to inform individual students about their proficiency in achieving learner outcome goals.

- Walking around, observing students during the learning process, and making generalised comments.

- formative assessment requires specific, indivualized, and action - oriented feedback.

Rick Stiggins: Specific Actions to Improve Assessment

1 balance assessment systems

2 Refine standards

3 Assure assessment quality

4 Turn leaners into assessors

5 Use feedback to support learning

6 Build leader confidence

7 Develop assessment literacy of teachers

Formative Assessment Strategies

Informal strategies

- facial expression

- body language / gestures

- voice related cues

- oral questioning

Formal strategies

- homework

- assignments, performance assessments, and in class quizzes.

- group activities

- student self - reflection

Informal Formative Assessment Strategies

Informal indicators include cues from the students:

- Attention

- Facial expressions

- questions asked by students

oral questioning can also be used

- involve students in the lesson

- promote students important content to be learned

- signal to students important content to be learned

Control student behaviours and manage the class

- obtain information about student understanding and progress

Formal Formative Assessment Strategies

Homework assignments

- homework can extend, expand, and elaborate student learning.

- homework can be used for checking student learning.

Teachers can use homework to identify which specific areas of knowledge and skill need further instruction.

Limitation: students can receive assistance from parents, siblings, and friends without learning the content.

In class assignments

- individual or group activities can be used for monitoring students.

- The assignment should allow the teacher to monitor students closely, give immediate feedback, and encourage students to do self assessment (based on rubrics provided before the assignment).

Limitation: Providing students with simple worksheets to work in class is NOT a formative assessment unless it is returned with specific feedback times at guiding improvement,

Formal Formative Assessment Strategies (2)

Quizzes

- Quizzes provide the teacher with an indication of current progress.

- The information from quizzes can be used by the teacher to identify which content areas need further instruction or clarification.

Limitations

- It is hard to motivate students to take a formative quiz seriously if the score does not count.

- Some students may not perform well on quizzes due to test anxiety or other reasons.

If feedback is not timely, it is not as effective.

Formal Formative Assessment Strategies

Group activités:

- group activities are effective in creating a collaborative learning environment. Student can receive peer feedback from their group members or from members of other groups.

Limitations: It is very hard to identify each students contribution to group work. Feedback to a group is not individualised feedback.

Student self reflection (self assessment)

- Students gather information about themselves and reflect on their learning. Students need to have clearly identified criteria, appropriate resources, and training in self reflection.

Limitations: Self - reflection is not effective if students do not understand how to reflect on their performance or do not have the resources to assist them.

Effective Praise and Feedback

Praise

- Is specific

- is timely

- relates to standrads

- focuses on attributions

- celebrates progress

- is genuine

Feedback

- relates to performance to standards

- Indicates current progress

- indicates corrective action

- Is given frequently and immediately

- is specific and descriptive

- addresses errors

Using formative assessment appropriately

Formative process with the goal of mastery

Summative assessment to. determine if mastery is achieved

Assessment for Learning vs. Assessment of Learning

Formative: FOR learning

- provides feedback to students to support learning

- provides feedback to teachers to inform instruction

- continuous stream of information to students and teachers about where students are at.

Summative OF learning

- measurement of student learning at the end of a learning process it can provide feedback to students and others.

- A "portrait" that tells students what they have achieved relative to program standards.

Balanced Assessments: Two Balancing Acts

Balance between formative and summative assessment:

- Summative assessments are essential but not enough.

- Formative assessments must be embedded in instruction to

- promote student learning

- inform instruction

- provide students with feedback

- formative assessment provide students with the "toolkit" needed to achieve their best possible results on summative assessments.

Balance in types of classroom assessment used:

- use a variety of assessments

- assessments should allow students to demonstrate and enhance their knowledge/skills.

Summative Assessment

- Summative assessments are perceived as important by both students and parents.

- Summative assessments are based upon formal evidence and are carried out relatively infrequently.

- The primary Goal of summative assessments is to give students a fair opportunity to demonstrate their achievement of program expectations.

Characteristics of Summative Assessment

High quality summative assessments have three important characteristics:

1 tasks include a representative sample of the content taught.

2 The skills / knowledge assessed have been emphasised in instruction.

3 The tasks, directions, and scoring are clear and straightforward.

Preparing high quality summative assessments

- Identify the learner outcomes

- selecting suitable assessment tasks

- seeking to minimise assessment error

- preparing an assessment that is appropriate in format and rigour.

Representative Sampling of Content

- Selecting a representative sample of concepts and skills

- Create an assessment blueprint.

Blueprints identify assessed content areas and how many tasks are used in each area.

Blueprints are shares with students to help them prepare for assessment.

preparing as assessment blueprint

A blueprint helps answer the following questions

- Are there any over or under sampled content areas?

- Does the content of the assessment fairly reflect content emphasis during instruction?

How appropriate is the cognitive complexity of the assessment?

- Is there too much emphasis on remembering / understanding questions?

What s the length of the assist?

- Is it practical and efficient given the time constraints?

Number and Length of Assessment

Decision regarding how many tasks depends on several factors:

- The age level of the students

- The length of classes

- The type of tasks

Ensure enough time is allowed for the test, range of difficulty is acceptable. how will open ended questions be graded objectively

Prepare students for summative assessments by:

Teaching FOR the test

Telling students well in advance when the exam is scheduled

Familiarising students with length, format, and question types

Sharing the blueprint with students

Telling students about the duration of the exam

Running a review session

Putting a Summative Exam Together

Assembly

- Create a directions section

- Arrange questions in a logical order

- Determine the physical layout

Preparation

- Print the exam '

- Review the exam to ensure there are no error

- Determine what resources are necessary

Administration, scoring, and reporting

- Administer the exam

- Score the questions objectively and accurately

- Report results in a timely fashion

Exam directions

Exam directions should include the following:

Purpose , time allocated, basis for responding, procedures for recording answers, what to do about guessing, how short answer items will be assessed.

Arranging questions

- Arrange tasks by topic

- Start with several easier items to help calm students who may be anxious taking tests.

- Students experience test fatigue with a longer exam.

Arrange task by format - helps student focus, makes scoring easier for teachers.

Physical layout of the Exam

- When the exam is printed, the questions should be easy o read and answer.

- Place an appropriate number of items on each page and avoid having part of a question or question set extend onto another page.

- When using selected response items make sure to provide instruction on how to indicate responses.

Administering the Exam

When administering an assessment:

- The environment should have proper conditions

- Work to prevent cheating and plagiarism, provide clear guidelines about cheating, format the assessments to make cheating difficult. Close monitoring during the exam.

Scoring and Reporting Results

Learner Outcome

LO is a statment regarding student performance that should be demonstrated at the end of an instructional unit or lesson.

Other terms for LO are: Expectations, learning goals, learning expectations

Instructional Objective

IO is a statement about a specific learning expectation that is stated in terms of specific, observable and measurable student behaviours.

An IOO identifies a student performance that will demonstrate the achievement of the learner outcome.

Other terms: behavioural or performance outcomes, learning target.

High Quality IO

high quality IO should..

- be specific

- be aligned with the intended knowledge or skills in the LO

- Identify observable and measurable student performance, and

- be concise and clear

Aligning the LO and IO

Learner outcome:

- Student will measure liquid and dry ingredients using appropriate instruments

Instructional objectives:

- Student will measure liquid and dry ingredients using appropriate instruments.

- Students will describe the role of each ingredient in a cake better

etc...

Differences between LO and IO

LO & IO

Scope - broad - narrow

Function - develop curriculum - plan the instruction

Time to accomplish - varies greatly depending on the scope of the outcome - usually anywhere from less than a class period to several classes

Who's responsible for developing it - government agencies - teacher

Bloom's Revised Taxonomy - Evaluating

make judgement based on criteria and standards

Bloom's Revised Taxonomy - Creating

Put different elements together to form a new structure or reorganise the elements together to form a new structure or reorganise the elects into a different structure.

Blooms taxonomy - Analysing

Break the knowledge into its core parts and determine how the parts relate to one another and the overall structure.

Applying

Students will adapt a cake recipe

Understanding

Students will provide reasons for using each ingredient in baking a cake.

Remembering

Student will identify key ingredients needed in baking a cake.

Four Layers of Learning

Fact - knowledge - concept - understand

Going from narrow and concrete to broad and abstract

Performance Assessment

- enables students to perform a real-life task.

- Typically, in a performance assessment, multiple solutions are possible.

Performance assessment involves ...

- Students construct original responses for a given task.

- The teacher observes students response - construction progress. A judgment is made about the final product and/or the process.

PA is also called..

- Authentic assessment because, in some cases, a PA may represent a real-world situation.

- Alternative assessment because it can differ from tradition paper - and - pencil tests.

- PA's are best used to assess learner outcomes at the analysing, evaluating, and creating levels.

- PA's should allow for varied student responses each of which may be valid.

Ideal Uses for Performance Assessments

Supporting the learning process and related to skills and measuring achievement using either or both processes and products.

PA's are best suited for assessment at the upper levels of Bloom's taxonomy.

Building PA's

- Task is aligned with program outcomes

- Instruction has taught knowledge and skills required to complete the task.

- Assignment instructions are clear, complete, and concise

- A high quality scoring rubric is provided using student friendly language.

- Scoring rubric criteria are aligned with program outcomes.

- Exemplars illustrating standards are provided.

- Raters are trained using exemplars and rubric prior to scoring completed tasks.

Creating a PA

Step 1 - Determine which learner outcomes should be addressed with the assessment.

Step 2 - Think of an authentic task that will measure the achievement of the learner outcomes

Step 3 - Develop scoring criteria that identify the tasks required to complete the performance assessment.

Step 4 - Organise the criteria into a scoring tool and define the levels of achievement.

Restricted performance tasks

- Requires performance of a specific and limited skill

- Are more structured and specific, limiting the scope of the response

- Are similar to come aspects of constructed - response items, such as short essay questions and interpretive exercises.

Extended performance tasks:

- Are more elaborate and time consuming

- Requiring students to demonstrate multiple skills.

- Assess complex cognitive skills, communicate skills, use of visual materials, realism, and complexity

Ways to Engage Students with a Performance Assessment

- Select topics that would be relevant and of interest

- Offer choice within the assessment

- When practical, provide the details about the assignment well in advance

- Ensure students are familiar with assessment criteria

- Provide exemplars to illustrate expectations/standards

- Encourage creative / risk taking

- Provide detailed / specific feedback

- Facilitate peer feedback and / or self reflection

Engaging students: provide meaningful context

- explain why were doing this

- engage the imagination .. "imagine you have been chosen to..."

Identify person benefits

"understanding the perspectives of others is an important life skill that helps you..."

Ideas for PA

Creating

- adapt, revise, design and construct

Evaluating

- Judge, assess, critique

Analysing

- Contrast, categorise, differenziate

Ideas for analysing

examine a questionnaire to determine if the questions are appropriate.

Categorise survey results from classmates to determine the opinion of the class on an issue.

Contrast information from two sources to ermine if there is factual agreement.

Estimate costs for building a cherry-wood bookcase.

Subdivide recreational cavities into those that are affordable for families and those that would be costly for many families.

Differentiate objects based on specific criteria

Classify items of clothing based on the type of weather for which each is best suited.

PA ideas for evaluating

Judge a debate on the topic of whether zoos are beneficial for animals

Assess the quality of three websites that provide language translations.

Support your choice for a list of the five most important ways we can be kind to others.

Critique the government's environmental protection plan by providing the pros and cons of the plan

Justify your choice for two animal to place on the school's new coat of arms.

Recommend a website that would best help new immigrants adapt to life in Canada.

PA ideas for creating

Invent a machine that dispenses candy when the correct sequence of button are pressed.

Adapt an advertisement to convince people to not buy the product.

Revise the rules of a sport to attract more participants / viewers.

Re write a scene from a tragedy to change it to a comedy

Create the images for a new $20.00 bill that celebrates a milestone in Canadian science.

Design the ideal house, from the perspective of a cat

Construct a model of a playground that would encourage more creative play.

Portray the Mona Lisa as it would have possibly looked if it had been: painted by Da Vinci when he was seven years old or created by a contemporary Indigenous artists

Checklist for PA

- Focus on essential content and skills related to the learner outcome

- Ensure a high degree of authenticity

- Be feasible and manageable for students

- Allow for multiple solutions and and paths / processes

- Be challenging and stimulating for students

- Be assessed through pre - specified criteria

- Provide sufficient time of r the task to be completes

- Be equitable

Advantages of PA

Integrate assessment with instruction

Provide opportunities for formative assessment

Can be more authentic and engaging than other types of assessments

Emphasise high level cognitive skills