511 Week 5 Assessment

what is assessment?

A set of procedures designed to answer questions about a child’s language & communication development & functioning

in assessment procedures and questions should

line up

what are usually the two main diagnostic questions we are asking?

Is this child’s language:

• development/use within expectations for their age?

• consistent with language difficulties/language disorder?

what other questions are we trying to answer?

What does this child know about language and how do they communicate?

• Form, Content, Use

• Comprehension, Production

Even more questions!

• General observations and specific descriptions; interests and style of communication

• STRENGTHS and CHALLENGES

• Functional implications of knowledge and use in different contexts? (what is their academic/social environment like)

• parent questions/concerns

What are the three types of procedures for language assessment?

• Normative

• Criterion-referenced

• Descriptive

What does normative referencing explicitly tell you?

how a child ranks compared to same-age peers

What purpose does normative referencing serve in an assessment?

identifies if a child is performing within age expectations

What clinical decision does normative referencing help you make?

It helps determine if a child's difficulties point to a language disorder

What does normative referencing use to prove a child stands out from typical peers?

It uses statistical variability (like a bell curve).

What does criterion-referencing explicitly tell you?

if a child can perform a specific skill or criterion

True or False: Criterion-referencing compares performance of a child to other children

False: Performance is never compared to other children.

What purpose does a criterion-referenced test serve?

It answers a specific question about a child's unique abilities.

What is an example of a criterion-referenced question?

"Does the child understand directions using familiar verbs?".

What three factors determine the questions asked in this criterion-referencing?

Developmental needs, functional needs, and parent questions.

Descriptive assessment tool

• consists of broad, open-ended questions with no right or wrong answers

• for ex; what does the child like to talk about? How do they respond if they are not understood?

what are the 4 main purposes of assessment?

• Screening

• Identification / Diagnosis

• Intervention planning - how can we help them now?

• Progress monitoring - track progress, fine tuning & likely more assessment

Question: What is the difference between screening and identification / diagnosis?

• screening; do we think there might be a need?, Universal screening.

• Identification / Diagnosis; official now

what are the two main categories of assessment tools?

• informal/non-standardized

• formal/standardized

which type of assessment is more flexible?

informal/non-standardized assessment

What is another term for a Formal assessment?

A Standardized test.

What is another term for an Informal assessment?

A Non-standardized procedure.

True or False: Informal tools are less important than standardized tools.

False. They are just as important clinically.

How are Formal tests administered?

Using a set, identical method for every child.

How are Informal tools administered?

They are adapted to the individual child.

What type of referencing do Formal tests typically use?

Norm-referenced (comparing the child to same-age peers).

What do Formal tests take into account when scoring?

The natural range of human variation.

What is dynamic assessment? Is it formal or informal?

• informal

• involves assessing, teaching, and reassessing to see if they learned

What are examples of types of non-standardized assessment tools?

• Interviews

• Observations

• Language Samples

• Probes (Criterion-referenced)

• Questionnaires

• Portfolios

• Dynamic assessment 

What is a Language & Communication Sample, and when do you use it?

It is a record of real-world talk/play used to track spontaneous skills, or nonverbal communication if the child is silent.

What are Checklists & Questionnaires, and what are they best for?

• may be more “formal”

• often normed so you can get scores like you would on a standardized test

• They are caregiver/teacher forms used to score everyday pragmatics and social skills that you can't easily see in a short clinic visit.

What are Criterion-Referenced Probes, and when do you use them?

• They are flexible, multi-trial tasks used to test a specific target skill if the child didn't show it spontaneously.

• if you have spontaneous evidence of a skill then you don’t need probes

What are some examples of “probes” we might ask?

• can ___ ask questions?

• does ___ produce complex sentences?

• does ___ use eye gaze and gestures to request if shown something interesting?

• does ___ produce past tense –ed?

• does ___ turn and look at you when you say their name?

Could you diagnose a child solely based on standardized assessments? Why or why not?

no, b/c that is not sufficient enough evidence to make a diagnosis on its own

formal assessment supports…

diagnosis and funding

informal assessment supports…

diagnosis, funding, interventional planning, progress monitoring

what questions do we need to ask ourselves

• What do I know?

• What do I suspect?

• What do I need to know? How?

• What procedures might be best to answer the questions that I have?

a standardized test refers to

how the test is given; rigid, identical rule and methods for every child

a norm-referenced test refers to

how the test is scored; comparing the child’s score against peer norming

Standardized tests are typically an important component of the process of assessment and diagnosis because

• they identify how a child performs relative to their peers; within/below/above

• help with the question of discrepency relative to typical expectaions

• take into account typical varibility and ensure clinical objectivity and fairness

• provide context for observation of non-scored behaviour (attention, persistence, fatigue, child strategies)

Why are standardized tests not well-suited for setting intervention goals?

• they test a small number of forms and limited number of examples of each

• patterns of error on tests and in real-life can differ

• goals should be based on the language needs in their specific, real -life contexts

Why are standardized tests are typically not well suited to monitoring progress in intervention?

• designed to be stable and not sensitive to small changes

• if they hold their position relative to the norming sample, their score will not changed even if they learned a lot

• the child may show massive gains in therapy and daily function but this is not consistent with test scores

limitations of standardized tests

• unfamiliar and highly decontextualized for the child (must plan breaks, scheduling, encouragement)

• sub-scales by modality or domain don’t often map great onto one domain or extend to functional communication

raw score

• Total number of items credited as correct

• Takes into account basal and ceiling items

• meaningless to compare to peers

Bassal

Bassal: Item below which all items credited as correct (need to be included in the raw score) - for tests with variable start points.

Ceiling

Ceiling: Item above which all items counted as incorrect. Stopping point.

standard score

• Any score that has been converted to a standard mean and SD

• Provides a comparison to age group that takes into account average performance and variability

• Relationship to z-score

in practice a standard score is usually

M = 10 and SD = 3 (subtest scores); or

M =100 and SD=15 (composite scores)

percentile score

• Indicates the percentage of children in the norming sample (for the age range of interest) that obtained a lower raw score than the individual taking the test

• for ex; If a child is in the 70th percentile, it means they scored higher than 70% of the children in the norming sample. The remaining 30% of children scored higher than them.

True or false: Percentiles may be easier for parents to understand?

True

What are Age Equivalent scores (AE)?

• translate a child’s test score into an age level

• by matching their raw score to the median score of a specific age group.

Give an example of an AE score translated

if a 5-year-old child gets an AE of "3;6" (3 years, 6 months), it simply means their raw score matches the median score of typical 3-and-a-half-year-olds in the testing sample.

What are the limitations of AEs?

• They cannot be used to diagnose a disorder or show age expectations (because they ignore standard scores).

• They do not account for normal variation in development.

• They don't compare a child to their same-age peers (only to older/younger ages).

• They cause false alarms by making typical scores sound like a scary delay.

Sensitivity

Degree to which a test accurately identifies that a child DOES have the disorder in question

Specificity

• Degree to which a test accurately identifies that a child DOES NOT have the disorder in question

Sensitivity

• we take ALL the kids that we know have a language disorder and we say, of those how many of those did the test identify?

• ex; there are 100 kids, 40 have a LD, test identified 10.

• sensitivity = 25% (not great)

Specificity

• child does not have a LD, and the child’s test score is consistent with that.

• take all the kids that do not have a LD, how many got a score that was high enough to agree with that reality

• take that number and divide by ALL of the kids with no LD

• test wants to catch the kids that have a LD and not catch the ones that do not

sensitivity and specificity help you choose a test that…

• has a better chance of catching hte kids that it needs to catch and letting the other kids pass through that should

• want .8 or 80%!

Positive Predictive Value

• likelihood that a child who scores in the range of a language disorder actually has one

• denominator or pool is all the kids that the test identified as having a LD

• so which ones actually do?

Negative predictive value

• likelihood of a child who on their test score looks like they don’t have a LD, whats the likelihood that they really don’t

• pool is kids who the test says they don’t have an LD, numerator is the ones who really dont vs all of the ones the test said don’t