1/28
Flashcards for KIN 3313 - Assessment and Evaluation lecture review.
Name | Mastery | Learn | Test | Matching | Spaced |
---|
No study sessions yet.
Reliability
The consistency or repeatability of scores resulting from a testing procedure.
Other terms to describe reliability
Consistency, dependability, stability, and precision.
Validity
The degree of truthfulness of a test score, measuring what it purports to measure, dependent on reliability and relevance.
Relevance
The degree to which a test pertains to its objectives.
Objectivity
A special kind of reliability, consistency between two or more raters (interrater) or within the same rater over time (intrarater).
Interrater reliability
Consistency between two or more independent judgments of the same performance made by two or more raters
Intrarater reliability
Consistency in scoring when a rater scores the same test or performance two or more times
Pearson product-moment (PPM) correlation coefficient
A correlation coefficient used to provide evidence of a test's reliability.
Observed score
True score + error score
True Score
Theoretically exists but is impossible to measure; the theoretical average of an infinite number of test takings.
Error Score
Results from anything that causes the observed score to differ from the true score.
Theoretical Conclusions of Reliability
Observed score variance = true score variance + error score variance
Interclass coefficients
Based on PPM correlation coefficient
Intraclass coefficients
Based on ANOVA
Interclass Reliability
Test-retest reliability, Equivalence reliability, Split-halves reliability
Test-Retest Reliability
Administering a single test twice to participants and correlating the scores.
Equivalence Reliability
Two parallel or equivalent forms of an exam are given to participants.
Split-Halves Reliability
A single test split into parts is given to participants.
Intraclass Reliability
Estimates reliability when scores from more than two trials are available and addresses constant differences between means.
Factors Affecting Reliability
Participant variability, time between testing, circumstances surrounding the testing periods, precision of measurement, environmental conditions, fatigue, practice and appropriate level of difficulty for testing participants.
Five Evidences of Validity
Evidence based on test content, relations to other variables, internal structure, response processes, and consequences of testing.
Test content
The themes, wording, and format of the items, tasks, or questions on a test.
Evidence Based on Relations to Other Variables
Analyzing the relationship between test scores and variables external to the test.
Convergent evidence
Relationships between test scores and other measures intended to assess similar constructs
Discriminant evidence
Relationships between test scores and other measures intended to assess different constructs
Evidence Based on Internal Structure
Degree to which the relationship among test items and test components conform to the construct on which the proposed test score interpretations are based
Evidence Based on Response Processes
Degree to which the processes of test takers or scorers are consistent with the intended interpretation of scores
Evidence Based on Consequences of Testing
Focuses on scoring meaning and the intended and unintended consequences of assessment use
Important considerations in reliability and validity
Reliability and validity results are specific to the group tested, the environment of testing, and the testing procedure and are not typically generalizable