1/33
Looks like no tags are added yet.
what is Bloom’s Taxonomy, and what are its cognitive categories/domains?
Bloom’s Taxonomy labels the ‘levels’ at which we think. From least to most effortful/cognitively demanding:
knowledge (remembering): memorization of basic knowledge
comprehension (understanding): being able to re-state information in your own words
application: being able to apply knowledge/comprehension of material to a new situation (ex., like solving a math problem by following its ‘rules’/formulas, etc)
analysis: being able to break down information/concepts into their individual parts (ex., giving a presentation in class or writing a paper)
synthesis (evaluating): critically thinking about content/information by putting its individual parts together to form a whole concept (ex., writing an essay about a book’s themes)
evaluation (creating): encompasses all levels of learning (ex., creating an exam or experiment)
what types of assessment items are “supply,” and which items are “selection”?
supply: short answer, fill in the blank (completion)
selection: multiple choice, matching, T/F
what kind of assessment item(s) are considered “performance” based?
Essays (short, long, restrictive, extended)
what assessment item type(s) have higher scoring reliability, and what assessment item type(s) have lower scoring reliability? which ones are easier to create?
Objective supply and selection assessment items (T/F, multiple choice, fill in the blank, etc) are harder to create, but have higher scoring reliability. Performance assessment items (ex., short answer, essays, etc) are easier to create, but have lower scoring reliability.
what is validity, and how does it differ from reliability?
validity: how well a set of scores reflect whatever the test is supposed to be measuring
reliability: how random a set of scores are
reliable tests are not always valid, but a test cannot be valid unless it is reliable.
describe each type of validity and when/in what context they are used.
content-based validity: used when trying to establish that the items on a test should reasonably be there, or are reasonably representative of other items that should (theoretically) be on the test
criterion-based validity (concurrent vs. predictive): when trying to determine whether a test’s scores are similar to other scores from tests that measure the same thing, now (concurrent) or in the future (predictive)
construct-based validity: when trying to demonstrate that the scores on a given test have a clear relationship to what the test aims to measure
what is criterion contamination?
when a confounding variable(s) influences how closely related two test scores are, making the validity of the test scores questionable.
describe what factors/events could cause a test score to be unreliable?
when a test is too easy or too hard, trait and method errors, lack of standardization, etc
what is the difference between trait and method errors?
trait errors: an error that comes from the individual taking the test (ex., waking up late, not studying enough, etc)
method errors: an error that comes from the environment/site at which the test is taken (ex., technological issues, temperature of the room, missing items on the test, etc)
what are the different levels of measurement and how do they differ from one another?
nominal (offers least information): when numbers signify different categories (always qualitative and mutually exclusive)
ordinal: describes how variables can be ordered along some kind of continuum
interval: scores are assigned value based on a continuum with equal intervals
ratio (offers most information): physical, direct, numerical data that can have an absolute zero
all levels of measurement, except for nominal, allow for an ‘order’ of data to be established.
what are the five main purposes for using a test?
selection
placement
diagnosis
classification
research
what is the difference between aptitude and intelligence tests?
aptitude tests can include intelligence tests, or function as both, but in general, an aptitude test is predictive, whereas intelligence tests on their own are not predictive, and focus on the test taker’s current capabilities.
who ‘pioneered’ the first ‘mental test’?
James Cattell
what three people were involved in the development of the Stanford-Binet test (1905)?
Alfred Binet, Theodore Simon, and Lewis Terman (who brought the test to Stanford from Europe)
what is the purpose of testing, and what different areas/domains do we test?
to measure an outcome (score) and make sense of that judgement
achievement (level of knowledge in a particular domain)
aptitude (potential to succeed)
personality
vocational/career interest (job-related interests… can sometimes be an aptitude test)
ability/intelligence (skill/competence)
neuropsychological
what is the difference between Item Response Theory (IRT) and Classical Test Theory (CTT)?
CTT: aims to increase the accuracy of the measure of a test taker’s ‘true score’
observed score = true score + error score
IRT: the idea that the likelihood of a correct response on a test is a complex function of the person taking the test and the characteristics of each individual item on the test
CTT focuses on the test as a whole, and IRT focuses on a test’s individual items
what three scores comprise an overall ‘test score’?
true score: the score of a test if taken an infinite number of times
observed score: the original score obtained on a test
error score: trait error and method error (the smaller the error score, the greater the reliability of the test score)
what is the 12 step process of test development?
choosing and defining constructs to test
determine best testing method (paper-pencil, online, observation, etc)
item development
pilot test the preliminary items
use data from pilot test to edit, revise, or make new items for the exam
pilot test again for the adjusted or new items
final configuration/grouping of items based off data collected from pilot tests
develop directions + guidelines for the test
conduct final reliability/validity tests for the exam
make revisions to test based on data collected from reliability/validity tests of the exam
conduct tests to find typical performance levels (norm) for a given population
develop test manual
how much does testing cost in the U.S.?
testing in U.S. schools costs $1.7 billion annually
describe the early history of intelligence testing in the U.S.
began in China ~ 3,000 years ago
Darwin’s Origin of Species insinuated that traits/attributes are passed down generationally as a result of adaptation
increased interest in differences between individuals
First mental test - James Cattell
founded Psychological corp. (1920)
Testing boom in 20th century
Stanford-Binet test
intelligence testing before WWII (verbal and non-verbal)
testing spreading into civilian life post-war (schools, businesses, hospitals)
emergence of personality testing
testing in education (SAT, GRE)
what is the definition of a test?
a measure of an outcome
what is the definition of a variable?
a value that is subject to change/takes on more than one value
what is the definition of measurement?
the assignment of labels to a variable or outcome
what kind of measurement constructs are tangible vs. intangible?
Tangible: MEASUREABLE. weight, time, height, etc.
ratio
Intangible: IMAGINARY CONSTRUCTS. depression, happiness, love, etc.
nominal/interval
how do you calculate percentile rank?
number of scores below any given score (divided by) total number of scores in a set (multiplied by) 100
if the mean of a Z score is 0, the T mean is…
what is the SD for T scores if Z = 0?
mean T = 50, SD = 10
given mean Z = 0 and SD = 1
what are the differences between Kuder-Richardson, Chronbach’s alpha, and Spearman Brown correlation coefficients? What information are they used for/communicate?
all three formulas are used to calculate the reliability of a given score, which tells us how random/due to chance a score is.
Kuder-Richardson: used for dichotomous questions (only either correct or incorrect)
Chronbach alpha: used for items that don’t have correct or incorrect answers (ex., answers from a likert scale)
Spearman Brown: predicts the reliability of a test if the length of the test was longer or shorter by utilizing the “split half” technique
what is the coefficient of determination? what does it tell us?
R-squared (divided by) 100 - r-squared
numerator tells us what we ‘know’/hypothesized, and the denominator tells us what we don’t know/couldn’t account for
r-squared tells us how varied the dependent variable is from the independent variable
ex., if r-squared = .75, 75% of the variation of the dependent variable is explained by the independent variable, and the remaining 25% is explained by chance/random
describe Standard Error of Measurement and how it relates to reliability.
As SEM decreases, the reliability coefficient increases
SEM tells us how much a score can vary if they took the test over and over (so the lower standard error of measurement, the more reliable the score is)
What are the core characteristics of a normal curve?
a normal curve is asymptotic (ends of the curve never touch the line, go on infinitely), and is symmetrical (mean in the middle)
34 34 (innermost), 14 14 (middle), 2 2 (tails)
what are the advantages/disadvantages of MPC and short answer questions?
MPC
advantages: scoring is easy and less subjective, most common type of question form, can be answered with all levels of bloom, utilizes (hopefully effective) distractors
disadvantages: harder to construct (requires a higher level of bloom), correct answers can still be guessed
Short Answer (fill in the blank, 1 word/1 sentence answers)
advantages: easier to create/write, less opportunities to guess a correct answer
disadvantages: can’t measure/be used for all levels of Bloom (only used for lower levels), harder to grade (more subjective)
list the different kinds of standardized tests
norm-referenced and criterion-referenced tests are both standardized.
norm-referenced (ex., SATs)
criterion-referenced (ex., driver’s test)
intelligence tests
psychological (and other diagnostic) tests
aptitude tests
achievement tests (ex., state exams like STAAR or CAASSP)
the closer a reliability coefficient is to 1, the more…
reliable it is, and the stronger the correlation is (coefficient of 1 = cause)
the closer to 0, the less reliable it is
correlation coefficients range from __ to __
-1 to 1
sign before coefficient indicates direction of relationship/direction of its slope