midterm: tests and measurement

0.0(0)
Studied by 3 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/33

encourage image

There's no tags or description

Looks like no tags are added yet.

34 Terms

1
New cards

what is Bloom’s Taxonomy, and what are its cognitive categories/domains?

Bloom’s Taxonomy labels the ‘levels’ at which we think. From least to most effortful/cognitively demanding:

  1. knowledge (remembering): memorization of basic knowledge

  2. comprehension (understanding): being able to re-state information in your own words

  3. application: being able to apply knowledge/comprehension of material to a new situation (ex., like solving a math problem by following its ‘rules’/formulas, etc)

  4. analysis: being able to break down information/concepts into their individual parts (ex., giving a presentation in class or writing a paper)

  5. synthesis (evaluating): critically thinking about content/information by putting its individual parts together to form a whole concept (ex., writing an essay about a book’s themes)

  6. evaluation (creating): encompasses all levels of learning (ex., creating an exam or experiment)


2
New cards

what types of assessment items are “supply,” and which items are “selection”?

supply: short answer, fill in the blank (completion)

selection: multiple choice, matching, T/F

3
New cards

what kind of assessment item(s) are considered “performance” based?

Essays (short, long, restrictive, extended)

4
New cards

what assessment item type(s) have higher scoring reliability, and what assessment item type(s) have lower scoring reliability? which ones are easier to create?

Objective supply and selection assessment items (T/F, multiple choice, fill in the blank, etc) are harder to create, but have higher scoring reliability. Performance assessment items (ex., short answer, essays, etc) are easier to create, but have lower scoring reliability.

5
New cards

what is validity, and how does it differ from reliability?

validity: how well a set of scores reflect whatever the test is supposed to be measuring

reliability: how random a set of scores are

reliable tests are not always valid, but a test cannot be valid unless it is reliable.

6
New cards

describe each type of validity and when/in what context they are used.

content-based validity: used when trying to establish that the items on a test should reasonably be there, or are reasonably representative of other items that should (theoretically) be on the test

criterion-based validity (concurrent vs. predictive): when trying to determine whether a test’s scores are similar to other scores from tests that measure the same thing, now (concurrent) or in the future (predictive)

construct-based validity: when trying to demonstrate that the scores on a given test have a clear relationship to what the test aims to measure

7
New cards

what is criterion contamination?

when a confounding variable(s) influences how closely related two test scores are, making the validity of the test scores questionable.

8
New cards

describe what factors/events could cause a test score to be unreliable?

when a test is too easy or too hard, trait and method errors, lack of standardization, etc

9
New cards

what is the difference between trait and method errors?

trait errors: an error that comes from the individual taking the test (ex., waking up late, not studying enough, etc)

method errors: an error that comes from the environment/site at which the test is taken (ex., technological issues, temperature of the room, missing items on the test, etc)

10
New cards

what are the different levels of measurement and how do they differ from one another?

nominal (offers least information): when numbers signify different categories (always qualitative and mutually exclusive)

ordinal: describes how variables can be ordered along some kind of continuum

interval: scores are assigned value based on a continuum with equal intervals

ratio (offers most information): physical, direct, numerical data that can have an absolute zero


all levels of measurement, except for nominal, allow for an ‘order’ of data to be established.

11
New cards

what are the five main purposes for using a test?

  1. selection

  2. placement

  3. diagnosis

  4. classification

  5. research


12
New cards

what is the difference between aptitude and intelligence tests?

aptitude tests can include intelligence tests, or function as both, but in general, an aptitude test is predictive, whereas intelligence tests on their own are not predictive, and focus on the test taker’s current capabilities.

13
New cards

who ‘pioneered’ the first ‘mental test’?

James Cattell

14
New cards

what three people were involved in the development of the Stanford-Binet test (1905)?

Alfred Binet, Theodore Simon, and Lewis Terman (who brought the test to Stanford from Europe)

15
New cards

what is the purpose of testing, and what different areas/domains do we test?

to measure an outcome (score) and make sense of that judgement


  1. achievement (level of knowledge in a particular domain)

  2. aptitude (potential to succeed)

  3. personality

  4. vocational/career interest (job-related interests… can sometimes be an aptitude test)

  5. ability/intelligence (skill/competence)

  6. neuropsychological


16
New cards

what is the difference between Item Response Theory (IRT) and Classical Test Theory (CTT)?

CTT: aims to increase the accuracy of the measure of a test taker’s ‘true score’

  • observed score = true score + error score

IRT: the idea that the likelihood of a correct response on a test is a complex function of the person taking the test and the characteristics of each individual item on the test


CTT focuses on the test as a whole, and IRT focuses on a test’s individual items

17
New cards

what three scores comprise an overall ‘test score’?

true score: the score of a test if taken an infinite number of times

observed score: the original score obtained on a test

error score: trait error and method error (the smaller the error score, the greater the reliability of the test score)

18
New cards

what is the 12 step process of test development?

  1. choosing and defining constructs to test

  2. determine best testing method (paper-pencil, online, observation, etc)

  3. item development

  4. pilot test the preliminary items

  5. use data from pilot test to edit, revise, or make new items for the exam

  6. pilot test again for the adjusted or new items

  7. final configuration/grouping of items based off data collected from pilot tests

  8. develop directions + guidelines for the test

  9. conduct final reliability/validity tests for the exam

  10. make revisions to test based on data collected from reliability/validity tests of the exam

  11. conduct tests to find typical performance levels (norm) for a given population

  12. develop test manual


19
New cards

how much does testing cost in the U.S.?

testing in U.S. schools costs $1.7 billion annually

20
New cards

describe the early history of intelligence testing in the U.S.

  • began in China ~ 3,000 years ago

  • Darwin’s Origin of Species insinuated that traits/attributes are passed down generationally as a result of adaptation

    • increased interest in differences between individuals

  • First mental test - James Cattell

    • founded Psychological corp. (1920)

  • Testing boom in 20th century

    • Stanford-Binet test

    • intelligence testing before WWII (verbal and non-verbal)

    • testing spreading into civilian life post-war (schools, businesses, hospitals)

    • emergence of personality testing

    • testing in education (SAT, GRE)


21
New cards

what is the definition of a test?

a measure of an outcome

22
New cards

what is the definition of a variable?

a value that is subject to change/takes on more than one value

23
New cards

what is the definition of measurement?

the assignment of labels to a variable or outcome 

24
New cards

what kind of measurement constructs are tangible vs. intangible?

Tangible: MEASUREABLE. weight, time, height, etc.

  • ratio

Intangible: IMAGINARY CONSTRUCTS. depression, happiness, love, etc.

  • nominal/interval


25
New cards

how do you calculate percentile rank?

number of scores below any given score (divided by) total number of scores in a set (multiplied by) 100

26
New cards

if the mean of a Z score is 0, the T mean is…

what is the SD for T scores if Z = 0?

mean T = 50, SD = 10

given mean Z = 0 and SD = 1

27
New cards

what are the differences between Kuder-Richardson, Chronbach’s alpha, and Spearman Brown correlation coefficients? What information are they used for/communicate?

all three formulas are used to calculate the reliability of a given score, which tells us how random/due to chance a score is.


Kuder-Richardson: used for dichotomous questions (only either correct or incorrect)

Chronbach alpha: used for items that don’t have correct or incorrect answers (ex., answers from a likert scale)

Spearman Brown: predicts the reliability of a test if the length of the test was longer or shorter by utilizing the “split half” technique

28
New cards

what is the coefficient of determination? what does it tell us?

R-squared (divided by) 100 - r-squared

numerator tells us what we ‘know’/hypothesized, and the denominator tells us what we don’t know/couldn’t account for

  • r-squared tells us how varied the dependent variable is from the independent variable

    • ex., if r-squared = .75, 75% of the variation of the dependent variable is explained by the independent variable, and the remaining 25% is explained by chance/random


29
New cards

describe Standard Error of Measurement and how it relates to reliability.

As SEM decreases, the reliability coefficient increases

  • SEM tells us how much a score can vary if they took the test over and over (so the lower standard error of measurement, the more reliable the score is)


30
New cards

What are the core characteristics of a normal curve?

a normal curve is asymptotic (ends of the curve never touch the line, go on infinitely), and is symmetrical (mean in the middle)


34 34 (innermost), 14 14 (middle), 2 2 (tails)

31
New cards

what are the advantages/disadvantages of MPC and short answer questions?

MPC

  • advantages: scoring is easy and less subjective, most common type of question form, can be answered with all levels of bloom, utilizes (hopefully effective) distractors

  • disadvantages: harder to construct (requires a higher level of bloom), correct answers can still be guessed

Short Answer (fill in the blank, 1 word/1 sentence answers)

  • advantages: easier to create/write, less opportunities to guess a correct answer

  • disadvantages: can’t measure/be used for all levels of Bloom (only used for lower levels), harder to grade (more subjective)


32
New cards

list the different kinds of standardized tests

norm-referenced and criterion-referenced tests are both standardized.

  • norm-referenced (ex., SATs)

  • criterion-referenced (ex., driver’s test)

  • intelligence tests

  • psychological (and other diagnostic) tests

  • aptitude tests

  • achievement tests (ex., state exams like STAAR or CAASSP)


33
New cards

the closer a reliability coefficient is to 1, the more…

reliable it is, and the stronger the correlation is (coefficient of 1 = cause)

  • the closer to 0, the less reliable it is


34
New cards

correlation coefficients range from __ to __

-1 to 1

  • sign before coefficient indicates direction of relationship/direction of its slope