Assessment of Learning: Validity, Reliability, Item Analysis, Central Tendency, and Variability

Comprehensive review flashcards covering measurement validity, reliability methods, item analysis criteria (difficulty and discrimination), principles of assessment, norm- vs. criterion-referenced testing, measures of central tendency, and measures of variability and skewness.

What condition demonstrates convergent validity in a measurement procedure?

Convergent validity is established when there is a strong relationship between the scores from two different measurement procedures measuring the same construct.

How does divergent validity support construct validity?

It demonstrates that the construct being assessed is different from other constructs present in the study by showing a weak relationship between them.

How is test reliability defined in educational assessment?

Reliability refers to the consistency of scores obtained by the same person when retested using the same instrument, its parallel form, or when compared with other students taking the same test.

What type of reliability measure does the test-retest method provide, and what is its procedure?

It provides a measure of stability by administering the same test twice to the same group with a time interval ranging from several minutes to several years.

What does the equivalent forms reliability method measure, and how is it conducted?

It provides a measure of equivalence by administering parallel forms of a test with a shorter or close time interval between the two forms.

What does the test-retest with equivalent forms method measure, and how does its administration differ from standard equivalent forms?

It measures both stability and equivalence by administering parallel forms with a longer or increased time interval between tests.

How is split-half reliability measured, and what does it assess?

It is a measure of internal consistency determined by giving a test once and scoring equivalent halves of the test, such as odd-numbered versus even-numbered items.

What is the procedure for assessing internal consistency using the Kuder-Richardson method?

The test is administered once, and the proportion or percentage of students passing and not passing each given item is correlated.

What is item difficulty and what formula is used to calculate it?

Item difficulty is the proportion of examinees who answer an item correctly, calculated as $\text{Item Difficulty} = \frac{\text{number of students with correct answer}}{\text{total number of students}}$.

What difficulty levels and actions correspond to item difficulty index ranges of $0\text{ to }0.25$, $0.26\text{ to }0.75$, and $0.76\text{ to }1.0$?

A range of $0\text{ to }0.25$ indicates a difficult item (revise or reject/discard); $0.26\text{ to }0.75$ indicates right difficulty (retain); and $0.76\text{ to }1.0$ indicates an easy item (revise or reject).

What does the discrimination index measure in item analysis?

It measures how well a test item distinguishes between students with high overall test scores and those with low overall test scores.

What is the formula for computing an item's discrimination index using upper and lower performing groups?

$\text{Discrimination Index} = D_U - D_L$, where $D_U$ is the difficulty index of the upper group and $D_L$ is the difficulty index of the lower group (typically the upper and lower $27\%$).

What is the difference between positive, negative, and zero discrimination?

Positive discrimination occurs when the proportion of students answering correctly in the upper group is greater than the lower group; negative discrimination occurs when the lower group proportion is greater; zero discrimination occurs when the proportions are equal.

What are the action recommendations for discrimination index ranges of $-1.0\text{ to }-0.56$, $-0.55\text{ to }0.45$, and $0.46\text{ to }1.0$?

Indices from $-1.0\text{ to }-0.56$ are questionable and should be rejected/discarded; $-0.55\text{ to }0.45$ are non-discriminating and should be revised; and $0.46\text{ to }1.0$ are discriminating items and should be retained.

Under what combined criteria of difficulty index and discrimination index should a test item be retained?

An item is retained when its difficulty index falls within $0.26\text{ to }0.75$ and its discrimination index is $0.20$ and above.

Under what combined criteria of difficulty index and discrimination index must a test item be revised?

An item must be revised if its difficulty index is within $0.26\text{ to }0.75$ but its discrimination index is $0.19$ and below, or if its difficulty index is NOT within $0.26\text{ to }0.75$ while its discrimination index is $0.20$ and above.

Under what combined criteria of difficulty index and discrimination index must a test item be discarded or rejected?

An item is discarded or rejected when its difficulty index is NOT within $0.26\text{ to }0.75$ and its discrimination index is $0.19$ and below.

What do the assessment principles of administrability and scoreability mean?

Administrability means the test is easy to administer with clear directions regarding how to respond and how much time to spend; scoreability means the test is easy to score with clear scoring directions and specified point values.

What do the assessment principles of economy and interpretability mean?

Economy means tests are administered at minimal cost in time and effort, often using separate answer sheets for reusability; interpretability means scores can easily be described by specific tasks mastered or by relative rank in a defined group.

How do criterion-referenced and norm-referenced assessments differ?

Criterion-referenced assessment compares student performance directly against a predetermined standard without considering other students; norm-referenced assessment compares performance against a norm group, typically reporting scores as percentile ranks.

What is the mean, and what is its primary vulnerability in a score distribution?

The mean is the average or arithmetic mean representing the center of the data; its main vulnerability is that it is easily affected by extreme scores (outliers).

Under what specific conditions should the mean be chosen as the measure of central tendency?

The mean should be used when each score needs equal weight, when the measure with the highest reliability is desired, and when computing standard deviation and correlation coefficients later.

What is the median, and when is it preferred over the mean?

The median is the centermost score when values are arranged by magnitude; it is preferred when a quick computation is desired or when extreme scores (badly skewed distributions) would distort the mean.

What is the mode, and what are its classifications based on the number of modes?

The mode is the score occurring most frequently in a distribution; it is classified as unimodal (one mode), bimodal (two modes), trimodal (three modes), or multi-modal (more than two modes).

What do small variability and large variability signify about a distribution of scores?

Small variability indicates scores are clustered, closer together, and homogeneous; large variability indicates scores are dispersed, scattered, spread apart, and heterogeneous.

What is the range in a distribution, and what formula defines it?

The range is a rough estimation of variation representing the difference between the highest score and the lowest score: $R = \text{HS} - \text{LS}$.

What does quartile deviation ($\text{QD}$) measure in a set of scores?

Quartile deviation refers to the average deviation of the third quartile ($Q_3$) and the first quartile ($Q_1$) from the median, considering only the middle $50\%$ of scores to reduce the influence of extreme scores.

What is the standard deviation ($\text{SD}$), and why is it preferred in research?

Standard deviation measures the variation of individual scores about the mean; it is the most important and commonly used measure of dispersion, particularly in research.

What defines a positively skewed score distribution?

A positively skewed distribution is skewed to the right, meaning the thin end or tail of the curve extends toward the right side.