Studied by 0 people
Reliability
The consistency of test scores; it indicates that scores can be expected to be stable over time
Validity
a test measures what it was designed to measure, evaluating its accuracy
Content Validity
test adequately represents the content or behavior domain it aims to measure.
How is content validity established
during test development through expert review
Face Validity
how valid a test appears to be to test-takers based on its content
not a formal type of validity, it can affect examinees' willingness to try their best
Face validity
Construct Validity
a test accurately measures a hypothetical construct
Construct Validity procedures
through procedures such as convergent and divergent validity.
Convergent Validity
The degree to which scores on a test correlate highly with scores on other tests that measure the same constructs
Divergent Validity (Discriminant Validity) correlation
The degree to which scores on a test do not correlate with scores on tests of unrelated constructs.
Multitrait-Multimethod Matrix
A table of correlation coefficients that helps assess a test’s reliability, convergent validity, and divergent validity.
Monotrait-Monomethod Coefficient
The reliability coefficient of the test being validated when using the same trait and method (e.g., coefficient alpha).
Monotrait-Heteromethod Coefficient
The correlation coefficient between two tests measuring the same trait but using different methods, showing evidence of convergent validity.
Heterotrait-Monomethod Coefficient
The correlation coefficient between a test measuring one trait and another test measuring a different trait using the same method.
what does a Small Coefficient indicate when using the Heterotrait-Monomethod
indicates divergent validity
Heterotrait-Heteromethod Coefficient
The correlation coefficient between tests measuring different traits using different methods.
Heterotrait-Heteromethod Small Coefficient
supports divergent validity
Factor Analysis
used to assess a test’s convergent and divergent validity by analyzing the relationships between a set of variables
Rotated Factor Matrix
The result of factor analysis, where factor loadings are adjusted to make the data easier to interpret.
What does a Rotated Factor Matrix indicate with it’s findings?
correlation between each test and the identified factors
Factor Loadings
The correlation coefficients in the rotated factor matrix, showing the relationship between each test and each identified factor.
Communality
The amount of variability in a test that is explained by the identified factors in a factor analysis.
Orthogonal Factors
Factors that are uncorrelated
What does Orthogonal Factors variability indicate?
their variability can be independently measured in factor analysis
Oblique Factors
Factors that are correlated with each other in factor analysis
Oblique Factors interpretations
their interpretation is more complex.
Factor Analysis: Step 1
Administer the test being validated alongside tests of the same or related traits and unrelated traits
Factor Analysis: Step 2
Correlate all pairs of scores and create a correlation matrix.Factor Analysis
Factor Analysis: Step 3
Use the correlation matrix to derive the initial factor matrix.
Factor Analysis: Step 4
Rotate the initial factor matrix and interpret the resulting data to name the factors.
Factor analysis supports convergent validity
the test correlates strongly with related traits
Factor analysis supports divergent validity
does not correlate with unrelated traits
Reliability is necessary for validity
A test can be consistent (reliable) but still fail to measure the right thing (not valid).
What type of validity comes first during test development?
Content validity
convergent and divergent validity provide evidence that…
the test is accurately measuring the intended construct
What type of validity does monotrait-heteromethod assess?
convergent validity
What type of validity does heterotrait-monomethod assess?
divergent validity
What does the weakest correlation support in heterotrait-heteromethod?
divergent validity
What does factor analysis identify?
factors (underlying traits) that explain variability in test scores
What are correlations between tests and factors?
Factor loadings
What are factor loadings used for?
show a test strongly relates to a specific factor
what does factor analysis provide evidence for?
construct validity
What does strong convergent validity mean?
a test loads highly onto it’s intended factors
What does strong divergent validity mean?
a test does not load highly onto unrelated factors
Note 
Flashcard (32)