Psychometrics - Semantic 1

Definitions, facts, general knowledge

Selection ratio

The proportion of applicants selected for a particular position or program compared to the total number of applicants. It is often used to assess the effectiveness of a selection process.

Predictive value

Indicates how well a test or assessment predicts the likelihood of a certain outcome, typically used in tests with dichotomous items.

Norm-referenced test

A type of test that assesses an individual's performance in relation to a defined group or norm, often used to compare test-taker scores to the average performance of a peer group.

Goodness of fit index

A statistical measure used to evaluate how well a model fits a set of observations, measures the proportion of variance predicted by the model.

It ranges from 0.00 - 1.00 and is ideally >0.90.

Adjusted goodness of fit index

A modified version of the goodness of fit index that decreases as the number of factors increases.

It ranges from 0.00 - 1.00 and is ideally >0.90.

Parsimony goodness of fit index

A modified version of the goodness of fit index that decreases as the degrees of freedom increases.

It ranges from 0.00 - 1.00 and is ideally >0.90.

Rank-order method

A technique of comparing the performance of two or more individuals in a way that accounts for their performance across items of varied difficulty.

Standard error of the measure

The average distance an obtained score is expected to fall from a test-taker’s true score.

Reliable change index

Used to determine if a change in an individual's score on a psychological test is significant, calculated as a ratio of their difference in pre- and post-test scores to the amount of error estimated in their obtained scores.

λ/λ_Maximum

The proportion of shared variance among variables in a factor, calculated as the factor’s Eigenvalue divided by the number of principal components (variables).

Exploratory factor analysis

A statistical technique used to identify the underlying relationships between variables by grouping them into factors. It helps to reduce data dimensionality and discover latent constructs.

Developmental norms

Standardized benchmarks used to assess the typical development of children across various domains, such as physical, cognitive, and social-emotional skills. They are measured on an interval scale.

Test bias

Difference in performance across groups not due to the true scores for either group. The proportion of variance not attributable to the true score variance between groups of scores.

Comparative fit index

A measure of goodness-of-fit for statistical models that evaluates how well the proposed model approximates the observed data compared to a “null” model (where all variables are assumed to be uncorrelated to one another).

It ranges from 0.00 - 1.00 and is ideally >0.90

r₁₂

Split-half reliability, the percent of score variance over different ordering of test items attributable to true score variance; internal consistency between two differentially-ordered halves of the same test.

Assessed using the Spearman-Brown formula.

Intercept bias

Occurs when an item characteristic curve (ICC) describes performance of two or more groups (or people) at different ability levels when their ability is known to be the same. In a linear regression model, the performance of each group intercepts the vertical axis at the same point.

Content validity

The extent to which a test adequately represents the construct it aims to measure; it ensures that all components of the construct are covered and in the same proportions expected in the construct itself.

Object

A measurable entity used to assess specific attributes or traits, often represented as a variable in testing.

λ_Maximum

The number of principal components (variables) in an exploratory factor analysis.

Empirical criterion keying

A method of selecting test items based on their ability to distinguish between groups defined by a known, pre-established, chosen criterion.

Non-uniform bias

Item characteristic curves between groups of people that differ in ability level and slope. It is considered to be an interaction between group and item.

Confirmatory factor analysis

A test of whether a proposed model (taken from an exploratory factor analysis) is preferable to alternative models with varying numbers of factors.

Test information curve

A graphical representation of the amount of information a test provides across various trait levels. The ability level at the apex of the curve (highest information) signifies the ability level that the test is most conducive to.

Within-group norms

Statistical benchmarks derived from test scores of individuals within a specific group, allowing for comparisons against the group's performance in the context of a specific trait.

Power test

In Item Response Theory, a test in which every item has the same item quality but designed for a different ability level.

In Critical Test Theory, a test in which the pass rate of each item varies and all items are meant to be completed in the time frame allotted.

Reliable clinically significant change

In a true experimental design, when a post-treatment score for an individual in the treatment group falls within one positive standard deviation of the control group mean.

Nested model

In a confirmatory factor analysis, a model that has a different number of factors (but the same number of principal components) than the proposed model created during exploratory factor analysis.

Test information

A measure in item response theory indicating the reliability and precision of the test across different ability levels, reflecting how much information the test provides about an examinee's ability. It is a summation of item information at every ability level.

Item information curve

A graph of item information versus ability level. It describes which ability level a given test item provides the most information for, and how much information it provides at all levels of ability.

Theta (IRT)

In item response theory, theta (θ) represents the latent trait or ability level of an examinee, estimated as the number of standard deviations from a mean of zero.