Validity

Introduction

Traditionally validity has been defined as the extent to which a test measures what it was designed to measure. This definition, however, implies that a test can have only one type of validity established by a single study.

A test may have different validities depending on the specific purposes for which it was designed, the target population, the conditions under which it was administered and the method for determining validity.

Validity can be determined as follows;

  1. Analysing the content of the test.

  2. Computing the correlation between scores on the test and those on the criterion of interest.

  3. Investigating the particular psychological characteristics of constructs measured by the test.

All these procedures are useful in that they improve understanding of what a test measures and provide information for making decisions about people. It may also be of interest to evaluate a test’s incremental validity which is how much the test adds to the prediction and understanding of the phenomenon at hand.

A test may be reliable without being valid. However, a test cannot be valid without being reliable. Hence, one must first check for reliability and then validity.

Content Validity

Face validity - the physical appearance of a test about its particular purposes to certainly an important consideration in marketing a test. Content validity refers to face validity and whether the content of a test elicits a range of responses that are representative of the entire domain or universe of skills, understandings and other behaviours that a test is designed to measure.

Content validity is of concern in achievement tests, measures of aptitudes, interests and personality. The content validity of an achievement test is evaluated by analysing the composition of the test to determine the extent to which it represents the objectives of instruction.

Compares the content of the test with an outline or table of specifications concerning the subject matter to be covered by the test. If the subject matter experts agree that a test looks and acts like an instrument that was designed to measure whatever it is supposed to, then it is said to possess content validity. Such a process should commence whilst the test is being constructed. For example this is testing whether a person has depression.

Criterion-Related Validity

Concurrent Validity

Test whether an individual has depression and those who don’t. These procedures are employed whenever a test is administered to people in various categories such as clinical diagnostic groups or socioeconomic levels to determine whether the test scores of people in one category are significantly different from those of people in other categories.

MMPI (a type of test) is useful for example in identifying specific mental disorders as patients who are diagnosed with particular disorders tend to make different scores on certain groups of items (Scales) than do people in general.

Predictive Validity

Concerned with how accurately test scores predict criterion scores as indicated by the correlation between the test score (predictor) and a criterion of future performance. Concerned with aptitude or intelligence test ….

Factors affecting criterion-related validity;

  • Incremental Validity: How much more accurate predictions and diagnoses are when a particular test is included in a battery of assessment procedures.

  • Group Differences: The characteristics of a group of people whom the test is validated for example sex; age and personality. These are called correlation variables because they tend to affect the correlation between the test and the correlation measures. Cross validation is when a test is administered to a second sample.

  • Test Length: Validity varies directly with the length of the test and the heterogeneity of the group of people who are tested. The more varied the sample, the longer the test, chances are that the test will have higher predictive validity.

  • Criterion Contamination: A test can be limited not only by the reliabilities of the test and criterion but also by the validity of the criterion itself as a measure of the variable of interest. Sometimes the criterion is made less valid or becomes contaminated by the particular methods of measuring the criterion scores.

Construct Validity

Construct Validation refers to the extent to which the instrument measures the particular construct of a concept such as anxiety, achievement, motivation, extroversion-introversion, etc. This validity involves a network of investigations and other procedures designed to determine whether an assessment instrument presumably measures what it’s supposed to.

Convergent and Discriminant Validation

Discriminant Validation is when a construct is validated instrument should have high correlations with other measures or methods of measuring the same construct convergent validity, but low correlations with measures of different construct.

Usually done as follows:

  • The same constructs using the same method.

  • Different constructs using the same method.

  • The same constructs using different methods.

  • Different constructs using different methods.