KIN 3502 - Norm-Referenced Reliability and Validity

what does reliability and validity refer to?

to the RESULT of the test, NOT refer to the TEST

true or false: The results obtained from the three-minute step test are valid and reliable

true

true or false: The three-minute step test is a valid and reliable test

false

what is reliability?

The degree to which repeated measurements of the same trait are reproducible under the same conditions; Stability, dependability, and precision (Consistently unchanged over a short period of time)

what are these examples of: weight, motor skills

reliability

what factors affect reliability?

performer, test, testing situation, measurement process, evaluator(s), and mode of statistical estimation

how does the evaluator affect reliability?

competencies, confidence in test, concentration on task, familiarity with instrument, motivation, and number of evaluators

how does statistical estimation affect reliability?

error, retest design, individual variability, and consistency among groups

how does th eprformer affect reliability?

ability range, motivation, mood, learning, forgetting, fatigue, mental/physical condition ,practice, and guessing

how does the test affect reliability?

length, difficulty, discriminative power, trial homogeneity, number of performers, and appropriate level of difficulty

how does the testing situation affect reliability?

instructions, organization, standardized procedures, timing, warm-up, time between testing, environmental conditions

how does the measurement process affect reliability?

scoring unit precision, errors in measure, number of trials, and recording errors

what are the main types of reliability?

test-retest, inter-rater, and intra-rater reliability

what is the test-retest reliability?

Measure of reliability for a specific measure when there are at least two data points; Ideal time between assessments is minimum two week (avoid a learning effect)

what is the inter-rater reliability?

“objectively”; measure of agreement between two or more raters

what is the purpose of objectivity?

ensures data is collected without bias; Depends on clarity of scoring system and degree to which judge can score accurately

differences between objective and subjective tests

what is the intra-rater reliability?

Measure of stability for one rater’s scores when conducted on multiple assessments

what is a common characteristic when reporting reliability?

Most commonly reported as a correlation (r) or intraclass correlation (ICC) and the score is closer to 1.0 is best

interpret reliablity scoring: 0.0 to 0.3

low

interpret reliability scoring: > 0.3 and <0.7

medium

interpret reliablity scoring: > 0.7

high

what is validity?

Degree to which a test measures what it is meant to measure; degree of soundness or truthfulness

true or false: Validity is predicated on reliability

YOU CAN’T HAVE A VALID RESULTS, IF THEY ARE NOT RELIABLE

true

true or false: THE TEST ITSELF IS NOT VALID, THE INTERPRETATIONS OF SCORES ARE VALID!

true

what is content validity?

Evidence of truthfulness based on logical decision making and interpretation.

AKA Face or Logical Validity

what is criterion validity?

Evidence that a test has a statistical relationship with the trait being measured

Also known as statistical or correlational validity

what are these examples of: Skinfold vs Bodyfat, Distance Run vs V02 Max, Steps in a Day vs Minutes of MVPA

criterion validity

what is concurrent validity?

Validity established by measuring the same construct using two different measures at the same time

what is predictive validity?

Validity established by examining the relationship between results at one point and results measured at some point in the future

what is the difference between concurrent and predictive validity?

concurrent measures taken at the same or near the same time while predictive measure at the Time 1 predict future measure of a trait at Time 2

what is this an example of: If we measure throwing capability, we can assess the form of the throw and the speed the ball was thrown at the same time

concurrent validity

what is this an example of: Does childhood motor skill competence predict adult physical activity?

predictive validity

what is this an example of: Does one’s score on the SAT/ACT predict success in university courses?

predictive validity

what is construct validity?

Highest form of validity; combines logic with statistical evidence of validity to determine the existence of the theoretical construct being measured; typically unobservable (e.g., intelligence, attitudes)

what does construct validity require?

hypothesis

what are the subcategories of construct validity?

convergent and discriminate validity

what is convergent validity?

Evidence to show that variables theoretically are correlated with one another

what is discriminate validity?

Variables that theoretically should have no correlation (i.e., they are different measures)