Characteristics of Assessment Methods

Chapter 9: Characteristics of Assessment Methods

Overview of Assessment Reliability Techniques

• Four main techniques to test reliability of assessment methods:
  • Test-Retest Method
  • Parallel Forms
  • Split Half
  • Internal Consistency

Test-Retest Method

• Definition: The same test administered twice to the same group of students.
• Purpose: To determine the stability of test scores over time.
• Correlation Coefficient: Determined using the Spearman rank correlation coefficient (Spearman's rho).

Advantages

• Useful for assessing the reliability of a test over two different occasions.

Disadvantages

1. Short Interval: May result in high correlation as respondents recall previous answers.
2. Long Interval: Factors like forgetting or unlearning might affect results negatively.
3. Environmental Variables: Variations in conditions (noise, temperature, lighting) can skew results.

Spearman Rank Correlation Coefficient

• Purpose: Measures relationship between paired ranks from two variables (first test administration X and second test administration Y).
• Formula: $\rho = 1 - \frac{6 \Sigma D^2}{N(N^2-1)}$ where:
  • $\rho$ = Spearman's rho
  • $\Sigma D^2$ = sum of squared rank differences
  • $N$ = number of cases

Steps to Compute Spearman’s Rho

1. Rank the scores for the first test (X) and assign as Rx.
2. Rank the scores for the second test (Y) as Ry.
3. Compute differences for each rank pair to get D.
4. Calculate D² (square of each difference).
5. Sum D² values to find $\Sigma D^2$.
6. Substitute into the Spearman rho formula.

Interpretation of Correlation Value

• Interpretation Ranges:
  • $(0.00)$: No correlation
  • $(0.01 - 0.20)$: Negligible correlation
  • $(0.21 - 0.40)$: Low correlation
  • $(0.41 - 0.70)$: Moderate correlation
  • $(0.71 - 0.90)$: High correlation
  • $(0.91 - 0.99)$: Very high correlation
  • $(1.0)$: Perfect correlation (any value above indicates potential error in computation).

Sample Problems and Solutions

• Example Problem: Fifteen students were tested.
  • Test Results:
  • Scores displayed in Table A.
  • Final Spearman's rho calculation provided a correlation of 0.99, indicating a very high reliability of scores between administrations.

Correlation with Computers

• Benefits: Makes the computation faster and reduces manual errors.
• Steps for Computer Calculation:
  1. Open Microsoft Excel.
  2. Input data as shown in sample tables.
  3. Utilize the Data Analysis tool and select correlation to compute quickly.
  4. Interpretation of values remains the same as the manual calculation.

Practice Activities

1. Manual and Computer Computation:
   • Practice using different sets of data as specified in the tables.
   • Aim to interpret results according to the expected correlation ranges.

Enrichment Exercises

• Validate additional results from specified tables using both methods of calculation.

Assessment Rubrics

• Manual Computation: Up to 5 points for accuracy.
• Computer Computation: Up to 5 points for accuracy.
• Interpretation: Up to 5 points for correct implications.
• Total Possible Score: 15 points assessing overall understanding.

Conclusion

• Understanding and applying the test-retest method is crucial for validating assessments in educational settings. With both manual and computer methods available, one can ensure reliability in academic testing.