Psychometrics_Lecture_12_Applications_and_Issues_in_Ability_Testing

Applications and Issues in Ability Testing

• Speaker: Katya DeGiovanni

Lecture Outline

• Introduction

• Assessment in Educational Contexts

• Criticisms and Issues in Ability Testing

• Legal Matters and Educational Selection Tests

• Employment Testing and Bias

Introduction

• Ability tests are widely used in schools and institutions to measure students' knowledge and skills.

• Knowledge tested should go beyond mere recall; it should include comprehension and practical application.

• Skills should ideally be transferable across contexts.

Assessment in Educational Contexts: Evaluating Student Competency

• General Findings: Scholastic attainment is low among various ethnic groups including European American, African American, and Hispanic students.

• Historic Progress: Since the 1970s, African American and Hispanic students have made improvements in reading, math, and science; however, their performance is still behind European American peers.

• Competency Testing: Passing functional literacy or minimum competency tests is mandatory for high school graduation. There are concerns about these tests being discriminatory.

• Critiques of Minimum Competency Tests: Critics argue that an 8th-grade competency test may not be a robust measure for high school graduation and can lead to 'teaching to the test'.

Assessment in Educational Contexts: Value Added Testing

• Definition: Value-added testing evaluates life competencies.

• Methodology: Student achievements are assessed before and after educational experiences; improvements indicate educational effectiveness.

Assessment in Educational Contexts: Teachers and Testing

• Roles: Involvement of school psychologists, counselors, special education directors, and teachers in testing activities.

• Types of Assessment: Includes formal and informal evaluations based on observations, classwork, homework, and tests (including standardized tests).

• Concerns about Standardized Tests: Potential errors in administering, scoring, and interpretation, necessitating properly trained administrators.

Assessment in Educational Contexts: Teacher Training in Testing

• Current Status: Teacher training in testing methodologies is often superficial.

• Misunderstandings: Teachers may not grasp test measurements or implications of scores, leading to overreliance on single test scores without considering student context.

Assessment in Educational Contexts: Testing the Teachers

• Standardized Tests: Examples include Praxis I in the US and the English Language Competency Test in Malta.

• Legal Challenges: The Praxis Series is widely debated due to its implications for teacher certification standards.

Criticisms and Issues in Ability Testing: Nature and Consequences of Criticisms

• Confidentiality Issues: Testing criticized for violating individual privacy and confidentiality.

• Limitations and Biases: Ability tests have been deemed inadequate predictors, unfair to minorities, and misused.

• Perpetuation of Inequality: Concerns that testing preserves status quo and rigid classifications based on static characteristics.

Criticisms and Issues in Ability Testing: Nature and Consequences of Criticisms II

• Ethical Treatment: Psychological assessments require conscientious handling of results, awareness of instrument limitations, and respect for examinee rights.

• Professional Standards: High ethical standards among examiners are essential; governing bodies like APA and BPS set codes of ethics to regulate testing practices.

Criticisms and Issues in Ability Testing: College Entrance Examinations

• Time Consumption: Testing processes occupy a considerable portion of school time and measure limited variables of achievement.

• Major Tests: SAT and ACT dominant. High school grades are often weighted more heavily than other performance indicators (interviews, recommendations, etc.).

• Reliability of SAT: The SAT is recognized for its design reliability in predicting college performance.

Criticisms and Issues in Ability Testing: MCQs Issues

• Critiques of MCQs: Authors Black and Hoffman highlight that multiple-choice questions favor quick readers and may undermine deeper thinking.

• Responses to Criticism: Calls for performance-based testing approaches, including authentic assessments and open-ended questions.

Criticisms and Issues in Ability Testing: Cheating on Tests

• Preventive Measures: Secure testing involves strict administration protocols to prevent cheating.

• Causes of Cheating: High pressures from various societal factors lead to potential cheating behaviors, including answer copying and test theft.

• Indicators of Cheating: Similar answer patterns and excessive erasures on answer sheets may indicate dishonest practices.

Criticisms and Issues in Ability Testing: National Educational Standards and Tests

• Comparative Education: US students, particularly in science and math, lag behind peers in other countries.

• PISA Studies: Malta's performance in PISA studies reflects educational shortcomings compared to EU counterparts.

Legal Matters and Educational Selection Tests: Effects of Coaching on Test Scores

• Coaching Organizations: Prominent entities include College PowerPrep, Kaplan, The Princeton Review.

• Disparities in Coaching: Coaching may variably enhance test scores, leading to unfair advantages for coached students.

• Research Findings: Mixed results regarding coaching effectiveness highlight methodological issues in studies.