Session 5: Selecting, Administering, Scoring, and Interpreting Assessments
Selecting Assessments and Interpreting Results
Assessment selection steps:
Identify info type needed.
Identify available info.
Determine info-gathering methods.
Search assessment resources.
Evaluate & select instrument/strategy.
Identify the Type of Information
Instrument must align with info needed.
Counselors determine info based on assessment purpose:
Screening
Identification/diagnosis
Intervention planning
Progress monitoring
Outcome evaluation
Identify Available Information
Use existing client info to prevent over-testing/redundancy.
Create information checklist.
Cumulative Folder of Student (Figure 7.1)
Sample school client info checklist includes:
Personal: Birth date, Family info, Attendance
Academic: Report cards, Academic records (other schools), State/achievement test results, College entrance exam results, Grades, Reading progress/level, Competencies, Work samples
Health: Health data/records
Behavioral: Disciplinary actions, FBA, BIP, IEP, Behavior problems, Anecdotal observations
Other: Parent conference notes, School counselor records, Interest data, Career/educational goals, School/non-school activities, Parent/guardian records, Transcripts/assignments
Determine the Methods for Obtaining Information
Assessors choose methods:
Interviews
Tests
Observation
Combine methods for in-depth evaluation.
Match methods to client/setting.
Ensure assessor is qualified.
Search Assessment Resources
Thousands of instruments available from:
Reference resources (e.g., MMY)
Publishers’ websites/catalogs
Instrument manuals
Research literature
Internet resources
Professional organizations
Directories of unpublished instruments
Reference Resources and Publisher Tools
Mental Measurement Yearbook (MMY):
Founded 1938 by Oscar K. Buros.
Evaluative info for test selection.
Describes >2500 tests (prices, publishers, population, psychometrics, expert reviews).
Available hardcopy, electronically, www.buros.org.
Test in Print (TIP):
Comprehensive listing of commercial English tests.
Basic info, no reviews (refers to MMY).
Tests (Pro-Ed, Inc):
Comprehensive listing of English tests.
Critiques: intro, practical application, technical aspects; includes psychometrics & reviews.
Publishers’ Websites and Catalogs; Manuals
Publisher sites: Provide cost, scoring, ancillary materials; often marketing, may lack full info.
Counselors can order specimen sets (manual, booklet, answer sheet, key).
Manuals: Admin and technical info:
Instrument nature, recommended uses, development.
Norming, validity, reliability.
Administration/scoring, interpretation procedures.
Checklist for Evaluating a Test Manual (Figure 7.2)
Key criteria (Yes/No):
Manual: available, complete, accurate, clear, updated.
Rationale/uses discussed; cautions against misuses.
Norming population described.
Reliability/validity evidence.
Special user qualifications.
Research bibliography.
Admin conditions/modes explained.
Interpretive aids for test takers; easy to understand.
Evidence for computer interpretations; automated service available.
Rationale for cutoff scores; clear concept organization.
Technical info on diverse groups (age, grade, language, culture, gender).
Linguistic modification descriptions.
Research Literature, Internet Resources, and Professional Organizations
Research literature: Journal articles, test reviews, studies.
Internet resources/professional organizations: Current info, discussions (e.g., www.testpublishers.org).
Evaluate and Select an Assessment Instrument or Strategy
Test manual is primary source for formal assessment evaluation.
Guiding questions:
Purpose, intended population?
Norm group makeup?
Reliable results?
Validity evidence?
Clear admin instructions?
Additional considerations:
Bias?
Examiner competency needed?
Practical: time, ease of admin/scoring/interpretation, format, readability, cost.
Instrument Evaluation Form (Figure 7.3)
Typical fields:
Title, Publisher, Authors, Pub Date.
Examiner quals, Description, Type, Population, Purpose, Scores, Item types.
Technical Evaluation: Reliability, Validity.
Norm Group Details: Composition, Size, Appropriateness.
Practical Features: Admin procedures/time, Scoring/interpretation, Manual adequacy, Cost.
Previous Reviews: MMY, other.
Evaluation: Strengths, Weaknesses, Recommendation.
Administering Assessment Instruments
Various modes, each with pros/cons:
Self-administered
Individually administered
Group administered
Computer administered
Video/audio administration
American Sign Language (ASL)
Nonverbal approaches
Modes of Administering Assessment Instruments (Table 7.5)
Self-Administered:
Desc: Examinees read instructions, take test alone.
Pros: No examiner, efficient for large groups.
Cons: Motivation unknown, potential confusion.
Individually Administered:
Desc: Examiner to one individual.
Pros: Assess motivation/attitudes/thought processes, most flexible.
Cons: Expensive, time-consuming, requires trained examiner.
Group Administered:
Desc: Simultaneously to many.
Pros: Cost-effective, efficient for large groups.
Cons: Less control over individual differences, variable motivation.
Computer Administered:
Desc: Directions/items on computer (video/audio possible).
Pros: Immediate scoring/analysis, scalable, adaptive tests save time.
Cons: Requires computer access, may disadvantage those with disabilities or limited computer familiarity.
Video/Audio Administration: Directions/items via video/audio.
ASL: Directions/items in ASL for hearing-impaired; examiner must be proficient.
Nonverbal: For certain disabilities; relies on nonverbal cues; requires trained administrators.
Administering Instruments: Overview of the Process
Admin activities: pre-, during, post-administration.
Before Administration
Tester's main responsibility: thorough instrument knowledge.
Review manual, forms, materials.
Familiarize with content, item types, directions.
Practice taking the test.
Pre-Administration Management Tasks (especially for group testing)
Secure materials (tests, answer sheets).
Schedule date, rooms.
Count materials accurately.
Arrange materials for distribution.
Obtain informed consent (nature, purpose, confidentiality limits, test security).
Physical Environment (Lingkungan Fisik)
Quiet, interruption-free room.
Clear speech from examiner; managed microphone.
Max 30 test-takers/room.
Even, glare-free lighting; adequate ventilation, appropriate temperature/humidity.
Room Setup (Susunan Ruangan)
Podium slightly higher.
Table (approx. 60 \times 80\text{ cm}) with classroom chairs.
Seating: smooth distribution/collection, approx. 60\text{ cm} between workstations (independent work).
Smooth, flat working surfaces for booklets/sheets.
Assistants and Orientation
Assistants must be trained (overview, practice).
Examiners can provide orientation sessions covering:
Instrument purpose & selection criteria.
Conditions for taking.
Skills/domains measured.
Administrative procedures/concerns.
Question types & overview.
Scoring type, method, reporting schedule.
During Administration
Tester performs final check.
Deliver verbatim instructions, precise sequence/timing.
Establish rapport: assess emotional/motivational factors (individual); ensure valid portrayal (group).
Examinee Conditions and Mental Preparation
Psychological: Treat with dignity/equality; respectful discipline.
Physical: Avoid testing when fatigued; schedule breaks.
Mental preparation: Explain purpose, usefulness, importance of optimal performance; reduce anxiety, encourage effort/focus; avoid rushing/insincere practices.
Giving Instructions and Managing the Administration
Avoid continuous reading of instructions; read aloud only as needed.
Speak clearly, even pace.
Monitor client for unique issues, document incidents.
Some instruments include observational forms/rating scales.
Observational Scale Sample (Figure 7.5)
Sample for test administration covers:
Response time (1-5: Slow to Quick)
Activity level (1-5: Passive to Active)
Security/Anxiety (1-5: Ill at ease to Calm/collected)
Relationship to examiner (1-5: Poor to Good)
Need for reinforcement (1-5)
Task orientation (1-5: Gives up to Sticks to task)
Reaction to failure (1-5: Poor to Good)
Interest in test (1-5: Low to High)
Problems Encountered in Test Administration (Table 7.6) and Solutions
Cheating: Create uncheatable environment (spacing).
Manual not addressing question: Use experience/judgment.
Guessing: Encourage item-focused work; allow if no penalty.
Lack of effort: Be positive, professional; use motivational strategies.
Questions during testing: Explain no questions during; circulate for clarity.
Distractions: Eliminate; apologize, provide extra time.
Refusal to answer: Repeat, check understanding; explore ability.
Examiner praise/encouragement: Use positive reinforcement appropriately; build rapport, reduce anxiety without being artificial.
Examiner effects: Acknowledge biases; remain positive, objective; listen, observe nonverbals.
General note: Maintain professionalism, ethical conduct.
After Administration
Group tests: collect/count materials, secure.
Record events affecting score validity.
Ensure scoring integrity.
Scoring Assessment Instruments
Types of scores: Percentiles, T Scores, Deviation IQ, Age/grade equivalents, Stanines, other standardized.
Hand-scoring: Qualified scorer needed; risks errors, judgment calls.
Computer scoring: Enter/scan data; advantages: ease, speed, reduced training, fewer errors.
Scoring Details and Scoring Rules
Raw Score: Basic count of correct responses; often converted to Weighted Score.
Accurate scoring crucial for validity/reliability.
Scorers must be trained, follow policy.
Manuals provide instructions/keys for integrity/confidentiality.
Scoring keys: Indicate correct/incorrect responses; protect confidentiality.
Scoring Performance Assessments
Projects, portfolios lack right/wrong answers.
Scored using rubrics: predefined criteria.
Rubrics include:
Dimensions/attributes (e.g., essay content, format).
Descriptors/examples for each.
Rating scale per dimension.
Scoring Errors and Quality Control
Common errors: incorrect assigning, recording, converting raw scores; arithmetic mistakes.
Computer scoring reduces errors, but incorrect data entry is a risk.
Interpreting Assessment Results
Interpret scores: norm- or criterion-referenced.
Norm-referenced: Compare individual to reference group.
Criterion-referenced: Compare performance against defined standard.
Additional Interpretive Models
Inter-individual (normative): Differences across test-takers on same construct.
Intra-individual (ipsative): Compare a test-taker’s scores on various scales within same test (e.g., WAIS).
Indonesian Section: Pelaksanaan Pemeriksaan Psikologi
Examinations by students under supervisor/psychologist (responsible for tools/procedures).
Examiner & supervisor roles interchangeable; cooperate, be fully responsible.
Tugas Pemeriksa (Examiner Duties)
Receive, count, return materials.
Arrange seating, assign exam numbers.
Lead exam, give instructions.
Cooperate with supervisor.
Responsible for complete results.
Tugas Pengawas (Supervisor Duties)
Assist receiving, counting, returning materials.
Assist seating, assigning exam numbers.
Cooperate with examiner; maintain order, observe class.
Follow examiner's instructions; assist individual explanations.
Distribute/collect materials.
Check materials, prepare others.
Help optimize results.
Perilaku Profesional: Pemeriksa dan Pengawas
Appearance: reasonable, neat, polite.
No smoking in room.
Must not leave room unless strictly necessary (one must remain).
Conduct professionally as a psychologist during exam.