Psychological Assessment

Psychological testing

process of measuring psychology-related variables by means of devices or procedures designed to obtain a sample of behavior

Psychological Assessment

gathering and integration of psychology-related data for the purpose of making psychological evaluation

Parts of a psychological test

1. Content

2. Form

3. Item

4. Admin procedure

5. Scoes

6. Scoring

7. Cut score

8. Psychometric soundness

1. Subject matter

2. form, plan, structure, arrangement, layout

3. a specific stimulus to which a person responds overtly and this response is being scored or evaluated

4. one-to-one basis or group administration

5. code or summary of statement, usually but not necessarily numerical in nature, but reflects an evaluation of performance on a tes

6. The process of assigning scores to performances

7. reference point derived by judgement and used to divide a set of data into two or more classification

8. technical quality

Ability or Maximal ability test

assess what a person can do (Overall capacity)

Types of ability or maximal ability test

1. Achievement

2. Aptitude

3. Intelligence

1. measurement of the previous learning. Specific time period, rely mostly on content validity. Fact or conceptual based

2. refers to the potential for learning or acquiring a specific skill. Uses predictive validity.

3. refers to a person’s general potential to solve problems, adapt to changing environments, abstract thinking, and profit from experience

Human ability

considerable overlap of achievement, aptitude, and intelligence test

Typical performance test

measure usual or habitual thoughts, feelings, and behavior - indicate how test takers think and act on a daily basis

Attitude test

Tests that elicit personal beliefs and opinions

1. the interest is the number of times a test taker can answer correctly in a specific period

2. reflects the level of difficulty of items the test takers answer correctly

1. Speed

2. Power

1. A tests-taker’s score is compared to the previous batch of takers

2. A test-taker must have a total score of 75% to pass

1. Norm reference

2. criterion reference

SORC Model of behavioral observation

Stimulus, organism (internal psycho and physio response), response (behavior), Consequence

Psychological traits

any distinguishable, relatively enduring way in which one individual varies from another

Psychological states

distinguish one person from another but are relatively less enduring. Specific moment in time

Cummulative scoring

assumption that the more the testtaker responds in a particular direction keyed by the test manual as correct or consistent with a particular trait, the higher that testtaker is presumed to be on the targeted ability or trait

Assumptions about Psychological Testing and Assessment

1. existence of traits and states

2. they can be measured

3. Test-Rlated Behavior Predicts NonTest-Related Behavior

4. Tests have strengths and weaknesses

5. Test errors are part of the assessment

6. Testing and Assessment can be conducted in a Fair and Unbiased Manner

7. Beneficial to society

Reliability

dependability or consistency of the instrument or scores obtained by the same person when re-examined with the same test on different occasions, or with different sets of equivalent items. More number = more _____________

Classical test theory or true score theory

score on a ability tests is presumed to reflect not only the testtaker’s true score on the ability being measured but also the error

Error

refers to the component of the observed test score that does not have to do with the testtaker’s ability

X = T + E

Formula for true test score or classical test theory

True score

When you average all the observed scores obtained over a period of time, then the result would be closest to the _____________

Random error

source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in measurement process. Can go either way.

Systematic error

Source of error in a measuring a variable that is typically constant or proportionate to what is presumed to be the true values of the variable being. Always biased towards a direction.

Test-retest

Inter-rater

Inter-item (Internal consistency)

Parallel forms reliability

Split-half reliability

Types of reliability

Test-retest

an estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the test

1. Carryover effect

2. Practice effect.

Errors of time sampling reliability

1. occur when the lingering influence of a previous experience or treatment alters a participant's response in subsequent conditions. It could be a positive or negative effect from fatigue or item familiarity

2. Scores on the second session are higher due to their experience in the first session of testing.

Statistical tool used for coefficient of equivalence. (test-retest, parallel forms, split-half)

Pearson R, Spearman Rho

Parallel Forms

established when at least two different versions of the test yield almost the same scores. Same items, different positioning and numbering. Means and variances must be similar. Can be administered in different times.

Internal consistency/Interitem

measures the internal consistency of the test which is the degree to which each item measures the same construct. Usually used for unstable traits (personality)

Statistical measures for inter-item reliability

1. Cronbach’s alpha

2. KR20

3. KR21

1. Internal consistency reliability for tests with right/wrong (0–1) items. With item difficulty and more precise

2. Internal consistency reliability for tests with right/wrong (0–1) items. Without item difficulty

1. KR-20

2. KR-21

Split-half

obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered ONCE. Randomly assign numbers, not merely dividing it.

1. Spearman brown.

2. Rulon’s Formula

1. allows a test developer of user to estimate internal consistency reliability from a correlation of two halves of a test, if each half had been the length of the whole test and had equal variances. predict how the reliability of a test will change if its length is increased or decreased.

2. Also estimates split-half reliability, but instead of correlation, it uses the difference in scores between halves. Uses variance. “How much do people’s scores differ between the two halves?”

Inter-scorer/rater reliability

the degree of agreement or consistency between two or more scorers with regard to a particular measure

1. Fleiss’s Kappa

2. Cohen’s Kappa

3. Krippendoff

4. Kendall’s W

Interrater Reliability

1. determine the level between TWO or MORE raters when the method of assessment is measured on CATEGORICAL SCALE

2. determine the level between TWO raters when the method of assessment is measured on CATEGORICAL SCALE

3. two or more rater, based on observed disagreement, corrected for disagreement expected by chance.Interval Data

4. Specifically for ordinal raters

Domain sampling theory

a psychometric framework that conceptualizes any psychological or educational test as a subset of items randomly drawn from an infinitely large "domain" of potential items.

Theory that suggests everyone has a “true score” on test

Classical test theory

Domain sampling theory

If you want to measure math ability, you cannot test everything in math.

So you:

• pick a limited number of questions

• assume they represent the whole “math domain”

This represents?

1. Generalizability theory

2. Universe

3. Facets

4. Generalizability theory

1. In domain sampling theory, this refers to the idea that a person’s test scores vary from testing to testing because of the variables in the testing situations. Your score changes because of different testing conditions, not just your ability.

2. This refers to every possible testing situation

3. number of items in the test, amount of review, and the purpose of test administration. Things that can change your score. (error)

4. According to ____________ given the exact same conditions of all the facets in the universe, the exact same test score should be obtained (Universe score)

5. the exact same conditions of all the facets in the universe, the exact same test score should be obtained (true score)

1. Item-response theory

2. Descrimination

3. attribute of not being easily accomplished, solved, or comprehended

1. Test making theory that suggests the probability that a person with X ability will be able to perform at a level of Y in a test. Ex. an intelligent person, person with lower anxiety levels, etc. (ex. COMPUTERIZED ADAPTIVE TESTING - humihirap kada tama)

2. degree to which an item differentiates among people with higher or lower levels of the trait, ability or etc.

3. Difficulty

“How many items did you get right?”

_________________ thoery asks:

“Which items did you get right, and how hard were they?”

Item-response theory

Cronbach’s alpha

1. unacceptable

2. poor

3. questionable

4. Acceptable

5. Good

6. Excellent

Cronbach’s alpha

Less than 0.5

0.51 - 0.6

0.61 - 0.7

0.71 - 0.8

0.81 - 0.9

0.91 - 1

1. Standard Error of Difference

2. Standard Error of Estimate

1. Used to determine whether the difference between two test scores is large enough to be considered meaningful and not just due to measurement error.

2. How far predicted scores usually are from actual scores.

1. Test sensitivity

2. Test specificity

1. detects true positive. The presence of the trait

2. detects true negative. The absence of the trait

Type of failure

1. Type 1

2. Type 2

Type of error

1. Predicted success but failed

2. Predicted failure but succeeded

Validity

a judgment or estimate of how well a test measures what it supposed to measure

1. Internal Validity

2. External Validity

1. degree of control among variables in the study (increased through random assignment)

2. generalizability of the research results (increased through random selection, or the sampling)

1. Face Validity

2. Content Validity

3. Criterion

4. Construct

Validity

1. Is the extent to which a test, survey, or research tool subjectively appears to measure what it claims to measure at first glance.

2. describes a judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample. (DOMAIN)

3. a judgment of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interestꟷthe measure of interest being the criterion

4. ability of the test to measure what it is meant to measure. Should measure a single construct that has no universal criterion. (Can include factor analysis)

Test Blueprint

a plan regarding the types of information to be covered by the items, the no. of items tapping each area of coverage, the organization of the items, and so forth

Types of criterion Validity

1. Measures how well a test relates to a criterion measured at the same time.

2. Measures how well a test predicts a future outcome.

3. A new depression inventory is given to patients, and their scores are compared with a psychiatrist’s current diagnosis.
   If the scores strongly match the diagnosis, the test has good _______ validity.

4. Focuses on future performance/behavior.

5. An entrance exam is used to predict students’ future GPA in college.
   If high scorers later perform well academically, the exam has good _______validity.

Types of criterion Validity

1. Concurrent

2. Predictive

3. Concurrent

4. Predictive

5. Predictive

CONSTRUCT VALIDITY

1. C.

2. D

3. E

4. F

5. A

6. B.

Construct ValidityIt

1. is an evaluative technique used to set passing scores (cut-offs) or establish the validity of an assessment. It works by administering a test to two groups known to have different levels of expertise (e.g., novices vs. experts) and finding the score that best separates them.

2. designed to identify factors or specific variables that are typically attributes, characteristics, or dimensions on which people may differ

3. constructs are not expected to correlate; (-) inverse

4. correlate constructs that are expected to correlate; must come from a new or not-so-well-established test

5. estimating or extracting factors; deciding how many factors must be retained

6. researchers test the degree to which a hypothetical model fits the actual data

A. Explanatory FA

B. Confirmatory FA

C. Methods of contrasting groups

D. Factor analysis

E. Divergent

F. Convergent

Rating errors

1. Rating error

2. Leniency error

3. Central tendency error

4. Severity error

5. Halo effect error

Type of rating errors

1. Misuse of the rating scale

2. too lax in rating

3. Gives middle rating

4. Too strict in rating

5. one standout positive trait or first impression dictates how a rater evaluates all other unrelated traits.One

1. Spearman-brown

2. Flannagan-Rulon

in the split-half reliability, use _________ to your test if the halves are highly parallel (similar means, variances, and correlations). Use _____________ when the test halves have unequal variances or differ in difficulty.

➔ Given: ◆ Observed score = 75 ◆ SEM = 3 ➔ For a 68% Confidence Interval:

Compute for the confidence interval

75 plus and minus (1×3) = 72 and 75

➔ Given: ◆ Observed score = 75 ◆ SEM = 3 ➔ For a 95% Confidence Interval:

Compute for the confidence interval

75 plus and minus (2×3) = 69 and 81

a range or band of the test scores that is likely to contain the true score

Confidence interval

1. Concurrent

2. Concurrent

3. Convergent

Concurrent vs Convergent Validity

1. A new depression inventory is given alongside the BDI-II. If scores are highly correlated, it demonstrates concurrent validity.

2. A new sales aptitude test is compared with employees' current supervisor performance ratings.

3. The questionnaire correlates with worry scales, physiological anxiety measures, and trait anxiety inventories.

➔ 50% of the score occur above the mean and 50% of the scores occur below the mean

➔ Approximately 34% of all scores occur between the mean and 1 standard deviation above the mean

➔ Approximately 34% of all scores occur between the mean and 1 standard deviation below the mean

➔ Approximately 68% of all scores occur between the mean and ±1 standard deviation

➔ Approximately 95% of all scores occur between the mean and ±2 standard deviations

• C (62)

• B (77.5)

• C (700)

• B (35)

• C (7.5)

• D (130)

• B (13.5)

• C (6.5)

• B (40)

• D (675)

• C (73)

• B (400)

• C (8.6)

• A (7.6)

• C (7.1)

1. A student obtained a z-score of +1.20. What is the equivalent T-score?

A. 58
B. 60
C. 62
D. 64

---

2. An examinee earned a T-score of 35. What is the equivalent IQ?

A. 72.5
B. 77.5
C. 85.0
D. 92.5

---

3. A participant has an IQ of 130. What is the equivalent GRE score?

A. 650
B. 675
C. 700
D. 725

---

4. An applicant received a GRE score of 350. What is the equivalent T-score?

A. 30
B. 35
C. 40
D. 45

---

5. A client obtained an IQ Subtest score of 13. What is the equivalent STEN score?

A. 6.5
B. 7.0
C. 7.5
D. 8.0

---

6. A student has a STANINE score of 7. What is the equivalent IQ?

A. 107.5
B. 115
C. 122.5
D. 130

---

7. A patient has a T-score of 65. What is the equivalent IQ Subtest score?

A. 13.0
B. 13.5
C. 14.0
D. 14.5

---

8. A participant earned a GRE score of 550. What is the equivalent STEN score?

A. 5.5
B. 6.0
C. 6.5
D. 7.0

---

9. An examinee has an IQ of 85. What is the equivalent T-score?

A. 35
B. 40
C. 45
D. 50

---

10. A student obtained a STEN score of 8.5. What is the equivalent GRE score?

A. 600
B. 625
C. 650
D. 675

---

11. An applicant earned a z-score of –1.80. What is the equivalent IQ?

A. 67
B. 70
C. 73
D. 76

---

12. A participant obtained an IQ Subtest score of 7. What is the equivalent GRE score?

A. 367
B. 400
C. 433
D. 467

---

13. A patient earned a T-score of 73. What is the equivalent STANINE score?

A. 7.3
B. 8.1
C. 8.6
D. 9.6

---

14. An examinee scored GRE = 420. What is the equivalent IQ Subtest score?

A. 7.6
B. 8.0
C. 8.4
D. 8.8

---

15. A student obtained an IQ of 112. What is the equivalent STEN score?

A. 6.3
B. 6.7
C. 7.1
D. 7.5