PSYCHOLOGICAL ASSESSMENT Prepared by Bianca Patricia Reyes, RPm

Reliability is concerned with the:

a. consistency of the scores

b. reproducibility of the scores

c. capability of the instrument to measure what it claims to measure

d. more than one is correct

D

Clemuel's wristwatch is broken; it is stuck at 2 o'clock. We could say that his clock is:

a. reliable and valid

b. reliable but not valid

c. valid but not reliable

d. not reliable and not valid

B

Nature of a Test

• Stable vs. Dynamic

- If the test measures a variable or a trait that is stable and we wish to assess the stability of the trait across different situations, then we use?

Test-Retest Reliability.

Nature of a Test

• Stable vs. Dynamic

- If the concern is still stability, but we would like to control for practice and carryover effect, and if a test has more than one form, then we use?

Alternate-Form Reliability.

- When measuring a characteristic that is dynamic, or a characteristic that is expected to change from time to time as a function of situational and cognitive experiences, a measure of internal consistency is more appropriate.

Cronbach's Alpha, Split Half Reliability

Ben constructed an anxiety scale for his psychological assessment project. To establish its reliability, he decided to get its test-retest reliability. With the knowledge that you have, you could say that:

a. He is correct because it is appropriate for homogenous variables.

b. He is incorrect because it is not appropriate for homogenous variables.

c. He is correct because it is appropriate for dynamic variables.

d. He is incorrect because it is inappropriate for dynamic variables.

D

Ben constructed an anxiety scale for his psychological assessment project. To establish its reliability, he decided to get its test-retest reliability. With the knowledge that you have, you could say that:

a. He is correct because it is appropriate for homogenous variables.

b. He is incorrect because it is not appropriate for homogenous variables.

c. He is correct because it is appropriate for dynamic variables.

d. He is incorrect because it is inappropriate for dynamic variables.

A

- estimate of reliability by correlating pairs of scores from the same people on two different administrations of the same test

- appropriate for: tests that measure a construct that is relatively stable over time & tests that employ outcome measures (e.g., reaction time, grip test)

Test-Retest Reliability

Establishing the inter-item consistency of tests like 16PF and BFI-2 to get their reliability as a whole can result in:

a. high reliability

b. low reliability

c. average reliability

d. none, the reliability of the test will depend on how well it is written.

B

Nature of a Test

• Homogeneity vs. Heterogeneity of the items

- Tests that measure a single trait or ability (unidimensional) obtain a ______ of internal consistency.

high degree

- Therefore, it is recommended that for tests that have many subscales (e.g., 16PF, NEO-PI-R, BFI-2), internal consistency must be computed per subscale to get a more accurate reliability estimate.

Nature of a Test

• Homogeneity vs. Heterogeneity of the items

- Tests that are multidimensional (i.e., measures more than one trait or characteristics, has a subscales) would result to a ______ of internal consistency.

lower degree

- Therefore, it is recommended that for tests that have many subscales (e.g., 16PF, NEO-PI-R, BFI-2), internal consistency must be computed per subscale to get a more accurate reliability estimate.

Ivie constructed a speed test and is now deciding what to use to establish its reliability. Based on your knowledge, which among the following should be considered by Ivie?

a. test-retest

b. alternate forms

c. split-half

d. all of the above

D

Nature of a Test

Speed Tests

- A speed test measures response speed; and because of this we cannot measure its reliability after a single administration of a test.

- Thus, for speed tests the test should be administered twice to measure the stability of the test scores after being subjected to the same time limit.

- What are appropriate reliability estimates?

> test-retest reliability

> alternate form reliability

> split-half reliability from two separately timed half tests

- A _____ measures response speed; and because of this we cannot measure its reliability after a single administration of a test.

speed test

for _____ the test should be administered twice to measure the stability of the test scores after being subjected to the same time limit.

speed tests

Doctor Miranda, one of the board members, wanted to get the reliability estimate of the board examination they will conduct this coming August. Anchoring from what you know about establishing reliability:

a. He should use inter-item consistency because it is the most recommended for board examinations.

b. He can use any reliability estimates, as they are all appropriate.

c. It is best for him to use test-retest reliability, which is the most recommended.

d. He should consider other ways of establishing reliability, as traditional reliability estimates may not fit such tests.

D

Nature of a Test

Criterion-referenced tests

Traditional ways of estimating reliability are not always appropriate for criterion-referenced tests.

Score variability is not the focus

> Measures of reliability rely on the variability of the test scores (or how different the scores are from on another).

> When individual differences decrease (meaning there is less variability across the scores), this will lead to lower reliability estimates.

- In CRTs, individual differences between examinees on the total scores may be minimal (e.g., everyone gets high scores), thus reliability estimates will also decrease despite the stability of individual performance.

- Other forms of reliability estimates such as Classification consistency and classification accuracy may be used instead (Alger, 2016).

I. In test-retest reliability, the recommended length between the intervals is 3 months to one year.

II. The longer the interval, the lower the TRR coefficient that it will yield.

a. I is true

b. II is true

c. both are true

d. both are false

B

The following are accurate about alternate forms reliability except for:

a. It is the administration of two tests at the same time

b. It can eliminate practice effects.

c. It measures both temporal stability and consistency of responses.

d. It can be time-consuming and costly.

B

- The same people/sample are tested with one form on the first testing (e.g., Test A) and with another equivalent form (e.g., Test B) on the second testing occasion.

- The correlation or degree of relationship between the scores on the two forms is the estimate of the reliability or also called as the coefficient of equivalence.

- It can reduce practice effects (but not eliminate it).

- It measures temporal stability and consistency of responses.

- It is time consuming & costly.

Alternate-Form Reliability

- The correlation or degree of relationship between the scores on the two forms is the estimate of the reliability or also called as the?

coefficient of equivalence

- two different forms, same occasion

Simultaneous Alternate-Form Reliability

- two different forms at two different occasions

Delayed Alternate-Form Reliability

After running a reliability analysis, Inigo found out that his new test for blirtatiousness has a Cronbach's alpha value of 0.97. Drawing from your knowledge about reliability, you can say that:

a. The test has an excellent internal reliability

b. The test-retest reliability is questionable

c. The internal consistency of the items is satisfactory

d. There might be redundancy in the items, thus, it would do Inigo good to revisit his items.

D

- it is the mean of all possible split-half correlations, corrected by the Spearman Brown formula

- useful for tests with non-dichotomous items (e.g., Likert scale)

- received the most acceptance and is widely used today

Coefficient Alpha/Cronbach's Alpha

Coefficient Alpha/Cronbach's Alpha

More than 0.90

Excellent

Coefficient Alpha/Cronbach's Alpha

0.80-0.89

Good

Coefficient Alpha/Cronbach's Alpha

0.70-0.79

Acceptable

Coefficient Alpha/Cronbach's Alpha

0.6-.69

Questionable

Coefficient Alpha/Cronbach's Alpha

0.5-0.59

Poor

Coefficient Alpha/Cronbach's Alpha

Less than 0.59

Unacceptable

Measures of test reliability can:

a. provide us with an estimate of what proportion of the total variance of the test score is the error variance

b. how consistent the test is

c. the extent to which the test measures what it purports to measure

d. none of the above

A

It's about the test score not the test

While taking the psychological tests given by the HR for their application process, Shuvee was unusually anxious because of a family emergency that occurred the night before. This affected the way she answered the questions. This is an example of what kind of error?

a. systematic

b. transient

c. trait

d. inadvertent

B

- is a type of measurement error caused by fluctuations in psychological states (Trujillo, 2022)

Transient Error

Group A scored higher in the achievement test because they were more familiar with the content of the test, as a huge portion of it was taught to them in class. On the other hand, Group B scored relatively lower than them because the contents were not familiar to them. What was the source of the error?

a. test administration

b. test construction

c. test scoring

d. examiner-related variable

B

> - the variation among the items within or between tests

Item/Content sampling

Sources of Error Variance

> Item/Content sampling - the variation among the items within or between tests

> When a group of test takers are familiar with the items sampled, they may get higher scores than other groups.

• Test Construction

> (e.g., room temperature, level of lighting, ventilation, changes in weather, broken pencil point, and noise)

Test environment

> (e.g., emotional problems, physical discomfort, lack of sleep, illness, fatigue, drugs or medications taken, worry)

Test-taker variables

> (e.g., physical appearance and demeanor, manner of speaking, emphasis on certain word, nodding, other nonverbal gestures)

Examiner-related variables

> Test environment (e.g., room temperature, level of lighting, ventilation, changes in weather, broken pencil point, and noise)

> Test-taker variables (e.g., emotional problems, physical discomfort, lack of sleep, illness, fatigue, drugs or medications taken, worry)

> Examiner-related variables (e.g., physical appearance and demeanor, manner of speaking, emphasis on certain word, nodding, other nonverbal gestures)

Sources of Error Variance

• Test Administration

Sources of Error Variance

> Hand-scoring vs. Computered-scoring

> Objective scoring vs. Subjective scoring

• Test Scoring

Which among the following is correct?

a. The higher the reliability, the higher the error.

b. The lower the reliability, the lower the error.

c. The lower the reliability, the higher the error

d. None. Reliability & error have no relationship.

C

Which of the statements is correct?

a. Once a test is valid, it can be used by a test user anytime.

b. If a test's reliability is high, we can be confident that its validity will also be high because of their positive relationship.

c. Validity refers to the appropriateness of the inferences drawn from the test scores.

d. All of the above

C

- a judgment or estimate of how well a test measures what it purports to measure in a particular context.

- it is a judgment based on the evidence about the appropriateness of inferences drawn from test scores.

Validity

- a "valid" tests is a test that has been shown to be valid for a particular use with a particular population of test takers at a particular time. No test is "universally valid" for all time, for all uses, and with all types of populations.

Ivan is interested in translating the existing BDI-II written in English to Braille. He contended that it is essential so that it can be administered to people with deficits in their vision. Given this, what process should Ivan go through?

a. validation

b. local validation

c. cross validation

d. co-validation

B

- necessary if the test user plans to alter in some way the format, instructions, language, or content of the test

- is also necessary if a test user seeks to use a test with a population of test takers that differed in some significant way from the population on which the test was standardized

• Local validation

On the other hand, Clemuel and Pia constructed two tests - an intelligence test and an aptitude test. To be practical and economical, they decided to validate these two tests using the same test takers because these tests were intended for the same population. What did Clemuel and Pia do?

a. co-norming

b. cross validation

c. local validation

d. co-validation

D

a test validation process conducted on two or more tests using the same sample of test takers.

Co-validation

- the process in which the same sample is used in the creation of norms or the revision of existing norms of two or more tests

co-norming

- the revalidation of a test on a sample of test takers other than those on whom test performance was originally found to be a valid predictor of some criterion

Cross validation

- the decrease in item validities that inevitably occurs after cross-validation of findings.

Validity shrinkage

A group of students developed a test that aims to measure the level of depression experienced by an individual. Most of its items focused on the affective characteristics of the said construct. They consulted three registered psychometricians to have their test validated. Which among the following would most likely be the comment of the validators?

a. The test is good because on the face of it, it covered the affect of individuals who experience depression.

b. The test is content valid because it sufficiently sampled items characteristic of depression.

c. The test lacks content validity; other domains of depression were not covered.

d. The test lacks face validity because it does not seem to discuss depression.

C

- evaluates how adequately an instrument covers all relevant parts of the construct it aims to measure

- extent on how appropriate/representative the items relative to the construct it is measuring

- ex: depression has behavioral, affective, and cognitive aspects which must be covered adequately in a scale

Content validity

- extent to which test scores may be used to estimate an individual's present standing on a criterion

- test scores are obtained at about the same time that the criterion measures are obtained

- ex: Ambrosini et al. (1991) conducted a concurrent validity study to explore the utility of BDI with adolescents, in which the criterion was diagnoses generated from an instrument previously validated for use w/ adolescents.

Concurrent validity

Pami is interested in knowing whether MMPI-II, which is validated for adults, is also appropriate and valid for those 18 years old and below. To do this, she decided to use a measure of psychopathology that is validated and intended for adolescents as a point of reference. This is an example of what validity?

a. content

b. criterion

c. construct

d. face

B

On the other hand, Bianca created a new measure for sensation seeking. To establish its validity, she correlated it with a scale for openness, a construct theoretically similar to it. The results showed that these two were not significantly correlated with each other. With the knowledge that you have, you could say that:

a. Bianca was able to establish evidence of the criterion validity of her test.

b. Bianca failed to establish evidence for the construct validity of her test.

c. Bianca was able to establish evidence of the construct validity of her test.

d. Bianca failed to establish evidence for the criterion validity of her test.

B

is shown when scores on test tend to correlate highly in the predicted direction with scores on older, more established, and already validated tests designed to measure the same or a similar construct

Convergent Evidence

Bonnie, an applicant, was told that she would be taking an integrity test. However, upon getting the test, she noticed that the items did not seem to measure integrity because there were numbers in them. With this information at hand, you can infer that:

a. The test lacks content validity and may produce inaccurate results.

b. The test lacks concurrent validity and may lead to dishonest answering.

c. The test lacks face validity. However, it's alright as face validity never affects the test administration.

d. None of the above is correct.

d

Donny recently established a scale for eating disorders. To establish its validity, he decided to gather two samples - one for those diagnosed with EDs and the other for those not. Results showed that those with EDs scored higher on the scale than those without. With this information, we can say that Donny established what kind of validity?

a. content

b. criterion

c. construct

d. face

c

After trying to establish the validity of her newly created test, wherein Pia correlated her General Anxiety Inventory to the existing General Anxiety Scale, it yielded a validity coefficient of 0.98, which was within her prediction because the two tests measured the same construct. With this, we could say that:

a. She has established convergent evidence that can support the construct validity of her scale.

b. She has established concurrent evidence that can support the criterion validity of her scale.

c. The new test seems to be a needless duplication of GAS.

d. GAI can now replace GAS because of its evidence of construct validity.

C

A criterion should not be:

a. relevant

b. uncontaminated

c. timely

d. valid

C

Which among the following is incorrect?

I. Confirming & contrary evidence contribute to the construct validity of a measure.

II. The homogeneity of a test is also evidence for construct validity.

III. The changes in test scores resulting from an intervention between a pretest and posttest are evidence for criterion validity.

IV. When a depression test is capable of discriminating between people with depression and those who do not, content validity is established.

a. I & II only

b. I & IV only

c. II & III only

d. III & IV only

D

Ralph established the psychometric properties of his newly made test for conscientiousness. The test's Cronbach's alpha is 0.56, whereas the test's construct validity is 0.80. With the knowledge that you have in terms of psychometric properties, you can infer that:

a. The test's validity is good, but the reliability is not acceptable.

b. The test can be used already.

c. There might be something wrong with the reliability and validation procedure.

d. The reliability is good, while the validity is excellent.

C

You aim to establish the content validity of your personality test that measures the Big Five. To do so, you want to retain the items that are relevant to the constructs that you intend to measure. Which of the following should the SMEs use?

a. Content validity index

b. Content validity ratio

c. Confirmatory factor analysis

d. Exploratory factor analysis

A

If you wish to establish the content validity of an achievement test that you have just completed by using items that are essential for the test, which of the following should the SMEs use?

a. Content validity index

b. Content validity ratio

c. Confirmatory factor analysis

d. Exploratory factor analysis

B

Maomao used a content validity ratio to establish the content validity of her newly made test, wherein she asked 10 SMEs to rate each item. Item #5 showed a CVR of 0.65. Given this, what should Maomao do?

a. Retain the item without revision since it shows strong expert agreement.

b. Accept the item, as the CVR meets the acceptable cut-off.

c. Consider revising or deleting the item due to insufficient expert agreement.

d. Retain the item because it is statistically difficult.

B

Will created a new personality test that measures Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. He wants to establish discriminant validity within the subscales by ensuring that each set of items measures its intended construct and that the subscales do not correlate too highly with each other. Which of the following should Will use?

a. Confirmatory factor analysis

b. Exploratory factor analysis

c. Construct Validity Ratio

d. Structural Equation Modeling

A

Based on item #30, what correlation coefficient is considered to violate discriminant validity between two subscales?

a. ≥ .80

b. ≥ .85

c. ≥ .90

d. ≥ .95

C

Your colleague established the reliability of an employment test, and you noticed that he only included the data of the successful employees who were considered for the job. With this information at hand, you can conclude that:

a. the reliability will be high because those who were successful were included in the sample

b. the reliability will be low because of a limited sample

c. the reliability will just be average

d. the reliability will not be affected at all

B

Range restriction only affects reliability estimates. On the other hand, range inflation affects both reliability & validity.

a. first statement is correct

b. second statement is correct

c. both statements are correct

d. both statements are incorrect

B

Taylor and Katy were American test developers who constructed an intelligence scale for Southeast Asians. Therefore, some of the test items were not appropriate to the experiences of the population to whom it was intended. This shows what kind of bias?

a. inadvertent bias

b. content validity bias

c. construct validity bias

d. norming bias

A

This bias occurs when the content of a test is more difficult for one group than for the others.

a. inadvertent bias

b. content validity bias

c. construct validity bias

d. norming bias

B

Diadem used an old, established, and validated IQ test that was created in 1989, called the TSIQ, to measure the intelligence of a group of adults. She ensured the test was appropriate for the test takers' population. Upon utilizing this test and its norms, we could say that:

a. It is okay if the test has been validated and established.

b. There's nothing wrong with it, as the population to whom the TSIQ was intended is similar to the characteristics of the test takers.

c. We can expect an inaccuracy in the scores and their interpretation.

d. None of the above is correct

C

Jinshi is developing a new test for anxiety. Currently, he has already written at least 60 items for this 30-item scale. Also, he has decided to use a Likert scale & get the summation for the scoring. Given this, what is Jinshi's next step?

a. further conceptualize the test

b. construct the test

c. analyze the items that he has constructed

d. conduct the test on a sample representative of the population to whom the test is intended for

D

Upon testing his scale for anxiety, Jinshi's sample was 105 people. Was he correct?

a. No, it should be 150 because the rule-of-thumb indicates that there should be at least five persons per item.

b. No, it should be 300 because the rule-of-thumb indicates that there should be at least 10 persons per item.

c. There are no rulings about the number of samples in the test tryout.

d. Yes, 105 is the recommended sample by Greene (1996).

A

After conducting his test on the sample, what should Jinshi do next?

a. analyze the items that he has constructed

b. start with the test revision

c. conduct the test one more time

d. none for he is already done with the test development

A

If Jinshi is now finished with the test development of his anxiety scale and plans to establish its reliability, what should he best use?

a. test-retest

b. KR20

c. KR21

d. Cronbach's alpha

D

Considering the construct, scaling, and scoring used by Jinshi, we could infer that the format of his items is_______with a______scoring system.

a. selected response & cumulative scoring

b. constructed response & cumulative scoring

c. selected response & ipsative scoring

d. constructed response & category scoring

A

An item is not good if:

a. everybody gets it right

b. everybody gets it wrong

c. some get it right while some get it wrong

d. two of the options are correct

D

Fifty out of 90 examinees answered item #2 incorrectly. Therefore, it has an item difficulty of:

a. 0.50

b. 0.45

c. 0.44

d. 0.55

C

Item 5 has an item difficulty value of 0.90, while item 6 has an item difficulty of .50. We could say that:

a. Item 5 is easier than item 6

b. Item 6 is easier than item 5

c. Both items are difficult

d. There's insufficient data, thus we can't make an inference.

A

Item 5 should be:

a. Used

b. Revised

c. Discarded

d. b & c are correct

D

If your item is a four-option multiple choice, the optimal item difficulty level should be taking into account the possibility of the test takers guessing the answer:

a. 0.63

b. 0.73

a. 0.66

d. 0.76

A

The fourth item on your test yielded a -.56 d-value upon analyzing the discriminability of your test. We could interpret the item as:

a. a good item that discriminates adequately between the high scorers and low scorers in the entirety of the test

b. a bad item because there was no discrimination at all

c. a bad item because it shows that low scorers are more likely to get the answer correctly

d. none of the statements above is correct

C

Pia noticed that most of the high scorers got item 9 incorrectly on the test she made. What should Pia do first?

a. Reject the item.

b. Revise the item.

c. Interview the high scorers.

d. None, it is a good item.

C

Going back to Jinshi's anxiety scale, if he wishes to analyze the difficulty level of his items, which of the following is accurate?

a. He may use the item difficulty index and ensure that each item's difficulty level is within the recommended range.

b. He should consider using item discrimination instead of item difficulty because the latter is not appropriate for his test.

c. He cannot analyze the difficulty level of each item; therefore, he should just analyze the endorsement index per item.

d. He should rely on item reliability and item validity index instead.

C

Regarding item analysis, which of the following is inaccurate?

a. Item difficulty index should be done, especially if the test being analyzed is an ability test.

b. Item discrimination index is important to know whether an item adequately distinguishes high performers and low performers.

c. Items can be analyzed qualitatively through interviews.

d. Item discrimination index cannot be used when the tests are polytomous.

D

Upon checking the distribution of the scores in the test he made, Keno saw that the distribution was negatively skewed. If he wants to get the middle score, what measure of central tendency should he use?

a. range

b. mean

c. median

d. mode

C

On the other hand, when Keno checked the modality of his data set, he noticed that there were three peaks. Drawing from what you know about measures of shape, you could say that:

a. Keno should recheck his data; modality is not a part of measures of shape.

b. It is acceptable and means that there might be three distinct subgroups in his test takers.

c. There's something wrong, as there can only be one modality.

d. It is correct and means that there are three frequently occurring scores.

B

If a data set is negatively skewed, we can infer that:

a. There are many test takers who got very high scores.

b. The test is very easy.

c. There are more test takers who got higher scores than those who got lower scores.

d. There are many test takers who got lower scores.

C

This refers to the "tailedness" of a distribution.

a. skewness

b. normality

c. kurtosis

d. variance

C

When Primo checked the distribution of the scores of the test he made, he noticed that it resembled a normal distribution. Thus, we could say that it is a:

a. platykurtic

b. leptokurtic

c. mesokurtic

d. Betakurtic

C

When Gino checked the distribution of the scores of the test he made, he noticed that there were many outliers at both ends of the tail of his distribution. Thus, we could say that this might be a what kind of distribution?

a. platykurtic

b. leptokurtic

c. normal

d. none of the above

B

Which among the following is a characteristic of a normal distribution?

a. the mean, median, and mode are in the middle

b. it is symmetrical

c. most of the test-takers are close to the average score

d. all of the above

D

If the distribution is normal, approximately how many of the scores will fall outside the interval between ±2SD?

a. 32%

b. 4.6%

c. 13.4%

d. 95%

B

If the mean is 50 and the SD is 10, we can infer that approximately. 99% of all scores lie between:

a. 10 and 80

b. 30 and 70

c. 20 and 80

d. 10 and 90

C

Niki finished as the 5th placer in a marathon with 50 runners. What is her percentile rank?

a. 80th

b. 85th

c. 90th

d. 95th

C

In their Psychological Statistics test, Iwi got a z-score of 2, while Ivan got a t-score of 60. We could say that:

a. Iwi scored higher than Ivan

b. Ivan scored higher than Iwi

c. No one was higher; they performed equally

d. We cannot make an inference due to insufficient data

A

In their Psychological Statistics test, Iwi got a z-score of 2. What is her T score?

a. 50

b. 60

c. 70

d. 80

C

For a score to be considered as "different", it must at least fall above or below the mean.

a. 1 SD

b. 2 SD

c. 3 SD

d. 4 SD

B