Scientific Research Methods (3)

What are the three basic methods for test construction in personality assessments?

• Rational Method

• Factor Analytic Method

• Empirical Method

What is the rational method?

Questions (test items) are directly and obviously related to what the test is intended to assess.

What the requirements for the rational method?

• Items must mean the same thing to everyone,

• Test-takers must be able to make an accurate self-assessment,

• Test-takers must be honest,

• Items must be valid indicators of what the test intends to assess

What is an example of a fake-good item?

“I have never told a lie.”

What is the factor analytic method?

Identifies groups of things that seem to have something in common.

How do you do the factor analytic method?

1. Create a large list of items

2. Administer to many participants

3. Correlated items are assembled into groups (i.e., factors

4. Name the factors

What is an example of the factor analytic method?

• The 16 Personality Factor questionnaire

• the Big Five Inventory;

• the NEO-PI

What is the empirical method?

Data driven.

How do you do the empirical method?

1. Create a large list of items

2. Identify two (or more) meaningful groups

3. Administer the items and compare responses on items between the groups

4. Cross-validate to predict behavior, diagnoses, or membership in a new sample

What is an example of the empirical method?

• Individuals diagnosed with depression vs those who are not

• MMPI

What is reliability?

Elicits the same response every time

What is validity?

The extent to which the assessment is measuring what it is meant to measure.

Why are personality tests valid?

• Include “fake good” and “lie” scales/items

• People are not very good at faking

• Good tests use a combination of the three approaches to test development

• Tested in many populations and cultures*

What is the issue of WWEIRD samples?

The lack of representation of the world's diverse population.

What is publication bias?

Higher likelihood of publishing interesting and significant results.

What is the file drawer effect?

Not submitting unexpected/undesirable results.

What is the novelty over replication?

Too much importance on new and exciting research.

What is p hacking?

Using questionable practices to manipulate data / find significant results.

Where else are personality assessments used besides in psychological research or self satisfaction?

Companies. They use personality tests to select employees who are likely to perform well in the job.

What is the best method for evaluating research results?

Replication

Why does scientific methods matter?

• Testable (can evaluate if it measures what it claims)

• Replicable (others can repeat it with consistent results)

• Predictive (scores relate meaningful to behavior/outcomes)

What are examples of scientific methods?

• Standardized administration

• Systematic scoring

• Theory or hypothesis driven (& based on empirical evidence)

Are personality measures reliable?

Yes, (~.80-.90).

Which of the three test-construction methods (rational, factor-analytic, empirical) is the easiest to understand?

The rational method.

Which of the three test-construction methods (rational, factor-analytic, empirical) is the hardest to understand?

The empirical method.

How many test-construction methods do modern test developers often use?

2 to 3

What are examples of projective tests?

Rorschach Inkblot & Thematic Apperception Test (TAT)

What is the Thematic Apperception Test?

A projective personality test where an individual creates stories about ambiguous pictures to reveal their underlying thoughts, emotions, conflicts, and personality traits.

Are projective tests reliable?

No, but can be useful in therapy.

What is an example of objective test?

Minnesota Multiphasic Personality Inventory (MMPI)

What is the Minnesota Multiphasic Personality Inventory (MMPI)?

An objective test (a series of true/false statements) helping psychologists recognize and diagnose mental health disorders.

What is face validity?

The degree to which a measure appears to be measuring what it's supposed to.