Personality Assessment Notes

Gordon Allport's Critique of Personality Scales

Gordon Allport questioned the validity of Likert-type personality scales.
He posited personality as inherently inconsistent, contradictory, and situation-dependent.
Allport suggested that situation influences personality traits in structured environments, while internal factors dominate in unstructured ones.
He argued personality evolves beyond simple maturation and traits cannot be quantitatively expressed on a continuum.
"Common traits" are approximations, not true personality characteristics.

Challenges to Allport's Theory

The Five-Factor Theory (FFT) challenges Allport's claims, assuming personality is consistent and transcontextual.

Study Overview

Two studies re-examine data (N = 870) and replicate findings (N = 1,423) using the NEO-PI-R inventory.
The aim was to evaluate response patterns relative to Allport's claims.

Key Findings Supporting Allport

Only some NEO-PI-R items are transcontextual (situation-independent).
Consistency Index (CI) and Decisiveness Index (DI) were developed to measure response consistency.
High levels of inconsistent and indecisive responses were observed.
Response consistency and decisiveness are linked to situational information in items and word meaning structure.

Implications of Findings

Inconsistency and indecisiveness are not random responding styles.
The NEO-PI-R is an unlikely test to refute FFT.
Summary scores from Likert-type questionnaires may be misleading without validating response consistency.

Introduction to Personality Testing

Personality testing is a multi-billion-dollar industry.
It assumes personality assessments reveal information about individuals.
Assessments aim to differentiate individuals using a limited number of personality types or dimensions.

Approaches to Personality Testing

Two main approaches exist:
- Type-based (e.g., Myers-Briggs Type Indicator - MBTI) that classify individuals into distinct types.
- Trait-based (e.g., NEO-Personality Inventory - NEO-PI) that maps individuals on a continuum of traits.
Empirical evidence suggests personality testing systems can combine within the Five-Factor Model.

Questions on Personality Assessments

What knowledge can be gained from assessments?
Assessments can predict job success, organizational fit, and intervention effectiveness to some extent.
However, this predictive ability doesn't guarantee assessments reflect an individual’s true character.

Gordon Allport's Views on Personality

Allport's understanding differs from popular tests like NEO-PI.
Key Concepts:
- Personality is situation-dependent, not stable or fixed.
- Personality is inconsistent and contradictory; individuals aren't neatly categorized.
- Individuals are often mixed types, exhibiting traits differently across contexts.
- "Moderately extraverted" individuals may show extraversion often but introversion in some situations.
- Situational determinants are crucial when duties and roles are defined, while personality determinants are more significant when tasks are free and unstructured.

Situation-Dependence vs. Transcontextuality

Situation-dependence contrasts with the belief that personality is transcontextual.
Transcontextuality means personality is independent of specific situations.
Transcontextual items lack situational structure, while those with any external situation include constraints.

Personality Development

Children are more situational, modifying behavior based on circumstances, than adults.
Adults are less prisoners of the immediate situation than children.
Personality traits aren't fixed; they're ranges of potential behaviors, not amounts along a continuum.
Allport rejected that individuals share the same fundamental personality structure.
Common traits are convenient approximations for the abstract average person, not individual cases.

Recent Views: Five-Factor Theory (FFT)

FFT contradicts Allport's principles; assumes personality is coherent and consistent.
Personality comprises five basic dimensions: Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness.
The dimensions are divided into facets and nuances, all seen as transcontextual.
This system is considered universal.
FFT posits traits are influenced solely by biology, and changes are quantitative.
Personality development results from biological maturation but is strictly quantitative.
FFT assumes personality measurement is possible, describing individuals quantitatively.

Questions Addressed in the Study

Is personality coherent or inconsistent?
Is personality situation-dependent?
Do humans behave differently in structured vs. unstructured situations?
Is personality shaped solely by maturation?
Can personality traits be expressed quantitatively?
Are “common traits” reflections of true nature?

Theoretical Foundation and Analytical Methods

The study uses NEO-PI-Revised to test Allport's ideas, demonstrating NEO-PI-R reflects Allport's characteristics.
Novel measures were devised to reveal shortcomings of personality questionnaires.

Measuring Inconsistency of Personality

NEO-PI-R, responses to individual items don't provide pure measures.
Items are summed, assuming statistical error variance cancels out with Likert-type responses (strongly disagree to strongly agree).
Responses are assigned numerical values (0–4) and summed for each domain.
Responses are treated as ordinal or interval scales.

Consider items from the Extraversion dimension:
* (1) I do not get much pleasure from chatting with people
* (2) I really enjoy talking to people.

Item (1) is reverse-coded (0 to 4 recoded as 4 to 0). A higher sum indicates more extraversion.

A sum of 4 is assumed to mean one is neutral, but one could respond with "strongly disagree" (4 after recode) or “strongly agree” (0 after recode) to both items.
This leads to 4 + 0 = 4; however, responding “I really like talking” and “I really do not like to chat” is contradictory to the idea of being neutral about both items.
The first pair of answers is internally contradictory, while the second is consistent.
Reverse coding the negatively worded item responses, assign “-1” to "strongly disagree" or "disagree", ‘0’ to “neutral/undecided” and “+1” to “agree” or "strongly agree".
All items related to the same domain are then aggregated after the recoding, and the absolute value of the sum is divided by the total number of non-neutral responses within that specific domain of the test.

Consistency Index (CI)

The resulting value is known as the Consistency Index (CI), which ranges from 0 (absolute inconsistency) to 1 (absolute consistency) and assesses between-item inconsistency
$CI = 0$ indicates absolute inconsistency (responses are completely opposing)
$CI = 1$ indicates absolute consistency (responses completely agree or disagree)

Decisiveness Index (DI)

DI measures within-item inconsistency.
"Neutral" responses may mean "It depends", "I don't know”, etc.
Proportion of "undecided" may reflect confusion.
Decisiveness Index (DI) measures the proportion of “decided” responses from the total number of answers.
Both CI and DI can reveal information about test performance.
Summary scores may obscure consistency/decisiveness levels.
Summary score interpretation depends on high consistency. Otherwise, the test results show "mixed-type" personality and are not interpretable.
Elevated DI also suggests non-interpretable summaries.

Measuring Situation Dependency

NEO-PI-R is designed to be situation-independent.
Using NEO-PI-R to demonstrate situation dependency provides a stringent test.
Content analysis categorizes items into:
- Transcontextual (no specific context: “I rarely feel fearful or anxious” and “I strive to achieve all I can.”)
- Situational (suggesting context: "Sometimes, I cheat when playing solitaire")
Responses to situational items should be less consistent than transcontextual items.

Development and Word Meaning Structure

Development involves not only the brain but also social and cultural experiences.
Vygotsky proposed that different information processing modalities exist in verbal cognition.
These forms develop hierarchically through interaction with a culturally structured environment.
Everyday conceptual structure vs. scientific conceptual structure.
DWMS relates to how verbal thought is structured, while intelligence pertains to the ability
to solve problems accurately.
Logical conceptual thinkers might argue that potatoes and carrots go together because both are vegetables. The hierarchical term “vegetables” unites various types of vegetables into one category. In contrast, an everyday conceptual thinker may justify this based on practical experience, stating that potatoes and carrots go together because both grow in a field.

Dominant Type of Word Meaning Structure (DWMS)

DWMS is not identical to intelligence.
Revealing logical concepts:
- Word triplets (e.g., soup-carrot-potato: carrot and potato belong together because both are vegetables (hierarchical)).
DWMS is seen as a tool for thought and should influence all areas of mental functioning.
DWMS affects perception, generalization, reasoning, imagination, self-analysis, etc.
logical conceptual thinkers perform better on visual-perceptual tests.
DWMS test assesses the developmental level of verbal thinking.
If Allport is correct, personality structure must be linked to DWMS.
Everyday conceptual thinkers should exhibit less consistent personalities.

Study 1: Re-examination of Previous Study

Re-examined results from a previous study concerning the consistency, decisiveness, and situational dependence of responses on the NEO-PI-R.

Methods

Participants: 870 native Estonian males (military personnel and undergraduate students, mean age = 24.64 years, SD = 8.14).
Educational Background:
- 255 (29.3%) had primary education (9 years or less),
- 498 (57.2%) held secondary education (12 years),
- 62 (7.1%) were first- or second-year university students,
- 55 (6.3%) had completed a university degree (Bachelor of Science or Bachelor of Arts in nearly all cases).
Personality Inventory:
- Evaluated using the Estonian version of the NEO-PI-R which contain five primary dimensions of personality: Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness.
- Each of these main dimensions is further divided into six facets, containing eight items per facet.
Situational and Transcontextual Items:
- Items from the NEO-PI-R were classified into two groups based on the criteria mentioned above.
- The situational category included items that suggest some external situation or context, while the
  transcontextual category comprised items that reference no context.
- High interrater agreement in coding (Fleiss’ κ = 0.77).
- 143 situational and 97 transcontextual items.
Consistency Index (CI):
- Responses to items phrased negatively regarding their respective dimensions and facets were reverse-coded.
- "Strongly disagree" and "disagree" responses were given a code of “-1”, “undecided” was coded as “0”, and “agree” and “strongly agree” received a code of “+1."
- All items within the same personality dimension of the NEO-PI-R were summed, and the absolute value of this sum was calculated.
- $CI = \frac{{\left| \sum Responses \right|}}{{Total Non-Neutral Responses}}$
- SCI and TCI were calculated separately
The Decisiveness Index (DI) is calculated as the number of decisive responses (all non-neutral responses) divided by the total number of items in the test (240).

Dominant Type of Word Meaning Structure

Original test designed based on Luria's suggestions, included:
- Definitions for eight concepts (concrete and abstract).
- Nine word pairs (similarity descriptions).
- Nine triplets of words (indicating which two “go together” and justifying the relation).
Responses were classified into everyday concepts (0) or logical concepts (1).
DWMS was determined by summing all item scores (max score of 26).

Results

The study search of answers to questions that address core assumptions underlying modern theories of personality.
Is personality inconsistent and contradictory?
CI varied from 0.05 to 1.00 (Mean = 0.44, SD = 0.17, Median = 0.44, 90th percentile = 0.67). Only one respondent showed complete consistency.
Median CI (0.44) means half the group had a response distribution of 72% or less.
DI ranged from 0.00 to 1.00 (Mean = 0.72, SD = 0.17, Median = 0.73, 90th percentile = 0.91). "Undecided" responses were quite common, with 50% answering “undecided” to ≥27% of items.
CI and DI demonstrated a correlation (r = 0.175, p < 0.0001), although the effect size was low.
Is personality situation-dependent? and Does it develop only through maturation, or are more complex developmental processes involved?
To analyze the all above question relationships DWMS, TCI, SCI, TDI, and SDI was used.
Participants were five groups based on the number of hierarchical answers on the DWMS test (H1 to H5 groups).
If personality is situation-dependent, SCI should be significantly lower than TCI.
If personality development is influenced by cultural factors, response consistency should increase with hierarchical DWMS answers.
An analysis of variance (ANOVA) revealed significant main effects related to the group, $F(4, 859) = 22.62$ , p < 0.0001, partial $\eta^2 = 0.096$ , CI, $F(1, 859) = 292.66$ , p < 0.0001, partial $\eta^2 = 0.254$ , along with a significant Group × CI interaction, $F(4, 859) = 5.13$ , p < 0.0005, partial $\eta^2 = 0.023$ .
SCI was significantly lower than TCI in all DWMS groups (p < 0.0001), with TCI increasing more than SCI as hierarchical answers rise.
Significant main effects attributable to the group, $F(4, 865) = 19.71$ , p < 0.0001, partial $\eta^2 = 0.084$ , and DI, $F(1, 865) = 7.04$ , p < 0.0082, partial $\eta^2 = 0.008$ . Furthermore, a significant Group × CI interaction was observed, $F(4, 865) = 8.16$ , p < 0.0001, partial $\eta^2 = 0.036$ .
SDI should be significantly lower than TDI.
Decisiveness of responses was correlated through a rise in the number of intralinguistic hierarchical answers on the DWMS test.
an examination of Figure 2 shows that the mean differences among the word meaning structure groups follow a systematic pattern: as the level of hierarchical responses increases, so do the levels of both TDI and SDI.
TDI was significantly higher than SDI in H3, H4, and H5 (p < 0.04); in H1 TDI, it was significantly lower than SDI (p < 0.001).

NEO-PI-R factor structure and response consistency

Exploratory factor analyses were conducted at the facet level to determine if the factor structure varies with CI.
A “universal” Big Five structure should emerge only among individuals with a high level of response consistency.
A chaotic factor structure must appear if the CI is low
Individuals with low levels of CI interpret test items more in terms of specific situations personally associated with the items rather than in terms of abstract, transcontextual characteristics of the psyche.
A five-factor structure was not found among groups of individuals who scored low on the DWMS, whereas the expected Big 5 factor structure emerged among individuals with a high level of hierarchical responses (more educated individuals).
The patterns of factor loadings: Showed that the expected Big Five structure was largely consistent with the original NEO-PI-R across the entire sample.
CI-high group, acceptable level of explained variance should be at least 50%, ideally exceeding 60%
For the CI-low group, the explained variance was significantly low.

Study 2

Replicated the findings of Study 1.
Employed shorter versions of both the NEO-PI-R and DWMS test.
All the main results of Study 1 were replicated.

Individual Level Analysis

Test performance at the individual level.
If personality is not consistent and transcontextual, there should be high within-individual variability.
Individuals with identical scores should often differ significantly.
Four cases from Study 1 are presented as examples of between-subject variability.

Why X doesn't equal X

Scores calculated from NEO-PI-R are interpreted as indicators of underlying constructs.
However, if Allport's assertion holds true, then identical results may not reflect the same underlying traits when examined individually.
The first two examples are from the lower spectrum of the Neuroticism dimension results.
- A total of 14 individuals achieved a score of 66 on this dimension.
The third and fourth examples were from the high end of the Neuroticism dimension.
- Case C and Case D represent two of the six cases that reported high but identical scores on the Neuroticism dimension (94th percentile).
Identical summary scores on a personality dimension hide substantial differences in the structure of personality.

Discussion

The study pursued answers that examines: the nature of personality and the possibility of characterizing personality using Likert-type scales.
It is found that individuals can hold conflicting attitudes about the same aspects of the world, which is in direct opposition of what the contemporary personality theories state.
Responses to situational items are less consistent with the theoretical Big Five model and those that did not imply in external situation (Transcontextual).
The analysis of CI and DI levels indicated that responses to items suggesting external situations were less consistent with the theoretical Big Five model compared to those that did not imply a situation.
Therefore, humans exhibit differences in both structured and unstructured contexts; in the latter, their self-reports align more closely with the Big Five model’s theory.
These finding aligned with Allport point of view in personality that any theory that regards personality as stable, fixed and invariable, is wrong.
Human psyche is a phenomenon that arises from the interaction between the individual and their environment, also suggest that certain developmental changes in the psyche are only possible when the individual’s environment is
structured in a specific way.
Individuals who primarily think in everyday concepts should display a less coherent and more contradictory personality structure, which was supported, in the theory.
*The findings suggested that, personality traits are not distributed along a quantitative continuum.
"common traits” identified through the factor that suggest that the Big Five Factor are an abstraction not describing individuals and it only applies to an abstract average person.
NEOAC five-factor structure is not a human universal.

Likert scale and Likert response format

There is no assumption for questionnaires using a Likert-type response to constructs a Meaningful Likert scale from items.

Scores may lead to an unjustified summary in psychologically, especially with the context of questionnaires and inventory utilizing Likert-type response and their consistency.

Conclusion

Analyzing responses to Likert-type response patterns on the NEO-PI-R to point out Allport idea about personality is accurate.
These findings have much broader implications since there Is no concrete belief the response patterns are inconsistent in the context of using the NEO-PI-R, Likert type results may cause high mislead and psychologically unjust.
It is important to use caution for summarizing scores without validated consistency.