Psychological Assessment Exam 3

Validity

The extent to which a test or research method accurately measures what it suppose to measure. It answers the question: "Are we truly measuring the construct we intend to measure?"

Reliability

The consistency of a measure, indicating how likely it is that the same results would be obtained if the measurement were to be repeated under similar conditions. It refers to the repeatability and precision of a measurement.

Practicality

The feasibility and cost-effectiveness of administering, scoring, and interpreting a test or assessment in a given setting. It considers if the method is sensible to implement and provides enough utility to justify its use.

Cross-Sectional Fairness

The extent to which a test or assessment provides an accurate and unbiased measure for individuals across different demographic, cultural, or social groups. It aims for the absence of systematic errors or biases that favor one group over another.

Correlation

A statistical measure that describes the strength and direction of a linear relationship between two variables. It indicates how closely two variables move in relation to one another, ranging from -1 (perfect negative) to +1 (perfect positive), with 0 indicating no linear relationship.

No Correlation

A situation where there is no discernible statistical relationship or pattern between two variables, meaning changes in one variable do not systematically predict changes in the other. A correlation coefficient close to 0 typically indicates no linear correlation.

Concurrent Validity

A type of criterion-related validity that assesses how well a new measure correlates with an already established and validated measure when both are administered at the same time. It checks if the test can accurately predict a present criterion or status.

Predictive Validity

A type of criterion-related validity that evaluates how well a test predicts future behavior, performance, or outcomes. It assesses the accuracy with which a measure can foretell a future criterion, such as academic success or job performance.

Construct Validity

The degree to which a test or measurement tool accurately measures the specific theoretical construct or unobservable trait (e.g., intelligence, anxiety) it is designed to assess. It involves determining if the underlying concept is truly being measured.

Convergent Validity

A subtype of construct validity, demonstrating that scores on a measure are highly correlated (converge) with scores on other measures that theoretically assess the same or very similar constructs. High correlations suggest the measures are tapping into the same underlying concept.

Discrimination Validity

Also known as divergent validity, it is a subtype of construct validity, demonstrating that scores on a measure are weakly or negatively correlated (discriminate) with scores on measures that theoretically assess different or unrelated constructs. Low correlations suggest the measures are distinct.

Hit Rate

In psychological testing and selection, the proportion of cases in which a test accurately predicts the outcome or criterion. It represents the percentage of correct predictions made by a test, such as correctly identifying who will succeed or fail.

Test Retest Reliability

A method for estimating the reliability of a test by administering the same test to the same group of individuals on two separate occasions and then correlating the two sets of scores to see their consistency over time. High correlation indicates good stability of the measure.

Coefficient Alpha

Also known as Cronbach's Alpha, it is a coefficient of internal consistency that measures how closely related a set of items are as a group. It is often described as the mean of all possible split-half reliabilities for a given test, indicating how well the items consistently measure the same construct.

Fluid Intelligence

The capacity to reason quickly and think flexibly to solve novel problems, adapt to new situations, and understand complex relationships, largely independent of acquired knowledge or learned skills. It's often associated with working memory and processing speed.

Crystallized Intelligence

The accumulated knowledge, facts, and skills acquired over a lifetime, often reflecting general knowledge, vocabulary, and verbal reasoning abilities, which tend to increase with age and experience. It is heavily influenced by education and cultural background.

Triarchic Theory of Successful Intelligence

Robert Sternberg's theory suggesting that successful intelligence is the ability to achieve success in life by capitalizing on one's strengths and correcting or compensating for one's weaknesses. It encompasses three distinct intelligences: analytical (academic problem-solving), creative (generating novel ideas), and practical (adapting to real-world contexts).

Existential Intelligence

One of Howard Gardner's proposed intelligences, characterized by the capacity to ponder deep questions about human existence, such as the meaning of life, death, and ultimate destinies. It involves a sensitivity and capacity to tackle profound philosophical inquiries.

Social Identity Threat

A psychological predicament in which individuals feel at risk of being judged or treated negatively based on a negative stereotype about their social group. This concern can lead to anxiety, self-consciousness, and impaired performance, often manifesting as stereotype threat.

Flynn Effect

The observed phenomenon that average scores on intelligence tests have been rising across the world over the past several decades, typically increasing by about 3 IQ points per decade. Proposed reasons include improved nutrition, education, and complexity of modern life.

Intelligence Quotient (IQ)

A standardized score derived from a set of intelligence tests, representing an individual's cognitive abilities relative to the general population of the same age. It indicates how one compares to others on a specific intelligence test.

What question does Validity aim to answer?

"Are we truly measuring the construct we intend to measure?"

What two qualities does Reliability refer to?

Repeatability and precision of a measurement.

What factors does Practicality consider regarding a test?

Its feasibility and cost-effectiveness of administration, scoring, and interpretation.

What is the aim of Cross-Sectional Fairness?

The absence of systematic errors or biases that favor one group over another.

What is the range of a correlation coefficient?

-1 to +1.

What does a correlation coefficient of 0 signify?

No linear relationship between two variables.

When are measures administered for Concurrent Validity?

At the same time as an established measure.

What does Predictive Validity aim to assess?

How well a test predicts future behavior, performance, or outcomes.

What does Construct Validity determine?

If the underlying concept or unobservable trait is truly being measured.

In Convergent Validity, what type of correlation is expected between measures of similar constructs?

High correlation.

What is another name for Discrimination Validity?

Divergent validity.

In Discrimination Validity, what type of correlation is expected between measures of unrelated constructs?

Weak or negative correlation.

In psychological testing, what does Hit Rate represent?

The percentage of correct predictions made by a test about an outcome or criterion.

How is Test-Retest Reliability estimated?

By administering the same test to the same group on two separate occasions and correlating the scores.

What does a high correlation in Test-Retest Reliability indicate?

Good stability of the measure over time.

What is another common name for Coefficient Alpha?

Cronbach's Alpha.

How is Coefficient Alpha often described?

The mean of all possible split-half reliabilities for a given test.

Fluid Intelligence is largely independent of what?

Acquired knowledge or learned skills. They max out around 20’s and 30’s, slows and faster around 75+. Crystallized maintained much longer 

What two cognitive processes is Fluid Intelligence often associated with?

Working memory and processing speed.

What type of abilities tend to increase with age and experience in Crystallized Intelligence?

General knowledge, vocabulary, and verbal reasoning abilities. These skills are maintained much longer than fluid skills 

Who proposed the Triarchic Theory of Successful Intelligence?

Robert Sternberg.

What are the three distinct intelligences in Sternberg's Triarchic Theory?

Analytical, creative, and practical intelligences.

Which intelligence in Sternberg's Triarchic Theory relates to academic problem-solving?

Analytical intelligence.

Which intelligence in Sternberg's Triarchic Theory relates to generating novel ideas?

Creative intelligence.

Which intelligence in Sternberg's Triarchic Theory relates to adapting to real-world contexts?

Practical intelligence.

Who proposed Existential Intelligence as one of his theories?

Howard Gardner.

What kind of questions does Existential Intelligence involve?

Deep questions about human existence, such as the meaning of life, death, and ultimate destinies.

What can Social Identity Threat lead to in individuals?

Anxiety, self-consciousness, and impaired performance.

The Flynn Effect describes an increase in average IQ scores over time, by approximately how many points per decade?

About 3 IQ points per decade.

What are some proposed reasons for the Flynn Effect?

Improved nutrition, education, and complexity of modern life.

What does an Intelligence Quotient (IQ) score indicate?

How one compares to others on a specific intelligence test, representing cognitive abilities relative to the general population of the same age.

Two Factor Theory of Intelligence 

• g factor and s factor!

• little g is the ability to reason and solve problems: impacts everything, explains high correlations among skills; s factor is specialized skills 

G factor 

general intelligence; the basic intelligence that you are born with; these skills don’t need to be polished or improved, they’re just qualities that you are born with; could be genetic; varies from person to person

S Factor

• The ability to excel in certain areas; specific intelligences (art, music, art, business, etc) the two factor model was too simple 

• Specific abilities; skills that you learn and then master; the more you practice, the better you get. You can have more than one, but you may lose them if you don’t practice.

Group Factor 

an intermediate construct that impacts must have but not all skills. Not general g or specific s  

The Hierarchical Model

a theory that intelligence is structured in multiple levels, from a broad, general factor at the top to more specific abilities at the bottom

What is the best measure of “g”?

• Raven’s Progressive Matrices (they say this is the purest measure of “g”). This test is supposed to be culture free. For ages 5+. You have to choose the missing piece that best suits the pattern of images

Howard Gardner thought 

intelligence was too vast and too complex to measure as we do. Initially came up with 7 forms of intelligence and then added throughout the years. (7, 8, 9, 10)

Deviation IQ

1. Raw scores are converted too In scores based on raw score distribution for age. (Scores fall along normal curve; can manipulate test if not)

• arbitrarily but consistently use to 100 as mean with a SD of 15.

We only use deviation now! This is the old way…

(ratio)    IQ = MA/CA * 100

MA = Mental Age (what you score like; could be lower, higher, or average) 

CA =Chronological Age (Actual Age)

• These are traditions set somewhat arbitrarily, but they allow you to compare across tests and ages

Pedagogical

understanding how learning happens and knowing how to shape one's own learning to help others learn 

Why are we testing intelligence?

• To assist people who are struggling to who is excelling 

• Used to identify intellectual disabilities/learning disorders 

• Guide admission to private schools

•  Helps predict academic success

• Helps predict occupational success 

• It DOES NOT predict mental health 

• Assess intellectual ability after brain injury 

• To assess/document impairment and or legal competence 

• To diagnosis disability — access to benefits 

• Personality testing as a part of full assessment for diagnosis and/or therapeutic intervention 

• Career Guidance 

Those with IQ’s 130+ are considered…

highly gifted

Those with low IQs (<70ish)…

are eligible for a diagnosis of intellectual developmental disorder. 

(a.k.a. intellectual disability, old name = mental retardation) 

Having a low IQ and being considered intellectually disabled you must…

also requires low adaptive functioning (age-appropriate ability to care for self social and practical skills)

and start before age 18 (a neurodevelopmental disorder)

(starts later?  Mild or major neurocognitive disorder

specify cause if known (disease or TBI))

Why don’t we use an absolute IQ cutoff of 70 anymore?  

DSM5 (+TR):  no longer uses a strict IQ cutoff for ID

             instead:  must show conceptual, social + practical deficits

                (guidelines provided)

Think:  importance of legal and clinical decisions being made

Think:  SEM  (~2.5)

     

To Assess Intellectual Development

(old name mental retardation) (intellectual disability)

• To ID certain learning disorders (disabilities)

These are the 6 accounts for the 95% of IQ Tests 

WAIS, S-Binet, CAS, WISC, DTLA (Detroit Test of Learning Aptitude), K-BIT (Kaufman Brief Intelligence Test). All provide global score (most subset/index scores). Global scores correlate  .8-.9  (almost as high as test –retest!). All possess large, stratified samples and have strong evidence for reliability and validity.

The Wechsler Series

• Wechsler Adult Intelligence Scale 

(started Wechsler-Bellevue 1930s, WAIS:1955)

(WAIS 5 (aug 2024)) (ages 16 – adult (90))  

• Wechsler Intelligence Scale for Children (1949)

(now WISC V) (6 to 17)  (Can be given w/ two iPads!)

• Wechsler Preschool and Primary Scale of Intelligence

(1967) (now WPPSI IV)  ages 0-7 (2.5-7.5)

Useful in assessing general cognitive functioning, assessment of intellectual disabilities, giftedness, and learning problems

WAIS IV scores:

General Ability Index (IQ);   

Verbal Comprehension and Perceptual Reasoning

Cognitive Proficiency Index:

Working Memory and Processing Speed

4 Indices (Verbal Comprehension, Perceptual Reasoning, Working Memory, Processing Speed) 

(factor support – vs. former verbal and performance IQ)

  up to 15 Subtests  (10 core; 5 supplemental)

Full Scale and Indices :  average 100; SD 15

Subtests :  average 10; SD 3

Standardization:  2200 individuals (cooperative and healthy)

stratified gender/race-ethnicity/ed level and geography age groups (2 yrs, 5yrs, 10 yrs, 5 yrs)

SB5 (Standford Binet 5)

• Standardization:

stratified random sample of 4800 (2000 census)

(gender, ethnicity, geo, ed levels)

• Reliability 

global, Verbal and nonVerbal :  High .90s

subtests .70-.85

• Validity

content validity: experts agree:  

gender, race, ethnicity, disability + religion fairness

other measures acceptable 

Ages: 2-85+

Average: 100; SD: 15 (meaning extended high and low items)

Cognitive Assessment System II (CAS)

• Ages 5-18 (~1 hour)

• 12 subtests,  full scale and 4 process scores (based on PASS)

• Stratified US sample 2200

• Acceptable reliability and validity 

• Given the same info Blacks and Whites show similar info processing skills (PASS-CAS) (FAGAN)

Black/white race differences smaller than traditional tests

(when control for other variables like SES)

• Correlate with grades high for both groups

• Use may help decrease over-rep of black children in special ed

The Flynn Effect Increase

- Increase in average IQ score

- about 3 pts/decade (then re-standardize) 

- biggest jump - prob solve, not gen knowledge

• Within groups there are generational differences  (ex: FLYNN)

(and average IQ for Blacks increasing more than Whites)

Infant/Child Trend:

• Early intervention is useful to identify development delays . Measure multiple domains: cognitive, social and motor

• We don’t identify if a young child is gifted because infants change so rapidly so they may easily fall behind as fast as they excel

When does IQ become reasonably predictive?

• By age 4 you can predict adolescence and then when they’re middle child to teenage years then you can better determine their intellectual level as an adult 

Heritability

proportion of variation among individuals that can be attributed to genes. estimates of intelligence range from 50-80%. Strong evidence nature (genes) AND nurture (environment) matter

Test Bias

• validity varies by group / systematic inaccuracies 

    (the accuracy of interpretations/predictions varies          by things like race/ethnicity, gender, SES, etc)

determined by data

(content v. predictive v. construct bias)

*group differences alone are not enough

Test Fairness

• reflects social values and test use philosophy

Being unbiased does not necessarily make it fair!

Test Bias: a statistical question

a biased test is one where the same test score means different things for people from different groups

    * * need more than group differences to show bias

Test Fairness:  a social justice question

tests may be considered unfair if current group differences lead to differential opportunities that maintain (or grow) the group differences over time

Neurodevelopmental

the development and functioning of the nervous system, particularly the brain. the processes involved in the formation, maturation, and organization of the brain and its connections, as well as the emergence of cognitive, motor, and behavioral abilities.