PSYC 102 - Ch. 8: Thought, Language, Intelligence

replication crisis

The discovery that many psychological studies cannot be successfully repeated (replicated), raising concerns about the reliability of findings.

paradigm shift

A major change in the model or framework scientists use to study something.

Why is psychology in a paradigm shift right now?

Because the field is reforming how research is done to improve replicability.

What study became the “canary in the coal mine” for the replication crisis?

Daryl Bem’s 2011 “Feeling the Future” study.

What did Bem (2011) claim?

That participants could predict future erotic images at above-chance levels (53% - statistically significant!!)

Why was Bem’s study controversial?

Because many later studies failed to replicate it.

What percentage of 100 major psychology studies successfully replicated?

Only 39%

Why did Bem’s results seem shocking?

They appeared to support precognition (paranormal ability).

Why were Bem’s studies important to the replication crisis?

Because if impossible claims (like feeling the future) could get published in top journals, it suggested a bigger systematic problem in psychological science.

What did Bem’s work reveal about scientific literature?

That statistically significant findings are not always true findings.

What was the goal of Simmons (2011) study?

Relatively common, unreported scientific research practices can create/find evidence for effects known to be false.

false-positive psychology

Finding statistically significant results for effects that are actually false.

The area of psychology called _______ proves that we can ______ ____ conclusions

false-positive psychology; “simulate”, false

Who conducted the false-positive psychology studies?

Joseph Simmons in 2011

“Hot Potato” study

Participants listened to the song “Hot Potato” and then reported feeling older.

“When I’m 64” study

Participants listened to the song and were supposedly “1.5 years younger.”

What does “chronological rejuvenation” mean for Simmons study?

The absurd false claim that music made people literally younger.

Did Simmons actually believe music changed age?

No—the study was designed to expose flaws in research practices.

Why was Simmons’ study important to the replication crisis?

It demonstrated the mechanism by which false claims can gain empirical support.

What mechanism did Simmons reveal?

Flexible analyses and questionable research practices (p-hacking).

Mechanism 1

“Filtering Published Research”

Criteria used to evaluate research for Mechanism 1

1. Finding answers important question

2. Finding has high internal validity

3. Finding is novel or surprising

4. Finding is statistically significant,p < .05

Pressures for publication for Mechanism 1

• Significant & novel effects are published, even if...

• Non-significant go to the ‘file drawer”, even if....

Mechanism 2

“Filtering Researchers”

Filters for scientists (Mechanism 2)

1. Publish to get a PhD

2. Publish to get a post-doc

3. Publish to become a prof

4. Publish to get grant funding

5. Publish to get tenure

Science is a ___ ____ and relies on ____ ______!

social process; human decisions

Social indicators (Mechanism 2)

Social indicator 1: Is the result statistically significant?

Social indicator 2: Did you, scientist, publish?

Indicator 2 outlines…

In your career have you engaged in any of the following practices:

• Not reporting all outcome variables: 66.5%

• Collecting more data if not significant: 58.0%

• Reporting only studies that ‘work’: 50.0%

• Not reporting all conditions: 27.4%

• Stopping data collection early: 22.5%•

• Rounding”p-values: 23.3%

• Falsifying data: 1.7%

Biases in research design

1. effectiveness study

2. side-effect study

__ ____ differ for effectiveness vs. side-effect studies, displaying what?

Rx doses; biases in research design

effectiveness study

use big doses

side-effect study

use small doses

What is p-hacking?

Manipulating data collection or analysis until p < .05 is found.

What is the file drawer problem?

Non-significant results are often not published.

Why is publication bias a problem?

It makes effects look stronger or more reliable than they actually are.

What percentage of researchers admitted to collecting more data after non-significant results?

58%

What percentage admitted to not reporting all outcome variables?

66.5%

Solutions to Replication Crisis

1. Preregistering hypotheses, data collection, and data analysis plans

2. Publishing null results (p > .05) and replications

3. Increasing sample sizes

4. Publishing raw data, methods, & analysis scripts

5. Embracing ‘messy’ data

6. Even (bravely) critiquing own work

preregistration

Publicly stating hypotheses, methods, and analyses before collecting data.

Why publish null results?

To reduce publication bias.

Why do larger sample sizes help?

They make findings more reliable.

Why publish raw data?

So others can verify analyses.

How did David Wechsler define intelligence?

The global capacity to act purposefully, think rationally, and deal effectively with the environment.

What is the difference between aptitude and achievement tests?

Aptitude = future ability; Achievement = what you already learned.

What do intelligence tests measure?

Broad, overall capability across many domains.

3 Approaches to Intelligence

1. Psychometric Approach

2. Multiple Intelligences

3. Information Processing Approach

What is the psychometric approach?

Children seem to have different potentials → assume existence of intelligence construct, then measure intelligence through standardized testing.

Who developed early intelligence testing?

Alfred Binet and Théodore Simon

Binet-Simon Natural Intelligence Test

The first widely accepted practical intelligence test. It was designed to identify schoolchildren needing academic support, laying the groundwork for modern IQ testing.

what did the Binet-Simon Natural Intelligence Test assess?

Instead of testing acquired knowledge like specific math or reading skills, the Binet-Simon scale focused on assessing:

• Memory

• Attention

• Abstract reasoning

• Problem-solving abilities

the tests were tailored to specific age levels based on developmental norms

Who proposed the two-factor theory?

Charles Spearman

Spearman’s 1904 General Ability/Intelligence Approach

A person's ultimate performance on any task is determined by their overarching general intelligence (g) combined with the specific ability (s) required for that exact task.

Factor 1 (g)

General ability — overall mental ability; single, foundational mental capacity that drives all of our cognitive performance

Factor 2 (s)

Specific abilities — strengths in particular areas.

• akin to aptitudes

Logical, Mechanical, Spatial, Arithmetical

Two-factor theory, Spearman’s Approach

predictive validity

A test’s ability to predict future performance.

What does g predict?

School performance and academic success.

What does Mechanical s predict?

Video game expertise (15%)

what is the Multiple Intelligences Approach?

• Intelligence is the product of many communicating systems

• Intelligence exists in many independent forms.

supporting evidence of Multiple Intelligences Theory

1. Brain damage often impacts a specific ability, but not other abilities

2. Development of systems happens at different ages

3. ‘Gifted’ individuals may possess strong ability in one domain, but not another

Gardner’s 7 (or 8) Intelligences

1. Logical/mathematical***

2. Verbal***

3. Visual-Spatial

4. Intra-personal

5. Social (interpersonal)

6. Body/kinesthetic

7. Musical

8. Naturalistic

Limitations to Multiple Intelligences Theory

1. Multiple intelligences are strongly correlated with each other - should be unique, implies existence of g

2. Multiple intelligences strongly correlate with g!!

3. Subjective theory - different researchers could defend entirely different sets of intelligences

4. Low predictive validity - 10% of elementary grades (vs. 49%)

Components of ‘g’

1. processing speed

2. fluid intelligence

3. crystallized intelligence (knowledge base)

4. general memory & learning

5. broad visual perception

6. broad auditory perception

7. broad retrieval ability

8. broad cognitive speediness

Why is Gardner’s theory criticized as subjective?

Different researchers could define different intelligences.

What does Information Processing Approach study?

The basic mental processes supporting intelligence.

what are the basic mental processes supporting intelligence?

1. Processing speed

2. Acquiring new mental processes/habits

3. Inhibiting previously learned mental processes

4. Knowledge base or crystallized intelligence

processing speed

How quickly you perform mental tasks.

fluid intelligence

• sequential reasoning

• induction

• quantitative reasoning

crystallized intelligence

Knowledge you’ve built up over time.

general memory & learning

• memory span

• associative memory

broad visual perception

• visualization

• spatial relations

• closure speed

broad auditory perception

• speech sound discrimination

• general sound discrimination

broad retrieval ability

• creativity

• ideational fluency

• naming facility

broad cognitive speediness

• rate of test-taking

• numerical facility

• perceptual speed

What factor has the strongest support from evidence in the Info Processing Approach?

Working memory.

What does the other evidence point to in the Info Processing Approach?

3 Factor Model

1. Updating

2. Shifting

3. Inibition

What IQ test is considered the “winner”?

Wechsler Adult Intelligence Scale (WAIS)

What four areas does WAIS-IV measure?

1. Verbal comprehension (g)

2. Perceptual reasoning (g)

3. Working memory (info-processing)

4. Processing speed (info-processing)

What makes a good intelligence test?

Reliability and validity.

reliability

Consistency over time.

validity

Measuring what it claims to measure.

How reliable is WAIS?

Very high (r = .96).

What does WAIS predict besides other IQ tests?

job performance

When does IQ predict job performance best?

In complex jobs requiring decision-making.

What is the key difference between Bem and Simmons in relation to the replication crisis?

• Bem = shows the problem (false/impossible findings can enter the literature).

• Simmons = shows the mechanism (how researchers can accidentally create false findings).

Why are Bem’s and Simmons’s studies often taught together?

Together they explain both what went wrong in psychology and how it happened.

Why was Bem’s study a canary in the coal mine?

It was an early warning sign that the scientific system could publish impossible findings, revealing deeper issues in replicability.

What did Simmons prove?

That common but hidden research practices can produce statistically significant results for things known to be false.