Narayanan and Kapoor - AI Snake Oil

What is the main argument of the introduction to AI Snake Oil?

"Artificial intelligence" is an umbrella term covering many different technologies, so we need to distinguish among types of AI and evaluate what each can and cannot actually do rather than treating AI as one technology.

What analogy do Narayanan and Kapoor use to explain the problem with the term "AI"?

They imagine a world where cars, buses, bicycles, trucks, and spacecraft are all simply called "vehicles," making meaningful discussion difficult because very different technologies are grouped under one term.

Why is the word "AI" potentially misleading?

AI refers to many loosely related technologies that differ in how they work, how they are used, who uses them, and how they fail.

What example demonstrates how different technologies can both be called AI?

ChatGPT and software used by banks to evaluate loan applicants are both called AI even though they work differently and serve very different purposes.

What are two major categories of AI emphasized in the introduction?

Generative AI and predictive AI.

What is generative AI?

AI that generates new content such as text, images, speech, music, or other media.

What are examples of generative AI?

ChatGPT and other chatbots, DALL-E, Stable Diffusion, Midjourney, and systems that generate speech or music.

How do the authors characterize recent progress in generative AI?

The progress is genuine and remarkable, but generative AI products are still immature, unreliable, and prone to misuse.

What is predictive AI?

AI that attempts to predict future outcomes in order to guide decisions being made in the present.

What are examples of predictive AI?

Systems predicting crime, machinery failure, job performance, healthcare needs, insurance behavior, or whether a defendant will commit another crime.

What can predictive AI sometimes do effectively?

It can analyze large datasets and identify broad statistical patterns.

What is the major problem with predictive AI according to the authors?

It is often marketed as being able to accurately predict individual human behavior and social outcomes even though these outcomes are extremely difficult to predict.

What is AI snake oil?

AI that does not and cannot work as advertised.

Where do the authors argue much AI snake oil is concentrated?

Predictive AI, especially systems claiming to predict consequential outcomes about individual people.

Why is AI snake oil a societal problem?

People often lack the knowledge needed to distinguish AI that can actually function as promised from AI making implausible or unsupported claims.

What is one major goal of AI Snake Oil?

To give readers vocabulary and reasoning tools for distinguishing useful AI from hype and evaluating whether claims about AI are plausible.

What do the authors argue about predicting human behavior?

They argue that predictive AI faces inherent limitations because individual human behavior and life outcomes are extremely difficult to predict.

When did ChatGPT burst into widespread public awareness?

After its release in November 2022.

How did OpenAI initially view ChatGPT's release?

As a research preview rather than a major product launch.

Why did ChatGPT become popular so quickly?

People shared entertaining and impressive examples of its ability to generate realistic and creative responses to prompts.

Approximately how many users did ChatGPT reportedly reach within two months?

More than 100 million users.

How did programmers begin using ChatGPT?

They used it to generate snippets of programming code from natural-language descriptions, accelerating some software-development tasks.

What coding product similar to ChatGPT existed before its release?

GitHub Copilot.

What happened after Microsoft integrated OpenAI technology into Bing?

Google viewed the development as a major competitive threat and quickly announced its own chatbot, Bard, which was later renamed Gemini.

What famous error appeared in Google's Bard promotional demonstration?

Bard incorrectly claimed that the James Webb Space Telescope took the first picture of a planet outside our solar system.

What did the Bard error demonstrate?

Generative AI can produce fluent and convincing responses that contain factual errors.

Why can chatbots produce incorrect factual information?

They learn statistical patterns from training data and generate text based on those patterns, which does not guarantee that the generated information is factually correct.

Does a convincing chatbot answer necessarily mean the answer is correct?

No. Fluency and plausibility do not guarantee factual accuracy.

What are examples of harmful misuse of generative AI described in the reading?

Error-filled AI-generated news and financial advice and AI-generated mushroom-foraging books where incorrect information could be dangerous.

Why is generative AI especially conducive to misuse?

It makes producing large quantities of content extremely easy and cheap, while people trying to profit quickly may have little incentive to verify whether that content is accurate.

Do Narayanan and Kapoor believe chatbots are useless?

No. They believe most knowledge industries can benefit from chatbots when the tools are used carefully and users understand their limitations.

How do the authors personally use chatbots?

For research assistance, formatting citations, and understanding jargon-heavy papers in unfamiliar research areas.

What is the "catch" when using chatbots productively?

Using them effectively while avoiding their pitfalls requires effort, practice, and verification.

What power issue can arise when search engines replace links with AI-generated answers?

AI systems may rewrite information created by other websites without directing users, traffic, or revenue back to the original creators.

Why does generative AI create copyright concerns?

AI-generated answers and content can be based on existing creators' work while potentially avoiding the normal mechanisms that compensate or direct attention to those creators.

What is text-to-image generation?

Generative AI technology that creates images based on written descriptions.

What labor concern about AI arose during the 2023 Hollywood strikes?

Actors worried studios could train AI on existing recordings of them and generate new performances using their likenesses without appropriate compensation.

Why are generative AI issues also issues of power and labor?

The technology can change who controls creative work, who receives compensation, and who benefits economically from people's previous labor.

What do the authors think could reduce the long-term harms of generative AI?

A combination of technological improvements and laws or regulations.

How do the authors characterize generative AI in the short term?

Highly capable but unreliable, meaning society must carefully learn how to integrate it into everyday life.

What technology do the authors compare AI's introduction into education with?

The calculator.

How do the authors' views of generative AI and predictive AI differ?

They are cautiously optimistic about generative AI's long-term benefits but much more skeptical of predictive AI used to predict human behavior and social outcomes.

What is the main weakness of predictive AI according to the authors?

It often claims to predict individual future outcomes that are inherently difficult or impossible to predict accurately.

How has predictive AI been used in healthcare?

For example, Medicare providers have used AI to estimate how long patients will require hospital or nursing care.

What happened in the Medicare example involving an 85-year-old patient?

AI predicted she would be ready to leave care after seventeen days, but she remained in severe pain and unable to use a walker independently; nevertheless, her insurance payments stopped based on the assessment.

What does the Medicare example illustrate?

Even predictive systems introduced for reasonable purposes can cause harm when inaccurate predictions are used rigidly in high-stakes decisions.

How can the goals of an AI system change after deployment?

A system initially created for accountability or efficiency can eventually be used primarily to cut costs or achieve other goals regardless of the human consequences.

How is predictive AI used in hiring?

Some companies claim AI can infer personality traits or future job performance from features such as body language and speech patterns in short videos.

What problem do the authors identify with AI hiring claims?

Companies making these claims often fail to provide verifiable evidence that their systems can accurately predict job performance.

Why is predicting individual life outcomes difficult?

Important information may be unavailable, human behavior is complex, circumstances change, and measurable statistical patterns may not reliably predict what one particular person will do.

What happened in the Allstate predictive-pricing example?

A predictive system identified customers whose insurance rates could potentially be increased substantially without causing them to leave.

What was the "suckers list"?

A group of insurance customers whose rates could be raised dramatically compared with their previous rates while they were predicted to remain customers.

Which group was disproportionately represented in Allstate's "suckers list"?

People over age 62.

What does the Allstate example demonstrate about prediction and ethics?

A prediction can potentially increase profits or identify real statistical patterns while still being discriminatory or ethically unacceptable.

Can an algorithm discover a statistically useful pattern that is morally problematic to use?

Yes. Statistical usefulness does not automatically make a decision fair or ethical.

How is predictive AI used in criminal justice?

Risk-prediction systems can estimate whether defendants are likely to commit future crimes and influence decisions such as pretrial release.

What biases have been documented in criminal risk-prediction systems according to the reading?

Racial bias, gender bias, and ageism.

Why can missing information undermine predictive AI?

The algorithm can only use information available to it, while important factors affecting a person's future behavior may be unavailable or impossible to measure reliably.

What example illustrates the missing-information problem in criminal risk prediction?

Three defendants could have identical measurable characteristics and receive the same risk score even though one is remorseful, another was wrongly arrested, and another intends to commit another crime.

How can predictive systems create perverse incentives?

People may change their behavior in response to how a prediction system rewards or penalizes them, sometimes producing incentives opposite to the system's intended goal.

What was the kidney-transplant predictive AI example?

A proposed system would prioritize people predicted to live longest after transplantation, but this could perversely reward patients whose kidneys failed at younger ages.

What happened to the proposed predictive system for kidney-transplant matching?

Stakeholders including patients and doctors recognized the incentive problem, and the use of predictive AI for the matching system was abandoned.

What tension exists between automation and accountability?

Automation can make decision-making more efficient, but that efficiency can reduce individualized human judgment and make accountability more difficult.

How should claims from predictive-AI companies be treated according to the authors?

With skepticism unless the claims are supported by strong evidence.

Is there a universally accepted definition of artificial intelligence?

No. There is no consensus about exactly what should or should not be labeled AI.

What three questions do the authors suggest for thinking about whether something qualifies as AI?

Whether the task requires creativity or training for humans; whether the system's behavior was directly programmed or emerged indirectly through learning/search; and whether the system acts autonomously with flexibility or adaptability.

What is the first possible criterion for determining whether something is AI?

Whether the task normally requires creativity, skill, or training for a human to perform.

Why is human difficulty not a perfect criterion for defining AI?

Some tasks that are effortless for humans, such as recognizing objects in images, were extremely difficult for computers but are still considered AI.

What is the second possible criterion for determining whether something is AI?

Whether the system's behavior emerged indirectly through learning from examples or searching data rather than being completely specified by programmers.

What is machine learning?

A form of AI in which a computer learns behavior or patterns from examples or data rather than having every behavior directly programmed.

What is the third possible criterion for determining whether something is AI?

Whether the system makes decisions relatively autonomously and demonstrates flexibility or adaptability to its environment.

Why isn't autonomy alone enough to define AI?

Simple systems can react automatically to their environment without being considered intelligent, such as a traditional mechanical thermostat.

What factors besides technical characteristics influence whether something gets called AI?

Historical usage, marketing, and social convention.

What does the humorous definition "AI is whatever hasn't been done yet" mean?

Once an AI technology becomes reliable, ordinary, and widely accepted, people tend to stop thinking of it as artificial intelligence.

What technologies were once considered difficult AI problems but are now ordinary?

Robot vacuum cleaners, airplane autopilot, phone autocomplete, handwriting recognition, speech recognition, spam filtering, and spell-check.

Why does the book focus heavily on problematic AI?

Successful and reliable AI often becomes ordinary and fades into the background, whereas unreliable or misleading AI creates more visible social problems.

Statistical pattern vs. individual prediction

A broad pattern in a population can be real and useful without allowing an AI system to accurately predict what will happen to a particular individual.

Accuracy vs. fairness

A system could identify a statistically or economically useful pattern while still producing unfair or discriminatory decisions.

Major theme: AI is not one thing

Different technologies are all labeled AI even though they operate differently, serve different purposes, and have different limitations.

Major theme: evidence over hype

AI claims should be evaluated according to strong evidence about what a particular system actually does rather than accepting claims simply because the product uses AI.

Major theme: generative AI

The technology represents genuine progress and can be useful, but it remains unreliable, can fabricate information, and is easy to misuse.

Major theme: predictive AI

Predictive systems are especially concerning when they claim to forecast individual human behavior or social outcomes because these outcomes are extremely difficult to predict.

Major theme: social consequences

Evaluating AI requires considering not only technical accuracy but also discrimination, incentives, labor, copyright, power, accountability, and who benefits or is harmed.

Central takeaway of AI Snake Oil

Do not ask whether "AI" works in general. Identify the specific type of AI, understand what it claims to do, examine the evidence supporting that claim, and consider the consequences of using it.