Melanie Mitchell - The Roots of AI

What event is generally considered the official founding of artificial intelligence?

The 1956 Dartmouth workshop organized primarily by John McCarthy.

Who coined the term "artificial intelligence"?

John McCarthy.

What was the Dartmouth AI workshop?

A 1956 summer workshop intended as a two-month study of artificial intelligence that helped establish AI as a field and outline its major research goals.

Who were the four organizers of the Dartmouth AI workshop?

John McCarthy, Marvin Minsky, Claude Shannon, and Nathaniel Rochester.

What was the central assumption behind the Dartmouth proposal?

That every aspect of learning or intelligence could, in principle, be described precisely enough that a machine could be made to simulate it.

What AI topics were identified in the Dartmouth proposal?

Natural-language processing, neural networks, machine learning, abstract concepts and reasoning, and creativity.

Who were the "big four" early AI pioneers discussed by Mitchell?

John McCarthy, Marvin Minsky, Allen Newell, and Herbert Simon.

What characterized early AI researchers' expectations?

They were extremely optimistic and often predicted that human-level or general machine intelligence would be achieved within a few decades.

What is a "suitcase word"?

Marvin Minsky's term for a word that contains many different meanings, such as intelligence, thinking, cognition, consciousness, and emotion.

Why is intelligence difficult to define?

It can be viewed as binary, continuous, or multidimensional and can refer to many different abilities such as verbal, emotional, spatial, logical, artistic, and social intelligence.

What are the two broad goals of AI described by Mitchell?

A scientific goal of understanding natural/biological intelligence by reproducing it in computers, and a practical goal of building computers that perform tasks as well as or better than humans.

What is the "anarchy of methods" in AI?

The existence of many competing approaches to creating intelligence because researchers do not fully understand intelligence or know the single best way to produce general AI.

What are three major approaches within AI's "anarchy of methods"?

Deductive/logical approaches, inductive/statistical approaches, and approaches inspired by biology and psychology.

What is the relationship among AI, machine learning, and deep learning?

AI is the broad field; machine learning is a subfield of AI in which systems learn from data or experience; deep learning is one family of methods within machine learning.

Is deep learning the same thing as artificial intelligence?

No. Deep learning is one approach within machine learning, which itself is a subfield of the broader field of AI.

What is symbolic AI?

An approach in which knowledge is represented using human-understandable symbols and explicit rules for manipulating those symbols.

What inspired symbolic AI?

Mathematical logic and descriptions of conscious human reasoning.

What is the General Problem Solver (GPS)?

An early symbolic AI program created by Herbert Simon and Allen Newell that attempted to solve problems by manipulating symbolic representations of states according to rules and operators.

Who created the General Problem Solver?

Herbert Simon and Allen Newell.

How does the General Problem Solver work in general?

It represents a current state and desired state and uses operators and rules to transform the current state step-by-step toward the desired state.

What are operators in the General Problem Solver?

Actions or subprograms that transform the current state into a new state.

What role do rules play in symbolic AI systems like GPS?

Rules encode the constraints that determine which operations are permitted.

Does a symbolic AI system necessarily understand the meaning of its symbols?

No. The computer can manipulate symbols correctly without understanding what those symbols mean.

What gives symbols "meaning" to the General Problem Solver?

Their relationships to other symbols and the ways they can be combined and operated on, rather than human-like understanding.

What were expert systems?

Symbolic AI programs in which human experts created rules that computers used to perform specialized tasks such as medical diagnosis or legal decision-making.

How long did symbolic AI dominate early AI research?

Roughly the first three decades of the field.

What is subsymbolic AI?

An approach inspired by neuroscience in which knowledge is encoded through numerical operations and learned parameters rather than explicit human-understandable symbols and rules.

What kinds of human abilities inspired subsymbolic AI?

Fast and sometimes unconscious perceptual abilities such as recognizing faces, objects, and spoken words.

What is the major difference between symbolic and subsymbolic AI?

Symbolic AI represents knowledge using explicit symbols and rules, whereas subsymbolic AI represents learned knowledge through numerical values and interactions that are often difficult for humans to interpret.

Who invented the perceptron?

Psychologist Frank Rosenblatt in the late 1950s.

What is a perceptron?

A simple brain-inspired computational system that makes a yes-or-no decision by comparing the sum of weighted inputs with a threshold.

Why are perceptrons historically important?

They were an important early form of subsymbolic AI and are ancestors of modern artificial neural networks and deep learning.

What biological structure inspired the perceptron?

A neuron.

How does a biological neuron roughly process information?

It receives inputs through connections of different strengths, combines those inputs, and fires when the combined input reaches a sufficient threshold.

What is a weight in a perceptron?

A numerical value assigned to an input that determines how strongly that input influences the perceptron's decision.

What is the threshold of a perceptron?

The value the sum of weighted inputs must reach or exceed for the perceptron to output 1.

When does a perceptron output 1?

When the sum of its weighted inputs is equal to or greater than its threshold.

When does a perceptron output 0?

When the sum of its weighted inputs is below its threshold.

What type of decision does a basic perceptron make?

A binary or yes/no decision represented by 1 or 0.

How does Mitchell demonstrate perceptron image recognition?

She describes a perceptron designed to determine whether a handwritten digit is an 8.

What does output 1 mean in Mitchell's handwritten-8 example?

The perceptron predicts that the image is an 8.

What does output 0 mean in the handwritten-8 example?

The perceptron predicts that the image is not an 8.

How are images converted into perceptron inputs?

Each pixel is represented by a numerical intensity value and becomes an input to the perceptron.

How many inputs does Mitchell's 18 × 18 pixel perceptron have?

324 inputs because 18 × 18 = 324.

How is knowledge represented in a perceptron?

Through its numerical weights and threshold rather than explicit symbolic rules.

What is supervised learning?

A learning method in which a system receives labeled examples, produces outputs, compares them with correct answers, and adjusts itself based on its errors.

What psychological idea influenced Rosenblatt's approach to perceptron learning?

B. F. Skinner's behaviorist idea of learning through conditioning, reinforcement, and repeated trials.

What is a supervision signal?

Information telling the learning system how its output differs from the correct output.

What are positive examples in the handwritten-8 task?

Images labeled as 8s.

What are negative examples in the handwritten-8 task?

Images of handwritten digits other than 8.

What is a training set?

The collection of labeled examples used to train a machine-learning system.

What is a test set?

A separate collection of examples used after training to evaluate how well the system performs on examples it has not already been trained on.

Why is the test set important?

It tests whether the system has learned to generalize to new examples rather than simply performing well on its training examples.

What is an algorithm?

A recipe or sequence of steps that a computer follows to solve a particular problem.

What was Rosenblatt's major algorithmic contribution?

The perceptron-learning algorithm, which allows a perceptron to learn appropriate weights and a threshold from examples.

How does the perceptron-learning algorithm begin?

The weights and threshold are initially assigned random values.

What happens when a perceptron makes a correct prediction during training?

Its weights and threshold remain unchanged.

What happens when a perceptron makes an incorrect prediction during training?

Its weights and threshold are adjusted slightly to make the correct answer more likely in the future.

Why are weights adjusted gradually during perceptron training?

Large changes based on individual examples could cause the system to learn incorrect generalizations; gradual learning over many trials is more reliable.

What happens after the perceptron repeatedly processes the training examples?

Its weights and threshold gradually settle on values intended to classify the training examples correctly.

How is a trained perceptron ultimately evaluated?

It is tested on examples in the test set that it did not encounter during training.

Why is subsymbolic AI harder to interpret than symbolic AI?

Its knowledge is stored in numerical parameters such as weights rather than explicit human-readable concepts and rules.

Why does interpretability become especially difficult with modern neural networks?

Modern neural networks can contain millions of learned weights, making it extremely difficult to translate their internal numerical operations into human-understandable rules.

What do subsymbolic AI advocates argue about symbols and intelligence?

They argue that intelligent symbolic abilities should emerge from neural-like architectures rather than having all symbols and rules explicitly programmed.

Who wrote the 1969 book Perceptrons?

Marvin Minsky and Seymour Papert.

What limitation did Minsky and Papert demonstrate about simple perceptrons?

The class of problems simple perceptrons could solve perfectly was limited, and the perceptron-learning algorithm did not scale well to tasks requiring many weights and thresholds.

What is a multilayer neural network?

A neural network with additional layers of simulated neurons, allowing it in principle to solve a much broader range of problems than a simple perceptron.

Why weren't multilayer neural networks widely developed at that time?

Researchers lacked a general algorithm comparable to the perceptron-learning algorithm for effectively learning the weights and thresholds of multilayer networks.

What happened to neural-network research in the late 1960s and early 1970s?

Funding declined substantially, Rosenblatt died in 1971, and perceptron/subsymbolic research largely halted outside a few isolated research groups.

What is an AI winter?

A period in which disappointment with AI's progress causes funding, investment, and research activity to decrease.

What is an AI spring?

A period of renewed excitement, optimism, investment, and research activity surrounding new AI ideas or breakthroughs.

What recurring AI hype cycle does Mitchell describe?

New ideas create optimism → researchers and media promise major breakthroughs → funding increases → results fail to meet expectations → funding and investment decline → an AI winter occurs.

Why did an early AI winter occur in the 1970s?

Researchers had promised major advances in areas such as language, commonsense reasoning, robotics, and autonomous systems, but general AI breakthroughs failed to materialize, leading governments to cut funding.

What kinds of AI systems did show some success before the AI winter?

Narrowly focused expert systems designed for specialized problem domains.

What lesson did John McCarthy draw from early AI research?

AI was harder than researchers initially thought.

What does "easy things are hard" mean in AI?

Abilities that humans perform effortlessly, such as perception, language understanding, and learning concepts from few examples, can be extremely difficult for computers.

What are examples of supposedly difficult tasks that computers have performed successfully?

Playing chess and Go, solving complex algebraic problems, and performing some forms of disease diagnosis.

What are examples of "easy" human abilities that have historically been difficult for AI?

Natural-language conversation, visual understanding, recognizing everyday objects, and learning new concepts from only a few examples.

Why is the "easy things are hard" paradox important?

It demonstrates that many effortless human abilities rely on extremely complex mental processes that humans themselves are often barely aware of.

What larger lesson does Mitchell draw from attempts to create AI?

Trying to reproduce intelligence in machines has revealed how complex and subtle human intelligence actually is.

Symbolic AI vs. subsymbolic AI

Symbolic AI uses explicit human-understandable symbols and rules; subsymbolic AI learns numerical representations such as weights and is generally harder for humans to interpret.

General Problem Solver vs. perceptron

GPS is symbolic and follows explicitly programmed symbols, operators, and rules; a perceptron is subsymbolic and learns numerical weights and a threshold from examples.

Training set vs. test set

The training set teaches the system; the test set evaluates how well the trained system generalizes to previously unseen examples.

Major theme: competing approaches

AI has never had one universally accepted method; symbolic logic, statistical learning, and brain-inspired approaches have competed throughout the field's history.

Major theme: understanding human intelligence

Attempts to build artificial intelligence have revealed that seemingly effortless human abilities depend on remarkably complex processes.