Introduction to Neural Networks and Learning Algorithms

These flashcards cover key concepts, definitions, and questions related to neural networks and their learning algorithms.

What is a perceptron?

A simple computational unit modeled after a neuron that receives multiple inputs, assigns weights, sums them, and fires if a threshold is reached.

What is supervised learning?

Learning from labeled examples where the system compares its output to the correct answer and adjusts weights to reduce error.

What is subsymbolic AI?

An approach to AI where intelligence emerges from patterns of activity and weight changes rather than explicit symbols or rules.

What defines a multi-layer neural network?

A network of perceptron-like units organized into an input layer, one or more hidden layers, and an output layer.

What is a hidden unit in a neural network?

A unit that is neither an input nor an output and learns intermediate representations not explicitly programmed.

What is activation in the context of neural networks?

A unit’s output value indicating how strongly it is responding, typically a continuous value rather than a binary on/off signal.

What is backpropagation?

A learning algorithm that reduces error by sending output errors backward through the network to adjust weights at all layers.

What is distributed representation?

Knowledge stored across many connections in a network rather than in explicit rules or symbols.

What is pattern transformation?

The idea that cognition involves transforming patterns of activity instead of manipulating symbols.

What does graceful degradation refer to in neural networks?

The ability of a neural network to continue functioning even when parts are damaged or inputs are noisy.

How does a perceptron resemble a biological neuron?

It receives multiple inputs, weights them, sums them, and fires only if activation reaches a threshold.

How does a perceptron learn to recognize handwritten digits like '8'?

It is trained on labeled examples and adjusts its weights through supervised learning.

What does Mitchell say about the rules of a perceptron?

The perceptron’s 'rules' are embedded in numerical weights, not in explicit, human-understandable rules.

Why is a multi-layer neural network advantageous?

It can learn complex patterns beyond the capabilities of a single perceptron.

What is the significance of a hidden unit?

It detects intermediate patterns and allows the network to form internal representations.

What does a unit's activation represent?

The degree to which a unit is active, represented as a graded continuous value.

How does backpropagation function?

It computes error at the output layer and sends it backward to adjust weights at all layers.

Why is defining the vowel 'a' difficult?

It varies across speakers, contexts, and environments, lacking a single defining acoustic feature.

How do neural networks learn to recognize the vowel 'a'?

By training on many examples, establishing 'a' as a region in activation space.

Why is NetTalk significant according to Churchland?

It learns pronunciation without explicit rules, showing that linguistic competence can emerge from distributed learning.

What features of NetTalk support the idea of connectionism?

Emergent structure, distributed knowledge, graceful degradation, and learning through error correction.

What are the key differences between connectionist networks and the brain's microstructure?

Brains are biologically complex and use multiple learning mechanisms, while ANNs are simplified and algorithm-driven.

Why does Churchland believe differences do not undermine connectionism?

Because ANNs capture essential cognitive features like learning, pattern sensitivity, and distributed representation.