reading ass. 6

CHAPTER 4: ARTIFICIAL NEURAL NETWORKS

4.1 Introduction

  • Artificial Neural Networks (ANNs): Used for learning real-valued, discrete-valued, and vector-valued functions from examples.

  • Learning Algorithm: BACKPROPAGATION utilizes gradient descent to adjust network parameters for optimal performance on a training set.

  • Applications: ANNs have successfully tackled challenges like visual data interpretation, speech recognition, and robotic control strategies.

  • Examples of Success:

    • Handwritten character recognition (LeCun et al. 1989)

    • Spoken word recognition (Lang et al. 1990)

    • Facial recognition (Cottrell 1990)

4.1.1 Biological Motivation

  • Inspired by biological learning systems made of interconnected neurons.

  • ANNs simulate this with interconnected simple units that take multiple real-valued inputs and produce a single output.

  • Human Brain: Approximately 10^11 neurons interconnected in complex ways.

    • Neuron firing time: 10^{-3} seconds, considerably slower than computer switching speeds.

    • Speculated that human cognitive abilities stem from parallel processes spread across networks of neurons.

  • Comparison with ANN: ANNs do not fully encapsulate biological neural properties; many complexities of biological systems are not modeled here.

    • Research groups explore either biological modeling or efficient machine learning algorithms.

4.2 Neural Network Representations

  • ALVINN System (Pomerleau 1993): Example of an ANN that commands an autonomous vehicle using learned steering commands based on pixel intensity input.

    • Input: 30x32 grid pixels from a camera.

    • Output: Steering direction.

    • Successfully drove at high speeds (up to 70 mph).

4.3 Appropriate Problems for Neural Network Learning

  • Best suited for problems involving noisy, intricate sensor data, such as image and audio inputs.

  • Also comparable to symbolic representations like decision trees.

4.4 Perceptrons

  • Functionality of a Perceptron:

    • Takes multiple real-valued inputs to produce a binary output based on a threshold.

    • Can be modified to represent AND, OR, NAND functions, but cannot represent XOR.

  • Learning in Perceptrons:

    • Implemented using the Perceptron Training Rule and Delta Rule (Gradient Descent).

4.5 Multilayer Networks and the BACKPROPAGATION Algorithm

  • Allows representation of complex, nonlinear decision surfaces as opposed to single-layer perceptrons.

  • BACKPROPAGATION: Algorithm for training multilayer networks utilizing gradient descent to minimize output errors.

4.8 Advanced Topics in Artificial Neural Networks

4.8.1 Alternative Error Functions

  • Analysis of gradient descent can adapt to various error definitions to refine ANN learning capabilities by adjusting weight-tuning rules.

4.9 Summary and Further Reading

  • ANNs provide robust learning frameworks adaptable to various datum types, maintaining effectiveness despite training noise.

  • The BACKPROPAGATION method is foundational in ANN applications, allowing for a range of practical implementations across different domains.

Further Reading

  • Topics like overfitting, robust algorithm learning, and alternative ANN architectures are further discussed in Chapters 5, 6, and 12.