ML-Lec12-ANN
Page 1: Introduction
Authors
Prof. Dr. Magdy M. Aboul-Ela (Email: magdy.aboulela@sadatacademy.edu.eg)
Dr. Heba Zaki (Email: heba.zaki@sadatacademy.edu.eg)
Page 2: Introduction to Machine Learning
Key Points
Machine Learning: Involves adaptive mechanisms allowing computers to learn from experience, examples, and comparisons.
Learning Capabilities: Enhance the performance of intelligent systems over time.
Popular Approaches: Include Artificial Neural Networks (ANN) and Genetic Algorithms (GA).
Page 3: Understanding the Brain's Functions
Neurons
Structure: The brain comprises a dense network of neurons, which are information-processing units.
Connections: Each neuron is interconnected, transmitting signals to others.
Signal Strength: The transmission of signals is influenced by the strength of the connections (synapses) between neurons.
Page 4: Overview of Neural Networks
Definition and Inspiration
Artificial Neural Networks (ANN): A subset of machine learning, central to deep learning algorithms, inspired by the human brain structure.
Human Brain: Contains about 10 billion neurons and 60 trillion synaptic connections.
Processing Capability: Enables the brain to perform functions faster than any existing computer.
Page 5: Functionality of Neural Networks
Information Processing
Global Processing: Information is processed simultaneously throughout the network.
Connections: Neurons are linked by weights to transmit signals.
Statistical Models: ANNs are nonlinear statistical models revealing complex input-output relationships.
Applications: Utilized in tasks like image recognition, speech recognition, machine translation, and medical diagnosis.
Page 6: Architecture of ANNs
Structure
Layers: Consist of input, hidden, and output layers, mimicking the functional aspects of the human brain.
Page 7: Analogy Between Biological and Artificial Neural Networks
Neuron as a Computing Element
Overview of the biological neuron structure and its function within a network.
Page 8: Detailed Architecture of ANNs
Layer Functions
Input Layer: Receives input data (text, numbers, audio, images).
Hidden Layers: Perform mathematical computations and pattern recognition.
Output Layer: Produces the final results based on the computations.
Parameters Influencing Performance: Include weights, biases, learning rate, etc.
Page 9: Calculating Outputs
Node Operations
Weighted Links: Each node has weights that influence output signals.
Transfer Function: Computes the weighted sum of inputs and biases.
Activation Function: Determined the final output based on the transfer function results.
Page 10: Activation Mechanism
Firing Mechanism
Activation Trigger: If the output exceeds 0.5, the activation function outputs a 1; otherwise, it outputs 0.
Page 11: Popular Activation Functions
Examples
Overview of commonly used activation functions in ANNs.
Page 12: Definition of Neural Network
Characteristics
Neural Network: Composed of parallel processing units (neurons) capable of acquiring, storing, and utilizing knowledge.
Single Neuron Learning: Explains the first training method using a perceptron.
Perceptron Structure: Simplest ANN model consisting of a single neuron with adjustable weights and a limiter.
Page 13: Single-Layer Perceptron
Structure Illustration
Components: Represents inputs, threshold, and outputs of a basic perceptron model.
Page 14: Multilayer Perceptron
Layers Configuration
Structure: Involves an input layer, two hidden layers, and an output layer.
Page 15: Perceptron Model
Operation
Components: Includes a linear combiner followed by a hard limiter to produce binary outputs (e.g., +1 or -1).
Types of Perceptrons: Single-layer and multilayer feed-forward perceptrons.
Page 16: Learning Process of Perceptron
Weight Adjustments
Adjustment Mechanism: Involves modifying weights to minimize actual vs. desired outputs.
Initial Weights: Typically start within the range [-0.5, 0.5].
Page 17: Feed-Forward Neural Networks
Information Flow
Direction of Flow: Information moves only from input to hidden to output layer without feedback loops.
Page 18: Back-Propagation Neural Networks
Layers and Learning
Structure: Can have multiple layers, including several hidden layers, processing millions of neurons.
Learning Process: Similar to perceptron, involves input pattern, output computation, and error correction.
Page 19: Learning Algorithm Phases
Two Main Phases
Forward Phase: Input pattern moves through layers to generate an output.
Backward Phase: If output differs from the desired pattern, errors are calculated and weights are adjusted accordingly.
Page 20: Three-Layer Back-Propagation Network
Structure Overview
Illustrates input, hidden, and output layers with respective signals and error signals.
Page 21: Training Back-Propagation Network
Training Steps
Initialize network with random weights.
For each training case:
Present inputs, compute output.
Adjust weights based on output error.
Page 22: Unused Page Content
Page 23: Unused Page Content
Page 24: Unused Page Content
Page 25: Unused Page Content
Page 26: Back-Propagation Training Algorithm
Initialization
Weight Setting: Randomly distribute weights and thresholds within a small range.
Page 27: Activation Step
Hidden Layer Output Calculation
Formulas: Calculate neuron outputs in the hidden layer using sigmoid activation function.
Page 28: Output Layer Calculation
Output Neuron Operations
Formulas: Determine outputs for neurons in the output layer.
Page 29: Weight Training Process
Error Gradient Calculation
Output Layer: Calculate weight corrections and update based on errors.
Page 30: Hidden Layer Adjustments
Error Correction in Hidden Layers
Error Gradients: Derive corrections for weights in the hidden layer based on outputs and errors.
Page 31: Iterative Learning
Repeated Process
Iteration: Increase iteration count and repeat until the convergence criterion is met.
Page 32: XOR Operation and Network Design
Application Example
Network Setup: Training set and randomly set weights to perform logical operations.
Page 33: Calculating Neuron Outputs
Error Evaluation
Compare Outputs: Determine actual outputs, calculate errors for training.
Page 34: Weight Training Updates
Propagating Errors
Weight Updates: Adjust weights and thresholds based on the determined errors from the output layer backwards.
Page 35: Hidden Layer Gradients
Final Adjustments
Update Process: Include final calculations to adjust weights in all layers based on accumulative errors.
Page 36: Convergence of Training
Final Updates
Continue Training: Process repeats until total error is minimized.