Genetic Algorithms in Machine Learning

Introduction

• Genetic algorithms are used in machine learning to find optimal solutions through an evolutionary process.

• Genes in genetic algorithms represent different characteristics, similar to a list of information in machine learning.

• Genetic algorithms involve creating a population of potential solutions, evaluating their fitness, selecting fit individuals for breeding, and creating offspring with combined genetic material.

• The process includes initialization, fitness calculation, selection of individuals for breeding, pairing individuals to create offspring, introducing random changes through mutation, and replacing the old population with new offspring.

• The goal is to repeat this process over multiple generations to reach an optimal solution.

Main Ideas:

• Genetic algorithms mimic the process of natural selection to find optimal solutions.

• Individuals in the population represent potential solutions encoded as strings of information.

• Fitness scores determine the quality of solutions in solving the problem.

• Selection of fit individuals for breeding simulates the concept of eugenics.

• Offspring are created by combining genetic material from selected individuals.

• Mutation introduces random changes to promote genetic diversity.

• The process is repeated over multiple generations to improve solutions iteratively.

Genetic Algorithms

Genetic Algorithms for Science

• Genetic algorithms evaluate solutions based on fitness functions

  • Fit solutions are selected for breeding

  • New solutions are generated through crossover and mutations

Genetic Algorithms for Everyday Use

• In genetic algorithms for itinerary planning

  • Initial set of itineraries is generated

  • Fitness is calculated based on travel time and costs

  • High fitness itineraries are selected for reproduction

  • Swapping segments of city sequences creates new itineraries

  • Mutations introduce small changes in itineraries

  • Less fit itineraries are replaced with new ones to improve overall quality

Genetic Algorithms for Good

• Genetic algorithms for endangered animal groups

  • Initial paths are set for reaching isolated animal groups

  • Paths are evaluated using fitness functions

  • Unsuitable paths are replaced with better ones

  • Mutation and crossover are applied iteratively to find the best solution

Conclusion

• Genetic algorithms are iterative processes for finding optimal solutions

• They involve evaluating fitness, breeding, mutations, and replacements

• Applied in various scenarios like itinerary planning and reaching endangered animal groups

• Key aspects include understanding the initial population and fitness functions

Neurons On Input

Genetic Algorithms

• Genetic algorithms involve:

  • Choosing initial population randomly or pseudo-randomly

  • Applying fitness function to each population

  • Selecting fit members for the next stage

  • Applying genetic operators like crossover and mutation

  • Repeating the process until acceptable fitness level is reached

• The process continues until a plateau is reached or a maximum number of generations is reached

Neural Networks

• Neural networks are simplified versions of the human brain

• They function similarly to neurons in the brain, recognizing patterns in data

• Neural networks help in identifying categories, predicting outcomes, and finding patterns

• They improve their task completion ability over time by learning from examples

• Neural networks consist of input layer, hidden layer, and output layer

• Input layer accepts data, hidden layer makes decisions, and output layer provides final prediction

• Training is essential for neural networks to accurately recognize patterns

Training Neural Networks

• Scenario: Predicting university admissions based on GPA, SAT score, and number of AP/IB classes

• Input layer consists of neurons representing the criteria

• Output layer predicts acceptance (1) or lack of acceptance (0)

• The number of input and output neurons depends on the criteria and possible results

• Neural networks are represented in code to process data, not physical entities.

The Neural Network

Training the Neural Network

• Data with known output is needed to train the neural network.

• Dataset includes information on students like GPA, SAT score, and number of AP or IB classes.

• Input data into the neural network and expect the output to represent successful students.

Adjusting Network Weights

• Adjust weights of connections between neurons to get the correct output.

• Input layer connected to a hidden layer with each connection having a weight.

• Values from input neurons are processed, multiplied by weights, and sent to hidden neurons.

Training Process Overview

• Feed data into the network through the input layer.

• Make predictions by passing data through the hidden layer to the output layer.

• Compare network's prediction with the correct answer from training data.

• Calculate the error using a cost function to minimize the difference between predicted and correct answers.

Hidden Neuron

Back Propagation Process

• Back propagation is the process where the network adjusts its neurons' calculations based on errors.

• It involves changing the weights inside the network, which act as multipliers for neuron outputs.

• Weights exist between input and hidden neurons, and between hidden layers and output.

Training the Neural Network

• Training involves repeating the process for every piece of data to adjust weights.

• Each cycle through the data is called an epoch.

• Back propagation helps the network improve its task performance.

Network Evaluation

• The network's performance is evaluated to check for improvements in tasks like recognizing images or predicting outcomes.

Neural Network Example

• Input values are represented by mathematical values in the neural network.

• Weights exist between input and hidden layers, and hidden layers and output.

• Neurons' values are multiplied by weights and added to get the final output.

Activation Function and Bias

• A bias value is added to adjust the network's precision.

• The total value is passed through an activation function like sigmoid or relu.

• The activation function helps in getting the final output value after processing through the network.Hidden Neuron Conclusion

Cost Function and Back Propagation

• Cost function addresses the difference between the desired output and the actual output.

• Back propagation adjusts weights and biases based on the cost function to improve predictions.

• Back propagation occurs during the training process to enhance accuracy in predicting outputs.

Terminology for Training

• Weights: Parameters that regulate the strength of input signals.

• Biases: Added to weighted inputs to shift values for better fitting complex patterns.

• Activation Functions: Alter the output based on weighted inputs and biases.

  • Two important activation functions are sigmoid and ReLU.

  • There are various other