1.ML-Ingredients

Introduction to Machine Learning System Ingredients

  • Overview of ingredients: 5 essential components in a machine learning system.

Problem Illustration: Classifying Cats and Dogs

  • Example of a binary classification problem.

  • Supervised learning using labeled training data (images of cats and dogs).

  • Goal: Classify unseen images as either cat or dog.

Key Components of the System

1. Input

  • Input can vary in complexity; in this case, it is an image.

  • Need to convert the image into a format suitable for the system.

  • Images represented as a matrix of pixel values.

    • Example: A 28x28 image corresponds to 784 pixel values (P1 to P784).

2. Output

  • For binary classification, there are two outputs:

    • Probability of belonging to the 'Dog' class.

    • Probability of belonging to the 'Cat' class.

  • Probabilities range from 0 to 1:

    • If the probability of 'Dog' is 0.7, then 'Cat' will be 0.3 (1 - 0.7 = 0.3).

    • Two output values simplify extension to multi-class classification.

3. Mapping Function

  • The mathematical function that maps inputs to outputs.

  • Also referred to as the hypothesis (F).

  • Different classification techniques (logistic regression, SVM, neural networks) will have different hypotheses.

  • The core function remains: mapping input (784 vector) to output (class probabilities).

4. Cost Function

  • Measures the error between predicted outputs and true outputs.

  • Also known as the loss function.

  • Essential for evaluating the performance of the mapping function.

    • True labels known from training data allows calculation of errors based on predictions.

  • Provides a mechanism to assess how well the system is performing.

5. Learning Procedure

  • The iterative process of minimizing the cost function by adjusting the mapping function.

  • Introduces flexibility into the mapping function via trainable parameters (W):

    • Changes in W result in changes in the mapping function, enabling minimization of error.

  • The goal of the learning process is to find parameter values (W) that reduce the cost function and thereby improve classification accuracy.

Example of Learning Process

  • Illustrative binary classification problem with two features (S1 and S2).

  • Hypothesis involves finding a straight line to separate classes.

    • W represents slope and b represents intercept.

  • Initial guess might yield high error; iterative adjustments lead to improved mappings:

    • Adjust W and b to minimize the cost function.

    • The process continues until an optimal line (and parameters) is found, resulting in minimal error.

Conclusion

  • Learning in machine learning involves minimizing error on training data by dynamically adjusting the mapping function through trainable parameters.

  • Common practice in programming is using a 'fit' method (e.g., logistic regression, etc.) to perform this learning process.