Machine Learning Study Notes

Introduction to Machine Learning

• Machine Learning (ML): Constructing programs that improve performance at tasks through experience.

  • Objective: Create adaptive systems for accurate predictions.

Well-posed Machine Learning Problem

• Learning from experience: Performance improves with experience.

  • Learning Task: Specific problems to address.

  • Measure of Performance: Criteria for evaluating model (e.g., accuracy).

  • Task Experience: Data involved in the learning process.

Learning from Experience

• Key behaviors:

  • Remembering: Use past actions for future decisions.

  • Adapting: Modify actions based on past results.

  • Generalizing: Apply knowledge across different contexts.

Machine Learning Applications

• Medical Diagnosis: Assess diseases from patient data.

• Drug Design: Explore databases of potential drugs.

• Speech Recognition: Convert spoken language to text.

• Natural Language Processing (NLP): Machines interpret human language.

• Fault Diagnostics: Identify software/machinery problems.

• Business Intelligence: Analyze data for decision-making.

Data Representation

• Experience in raw data form requires preprocessing.

  • Attributes: Data characteristics.

  • Instances: Rows in dataset.

  • Features: Columns representing attributes.

Types of Attributes

• Nominal: Categories without order (e.g., eye color).

• Ordinal: Categories with order (e.g., movie ratings).

• Interval: Numeric values without true zero (e.g., temperature).

• Ratio: Numeric values with true zero (e.g., weight).

Big Data in Machine Learning

• Big Data: Extraction of patterns from large data volumes.

Need for Machine Learning

• Used when:

  • Lack of Human Expertise: e.g., navigation on Mars.

  • Insufficient Human Understanding: e.g., speech recognition.

  • Dynamic Solutions Needed: e.g., network routing.

  • User-specific Adaptations Needed: e.g., biometrics.

Machine Learning Process Overview

• Unseen Data: New, unencountered data.

• Training Set: Data used for model training.

• ML Algorithm: Method for learning from training data.

• ML Model (Hypothesis): Model used for predictions.

Supervised Learning Paradigms

• Supervised Learning: Learning with labeled data.

• Unsupervised Learning: Finding patterns in unlabeled data.

• Reinforcement Learning: Based on environment interaction for rewards.

Logistic Regression Model

• Utilizes sigmoid function to map values to probabilities: $g(z) = \frac{1}{1 + e^{-z}}$

Cost Functions in Machine Learning

• Mean Squared Error (MSE): $J(w) = \frac{1}{m} \sum_{i=1}^{m} \text{error}$

• Cross-Entropy Loss Function: $L = -y \ln(\hat{y}) - (1 - y) \ln(1 - \hat{y})$

Conclusion

• Machine learning leverages data and algorithms to provide insights and automation across various fields.