CONCEPTS IN MACHINE LEARNINGPPT1

Artificial Intelligence (AI)

• Branch of computer science aiming to create machines capable of mimicking human intelligence.
  • Targets thinking, learning, problem-solving, language understanding, perception, etc.
• Serves as the umbrella field that contains Machine Learning (ML) as a subset.

Machine Learning (ML)

Core Definition & Historical Context

• Subset of AI that provides systems the ability to learn and improve automatically from experience without explicit programming.
• Arthur Samuel (IBM, 1959) coined the term “Machine Learning” and defined the field as “the study that gives computers the ability to learn without being explicitly programmed.”
• Modern wording: “Programming computers to optimize a performance criterion using example data or past experience.”
  • Model specified up to parameters; learning = algorithmic optimization of those parameters with training data.
  • Models can be predictive (future estimates), descriptive (knowledge extraction), or both.

Optimization Perspective

• We assume a model $f(x,\Theta)$.
• Learning = running a computer program that adjusts $\Theta$ to minimize error per a performance measure.

Traditional Programming vs. Machine Learning

Formal Definition of Learning (Tom Mitchell, 1997)

• “A computer program is said to learn from experience E with respect to some tasks T and performance measure P if its performance at tasks T, as measured by P, improves with experience E.”

Illustrative Learning Problems

• Hand-written word recognition
  • T: Classify words in images.
  • P: % of correctly classified words.
  • E: Labeled dataset of word images.
• Autonomous highway driving
  • T: Drive using vision sensors.
  • P: Avg. distance before an error.
  • E: Video + steering data from human driver.
• Chess playing
  • T: Win chess games.
  • P: % games won.
  • E: Self-play practice games.

Components of the Learning Process

1. Data Storage
   • Ability to store/retrieve vast data (brain vs. HDD/SSD/RAM).
2. Abstraction
   • Extracting knowledge; fitting a model to data (training) ⇒ abstract representation.
3. Generalization
   • Converting learned knowledge into a form usable for future/unseen instances.
   • Key property: algorithm’s accuracy on new data.
4. Evaluation
   • Providing feedback on model utility; drives iterative improvement.

Visualization: Data → Storage → Abstraction (concepts) → Generalization (inferences) → Evaluation (feedback).

Real-World Applications of Machine Learning

• Manufacturing, Healthcare, Insurance, Transportation, Automotive, E-commerce, Customer Service.
• Data-mining: applying ML to large databases to derive high-accuracy yet simple models.
• Sample domain uses:
  1. Retail – consumer-behavior analysis.
  2. Finance – credit scoring, fraud detection, stock modeling.
  3. Manufacturing – process optimization & troubleshooting.
  4. Medicine – diagnostic support.
  5. Science – physics/astronomy/biology data analysis, web search.
  6. AI research – adaptability in unforeseen scenarios (vision, speech, robotics).
  7. Autonomous vehicles – steering on diverse roads.
  8. Games – chess, backgammon, Go AI.

Data Foundations for Machine Learning

Units of Observation

• Smallest entity whose properties are measured (person, object, time point, region, measurement, person-years, …).

Examples & Features

• Example / Instance / Case / Record: a single unit with recorded properties.
• Feature / Attribute / Variable: individual recorded property.

Illustration (Cancer Detection)

• Unit: patient.
• Examples: sampled patients.
• Features: gender, age, blood pressure, biopsy findings.

Illustration (Spam Email)

• Unit: email message.
• Examples: specific emails.
• Features: words contained in message.

Dataset Matrix (Automobile sample)

• Rows = examples (cars).
• Columns = features: year, model, price, mileage, color, transmission.
  • “year”, “price”, “mileage” → numeric.
  • “model”, “color”, “transmission” → categorical.

Data Types & Forms

• Numeric (Quantitative)
  • Continuous (infinite values): age, weight, blood pressure.
  • Discrete (finite counts): shoe size, number of children.
• Categorical (Qualitative)
  • Nominal (no intrinsic order): eye color, dog breed.
  • Ordinal (ordered categories): clothing sizes, pain severity.

Seven-Step Workflow in ML Projects

1. Data collection
2. Data preparation (cleaning, formatting)
3. Choosing a model
4. Training
5. Parameter tuning
6. Evaluation
7. Prediction (deployment)

Machine-Learning Task Families

1. Association (Learning Associations)

• Goal: discover interesting relations (association rules) between variables in large datasets.
• Example rule: {onion, potato} ⇒ {burger}
  • Strength quantified by conditional probability $P({\text{burger}}\mid{\text{onion},\ \text{potato}})$.
  • If $P=0.8$ → “80 % of customers who buy onion & potato also buy burger.”
• Marketing use: customers who bought X but not Y are potential Y targets (cross-sell, promo pricing, product placement).
• Algorithms: Apriori, FP-Growth (Frequency Pattern), …

2. Classification

• Problem: assign new observations to one of predefined categories using labeled data.
• Requires discriminant rule/function.

Tabular Example (Scores → Pass/Fail)

Score1 Score2 Result
29     43      Pass
22     29      Fail
10     47      Fail
31     55      Pass
…

• Given new scores (Score1 = 25, Score2 = 36) → need rule to predict Result.

Discriminant Illustration

• Finance loan-risk rule: IF income $x<em>1 > \theta</em>1$ AND savings $x<em>2 > \theta</em>2$ THEN “low-risk” ELSE “high-risk”.