Slides w2

Features

Feature-Based Supervised Learning

Features = attributes/properties of the object you want to classify

Classifiers rely on features encoded as dimensions in vectors (interpretable)

Logistic Regression is part of the family: Feature-Based classifiers

Reductionist Perspective

Feature Encoding

Reductionist Perspective
Explains the behavior of large structures through the combined actions of smaller primitives (e.g., matter → atoms; word meanings → semantic features).

Feature Encoding

• manual selection

• features can be very diverse and can combine different information

Feature Types (NLP example)

Feature-based ML algorithm

3 feature-based classifiers

Each feature-based ML algorithm uses features differently to learn “how to put things in different boxes”

We will look into 3 feature-based classifiers:

• Decision Trees

• Naïve Bayes

• Support Vector Machines (SVM)

Feature-based classifiers

Decision Trees: Overview

Feature-based classifiers

Decision Trees: Training

Feature-based classifiers

Decision Trees: Properties

Feature-based classifiers

Naïve Bayes: Overview

Feature-based classifiers

Naïve Bayes: Decision Rule

Feature-based classifiers

Naïve Bayes: Bayes Rule

Feature-based classifiers

Naïve Bayes: Properties & Differences wrt DT and LR

Feature-based classifiers

SVM: Overview

Feature-based classifiers

SVM: Support Vectors

Feature-based classifiers

SVM: Margins

Feature-based classifiers

SVM: Non-linear

Unsupervised ML

Overview

Unsupervised ML

Clustering

Unsupervised ML

Clustering: K-means

Unsupervised ML

Clustering: K-means limitations & choosing K

Unsupervised ML

Evaluation

DBSCAN

Dimensionality Reduction via PCA

PCA (Principal Component Analysis) squashes those 13 dimensions down to just 2, while trying to keep the overall shape of the data.

Think of it as taking the most informative "photo" of the data from the best angle.

Kernel Trick vs PCA

Curse of dimensionality

K-means

Inertia