Data Mining Chapter 9

Support Vector Machine

Out of box classifier (can be used immediately without modification). An extension of the support vector classifier that allows for non-linear boundaries

Hyperplane

Flat objetct that divides the space of dimension

Divides the p-dimensional space into two parts

Maximal Margin Classifier

The optimal separating hyperplane

Hyperplane that is the furthest away from all training observations

Tends to overfit when p is larger

Support vectors

Vectors that are on the margin, observations that are “supporting” the margin

Support Vector Regressor

Extends the SVM framework to continuous response variables, meaning it can handle regression problems where the target value is numeric and continuous

Downfalls of Maximal Margin Classifiers

Maximal margin classifier can be overly sensitive to changes in a single observation

Maximal margin classifier is only applicable when classes are perfectly separable

Support vector classifier

Also known as the soft margin classifier

Have a “budget” as to how many observations are allowed to be on the wrong side of the hyperplane

Support Vector Machines

Extends support vector classifiers to non-linear decision boundaries

Enlarge the feature space using predefined kernels

● Kernels - a function that quantifies the similarity of two observations

○ Linear kernel

○ Polynomial kernel

○ Radial kernel