Machine Learning Midterm

Supervised

learning from labled data

unspervised

finding trends

regression

predicts numerical output

classification

predicts categorical output

eigen vector

survivor vector

rule 1

w^Tx=x

rule 2

w^TW=2w

rule 3

w^TAw=2Aw

cost function

error ²

gradient descent

keep testing to find w when error is lowest

normal equation definition

closed for solution of linear regression

normal equation

w=(X^TX)^-1X^Ty

normalization

gives context/ weights to data

non linear data regression adpatation

add higer order features

over fitting trend

high variance

over fitting reason

too much data

overfitting problem

data has no trend, is perfeclty curved

under fitting probem

no curve to data

under fitting cause

too little data

underfitting trend

high test and train error

regulization

adds penalty to cost function to keep weights low

lasso

non essentail weights to zero

ridge

all weights gradually towards zero

dual form definition

linear combination of linear regression

dual form equation

w=x^Ta

kernel method

add more dimension, then do inner dot product

logistic regression purpose

binary outputs

sigmoid function

o(z)=1/1+e^-z

sigmoid function purpose

turns linear data into a probability between 1 and 0

multiclass logistic regression

uses one vs all (aka 3>2>1)

soft max logistic regression

logistic regression when all outputs add up to 1

covarience

the change in 2 variables relative to each other

possitive covarience

increases together

negative covarience

drifts apart from each other

naive bayes classification

assumes all variables are independent