BMGT 394H #4: Classification models

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/33

encourage image

There's no tags or description

Looks like no tags are added yet.

34 Terms

1
New cards

classification

predicts which class (category) a record belongs to

2
New cards

why not just use linear regression?

logistic regression instead fits an s-shaped curve that is always between 0 and 1, which better matches how probabilities behave

3
New cards

a logit of 0 represents

even odds (p = 0.5)

4
New cards

implementation for classification

  1. estimate the probability of belonging to a class

  2. use a cutoff value on that probability to classify records

  3. the cutoff determines the sensitivity and specificity of the classifier, which is visualized by the ROC curve


5
New cards

β

the change in log odds of yes for a one-unit increase in x, holding others fixed; positive raises probability of x, negative lowers it

6
New cards

maximum likelihood estimation

finds parameter values that maimize likelihood that the model produced the actual data observed

7
New cards

odds ratio (e^B)

each additional variable increases odds by a multiplier of odds ratio

8
New cards

confusion matrix

actual vs predicted

9
New cards

accuracy

what share of all predictions were correct?

= TP + TN / total

10
New cards

sensitivity (aka recall, TPR)

TPR; of the actual positives, how many did we catch?

TP / (TP + FN)

11
New cards

specificity

TNR; of the actual negatives, how many did we correctly reject?

TN / (TN + FP)

12
New cards

precision

of the ones we flagged positive, how many really were?

TP / (TP + FP)

13
New cards

false positive rate

of the actual negatives, how many did we wrongly flag?

14
New cards

F1 score

one number balancing precision and recall

15
New cards

sckit-learn’s classifcation_report shows

precision, recall, and F1 for each class + support (number of actual records)

16
New cards

lower cutoff →

more recrods predicted “yes” → catch more true positives (increases sensitivity), but decreases precision and specificity

17
New cards

higher cutoff →

opposite of low cutoff

18
New cards

ROC curve

plots true positive rate (sensitivity) on y-axis vs. false positive rate (1-specificity) on x-axis

19
New cards

optimal cutoff from ROC

maximizes sensitivity + specificity TPR + (1-FPR)

corner point of the curve (top left)

20
New cards

gain and lift chart

if i can only contact the top x%, how many postiives will i reach?


21
New cards

gain

total number of cases of interest identified by model as you go down the sorted list

22
New cards

lift

ratio of cases found by the model to cases found by random selection

% of positives in decile / % of positives overall

23
New cards

a good model has __ lift in first deciles

high

24
New cards

validation

testing on data the model hasn’t seen

25
New cards

in a gains chart, is a steep rise at the start good or bad?

good; it means the model put the real yess at the top

26
New cards

t/f: the bigger the gap between the diagonal and the curve, the better the model

true

27
New cards

con of validation set approach

results depend on which rowrs landed where, less data for training

28
New cards

k-fold cross validation

split k into equal folds, train on k-1 and validate on the remaining ones; repeat k times so each fold is the validation fold once, average the k scores

29
New cards

LOOCV

train on all records but one, test on that one, repeat n times

30
New cards

when do you split data?

when you want to predict, not explore. splitting for exploratory misses out on a lot of data

31
New cards

when the class you care about is rare, judge the model by __ instead of __

precision; accuracy

32
New cards

missing a disease is very costly, so you should favor

recall

33
New cards

missing a real email is costly, so favor

precision

34
New cards

should you lower or raise the cutoff?


lower; recall suffers because most customers have predicted probabilities closer to 0