BUAN 348 - Concepts Test #2

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/45

encourage image

There's no tags or description

Looks like no tags are added yet.

46 Terms

1
New cards

Naive benchmark

A simple baseline rule (what you start with) where you predict the overall average outcome for every single record. Any useful predictive model must beat this baseline to prove it adds value.

2
New cards

Mean Absolute Error (MAE)

The average size of your prediction errors, measured in the exact same units as the outcome (such as dollars).

3
New cards

Mean Squared Errors (MSE)

The average of all squared errors. Because errors are squared, big misses are penalized much more heavily than small ones.

4
New cards

Root Mean Squared Error (RMSE)

The square root of MSE. Converts the metric back to its original units, making it easier to read, but punishes large errors extra heavily.

5
New cards

(Model evaluation & interpretation) What metrics are used for numeric outcomes?

Naive benchmark, MAE, MSE, RMSE

6
New cards

Why might a model scored only on its training data look great?

Because it could have memorized the training data; overfitting.

7
New cards

What is a confusion matrix?

A summary table that counts correct and incorrect predictions for each category, serving as the foundation for all classification metrics.

8
New cards

What is accuracy? (provide the formula too)

The overall share of all predictions that the model got right. It can be misleading when one outcome is very rare (like fraud or a rare disease). (TP + TN / (TP + FN + TN + FP))

9
New cards

What is precision? (provide the formula too)

Out of all the cases the model flagged as positive, how many are actually positive? (TP / (TP + FP))

10
New cards

What is recall / sensitivity? (provide the formula too)

Out of all the positive cases in the data, how much did the model successfully catch? (TP / (TP + FN))

11
New cards

What is specificity? (provide the formula too)

Out of all the actual negative cases, how many did the model correctly leave alone? (TN / (TN + FP))

12
New cards

What is the F1 score? (provide the formula too)

A single combined metric that balances precision and recall. (F1 = (2 x precision x recall) / (precision + recall))

13
New cards

What is the AUC [area under ROC curve]?

Measures how well a model separates positive and negative classes across every possible cutoff threshold. A score of 0.5 means the model is no better than a coin flip, while 1.0 is perfect separation.

14
New cards

What is overfitting?

When a model memorizes the quirks and noise of training data, making it look great during training but fail on new, unseen data. Always evaluate models on separate validation or test sets to get an honest score.

15
New cards

What is classification cutoff?

The probability threshold used to decide when a prediction counts as “positive”. Lowering the cutoff flags more cases as positive (boosting recall, hurting precision and specificity), while raising it does the opposite. Choose a cutoff based on the real-world cost of false alarms versus missed cases.

16
New cards

What is a lift chart?

A visual chart showing how well a model ranks records from most likely to least likely (for example, putting the most likely buyers at the very top of a marketing list).

17
New cards

What is cross-validation?

Testing a model across multiple different splits of your dataset to ensure its performance ranking isn’t just the result of one lucky split.

18
New cards

What is feature importance?

Score showing which variables matter most to the model's predictions overall.

19
New cards

What is a Partial Dependence Plot (PDP)?

A graph showing how the predicted outcome changes, on average, as one variable changes.

20
New cards

What are SHAP values?

A method that calculates how much each variable pushed one specific prediction up or down.

21
New cards

What is LIME?

A tool that creates a simple, local explanation of one specific prediction.

22
New cards

What is sensitivity analysis?

What happens to the output when one input changes and the others are held constant.

23
New cards

Is the most statistically accurate model always the best choice? Why?

No. In regulated fields like banking or healthcare, a slightly less accurate model that is fast, scalable, and easy to explain is often preferred.

24
New cards

What is the first thing you should do when evaluating a model?

Benchmark first; a model that cannot beat “predict the average” adds no value, however sophisticated it is.

25
New cards

RMSE vs. MAE, how are they different?

Both are measurements in the outcome’s units, but RMSE punishes big misses more, so it is always at least as large as MAE.

26
New cards

What is the main difference between recall and specificity?

Recall is about catching the actual positives, while specificity is about correctly clearing the actual negatives.

27
New cards

What are the simple ways to summarize metrics like accuracy, precision, recall, and specificity? What types of variables use these metrics?

Accuracy is overall correctness, precision is positive prediction correctness, recall is finding all the positives, and specificity is finding all the negatives.

28
New cards

(Model evaluation and interpretation) What does a “lift of 4” mean in the case where you’re trying to predict and target potential shoppers?

By targeting the top 10% of buyers, your model captures 4x as many actual buyers as you would by targeting a random 10%.

29
New cards

Which methods use standardization (k-NN, Naive Bayes, SVM)?

k-NN and SVM

30
New cards

When are dummy variables used?

To convert categorical text labels into 0/1 numeric columns so distance-based algorithms like k-NN can measure similarity

31
New cards

How is the “k” for “k-NN” chosen?

Cross validation or validation data

32
New cards

In two class problems, how can k avoid ties? (k-NN)

By choosing an odd value for k

33
New cards

k-NN is slow on large datasets. True or false? Why?

True. It must keep all its training data in memory because it evaluates predictions on the spot (“lazy learner”).

34
New cards

When is k-NN useful? What use case could k-NN be for?

Tasks built on similarity. k-NN could be used for product recommendations (“customers like you also bought…”).

35
New cards

Where does the “naive” in Naive Bayes come from?

It assumes predictors are independent from one another within each class

36
New cards

How does Naive Bayes work?

It combines values of the prior (how common each class is overall) with the likelihoods (how common each clue is within each class) to get the posterior (the updated probability after seeing the clues), then predicts based off those values.

37
New cards

What is different about Gaussian Naive Bayes?

It assumes a bell curve for each predictor within each class.

38
New cards

Naive Bayes is better at predicting probabilities over ranking and classifying. True or false?

False

39
New cards

What is the “zero-frequency problem” of Naive Bayes? What can fix it?

It’s when a model assumes a probability is zero because it doesn’t spot a predictor value in its training set (doesn’t mean it is actually zero); the solution is laplace smoothing, which adds one to every probability, so that no value has a zero chance.

40
New cards

When is Naives Bayes used? What can Naive Bayes be used for (use case)?

Tasks that need speed and scale; Spam email classification

41
New cards

How does Support Vector Machines (SVM) work?

An SVM finds the boundary (a line, plane, or hyperplane) that separates the classes with the widest possible margin, meaning the boundary sits as far as possible from the closest records of each class.

42
New cards

What are the closest records (to a boundary) for SVM called?

Support vectors

43
New cards

What is a kernel in SVM? What are common kernels?

SVMs use a kernel to map the data into a higher-dimensional space where a separating boundary can be found; used to handle nonlinear patterns.

Common kernels are linear, polynomial, and RBF.

44
New cards

What does a soft-margin SVM do? How do you adjust it?

Lets some records fall within the boundary margin or cross the line, at a penalty set by C. A small C is more tolerant (risks underfit), a larger C is more strict (risks overfit).

45
New cards

How is C in SVM tuned?

Through cross-validation

46
New cards

When would you use SVM; what is a use case for SVM?

Tasks where accuracy and a clean decision boundary matter most ;credit scoring