3. Bias-complexity trade-off

0.0(0)
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
Card Sorting

1/8

encourage image

There's no tags or description

Looks like no tags are added yet.

Study Analytics
Name
Mastery
Learn
Test
Matching
Spaced

No study sessions yet.

9 Terms

1
New cards

Hypothesis class

Can be viewed as reflecting some prior knowledge that the learner has about the task

2
New cards

No Free Lunch Theorem

No learner can succeed on all learnable tasks Every learner has tasks on which it fails while other learners succeed

<p><span>No learner can succeed on all learnable tasks Every learner has tasks on which it fails while other learners succeed</span></p>
3
New cards
term image

Error decomposition

4
New cards
term image

Approximation error

5
New cards
term image

Estimation error

6
New cards

Approximation error

Measures how much inductive bias

7
New cards

Estimation error

Derives from inability to choose (with ERM) the best hypothesis

8
New cards

Bias-complexity tradeoff

  • larger (more complex) H: decreases app err but increases est err → overfitting

  • smaller H: increases app err but decreases est err → underfitting

9
New cards

Test set

New set of samples not used for picking h_S (=the training set), so to be able to estimate true error