Read w1

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/34

encourage image

There's no tags or description

Looks like no tags are added yet.

35 Terms

1
New cards

Text categorization

Sentiment analysis

Spam detection

Language id

Authorship attribution

Text categorization = assigning a label or category to a text or document

Sentiment analysis = classifying sentiment, the positive or negative orientation that sentiment analysis a writer expresses toward some object

Spam detection = assign an email to one of the two classes spam or not-spam

Language id = determining what language a text is written in

Authorship attribution = determining a text’s author, relevant authorship attribution to both humanistic and forensic analysis

2
New cards

What makes classification so important?

Language modeling can also be viewed as classification: each word can be thought of as a class, and so we can think of predicting the next word as classifying the context-so-far into a class for each next word! This intuition underlies large language models.

3
New cards

The goal of classification

The goal of classification is to take a single input (we call each input an observation), extract some useful features or properties of the input, and thereby classify the observation into one of a set of discrete classes.

We’ll call the input x, and say that the output comes from a fixed set of output classes Y = {y1, y2, ..., yM}. Our goal is to return a predicted class from Y.

Sometimes you’ll see the output classes referred to as the set C instead of Y.

4
New cards

Methods for Classification

Handwritten rules can be components of modern NLP systems, but it can be quite hard for humans to come up with rules that are successful over many situations.

Another method is to ask a LLM by prompting the model to give a label to some text. Prompting can be powerful, but again has weaknesses: language models often hallucinate, and may not be able to explain why they chose the class they did.

For these reasons the most common way to do classification is to use supervised machine learning:
Supervised machine learning is a paradigm in which, in addition to the input and the set of output classes, we have a labeled training set and a learning algorithm.

<p>Handwritten rules can be components of modern NLP systems, but it can be quite hard for humans to come up with rules that are successful over many situations.</p><p>Another method is to ask a LLM by prompting the model to give a label to some text. Prompting can be powerful, but again has weaknesses: language models often hallucinate, and may not be able to explain why they chose the class they did.</p><p>For these reasons the most common way to do classification is to use <strong>supervised machine learning</strong>:<br>Supervised machine learning is a paradigm in which, in addition to the input and the set of output classes, we have a labeled training set and a learning algorithm.</p>
5
New cards

Probabilistic Classifiers

knowt flashcard image
6
New cards

Probabilistic Classifiers

Two Phases

knowt flashcard image
7
New cards

Sigmoid function

Goal


<p></p>
8
New cards

Weighted Sum (The Logit)

Logit = the score (z) = logit = input of sigmoid function

<p>Logit = the score (z) = logit = input of sigmoid function</p>
9
New cards

Sigmoid Function

knowt flashcard image
10
New cards

Decision Boundary

knowt flashcard image
11
New cards

Designing vs Learning features

Scaling input features

knowt flashcard image
12
New cards

Processing many examples at once

knowt flashcard image
13
New cards

Cross-entropy loss function

Overview & Goal

knowt flashcard image
14
New cards

Cross-entropy loss function

Derivation

knowt flashcard image
15
New cards

Cross-entropy loss function

Intuition and Behavior
Examples

knowt flashcard image
16
New cards

Gradient Descent

Optimization Goal and Intuition

knowt flashcard image
17
New cards

Gradient Descent

Update Rule

knowt flashcard image
18
New cards

Gradient Descent

Partial Derivatives

knowt flashcard image
19
New cards

Stochastic Gradient Descent (SGD)

knowt flashcard image
20
New cards

Gradient Descent

Working Through an Example

knowt flashcard image
21
New cards

Gradient Descent

Mini Batch Training

Loss = error on single training example i

Cost = average loss

<p>Loss = error on single training example i</p><p>Cost = average loss</p>
22
New cards

Confusion Matrix

knowt flashcard image
23
New cards

Accuracy

Failure Mode

knowt flashcard image
24
New cards

Precision

Recall

knowt flashcard image
25
New cards

Combining Metrics: F-measure

knowt flashcard image
26
New cards

Multi-Class Evaluation

knowt flashcard image
27
New cards

Statistical Significance Testing

Effect Size
Hypotheses
p-value
Decision Rule

knowt flashcard image
28
New cards

Testing in NLP

Parametric vs Non-Parametric

knowt flashcard image
29
New cards

Paired Bootstrap Test

knowt flashcard image
30
New cards

The 2δ(x) Shift for the Bootstrap p-value

knowt flashcard image
31
New cards
32
New cards

Categories of Classifier Harms

Sources of Model Bias

knowt flashcard image
33
New cards

Model Cards

knowt flashcard image
34
New cards

Interpretability

knowt flashcard image
35
New cards