1/34
Looks like no tags are added yet.
Text categorization
Sentiment analysis
Spam detection
Language id
Authorship attribution
Text categorization = assigning a label or category to a text or document
Sentiment analysis = classifying sentiment, the positive or negative orientation that sentiment analysis a writer expresses toward some object
Spam detection = assign an email to one of the two classes spam or not-spam
Language id = determining what language a text is written in
Authorship attribution = determining a text’s author, relevant authorship attribution to both humanistic and forensic analysis
What makes classification so important?
Language modeling can also be viewed as classification: each word can be thought of as a class, and so we can think of predicting the next word as classifying the context-so-far into a class for each next word! This intuition underlies large language models.
The goal of classification
The goal of classification is to take a single input (we call each input an observation), extract some useful features or properties of the input, and thereby classify the observation into one of a set of discrete classes.
We’ll call the input x, and say that the output comes from a fixed set of output classes Y = {y1, y2, ..., yM}. Our goal is to return a predicted class from Y.
Sometimes you’ll see the output classes referred to as the set C instead of Y.
Methods for Classification
Handwritten rules can be components of modern NLP systems, but it can be quite hard for humans to come up with rules that are successful over many situations.
Another method is to ask a LLM by prompting the model to give a label to some text. Prompting can be powerful, but again has weaknesses: language models often hallucinate, and may not be able to explain why they chose the class they did.
For these reasons the most common way to do classification is to use supervised machine learning:
Supervised machine learning is a paradigm in which, in addition to the input and the set of output classes, we have a labeled training set and a learning algorithm.

Probabilistic Classifiers

Probabilistic Classifiers
Two Phases

Sigmoid function
Goal

Weighted Sum (The Logit)
Logit = the score (z) = logit = input of sigmoid function

Sigmoid Function

Decision Boundary

Designing vs Learning features
Scaling input features

Processing many examples at once

Cross-entropy loss function
Overview & Goal

Cross-entropy loss function
Derivation

Cross-entropy loss function
Intuition and Behavior
Examples

Gradient Descent
Optimization Goal and Intuition

Gradient Descent
Update Rule

Gradient Descent
Partial Derivatives

Stochastic Gradient Descent (SGD)

Gradient Descent
Working Through an Example

Gradient Descent
Mini Batch Training
Loss = error on single training example i
Cost = average loss

Confusion Matrix

Accuracy
Failure Mode

Precision
Recall

Combining Metrics: F-measure

Multi-Class Evaluation

Statistical Significance Testing
Effect Size
Hypotheses
p-value
Decision Rule

Testing in NLP
Parametric vs Non-Parametric

Paired Bootstrap Test

The 2δ(x) Shift for the Bootstrap p-value

Categories of Classifier Harms
Sources of Model Bias

Model Cards

Interpretability
