Semantics week 10

LING2121

What is distributional semantics?

A theory that models word meaning based on distributional patterns - i.e., the idea that words with similar meaning occur in similar contexts (the distributional hypothesis)

What is the distributional hypothesis?

‘you shall know a word by the company it keeps’

Words appearing in similar contexts tend to have similar meanings.

What kind of data is used in distributional semantics?

Large corpora of natural language text, used to gather co-occurence statistics between words and their surrounding context.

What is a vector space model of meaning?

Each word is represented as a vector in high-dimensional space, where each dimension corresponds to a contextual feature (e.g., a nearby word, syntactic role, etc)

What is the difference between count-based and prediction based models?

Count-based models: Build vectors directly from co-occurance counts (e.g., LSA, PMI)

Prediction-based models: Use neural nets to predict context from target words or vice versa (e.g., Word2vec, GloVe)

What kinds of meaning relations do distributional models capture well?

Semantic similarity (e.g., dog vs. wolf)

Analogy (e.g., king - man + woman = queen)

Topical relatedness (e.g., bank vs. money)

What are limitations of distributional semantics?

-Poor at logical inference and truth-conditions

-Struggle with compositionally (combining word meanings into sentence meanings)

-Contextual meaning is often flattened in static models

Why is compositionally a problem in distributional semantics?

Because simple vector addition/multiplication often fails to capture syntactic structure and logical scope (e.g., negation, quantification)can