L1 Foundations of Natural Language Processing

[T] What is Natural Language Processing (NLP)?

Developing software that interacts with users in natural language or analyzes large amounts of language data.

[F] What closely related field is mentioned alongside NLP?

Computational Linguistics.

[T] What are major examples of NLP applications introduced in Lecture 1?

Sentiment analysis, summarization, translation, dialogue systems, and question answering.

[T] What does sentiment analysis determine?

The sentiment/opinion expressed in text, such as whether customers feel positively, negatively, or neutrally about something.

[T] What does summarization do?

Produces a condensed representation of the important information in one or more texts.

[T] What does machine translation do?

Converts text from one natural language to another.

[T] What are two examples of dialogue systems from the lecture?

Personal assistants and chatbots.

[T] What does question answering (QA) aim to do?

Answer natural-language questions.

[F] What two NLP-adjacent tasks are mentioned?

Information Retrieval (IR) and speech processing.

[T] What are the four broad stages of the classic Watson-style QA system shown in the lecture?

1. Question processing, 2. Candidate answer generation, 3. Candidate answer scoring, 4. Confidence merging and ranking.

[IT p.22] IMAGE: Classic QA system architecture

Be able to identify the overall flow: question processing → candidate answer generation → candidate answer scoring → confidence merging/ranking → answer.

[T] In a classic QA system, what happens during question processing?

The question is analyzed to determine information such as lexical answer type, question classification, parsing, named entities, and relations.

[T] In classic QA, what is candidate answer generation?

Retrieving or producing possible answers from text resources and/or structured data.

[T] In classic QA, what is candidate answer scoring?

Evaluating candidate answers using evidence and assigning scores.

[T] In classic QA, what is confidence merging and ranking?

Combining evidence/scores for equivalent answers and ranking candidates to produce the final answer.

[F] What kinds of answer types might a classic QA system identify?

Date, time, location, event, person, drug, etc.

[T] How do modern language-model-based QA systems differ broadly from classic QA pipelines?

Much of the linguistic processing and answer generation can be handled within a pretrained language model rather than by many manually separated modules.

[T] What does RAG stand for?

Retrieval-Augmented Generation.

[T] What is the basic idea of Retrieval-Augmented Generation (RAG)?

Retrieve relevant documents for a query and provide them to an LLM to help generate the answer.

[IT p.25] IMAGE: RAG architecture

Know the flow: source documents → encoder → vectorized index; query → encoder → matching → relevant documents → LLM → answer.

[A] Given a RAG diagram, where are source documents stored after encoding?

In a vectorized index.

[A] In RAG, what is matched against the vectorized index?

The encoded query.

[A] In RAG, what is passed to the LLM before it generates the answer?

Relevant retrieved documents.

[F] What example of complex QA/search is shown using Bing?

Breaking a complex question into simpler questions, retrieving answers for them, and merging the answers.

[T] Is question answering considered solved?

No.

[T] What major problem do LLM-based QA systems have?

Hallucination.

[T] What reasoning limitation of LLM-based QA systems is mentioned?

Limited reasoning abilities can hurt their ability to answer complex questions.

[T] Why can benchmark performance give a misleading picture of QA ability?

Systems are often tested on artificial benchmarks, and strong benchmark performance does not necessarily transfer to useful real-world applications.

[T] What are the major levels/components listed in the classic NLP pipeline?

Phonetics and phonology, morphology, syntax, semantics, pragmatics, and discourse/dialogue.

[T] What do phonetics and phonology concern?

Sounds.

[T] What does morphology concern?

The structure of words.

[T] What does syntax concern?

The structure of sentences.

[T] What does semantics concern?

Meaning.

[T] What does pragmatics concern?

Language use / meaning arising from context of use.

[T] What do discourse and dialogue concern?

Units of language larger than a single utterance.

[IT p.35] IMAGE: Classic NLP Pipeline

Know the staged classical pipeline: speech may undergo speech-to-text, then text is tokenized, followed by morphological analysis, syntactic analysis, semantic analysis, and finally the NLP application.

[T] What is morphology?

The study of how words are formed from minimal meaning-bearing units called morphemes.

[T] What is a morpheme?

A minimal meaning-bearing unit.

[T] Give examples of morphology involving plural formation.

cat-s, fox-es, fish.

[T] Give examples of morphology involving tense.

walk-s, walk-ed.

[T] Give examples of nominalization from the lecture.

kill-er, fuzz-iness.

[T] Give examples of compounding from the lecture.

book-case, over-load, wash-cloth.

[T] Why is morphological analysis nontrivial?

Natural language contains many irregular forms.

[F] What sentence illustrates morphological irregularity in the lecture?

“women and children begun running away as the wolves showed their teeth”

[T] What does a morphological analyzer output in the lecture's example for “am”?

“be” + 1 PERSON + PRESENT.

[T] What is lemmatization?

Reducing inflectional forms of a word to a common base form.

[T] Give two lemmatization examples from the lecture.

children → child; running → run.

[A] What is the lemma of “children”?

child.

[A] What is the lemma of “running”?

run.

[T] What is syntax?

The study of how sentences are formed by grouping and ordering words.

[T] What does a part-of-speech (POS) tagger do?

Assigns a part-of-speech category to each token.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “Apple”?

PROPN.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “is”?

AUX.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “looking”?

VERB.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “at”?

ADP.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “buying”?

VERB.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “U.K.”?

PROPN.

[A] In “Apple is looking at buying U.K. startup for $1 billion,” what POS does the lecture assign to “startup”?

NOUN.

[A] In the lecture POS example, what is the lemma sequence for “Apple is looking at buying U.K. startup for $1 billion”?

apple be look at buy u.k. startup for $ 1 billion.

[T] What is a constituent?

A group of words that functions together as a unit within a sentence.

[IT p.54] IMAGE: Constituents in “Ming and Sue prefer morning flights”

Recognize the nested constituent structure shown in the lecture and how a constituency parser represents it.

[T] What three tests for constituency are given in the lecture?

Substitution, movement/cleft, and coordination.

[A] How can substitution test whether a sequence is a constituent?

Replace the sequence with another expression that can occupy the same grammatical position; if the result remains grammatical, that supports constituency.

[A] What substitution example is used for “Ming and Sue”?

“They prefer morning flights.”

[A] What substitution example is used for “morning flights”?

“Ming and Sue prefer coffee.”

[A] What movement/cleft example supports “Ming and Sue” being a constituent?

“It was Ming and Sue that prefer morning flights.”

[A] What coordination example supports “morning flights” as a constituent?

“Ming and Sue prefer morning flights and evening drives.”

[T] What does a constituency parser represent?

The hierarchical grouping of words into nested constituents.

[T] What does a dependency parser represent?

Dependency relations between words, showing which words depend on which other words.

[IT p.55] IMAGE: Dependency parse

Be able to interpret the dependency links between words rather than a nested phrase-structure tree.

[A] What is the key representational difference between constituency parsing and dependency parsing?

Constituency parsing groups words into hierarchical phrases/constituents; dependency parsing directly links words through syntactic dependency relations.

[T] What is semantics?

The study of the meaning of words, intermediate constituents, and sentences.

[T] What is lexical semantics?

Meaning and semantic relationships at the word level.

[T] What lexical semantic relations are illustrated in the lecture?

Synonymy, antonymy, and part-of relations.

[T] Give the lecture's synonym example.

purchase / buy.

[T] Give the lecture's antonym example.

hot / cold.

[T] Give the lecture's part-of relation example.

wheel is part of car.

[T] What is compositional semantics?

Understanding how the meanings of components combine to determine the meaning of a larger expression.

[A] Why do “olive oil” and “baby oil” illustrate compositional semantics?

The relationship between the modifier and “oil” differs: olive oil is oil from olives, while baby oil is not oil from babies.

[A] What question about “good” illustrates compositional meaning?

“Is a good driver a good person?” — the adjective's meaning depends on what it modifies.

[T] What does sentence-level semantics attempt to represent?

The meaning of an entire sentence, potentially using a formal logical representation.

[IT p.60] IMAGE: Sentence-level semantic representation of “Mary has a new car”

Recognize that sentence meaning can be represented formally with variables, predicates, quantification, and relations.

[F] What semantic/NLP tasks are listed on the semantics slide?

Word representations, word sense disambiguation, ontologies/taxonomies, and relation extraction/classification.

[T] What are discourse and pragmatics concerned with?

Meaning that comes from the context of use.

[T] What is implicit meaning?

Meaning that is inferred even though it is not stated directly.

[A] What implicit meaning is inferred from “I didn’t eat anything since the morning”?

“I’m hungry.”

[T] What is non-literal meaning?

Meaning where the intended interpretation differs from the literal wording.

[A] What does “I could eat a horse!” mean in the lecture?

The speaker is extremely hungry, not that they literally intend to eat a horse.

[T] What is a pragmatic inference in the lecture's food example?

“Do you have some food here?” can function as an indirect request meaning roughly “please give me some.”

[T] What is broad discourse context?

Earlier or surrounding utterances that are needed to interpret the current utterance.

[A] In the dialogue “Would you like some food?” → “Yes, please,” what does “Yes, please” mean?

“I would like some food.”

[T] What is coreference?

The phenomenon where multiple expressions refer to the same entity.

[A] In “I have an apple. I don’t like them. I don’t like apples,” what contextual phenomenon is being illustrated?

Coreference/discourse interpretation.

[T] What is common background/context?

Shared knowledge or situational information needed to interpret an expression.

[A] In “There’s a new restaurant in the city,” what might “the city” mean from common background?

The city the speakers are currently in; interpretation can depend on shared context.

[T] What is the first major NLP challenge discussed?

Ambiguity.

[T] What is ambiguity?

A form or expression having multiple alternative interpretations.

[T] What is lexical ambiguity?

A word/form having multiple possible meanings or lexical categories.

[A] Why is “kiwi” lexically ambiguous?

It can refer to different things depending on context, such as the fruit versus another meaning.

[T] What NLP task resolves different possible senses of a word?

Word Sense Disambiguation.