Natural Language Processing (NLP) Comprehensive Question Bank

Vocabulary practice flashcards covering linguistic levels, text preprocessing, text representation, sequence labeling, semantic processing, discourse, generative AI, and NLP applications from the extended model answers.

Lexical Analysis

The linguistic level that identifies individual words and assigns their grammatical categories, such as determiners, nouns, and verbs.

Syntactic Analysis

The linguistic level that determines how individual words combine into structured constituents, such as subject noun phrases, verb phrases, and objects.

Semantic Analysis

The linguistic process that interprets the literal meaning or proposition expressed by a sentence.

Pragmatic Analysis

The analysis of language that interprets speaker intention and meaning by leveraging the situation, physical setting, and conversational context.

Stemming

A heuristic preprocessing technique that strips affixes from words based on fixed rules, which may yield an invalid dictionary word.

Lemmatization

A linguistically informed preprocessing technique that uses morphological analysis, vocabularies, and part-of-speech context to return a valid base dictionary form.

Tokenization

The fundamental preprocessing step that divides continuous raw text into discrete processing units, such as words or subwords.

Bag-of-Words (BoW)

A sparse lexical text representation that counts the frequencies of vocabulary words in a document while ignoring word order and contextual semantics.

TF–IDF

A numerical weighting metric defined by $\text{TF-IDF}(t,d) = \text{TF}(t,d) \times \text{IDF}(t)$, where $\text{IDF}(t) = \log(N / \text{df}(t))$, scoring words high if they are frequent in a specific document but rare across the corpus.

FastText

An embedding method that represents words as bags of character n-grams, making it robust for handling rare or morphologically complex vocabulary items.

Self-Attention

A mechanism in Transformers that computes pairwise relevance weights across tokens in a sequence, allowing each token representation to combine information from other positions.

Masked Self-Attention

A variant of self-attention applied in decoder layers that prevents current positions from attending to future tokens, preserving autoregressive generation.

Hard Attention

An attention mechanism that selects discrete input positions to focus on rather than computing a soft, weighted sum across all tokens.

WordPiece

A subword tokenization algorithm that greedily segments words into subword pieces from a learned vocabulary, using the prefix "##" to denote internal subword continuations.

Inflectional Morphology

A morphological process that modifies a word's grammatical form (e.g., number or tense) without changing its core lexical category or creating a distinct dictionary entry.

Derivational Morphology

A morphological process that forms a new word by adding affixes, typically altering its semantic meaning or shifting its grammatical category (such as verb to noun).

Part-of-Speech (POS) Tagging

A sequence labeling task that classifies each token in an utterance into a specific grammatical category, such as noun, verb, or adjective.

Named Entity Recognition (NER)

A sequence labeling task that locates spans of text mentioning real-world entities and classifies them into categories such as persons, organizations, locations, or dates.

Dependency Parsing

A syntactic parsing approach that extracts binary, directed grammatical relationships connecting head words directly to their dependents rather than nesting them into phrases.

Constituency Parsing

A syntactic parsing method that decomposes sentences into nested, hierarchical phrase and clause structures based on phrase structure rules.

Context-Free Grammar (CFG)

A formal grammatical system governed by production rules (such as $S \rightarrow NP \; VP$) that specifies valid syntactic derivations and hierarchical phrase structures.

Hidden Markov Model (HMM)

A generative sequence modeling algorithm that models transitions between hidden label states and emission probabilities of observed tokens under strict independence assumptions.

Conditional Random Field (CRF)

A discriminative sequence model that models the conditional probability of an entire label sequence given the observed input, capable of utilizing overlapping features without modeling input generation.

Word Sense Disambiguation (WSD)

The semantic processing task of identifying which particular sense of an ambiguous, polysemous word is intended based on its surrounding context.

FrameNet

A lexical database organized around semantic frames representing structured situations, linking words to specific participant roles called frame elements.

Transfer Learning

A machine learning paradigm where knowledge learned from a pretrained model is adapted to a new downstream target task via prompting, feature extraction, or fine-tuning.

Zero-Shot Learning

An NLP paradigm in which a model executes a target task without having seen any labeled training examples for the specific task labels, relying on general learned representations and prompts.

Few-Shot Learning

A learning approach where a pretrained language model is provided with a small number of demonstrative examples to guide inference on the target task.

Sequence-to-Sequence (Seq2Seq)

An architecture comprising an encoder that reads an input sequence to generate hidden representations and a decoder that autoregressively generates an output sequence.

Coreference Resolution

The discourse task of determining all linguistic mentions (such as pronouns and noun phrases) that refer to the identical real-world entity within a text.

Closed-Domain Question Answering

A QA configuration where the system extracts or generates answers exclusively from a restricted, predefined document collection or domain knowledge base.

Open-Domain Question Answering

A QA configuration where the system resolves questions by querying large-scale, general knowledge collections or extensive external corpora.

Extractive Summarization

A summarization approach that directly identifies and extracts verbatim sentences or phrases from the source text to form a summary.

Abstractive Summarization

A summarization technique that generates newly phrased, concise text to express the core concepts of the source document, potentially synthesizing across multiple sentences.

Retrieval-Augmented Generation (RAG)

An architecture that retrieves relevant document passages using vector and lexical search to condition a language model, grounding generated responses in verified external evidence.

Rule-Based Machine Translation (RBMT)

A translation methodology that converts text between languages using manually crafted grammatical rules, bilingual dictionaries, and syntactic transformations.

Statistical Machine Translation (SMT)

A data-driven machine translation approach that computes translation hypotheses using statistical models and translation probabilities estimated from bilingual parallel corpora.

Neural Machine Translation (NMT)

An end-to-end translation approach that models source-to-target sentence mappings using neural architectures, typically Transformer encoder-decoder models.

Semantic Search

A search paradigm that maps queries and documents into continuous vector spaces to retrieve results based on conceptual meaning rather than exact lexical overlap.

Multimodal NLP

NLP methods and architectures that integrate, align, and process textual data in conjunction with other modalities, such as images, audio, video, or structured data.