GenAI for Text, Language and GenAI Quality

Flashcards covering text analysis history, Transformer architecture, decoding methods, training techniques, chatbot system layers, and evaluation metrics for text, audio, and images.

Which seminal 2017 paper by Vaswani et al. introduced the Transformer architecture that forms the foundation of models like GPT, BERT, and T5?

"Attention is All You Need"

In the Transformer attention mechanism, which three matrices are computed from input embeddings via learned weight matrices?

Query ($Q$), Key ($K$), and Value ($V$)

According to the general LLM attention architecture, what mathematical formula defines the attention score matrix $S$?

$$S = \text{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right)$$

In transformer output decoding, how does Greedy Decoding differ from Beam Search?

Greedy Decoding always selects the token with the highest probability, whereas Beam Search tracks multiple candidate sequences ("beams") and explores the most likely paths to generate more balanced and coherent text.

What training method uses human evaluators to rank responses to optimize a model's policy for helpfulness, safety, and relevance?

RLHF (Reinforcement Learning from Human Feedback)

In a chatbot system architecture, what is the core function of the Dialogue Manager layer?

To track conversation state using session memory and vector stores.

What does the classical text evaluation metric BLEU (Bilingual Evaluation Understudy) measure?

The weighted geometric mean of modified $n$-gram precisions.

How is Word Error Rate (WER) calculated in speech evaluation?

$$\frac{\text{substitutions} + \text{deletions} + \text{insertions}}{\text{total words}}$$

What does the Fréchet Inception Distance (FID) metric measure when evaluating image generation?

The distributional similarity between generated and real images, measured via features extracted from the Inception network.

What is the human evaluation method known as Pairwise Comparison?

An evaluation approach where the evaluator is presented with two AI-generated responses and tasked with selecting the better one.