Certified Tester Specialist Level - Testing with Generative AI (CT-GenAI) Practice Flashcards

Flashcards covering foundational concepts, prompting techniques, risk management, infrastructure, and organizational integration of Generative AI in software testing based on the ISTQB CT-GenAI syllabus.

Generative Artificial Intelligence (GenAI)

A branch of artificial intelligence that uses large, pre-trained models to generate human-like output, such as text, images, or code.

Symbolic AI

An AI approach that uses a rule-based system to mimic human decision-making by representing knowledge using symbols and logical rules.

Classical Machine Learning

A data-driven approach that requires data preparation, feature selection and model training, often used for tasks like defect categorization.

Deep Learning

A type of machine learning that uses neural networks to automatically learn features from large and complex datasets without manual feature definition.

Large Language Models (LLMs)

GenAI models pre-trained on large textual datasets, enabling them to determine context and produce relevant responses according to user prompts.

Tokenization

The process of breaking down text into smaller units called tokens, which can be characters, sub-words, or words.

Embeddings

Numerical representations of tokens as vectors in a high-dimensional space that encode semantic, syntactic, and contextual relationships.

Transformer Model

A neural network architecture that excels in language tasks by processing the context of extensive text sequences and learning how tokens relate to each other.

Context Window

The amount of preceding text, measured in tokens, that the model can consider when generating responses.

Non-deterministic behavior

Inherent randomness in LLMs due to the probabilistic nature of inference and hyperparameter settings, leading to varied outputs for the same input.

Foundation LLMs

General-purpose models trained on vast and diverse datasets including text, code, and images, typically requiring further adaptation for specific tasks.

Instruction-tuned LLMs

Models derived from foundation models that are fine-tuned using datasets pairing prompts with expected responses to improve alignment with human instructions.

Reasoning LLMs

Models that emphasize structured cognitive abilities such as logical inference, multi-step problem-solving, and chain-of-thought reasoning.

Multimodal LLMs

Models extended to process multiple data modalities, including text, images, sound, and video.

Vision-Language Models

A subset of multimodal LLMs that integrate visual and textual information for tasks like image captioning and visual question answering.

Prompt Engineering

The process of designing and refining input prompts to guide LLMs toward producing desired, accurate, and relevant outputs.

Structured Prompt Components

The six key elements of an effective prompt: Role, Context, Instruction, Input Data, Constraints, and Output Format.

Prompt Chaining

A technique involving breaking a complex task into a series of intermediate steps where the result of each step informs the next prompt.

Few-shot Prompting

Providing the LLM with more than one example within the prompt to guide and consolidate the desired response behavior.

Meta Prompting

Leveraging the AI's ability to generate or refine its own prompts through an iterative cycle to optimize prompt quality.

System Prompt

A predefined instruction set that establishes the LLM's behavior, personality, context, and operational parameters for the entire conversation.

User Prompt

The actual input, question, or specific instruction from the user that forms the immediate context for each response.

Hallucinations

A GenAI defect where the model generates output that appears factually incorrect, irrelevant, or fictitious to the given task.

Reasoning Errors

Defects occurring when LLMs misinterpret logical structures like cause-and-effect or conditional logic, leading to incorrect conclusions.

LLM Biases

Favored information or assumptions in LLM output originating from the data on which the model was trained.

Temperature (Parameter)

A parameter that controls the randomness of an LLM's outputs; lowering it narrows the probability distribution for more consistent results.

Retrieval-Augmented Generation (RAG)

A technique that enhances LLMs by incorporating additional data sources retrieved from vector databases into the response generation process.

LLM-powered Agent

A specialized GenAI application designed for autonomous or semi-autonomous task processing by invoking a predefined set of functions or tools.

Fine-Tuning

A process of further training a pre-trained model on a targeted dataset to adapt it for specific tasks, domains, or organizational vocabularies.

LLMOps

The practices, tools, and processes designed to streamline the development, deployment, and maintenance of LLMs in production environments.

Shadow AI

The use of GenAI tools or systems within an organization without formal approval or oversight, leading to security and compliance risks.

Small Language Model (SLM)

Compact models with fewer parameters than LLMs, designed to provide lightweight, efficient, and focused GenAI solutions.

ISO/IEC 42001:2023

A standard that specifies requirements for establishing, implementing, maintaining, and continually improving an AI Management System.

Vector Database

A database optimized for storing and querying high-dimensional vector representations (embeddings) of data.