Topic 11 - Validation & Verification of Simulation Models

  • Key Concepts

    • Verification:

    • Definition: Building the model right

    • Focus: Comparison of conceptual model vs. computer representation

    • Questions:

      • Is the model correctly implemented in the computer?

      • Are input parameters and logical structure accurately represented?

    • Validation:

    • Definition: Ensuring the model accurately represents the real system

    • Achieved through calibration:

      • Iterative comparison between model and actual system behavior

      • Adjustments made based on discrepancies until acceptable accuracy is reached

  • Simulation Modeling Process

    1. Problem Formulation

    2. Objectives Definition

    3. Model Construction and Experimental Design

    4. Simulation Results

    5. Model Validation (Operational Validation, Verification)

    6. Programming & Implementation

  • Verification Techniques

    1. Code Review: Have the code checked by someone other than the programmer

    2. Flow Diagrams: Create diagrams to outline possible actions in model logic

    3. Output Examination: Check model output against a variety of input settings

    4. Parameter Tracking: Ensure input parameters aren't altered during execution

    5. Self-Documentation: Define every variable and describe the purpose of code sections

  • Calibration and Validation

    • Iterative process comparing the model with the real system

    • Repeat revisions until model behavior matches expectations

  • Validation of Simulation Models (Naylor and Finger's Approach)

    1. Build model with high face validity

    2. Validate model assumptions

    3. Compare input-output transformations with real systems

  • Validation of Model Assumptions

    • Structural assumptions: How does the system operate?

    • Data assumptions: Use reliable data and correct statistical analysis

    • Examples: Interarrival and service times for customers in a bank

  • Data Analysis Steps

    1. Identify appropriate probability distribution

    2. Estimate parameters of the distribution

    3. Validate statistical model using goodness-of-fit tests (chi-square, Kolmogorov-Smirnov)

  • Example: Fifth National Bank of Jaspar

    • Service time assumed random sample from an underlying population

    • Data collection over 90 customers during peak hours

    • Arrival process modeled as Poisson process, rate: extl=45ext{l} = 45 customers per hour

    • Service times normally distributed: Mean = 1.11.1 minutes, Std Dev = 0.20.2 minutes

  • Discussion Questions

    • Q1: What is the primary purpose of verification in simulation modeling?

    • Q2: How can you validate a simulation model?

    • Q3: Why are both verification and validation necessary in simulation modeling?