Computer Simulation Module 8: Input Data Analysis

Computer Simulation Module 8: Input Data Analysis

Instructor Details

• Professor: Dave Goldsman, Ph.D.

• Institution: Stewart School of Industrial and Systems Engineering

Module Overview

• Last Module: Random variate generation techniques

• This Module Focus: Input Data Analysis

  • Selecting appropriate random variables (RVs) as inputs to simulations.

  • Importance of preventing Garbage-In-Garbage-Out (GIGO) which can invalidate results if improper data is used.

Key Topics and Structure

1. Introduction

2. Identifying Distributions

3. Unbiased Point Estimators

4. Mean Squared Error

5. Maximum Likelihood Estimators

6. MLE Examples

7. Invariance Property of MLEs

8. Method of Moments

9. Goodness-of-Fit Tests

10. Exponential Example

11. Weibull Example

12. More Goodness-of-Fit Tests

13. Problem Children

14. Demos

Input Analysis

• Goal: Use RVs that accurately reflect real-world scenarios, such as:

  • Interarrival times

  • Service times

  • Breakdown times

• Emphasis on Data Specification:

  • Properly specify the distributions based on empirical data.

Importance of Appropriate Data Input (GIGO)

• Warning: Incorrectly specified RVs can lead to disastrous outcomes in simulation models.

• Illustrative Example: In a queuing system with constant service times of 10 minutes, if the simulation incorrectly assumes constant interarrival times of 12 minutes instead of exponentially distributed times with a mean of 12, the simulation fails to account for the reality of customer arrival patterns.

High-Level Game Plan for Data Analysis

1. Collect relevant data for analysis.

2. Estimate the underlying distribution and its parameters, e.g., Normal distribution as $N(30,8)$.

3. Conduct statistical tests to validate the chosen distribution.

Summary of Input Data Analysis Subtopics

• Discussed the upcoming subjects related to input data analysis. Next topics will introduce methods for identifying distributions.

Identifying Distributions

Lesson Overview

• Last Module: Introduction to simulation input analysis.

• This Lesson: Basic methods to analyze data for distribution identification.

  • Gist expressed humorously: "It's Weibull, and please change my diaper."

Histogram Usage

• Histograms: A basic method for data visualization. Key considerations:

  • Too few cells may oversimplify data.

  • Too many cells may obfuscate trends.

  • Ideal presentation allows for observation clarity.

  • Fun Fact: Sufficient data will cause the histogram to converge to the true distribution.

Initial Considerations for Selecting Distribution

• Key questions when starting distribution analysis:

  • Is the data discrete or continuous?

  • Is it univariate or multivariate?

  • How much data is available?

  • Are there experts available for insights on data characteristics?

  • What to guess if data volume is low?

Discrete Distributions

1. Bernoulli($p$): Single trial outcome (success probability $p$).

2. Binomial($n,p$): Number of successes in $n$ Bernoulli trials.

3. Geometric($p$): Number of trials until the first success.

4. Negative Binomial: Generalization of geometric trials.

5. Poisson($ heta$): Counts of events in fixed intervals.

6. Empirical Distribution: Based on observed sample data.

Continuous Distributions

1. Uniform: Limited information, only minimum and maximum known.

2. Triangular: Knows min, max, and mode.

3. Exponential($ heta$): Used for modeling time until event occurrences.

4. Normal: Effective model for many natural data types (e.g., heights, weights).

5. Beta: Useful for bounded data modeling.

6. Gamma, Weibull, Gumbel, Lognormal: Preferred for reliability or time-to-event data.

7. Empirical Distribution: Based on observed continuous data.

Game Plan for Distribution Evaluation

• Choose a reasonable distribution based on prior knowledge and available data,

• Use goodness-of-fit tests to confirm the distribution's suitability, e.g., expect normal data to align on a normal probability plot.

Summary of Distribution Identification

• Overview covered essential types of distributions useful for simulating input RVs.

• Next steps include estimators for testing hypotheses about distribution adequacy.

Unbiased Point Estimation

Lesson Overview

• Last Lesson: Explored useful input distributions.

• This Lesson: Focus on narrowing down distribution parameters with an emphasis on unbiased estimation.

Definition of Statistics

• A statistic is a function based on observations that estimates an unknown parameter of the underlying probability distribution.

  • Example of Statistic: $ar{x} = rac{1}{n} extstyleig( ext{i=1}^n x_i ig)$

• Statistical analysis typically yields different statistic values from different samples.

Point Estimators

• If $Tn(x) = (X1,…,Xn)$ aims to estimate an unknown parameter $ heta$, then $Tn(x)$ is a point estimator for $ heta$.

  • Examples: Sample mean $ar{X}$ is usually a point estimator for the mean $ heta$, while $S^2$ is a point estimator for variance.

• Desirable properties for $T_n(x)$:

  • Expected value property: $E[T_n] = heta$ indicates unbiasedness.

  • Low variance for precision.

Definition of Unbiased Estimator

• An estimator $Tn$ is unbiased for $ heta$ if $E[Tn] = heta$.

• Theorem Example: Assume $X1…Xn$ are independent and identically distributed (iid) with mean $ heta$. Thus, $ar{X}$ is unbiased since it is derived as an average over $n$ samples: $ar{X} = rac{1}{n} extstyleig( ext{i=1}^n X_iig)$.

• Special Case: If $X_i$ follows a standard uniform distribution, then $E[ rac{1}{X}] = rac{1}{E[X]}$.

Mean Estimators’ Properties

• For $n$ iid samples $X_i$ taking values from a distribution with mean $ heta$ and variance $ heta^2$, it can be shown thategin{equation} Var(ar{X}) = rac{ heta^2}{n}, ext{so } ar{X} ext{ has lower variance. }egin{equation}

Summary of Unbiased Estimators

• This lesson initiated an examination of unbiased point estimation.

• Next steps involve discussing Mean Squared Error as an estimator performance measure combining bias and variance.

Mean Squared Error (MSE)

Lesson Overview

• Last Lesson: Discussed unbiased point estimators.

• This Lesson: Introduction to the Mean Squared Error (MSE) and its relation to estimator quality.

Good Estimator Properties

• Low Bias: Discrepancy between the expected estimator value and the true parameter value.

• Low Variance: Indicates reliability across samples.

• Combination of Both: Necessitates a balance between bias and variance; either alone may lead to poor estimators:

  • Low Bias + High Variance = Untrustworthy estimator.

  • High Bias + Low Variance = Misleading confidence.

Practical Example

• Consider $X1, … Xn$ iid with hypothetical mean $ heta$ over the interval $(0, 1)$.

  • Estimators: $T1 = rac{1}{n} extstyleig(i=1^n Xiig)$ (unbiased) and $T_2 = m$ (mean of squared values).

  • Use MSE analysis for $ heta$ based on collected data and distributions.

MSE Formula

1. $MSE(T<em>n)= E[(T</em>n - heta)^2] = Var(T<em>n) + (E[T</em>n] - heta)^2.$

2. Remark: A lower MSE is indicative of a superior estimator, even if small bias is present.

Relativity of Estimators

• Definition: Relative efficiency compares two estimators; if $E(T1)/E(T2) < 1$, then $T_1$ is preferable.

Summary of MSE

• Discussed MSE in the context of estimator performance.

• Next lesson will focus on Maximum Likelihood Estimators (MLE) for distribution fitting.

Maximum Likelihood Estimators (MLE)

Lesson Overview

• Last Lesson: Continued unbiased estimator and MSE discussions.

• This Lesson: Introduction to the widely-used Maximum Likelihood Estimation method.

Definition of MLE

• Assume an iid random sample from a distribution with parameter $ heta$. The likelihood function is defined as:
  $L( heta) = extstyle\big( ext{i=1}^n f(x_i; heta)\big).$

• The MLE is the value of $ heta$ that maximizes $L( heta)$.

Example of Finding MLE

• Hypothetical Example: For iid Poisson with mean $ heta$. Maximize the likelihood function:
  $L( heta) = extstyle\bigg( rac{ heta^x e^{- heta}}{x!}\bigg)$

• Maximizing requires derivatives and algebraic simplifications; can also utilize the log-likelihood for more straightforward calculations.

Traits and Characteristics of MLE

• Remark: Software utilities often use MLE methods due to its flexibility and efficiency.

• The habit of ensuring MLE results by evaluating against primary statistical tests—maintaining a degree of reliance on accurate validation.

Summary of MLE Examples

• Provided foundational knowledge on MLE, inclusive of examples.

• Next lesson will continue exploring MLEs through various illustrative cases.

MLE Examples

Lesson Overview

• Prior Lesson: Defined MLE with simple examples.

• Current Lesson: Deeper exploration into additional MLE illustrations.

Complex MLE Examples

• Example 1: iid exponential examples; maximize likelihood for $ heta$, where $L( heta) = extstyle\bigg( rac{ heta^x e^{- heta}}{p!}\bigg).$

• Analysis focuses on coherence with theoretical distributions through careful calculations based on known MLE traits.

Analytical Techniques for MLE

• Often necessitated knowledge of chain rules and derivatives for resolution, particularly through complex distributions like the Weibull.

• Parallel notation to ensure accuracy in syntax helps maintain clarity in derivational components and functional inequalities.

Summary of Additional MLE Cases

• Covered challenging MLE examples.

• Next lesson focuses on exploring the Invariance Property of MLEs.

Invariance Property of MLEs

Lesson Overview

• Prior Lesson: Focused on examples expanding MLE applications.

• Current Lesson: Discussion on the Invariance Property.

Theorem: Invariance Property Explained

• Property Statement: If $ hetaA$ is the MLE of a parameter $ heta0$, and $h( heta)$ is a one-to-one function, then $h( hetaA)$ is the MLE of $h( heta0)$.

• Example: Consider pairs from Poisson data to leverage statistical models effectively.

• Sensitivity Notes: Evaluating variances effectively facilitates insights into distributions on analysis levels.

Summary of Invariance Property

• Overview of the Invariance Property with thorough practical examples.

• Preparatory discussions for transitioning into discussing the Method of Moments in upcoming lessons.

Method of Moments (MoM)

Lesson Overview

• Prior Lesson: Invariance property discussions related to MLE.

• Current Lesson: Introduction to the Method of Moments (MoM).

Definition of MoM

• Definition Context: The k-th moment of a RV represents $E[X^k]$.

• For iid observations from p.m.f. or p.d.f. $f(x)$, the MoM estimator is refined and calculated systematically through expected values,

• Derived estimators should facilitate smoother transitions for distribution identifications when embarking on empirical tests.

Examples of MoM Estimators

• Mean Estimators: MoM must include comprehensive evaluations against empirical data points.

• Relevant for normalized variables exemplified through real-world applications involving services, sales rates, and product evaluations.

Summary of MoM

• Engagement over MoM was substantive, leading to robust explorations on point estimation and its implications.

• Finalizing discussions leads us towards goodness-of-fit tests.

Goodness-of-Fit Tests

Lesson Overview

• Prior Lesson: Covered Method of Moments comprehensively.

• Current Lesson: Transitioning into goodness-of-fit tests.

Purpose and Objectives of Goodness-of-Fit Tests

• Goal: To assess if a chosen simulation distribution accurately reflects real-world behavior.

• Goodness-of-fit tests provide serious statistical insights through structured hypothesis evaluation, thereby fortifying model reliability.

General Approach to Goodness-of-Fit Testing

1. Hypothesis Setup: Define null hypotheses to guide empirical evaluations.

2. Statistical Calculations: Compute expected values against observed values to facilitate examination of discrepancies using chi-squared tests.

3. Data Evaluation: Importance of tailoring tests based on sample sizes and likelihood functions deriving the expected outcomes.

Summary of Goodness-of-Fit Tests

• Engaged with structured goodness-of-fit testing, leading into precise tests for exponential distributions next.

Exponential Example

Lesson Overview

• Prior Lesson: Discussed chi-square goodness-of-fit tests.

• Current Lesson: Application of goodness-of-fit tests on exponential distributions systematically.

Testing Procedure for Exponential Distribution

• Procedure Steps Include:

  • Identifying cumulative distribution function and expected results based on empirical calculations.

  • Chi-square determination and analysis yields insightful evaluations for observed versus estimated frequency occurrences.

Summary of Exponential Example

• Implemented chi-square tests on exponential distributions, reinforcing understanding of goodness-of-fit testing.

Weibull Example

Lesson Overview

• Prior Lesson: Chi-square testing on exponential distributions.

• Current Lesson: Focus on performing goodness-of-fit tests for Weibull distributions.

Weibull Testing Approach

• Hypothesis Statement: Similar structures from previous tests adjusted to Weibull specifications.

• Deviations and predictions come from empirical exploration into distribution behaviors and moments of analysis in test structures.

Summary of Weibull Example

• Engaged thoroughly with goodness-of-fit tests for Weibull distributions while preparing for more advanced tests using alternative statistical methodologies.

Still More Goodness-of-Fit Tests

Lesson Overview

• Prior Lesson: Explored Weibull goodness-of-fit tests.

• Current Lesson: Delve into varied alternative goodness-of-fit tests, notably Kolmogorov-Smirnov techniques.

Kolmogorov-Smirnov Goodness-of-Fit Test

• Overview: Effective for smaller sample sizes, offering a practical alternative.

• Structure tests hypothesis pertaining to observed data versus predicted distributions.

Summary of Other Tests

• Discussed additional tests including Cramer-von Mises, Anderson-Darling, etc.

• Mentioned Shapiro-Wilk for normality verification alongside graphical analysis for enhanced normalization assessments.

Problem Children

Lesson Overview

• Prior Lesson: Focused on goodness-of-fit tests intricacies.

• Current Lesson: Address issues with difficult datasets leading to poor fitting challenges.

Problematic Data Characteristics

• Identification Factors: Low data volume, distorted distributions, non-stationary behaviors, and correlated datasets.

Data Solutions and Alternatives

• Strategies: Utilizing expert insights, employing empirical models, or considering adaptations including non-homogeneous frameworks for better data representations.

Summary of Problem Children’s Impact

• Engaged approaches uniquely intending to respond to flawed datasets.

Demo Time

Lesson Overview

• Previous Lesson: Discussed concerns generated by problematic datasets.

• Current Lesson: Application of input analysis through demo functionalities.

Demo Overview

• Tools Utilized: Arena Input Analyzer demonstrates clear data fitting functionalities.

• Efficiency noted in comparing G-o-F statistics while analyzing inherent characteristics within datasets.

Summary of Demo Activities

• Concluded module efficacy through demos, aligning with successive explorations in output analysis for future engagements.