Design of Experiments and Analysis of Variance

1. Elements of a Designed Experiment

• Response Variable: The dependent variable of interest in the experiment, typically quantitative in nature. It is crucial to identify this variable as it represents the outcome that is being measured.

• Factors: Variables whose effects on the response are studied. Two categories:

  • Quantitative Factors: Measured on a numerical scale, such as temperature or time, allowing for mathematical operations.

  • Qualitative Factors: Not naturally measured on a numerical scale, often represented as categories or labels (e.g., type of treatment or brand).

• Factor Levels and Treatments:

  • Factor Levels: Specific values or settings of the factors used in the experiment, such as low, medium, and high. Understanding these levels is vital for analyzing how changes impact the response variable.

  • Treatments: Combinations of factor levels administered to experimental units, representing different conditions under which observations are made.

• Experimental Unit: The object on which measurements are taken, which could be individuals, plots of land, or any individual entity involved in the experiment. Each unit should be similar in relevant characteristics to minimize variability unrelated to the treatment.

• Designed vs. Observational Experiment:

  • Designed Study: The analyst controls treatments and assigns experimental units to ensure valid comparisons. Random assignment helps to reduce bias and confounding variables.

  • Observational Study: The analyst merely observes treatments and responses without manipulation, which could introduce confounding factors and make it more challenging to determine causality.

2. The Completely Randomized Design: Single Factor

• Definition: Experimental units are randomly assigned to treatments. This design helps to ensure that each treatment group is comparable and minimizes selection bias.

• Assumptions:

  • Subjects are assumed homogeneous, meaning they have similar characteristics that would not influence the outcome. This is essential for valid results.

  • Analyzed by one-way ANOVA, where the variability in the response can be attributed to the differences in treatment.

• Example: Study on taste preference of different bottled water brands with assigned consumer groups.

  • Response: Taste Preference Scale (1-10), indicating how much participants favor a particular brand out of a set of choices.

3. ANOVA F-Test

• Purpose: Tests the equality of two or more population means, which is vital for determining whether the means differ statistically in efficacy or effect.

• Requirements:

  • One nominal independent variable with two or more treatment levels, providing a categorical comparison group.

  • One interval or ratio dependent variable, ensuring that the response can be quantitatively measured.

• Partitioning Variability:

  • Total Variation (SS Total): The sum of variation within groups (error) and among groups (treatment). This concept is essential for understanding overall variability in response.

  • Treatment Variation (SS Treatment): Variation attributed to differences in treatment means and indicates the effect of the treatment on the response variable.

  • Error Variation (SS Error): Variation among individual observations within each treatment group, reflective of random fluctuations or noise in the data.

• F-Test Calculation:

  • Formulate the test statistic: F = \frac{MST}{MSE} where MST = Mean Square Treatment (calculated by dividing SS Treatment by its degrees of freedom), and MSE = Mean Square Error (calculated by dividing SS Error by its degrees of freedom).

  • Determine degrees of freedom:

    • u_1 = k - 1 (for treatments), where k is the number of treatment levels.

    • u_2 = n - k (for residuals), where n is the total number of observations.

• F-Test Hypotheses:

  • Null Hypothesis (H0): All treatment means are equal, indicating no effect of treatments.

  • Alternative Hypothesis (Ha): At least two treatment means differ, suggesting a significant effect of at least one treatment.

4. Conditions for Valid ANOVA F-Test

• Samples are randomly selected and independent, ensuring that one sample does not affect another, which is vital for standard statistical assumptions.

• All populations have an approximately normal distribution, particularly important for small sample sizes to ensure accurate p-values.

• All population variances are equal (homogeneity of variances), which is a key assumption for the validity of the F-test results.

5. Steps for Conducting an ANOVA for a Completely Randomized Design

1. Ensure the design is completely randomized and that all experimental units have an equal chance of receiving any treatment.

2. Verify assumptions of normality and equal variances using tests like Shapiro-Wilk and Levene’s test, respectively.

3. Create an ANOVA summary table to calculate variability, which includes SS, degrees of freedom, MS, F-statistic, and p-value.

4. If the F-test indicates means differ (p-value < α level), conduct a multiple comparisons procedure to understand which groups differ significantly.

5. In case of non-rejection of null hypothesis:

   • Treatment means may indeed be equal, suggesting no significant effect.

   • Consider unaccounted factors affecting response variability that may not have been included as factors in the study, which warrants further investigation.

6. Multiple Comparisons of Means Methods

• Acknowledge that multiple pairwise comparisons can inflate Type I error rates, leading to misleading conclusions.

• Methods include:

  • Tukey Method: Appropriate for balanced designs allowing pairwise comparisons and controlling for overall error rates.

  • Bonferroni Method: More conservative, can handle both balanced and unbalanced designs, reducing the likelihood of false positives at the expense of power.

  • Scheffé Method: Flexible for general contrasts of means, allowing more complex comparisons while controlling error rates.

7. Key Ideas

• Completely Randomized Design Elements:

  • Response and factors must be clearly defined to ensure the integrity of the study.

  • Treatments should properly reflect combinations of factor levels to effectively test hypotheses.

  • Important to have equal sample sizes for treatments in balanced designs to prevent bias and ensure statistical power.

• ANOVA Validity Conditions: Recognize that adhering to normality and variance conditions is critical for reliable F-test results, enhancing the validity of conclusions drawn from the data.