Experimental Design, Sampling Errors, and Frequency Distributions

Principles of Experimental Design

• Experimental design centers around controlling a major statistical challenge known as confounding.
• Confounding occurs when an effect is observed in an experiment or study, but it is impossible to identify the specific factor or variable that caused it.
• Confounding is closely related to the concept of a lurking variable.
  • Classic Example: In an observational study showing a positive correlation between ice cream consumption and drowning deaths, the lurking variable is hot weather/temperature. High temperatures cause an increase in both ice cream consumption and outdoor swimming activity, creating a non-causal correlation between ice cream and drowning.
• Experimental design methodologies are specifically structured to eliminate lurking variables and control confounding effects.
• Distinguishing Sampling Methods vs. Experimental Design:
  • Sampling methods deal with selecting a representative subset of individuals from a broader population.
  • Experimental design deals with assigning selected subjects to different treatment groups to observe outcomes.

Types of Experimental Designs

• Completely Randomized Experimental Design:

  • Definition: Assigns subjects to different treatment groups strictly through a process of random selection.
  • Characteristics: Shares the same foundational logic as simple random sampling. Every subject has an equal chance of being assigned to any treatment group, with no prior grouping or filtering.

• Randomized Block Design:

  • Definition: Subjects are first categorized into blocks, and then treatments are randomly assigned to subjects within each block.
  • Block Definition: A group of subjects that share similar characteristics that might influence the outcome of the experiment.
  • Procedure:
  1. Form blocks (groups of subjects) based on shared traits.
  2. Randomly assign treatments to subjects within each block.
  • Analogy: Parallel to stratified sampling in sample selection, where a population is partitioned into strata and random samples are drawn from within each stratum.

• Matched Pairs Design:

  • Definition: Compares two treatment conditions (such as a treatment group versus a placebo group) by utilizing subjects paired based on specific shared relationships or identical characteristics.
  • Common Implementation Scenarios:
  • Before-and-After Measurements: A single subject receives a treatment, and two measurements are taken from that same subject (one before treatment, one after). The pair consists of the before-and-after measurements of that single individual.
  • Twin Studies: Experiments conducted using identical twins as pairs. One twin receives the treatment, while the other does not (or receives a placebo). Identical twins control for genetic and baseline physical differences (e.g., testing toothpaste efficacy on twin pairs with identical dental characteristics).
  • Split-Subject Design: Applying two different treatments to two similar parts of the same subject (e.g., applying one lotion to the left hand and another to the right hand).

• Rigorously Controlled Design:

  • Definition: Subjects are explicitly and carefully assigned to distinct treatment groups such that subjects receiving each treatment are as similar as possible in all factors relevant to the study.
  • Characteristics: Highly common in strict laboratory environments. Requires extensive monitoring and precise control over environmental and procedural variables to construct ideal experimental conditions.

Practical Application of Experimental Designs

• Evaluating Fertilizer Types:

  • Scenario: A researcher evaluates $3$ fertilizers across $60$ land plots with uniform soil, drainage, and sunlight. She randomly assigns $20$ plots to each fertilizer.
  • Design: Completely Randomized Design (plots are uniform; treatments are assigned purely at random without grouping).

• Sibling/Twin Medication Study:

  • Scenario: A sleep researcher compares a new medication against a placebo using pairs of twin siblings.
  • Design: Matched Pairs Design (subjects are paired based on twin status).

• Clinical Trial Testing:

  • Scenario: A pharmaceutical trial with $200$ patients enforces strict, identical schedules (diet, exercise, sleep, dosing times) while randomly assigning patients to drug or placebo.
  • Design: Rigorously Controlled Design (extreme monitoring and environmental standardization).

• Instructional Method Evaluation Across Time Periods:

  • Scenario: A teacher tests a study method across $4$ class periods, suspecting time of day affects student alertness. Within each period, half the students are randomly assigned the new method and half the old method.
  • Design: Randomized Block Design (class periods serve as blocks to isolate the effect of time of day).

• Dermatological Hand Testing:

  • Scenario: A cosmetic firm compares two moisturizers by having volunteers apply Product A to one hand and Product B to the opposite hand.
  • Design: Matched Pairs Design (paired units are the two hands of the same subject).

• Greenhouse Pesticide Evaluation:

  • Scenario: Testing $4$ pesticides in a single uniform greenhouse with precise automated control over light, water, temperature, and humidity.
  • Design: Rigorously Controlled Design (complete control over external physical parameters).

• Educational Software Trial:

  • Scenario: A university tests an app on $500$ homogenous students, randomly assigning $250$ to the app and $250$ to a control group.
  • Design: Completely Randomized Design (no confounding differences identified; pure random allocation).

• Metabolic Diet Study:

  • Scenario: Nutrition study categorizing participants into age brackets due to metabolic differences, then randomly assigning participants within each bracket to one of two diet plans.
  • Design: Randomized Block Design (age brackets act as blocks).

• Footwear Sole Durability:

  • Scenario: Testing sole wear by having runners wear Sole Material A on one foot and Sole Material B on the other foot for one month.
  • Design: Matched Pairs Design (paired units are the left and right shoes of individual runners).

• Materials Stress Testing:

  • Scenario: Testing metal alloy composition in a lab where temperature, humidity, and mechanical stress loads are kept constant.
  • Design: Rigorously Controlled Design (laboratory experiment with total control over environmental variables).

Types of Errors in Statistics and Sampling

• Even when rigorous design and sampling techniques are employed to eliminate bias, statistical errors occur due to human mistakes or natural variation.

• Sampling Error:

  • Definition: An error resulting strictly from measuring a sample statistic rather than conducting a full census of the population.
  • Cause: Inherent limitation of drawing conclusions about an entire population from a subset.

• Random Sampling Error:

  • Definition: Discrepancy between a sample statistic and the true population parameter that occurs even when a perfectly valid random sampling method is used.
  • Cause: Pure chance variation in selection.
  • Example: A university population of $20{,}000$ students has a true mean height of $\text{4 ft 5 in}$. A randomly selected sample of $100$ students yields a mean height of $\text{6 ft 0 in}$ purely because an unusually tall group of students was drawn by chance.

• Non-Sampling Error:

  • Definition: Errors arising from human error, operational mistakes, or flawed execution during data collection or processing.
  • Causes/Examples:
  • Data entry errors or computational mistakes.
  • Biased question wording in surveys.
  • False information provided by respondents.
  • Biased conclusions drawn by researchers.
  • Applying statistical analysis methods unsuited for the data or situation.

• Non-Random Sampling Error:

  • Definition: An error caused by utilizing a non-random sampling method.
  • Example: Using convenience sampling, such as having a student select only their friends to measure student heights across a school, introducing systematic selection bias.

Data Organization and Frequency Distributions

• Raw data gathered from surveys, experiments, or observational studies consists of unstructured data points that must be organized to reveal underlying patterns.

• Frequency Distributions serve as a primary quantitative method for summarizing raw data.

• Core Insights Provided by Frequency Distributions:

  1. Shape: Shows whether data distribution is symmetric, skewed (left or right), or bell-shaped.
  2. Center: Identifies the primary interval or value where the majority of observations concentrate.
  3. Spread / Variation: Demonstrates how widely data spreads relative to the minimum, maximum, and central values.
  4. Outliers: Highlights unusual data points that deviate substantially from the rest of the distribution.

Key Components and Metrics of Frequency Distributions

• Frequency Distribution: A tabular arrangement listing data values (individually or grouped in class intervals) alongside their corresponding frequencies.

• Frequency: The total count of data observations falling into a specific class interval.

• Class: A single interval or grouping used to categorize raw data.

• Class Limits:

  • Lower Class Limits: The smallest numerical values that can belong to each distinct class.
  • Upper Class Limits: The largest numerical values that can belong to each distinct class.

• Class Boundaries:

  • Definition: Values that separate adjacent classes without leaving gaps created by class limits.
  • Calculation of Intermediate Boundaries:     $\text{Class Boundary} = \frac{\text{Upper Limit of Lower Class} + \text{Lower Limit of Upper Class}}{2}$
  • Adjustment for Outer Boundaries: Determine the gap width $\text{Gap} = \text{Boundary} - \text{Upper Limit}$. Subtract $\text{Gap}$ from the lowest lower limit, and add $\text{Gap}$ to the highest upper limit.
  • Example:
  • Given classes $0 - 5$, $6 - 11$, and $12 - 17$:
  • Lower Class Limits: $0, 6, 12$
  • Upper Class Limits: $5, 11, 17$
  • Midpoint between limits $5$ and $6$: $\frac{5 + 6}{2} = 5.5$
  • Midpoint between limits $11$ and $12$: $\frac{11 + 12}{2} = 11.5$
  • Gap offset: $5.5 - 5 = 0.5$
  • Lowest Boundary: $0 - 0.5 = -0.5$
  • Highest Boundary: $17 + 0.5 = 17.5$
  • Complete Set of Class Boundaries: $-0.5, 5.5, 11.5, 17.5$

• Class Midpoint:

  • Definition: The numerical value positioned at the exact center of a class interval.
  • Formula:     $\text{Class Midpoint} = \frac{\text{Lower Class Limit} + \text{Upper Class Limit}}{2}$
  • Examples (using classes $0 - 5$, $6 - 11$, $12 - 17$):
  • Midpoint 1: $\frac{0 + 5}{2} = 2.5$
  • Midpoint 2: $\frac{6 + 11}{2} = 8.5$
  • Midpoint 3: $\frac{12 + 17}{2} = 14.5$

• Class Width:

  • Definition: The absolute difference between two consecutive lower class limits or two consecutive lower class boundaries.
  • Formula:     $\text{Class Width} = \text{Lower Class Limit}_2 - \text{Lower Class Limit}_1$
  • Example (using limits $0$ and $6$):     $\text{Class Width} = 6 - 0 = 6$$\text{Class Width} = 5.5 - (-0.5) = 6$

Step-by-Step Procedure for Constructing Frequency Distributions

1. Select the Number of Classes:

   • Typically choose between $5$ and $20$ classes.
   • Must select enough classes to illustrate structure without making intervals so narrow that data becomes overly fragmented, or so wide that summary details are lost.

2. Calculate the Class Width:

   • Use the class width formula:      $\text{Class Width} = \frac{\text{Maximum Data Value} - \text{Minimum Data Value}}{\text{Number of Classes}}$
   • Round the result up to a convenient whole number if necessary.

3. Set the First Lower Class Limit:

   • Choose a clean, easy-to-use round number equal to or slightly lower than the minimum value in the raw dataset.

4. Determine Remaining Lower Class Limits:

   • Calculate successive lower class limits by adding the class width to the previous lower class limit:      $\text{Next Lower Limit} = \text{Previous Lower Limit} + \text{Class Width}$

5. Establish Upper Class Limits and Tally Frequencies:

   • List corresponding non-overlapping upper class limits.
   • Iterate through the dataset sequentially, placing tally marks into their respective class intervals, and count tallies to record final class frequencies.

• Sample Table Structure:

Creating Frequency Distributions in Microsoft Excel

• Enabling the Analysis ToolPak Add-In:

  1. Open Microsoft Excel and navigate to File -> Options.
  2. Select Add-ins on the left navigation panel.
  3. At the bottom dropdown menu (Manage), select Excel Add-ins and click Go.
  4. Check the box for Analysis ToolPak and click OK.
  5. Verify that Data Analysis appears on the far right of the Data tab.

• Generating a Frequency Distribution/Histogram in Excel:

  1. Go to the Data tab and click Data Analysis.
  2. Select Histogram from the tool list and click OK.
  3. Set Input Range to the cell range containing raw data (e.g., A2:A21).
  4. Set Bin Range to a cell range containing user-defined upper class limits (e.g., 59, 69, 79, 89, 99). In Excel, Bin values represent upper class limits. If left blank, Excel calculates bin limits automatically.
  5. Specify an Output Range cell destination and click OK to produce the frequency table.