Unit 4 - Design of Experiments

Two Ways to Obtain Data

• Sample: Sample units from a larger population and just simply observe some characteristic of these sampled units.

• Experiment: Carry out an experiment where we observe the response to some treatment imposed on units.

Observational Study

The researcher simply observes the units or subjects, and measures the variables or characteristics of interest. The research does not actively interfere with the units. Treatments (combinations of factor levels) are not imposed on the experimental units.

• Researchers observe only.

• Example is that cellular phone usage seems to have a correlation with brain cancer.

Experiment

Only in an experiment, in which the researcher imposes some treatment on the units or individuals, are we able to establish the cause of differences observed for different groups.

Imposing treatments to experimental units is one of the defining characteristics of an experiment. In an observational study, treatments (combinations of factor levels) are not imposed on the experimental units.

• Imposed values of x.

Confounding

Happens when an outside factor influences both the variable you are studying and the outcome, making it appear that there is a relationship when the true cause may be something else.

• For example, suppose researchers find that people who carry lighters have a higher risk of lung cancer. Carrying a lighter does not cause cancer, but smoking is a confounding variable because smokers are more likely to carry lighters and smoking increases the risk of lung cancer.

• Confounding can lead to misleading conclusions if it is not identified and controlled for in a study.

Experimental Units

The smallest unit that receives a treatment independently. The objects on which our actual experiment is being conducted. The thing you're applying the treatment to and measuring a response from within an experiment. If the experiment is done on humans, they are referred to as subjects.

• Examples: Each human, each mouse, each plant. When the treatment is assigned to an overarching population (e.g. each school, each hospital), the experimental units would be the school or hospitals. Not the individuals within.

• Example: In the fertilizer experiment, if each plant is given a fertilizer treatment separately, then each plant is an experimental unit.

  • Treatment applied to individual plants → plants are experimental units.

Factors

A factor is a variable that the researcher changes or controls to see its effect on the response. A variable that is used to explain or influence another variable.

• In experiments, factors are often the treatments or conditions that researchers deliberately change.

Example: If you want to know whether fertilizer affects plant growth, then fertilizer type is a factor.

• Factor: Fertilizer type.

• Categorical (x)

Levels or Factor Levels

Levels are the different values or categories of a factor.

• Factor = the characteristic you're changing or comparing.

• Levels = the different versions of that characteristic.

Example: For the factor "fertilizer type," the levels might be:

• No fertilizer

• Fertilizer A

• Fertilizer B

Here, the factor has 3 levels.

Another example:

• Factor: Amount of water

• Levels: 100 mL, 200 mL, 300 mL per day

Subjects

A subject is an experimental unit when the unit is a living person (or sometimes an animal).

Example: Suppose researchers test three diets on people.

• Factor: Diet type

• Levels: Diet A, Diet B, Diet C

• Subjects: The people participating in the study

Every subject is an experimental unit, but we usually use the word subject when talking about people.

Categorical Factors (not Quantitative)

A factor whose levels are categories or groups, not numerical amounts. The levels represent different types or labels rather than quantities that can be measured.

Examples of Categorical Factors

1. Fertilizer Type

• Factor: Fertilizer Brand Type

• Levels:

  • No fertilizer

  • Fertilizer A (Green Grow)

  • Fertilizer B (Quick Sprout)

  • Fertilizer C (Off Spray)

Value

A value is a specific result or category that a variable can take. A variable can be thought of as the specific thing that you are recording, whereas the value can be thought of as the specific answer that you get.

Examples:

Variable: Test Score

• Values: 78, 85, 92

Variable: Eye Colour

• Values: Blue, Brown, Green

Factor: Teaching Method

• Values (Levels): Online, In-person, Hybrid

Treatment

A treatment is the specific condition applied to an experimental unit or subject. When there is more than one factor, a treatment is usually a combination of levels from those factors.

For example:

• Factor = Fertiliser Type → Levels = A, B

• Factor = Water Amount → Levels = Low, High

Treatment = Fertiliser A + High Water

When there is only one factor, the treatments are often the same as the factor levels.

Response Variable

A response variable is the outcome that is measured to see the effect of the treatment. This is the result that you are interested in measuring.

Example: Fertilizer Experiment

• Experimental Unit: Plant

• Factor: Fertiliser Type

• Levels: None, A, B

• Treatment: Fertiliser A (for a particular plant)

• Response Variable: Plant Height

The researcher wants to see whether the different treatments affect plant height.

Placebo

A pill that looks the same as real medication, but has no medical effect. It may have a psychological effect, like any real medication.

In a study of sickle cell anemia, 150 patients were given the drug hydroxyurea, and 150 were given a placebo. The researchers counted the episodes of pain in each subject.

Identify the experimental units, factor, levels, treatment and response variable.

Step 1: Identify the experimental units (subjects). “Who received the treatment?”

• 150 patients receiving hydroxyurea

• 150 patients receiving placebo

  • Subjects = All 300 patients.

Step 2: Identify the factor. “What did the researchers purposely change?”

• Changed which pill each patient got.

• Factor = The type of pill received.

Step 3: Identify the levels. “What specific values or categories did the factor level have?”

• Possible values include hydroxyurea, and placebo.

• Categorical factors.

Step 4: Identify the treatments. "What is the actual condition applied to a subject?”

• There is only one type of pill with two levels. Hydroxyurea, and placebo treatments.

Step 5: Identify the Response Variable. “What did the researchers measure?”

• Counted the episodes of pain.

Design Experiments Memory Trick

When you see an experiment, ask:

• Who received something? Experimental units/subjects

• What was changed? Factor

• What choices/values did it have?Levels

• What was measured? Response variable

For the previous example

• Patients = Subjects

• Type of pill = Factor

• Hydroxyurea & Placebo = Levels

• Actual condition applied = Treatment

• Pain episodes = Response variable

The effectiveness of three laundry detergents is being compared. Nine white sheets are each stained with grape juice, motor oil and mustard. Three sheets will each be washed with Tide detergent, three will be washed with Cheer and the remaining three will be washed with Sunlight. The sheets will be randomly assigned which detergent they will be washed with, and all of them will be washed in the same washing machine. After they are washed, the amount of stains removed will be compared for the three detergents.

What is the units, factor, levels, treatment and response variable?

• Experimental Units: The sheets.

• Factor: Brand of Laundry Detergent (Sunlight, Cheer or Tide)

• Levels: Sunlight, Cheer or Tide (Laundry Detergent).

• Treatment: Detergent assigned to a sheet.

• Response Variable: The effectiveness of the detergents for removing stains.

Since there is only one factor, with three levels, there are only three treatments. Since we're only changing one thing (the detergent), each detergent represents one treatment, giving us three treatments in total

A study is to be conducted to determine the effect of giving employees on an assembly line 1, 2, or 3 rest breaks and varying the speed of the assembly line to 20, 25, or 30 items per hour on the number of defects in a production process per 1000 produced items. The manager of the factory uses 90 of his employees to participate in the experiment. Each employee will be assigned one of the treatments. Outline the experimental units, response variable, factors, levels and treatments.

What is the units, factors, levels, treatments and response variables?

Experimental Units: 90 employees

Response Variables: The # of defects per 1000 produced.

Factors:

• Factor A: Number of rest breaks

• Factor B: Speed of the assembly line.

Levels:

• Factor A: 1 break, 2, breaks or 3 breaks.

• Factor B: 20, 25, 30 items per hour.

Treatments: Consists of the various combinations of factor levels (Number of breaks) + (Assembly line speed).

Formula for Number of Treatments

Number of Treatments = (Levels of Factor 1) x (Levels of Factor 2) x (Levels of Factor 3)

So in reference to the previous example

• Rest breaks = 3 levels (1,2, 3)

• Assembly speed = 3 levels (20, 25, 30)

• Treatments = 3 × 3 = 9

• 9 treatments

Formula for Experimental Units per Treatment

Once you know the # of treatments, divide the total number of experimental units by the number of treatments.

• Units per Treatment = Total experimental units divided by the number of treatments

• Units per Treatment = 90/9 = 10

• 10 employees (experimental units) per treatment

Final answer is 9 treatment combinations and 10 employees assigned to each treatment.

3 Principles of Experimental Design

Impose treatment:

• Researchers deliberately apply one or more treatments (conditions) to experimental units (people, animals, plots of land, etc.).

• This allows the experimenter to study the effect of the treatment on the response variable.

• Example: Giving one group a new fertilizer and another group the standard fertilizer.

Randomization:

• Experimental units are assigned to treatments by chance.

• Randomization helps balance out lurking or confounding variables across treatment groups, reducing bias.

• Example: Randomly assigning patients to receive either a new drug or a placebo.

Replication control:

• The treatment is applied to many experimental units, not just one.

• Replication increases the reliability of results and helps distinguish real treatment effects from random variation.

• Example: Testing a fertilizer on 50 plants instead of just 1 or 2.

Randomization in Experimental Design

The process of assigning experimental units (such as people, plants, or animals) to treatment groups by chance rather than by choice.

• It reduces bias by preventing researchers from influencing who receives which treatment.

• It helps ensure that other variables (such as age, health, or environmental conditions) are spread evenly among the groups.

• This makes it more likely that any differences in outcomes are due to the treatment itself, not some other factor.

Completely Randomized Design (CRD)

An experimental design in which all experimental units are assigned to treatments completely by chance. Every unit has an equal opportunity of receiving any treatment, which helps reduce bias and ensures that differences in results are due to the treatments rather than other factors.

• Example: Suppose a researcher wants to test three different fertilizers on 30 plants. The 30 plants are randomly assigned so that 10 plants receive Fertilizer A, 10 receive Fertilizer B, and 10 receive Fertilizer C. Because the assignment is random, any differences in plant growth can be more confidently attributed to the fertilizers rather than to differences among the plants themselves.

Steps in CRD

1. Select the experimental units
   Choose the subjects or items to be used in the experiment (e.g., plants, animals, students).

2. Identify the treatments
   Determine the different treatments or conditions that will be compared.

3. Randomly assign treatments
   Assign the experimental units to the treatments completely at random so that each unit has an equal chance of receiving any treatment.

4. Apply the treatments
   Administer the assigned treatment to each experimental unit.

5. Collect data
   Measure and record the response variable or outcome for each unit.

6. Analyze the results
   Use statistical methods to compare the treatment effects and determine whether any observed differences are significant.

Example of CRD

A researcher wants to compare three fertilizers (A, B, and C) using 30 plants.

1. Select 30 plants.

2. Define the treatments: Fertilizers A, B, and C.

3. Randomly assign 10 plants to each fertilizer.

4. Apply the fertilizers.

5. Measure plant growth after a set period.

6. Analyze the growth data to determine which fertilizer performs best.

Control

Means taking steps to make sure that factors other than the treatment being studied do not affect the results. The goal is to ensure that any differences in the response variable are caused by the treatments and not by outside influences.

• Control is keeping other conditions as similar as possible so that the effect of the treatment can be measured fairly.

• For example, when testing fertilizers on plants, all plants should receive the same amount of water, sunlight, and soil. The only thing that should differ is the fertilizer.

Methods of Control

This ensures that things are being kept fair by following one of the following methods of control. This help ensures that the treatment is the main reason for any differences in the results.

1. Same Environment: Keep all conditions the same except for the treatment.

2. Control Group: Use a group that receives no treatment for comparison.

3. Double-Blinding: Neither the participants nor the researchers know who receives which treatment.

4. Blocking: Group similar experimental units together before randomly assigning treatments.

Control Group

A group in an experiment that is treated the same way as the treatment group, except it does not receive the treatment being tested. Instead, it may receive a placebo or no treatment at all.

The control group provides a baseline for comparison, allowing researchers to determine whether the treatment actually caused any observed changes.

• Example: In a study testing PROZAC for depression, one group receives PROZAC (treatment group) and another group receives a placebo (control group). If the PROZAC group shows greater improvement, researchers can conclude that PROZAC may be effective.

Double-Blind Experiment

An experiment in which neither the participants nor the researchers know who is receiving the actual treatment and who is receiving the placebo or comparison treatment.

• This helps prevent bias from affecting the results. Participants cannot change their behavior based on what they think they are receiving, and researchers cannot unintentionally influence the results.

• this is popular when using human subjects.

• This helps to reduce the placebo effect and biased interpretation of results.

Replication

Means applying each treatment to more than one experimental unit instead of just one. The purpose of replication is to make the results more reliable and to reduce the impact of random variation or chance. Repeat each treatment on many subjects to reduce variation in results.

Example: If you are testing three fertilizers, you would not apply Fertilizer A to only one plant. Instead, you might apply:

• Fertilizer A to 10 plants

• Fertilizer B to 10 plants

• Fertilizer C to 10 plants

By using multiple plants for each treatment, researchers can be more confident that differences in growth are due to the fertilizers and not because of one unusually large or small plant.

Blocking

A method of control where researchers group similar experimental units together before randomly assigning treatments within each group.

• The purpose of blocking is to reduce the effects of known differences among the experimental units, making it easier to see the true effect of the treatment.