Design of Experiments Midterm

Experiment

A structured set of coherent tests that are analyzed as a whole to gain an understanding of the process

Efficiency

A efficient experiment derives the required information with the least expenditure of resources

Examples of resources

Time, Material, Energy, Money

Stages of deigning an experiment

1. Gain knowledge of the process 

2. Develop clear goals and objectives

3. Choose a response variable

Precise (definition)

Repeatable but not necessarily accurate

Relevant (definition)

Related to the design objective

Statistical 

if it unlikely to have been caused by chance

Meaningful

if it large enough to have a real world impact

Degrees of freedom

A way of counting the amount of data in an experiment. For every average of the data reduces the total DF by 1

Something is statistically different if

The data sets have a low likely hood that the data groups come from the same population

T test

T is the calculated difference represented in units of standard error. The greater the T the more likelihood they are from difference populations.

P Values do not tell you

1. The probability that Ho is true

2. The meaningfulness (or impact importance) of the observed effect

3. Note that P=0.05 is an arbitrary convention

Alpha Risk (type 1)

The probability of deciding the samples are different when they are the actually the same

Result: Supplier is scrapping good parts

Beta Risk (Type 2)

The probability of deciding the samples are the same when they are really different (More difficult to control)

Result: The company buys bad parts

F- Test

Compares the variance looking for a difference in variance that is larger than can be expected by random chance

F<=1 Variances are likely the same

F>>1 The variances are likely different

ANOVA (Analysis of Variance)

Variance between groups is the “signal” (Main effect)

Variance within groups “groups” (Error)

F stat for Anova

Signal/Noise

Requirements for ANOVA

1. Normally distributed

2. Independent

3. Homogenous 

Total Value for ANOVA (Sum of squares)

Calculated by summing the square of the difference between the values and the mean. It is also equal to the main effects and the error. 

What is the purpose of a 2 way ANOVA

Identify the factors that cause significant changes in the response variable

How an ANOVA avoids a large alpha risk

Variance is pooled

Error (definition)

Difference between data point and true mean

Residual (definition) 

Difference between data point and population mean

What do we if the null hypothesis is rejected in a 2 way ANOVA

If we have more than 2 groups we should run a post-hoc analysis to find which individual groups are actually different

Interpreting a 2 way ANOVA

1. Interaction: Always look at this first, if there is an interaction effect, we ignore the main effects and must move to step three

2. Main effect: If there is no significant interaction, we can look at the main effects. If nothing is significant, we accept the null hypothesis and stop

   1. Post Hoc. If there are significant main effects (or interaction) we want to explore, we can then run a post-hoc

Why should we be careful when we use a T-Test

Repeated tests run the risk of a type 1 error

What is factorial design

Factorial design is a way of setting up experiments so that all possible combinations of factors are tested. Full Factorial designs allow us to

1. Detect interaction

2. Asses main effects

3. Create a maximally efficient experimental setup

How Full Factorial Summary

Efficiency: The full factorial gives the same precision for main factors but with fewer total runs

Requires attention to detail: With fewer runs, the influence of each data point has a large effect on the results. Record keeping is critical! Label everything with the correct treatment combination not just the run number.

Assumptions for full factorials

1. Errors are homogenous

   1. Errors are Normally distributed

2. Errors are independent (Randomized from testing order)

ANOVA vs Factorial

Factorial: Full factorials describe a method of setting up or disning your experiment before you run it. it is especially helpful when you need to move beyond 2 factors and 2 levels.

ANOVA: When you want to analyze the results, use ANOVA to compare populations and find interactions

Summary of all tests discussed

T-Test: Allows us to directly compare2 groups. BUT must use judiciously, as repeated tests run the risk of a type 1 error

ANOVA: Allows us to compare multiple groups by pooling variance

Full factorials: helps us catch interaction effects efficiently by designing multi-factor experiments. BUT number of tests grows really quickly the more factors we need to analyze.

Fractional Factorials: Very similarly to full factorials by allowing for efficient testing of main effects and interaction but allows you to ignore higher level interaction or interactions you think are unlikely to matter.

Equation for 2 level interaction

K(k-1)/2

Confounding Rules

1. NEVER confound single effects with each other

2. Do not confound single effects wiht 2 factor interactions without prior knowledge

3. Avoid confounding 2-factor interactions with other 2 factor interaction

Generator 

A single effect is confounded with a multi-level interaction 

Resolution

The number of factors in the shortest set of a defining contrast (or generator)

Why Confound

Confounding us usually resource driven

• We get to reduce a factorial test that may need a lot of tests to one with much less

3 Uses of experimental designq

1. Characterizing a process

2. Troubleshooting

3. Quantification of errors

Unlike tests, experiments are _____

Information-oriented

T/F A systematic experimental approach to optimizing variables is usually more efficient than testing all combinations

True

Fill in the blank: An experiment is a ______ set of coherent ______ that are ______ as a whole to gain an _______ of the process

Structured, Tests, Analyzed, Understanding

Even with an unrestricted budget, which of these is the most limited resource

Time

What are 4 ways why a careful approach to designing experiments is superior to testing in the “School of hard knocks”

1. A structured plan of attack

2. Meshes with statistical analysis tools

3. Forces experimenter to organize

4. Is more efficient

What are examples of possible sources of prior knowledge

Pilot Studies (small experiments), Formal education, previous reports

Why is it important to include operators/Technicians in the brainstorming process

They know from hands-on experience how the device or system actually functions

Unlike Noise factors, control factors are…

Under our design authority

quiz 1 questions 13

For many types of experiments, we do not know what the target is. Therefore, the only measure we can look at to assess the quality of our response data is...

Prevision/Repeatability

Your text discusses the challenge of choosing meaningful response variables that are indicative of performance. Why is it often not feasible to directly test product performance (e.g. failure rates, useful life)?

Sometimes testing the product performance would take too much time or resources so by testing something indicative of performance limits what needs to be tested

Hypothesis testing is centered around a starting assumption, or null hypothesis, that the compared means are...

The Same

One way to reduce the risk of coming to an incorrect conclusion is to take a ____ number of observations. The central limit theorem assures us that if we do, the distribution of our data will approximate the ________ __________

Larger, Normal Distribution

o determine how many samples you need to come to a correct conclusion, you could use the formula shown below (12.11 in your textbook).

This formula shows that sample size is dependent on... (select all)

Chosen beta risk, minimum difference we want to find, and chosen alpha

My sample size calculation is giving me a required number that is far to large for me to test with the resources I've been assigned. If I'm designing an experiment where I need to keep a low risk of both a type 1 and a type 2 error, what else can I change about my experimental design to reduce the number of measurements I have to make?

You can increase the minimum difference you want to find or in other terms increase the delta

Refer to Table 27.3 in the back of your textbook. If I'm designing an experiment where I want to...

• detect a difference that is twice the size of the population standard deviation

• with a type 1 risk of 0.05

• with a type 2 risk of 0.01

... how many samples will I need?

7

We skipped over this in class, but an important step in a 2-sample t-test is to check if the 2 groups have approximately equal _____. We do this by calculating the _____ statistic, which is also central to ANOVA analysis.

variance, F

I've calculated an experimental t-value of 3.5. If the critical t-value for my test conditions is 2, I should _______ the null hypothesis.

Reject

The Numerator of the F-Statistic calculation (Signal/Noise)

Signal

The portion of the F stat that is calculated by treating all groups as if they came from the same population (Signal/Noise)

Signal

The denominator of the F-Statistic calculation (Signal/Noise)

Noise

The portion of the F stat that is calculated by pooling the variance from within each group

Noise

(T/F) A one-way ANOVA will tell you which specific group is different from all the others (e.g. after comparing A, B, C, & D, it will tell you if A does not equal B).

False

I'm performing a one-way ANOVA, and I've calculated an experimental F-statistic of 0.05. If the critical F-value for my test conditions is 3, I should            the null hypothesis.

Accept

Your text makes a distinction between pure error and residual sources of variation. This is because strictly speaking, "left over" variation is the portion of variation to which we are presently unable to _____ a _______ It might not be truly random in the way we would expect from pure error.

Assign, Cause

Assumptions built into the ANOVA test

1. Errors are Independent

2. Variances van be pooled (they are homogenous)

3. The error variance must be normally distributed

Your textbook give an equation for calculating the number of treatment combinations in a full factorial design (4.1):This equation assumes you have a                  experiment (select all).

2 Level, Multi-factor

What is the rationale for converting factor levels (e.g. 100 and 200 psi) to +1/-1 design units prior to analysis? Give a brief explanation.

It allows independent analysis, and simplifies the mathematics of the calculation

Factorial designs are more efficient than 1-factor-at-a-time because we use runs more than once in our computations. We can do this because we assume the factors are... (give the 2 reasons)

Mathematically Orthogonal (independent), Balanced

As the number of factors increases, the efficiency of a full factorial experiment goes...

Up

Interactions are important but not common. In some disciplines, such as mechanical assembly, interactions...

Dont even exist

Your text states that 2-level full-factorial designs are not efficient when the number of factors increases above...

5 or 6

What does it mean for two or more effects to be "confounded"? Give a brief explanation.

When you confound 2 or more effects it means that you are mathematically considering their effects to be the same. By doing this you reduce the number of trials you need but you now are unable to distinguish between either effect.

Following the modulus algebra introduced in the chapter, how can the following equation be most simplified: CDD = ABCCDD

C=AB

Let's say I'm designing an experiment with six 2-level factors, and I only want to investigate main effects and 2-factor interactions. How many degrees of freedom do I need? (Hint: you can use Eqn 5.1 or Table 5.1 to find the number of 2-factor interactions).

21

Given 21 degrees of freedom for the wanted analysis, how many runs would I need in base 2-level design?

2^5 (32)

With 6 2 level factors and a run size of 2^5, what is the P=(k-n)

1

If for another experiment the exponent of the number of levels is 3 and the number of factors is 5, what is the fraction of the design?

1/4