Exam three statistics

Effect Size

One way to measure the impact of the independent variable on the dependent variable

If we reject the null hypothesis what does that mean?

The likelihood of observing our results is less than alpha, does not indicate how much of an impact the independent variable has on the dependent variable this is why you need effect size

Eta Squared ( n squared)

One way between subjects ANOVA, indicate the proportion of variance or percent of variability in the dependent variable that can be explained by the treatment or accounted for the independent variable, SSb / Set

How and when it eta squared reported?

reported in the research literature, and reported as a percentage

Are omega squared and eta squared more conservative than cohen’s d?

Yes

Omega Squared

One way between subjects, more conservative than eta squared, reported as percentage and reported in research literature, w squared = SS effect - (df effect)(MSerror)/ SS total + MS error

Partial Eta Squared, n squared p

One way within subjects: SS effect/ SS effect + SS error

Partial Omega Squared

One way within subjects, more conservative then eta w squared p = df effect times (k-1)(MS effect - MS error) / (k-1) MS effect + (n-k-1)(MS error)

Type One error

Is false positive, there really is not an effect nut you find one in your study (you reject a true null hypothesis)

Type Two error

There really is an effect but you did not find one in your study. The probability of retaining a null hypothesis that is actually false.

Type 3 error

A source of error that occurs for one tailed tests where the researcher decides to retain or reject the null hypothesis because the region of rejection was located on the wrong tail

Why is the statistically significant decision wrong for type 3 error?

Because the region of rejection is in the wrong location

Power

The probability of rejecting a false null hypothesis

The more powerful the test of the null hypothesis, will you be more or less likely to reject the null hypothesis?

The more likely we will end up rejecting the null hypothesisW

What is power impacted by?

Sample size, alpha, variability in the measured variable, effect size

What should power be?

0.70 or higher

What is the probability of making a type 2 error then if power is 0.70?

0.30

How to increase power?

• Increase sample size

• measure the variables using interval or ratio scales because we can do more powerful parametric analysis

• utilize designs that control for extraneous variables that may influence participants

• be more selective about participants

• maximize treatment techniques

Beta

Probability of making a type two error is called beta

What type of relationship exists between alpha and beta?

inverse relationship between alpha and beta

Alpha

Probability of making a type one error, level of significance

Alpha is what

The largest probability of a type one error that we will allow and still reject the null hypothesis

Multiple Comparisons

Conducted after the reaction of the null hypothesis of the ANOVA

Multiple comparison tests from most liberal to most conservative (most statistical power to least)

1. Fisher’s Protected T test or Fishers LSD

2. Tukey’s Honestly Significant Difference

3. Bonferroni t-test

4. Scheffe’s test

Steps of Multiple Comparisons

1. Compute the mean differences

2. Compute the critical value

3. Compare the test statistic

What does it mean is mean difference > critical value

statistically significant

What does it mean when mean difference < critical value

not statistically significant

Fisher’s Protected t-test

Also known as Fisher’s LSD

Fisher’s LSD

Performed after a significant ANOVA, computes a protected t test instead of other t tests, more powerful statistical tests than other t tests

How many groups for Fisher’s LSD?

k = 3

When is Fisher’s LSD used?

post hoc (A posteriori)

How do you calculate Fisher’s LSD?

1. Identify all of the pairs and calculate the mean difference for each pair

2. Calculate LSD

   1. T crit

   2. MS within

   3. n1

   4. n2

3. Compare EACH mean difference to LSD

Tukey’s HSD

Alternative to Fisher’s LSD, more conservative than Fisher’s, start with comparing the smallest group mean to the largest mean to the critical value, then compares the other critical value if necessary

How many conditions for Tukey’s HSD?

More than 3 conditions

When is Tukey’s HSD done?

post hoc (A posteriori)

Steps to Tukey’s HSD

1. Identify all pairs and calculate the mean difference for each pair

   1. start with largest mean - smallest mean

2. Calculate HSD

   1. degrees of freedom within

   2. MS within

   3. N

3. Compare each mean difference to HSD

q

standardized range statistic

what is q’s definition?

a statistic use determine critical values for comparing pairs of means at a given range

Bonferroni test

A conservative multiple comparison test that maintains experiment wise alpha by dividing the number of pairs (j), ideal with one way within subjects ANOVA

How many conditions for Bonferroni?

The number of conditions depends: as number of samples increases, the test becomes more conservative

When is Bonferroni done?

A priori

Steps of Bonferroni Test

1. Calculate testwise alpha

2. determine critical value from the t table using

   1. testwise alpha

   2. df = n-1

   3. Two tailed test

3. Performed related samples t test with each pair

4. Compare each related samples t statistic to the new t critical and make a decision

Scheffe’s Test

A conservative statistical analysis that is often used for unplanned complex comparisons. It can also be used with unplanned pairwise analyses but it is not recommended. Can be used with balances and unbalanced designs.

How many conditions for Scheffe’s

Recommended for more than two

When is Scheffe’s done?

post hoc (A posteriori)

Steps for doing the Scheffe’s Test

1. Gather descriptive statistics

   1. n for each group

   2. Means for each group

   3. MS within

   4. k

   5. F crit

2. Calculate F contrast for each pair

3. Calculate F scheffe

4. Calculate F contrast to F critical for each pair and then make your decision

pairwise comparison

A statistical comparison for the difference between two group means (pair), done after you determine if the difference is statistically significant

Complex comparisons

A statistical comparison fo the difference involving more than two group means, example: comparing group means all at the same time

A priori multiple comparisons

Comparisons that are planned before collecting the data and conducting the study, tend to be well-grounded theory and/or existing research, can have a lower critical value, does not involve the same risk of a high experiment wise alpha as the posteriori comparison, not appropriate with exploratory research

A posteriori comparisons

Researcher plans to examine the comparison after a significant ANOVA to determine which group means significantly differ, typically seen in exploratory research studies, used when k > 2 because multiple comparisons are needed

Post hoc analyses

Statistical tests performed after an initial study or experiment has concluded and the data have already been collected and examined

Multifactorial design

Complex design

Complex Research Design

Require at least two independent variables, both between subjects, both within subjects, or mixed, simplest version 2 × 2

2 × 2 complex research design meaning

1st independent variable = 2 levels

2nd independent variable = 2 levels

Experimentwise alpha

The alpha level or probability of committing a type one error, for each test or pairwise comparison made on the same data

Testwise alpha

Experimentwise alpha/j

A priori

if you decide BEFORE do this

A posteriori

If you decide this AFTER do this

j

number of pairwise comparisons: the statistical comparison for the difference between two group means

Beta

β (beta)

d

Cohen’s d (eta and omega are more conservative than this)

F Scheffe

critical value = (k − 1) × F_crit

Experimentwise alpha

α_EW

Testwise alpha

α_TW

Similarities between One way Between subjects and Two way Between subjects

• tests evaluate whether group means differ for a quantitative dependent variable

• Categorical independent variable

• Use of F tests

• Both require independence of observations

• Normal distributions

• Homogeneity of variances

One way Between Subjects ANOVA

• one categorical

• tests one main effect

• interaction effect not tested

• groups formed by the levels of a single factor

• generated a single main F statistic

• Null: Mus are equal

• Alternative: At least one mu differs

• F obtained: MS between divided by MS error

How to interpret one way between subjects

Statement about what the researcher predicted. Statement about the averages and standard deviation. Statement about the statistical significance. F ( degrees of freedom between, degrees of freedom within, ) = f obtained, and then p value is less than or greater than 0.05.

Two Way Between Subjects

• Two categorical independent variables

• tests three hypotheses

  • Main effect A, Main effect B and an A x B interaction effect

• Cells formed by factorial combinations

• Separates F statistics for Factor A, then B, and then interaction

  • three F statistics

Main effect

Null: Mus are equal

Alternative: At least one mu differs

Interaction effect

Null: Factors are not dependent

Alternative: Factors are dependent

What is grand mean?

the mean of the means: or the average for all the data points if they weren’t divided into smaller conditions.