Week 9; research methods; power analysis

Power analysis:

Power is the probalability of NOT making a type II error, so finding the effect that exists in a relationship

  • 1- beta (1- chance of type II error)

Type 1 error- false positive, rejecting null hypothesis when there is no relationship between the variables

Type 2 error- false negative, accepting null hypothesis when there IS a relationship between variables

What effects the chances of a type I or type II error:

effect size

We want to know if a relationship exists between a relationship BUT we also need to know how STRONG this relationship is. The magnitude of this is the effect size.

the smaller the effect size, the larger the sample needed to avoid type II error.

for T tests, Cohen’s d is often the effect size measure of choice. Cohen’s d helps when we want to make direct comparisons but have different measures used for different variables.

  • 0.2→ is a small or weak effect size

  • 0.5→ moderate

  • 0.8→ strong

Whereas with r

  • 0.1, small, effect explains 1% of the total variance

  • 0.3, moderate, explains 9% of the total variance

  • 0.5, strong, 25$

Pearson’s r ranges from -1 to 1

we can square r and this would explain total variance

Partial Eta squared (np²):

is used to measure effect size in ANOVA and explains that it is not explained by other variables in analysis

  • can range from 0-1

  • 0.01. small effect

  • 0.06 moderate effect

  • 0.14 strong effect

Cohen’s f²

  • used to measure effect size for each of the (continuous) variables or construct in multiple regression

    • 0.02

    • 0.15

    • 0.35

sample size:
smaller sample size means that there is a greater chance for null hypothesis to be accepted, especially if researchers are trying to detect a small effect size

small samples may be enough when researcher are confident that they are trying to find a larger effect size

Unsystematic variability:

sources of UV:

  • measurement error

  • irrelevant individual differences

  • situation noise

They use the most appropriate statistical analysis

SO SIGNIFICANCE CRITERION (ALPHA)

lower alpha means less strength to test for a relationship

typical significance criterion is 0.05 (alpha rate), in a typica analysis there is a 1:4 ratio of a type I to type II error

Power is effect by:

  • sample size

  • effect size

  • alpha level

retrospective power analysis: calculate power

by knowing our

  • alpha level

  • calculated effect size

  • sample size

then we can find our power

power < 0.8 = underpowered

power > 0.8 = confident

Prospective power analysis

These are good to determine the required sample size for our study

We also would estimate our effect size based off of…

  • closely related data used in past research

  • conducting a pilot study

  • or decide on small/ medium/ large effect size like a rule of thumb

We use G* power to measure power analysis

Types of analysis

A priori analysis→ useful to determine sample size you need for your study

Post- hoc→ retrospective, to test if your study had enough power to determine the effect

Sensitivity→ to determine critical population effect size

Compromise→

How do we report a power analysis?

include:

  • effect size

  • sig level

  • required power

  • sample size