Week 9; research methods; power analysis



Power analysis:


Power is the probalability of NOT making a type II error, so finding the effect that exists in a relationship

  • 1- beta (1- chance of type II error)


Type 1 error- false positive, rejecting null hypothesis when there is no relationship between the variables


Type 2 error- false negative, accepting null hypothesis when there IS a relationship between variables


What effects the chances of a type I or type II error:


effect size

We want to know if a relationship exists between a relationship BUT we also need to know how STRONG this relationship is. The magnitude of this is the effect size.

the smaller the effect size, the larger the sample needed to avoid type II error.

for T tests, Cohen’s d is often the effect size measure of choice. Cohen’s d helps when we want to make direct comparisons but have different measures used for different variables.

  • 0.2→ is a small or weak effect size

  • 0.5→ moderate

  • 0.8→ strong

Whereas with r

  • 0.1, small, effect explains 1% of the total variance

  • 0.3, moderate, explains 9% of the total variance

  • 0.5, strong, 25$

Pearson’s r ranges from -1 to 1

we can square r and this would explain total variance

Partial Eta squared (np²):

is used to measure effect size in ANOVA and explains that it is not explained by other variables in analysis

  • can range from 0-1

  • 0.01. small effect

  • 0.06 moderate effect

  • 0.14 strong effect


Cohen’s f²

  • used to measure effect size for each of the (continuous) variables or construct in multiple regression

    • 0.02

    • 0.15

    • 0.35


sample size:
smaller sample size means that there is a greater chance for null hypothesis to be accepted, especially if researchers are trying to detect a small effect size

small samples may be enough when researcher are confident that they are trying to find a larger effect size



Unsystematic variability:


sources of UV:


  • measurement error

  • irrelevant individual differences

  • situation noise


They use the most appropriate statistical analysis


SO SIGNIFICANCE CRITERION (ALPHA)

lower alpha means less strength to test for a relationship

typical significance criterion is 0.05 (alpha rate), in a typica analysis there is a 1:4 ratio of a type I to type II error


Power is effect by:

  • sample size

  • effect size

  • alpha level


retrospective power analysis: calculate power

by knowing our

  • alpha level

  • calculated effect size

  • sample size

then we can find our power

power < 0.8 = underpowered

power > 0.8 = confident


Prospective power analysis

These are good to determine the required sample size for our study

We also would estimate our effect size based off of…

  • closely related data used in past research

  • conducting a pilot study

  • or decide on small/ medium/ large effect size like a rule of thumb


We use G* power to measure power analysis



Types of analysis

A priori analysis→ useful to determine sample size you need for your study

Post- hoc→ retrospective, to test if your study had enough power to determine the effect

Sensitivity→ to determine critical population effect size

Compromise→


How do we report a power analysis?


include:

  • effect size

  • sig level

  • required power

  • sample size