POWER AND EFFECT SIZE

what is one of the first tasks when designing your experiment?

to determine the sample size (i.e., how many participants) you will be using.

what is the rule of thumb?

use as few subjects as possible - but you wouldn’t want your sample size to be too small (i.e., 3 participants)

what is significance?

• statistical significance is a way of determining whether the results obtained from a study are likely to be due to a real effect or if they could have occurred by chance.

• significance is often assessed using a measure called the p-value

• p-value below a certain threshold (commonly 0.05) is considered statistically significant

• significant means that there is a small chance (usually less than 5%) of getting these results under the Ho (the null hypothesis)

what does the p-value represent?

• p-value represents the probability of obtaining the observed results (or more extreme results) if there is no actual effect or difference in the population being studied.

what is ‘effect size’

• the size of the difference between your null and alternative hypotheses that you hope to detect

• reflects the strength of the relationship between the IV and DV.

what happens when the IV has a strong effect on the DV?

effect size will be large

what happens when the IV doesn’t have an effect on the DV?

effect size will be small

what did Jacob Cohen (1992) propose?

that the use of effect sizes in statistics could help to avoid the problems associated with NHST (p values) by providing an unbiased estimate of the true size of an effect.

what is meta-analysis?

• meta-analysis is a quantitative technique for synthesising the results of multiple studies of a phenomenon into a single study (meta experiment)

• it does so by combining the effect size estimates from each study into a single estimate of the combined effect size.

when is Cohen’s d used?

for studies comparing means of two independent groups (e.g., experimental vs. control group), Cohen's d is commonly used.

how do u calculate effect size?

(refer back to lecture notes)

what is the effect size measure?

Hedge’s g

what is Type 1 error?

• we reject the null hypothesis when the effect did in fact occur by chance and there is no difference between our conditions

  • we find effects that don’t exist!

  • FALSE POSITIVE RESULTS

what is Type 2 error?

• type II error: we do not reject the null hypothesis when there is in fact a true difference

  • we miss effects that do exist!

  • FALSE NEGATIVE RESULTS

what is power?

defined as the probability of NOT making a Type 2 error.

how do you calculate standard deviation?

• work out the mean.

• then for each number: subtract the Mean and square the results.

• then work out the mean of those squared differences.

• take the square root of that.

how do u calculate power?

• calculate the effect size using Cohen’s d

how do you increase power?

• increase sample size:

  • easiest and most common

  • when we increase the sample size, the mean that we obtain will be closer to the actual population mean

  • hence the distribution will be narrower - with lower variance and SD

• smaller beta:

  • therefore more power

• increase alpha:

  • eg - use .1 instead of .05

  • the problem with this is that it increases our chances of Type I error, and we don’t want that!

  • p should always be less than .05 for your findings to be taken seriously

• increase effect size:

  • if you look at prior literature, then you can change levels of the IV to make the effect higher

  • if testing caffeine, increase the dose

• decrease population variance:

  • this will narrow the distributions

  • in some tasks, giving extended practice to participants will decrease variance and increase power.