psych analysis 11 - inferential statistics

two major approaches to make predictions about a population based on sample data

• null hypothesis significance testing

• confidence intervals and effect sizes

null hypothesis significant testing (NHST)

a method researchers use to determine whether their results provide enough evidence to reject the null hypothesis.

• u - population mean of DV

• subscript - group

alternative hypothesis

regular hypothesis

u

population mean of DV

• subscript: group

p

population correlation between 2 variables

• subscript - two continuous variables we are correlating

method for making predictions from our sample data about a population

1. gather sample

2. compute test statistic

3. compute p value for test statistic

   1. is test one or two tailed?

4. compare test statistic p value to alpha level

5. decide whether to reject/retain null

6. decide whether t reject/retain the null

   1. if p value is less than alpha level - reject null

   2. p value greater than or equal to - retain null

how do we interpret statistics using NHST?

test statistic —→ probability ——→ desicion

what is p?

• probability that results are due to chance

• we compare our result probability with a pre-determined probability threshold known as Alpha.

  • used to decide whether hypothesis is supported or not

  • set by researches

  • psychology: p<0.05

steps for using alpha to make decisions in NHST

1. decide alpha level (p<0.05)

2. use stats to calculate probability of our test statistic (mean difference)

3. compare test statistic probability to alpha

   1. if less than alpha, reject the null (significant relationship)

   2. if equal to or greater than alpha, retain the null (non-significant relationship)

alternative hypothesis

• differences are due to IV

• we reject the null

• IV had an effect

• probability less than 0.05 under null

• p<0.05

null hypothesis

• difference explained by chance

• fail to reject the null

• IV had no effect

• probability greater than 0.05 under null

• p>0.05

one tailed test (directional)

• tests whether a parameter is strictly greater than or less than a reference value

• uses < and >

• entire significance level sits on one tail of the distribution curve

• should only use when thee is reason to avoid change in the opposite direction

two tailed test

• tests whether a parameter is different from a reference value in either direction

• uses =

• singificance level is split equally between both tails

• often need a larger sample size

NHST limitations

• cannot test no relationship

  • can never accept the null

  • can never conclude groups are equal, or the same

  • NHST is a binary decision (supported or not supported)

type 1 error

false positive that happens when you reject a true null

type 2 error

false nefgative which happens when you fail to reject a null

NHST vs confidence intervals

NHST

• goal - decide whether to reject or retain null

• what we interpret - point estimate (test statistic) and it’s p value

• what we can conclude - is the relationship (or difference) significantly different from zero?

Confidence intervals

• goal - estimate a relationship

• what we interpret - point estimate (test statistic) and confidence interval around point estimate

• what we can conclude - how strong is the relationship (or difference)

confidence intervals

• range of plausible values for the population parameter

  • we are XX% confident that the interval contains the true population parameter

  • width (XX%) is set but the researcher. Usually 95% in psychology