Null Hypothesis Significance Testing

Steps in Hypothesis Testing

1. formulate research hypothesis

2. set up the null hypothesis

3. obtain the sampling distribution under the null hypothesis (choosing number of participants: past literature, power analysis)

4. obtain data; calculate statistics

5. given the sampling distribution, calculate the probability of obtaining a value that is as different as the one you have

6. on the basis of probability, decide whether to reject or fail to reject the null hypothesis

When do you reject the null hypothesis

• significance levels

• conventional levels

• the score that corresponds to alpha = critical value

  • if p < alpha, reject H0

  • if p > alpha, fail to reject H0

• the smaller the alpha, the more conservative the test (we are more likely to conserve H0)

Directional hypothesis

• indicates a direction (ex; time spent in class increases mind wandering)

• use a one tailed test

Nondirectional hypothesis

• does not indicate a direction (ex; time spent in class could increase or decrease mind wandering)

• use a two tailed test

Type I Error

• your test is significant (p < .05), so you reject the null hypothesis, but the null hypothesis is actually true

Type II Error

• your test is not significant (p > .05), you don’t reject the null hypothesis but you should have because it’s false

True or False — A significant result means the effect is important

• False

  • just because it is statistically significant does not mean it is actually significant in the real world

True or False — A non-significant result means that the null hypothesis is true

• False

  • tells us only that the effect is not big enough to be found with the sample size we had

  • fail to reject the null — doesn’t mean the effect is not there, just means you didn’t find it

True or False — a significant result means that the null hypothesis is false

• False

  • not a distinct yes or no, probabilistic reasoning — only 95% confident

True or False — The p-value gives you the effect size

• false

  • we need to do some further calculations to find effect

True or False — The population parameter (μ) will always be within a 95% confidence interval of the sample mean

• false

  • it’s an estimate

True or False — The sample statistic (M) will always be within a 95% confidence interval of the mean

• True

  • we calculate the confidence interval based on the sample mean so it is always right in the middle

Trends to Circumvent Problems with NHST

• effect size calculations — standardized so we can compare

• confidence intervals

Outcome = __________

(model) + error

• outcome — dependent variable

• model — independent variables

B₁

Slope

• increase in y for every increase in x

B₀

y-intercept

• y when x is 0

Parameter Estimates

• different samples drawn from the same population will likely yield different values for the mean

Sampling Error

• the difference between my sample and the population value

Sampling error formula

Interval Estimates using σ

• 95% confidence interval

  • lower limit: M - [z(critical)σ_M]

  • upper limit: M + [z(critical)σ_M]

Standard Error of the Sampling Distribution of Means

• used when we don’t know σ

Standard Error of the Sampling Distribution of Means Formula

Interval Estimates without σ

• 95% confidence interval

  • lower limit: M + [t(critical)SE_M]

  • upper limit: M + [t(critical)SE_M]

z scores vs. t scores

• z scores

  • σ is known

  • large sample sizes

• t scores

  • σ is not known (we introduce more error with SE_M when σ is not known and t corrects for this)

  • small sample sizes

t critical

• t[critical] = t(n-1)

t critical table

• degrees of freedom = n-1

  • then find percentage