Issues with NHST Logic and One-Sample t-Tests

What is a major limitation of NHST?

It only gives a binary decision: Reject 𝐻₀ or Fail to Reject 𝐻₀, lacking context such as the magnitude of effect.

Why can sample size affect NHST decisions?

Larger sample sizes reduce standard error, which can increase test statistics and lead to rejecting 𝐻₀ even for small differences.

What additional test estimates are recommended alongside NHST?

Effect sizes and confidence intervals (CIs).

What is Cohen’s d in a one-sample z-test?

d= xˉ−μ

————

σ​

What are Cohen’s guidelines for interpreting d?

• Small: ~0.2

• Medium: ~0.5

• Large: ~0.8

What is the formula for a 95% CI in a z-test?

xˉ±(1.96×SE)

What does a CI actually mean (correct interpretation)?

In the long run, 95% of CIs calculated from all possible samples would contain the true population mean.

What factors narrow a CI?

• Larger sample size (↓SE)

• Lower population standard deviation

What is a Type I error (α)?

Incorrectly rejecting a true null hypothesis.

What is a Type II error (β)?

Failing to reject a false null hypothesis.

What is statistical power?

1−β; the probability of correctly rejecting a false null hypothesis.

What increases statistical power?

• Larger effect size

• Larger sample size

• Smaller standard error

When do we use a t-test instead of a z-test?

When population standard deviation (σ) is unknown and must be estimated.

What is the formula for t in a one-sample t-test?

t=​xˉ−μ

—————-​
s(sqaure rooted)N

What is the degrees of freedom (df) in a one-sample t-test?

df=N−1

What happens to the t-distribution as df increases?

It approaches the normal (z) distribution.

What is the formula for Cohen’s d in a t-test?

d=xˉ−μ​

———

s

What is the formula for CI in a one-sample t-test?

xˉ±(tcrit×s/(square rooted) N)

What are key differences between z and t distributions?

t-distributions have heavier tails and critical values vary by df.

What is the only factor affecting power that researchers can directly control?

Sample size