Stats 6 - Effect sizes and confidence intervals

What is a point estimate?

A single-number estimate of a value

This value alone doesn’t tel the whole story of the data
Like an IQ test is a single measure of intelligence

What is a margin of error?

An estimate of the sampling error; the portion of the confidence interval formula to the right of the plus/minus sign

IQ is ± 5

P value

Tells you the probability of finding the experimental sample mean within the distribution of control-group sample means

What are some common mis-interpretations of what p values mean?

P is treated incorrectly as the arbiter of worth/goodness or correctness of a research study

Assigned too much value

How do sample sizes relate to point estimates, margin of error, and statistical significance?

A sample size helps point estimates to be more accurate

Increasing the sample size decreases the margin of error

However, when the sample size is extreme, you may achieve statistical significance because your sample size is so big, not necessarily because the control group differs from the experimental group

What is a Type I error?

False positive

Alpha

Rejecting the null hypothesis when the data aren’t statistically significance

What is a Type II error?

False negative

Beta

Failing to reject the null hypothesis when the data are actually statistically significant

What is an effect size?

Practical significance

Measurement of how substantial an effect is in the real world

What is the difference between practical significance and statistical significance?

Statistical significance indicates a result is unlikely to be due to chance, while practical significance indicates the result is meaningful or important in a real-world context

How can we interpret Cohen’s d?

Cohen’s d is interpreted using benchmarks for effect size range

What is a confidence interval? What does it tell us compared to a p value or an effect size?

An interval of variable values that is likely to contain a specified value, usually a population mean

An interval/range of data that we are reasonably confident, but not certain, includes the true value of mu

What does the confidence interval try to solve?

The CI focuses on a solution to the problem that the sample mean will rarely be exactly the same as the population mean