5 - Statistical theory and statistical thinking

5a - Confidence Interval for the mean

If the sampling distribution of the mean has a normal distribution:
- then 95% of sample means from that population are no more than 1.96 standard errors of the mean away from the population
- and we have a 95% confidence that the population mean is within 1.96 standard error of the mean of a sample mean

5a - Factors that influence confidence intervals

Because the width of a 95% CI varies depending on the standard error of the mean - which is calculated from the standard deviation and the sample size - the width of the interval varies with the standard deviation and the sample size

Confidence interval changes depending on:
- Varying the standard deviation
- the interval will be bigger the bigger the standard error is
- Varying the sample size
- the interval gets smaller the bigger the sample size is

5a -Varying the level of confidence for the interval

How wide are confidence intervals if I specify different confidence levels?

Varying the level of confidence:
- find the standard error of the mean
- and then specify a multiple of the SE that gives the required interval width. If the sampling distribution of the mean is normally distributed and we know the population SD, we can consult Tables for the Normal Distribution to identify our multiplier

5b - Different kinds of mean differences

Often in psychology we are interested in average differences, or differences between averages

When the scores of interest in such studies are measurement data (i.e. values from an ordered scale), the most commonly used average is mean

5b - Three kinds of differences between means

One sample differences:
- interested in the mean of one sample of scores
- the mean is compared against some 'special' value
- a known overall mean (e.g. population)
- a neutral value
- a performance 'benchmark' (e.g. number correct if guessing randomly)

Related-samples difference
(paired samples/matched samples/repeated measures/within-subjects)
- interested in two means for two samples of scores from the same source (set of cases/people)
- the means are compared with each other - this is equivalent to comparing pairs of scores
- participants take part in both conditions in an experiment (the same quantity is measured twice for each person)

Independent-samples differences (unpaired/unmatched/between-subjects)
- interested in two means for two samples of scores from different sources
- the means are compared with each other
- participants are randomly allocated to one of two conditions in an experiment

5b - The standardised effect size for a mean difference

Cohen's d - the standardised effect size for the mean difference

effect size for a population mean difference (two sample difference)

effect size = difference between two population means/standard deviation of either population

estimating effect size for a mean difference from sample data (two sample difference)

effect size = difference between two sample means/estimate of within-population SD

5b - Interpreting and calculating the standardised effect size (d)

For a two-sample difference, the effect size is a measure of the group difference between two groups of scores relative to the variability of those scores in each group. In other words, it reflects how much 'overlap' there is between two distributions

Rough guidelines to interpreting effect size:
- small = 0.2
- medium = 0.5
- large = 0.8

5b - Effect size

which standard deviation to use for effect size (when there is a choice):
- ideally, we would use the population standard deviation, but this is rarely known
- if there is a control group, or a 'baseline' group, can use the standard deviation for this group
- if there is no control group, we should combine ('pool') our two samples of data and use the average (pooled) variance to obtain a single standard deviation, and use that.

Pooled variance =

for standard deviation S1 and S2, we have variances S2/1 and S2/2.

average pooled variance = (S2/1 + S2/2) / 2

Standard deviation using pooled variance = square root of average pooled variance