comparing means

inferential statistics

statistics concerned with testing hypotheses and using sample data to make generalizations concerning populations

statistical reasoning

- probablitiy

- sampling error

probability

the likelihood that an event will occur, given all possible events

sampling

the difference between values observed in the sample and in the population

confidence intervals

a range that should contain the population mean

- expressed as %

- eg 95 or 99% confidence

95% of CI means

95% of the time the confidence interval would contain the true population mean

statistical hypothesis testing

- null hypothesis

- alternative hypothesis

null hypothesis

- a statement of no difference or no relationship between variables (H0)

alternative hypothesis

hypothesis stating the expected relationship between independent and dependent variables (H1)

we can only legitimately say that

we reject or do not reject the null hypothesis

alternative hypothesis

- non-directional hypothesis

- directional hypothesis

non-directional hypothesis

does not specify which mean is expected to be higher (use a two-tailed test)

directional hypothesis

indicating an expected direction in the difference between means (use a one-tailed test)

errors in hypothesis testing

- type 1 error

- type 2 error

type I error

rejecting the null when it was true

type II error

failing to reject the null when it is incorrect

type 1 error and significance

- there is always some degree for risk when rejecting the H0

- level of significance provides a standard for rejecting

- alpha level (maximal acceptable risk)

- the probability that the observed difference occurred due to chance (reported as the p value)

- clinically, p= 0.05 is a typically accepted standard

when is p = 0.05 risk too high?

Why not always set it low (p=-.001) to avoid type 1 error?

- drug trials (the risk does not out weigh the benefit)

- usually what we are doing does not significantly harm someone so it is sufficient

type II error and power

- the probability of making type II error (beta)

- power = 1-B

- power is the probability that a test will lead to rejection of the null hypothesis

- B= 0.20 (80% power) is commonly considered reasonable to protect against type II error

determining statistical power

- power

- alpha level

- number of subjects

- effect size

power

desired levels can be determined in planning stages

alpha level

best level needs to be determined considering impact on type I and type II error

number of subjects

larger the sample, the greater the statistical power

effect size

the degree to which the null hypothesis is false

effect size should be considered a

relative measurement

cohen's D calculator

- 0.2 = small

- 0.5 = medium

- 0.8 = large

cohen's D equation

d = (M1 - M2)/SD

effect size is the magnitude of

the difference between groups

parametric is used to

estimate population parameters

parametric must meet certain assumptions to be valid

- random sampling

- interval or ratio (continuous data)

- normal distribution

- homogeneity of variance (equality of variances)

nonparametric

less powerful analogs

- a valid alternative when assumptions are not met

comparing 2 means

t test

paired sample t test is used when

subjects serve as their own control

- data is paired: there is a matched value for each subject

- compares differences in scores for each pair so the subject is only compared with themself

paired sample t test reported as t statistic:

- t (df) = t statistic

- p= p vlaue

- CI 95(lower, upper)

- provides degree of freedom, p value, and 95% CI

- effect size (cohen's d)

- mean and standard deviation

nonparametric alternative for paired sample t test =

wilcoxon test

independent t-test

- each group consists of a different set of subjects

- no relationship or matching between the factors

- assumptions include equality of variances (levees test)

- reporting t statistic: t(df) = t statistic, p = p value, CI95 (lower, upper)

nonparametric alternative for independent t-test

Mann whitney U test

ANOVA

- 3 + treatment groups or conditions

- between groups and within groups

- manipulation of 2 or more variables

- based on F statistic

- parametric

- effect size - eta squared (n^2)

non parametric test for ANOVA

kruskal Wallis test

n^2 =.01

small

n^2 = 0.06

medium

n^2 = 0.14

large

ANOVA types

- one way

- two way

- repeated measure

- mixed design

one-way

one independent variable with 3 or more levels (factors)

two-way

2 or more independent variables (factors)

repeated measure

- within subjects design

- use with same subject under multiple conditions (k)

mixed design

at least 1 independent and 1 repeating factor between and within