Inferential Statistics

inferential statistics

allows for references to be made about populations based off sample data sets

• estimate population parameters

• compare b/t groups

step 1

state statistical hypotheses (null vs alternative)

hypothesis testing

use of mechanisms for deciding if an observed effect reflects chance only or if we can argue with confidence that differences represent real effects

null hypothesis

no difference exists between group means

alternative hypothesis

true difference exists between group means

goal of most research

reject null hypothesis

step 2

select level of significance = probability of rejecting the null hypothesis when it is true

0.05

0.05

5% risk of concluding that a difference exists when there is no actual difference

probability

likelihood one event will occur given possible outcomes; sum of all will equal 1

p=0 → no chance

p=1 → 100% chance

significant difference

difference likely not due to chance; null hypothesis is rejected

alpha level

level of significance; threshold selected to detect a significant difference

• set priori: decision rule, prior to data collection, threshold for significant or not

• typical level= 0.05 (95% probability of true difference)

p value

major resulting value from running statistical hypothesis testing; quantifies how consistent sample values are with null hypothesis; calculated from deviation b/t observed value and chosen reference value

larger values- p

sample consistent with null hypothesis; observed difference in groups is due to chance and not significant

smaller values- p

sample consistent with alternative hypothesis; observed difference in groups is significant

steps 3 & 4

3- decide which test to use

4- decide to reject or retain null hypothesis based on p value

sampling error

inevitable and unpredictable tendency for sample values to differ from population values

sampling distribution of means

all possible sample means within a population create a normal distribution curve

central limit theorem

sampling distribution of sample means approaches normal distribution and demonstrates a decreased SD as sample size gets larger, no matter the shape of original population distribution

standard error of mean

estimate of the population standard deviation

increase SEM= more variability within sample = less representative of population

increase sample size = decrease SEM

confidence intervals

provide an estimated range of values which is likely to include an unknown population parameter, estimated range being calculated from a given set of sample data

• 95 % CI (1- alpha)

z score

confidence level value; normal area under sampling distribution curve (use T score for smaller samples)

type I error

conclusion is made that difference exists, when in fact difference was due to chance (FALSE POSITIVE)

type II error

conclusion is made that no difference exists, when in fact there is one (FALSE NEGATIVE)

alpha level inflation

increased concern for type 1 error

more statistical tests run= greater likelihood of finding significant difference

solution: more stringent alpha level; Bonferroni correction (divide alpha level by # of tests run)

beta

probability of making type II error

likelihood that we will be unable to statistically ID real differences

power

probability that a test will reject null hypothesis; more powerful = less likely to make type II error

beta level

0.20 = 80% corresponding power

most reasonable to protect against type II errors, best to conduct power analysis prior to start of study, can use to determine appropriate sample size

significance criterion

lowering alpha level will decrease probability of committing type I error but will increase type II error probability

variance

• increased variability within a group will lead to less obvious difference

• decreased variance = increased power

sample size

increased = greater power

effect size

degree to which null hypothesis is false

larger= greater effective difference b/t groups

non directional test (2 tailed)

does not predict direction of difference b/t groups

directional test (1 tailed)

used when researchers predict the direction of change; more power to detect difference

violated assumptions

• procedure based on assumptions related to data and sample

• errors in inference if not met

problems with reliability and variance

• increased threats to statistical conclusions with increased variability in data may be due to: use of unreliable measurements, failure to standardize protocol, environmental interference, heterogenity of subjects

intention to treat

everyone who starts study is included in data analysis regardless of drop out