Inferential statistics & hypothesis testing FINAL

• null hypothesis

• null hypothesis

two possible decisions in hypothesis testing

• Reject the ____

• Fail to reject the ____

• Type I Error (Alpha Error)

• Type II Error (Beta Error)

What are the two types of error in hypothesis testing?

type I (alpha) error

error type

Rejecting the null hypothesis when it is actually true.

We mistakenly conclude that a difference exists (treatment works) when it is actually due to chance

What does a Type I error imply in terms of treatment effectiveness?

"Mistakenly finding a difference."

What phrase summarizes Type I error?

We decide to use a treatment that is not actually effective.

What is a consequence of a Type I error in clinical research?

We accept ≤ 5% chance of making a Type I error (finding a difference due to chance).

What does an alpha/p-value ≤ 0.05 represent?

“How confident are we that population A (control) is different from population B (treated) based on walking improvement?”

What question represents a Type I error example?

type II (beta) error

error type

Failing to reject the null hypothesis when it is actually false

We mistakenly conclude that there is no difference when a real difference exists.

What does a Type Il error imply in terms of treatment effectiveness?

"Mistakenly finding no difference."

What phrase summarizes Type I| error?

We might ignore or discard an effective treatment.

What is a consequence of a Type Il error in clinical research?

“How confident are you when saying there is no difference between treated and untreated groups when one really exists?”

What question represents a Type II error example?

0.05

What is the commonly used significance level (alpha)?

• Reject

• alternative hypothesis

• significant

• group means

What do we conclude if p ≤ 0.05?

• ___ the null hypothesis

• Accept the ___

• Result is statistically ___

• There is a difference between ____

• null hypothesis

• statistically significant

• chance

What do we conclude if p > 0.05?

• Fail to reject the ___

• Result is not ____

• Observed difference may be due to ___

level of significance (α)

defined as being the maximum acceptable risk of making a Type I error when rejecting the null hypothesis (H₀)

We accept a 5% chance of rejecting the null hypothesis when it is actually true (Type I error).

What does α = 0.05 mean?

α = 0.05 (5%)

What is the typical level of significance used in research?

When p ≤ 0.05

When do we reject the null hypothesis based on p-values?

When p > 0.05 (the difference is not statistically significant)

When do we fail to reject the null hypothesis based on p-values?

Only a 1 out of 100 chance of making a Type I error.

What does level of significance of a = 0.01 mean?

Rigor in avoiding Type I errors increases, but the risk of Type ll errors increases.

What happens when level of significance is extremely small (e.g., 0.0001)?

It provides a reasonable balance between Type I and Type || errors

Why is level of significance = 0.05 often used?

There is a 20% chance of making a Type II error

What does β = 0.20 mean?

Statistical power (1 − β), meaning an 80% chance of correctly rejecting a false null hypothesis

What does an 80% chance of not making a Type II error represent?

type II error

error type

Failing to reject the null hypothesis when it is actually false

confidence interval (CI)

defined as a range of values (with upper and lower limits) that is expected to contain the true population mean

The sample mean and its standard error (SEM).

What does a Cl depend on?

As a percentage

How is a confidence interval expressed?

If 100 samples are taken, about 95 of them would contain the true population mean.

What does a 95% confidence interval mean?

That sample is part of the 5% expected not to contain the true population mean.

What does it mean when a sample's 95% Cl does not include the population mean?

99% of samples would contain the true population mean, and only 1% would not

What does a 99% confidence interval indicate?

Higher confidence levels (e.g., 99%) provide more accuracy but result in wider intervals

How does increasing the confidence level affect accuracy?

effect size

defined as a measure of the degree to which the null hypothesis is false; indicates how large an effect is in an intervention or how strong a relationship is between two variables

How large the effect of the intervention is

In an intervention study, what does effect size indicate?

How strong the relationship is between two variables

In a correlation study, what does effect size indicate?

A larger difference results in a larger effect size

How does the size of the difference in a study relate to the effect size?

No, effect size is not always reported

Is effect size always reported in studies?

effect size index

defined as a standardized value used to report effect size

cohen’s D

defined as a common type of effect size index

small

0-0.2 cohens D size

medium

0.3-0.5 cohens D size

large

0.6+ cohens D size

odds ratio

For categorical data, what is effect size based on?

It helps readers understand the results section and analyze whether findings are bias-free

Why is inferential statistics important when appraising a study?

• sample size

• reliable estimates

• methods section

key questions to ask about sample size

• what is the ____

• is it large enough to provide ____

• where is it reported —> _____

standard error of the mean (SEM)

estimates how far the sample mean is likely to be from the true population mean