inferential statistics

what is inference?

drawing conclusions about a population based on sample results

used to draw conclusions for larger pop

from one or more samples

limitations of inference

sample is just a part of the whole population,

limited information

parametric data

Data that follow a normal distribution

central measure: mean

e.g independent t-test, paired t-test, and ANOVA

preferred if the assumptions can be justified

non parametric data

Data that doent follow a normal distribution

central measure: median

e.g chi-square test, Mann-Whitney U test, Wilcoxon signed rank tes

Criteria for Choosing a Statistical Test

- Hypothesis Type: comparing groups or finding associations between variables?

- Number of Groups/Variables: How many groups or variables are being analyzed?

- Data Type: Is the data nominal, ordinal, or continuous?

- Data Distribution: Can you use parametric tests? They require the outcome variable to be normally distributed

hypothesis testing

method for making decisions based on sample data

Use sample data to choose between two hypotheses about a population

Steps in Undertaking a Hypothesis Test

1) define study question

2) Set Hypotheses:

Null Hypothesis (H0): Assumes no difference or effect exists in the population.

Alternative Hypothesis (HA): Assumes a difference or effect does exist

3) choose statistical test

4) Compute a value that summarizes the data relative to the hypotheses

5) p-value (probability of null hypothesis is true)

6) Make Decision:

- Compare p-value to pre-defined significance level (e.g., 0.05)

- If p-value < significance level, reject H0 (suggests a difference exists)

- If p-value ≥ significance level, do not reject H0 (suggests no evidence of a difference)

7) Report whether a difference was found based on the results

types of errors

Type I Error (False Positive):

when you reject null hypothesis (H0) when it is actually true

This means concluding that a difference exists when it does not

Type II Error (False Negative):

when you do not reject the null hypothesis (H0) when alternative hypothesis (HA) is true

This means concluding that no difference exists when there actually is one

normal distribution

In a normal distribution, three measures of central tendency (mean, median, mode) are all located at the center of the distribution. This indicates that the data is symmetrically distributed around the mean

data is symmetrically distributed around the mean

SD

determines how spread points are around the mean

68% Rule: center 2 squares

97.5% Rule: center 4 squares

2.5% Rule: end bits

see slide 20

statisticial tests which assess differences

Independent t test

Paired t test

Analysis of variance (ANOVA)

Independent t test

compare the means (averages) of two independent groups

tests whether there is a significant difference between the average values

assumptions

- data from one group does not influence the data from the other.

- normally distributed (data points are symmetrically distributed around the mean)

- variability (spread) of two groups should be similar. Specifically, standard deviation of one group should not be more than twice that of the other group

Paired t test

compare the means of two related groups

whether there is a significant difference between the average values of two related groups

assumptions:

- two groups must be related or matched in some way. This means the data points are connected (e.g., measurements from the same individual)

- differences between the paired measurements should follow a normal distribution. This means that when you subtract one score from the other, the resulting differences should be symmetrically distributed around their mean

Analysis of variance (ANOVA)

compare the means of three or more independent groups

whether there are significant differences between the average values of multiple groups. It helps determine if at least one group mean is different from the others

an handle multiple groups simultaneously. It assesses overall differences among the groups rather than pairwise comparisons

produces f statistic

assumptions:

- samples from each group must be independent of one another

- normal distribution

- homogenicity of variance: variances (spread) of the groups being compared should be approximately equal

t test vs anonva

Number of Groups:

> T-Test: Compares the means of two groups.

>ANOVA: Compares the means of three or more groups.

Types of Tests:

> T-Test: Independent T-Test: Compares means of two unrelated groups. Paired T-Test: Compares means of two related groups (e.g., the same subjects tested twice).

> ANOVA: One-Way ANOVA: Compares means of three or more independent groups. Two-Way ANOVA: Compares means based on two factors (e.g., comparing different groups across genders).

Test Statistic:

> T-Test: Uses the t-statistic to determine if there's a significant difference between the two means.

> ANOVA: Uses the F-statistic to determine if there are significant differences among the means of multiple groups.

When to Use:

> T-Test: When comparing two groups. Example: Comparing average test scores of two classes.

> ANOVA: When comparing three or more groups. Example: Comparing average test scores of three different classes.

Post-hoc Testing:

> T-Test: No post-hoc testing is needed because it only compares two groups.

> ANOVA: Post-hoc tests (like Tukey's test) are often needed to identify which specific groups differ after finding a significant result.

chi squared

assesses whether there is an association between two categorical variables. For example, it can be used to determine if gender is related to satisfaction with a smoking cessation service

H0/ H1 hypothesis

The test compares the observed frequencies (actual data) with the expected frequencies (what we would expect if there were no association).

A significant result indicates that the two variables are related

In a clinical study, 60 patients were prescribed the drug 'Reductox' to assist in weight loss and 60 different patients were prescribed a placebo. The weight loss of each patient was recorded over an 8-week period. Which of the following tests should be used to decide whether 'Reductox' is effective in assisting weight loss?

Paired t-test​

Chi-squared test​

Independent t-test​

ANOVA

independent t test

- A. Paired T-Test:

Used when comparing two related groups (e.g., measurements taken from the same individuals before and after treatment). Not appropriate here.

- B. Chi-Squared Test:

Used for categorical data to assess the association between two categorical variables. Not appropriate here since weight loss is continuous data.

- C. Independent T-Test:

Used to compare the means of two independent groups (like in this case). This is the correct choice.

- D. ANOVA:

Used to compare the means of three or more groups, or to evaluate multiple factors. Not necessary here since we only have two groups

statistical tests which assess confidence

p value

confidence interval

p value

measure the strength of the evidence against the null hypothesis

lower P-Value indicates observed results are unlikely to have occurred by chance.

This suggests stronger evidence AGAINST null hypothesis

For example, P-Value greater than 0.01 means there is bigger than 1% probability that the results happened by random chance, so accept null, no correlation

Significance Level:

A common threshold for determining significance is P < 0.05. If the P-Value is less than 0.05, we consider the results statistically significant, meaning there is strong evidence to suggest an effect or difference.

This is often associated with a 95% confidence interval, indicating that we can be 95% confident the true effect lies within a certain range

Other Levels of Significance:

P < 0.01: Strong evidence against the null hypothesis (1% chance of being due to chance).

P < 0.0001: Very strong evidence against the null hypothesis (0.01% chance of being due to chance)

p < 0.05- sufficient evidence against H0

p <0.01= strong evidence against H0

p< 0.001- very strong evidence against H0

Which of these p-values are statistically significant at α = 0.05? In addition, state whether you would reject or fail to reject the null hypothesis

1. P=0.23

2. P=0.05

3. P=0.02

4. P=0.10

5. P=0.001

3, 5-> reject null hypothesis

1,2,5-> fail to reject null hypothesis

confidence intervals

P-Values indicate whether results are statistically significant (e.g., P < 0.05), but they do not provide information about the size or importance of the effect or difference observed

gives a range of values, offering insight into the magnitude and precision of the estimated effect

- range of values within which we can be fairly certain (confident) that the true value (like the true mean difference or effect size) lies

- e.g if a 95% confidence interval for a treatment effect is (2, 5), we can be 95% confident that the true effect is between 2 and 5 units

relationship between confidence intervals and significance levels =

CI = (1 - α) × 100%

α represents sig level, the probability of rejecting the null hypothesis when it is actually true (a type I error)

e.g

A 95% Confidence Interval (CI) corresponds to a significance level of α = 0.05. This means we are willing to accept a 5% chance of being wrong when we say the results are significant (p < 0.05), so 95% chance of being significant

A 99% Confidence Interval (CI) corresponds to a significance level of α = 0.01. This indicates we accept only a 1% chance of being wrong (p < 0.01)

clinical significance

whether the observed difference or effect is meaningful in a practical or medical sense.

whether the difference is large enough to have real-world implications for patient care

statistical and clinical significance is not always aligned e.g small effect might be statistically significant in a large study but may not be important in a clinical context

If a study shows a statistically significant difference in weight loss between two diets, but the actual weight loss is only a few grams, that difference may not be clinically significant. It may not impact a patient's health or treatment decisions

A statistically significant result must first be significant before it can be evaluated for clinical relevance

Methods of Establishing Clinical Importance

- Research Evidence: Refer to existing studies + data to determine what changes are necessary for meaningful clinical improvements

- Professional Judgment: Rely on healthcare professionals' expertise to determine needed improvements. Considering clinical guidelines and best practices

- Patient Involvement: Include patients in discussions to understand what changes would positively impact their quality of life

statistics which compare risk

relative risk

odds ratio

risk

probability that a specific event (often a negative outcome) will occur

e.g if 5 out of 100 patients experience a side effect-> 5/100= 0.05

relative risk

compares the risk of an event in two groups

risk= risk in treatment group/ risk in control group

RR > 1: Indicates treatment increases the risk of the negative outcome. For example, if RR is 1.5, the treatment group has a 50% higher risk of the bad outcome compared to the control group

RR < 1: Indicates treatment decreases the risk of the negative outcome. For instance, if RR is 0.75, the treatment group has a 25% lower risk of the bad outcome compared to the control group.

odds

likelihood of an event happening compared to it not happening

odds ratio

compares the odds of an event occurring in two different groups

odds ratio= odds of event in exposed group/ odds of event in non-exposed group

OR = 1: Indicates no difference in odds between the two groups.

OR > 1: Suggests that the event is more likely to occur in the exposed group.

OR < 1: Indicates that the event is less likely to occur in the exposed group

exposed statin group

myopathy= 45

non myopathy= 55

non exposed group

myopathy= 28

non- myopathy= 72

what is odds ratio?

(45/55)/(28/72) = 2.1