PHT5156 - Confidence Intervals and P Values

This session delves into two essential tools in statistical analysis: confidence intervals and p-values.
The session aims to examine how data from samples can be utilized to estimate population parameters and interpret the significance of p-values in hypothesis testing.
Confidence intervals provide a plausible range for population values based on sample data.

In traditional statistical inference, it is essential to understand the relationship between sample data and the population:
- It is unknown how closely data from a sample corresponds to the actual population.
- However, it is possible to analyze how much a statistic can deviate from a theoretical population parameter.

State the null hypothesis (H0): Presumes no effect or difference.
State the alternative hypothesis (H1): Indicates the existence of an effect or difference.
Choose a statistical test: Select the appropriate test based on data characteristics and hypotheses.
Set the alpha level (α): Represents the likelihood of committing a Type I error (false positive); commonly set at 0.05.
Analyze the data: Calculate relevant statistics.
Decision-making: Compare the computed p-value against the alpha level to accept or reject the null hypothesis.

A p-value is the probability of observing results as extreme as those in your sample, assuming the null hypothesis is true:
- If p < ext{α} (e.g., 0.05), reject the null hypothesis (suggests a true difference exists between groups).
- If p ext{ > α}, accept the null hypothesis (indicates no significant difference).

P-values do not:
- Indicate the size or magnitude of differences.
- Distinguish between statistical and clinical significance.
- Reflect the actual effect size observed.
The conventional threshold of 0.05 is considered arbitrary, oversimplifying the complexities of reality.
Misinterpretation or overemphasis on p-values may lead to misleading conclusions regarding research findings.

P-values represent probabilities:
- A p-value signifies the chance that the researcher incorrectly rejected the null hypothesis.
The interpretation of p-values tends toward a binary view:
- $p = 0.04$ indicates a difference exists, while $p = 0.06$ implies no observed difference.
Avoid overinterpreting results based on non-significant p-values as it does not equate to proof of no differences across all samples.
Context and additional supportive data should be considered for informed conclusions.

Definition: Confidence intervals (CIs) provide a range of values likely to contain the population parameter, reflecting precision in estimation.
- A 95% confidence interval signifies that there is a 95% probability that the interval includes the true population mean.
- Example: Knee pain data post-surgery showed no significant difference with a CI ranging from 0.19 to 1.35, indicating 95% certainty that the true population mean falls within this range.

Variance: Higher variance leads to wider confidence intervals.
Sample Size (N): Larger samples yield narrower intervals due to increased reliability of estimates.
Confidence Level: Higher confidence levels result in wider intervals to enhance certainty.

For a 95% confidence interval:
- If the interval does not include 0, the null hypothesis is rejected, indicating a significant difference.
- For example, if the mean difference CI ranges from -5 to -1, it indicates a significant difference.

In a study comparing patients with neck pain receiving different treatment methods:
- At 4 weeks, patients receiving manual therapy and exercise had lower disability scores (3.5 points lower) with a CI ranging from -5.6 to -1.2, confirming significance.
- At 6 months, the CI was from -4 to 0.3, crossing 0, thus indicating non-significance, although close to significant.
- Another instance at 6 months presented a CI from -3.1 to 1.7 capturing 0, confirming no significant difference.

Confidence intervals allow for nuanced understanding beyond the binary outcomes in traditional methods.
They quantitatively express the precision of estimations which is essential for evaluating clinical significance.
- In negative trials, the upper limit of CI represents the maximum plausible effect. If this limit is below the Minimum Clinically Important Difference (MCID), the trial is clearly negative.
- For example, if the observed difference is below MCID, and the CI captures 0, the result is definitely negative.

Positive Trials: For positive outcomes, the lower limit of CI reflects the smallest effect assumed clinically significant. If greater than the smallest important difference, it confirms a positive trial.
- Example: Observed difference is above the MCID with a CI above 0 indicates a definitive positive trial.
This must be carefully evaluated as an interval that captures the MCID might indicate ambiguity, possibly hinting at a Type II error (false negative).

Both p-values and confidence intervals hold substantial value in statistical analysis.
P-values provide evidence against the null hypothesis, demonstrating the likelihood of observable data under H0.
Confidence intervals contextualize p-values by estimating the precision and plausibility of observed effects. Together, they enhance the robustness of conclusions drawn from data analysis.