Hypothesis Testing and P-Value Evaluation
P-Values and Decision Rules
- P-value definition: The probability, assuming the null hypothesis is true, of obtaining a sample statistic with a value as extreme or more extreme than the observed data, as introduced by Hari.
- Significance level (α): The predetermined cutoff; for example, setting α=0.05 yields a 5% chance of rejecting the null hypothesis when it is true.
- Decision criteria:
- Reject the null hypothesis if p≤α.
- Fail to reject the null hypothesis if p>α.
Test Directions and P-Value Calculations
- Left-tailed test:
- Null hypothesis: Mean is greater than or equal to a specified value (mean≥k).
- Rejection: Occurs when the observed sample statistic is considerably lower than k.
- P-value: Area under the sampling distribution curve to the left of the observed standardized test statistic.
- Right-tailed test:
- Null hypothesis: Mean or population parameter is less than or equal to a value (mean≤k).
- Rejection: Occurs when the standardized test statistic is a high positive value.
- P-value: Area under the sampling distribution curve to the right of the observed test statistic.
- Two-tailed test:
- Null hypothesis: Parameter equals a specific single point (mean=k).
- Rejection: Occurs if the statistic is significantly lower or significantly higher than k.
- P-value: Twice the area under the curve to the left of a negative test statistic or to the right of a positive test statistic.
Sampling Distributions
- Population proportion (z-test): Evaluated using the standard normal distribution.
- Population mean (t-test): Evaluated using the t-distribution because the population variance is unknown, using degrees of freedom equal to n−1.
P-Values versus Critical Values
- Functional equivalence: Evaluating hypotheses via critical values and via p-values yields identical conclusions.
- Advantage of p-values: Provides a continuous measure of evidence, enabling null hypothesis rejection at more stringent levels of significance (e.g., 0.001) without recalculating cutoffs.