Hypothesis Testing, Model Assumptions, Errors, and Confidence Intervals

Hypothesis Testing and Point Estimation

• Point Estimate: A sample statistic used to estimate an unknown population parameter. For example, the sample mean ($\bar{y}$) approximates the population mean ($\mu$). In the tipping dataset, the point estimate for rainy day tips is $18.464$.

• Hypotheses Setup:

  • Null Hypothesis ($H_0$): A statement of equality setting the baseline comparison point (e.g., $H_0: Α = 18$).

  • Alternative Hypothesis ($H_a$): A statement of inequality or difference (e.g., $H_a: Α \neq 18$, $H_a: \mu > 20$, or $H_a: \mu < 2$).

• Test Statistic ($t$-value): Represents the number of standard errors the sample statistic ($\bar{y}$) lies away from the hypothesized parameter under $H_0$.

• Decision Rule: If the $p$-value is less than the significance level ($\alpha$ = 0.05$), reject$H_0.\n\n# Model Assumptions and Sample Size Rules\n\n* **Normality Assumption:** To apply the t-distribution appropriately, the underlying population should be approximately normal.\n * **Skewness Criterion:** Sample skewness should fall between -0.5$and$0.5.\n* **Central Limit Theorem:** If sample data is skewed or bimodal (such as the building energy efficiency dataset with 768$observations), testing under the$t$-distribution remains valid provided the sample size is sufficiently large ($N \ge 30).\n* **Sample Size Impact:** Increasing sample size (N$) increases the magnitude of the$t$-statistic and reduces the$p-value, making it easier to detect statistical differences.\n\n# Testing Errors and Types of Significance\n\n* **Type I Error (False Positive):** Rejecting H_0$when$H_0$is actually true. The probability of committing a Type I error is bounded by the chosen significance level ($\alpha).\n* **Type II Error (False Negative):** Failing to reject H_0$when$H_0 is actually false.\n* **Significance Thresholds (\alpha$):** Standard levels include$0.05$(typical balance),$0.01$(used when Type I error carries severe costs), and$0.1.\n* **Statistical vs. Practical Significance:**\n * **Statistical Significance:** Indicated strictly by p\text{-value} < \alpha$.

  • Practical Significance: Evaluates whether the real-world magnitude of an effect is meaningful (e.g., a weight loss program yielding a statistically significant average loss of $0.25\, \text{pounds}$ over 6 months lacks practical significance).

Confidence Intervals

• Definition: A confidence interval provides an interval estimate containing a range of plausible values for a population parameter ($\mu$).

• Structure:     $\text{Point Estimate} \pm \text{Margin of Error}$     $\bar{y} \pm t^* \times SE(\bar{y})$

• Properties:

  • The center of a confidence interval is always the sample statistic ($\bar{y}$).

  • Interpretation: A $95\%$ confidence interval means that $95\%$ of all random samples taken from the population using the same procedure will produce an interval containing the true population parameter ($\mu$).