Statistical Methods for Computer Science

Experimental and Empirical Research

• Empirical research involves observation-based investigations to establish causal relationships by manipulating independent variables and measuring dependent variables.

• Pre-Experimental Designs:

  • One-Shot Experimental Case Study: A single group receives treatment followed by observation ($Tx \rightarrow \text{Obs}$).
  • One-Group Pretest-Posttest Design: A single group is observed before and after treatment ($\text{Obs} \rightarrow Tx \rightarrow \text{Obs}$).
  • Static Group Comparison: Two groups are compared where one receives treatment and the other does not ($\text{Group 1}: Tx \rightarrow \text{Obs}$, $\text{Group 2}: - \rightarrow \text{Obs}$).

• True-Experimental vs. Quasi-Experimental Designs:

Central Measures and Data Distributions

• Measures of Central Tendency:
  • Mean: Average value of a set of scores.
  • Median: Middlemost item in an ordered set; unaffected by extreme values.
  • Mode: Most frequently occurring value; unaffected by extreme values and can be unimodal, bimodal, or multimodal.

• Central Limit Theorem (CLT):

  • States that the distribution of sample means approaches a normal distribution with standard deviation $\sigma' = \frac{\sigma}{\sqrt{n}}$.
  • Probability density function equation:   $p(x) = \frac{1}{\sqrt{2\pi\sigma'^2}} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma'}\right)^2}$

• Kernel Density Estimation (KDE): A non-parametric technique for estimating the probability density function of a random variable.

• Normal vs. Student's t-Distribution:

  • Normal Distribution: Symmetric curve centered at $\mu = 0$ with standard deviation $\sigma = 1$ ($68\,\%$-$\!95\,\%$-$\!99.7\,\%$ rule).
  • t-Distribution: Uses degrees of freedom ($d.f. = n - 1$); has heavier tails than a standard normal distribution but approaches the standard normal distribution ($Z$-distribution) as $n \rightarrow \infty$.

Statistical Hypothesis Testing and T-Tests

• Assumptions for T-Tests: Continuous or ordinal data, random sampling, normal distribution, sufficient sample size, and equal variance across groups (for independent two-sample tests).

• Types of T-Tests:

  • One-Sample t-Test: Compares a group mean $m$ against a population or theoretical mean $\mu$:     $t = \frac{m - \mu}{\frac{s}{\sqrt{n}}}$
  • Independent Two-Sample t-Test: Compares means of two independent groups ($m_A$ and $m_B$) with $d.f. = n_A + n_B - 2$:     $t = \frac{m_A - m_B}{\sqrt{\frac{S^2}{n_A} + \frac{S^2}{n_B}}}$
  • Paired Sample t-Test: Compares mean differences in repeated measurements on the same subjects with $d.f. = n - 1$:     $t = \frac{m}{\frac{s}{\sqrt{n}}}$

Analysis of Variance (ANOVA)

• Purpose: Evaluates if statistically significant differences exist across three or more group means.
• ANOVA Variations:
  • One-Way ANOVA: Compares multiple independent groups across a single factor ($H_0: \mu_1 = \mu_2 = \dots = \mu_k$).
  • Two-Way ANOVA Without Replication: Double-tests a single group across two factors.
  • Two-Way ANOVA With Replication: Evaluates multiple groups undergoing multiple treatment conditions to assess both main effects and interaction effects.

Pearson's Chi-Square Test

• Purpose: Evaluates independence between categorical variables in a contingency table.

• Calculations:

  • Expected Frequencies ($c$):     $c = \frac{\text{row total} \times \text{column total}}{\text{grand total}}$
  • Chi-Square Statistic ($\chi^2$):     $\chi^2 = \sum \frac{(o - c)^2}{c}$
  • Degrees of Freedom ($d.f.$):     $d.f. = (\text{rows} - 1) \times (\text{columns} - 1)$

• Decision Rule: Reject null hypothesis $H_0$ if calculated $\chi^2 > \text{critical } \chi^2$ value or if $p \le \alpha$ ($\alpha = 0.05$).