Sampling Distributions and Point Estimation Comprehensive Study Notes

Fundamental Questions in Sampling Distributions

• What is a sampling distribution? It is a probability distribution of a statistic (such as the mean) obtained from all possible samples of a specific size ($N$) drawn from a population.
• What is the standard error? The standard error ($\text{SD}_{\bar{X}}$) is the standard deviation of the sampling distribution of a statistic, most commonly the sample mean.
• What is the principle of maximum likelihood? (Topic introduced for foundational understanding of parameter estimation).
• What is bias? In a statistical sense, bias refers to whether the expected value of a statistic equals the population parameter it is intended to estimate.
• What is a confidence interval? An interval used to estimate a population parameter with a specific level of certainty.
• What is the central limit theorem (CLT)? A theorem stating that as sample size increases, the sampling distribution of the mean approaches a normal distribution regardless of the population's original distribution.
• Why is the number 1.96 a big deal? This value is the critical Z-score for a $95\%$ confidence interval in a standard normal distribution (leaving $2.5\%$ in each tail).

Population vs. Sample

• Population: The entire collection of elements or individuals of interest. It is also referred to as the Sample Space.
• Sample: A subset of the population selected for analysis.
• Population Parameter: A numerical characteristic of a population (e.g., $\mu$ for mean, $\sigma$ for standard deviation).
• Sample Statistic: A numerical characteristic of a sample (e.g., $\bar{X}$ for mean, $s$ for standard deviation).
• Parameter Estimation: Statistics are used to estimate parameters. For example:     * Determining the effectiveness of pilot training.     * Determining the effectiveness of psychotherapy.     * Estimating the population standard deviation ($\sigma$) using the sample standard deviation ($SD$).\n

Defining the Sampling Distribution

• Core Definition: A sampling distribution is a distribution of a statistic over all possible samples.
• The Sample Mean Distribution: Because every sample contains different observed values, the sample means ($\bar{X}$) will differ. The sampling distribution is the probability distribution of these sample means for all possible samples of the same size ($N$) drawn from the population.
• Steps to Create a Sampling Distribution:     1. Select a sample of size $N$ (e.g., $N = 5$, $N = 10$, or $N = 1000$) from a population.     2. Compute the desired statistic (e.g., the mean $\bar{X}$) and record its value.     3. Repeat steps 1 and 2 a massive number of times (ideally an infinite number of times for large populations).     4. Plot the resulting statistics. The resulting plot is the sampling distribution.
• Purpose: It shows the relationship between the probability of a statistic and the statistic's value for all possible samples of size $N$.

Discrete Example 1: Supervisor and Six Employees

• Scenario: A supervisor has six employees with experience levels (in years) of: $2, 4, 6, 6, 7, 8$.
• Population Size ($N$): $6$.
• Population Mean ($\mu$): $\frac{2 + 4 + 6 + 6 + 7 + 8}{6} = 5.5$.
• Sampling Condition: We want to find the sampling distribution of the mean for samples of size $n = 2$, sampled without replacement.
• Combinations: There are $15$ possible iterations when sampling without replacement from a population of $6$.

Possible Samples and Their Means:

Probability Distribution Table for the Sample Mean:

Discrete Example 2: Rolling Two Dice

• Population: Six elements: $1, 2, 3, 4, 5, 6$.
• Population Mean ($\mu$): $\frac{1+2+3+4+5+6}{6} = 3.5$.
• Sampling Condition: Sample size $n = 2$ with replacement. Each element has a $\frac{1}{6}$ chance of being selected.
• Possible Outcomes ($36$ total):

• Observations: There is only 1 way to get a mean of 1 (1,1), but 6 ways to get a mean of 3.5.

Properties of the Sampling Distribution of the Mean

Expected Value

The mean of the sampling distribution ($\mu_{\bar{X}}$) is equal to the population mean ($\mu$):

$E(\bar{X}) = E\left(\frac{1}{n} \sum_{i=1}^{n} X_i\right) = \mu$

Variance and Standard Error

The variance of the sampling distribution is:

$\text{Var}(\bar{X}) = \text{var}\left(\frac{1}{n} \sum_{i=1}^{n} X_i\right) = \frac{\sigma^2}{n}$

The standard deviation of the sampling distribution is called the Standard Error ($\sigma_{\bar{X}}$):

$\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$

Finite Population Correction (FPC)

When observations are not selected independently (sampling without replacement from a finite population), the variance is adjusted:

$\text{var}(\bar{X}) = \frac{\sigma^2}{n} \times \left(\frac{N - n}{N - 1}\right)$

The Standard Normal Distribution for Sample Means

If the sampling distribution is normal, we can calculate the standardized Z-score for a sample mean:

$Z = \frac{\bar{X} - \mu}{\sigma_{\bar{X}}} = \frac{\bar{X} - \mu}{\frac{\sigma}{\sqrt{n}}}$

Case Study: Spark Plug Manufacturer

• Claim: Spark plug lives are normally distributed with $\mu = 60,000\text{ miles}$ and $\sigma = 4,000\text{ miles}$.
• Sample: $n = 16$, $\bar{X} = 58,500\text{ miles}$.
• Task: Find the probability of finding a sample mean of $58,500$ or less.
• Step 1: Calculate Standard Error:$\sigma_{\bar{X}} = \frac{4,000}{\sqrt{16}} = 1,000$
• Step 2: Calculate Z-score:$Z = \frac{58,500 - 60,000}{1,000} = -1.50$
• Step 3: Determine Probability:$P(\bar{X} < 58,500) = P(Z < -1.50) = 0.0668$

The Central Limit Theorem (CLT)

1. Normalization of Means: The sampling distribution of the means becomes normal as the sample size $N$ increases, regardless of the shape of the original population distribution.
2. Binomial Approximation: A binomial distribution becomes normal as $N$ increases.
3. Broad Application: This principle applies to other statistics as well, including variance.

Acceptance Intervals

• Definition: An interval within which a sample mean has a high probability of occurring, given a known population mean and variance.
• Logic: If a calculated sample mean falls within this interval, we accept the conclusion that the random sample likely came from the population with the known parameters.
• Calculation:$\mu \pm Z_{\alpha/2} \sigma_{\bar{X}}$

Sampling Distribution of Sample Proportions

• Context: Used in Bernoulli and Binomial distributions where there is a probability of success ($P$).
• Sample Proportion ($p$):$p = \frac{X}{n}$     Where $X$ is the number of successes in $n$ independent Bernoulli trials.
• Central Limit Theorem Application: For larger sample sizes, the distribution of $p$ can be modeled as a normally distributed random variable.

Mean and Variance of Proportions

• Expected Value: $E(p) = P$
• Variance: $\text{Var}(p) = \frac{P(1 - P)}{n}$
• Standard Error: $\sigma_p = \sqrt{\frac{P(1 - P)}{n}}$
• Z-score for Proportions:$Z = \frac{p - P}{\sigma_p}$

Case Study: Unsafe Wiring

• Scenario: A sample of $n = 270$ homes is taken. $20\% ($P = 0.2) of homes have unsafe wiring.\n* **Task:** What is the probability that the sample proportion (p$) will be between$16\%$and$24\%?\n* **Calculation:**\n    1. \sigma_p = \sqrt{\frac{0.2 \times 0.8}{270}} = 0.024\n    2. Check range: P(0.16 < p < 0.24)\n    3. Standardize: P\left(\frac{0.16 - 0.2}{0.024} < Z < \frac{0.24 - 0.2}{0.024}\right) = P(-1.67 < Z < 1.67)\n    4. Result: 0.9050\n\n# Sampling Distribution of Sample Variance\n\n* **Population Variance:** \sigma^2 = E(X - \mu)^2\n* **Sample Variance:** s^2 = \frac{\sum_{i=1}^{n} (X_i - \bar{X})^2}{n - 1}\n* **Unbiased Property:** The expected value of sample variance is the population variance (E(s^2) = \sigma^2).\n\n### Chi-Square (\chi^2) Distribution\nIf a random sample of n observations is drawn from a **normally distributed** population, the statistic relating sample and population variance follows a Chi-square distribution:\n\n\chi^2_{(n-1)} = \frac{(n - 1)s^2}{\sigma^2} = \frac{\sum_{i=1}^{n} (X_i - \bar{X})^2}{\sigma^2}\n\n* **Degrees of Freedom (v$):**$n - 1.\n* **Nature:** Defined only for positive values.\n* **Mean of \chi^2$:**$E(\chi^2_v) = v.\n* **Variance of \chi^2$:**$\text{var}(\chi^2_v) = 2v.\n* **Variance of s^2$:**$\text{var}(s^2) = \frac{2\sigma^4}{n - 1}.\n\n# Chi-Square Values and Applications\n\n### Reading Chi-Square Values\n* Example: For a distribution with df = 39:\n    * P(\chi^2_{39} < 25.695) = 0.05\n    * P(\chi^2_{39} > 54.572) = 0.05$(This defines a two-tailed test with a$5\% significance region in each tail).\n\n### Case Study: Green Valley Foods\n* **Subject:** Shirley Mendez, Quality Assurance Manager.\n* **Goal:** Ensure package weight variation is small.\n* **Task:** Find the upper limit for the ratio of sample variance (s^2$) to population variance ($\sigma^2$) for$n = 20$where the probability the ratio is above this limit is$0.025$($97.5\% will be below).\n* **Solution Steps:**\n    1. Identify n = 20$, so$df = 19.\n    2. Goal: Find k_u$such that$P\left(\frac{s^2}{\sigma^2} < k_u\right) = 0.975.\n    3. Convert to Chi-square: P\left(\frac{(n-1)s^2}{\sigma^2} > (n-1)k_u\right) = 0.025.\n    4. Lookup \chi^2_{19, 0.025}$on the distribution table:$32.852.\n    5. Set up equation: 19 \times k_u = 32.852\n    6. Solve: k_u = \frac{32.852}{19} = 1.729\n    7. Conclusion: There is a 97.5\%$probability that the ratio$\frac{s^2}{\sigma^2}$is below$1.729$$.