Flashcards for Engineering Statistics Lecture Notes

Chapter 6: Point Estimation

6.1 Motivation: Point and Interval Estimations

• Major Problem of Statistics: Estimation of a parameter from an experiment involving a random variable $X$.
• We have $N$ measurements resulting in a data sample $x1, x2, …, x_N$.
• We aim to estimate a parameter $ heta$ (e.g., mean of $X$) characterizing the population, which we can only approximate with a function of the data:
  \hat{\theta} = f(x1, x2, \ldots, x_N) \approx \theta.
• The process of generating this function and computing values based on data is called estimation.

6.1. Point and Interval Estimations

• Estimation can be performed as:
  • Point Estimation: Achieving a single value $\hat{\theta}$ as the best approximation of $ heta$.
  • Interval Estimation: Finding an interval $[\hat{\theta}{P1}, \hat{\theta}{P2}]$ where the true value $ heta$ is expected to lie with probability $P$.
• Both methods yield approximate results, and characterizing their accuracy is essential.{\hat{\theta}} \

6.2 Point Estimators and Estimates

• Concepts:
  • Estimand: The parameter we want to estimate ($\theta$).
  • Point Estimator: A statistical rule $(\hat{\theta} = g(x1, x2, \ldots, x_N))$ that provides an estimate from the sample.
  • Point Estimate: The computed value from the estimator using a specific data sample.
• Error and Bias: The error of the point estimator is:
  e{\hat{x}} = \hat{\theta}{\hat{x}} - \theta;
  the bias is given by:
  B(\hat{\theta}) = E[\hat{\theta}] - \theta.
• An estimator is unbiased if $B(\hat{\theta}) = 0$.
• Minimum Variance: Among the unbiased estimators, the Minimum Variance Unbiased Estimator (MVUE) has the minimum variance:
  V(\hat{\theta}). Estimators with lower variance provide better estimates across samples.

6.2. Example of Estimation

• Considering a random variable $X$ (e.g., device lifetime), with data sample lifetimes $x1 = 1.7$, $x2 = 2.3$, and $x_3 = 0.8$ years:
  • The average estimate is:
    \hat{\theta} = \frac{x1 + x2 + x_3}{3} = \frac{1.7 + 2.3 + 0.8}{3} = 1.6 \text{ years}.

6.3 Estimators for the Mean and Variance

• Estimators for Mean:
  • Proposition: For a random sample of $X$ with same mean $\mu$, the sample mean is the unbiased estimator:
    \hat{\theta} = \bar{X} = \frac{1}{N} \sum{i=1}^{N} Xi.
• Estimators for Variance:
  • Proposition: The unbiased estimator for population variance is:
    \hat{\sigma}^2 = \frac{1}{N - 1} \sum{i=1}^{N} (Xi - \bar{X})^2.

6.4 Methods of Point Estimation

1. Formulation of the Problem:
   • For an experiment with PDF $f(x; \theta1, \theta2, …)$, objective is to estimate the parameters.
2. Method of Moments: Equates sample moments to theoretical moments.
3. Maximum Likelihood Estimation: Based on maximizing the likelihood function given the data sample.

Notes

• Various estimators can be generated from the same data set, and their efficiency can vary. Choose the unbiased estimator with the smallest variance whenever possible.
• The ambiguity of the best estimator necessitates careful selection based on distribution and sample data.