Populations, Samples, Descriptive Statistics, and Probability

Core Concepts of Statistics: Populations, Samples, Processes, and Data

• Population: A complete set of items or individuals that are of interest for a statistical study.
  • Examples: All manufactured pills of a specific type, or all individuals who have earned an engineering degree.
• Census: The collection of information from every single member of a population.
  • Conducting a census is often impractical or impossible due to severe constraints regarding time, financial cost, or logistical execution.
• Sample: A defined subset of a population selected according to a specified sampling method.
  • Example: A selection of ball bearings drawn from a production batch to evaluate whether they meet manufacturing specifications.
• Variable: Any characteristic that can differ among individual entities or items within a population.
  • Categorical Variable: A variable that takes on values corresponding to qualitative categories.
  • Examples: Gender, or the specific type of manufacturing defect.
  • Numerical Variable: A variable that takes on quantitative numerical values.
  • Examples: Age, or the diameter of a mechanical component.
• Data: The specific results obtained from observing one or more variables across items or individuals.
  • Univariate Data: Observations recorded on a single variable (e.g., recording the transmission type of various cars).
  • Bivariate Data: Observations recorded simultaneously on two variables for each unit (e.g., measuring both the height and weight of a basketball player).
  • Multivariate Data: Observations recorded on more than two distinct variables (e.g., measuring systolic blood pressure, diastolic blood pressure, and cholesterol levels of patients).
• Process: Any system, operation, or sequence of actions that generates observable data. Statistical methods are routinely applied to model, understand, and improve operational processes.
• Branches of Statistics:
  • Descriptive Statistics: Methods focused on organizing, summarizing, and presenting the key characteristics of a dataset in an informative manner.
  • Tools: Graphical displays (such as histograms, box plots, and scatter plots) and numerical summary measures (such as sample means, standard deviations, and correlation coefficients).
  • Inferential Statistics: Techniques utilized to analyze sample data in order to draw broader conclusions, estimations, or decisions regarding an entire underlying population.
  • Key Procedures: Point estimation, hypothesis testing, and confidence interval estimation.

Reasoning and Study Types in Statistics

• Comparison of Probability and Inferential Statistics:
  • Probability: Employs deductive reasoning to move from the population to a sample. It assumes known properties of a given population to predict or determine properties and outcomes of random samples.
  • Inferential Statistics: Employs inductive reasoning to move from a sample to the population. It analyzes observed sample characteristics to infer or conclude unknown properties of the general population.
• Types of Statistical Studies:
  • Enumerative Studies: Studies that focus on a finite, clearly identifiable collection of individuals or objects.
  • A complete sampling frame (a formal list of all population units) is typically available.
  • Example: Verifying the validity of physical signatures on a political petition.
  • Analytical Studies: Studies that are generally non-enumerative, aiming to analyze dynamic systems to inform actions or improve future products and processes.
  • A finite sampling frame is usually unavailable because the potential population includes future output.
  • Example: Testing prototype designs in a laboratory to refine future manufacturing techniques.
• Data Collection Methods:
  • Simple Random Sampling: A sampling technique in which every possible subset of a given size has an equal probability of being chosen from the population.
  • Stratified Sampling: A technique where the population is partitioned into distinct, non-overlapping subgroups called strata, after which a sample is independently selected from within each stratum.
  • Convenience Sampling: A non-systematic collection method where items or individuals are selected based on ease of access rather than randomized protocols. This approach frequently fails to yield a representative sample.
  • Planned Experiments: Methodical investigations where specific treatments are actively assigned to experimental units, or factor levels are deliberately manipulated to observe their precise effect on a response variable. Random assignment is critical to eliminate confounding factors and prevent systematic bias.

Visual and Graphical Methods for Descriptive Data

• Notation and Data Representation:
  • Sample Size ($n$): Represents the total number of observations contained within a given sample.
  • Individual Observations: Represented as a sequence $x_1, x_2, \dots, x_n$.
• Stem-and-Leaf Displays:
  • A visual display that simultaneously depicts the overall shape of the data distribution while retaining every individual quantitative data value.
  • Highly effective for identifying central tendency, dispersion (spread), presence of gaps, degree of symmetry, and potential outliers in small-to-moderate datasets.
• Histograms:
  • A graphical display showing the frequency distribution of a continuous or discrete numerical variable.
  • Discrete Data: Constructed by placing a vertical rectangle above each distinct possible value, where the height of the bar is proportional to the observed frequency or relative frequency.
  • Continuous Data: Constructed by dividing the continuous measurement axis into class intervals (bins). Rectangle heights correspond to class frequency, relative frequency, or density (used when class widths are unequal).
  • Distribution Shapes: Can be categorized as symmetric, unimodal, bimodal, multimodal, positively skewed (extended tail pointing to the right), or negatively skewed (extended tail pointing to the left).
• Dotplots:
  • A display where each numerical observation is marked as a dot situated above a continuous measurement scale.
  • Ideal for displaying visual patterns in small datasets.
• Qualitative Data Representation:
  • Categorical observations can be organized and presented using frequency tables and bar charts (which share visual similarities with histograms, though bars are separated by gaps).

Numerical Measures of Location and Dispersion

• Measures of Location (Central Tendency):
  • Sample Mean ($\bar{x}$): The arithmetic average of all sample observations.
  • Formula:       $\bar{x} = \frac{\sum_{i=1}^n x_i}{n}$
  • Sensitivity: Highly sensitive to extreme values or severe outliers.
  • Population Mean ($\mu$): The arithmetic average of all individual values present across an entire population.
  • Sample Median ($\tilde{x}$): The physical middle value when sample observations are arranged in ascending numerical order.
  • If the sample size $n$ is odd: The sample median is the exact single middle observation.
  • If the sample size $n$ is even: The sample median is defined as the arithmetic average of the two central observations.
  • Resistance: Highly resistant to the influence of extreme values or outliers.
  • Population Median ($\mu_{\text{median}}$): The exact median value dividing a population distribution into two equal halves.
  • Quartiles: Values that split an ordered dataset into four equal segments.
  • Lower Quartile ($Q_1$): The median of the lower half of the ordered data set.
  • Upper Quartile ($Q_3$): The median of the upper half of the ordered data set.
  • Second Quartile ($Q_2$): Identical to the median ($\tilde{x}$).
  • Percentiles: Values dividing an ordered dataset into $100$ equal parts.
  • Trimmed Mean: A modified mean computed after discarding a predetermined percentage of the smallest and largest numerical observations. Provides an effective middle ground between the extreme sensitivity of the standard sample mean and the high resistance of the median.
• Measures of Variability (Spread):
  • Range: The arithmetic difference between the absolute maximum and minimum observed values:     $\text{Range} = x_{\max} - x_{\min}$
  • Extremely sensitive to extreme values or sample outliers.
  • Sample Variance ($s^2$): The effective average of squared deviations of each sample point from the calculated sample mean:     $s^2 = \frac{\sum_{i=1}^n (x_i - \bar{x})^2}{n - 1}$
  • Measures dispersion expressed in squared measurement units.
  • Population Variance ($\sigma^2$): The mean of squared deviations taken from the overall population mean:     $\sigma^2 = \frac{\sum_{i=1}^N (x_i - \mu)^2}{N}$
  • Sample Standard Deviation ($s$): The positive square root of the sample variance:     $s = \sqrt{s^2}$
  • Measures spread in the original measurement units of the collected data, representing a typical distance of observations from the sample mean.
  • Population Standard Deviation ($\sigma$): The positive square root of the population variance:     $\sigma = \sqrt{\sigma^2}$
  • Fourth Spread ($FS$) / Interquartile Range ($IQR$): The numerical distance separating the upper and lower quartiles:     $FS = Q_3 - Q_1$
  • Resistant to extreme values and outliers.
  • Boxplot: A standardized visual chart representing the five-number summary (Minimum, $Q_1$, Median, $Q_3$, Maximum). Modified boxplots explicitly mark individual outliers beyond calculated thresholds.
  • Outliers: Observations located far outside the main cluster of data.
  • Regular Outliers: Points positioned more than $1.5 \times FS$ below $Q_1$ or above $Q_3$
  • Extreme Outliers: Points positioned more than $3 \times FS$ away from $Q_1$ or $Q_3$
• Proportions, Interpretation Rules, and Transformations:
  • Proportion: The ratio or fraction of a total dataset belonging to a specified outcome category.
  • Interpreting Interval Descriptions:
  • "At most X": Encompasses $X$ and all numerical values strictly less than $X$.
  • "At least X": Encompasses $X$ and all numerical values strictly greater than $X$.
  • "Between X and Y, inclusive": Encompasses $X$, $Y$, and every number located between them.
  • "Strictly between X and Y": Excludes both boundaries $X$ and $Y$, including only numbers situated between them.
  • Data Transformations: Applying mathematical functions such as a logarithmic transformation (e.g., $\log(x)$) can convert heavily right-skewed data into a symmetrical, bell-shaped distribution suitable for parametric statistical procedures.

Fundamental Probability Concepts and Axioms

• Experiments, Sample Spaces, and Events:
  • Experiment: Any well-defined process or procedure that generates an uncertain or non-deterministic outcome.
  • Sample Space ($\mathcal{S}$): The comprehensive set of all possible outcomes of an experiment.
  • Example: For a single roll of a standard six-sided die, the sample space is:       $\mathcal{S} = \{1, 2, 3, 4, 5, 6\}$
  • Event: Any specific subset of outcomes contained within the sample space $\mathcal{S}$.
  • Simple Event: An event comprising exactly one individual outcome.
  • Compound Event: An event comprising two or more distinct outcomes.
    • Example: For rolling a die, the event $E$ defined as "rolling an even number" corresponds to:         $E = \{2, 4, 6\}$
• Set Operations on Events:
  • Complement ($A'$): The event containing all outcomes in the sample space $\mathcal{S}$ that are not in event $A$.
  • Union ($A \cup B$): The event containing all outcomes that belong to $A$, to $B$, or to both (signifying that at least one of the events occurs).
  • Intersection ($A \cap B$): The event containing outcomes that belong simultaneously to both event $A$ and event $B$ (signifying that both events occur together).
  • Mutually Exclusive (Disjoint) Events: Events that share no common outcomes, meaning their intersection equals the empty set ($\emptyset$):     $A \cap B = \emptyset$
• Axioms of Probability:
  • Non-negativity: For any event $A$, the assigned probability must be non-negative:     $P(A) \ge 0$
  • Certainty: The total probability assigned to the complete sample space $\mathcal{S}$ equals 1:     $P(\mathcal{S}) = 1$
  • Additivity: For any infinite sequence of mutually exclusive (disjoint) events $A_1, A_2, A_3, \dots$:     $P(A_1 \cup A_2 \cup A_3 \cup \dots) = P(A_1) + P(A_2) + P(A_3) + \dots$
• Properties Derived from Probability Axioms:
  • Probability of an impossible event (empty set):     $P(\emptyset) = 0$
  • Probability of a complement event:     $P(A') = 1 - P(A)$
  • Upper boundary for probability:     $P(A) \le 1$
  • Special Addition Rule for two mutually exclusive events $A$ and $B$:     $P(A \cup B) = P(A) + P(B)$
  • General Addition Rule for any two arbitrary events $A$ and $B$:     $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• Interpretations and Calculation of Probability:
  • Objective Interpretation (Relative Frequency): Probability represents the long-run limiting proportion of occurrences of an event across an infinite series of independent, identical trials. Applicable strictly to repeatable phenomena.
  • Subjective Interpretation: Probability expresses a person's subjective measure of belief or confidence that a particular event will occur. Applicable to unique, non-repeatable situations.
  • General Systematic Approach to Computing Probability:
  1. Define the full sample space ($\mathcal{S}$).
  2. Assign valid probability values to every simple event such that their sum equals 1.
  3. Calculate the total probability of any compound event by summing the probabilities of all simple events composing it.
  • Equally Likely Outcomes:
  • If a sample space $\mathcal{S}$ consists of $N$ outcomes that are all equally likely to occur, the probability assigned to each individual simple outcome is $\frac{1}{N}$.
  • For any event $A$ containing $N(A)$ outcomes:       $P(A) = \frac{\text{Number of outcomes in } A}{\text{Total number of outcomes in } \mathcal{S}} = \frac{N(A)}{N}$