Describing Distributions

Introduction to Distributions

• Lecturer: Jennifer Linden

• Lecture Title: Describing Distributions

• Objectives: - Explain the fundamental significance of the normal distribution (also known as the Gaussian distribution) in statistical analysis, particularly its ubiquitous appearance in large samples of experimental data due to natural variation and random processes.

  • Describe the key characteristics of a normal distribution with precision and demonstrate how to use these properties, especially the empirical rule, to estimate probabilities of data points occurring within certain ranges.

  • Define and differentiate between the various features of data distributions: modality (number of peaks), kurtosis (tail heaviness), and skew (asymmetry).

  • Calculate and interpret the three primary measures of central tendency: mode, median, and mean, understanding their individual strengths and weaknesses.

  • Calculate and interpret the key measures of dispersion: range, interquartile range (IQR), variance, and standard deviation, explaining what each measure conveys about data spread.

  • Explain the profound effects of non-normality in data distributions on these calculated measures, highlighting when certain measures become more or less representative.

Normal Distribution

• Definition: - The normal distribution is a continuous probability distribution that describes the probabilities of different sample values from a single underlying process, assuming that variations are purely random and follow a specific pattern. It's often observed when many small, independent random effects are added together.

• Also known as: - Gaussian distribution, named after mathematician Carl Friedrich Gauss.

• Conditions for Normal Distribution: - Large sample size from a single, homogeneous population: This condition is crucial because with more data points, random fluctuations tend to average out, allowing the underlying stable process to emerge.

  • Multiple independent sources of measurement error: When several independent factors contribute to the variability of a measurement, their combined effect often leads to a normal distribution. For example, errors in measurement tools, environmental variations, and individual differences.

• Central Limit Theorem: - This foundational theorem in statistics establishes that, given a sufficiently large sample size (typically $n \ge 30$ is considered sufficient, but it can vary), the sampling distribution of the sample mean will approximate a normal distribution, regardless of the shape of the population distribution itself. This theorem is extremely powerful as it allows parametric statistical tests to be applied to many real-world data sets, even if the original population is not normally distributed.

• Example: Quincunx Machine - This mechanical device (also known as a Galton board) visually simulates the creation of a normal distribution. Balls are dropped from the top and fall through a series of pins, bouncing left or right with equal probability at each pin. The random path of each ball, accumulating small independent choices, results in a bell-shaped pile at the bottom, demonstrating how random processes converge to a normal distribution. It provides a tactile and intuitive understanding of statistical concepts like central tendency and dispersion.

  • Encouragement to explore the Quincunx machine for practical understanding through online simulations or physical models.

Characteristics of Normal Distribution

• Shape: - A distinctive bell-shaped curve that is perfectly symmetric around its central point. This symmetry implies that the likelihood of observing a value below the mean is identical to the likelihood of observing a value equivalently above the mean.

• Central Point: - In a perfectly normal distribution, the mode (most frequent value), median (middle value), and mean (average value) all coincide precisely at the center of the distribution. This is a direct consequence of its perfect symmetry.

• Empirical Rule (68-95-99.7 rule): - A practical guideline that provides approximate percentages of data within specific standard deviation ranges from the mean for a normal distribution:

  • Approximately 68% of the data falls within 1 standard deviation ($\pm 1 \sigma$) from the mean.

  • Approximately 95% of the data falls within 2 standard deviations ($\pm 2 \sigma$) from the mean.

  • Approximately 99.7% of the data falls within 3 standard deviations ($\pm 3 \sigma$) from the mean. This means almost all data in a normal distribution lies within three standard deviations.

• Probability Estimations: - The Empirical Rule allows for quick estimations of probabilities. More precisely, by calculating how many standard deviations ($z$-score) a data point is from the mean, one can determine the likelihood (or cumulative probability) of observing that data point or one more extreme. For instance, values beyond 2 standard deviations from the mean in a normal distribution are considered relatively rare.

Hypothetical Research Problem Example

• Research Context: - An experiment designed to validate a new speech and noise perception test, involving quantitative measurements of speech recognition thresholds (SRTs).

• Experimental Setup: - The study was conducted with a substantial sample of 200 participants. The SRTs for 190 of these participants were measured to fall within a specific range, between -10 and +14 dB SNR (decibels Signal-to-Noise Ratio).

• Questions to Answer: - Assuming the data is normally distributed, what is the mean SRT of all participants?

  • What is the standard deviation of this SRT distribution?

  • What percentage of the data would be expected to fall between -13 and +17 dB SNR?

• Answers: - The calculated mean SRT is 2 dB.

  • The standard deviation of the SRT distribution is 6 dB.

  • Based on these parameters, the percentage of data expected between -13 and +17 dB SNR is approximately 99.5%, suggesting this range covers almost the entire distribution.

Features of Data Distributions

Modality

• Definition: - Modality refers to the number of distinct peaks or prominent humps observed in the probability distribution of a dataset. Each peak represents a region of higher frequency or probability of values.

• Types: - Unimodal: A distribution with a single, clear peak, such as the normal distribution.

  • Bimodal: A distribution with two distinct peaks, often indicating the presence of two different groups or populations within the dataset (e.g., height distribution in a sample of both men and women).

  • Multi-modal: A distribution exhibiting more than two peaks, suggesting multiple distinct clusters or subgroups in the data.

  • Uniform: A distribution where all values within a given range occur with roughly the same frequency, resulting in a flat, rectangular shape with an infinite number of (equally weighted) peaks or no discernible peak at all.

Kurtosis

• Definition: - Kurtosis measures the "tailedness" of a distribution, specifically the concentration of the data in its tails and the sharpness of its peak relative to a normal distribution. Higher kurtosis implies more extreme outliers (heavier tails) and a sharper central peak.

• Types: - Mesokurtic: Describes a distribution with kurtosis similar to that of a normal distribution. By convention, a normal distribution has a kurtosis value of 3 (or an excess kurtosis of 0 when normalized). The tails are neither too heavy nor too light, and the peak is moderately rounded.

  • Leptokurtic: Characterized by higher kurtosis (excess kurtosis > 0), meaning it has heavier tails and a sharper, more pronounced peak than a normal distribution. This indicates that more of the variance arises from infrequent extreme deviations, or outliers.

  • Platykurtic: Characterized by lower kurtosis (excess kurtosis < 0), meaning it has lighter tails and a flatter, more rounded peak than a normal distribution. This indicates that observations are more spread out from the center, and extreme values are less likely to occur.

Skew

• Definition: - Skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable about its mean. A distribution is skewed if one of its tails is longer than the other.

• Types: - Symmetric: A distribution where both tails are equal in length and shape, and the data is evenly distributed around the mean. In a perfectly symmetric distribution (like the normal distribution), the mean, median, and mode are all equal.

  • Positively skewed (Right-skewed): The heavier or longer tail is on the right side (positive direction) of the distribution. This typically occurs when there are a few unusually high values. In a positively skewed distribution, the mean is usually greater than the median, which is often greater than the mode (Mean > Median > Mode).

  • Negatively skewed (Left-skewed): The heavier or longer tail is on the left side (negative direction) of the distribution. This typically occurs when there are a few unusually low values. In a negatively skewed distribution, the mean is usually less than the median, which is often less than the mode (Mean < Median < Mode).

Measures of Central Tendency

Mode

• Definition: - The mode is the most frequently occurring value or values in a dataset. It is the only measure of central tendency that can be used effectively for nominal (categorical) data.

• Calculation: - To find the mode, one simply counts the occurrences of each unique value in the dataset. The value (or values) with the highest frequency is the mode.

  • A dataset may have more than one mode (e.g., bimodal if two values occur with the same highest frequency) or, in the case of a uniform distribution, an infinite number of modes (as all values have equal frequency).

  • The mode is not sensitive to outliers.

Median

• Definition: - The median is the middle value in a dataset when the values are arranged in ascending or descending order. It divides the dataset into two equal halves, meaning 50% of the data points are below it and 50% are above it.

• Calculation: - First, the data must be ranked from the lowest to the highest value.

  • If there is an odd number of samples ($n$), the median is the value at the $(n+1)/2$ position.

  • If there is an even number of samples ($n$), the median is the average of the two middle values, specifically the values at the $n/2$ and $(n/2)+1$ positions.

  • The median is highly resistant to the influence of extreme values (outliers) and skew, making it a robust measure of central tendency for skewed distributions.

Mean (Arithmetic Mean)

• Definition: - The mean, or arithmetic average, is the sum of all sample values divided by the number of samples. It is the most commonly used measure of central tendency.

• Calculation: - The formula for the sample mean ($\bar{x}$) is:
  $\bar{x} = \frac{\sum<em>{i=1}^{n} x</em>i}{n}$
  where $\sum x_i$ is the sum of all individual data points, and $n$ is the total number of data points in the sample.

• Sensitivity: - The mean is highly sensitive to extreme values (outliers) and the presence of skew in the distribution. Outliers can pull the mean significantly in their direction, making it less representative of the typical value in skewed datasets.

  • This sensitivity is particularly illustrated in skewed distributions, where the mean shifts towards the tail of the skew more dramatically than the median or mode.

Measures of Dispersion

Range

• Definition: - The range is the simplest measure of dispersion, calculated as the difference between the maximum and minimum values in a dataset.

• Characteristics: - While easy to calculate, the range is highly sensitive to outliers, as a single extremely high or low value can greatly inflate or deflate it.

  • It provides limited information about the distribution or clustering of data points between the extremes, only indicating the full span.

Interquartile Range (IQR)

• Definition: - The Interquartile Range (IQR) is a measure of statistical dispersion, representing the range of the middle 50% of the data. It is particularly useful for skewed data or data with outliers because it's based on values that come from the center of the distribution.

• Calculation: - The data is first divided into quartiles:

  • The first quartile ($Q_1$) is the median of the lower half of the dataset.

  • The second quartile ($Q_2$) is the median of the entire dataset.

  • The third quartile ($Q_3$) is the median of the upper half of the dataset.

  • The IQR is then calculated as the difference between the third quartile and the first quartile: $IQR = Q<em>3 - Q</em>1$.

  • The IQR is less sensitive to outliers than the simple range, providing a more robust measure of spread.

Variance

• Definition: - Variance measures the average of the squared differences from the mean for each data point in a dataset. It quantifies how much the data points deviate from the average.

• Calculation: - The process involves several steps:

  1. Compute the mean ($\bar{x}$) of the dataset.

  2. Subtract the mean from each individual value ($x_i - \bar{x}$) to find the deviations.

  3. Square each of these differences $(x_i - \bar{x})^2$ to eliminate negative values and give more weight to larger deviations.

  4. Sum all the squared differences ($\sum (x_i - \bar{x})^2$).

  5. Divide the sum by the number of values ($n$) for population variance or by $(n-1)$ for sample variance (to provide an unbiased estimate of the population variance).

  • The formula for sample variance ($s^2$) is:
    $s^2 = \frac{\sum<em>{i=1}^{n} (x</em>i - \bar{x})^2}{n-1}$

• Sensitivity: - Like the mean, variance is sensitive to outliers because extreme values result in large squared differences, significantly increasing the variance.

  • Another characteristic is that its units are the square of the original sample units, making it less intuitive to interpret directly in relation to the data.

Standard Deviation

• Definition: - The standard deviation is the square root of the variance. It is a widely used measure of the spread of data around the mean.

• Characteristics: - Because it is the square root of the variance, the standard deviation is expressed in the same units as the original sample data, making it a much more convenient and interpretable measure of dispersion than variance.

  • A higher standard deviation indicates greater variability or spread in the data, while a lower standard deviation suggests that data points tend to be closer to the mean.

  • The formula for sample standard deviation ($s$) is:
    $s = \sqrt{\frac{\sum<em>{i=1}^{n} (x</em>i - \bar{x})^2}{n-1}}$

Conclusion and Review

• End of lecture expectations: - Graduates of this lecture should possess the ability to comprehensively explain the concept and significance of the normal distribution, recognizing its importance in statistics.

  • They should be able to accurately describe the inherent characteristics of a normal distribution and competently use these features for practical probability estimation.

  • They are expected to precisely define and differentiate between the key features of data distributions: modality, kurtosis, and skew, understanding what each implies about the data's shape.

  • They should be proficient in calculating and appropriately applying measures of central tendency (mode, median, mean) and measures of dispersion (range, interquartile range, variance, standard deviation).

• Gratitude expressed for attention and engagement throughout the lecture.