Comprehensive Study Guide: Statistical Methods and Analysis

Foundations of Statistical Analysis

• Core Definitions in Statistics

  • Statistics: The science of collecting, organizing, summarizing, and interpreting data. It is fundamentally defined as the science of learning from data.

  • Data: Information or facts describing a group of interest. Data consists of more than just numbers; context is critical to interpreting data accurately.

  • Key Preliminary Questions: Before producing data, three primary questions must be defined:

  • What is the specific group of interest?

  • What information about this group do we want to obtain?

  • How will this information be collected?

• Population versus Sample

  • Population: The entire group of individuals or units about which information is desired.

  • Census: The collection of data from every single member of a population. A census is performed when it can be executed quickly and cheaply. However, a census is frequently unfeasible due to time and financial constraints.

  • Sample: A subset of the population from which data is actually gathered, selected according to a prescribed sampling method. Sample data is organized, described, and analyzed to make inferences.

  • Case Studies in Population and Sample Identification:

  • Gallup Poll: Surveys a group of 1,500 adult U.S. residents weekly to assess national opinion.

    • Population: All adult U.S. residents.

    • Sample: The specific 1,500 adult U.S. residents surveyed in that week's poll.

  • 2000 U.S. Census Long Form: Gathered basic data from all U.S. households while sending an in-depth long form to 17% of households.

    • Population: All U.S. households in the year 2000.

    • Sample: The 17% of U.S. households in the year 2000 that received and completed the long form.

  • Voltage Regulator Study: Studying output voltage variation affecting overall product performance by analyzing 5 regulator devices in a laboratory.

    • Population: All regulator devices.

    • Sample: The 5 specific regulator devices sent to the laboratory for study.

• Branches of Statistics and the Analytical Process

  • Descriptive Statistics: Methods for summarizing, organizing, and displaying key features of a dataset through visual/graphical representations and numerical summary metrics.

  • Inferential Statistics: Techniques for generalizing sample findings and drawing formal conclusions about the broader population based on sample data.

  • The Statistical Analysis Process Workflow:

• Bridging Concepts:

  • Producing Data: Designing studies and sampling methods to ensure the sample accurately represents the target population.

  • Probability: The mathematical framework that quantifies randomness and chance, allowing the transition from describing sample statistics to making population-level inferences.

  • Observational Studies versus Experiments

• Observational Study: The researcher observes and measures subjects without applying treatment, interference, or manipulation (e.g., sample surveys).

• Experiment: A test or series of tests where purposeful changes (treatments) are made to input variable(s) of a process or system to observe and quantify the specific reasons for changes in output response variable(s).

  • Factory Defect Case Study: When items from a specific machine display higher defect rates, an experiment manipulates inputs (replacing the machine, changing operators, controlling localized humidity) to isolate the root cause.

Data Classification and Types of Variables

• Preliminary Questions for Data Examination

  • Who: What individuals/cases are described, and how many individuals are in the dataset?

  • What: How many variables are present, and what are their specific definitions?

  • Why: What is the primary objective of the data analysis, and does the data support achieving this goal?

• Main Categories of Variables

  • Categorical (Qualitative) Variable: Places an individual into one of several groups, classifications, or categories.

  • Examples: Course letter grade (A, B, C, D, F), textbook chapter identifiers.

  • Quantitative (Numerical) Variable: Takes numeric values for which standard arithmetic operations (addition, averaging) make physical and mathematical sense. Quantitative variables are recorded in specific units of measurement.

  • Examples: Examination scores, physical height, weight, monetary amounts.

• Classification of Quantitative Variables

  • Discrete Quantitative Variable: The set of possible values is finite or countable (can be listed as a sequence $1, 2, 3, \dots$).

  • Examples: Count of defective items, SUV sales volume per day.

  • Continuous Quantitative Variable: The set of possible values forms an unbroken numerical continuum or interval along the real number line.

  • Examples: Height, weight, reaction temperature, exact waiting times.

• Dimensionality of Data

  • Univariate Data: Observations made on a single variable per individual.

  • Bivariate Data: Simultaneous observations made on two variables per individual.

  • Multivariate Data: Simultaneous observations made on more than two variables per individual.

Graphical Methods for Categorical Data

• Frequency Tables

  • Standard frequency tables display three distinct parameters for categorical variables:

  • Category names/labels.

  • Frequency ($f$): The raw count of observations belonging to a category.

  • Relative Frequency: The proportion or percentage of total observations in a category: $\text{Relative Frequency} = \frac{\text{Frequency}}{\text{Total Number of Observations}}$

• Bar Graphs

  • Uses distinct rectangular bars to represent categorical counts or relative percentages.

  • Height: Directly proportional to category counts or percentages.

  • Width: Arbitrary with no quantitative meaning.

  • Spacing: Bars must be explicitly separated by gaps to visually reflect category boundaries.

• Pie Charts

  • Divides a complete circle into radial wedges corresponding to categories.

  • Wedge Size: Proportional to category percentages relative to the whole ($100\%$).

  • Best utilized when explicitly emphasizing a category's proportion relative to the entire dataset.

• Comprehensive Categorical Example: SUV Sales Volume Data ($n = 138$ total vehicles sold):

  • Monday: Count = $15$, Relative Frequency = $\frac{15}{138} = 10.9\%$

  • Tuesday: Count = $23$, Relative Frequency = $\frac{23}{138} = 16.7\%$

  • Wednesday: Count = $35$, Relative Frequency = $\frac{35}{138} = 25.4\%$

  • Thursday: Count = $11$, Relative Frequency = $\frac{11}{138} = 7.9\%$

  • Friday: Count = $12$, Relative Frequency = $\frac{12}{138} = 8.7\%$

  • Saturday: Count = $42$, Relative Frequency = $\frac{42}{138} = 30.4\%$

Graphical Methods for Quantitative Data

• Histograms

  • Displays the distribution of continuous or grouped quantitative variables across equal-width intervals (classes/bins).

  • Bar Height: Proportional to category frequency or count.

  • Bar Width: Represents range limits of the class interval.

  • Bar Contiguity: Adjacent bars touch each other directly, demonstrating continuous data coverage along the axis.

  • Construction Procedure:

  1. Group data into non-overlapping class intervals of equal width.

  2. Construct a frequency table of class counts.

  3. Draw vertical bars above each class boundary matching class frequencies.

• Best Actresses Frequency Distribution Case Study:

  • Age $21-30$: Frequency = $28$

  • Age $31-40$: Frequency = $30$

  • Age $41-50$: Frequency = $12$

  • Age $51-60$: Frequency = $2$

  • Age $61-70$: Frequency = $2$

  • Age $71-80$: Frequency = $2$

• Cal Men's Basketball Game Scores Dataset ($n = 33$ scores):     $49, 56, 58, 65, 66, 67, 68, 69, 69, 69, 70, 70, 73, 74, 74, 74, 75, 75, 76, 77, 77, 79, 80, 81, 84, 84, 86, 86, 89, 90, 92, 102, 117$

  • Stem-and-Leaf Plots (Stem Plots)

• Functions like a horizontal histogram while retaining exact numeric observation values.

• Stem: Consists of leading digit(s).

• Leaf: Consists of final trailing digit(s).

• Construction Procedure:

  1. Sort raw data in ascending order.

  2. Separate each observation into stem and leaf components.

  3. List stems vertically in ascending order and draw a vertical boundary line.

  4. Record leaves horizontally next to stems in ascending order.

  5. State scale units for stems and leaves.

  6. Split Stems: Used to expand detail by dividing stems into two rows (e.g., leaf values $0-4$ on line 1, $5-9$ on line 2).

• Distributional Features and Patterns

  • Modality: Number of major peaks in distribution curve:

  • Unimodal: One distinct peak.

  • Bimodal: Two distinct peaks.

  • Multimodal: More than two distinct peaks.

  • Symmetry versus Skewness:

  • Symmetric: Left and right halves are mirror images (e.g., height, weight).

  • Right-Skewed (Positively Skewed): Long right tail extends toward higher values (e.g., household income, scores on a difficult exam).

  • Left-Skewed (Negatively Skewed): Long left tail extends toward lower values (e.g., retirement age).

• Time Plots

  • Tracks quantitative observations over time.

  • Horizontal Axis: Time period.

  • Vertical Axis: Variable value.

  • Line Segments: Connect consecutive points to highlight temporal change.

  • Key Features:

  • Cycles: Regular, repeating up-and-down fluctuations.

  • Trends: Persistent long-term upward or downward directional shifts.

Measuring Center: Mean and Median

• The Sample Mean $\bar{x}$

  • Represents the arithmetic average and physical balance point of a quantitative distribution.

  • Formula: $\bar{x} = \frac{\sum_{i=1}^{n} x_i}{n} = \frac{x_1 + x_2 + \dots + x_n}{n}$

• Housing Price Example 1 ($n = 7$ values in thousands of dollars):     $\text{Prices} = \{\$144, \$98, \$204, \$177, \$155, \$316, \$100\}$ $\bar{x} = \frac{144 + 98 + 204 + 177 + 155 + 316 + 100}{7} = \frac{1194}{7} = 170.6\,\text{thousand dollars}$

  • The Sample Median $M$

• The exact 50th percentile midpoint of an ordered distribution; half of the observations are smaller and half are larger.

• Calculation Rules:

  1. Order observations from smallest to largest.

  2. If sample size $n$ is odd, $M$ is the middle value.

  3. If sample size $n$ is even, $M$ is the average of the two middle values.

• Housing Price Example 1 ($n = 7$, odd):     $\text{Ordered}: \$98, \$100, \$144, \$155, \$177, \$204, \$316$ $M = \$155\,\text{thousand dollars}$

• Housing Price Example 2 ($n = 10$, even):     $\text{Ordered}: \$98, \$100, \$144, \$155, \$170, \$177, \$177, \$177, \$204, \$316$ $M = \frac{170 + 177}{2} = 173.5\,\text{thousand dollars}$

  • Comparative Properties: Mean versus Median

• Sensitivity to Outliers: The mean is non-resistant and heavily affected by extreme outliers. The median is resistant to outliers.

• Demonstration of Outlier Influence:

  • Initial Dataset ($n = 5$): $1, 2, 3, 4, 5$ $\bar{x} = \frac{15}{5} = 3, \quad M = 3$

  • Dataset with Outlier ($n = 6$): $1, 2, 3, 4, 5, 100$ $\bar{x} = \frac{115}{6} = 19.3, \quad M = \frac{3 + 4}{2} = 3.5$

• Relationship to Distribution Shape:

  • Symmetric Distribution: $\bar{x} \approx M$

  • Right-Skewed Distribution: $\bar{x} > M$

  • Left-Skewed Distribution: $\bar{x} < M$

Measuring Spread: Range, Variance, Standard Deviation, and CV

• Dataset Case Study: Bank Customer Wait Times

  • Big Bank ($n = 11$): $4.1, 5.2, 5.6, 6.2, 6.7, 7.2, 7.7, 7.7, 8.5, 9.3, 11.0\,\text{minutes}$

  • Best Bank ($n = 11$): $6.6, 6.7, 6.7, 6.9, 7.1, 7.2, 7.3, 7.4, 7.7, 7.8, 7.8\,\text{minutes}$

• Sample Range

  • Difference between the maximum and minimum observations: $\text{Range} = \text{Maximum} - \text{Minimum}$

  • Big Bank Range: $11.0 - 4.1 = 6.9\,\text{minutes}$

  • Best Bank Range: $7.8 - 6.6 = 1.2\,\text{minutes}$

  • Best Bank has less overall variability than Big Bank.

• Sample Variance $s^2$

  • Average of squared deviations of observations from the sample mean: $s^2 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n - 1}$

• Sample Standard Deviation $s$

  • The positive square root of sample variance, returning the metric to the original data scale: $s = \sqrt{s^2} = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n - 1}}$

• Step-by-Step Variance and Standard Deviation Computation Example ($n = 6$ monetary observations):

  • Observations ($x_i$): $1600, 1157, 937, 800, 707, 700\,\text{dollars}$

  • Sample Mean: $\bar{x} = 983.5\,\text{dollars}$

  • Deviations ($x_i - \bar{x}$): $616.5, 173.5, -46.5, -183.5, -276.5, -283.5$

  • Squared Deviations ($(x_i - \bar{x})^2$): $380072.25, 30102.25, 2162.25, 33672.25, 76452.25, 80372.25$

  • Sum of Squared Deviations: $\sum (x_i - \bar{x})^2 = 602833.5\,\text{dollars}^2$

  • Sample Variance calculation: $s^2 = \frac{602833.5}{6 - 1} = \frac{602833.5}{5} = 120566.7\,\text{dollars}^2$

  • Sample Standard Deviation calculation: $s = \sqrt{120566.7} = 347.2\,\text{dollars}$

• Properties of Variance and Standard Deviation

  • Both $s^2$ and $s$ are non-resistant and heavily distorted by outliers.

  • Both values are strictly non-negative ($s^2 \ge 0, s \ge 0$).

  • $s = 0$ if and only if all observations take identical numerical values.

• Coefficient of Variation (CV)

  • Ratio of standard deviation to the absolute value of sample mean, measuring relative variability as a unitless ratio or percentage: $CV = \frac{s}{|\bar{x}|}$

  • Bank Account Scale Comparison ($s = \$100$):

  • Account A ($\bar{x} = \$500,000$): $CV = \frac{100}{500000} = 0.0002 = 0.02\%$

  • Account B ($\bar{x} = \$500$): $CV = \frac{100}{500} = 0.20 = 20\%$

  • Travel Duration Comparison ($s = 20\,\text{min}$):

  • Flight AA127 ($\bar{x} = 920\,\text{min}$): $CV = \frac{20}{920} \approx 2.17\%$

  • Commute ($\bar{x} = 60\,\text{min}$): $CV = \frac{20}{60} \approx 33.33\%$

  • The daily commute exhibits far greater relative variation than the international flight.

• Linear Transformation Theorems for Mean and Standard Deviation

  • Given sample $x_1, x_2, \dots, x_n$ with mean $\bar{x}$ and standard deviation $s_x$:

  • Additive Transformation ($y_i = x_i + c$): $\bar{y} = \bar{x} + c$ $s_y = \sqrt{\frac{\sum (x_i + c - (\bar{x} + c))^2}{n - 1}} = \sqrt{\frac{\sum (x_i - \bar{x})^2}{n - 1}} = s_x$

  • Multiplicative Transformation ($y_i = c x_i$): $\bar{y} = c \bar{x}$ $s_y = \sqrt{\frac{\sum (c x_i - c \bar{x})^2}{n - 1}} = \sqrt{\frac{c^2 \sum (x_i - \bar{x})^2}{n - 1}} = |c| s_x$

  • Linear Combination ($y_i = a + b x_i$): $\bar{y} = a + b \bar{x}, \quad s_y = |b| s_x$

  • Unit Conversion Applications:

  • Celsius to Kelvin ($Y = X + 273.15$ $\bar{y} = \bar{x} + 273.15, \quad s_y = s_x$

  • Kilograms to Grams ($Y = 1000 X$ $\bar{y} = 1000 \bar{x}, \quad s_y = 1000 s_x$

  • Celsius to Fahrenheit ($F = \frac{9}{5} C + 32$ $\bar{F} = \frac{9}{5} \bar{C} + 32, \quad s_F = \frac{9}{5} s_C$

Five-Number Summary, Quartiles, and Boxplots

• Quartile Definitions

  • Quartiles divide sorted quantitative data into four equal portions:

  • First Quartile ($Q_1$): 25th percentile; median of lower half of sorted data.

  • Second Quartile ($Q_2$): 50th percentile; overall sample median $M$.

  • Third Quartile ($Q_3$): 75th percentile; median of upper half of sorted data.

• The Five-Number Summary

  • Consists of five numerical benchmarks: $\text{Minimum}, Q_1, M, Q_3, \text{Maximum}$

  • Measures central location ($M$), spread ($\text{Range} = \text{Max} - \text{Min}$), and relative dispersion within quarters.

• Five-Number Summary Worked Example ($n = 13$ observations):

  • Raw Data: $6, -5.2, -1, 3.5, 3.2, -1.7, -1.5, 1, 4.9, 1.4, 7, 3.7, -12$

  • Sorted Data: $-12, -5.2, -1.7, -1.5, -1, 1, 1.4, 3.2, 3.5, 3.7, 4.9, 6, 7$

  • Overall Median $M = 1.4$

  • Lower Half ($n = 6$ values): $-12, -5.2, -1.7, -1.5, -1, 1$ $Q_1 = \frac{-1.7 + (-1.5)}{2} = -1.6$

  • Upper Half ($n = 6$ values): $3.2, 3.5, 3.7, 4.9, 6, 7$ $Q_3 = \frac{3.7 + 4.9}{2} = 4.3$

  • Five-Number Summary: $-12, -1.6, 1.4, 4.3, 7$

• Interquartile Range (IQR) and Outlier Rule

  • Interquartile Range: Distance between third and first quartiles: $IQR = Q_3 - Q_1$

  • $1.5 \times IQR$ Outlier Criterion: An observation is classified as a potential outlier if it falls below the lower fence or above the upper fence: $\text{Lower Fence} = Q_1 - 1.5 \times IQR$ $\text{Upper Fence} = Q_3 + 1.5 \times IQR$

  • Outlier Calculation for Dataset above:

  • $IQR = 4.3 - (-1.6) = 5.9$

  • $1.5 \times IQR = 1.5 \times 5.9 = 8.85$

  • Lower Fence: $-1.6 - 8.85 = -10.45$

  • Upper Fence: $4.3 + 8.85 = 13.15$

  • Because $-12 < -10.45$, $-12$ is confirmed as an outlier.

• Boxplots

  • Graphical representation of five-number summary.

  • Central box spans from $Q_1$ to $Q_3$; vertical line drawn inside box at median $M$

  • Modified boxplot whiskers extend to the smallest and largest observations that are not outliers. Outliers are plotted as individual points beyond whiskers.

• Comparative Boxplots Case Study: Bank Waiting Times:

  • Big Bank 5-Number Summary: $4.1, 5.6, 7.2, 8.5, 11.0$

  • Best Bank 5-Number Summary: $6.6, 6.7, 7.2, 7.7, 7.8$

• Criteria for Selecting Summary Measures

  • Use $\bar{x}$ and $s$ for roughly symmetric distributions containing no extreme outliers.

  • Use the Five-Number Summary ($M$ and $IQR$) for skewed distributions or datasets containing extreme outliers.

Density Curves and Uniform Distributions

• Density Curve Foundations

  • Smooth mathematical curves representing idealized distributions of continuous quantitative variables.

  • Non-negativity: Curve remains strictly on or above the horizontal axis ($f(x) \ge 0$).

  • Total Area: Area underneath density curve equals exactly $1.0$ ($100\%$

  • Proportions: Proportion of population falling within an interval equals area bounded under curve across that interval.

  • Mean and Median in Density Curves:

  • Median $M$: Point that divides total area under curve into equal halves ($0.50$ and $0.50$

  • Mean $\mu$: Physical balance point of density curve density.

• Uniform Distributions

  • A continuous distribution where density is constant over a bounded interval $[a, b]$.

  • Density Height Formula: $h = \frac{1}{b - a}$

  • Interval Proportion Formula: $\text{Proportion} = \text{Interval Length} \times h$

  • Reaction Temperature Case Study ($X \sim \text{Uniform}(-5, 5)$, interval range $= 10$

  • Density Height: $h = \frac{1}{5 - (-5)} = \frac{1}{10} = 0.10$

  • Proportion $X < 0$: $\text{Proportions} = (0 - (-5)) \times 0.10 = 5 \times 0.10 = 0.50$

  • Proportion $-2.5 < X < 2.5$: $\text{Proportions} = (2.5 - (-2.5)) \times 0.10 = 5 \times 0.10 = 0.50$

  • Proportion $-2 < X < 3$: $\text{Proportions} = (3 - (-2)) \times 0.10 = 5 \times 0.10 = 0.50$

The Normal Distribution and the Empirical Rule

• Normal Distribution Characteristics

  • Symmetric, unimodal, bell-shaped continuous probability distributions spanning $-\infty$ to $+\infty$

  • Parameterized entirely by population mean $\mu$ and standard deviation $\sigma$, denoted $N(\mu, \sigma)$.

  • Mean $\mu$ locates center peak; standard deviation $\sigma$ controls width and flatness.

• The 68-95-99.7 Rule (Empirical Rule)

  • For any normally distributed variable $N(\mu, \sigma)$, standard area proportions are approximated as follows:

  • Approximately 68% of observations fall within $1\sigma$ of $\mu$ ($[\mu - \sigma, \mu + \sigma]$).

  • Approximately 95% of observations fall within $2\sigma$ of $\mu$ ($[\mu - 2\sigma, \mu + 2\sigma]$).

  • Approximately 99.7% of observations fall within $3\sigma$ of $\mu$ ($[\mu - 3\sigma, \mu + 3\sigma]$).

• Specific Empirical Tail Areas:

  • Outside $1\sigma$: $32\%$ total ($16\%$ in each outer tail).

  • Outside $2\sigma$: $5\%$ total ($2.5\%$ in each outer tail).

  • Outside $3\sigma$: $0.3\%$ total ($0.15\%$ in each outer tail).

• Law Test Scores Application ($\mu = 50, \sigma = 5$

  • Percent scoring below 50: $\mu = 50$ is the median, so $50\%$ score below 50.

  • Middle 68% range: $50 \pm 1(5) \implies [45, 55]$.

  • Percent scoring above 65: Score $65$ lies $3\sigma$ above mean ($\frac{65 - 50}{5} = +3$). Area above $+3\sigma = 0.15\%$

The Standard Normal Distribution and Z-Scores

• Standard Normal Random Variable $Z \sim N(0,1)$

  • Standard normal distribution with mean $\mu = 0.0$ and standard deviation $\sigma = 1.0$

• Standardization Formula (Z-Score)

  • Converts raw score $x$ from $N(\mu, \sigma)$ into standard units $z$: $z = \frac{x - \mu}{\sigma}$

  • Represents the number of standard deviations $x$ lies away from mean $\mu$.

• Cumulative Proportions via Z-Table (Table A)

  • Table A provides cumulative left-tail area $P(Z \le z)$.

  • Complement Rule for Right Tail: $P(Z > z) = 1 - P(Z \le z)$

  • Difference Rule for Interval: $P(a < Z < b) = P(Z \le b) - P(Z \le a)$

  • Table Look-up Practice Examples:

  • $P(Z \le 0.12) = 0.5478$

  • $P(Z \le -0.21) = 0.4168$

  • $P(Z < 2.85) = 0.9978$

  • $P(Z > 2.85) = 1 - 0.9978 = 0.0022$

  • $P(Z < -1.66) = 0.0485$

  • $P(-1.66 < Z < 2.85) = P(Z < 2.85) - P(Z < -1.66) = 0.9978 - 0.0485 = 0.9493$

• Baby Elephants Case Study Application ($\mu = 224\,\text{lbs}, \sigma = 53\,\text{lbs}$):

  • Percent lighter than 200 lbs: $z = \frac{200 - 224}{53} = \frac{-24}{53} = -0.45$ $P(X < 200) = P(Z < -0.45) = 0.3264 = 32.64\%$

  • Percent between 200 and 300 lbs: $z_{200} = -0.45, \quad z_{300} = \frac{300 - 224}{53} = \frac{76}{53} = 1.43$ $P(200 < X < 300) = P(-0.45 < Z < 1.43) = P(Z < 1.43) - P(Z < -0.45) = 0.9236 - 0.3264 = 0.5972 = 59.72\%$

  • Percent heavier than 445 lbs: $z = \frac{445 - 224}{53} = \frac{221}{53} = 4.17$ $P(X > 445) = P(Z > 4.17) = 1 - P(Z < 4.17) \approx 1 - 1.0000 = 0.0000 = 0\%$

Percentiles and Inverse Normal Calculations

• Definitions

  • Deciles: Cutoff points dividing distribution into 10 equal proportions ($10\%, 20\%, \dots, 90\%$

  • Percentiles: Cutoff points dividing distribution into 100 equal proportions ($1\%, 2\%, \dots, 99\%$

• Inverse Normal Calculation Procedure

  1. Locate target percentage/area inside Table A grid to find corresponding $z$

  2. Unstandardize $z$ back to original continuous scale: $x = \mu + z \sigma$

• Steel Rebar Length Case Study Application ($\mu = 143\,\text{in}, \sigma = 29\,\text{in}$):

  • Goal: Find length below which 10% of rebar lengths fall (10th percentile).

  • Target area = $0.1000$

  • Table A look-up: Closest entry = $0.1003$ at $z = -1.28$

  • Unstandardization: $x = 143 + (-1.28)(29) = 143 - 37.12 = 105.88\,\text{inches}$

Bivariate Data, Scatterplots, and Correlation

• Explanatory and Response Variables

  • Response Variable (Dependent Variable): Measures study outcome; plotted on vertical axis ($y$

  • Explanatory Variable (Independent / Predictor Variable): Influences or explains changes in response variable; plotted on horizontal axis ($x$

  • Variable Identification Examples:

  • Touchdowns thrown (Explanatory) vs. Quarterback salary (Response).

  • Amount of exercise (Explanatory) vs. Weight loss (Response).

  • Years on job vs. Income (Role depends on study goals).

• Scatterplots

  • Visual display of relationship between two quantitative variables measured on the same individuals.

  • Analytical Assessment Criteria:

  • Form: Linear, curved, or clusters. Assessed using rectangle method surrounding data.

  • Direction: Positive (variables move together), Negative (one increases as other decreases), No relationship.

  • Strength: Length-to-width ratio of bounding rectangle (narrow rectangle indicates strong relationship; wide rectangle indicates weak relationship).

  • Categorical Layering: Categorical variables can be added to scatterplots by using distinct point colors or shapes (e.g., metabolic rate versus body mass stratified by sex: Females mass $33.1-54.6\,\text{kg}$, rate $913-1502\,\text{cal}$; Males mass $46.9-62.9\,\text{kg}$, rate $1362-1867\,\text{cal}$).

• The Correlation Coefficient $r$

  • Quantifies direction and strength of linear association between two quantitative variables: $r = \frac{1}{n - 1} \sum_{i=1}^{n} \left( \frac{x_i - \bar{x}}{s_x} \right) \left( \frac{y_i - \bar{y}}{s_y} \right)$

• Properties of Correlation $r$

  • Bounded between $-1$ and $+1$ ($-1 \le r \le 1$

  • Sign gives linear direction ($+$ positive, $-$ negative).\n * Magnitude gives strength ($|r| \approx 1$ strong linear, $r \approx 0$ weak linear).

  • Requires both variables to be quantitative.

  • Unitless metric.

  • Invariant to variable role swapping ($x$ and $y$ labels are interchangeable).

  • Invariant to linear scale/unit changes in $x$ or $y$

  • Non-resistant to outliers.

  • Measures ONLY linear trends.

• Device Capacity versus Price Worked Calculation Example ($n = 7$ devices):

  • Size ($x$): $\bar{x} = 13.6\,\text{GB}, s_x = 9.5\,\text{GB}$

  • Price ($y$): $\bar{y} = \$407.1, s_y = \$209.0$

  • Point Z-score products ($z_x z_y$):

  • Device 1 ($8\,\text{GB}, \$310$): $z_x = -0.6, z_y = -0.5 \implies z_x z_y = 0.30$

  • Device 2 ($6\,\text{GB}, \$290$): $z_x = -0.8, z_y = -0.6 \implies z_x z_y = 0.48$

  • Device 3 ($18\,\text{GB}, \$500$): $z_x = 0.5, z_y = 0.4 \implies z_x z_y = 0.20$

  • Device 4 ($30\,\text{GB}, \$800$): $z_x = 1.7, z_y = 1.9 \implies z_x z_y = 3.23$

  • Device 5 ($20\,\text{GB}, \$470$): $z_x = 0.7, z_y = 0.3 \implies z_x z_y = 0.21$

  • Device 6 ($10\,\text{GB}, \$330$): $z_x = -0.4, z_y = -0.4 \implies z_x z_y = 0.16$

  • Device 7 ($3\,\text{GB}, \$150$): $z_x = -1.1, z_y = -1.2 \implies z_x z_y = 1.32$

  • Sum of Z-score products = $5.90$

  • Correlation computation: $r = \frac{5.90}{7 - 1} = \frac{5.90}{6} = 0.98$

• Analytical Cautions in Correlation Analysis

  • Non-linearity: $r$ measures linear patterns only. Perfect non-linear relationships (e.g., parabolas) can produce $r \approx 0$.

• Lurking Variables: Unobserved variables that drive apparent associations between $x$ and $y$

• Correlation Does Not Imply Causation: High correlation ($r = 0.90$) between ice cream sales and drowning rates is driven by summer temperature as a lurking variable.