Bivariate Data Analysis: Scatterplots, Pearson Correlation, and Significance Testing

Foundations of Bivariate Data and the Statistical Workflow

• Univariate Versus Bivariate Data:

  • Univariate Data: Consists of observations gathered on a single quantitative or qualitative characteristic per individual. Summarization focuses on center, shape, and spread.

  • Bivariate Data: Involves measurements collected on two separate variables for each individual or observational unit simultaneously. The primary investigative goal is to discover whether a mathematical or statistical relationship exists between the two variables, characterize that relationship, or conclude that the variables operate independently.

• Analytical Toolkit for Bivariate Analysis:

  • Scatterplots: Two-dimensional graphical displays utilized to visually identify patterns, trends, clusters, and potential relationship shapes.

  • Correlation: A standardized quantitative metric measuring the direction and linear strength between two paired quantitative variables, assessing how closely plotted points cluster along a straight line.

  • Regression: Mathematical modeling used to determine the best-fit line or curve, allowing researchers to predict values of one variable based on known values of another when a linear trend is supported.

• Placement Within the Four-Stage Statistical Process:

  • Stage 1: Identify the question and population.

  • Stage 2: Collect the data.

  • Stage 3: Organize and summarize the information (the focus of bivariate correlation and exploratory data analysis).

  • Stage 4: Make an inference if appropriate.

Bivariate Variables: Explanatory and Response Definitions

• Association:

  • Two variables are considered associated when specific values of one variable tend to occur concurrently with specific values of the other. Knowing the value of one variable provides meaningful predictive or explanatory insight into the other.

  • Associations can occur between two qualitative variables, one qualitative and one quantitative variable, or two quantitative variables.

• Correlation:

  • Defined specifically as an association between paired quantitative (numerical) variables.

• Formal Variable Roles in Paired Data:

  • Paired Data Representation: Bivariate observations are expressed mathematically as ordered pairs: $(x, y)$.

  • Explanatory Variable ($x$): Also known as the independent or predictor variable; plotted on the horizontal axis. It serves as the input used to explain, model, or predict changes in the second variable.

  • Response Variable ($y$): Also known as the dependent or outcome variable; plotted on the vertical axis. It represents the outcome or measurement of interest whose variation is being studied.

Constructing and Interpreting Scatterplots

• Definition and Purpose:

  • A scatterplot (or scatter diagram) is a Cartesian coordinate plot displaying bivariate data as individual coordinate points $(x, y)$. It serves as the primary visual diagnostic tool for examining bivariate relationships before computing summary statistics.

• Manual Construction Procedure:

  1. Designate the explanatory variable as the horizontal axis ($x$) and the response variable as the vertical axis ($y$).

  2. Establish uniform numerical scales along both axes covering the full data range.

  3. Plot each individual observation as a discrete point at its respective $(x, y)$ coordinates.

• Case Demonstration: Student Exam Scores (Example 1):

  • An instructor tracked performance across two exams to evaluate whether Exam 1 scores ($x$, explanatory) could predict Exam 2 scores ($y$, response).

• Interpretation: The scatterplot demonstrates an upward trend: students achieving higher scores on Exam 1 generally attained higher scores on Exam 2, while students with lower Exam 1 scores tended to register lower Exam 2 scores. This visual trend indicates a positive association.

• StatCrunch Software Workflow for Scatterplots:

  1. Enter or import the bivariate data into two designated adjacent columns.

  2. Navigate the top menu: click Graph $\rightarrow$ Scatter Plot.

  3. Assign the respective column headers to the X variable and Y variable drop-down menus.

  4. Click Compute! to render the scatterplot.

Classifying Types, Direction, and Strength of Correlation

• Directional Classifications:

  • Positive Correlation: Occurs when the points follow a general linear trajectory with a positive slope (rising from lower-left to upper-right). As the explanatory variable ($x$) increases, the response variable ($y$) tends to increase.

  • Negative Correlation: Occurs when the points follow a linear path with a negative slope (falling from upper-left to lower-right). As the explanatory variable ($x$) increases, the response variable ($y$) tends to decrease.

• Structural Classifications:

  • Linear Correlation: Points cluster along a straight line path.

  • Non-linear (Curvilinear) Correlation: Data display a systematic relationship that is not straight (e.g., an inverted U-shaped parabolic curve resembling $y = -x^2$).

  • No Correlation: Points are dispersed uniformly or randomly across the Cartesian space, exhibiting neither an upward, downward, nor curved pattern.

• Qualitative Descriptors of Strength:

  • Relationships are qualitatively characterized as weak, moderate, or strong based on how tightly the points cluster along the trend trajectory.

  • Subjectivity Warning: Qualitative verbal descriptions of scatterplots without numerical statistics are subjective and prone to differing interpretations.

• Visual Case Studies:

  • Fast Food Nutrition (Example 3): Comparing Total Fat ($x$, in grams) to Protein ($y$, in grams) across restaurant chain meals reveals a strong positive linear pattern rising from left to right.

  • Automotive Specifications (Example 4): Comparing Vehicle Curb Weight ($x$, in pounds) to Fuel Economy ($y$, in Miles Per Gallon [MPG]) demonstrates a strong negative linear trend where heavier vehicles systematically exhibit poorer fuel economy.

• Cardiovascular Clinical Metrics (Example 5): Plotting Resting Heart Rate ($x$, in beats per minute [BPM]) against HDL Cholesterol Level ($y$, in milligrams per deciliter [mg/dL]) produces a diffuse, unstructured cloud of points, indicating no correlation.

• Common Real-World Directional Examples:

  • Hours of Study versus Exam Score: Positive correlation.

  • Hours of Television Watched per Week versus GPA: Negative correlation.

  • Human Height versus Body Weight: Positive correlation.

  • Amount of Aerobic Exercise versus Body Fat Percentage: Negative correlation.

  • Cell Phone Brand versus Resting Blood Pressure: No correlation.

The Pearson Linear Correlation Coefficient

• Origin and Purpose:

  • Developed by Karl Pearson to eliminate subjective assessments of scatterplots by providing an objective numerical index measuring how closely data points adhere to a straight line.

• Notation:

  • Sample linear correlation coefficient: denoted by $r$.

  • Population linear correlation coefficient parameter: denoted by $\rho$ (Greek letter rho).

• Mathematical Formula:

  • The sample linear correlation coefficient evaluates the mean product of standardized paired $z$-scores:

$r = \frac{\sum z_x \cdot z_y}{n - 1}$

• Where standardized scores for each observation are defined as:

$z_x = \frac{x_i - \bar{x}}{s_x}$

$z_y = \frac{y_i - \bar{y}}{s_y}$

• Expanded algebraic form:

$r = \frac{\sum \left(\frac{x_i - \bar{x}}{s_x}\right)\left(\frac{y_i - \bar{y}}{s_y}\right)}{n - 1}$

*   Where:
    *   $n$ = total number of paired data coordinates.
    *   $\bar{x}$ and $\bar{y}$ = sample means of variables $x$ and $y$.
    *   $s_x$ and $s_y$ = sample standard deviations of variables $x$ and $y$.

• Mathematical Properties of the Correlation Coefficient ($r$):

  • Bounded Range: The coefficient is strictly bounded within the closed interval $-1 \le r \le +1$.

  • Perfect Negative Linearity ($r = -1$): All coordinate pairs lie precisely on a downward-sloping straight line.

  • Perfect Positive Linearity ($r = +1$): All coordinate pairs lie precisely on an upward-sloping straight line.

  • Magnitude Interpretation: The closer $|r|$ approaches $1$, the stronger the linear relationship and the tighter points align to a line. As $r$ approaches $0$, linear correlation weakens toward total dispersion.

  • Crucial Diagnostic Caveat ($r = 0$): An outcome of $r = 0$ denotes the total absence of a linear association. It does not prove the absence of a relationship, as strong non-linear or curvilinear patterns (such as parabolas) can produce $r = 0$.

• Numerical Spectrum Guidelines:

  • $r = -1.0$: Perfect Negative Linear Correlation

  • $-1.0 < r < -0.5$: Strong Negative Linear Correlation

  • $r \approx -0.5$: Moderate Negative Linear Correlation

  • $-0.5 < r < 0.0$: Weak Negative Linear Correlation

  • $r = 0.0$: No Linear Correlation

  • $0.0 < r < 0.5$: Weak Positive Linear Correlation

  • $r \approx 0.5$: Moderate Positive Linear Correlation

  • $0.5 < r < 1.0$: Strong Positive Linear Correlation

  • $r = 1.0$: Perfect Positive Linear Correlation

• Worked Manual Calculation:

  • Given the five bivariate observations ($n = 5$): $(4, 3)$, $(2, 2)$, $(1, 8)$, $(3, 5)$, $(8, 4)$.

  • Summary statistics:

    • $\bar{x} = 3.6$

    • $s_x = 2.7018512 \approx 2.7019$

    • $\bar{y} = 4.4$

    • $s_y = 2.3021729 \approx 2.3022$

• Sum of products of standardized scores:

$\sum z_x \cdot z_y = -1.319$

• Compute Pearson correlation coefficient ($r$):

$r = \frac{-1.319}{5 - 1} = \frac{-1.319}{4} = -0.32975 \approx -0.3297$

• StatCrunch computational verification yields $r = -0.3296$.

• Interpretation: Weak negative linear correlation.

Technology-Based Computation of Correlation

• StatCrunch Software Workflow for Linear Correlation:

  1. Organize bivariate data into two designated columns within the spreadsheet.

  2. Navigate the top menu: click Stat $\rightarrow$ Summary Stats $\rightarrow$ Correlation.

  3. Hold down the Shift key (Windows) or Command key (macOS) and click both variable names to highlight them simultaneously.

  4. Click Compute! to open the numerical summary window.

• Empirical Application Outputs:

  • Exam Scores (Example 6):

    • Data: Table 1 student grades ($n = 7$).

    • StatCrunch result: $r = 0.81457979 \approx 0.815$.

    • Conclusion: Strong positive linear correlation.

  • Fast Food Nutrition: Fat vs. Protein (Example 7):

    • Data: Table 2 nutritional profiles ($n = 16$ meals):

*   StatCrunch result: $r = 0.84807168 \approx 0.848$.
*   *Conclusion:* Strong positive linear correlation.

• Vehicle Weight vs. Fuel Economy (Example 8):

  • Data: Table 3 automotive measurements ($n = 15$ vehicles):

*   StatCrunch result: $r = -0.88435876 \approx -0.884$.
*   *Conclusion:* Strong negative linear correlation.

• Resting Heart Rate vs. HDL Cholesterol (Example 9):

  • Data: Table 4 clinic observations ($n = 20$ patients):

• StatCrunch result: $r = 0.040196208 \approx 0.040$.

• Conclusion: No linear correlation ($r$ is near zero).

Testing Statistical Significance of Correlation

• Formal Definition:

  • A linear correlation coefficient is defined as statistically significant when sufficient sample evidence exists to conclude that the true population correlation reflects a linear trend, confirming the data points follow a genuine linear pattern rather than random sampling fluctuations.

• Dependence on Sample Size ($n$):

  • The evaluation of correlation depends heavily on sample size. A moderate correlation value (e.g., $r = 0.72$) provides strong evidence of linearity in large samples ($n = 45$), but weak evidence in very small samples ($n = 7$), where large random deviations occur more frequently.

• Analogy to Diagnostic Medical Testing:

  • Statistical significance indicates only the presence of a detectable linear relationship, not its strength. This functions like a COVID antigen test: a positive result confirms that viral antigen is present, but provides no measurement of disease severity. Significance proves a linear trend exists, but does not dictate whether that trend is weak, moderate, or strong.

• Five-Step Significance Testing Procedure:

  1. Calculate or obtain the sample linear correlation coefficient ($r$).

  2. Determine the sample size ($n$), defined as the total number of paired observations.

  3. Reference the Critical Values Table for Correlation to locate the critical threshold value ($CV$) corresponding to row $n$.

  4. Compute the absolute value of the correlation coefficient: $|r|$.

  5. Execute the decision rule:

     • If $|r| > CV$, the correlation is statistically significant (sufficient evidence of a linear pattern).

     • If $|r| \le CV$, the correlation is not statistically significant (insufficient evidence of a linear pattern).

• Critical Values Reference Table for Linear Correlation:

• Convention for Samples Exceeding Table Limits: For sample sizes where $n > 50$, the standard convention utilizes the critical value for $n = 50$ ($CV = 0.279$) as a conservative decision threshold.

• Evaluating Significance Across Sample Sizes (Example 10):

  • (a) Given $r = 0.647$, $n = 35$:

    • $CV = 0.334$

    • Condition: $|0.647| = 0.647 > 0.334$

    • Decision: The correlation is statistically significant.

  • (b) Given $r = -0.412$, $n = 16$:

    • $CV = 0.497$

    • Condition: $|-0.412| = 0.412 < 0.497$

    • Decision: The correlation is not statistically significant.

  • (c) Given $r = -0.581$, $n = 28$:

    • $CV = 0.374$

    • Condition: $|-0.581| = 0.581 > 0.374$

    • Decision: The correlation is statistically significant.

  • (d) Given $r = 0.871$, $n = 5$:

    • $CV = 0.878$

    • Condition: $|0.871| = 0.871 < 0.878$

    • Decision: The correlation is not statistically significant. Despite an apparently large correlation ($r = 0.871$), the sample size of five is too small to provide sufficient statistical certainty.

  • (e) Given $r = 0.512$, $n = 51$:

    • Because $n > 50$, use the conservative threshold $CV = 0.279$.

    • Condition: $|0.512| = 0.512 > 0.279$

    • Decision: The correlation is statistically significant.

• Synthesized Bivariate Analysis Evaluations:

  • Total Fat vs. Protein in Fast Food (Example 12):

    • Sample size: $n = 16$

    • Observed correlation: $r \approx 0.848$

    • Table threshold: $CV = 0.497$

    • Test: $|r| = 0.848 > 0.497$

    • Conclusion: Statistically significant. The data exhibit a moderate-to-strong positive linear relationship.

  • Curb Weight vs. Gas Mileage (Example 13):

    • Sample size: $n = 15$

    • Observed correlation: $r \approx -0.884$

    • Table threshold: $CV = 0.514$

    • Test: $|r| = 0.884 > 0.514$

    • Conclusion: Statistically significant. The data exhibit a strong negative linear relationship.

  • Heart Rate vs. HDL Cholesterol (Example 14):

    • Sample size: $n = 20$

    • Observed correlation: $r \approx 0.040$

    • Table threshold: $CV = 0.444$

    • Test: $|r| = 0.040 < 0.444$

    • Conclusion: Not statistically significant. There is no evidence of a linear relationship.

Correlation Versus Causation and Spurious Relationships

• The Non-Causality Principle:

  • Correlation demonstrates co-variation and predictive association, but does not prove cause-and-effect. A strong statistical correlation between variable $x$ and variable $y$ does not establish that changes in $x$ directly cause changes in $y$.

• Lurking and Confounding Variables:

  • An unmeasured underlying factor—known as a lurking variable—can influence both variables simultaneously, creating a strong statistical correlation between them even though no direct causal mechanism connects the two.

• Case Demonstration: Macroeconomic Trends (1990–2020):

  • A bivariate analysis of historical unemployment rates for Black and White Americans between 1990 and 2020 reveals a strong positive linear correlation ($r \approx 0.919$).

  • Causal Fallacy: Changes in one group's unemployment rate do not directly cause changes in the other's.

  • Lurking Variable Explanation: Broader economic forces—such as business cycle recessions, inflation shocks, monetary policy shifts, and global financial crises—simultaneously drive unemployment up or down across both demographic groups.

• Spurious Correlations:

  • A spurious correlation is a high mathematical correlation between two fundamentally unrelated variables, arising entirely through coincidence, shared temporal trends, or unlinked lurking variables.

  • Empirical Example: Cheese Consumption vs. Bedsheet Tangling Deaths:

    • Data tracked by Tyler Vigen using records from the U.S. Department of Agriculture (USDA) and the Centers for Disease Control and Prevention (CDC) across 2000–2009.

    • Variables: Per capita cheese consumption versus deaths caused by becoming tangled in bedsheets.

    • Observed correlation: $r \approx 0.947$

    • Interpretation: High per capita cheese consumption does not cause bedsheet strangulation, nor do bedsheet accidents promote cheese consumption. Both variables drifted upward concurrently over the decade due to population growth, changes in reporting standards, or pure mathematical coincidence.

• Additional Documented Spurious Trajectory:

  • A parallel study tracked by the National Vital Statistics Reports and the CDC demonstrates an empirical correlation of $r \approx 0.952$ between unrelated societal indicators, reinforcing that strong statistical metrics alone never establish physical causality.