Bivariate Data Analysis: Scatterplots, Correlation Coefficient, and Least-Squares Regression Models
Foundations of Bivariate Data and Variable Classification
Bivariate data analysis involves the simultaneous study of two quantitative variables to investigate patterns, associations, and relationships between paired observations.
Paired data structure:
Data points must always exist as linked ordered pairs .
Data lists must never be sorted independently from low to high, as sorting severs the pairing between individual observations and invalidates any subsequent statistical regression analysis.
Regression analysis cannot be performed on categorical variables; both variables under study must be strictly quantitative.
Explanatory variable ():
The variable that explains, affects, or accounts for changes in the response variable.
Known historically across mathematics as the independent variable, the input variable, or the domain variable.
Plotted exclusively along the horizontal axis (-axis).
In experimental designs, the explanatory variable represents the treatment or the condition actively manipulated by the experimenter (e.g., amount of time, applied pressure, fluid volume, or number of attachments).
Used as the basis for predicting the outcome of the response variable.
Response variable ():
The variable that measures the outcome of an experiment or observational study and depends on the changes of the explanatory variable.
Known historically across mathematics as the dependent variable, the output variable, or the range variable.
Plotted exclusively along the vertical axis (-axis).
Represents the variable being predicted by the statistical model.
Distinguishing variable roles:
When establishing a model, the investigator must explicitly assign the explanatory and response variables before running calculations.
In situations where neither variable naturally causes or predicts the other (e.g., paired skeletal dimensions such as femur length and humerus length), either variable may arbitrarily serve as the explanatory variable, provided that the assignment remains consistent throughout the analysis.
Constructing and Interpreting Scatterplots
Scatterplot construction principles:
Scatterplots serve as the primary graphical tool for displaying bivariate quantitative data.
The horizontal axis is designated for the explanatory variable (), and the vertical axis is designated for the response variable ().
Both axes must be labeled clearly with variable names, appropriate numerical scales, and units of measurement.
Axes are not required to originate at ; truncation of an axis is permissible to avoid compressing the data points or plotting them directly on top of the axes, provided the numerical intervals are scaled consistently.
The DUFS descriptive framework:
While single-variable distributions are described using SOCS (Shape, Outliers, Center, Spread), bivariate distributions displayed via scatterplots are evaluated using DUFS:
Direction:
Positive direction indicates that as the explanatory variable () increases, the response variable () also tends to increase.
Negative direction indicates that as the explanatory variable () increases, the response variable () tends to decrease.
No direction indicates a random, amorphous scattering of points showing no consistent upward or downward trend.
Unusual features:
Outliers: Individual data points that fall outside or deviate substantially from the overall pattern of the data. Outliers can occur in the horizontal direction (-direction), the vertical direction (-direction), or both.
Clusters: Distinct groupings of data points separated by spaces. Different clusters within the same dataset may exhibit conflicting internal trends (e.g., one cluster trending positively while another trends negatively).
Gaps: Noticeable intervals along the domain or range where no observations appear.
Form:
Refers to the underlying geometric shape traced by the data points.
Classifications include linear (tracing a straight-line pattern) and nonlinear.
Nonlinear patterns encompass curved structures such as quadratic/parabolic, exponential, power, logarithmic, polynomial, or logistic growth curves.
Strength:
Measures how tightly the plotted coordinates adhere to the identified form.
Described qualitatively using modifiers such as very strong, strong, moderately strong, fairly weak, or weak.
Determining strength requires examining the scale of the vertical () axis: closely bounded values on a small scale indicate tight adherence and less dispersion around the overall trend line.
Mandatory context requirement:
All written descriptions of scatterplots must incorporate the real-world context of both variables (identifying the units and attributes being measured) rather than relying exclusively on abstract statistical terminology.
Explicit prohibition on using "correlation" as a descriptive visual adjective:
The word "correlation" represents a specific numerical coefficient ().
The term "correlation" must never be utilized to describe the visual appearance of a scatterplot unless the exact numerical value of has been computed.
Appropriate alternative terminology includes "linear relationship" or "linear association".
Illustrative case studies:
Barbie bungee jump:
Context: Dropping a doll secured with varying counts of rubber bands to measure descent distance.
Setup: Doll's feet attached to rubber bands; the baseline rubber band around the ankles represents rubber band . Two yardsticks stacked end-to-end measure vertical drop.
Explanatory variable: Number of rubber bands (manipulated discrete integer values: ).
Response variable: Distance traveled by the doll's head in centimeters.
Scatterplot characteristics: Strong, positive, linear association between the count of rubber bands and the total distance traveled, with no outliers in either the horizontal or vertical directions.
Star Wars characters screen time vs. popularity:
Explanatory variable: Screen time exposure in minutes.
Response variable: Character popularity rank ( to ).
Scatterplot characteristics: Weak, negative linear trend. Overall, as screen time increases, rank value decreases (which numerically represents a higher rank, though visually sloping downward).
Unusual features: Darth Vader represents a prominent outlier, holding a rank of despite having very limited screen time. The distribution displays two distinct clusters separated by a central gap; the lower cluster displays a positive trend, whereas the upper cluster trends negatively.
Ninth-grade student body weight vs. backpack weight:
Explanatory variable: Body weight in pounds.
Response variable: Backpack weight in pounds during a hiking field trip.
Scatterplot characteristics: Positive, strong linear association; generally, students with lower body weights carry lighter packs.
Unusual feature: A student weighing approximately carries a pack significantly lighter than predicted by the linear model, creating an outlier that deviates noticeably from the dominant trend.
State SAT participation rate vs. mean SAT math score:
Explanatory variable: Percentage of high school students taking the SAT ( state data points).
Response variable: State mean math SAT score.
Scatterplot characteristics: Fairly strong, negative, nonlinear association; states with higher test-taking percentages exhibit lower mean math scores.
Cluster structure: Displays two separate clusters. The upper, more compact cluster consists of states where the SAT is voluntary (taken primarily by self-selected, college-bound students seeking competitive admissions), producing higher scores along a tight linear path. The lower cluster consists of states where SAT testing is universally mandated for graduation, generating lower, more uniform average scores across broader populations.
The Correlation Coefficient ()
Definition and purpose:
The correlation coefficient, denoted by , is a standardized numerical statistic that quantifies the direction and strength of the linear association between two quantitative variables.
Mathematical bounds and interpretation:
The value of is strictly bounded: .
Direction:
A positive value () indicates a positive linear association ( increases as increases).
A negative value () indicates a negative linear association ( decreases as increases).
Strength:
Values of or indicate a perfect linear relationship where all observed points lie exactly upon a single straight line.
Values near indicate an extremely weak linear association or an absence of any linear pattern.
General benchmarks:
Absolute values exceeding (e.g., ) indicate strong or very strong linear relationships.
Absolute values in the vicinity of (e.g., to ) reflect moderate to fairly strong linear associations.
Absolute values below indicate weak linear associations.
Algebraic definition and standard score derivation:
The formal calculation for the sample correlation coefficient is defined as:
Equivalent formulation using standardized scores (-scores):
Here, denotes the standardized -score of the explanatory coordinate, and denotes the standardized -score of the response coordinate.
Conceptual meaning: The correlation coefficient represents the average product of the standardized horizontal and vertical deviations from the mean centroid .
Geometric interpretation: Every data point forms a rectangle whose dimensions are its respective horizontal and vertical -scores relative to the intersection lines and . The statistic is the sum of these signed rectangular areas scaled across degrees of freedom.
Statistical properties of :
Non-resistance to outliers: Because the calculation of relies directly on sample means () and sample standard deviations ()—none of which are resistant to extreme values—the correlation coefficient is highly sensitive to outliers.
Influence of added points:
Adding a coordinate that falls directly in line with the existing linear pattern maintains or strengthens the value of .
Adding a coordinate that deviates substantially from the linear pattern (such as an erroneous entry of instead of ) degrades the tightness of the association and decreases the magnitude of .
Variable ordering: Because multiplication is commutative (), switching the explanatory and response variable assignments does not change the numerical value of . However, contextual interpretation requires precise variable assignment.
Unit invariance: Standardizing data points into dimensionless -scores renders completely independent of the units of measurement chosen for either or .
Correlation versus causation:
A high correlation coefficient demonstrates association, not causality.
Even an exceptionally high linear correlation can arise purely from coincidence or confounding variables.
Example: The length of winning words in the National Spelling Bee exhibits a strong positive correlation (, ) with the annual number of deaths caused by venomous spider bites. This statistical association is coincidental and cannot establish that word length influences arachnid mortality.
Causation can only be established through a rigorously controlled, randomized comparative experiment.
The Least Squares Regression Line (LSRL)
Definition and objective:
While infinitely many arbitrary lines of fit can be drawn through a scatterplot, there is exactly one unique Least Squares Regression Line (LSRL), also called the line of best fit or the equation.
The LSRL mathematical equation is defined as:
Here, represents the predicted value of the response variable for a given value of , denotes the slope of the regression line, and denotes the -intercept.
The caret symbol () over the variable is mandatory to distinguish modeled, predicted outputs from observed data points ().
Fundamental properties of the LSRL:
Least-squares criterion: The line minimizes the sum of the squared vertical distances (squared residuals) from each observed data point to the line:
Centroid property: The LSRL always passes directly through the point of averages (centroid), defined by coordinates .
Mathematical derivation of slope and intercept:
Slope ():
Conceptual derivation: The ratio represents the baseline rate of vertical variation to horizontal variation (). If or , the slope equals this ratio directly. For any real-world dataset where , the slope is scaled by the correlation coefficient , pulling the line toward the horizontal axis relative to standard deviations.
Intercept ():
Because the regression line must pass through , substituting these coordinates into the point-slope form gives:
Solving for yields:
Contextual interpretation of regression parameters:
Slope ():
Standard sentence structure: "On average, for each additional unit increase in [explanatory variable ], the [response variable ] is predicted to increase/decrease by units."
The phrase "on average" or "predicted" is strictly required, as the slope describes a modeled rate of change rather than an absolute deterministic rule.
Intercept ():
Standard sentence structure: "When the [explanatory variable ] is equal to zero units, the predicted value of the [response variable ] is units."
Meaningful application: In certain contexts, the intercept has concrete physical meaning (e.g., in a model of rat growth, represents the average birth weight at ; for vehicle valuation, reflects the baseline price of a brand-new vehicle at ).
Non-physical application: In other contexts, the intercept serves only as a mathematical anchor to establish the slope across the observed domain. For instance, in modeling fossilized dinosaur bones where femur length predicts humerus length, a -intercept of at a femur length of has no biological reality, as negative limb lengths cannot exist.
Evaluating Regression Models: Predictions, Extrapolation, and Residual Analysis
Generating predictions:
Predicted response values are computed by substituting a designated value of directly into the regression model: .
Example: In the Ford F-150 pricing model (where is odometer mileage and is resale price in dollars):
At :
The danger of extrapolation:
Extrapolation is the practice of using a regression model to make predictions for values of the explanatory variable that lie far beyond the domain of the data used to construct the model.
Extrapolated predictions are notoriously unreliable, unrealistic, and mathematically invalid because the established linear relationship cannot be assumed to hold outside the observed interval.
Example 1 (Automotive depreciation):
Evaluating the Ford F-150 model at yields:
A negative market value implies the owner must pay a buyer to haul the vehicle away, which is commercially absurd. The vehicle's scrap materials maintain positive value, showing that depreciation slows down nonlinearly at higher mileages.
Example 2 (Biological growth model):
A laboratory rat study models weight over the first of life as ( in weeks, in grams).
Evaluating the model at an age of () yields:
Converting units ():
A lab rat weighing nearly is biologically impossible. Organism growth follows a logistic growth curve that levels off horizontally at adulthood rather than continuing along an unbounded linear path.
Residual calculation and interpretation:
A residual measures the vertical prediction error between an observed coordinate and the corresponding point on the regression line.
Formal formula:
Interpretation of residual signs:
Positive residual ():
The actual data point lies vertically above the regression line.
The regression model underestimates the true observed outcome.
Negative residual ():
The actual data point lies vertically below the regression line.
The regression model overestimates the true observed outcome.
Numerical examples:
Ford F-150: Observed truck with and actual resale price .
Predicted price:
Residual calculation:
Interpretation: The linear model overestimates the resale price of this specific vehicle by (the vehicle sells for less than predicted).
Dinosaur fossil: Observed pair with femur length and humerus length .
Model parameters:
Predicted humerus length:
Residual calculation:
Interpretation: The model underestimates the humerus length by (the observed bone is longer than predicted).
Residual plots:
A residual plot is a diagnostic scatterplot that displays the explanatory variable () along the horizontal axis and the calculated residuals () along the vertical axis.
The horizontal axis at corresponds directly to the LSRL.
Conservation property of the LSRL:
By mathematical construction, the algebraic sum of all residuals produced by the least-squares regression line is always identically zero:
Diagnostic criteria for assessing model fit:
Appropriate linear fit:
The residual plot displays an unstructured, uniform, random scatter of points distributed symmetrically above and below the line .
No distinct curves, trends, or systematic variations in point spread should appear.
Inappropriate linear fit (curved pattern):
If the residual plot exhibits a distinct curved or parabolic shape, a linear model is structurally inappropriate for the data.
Even if the correlation coefficient is high (e.g., or ), a curved residual plot confirms that the underlying relationship is nonlinear.
Inappropriate linear fit (fanning or heteroscedasticity):
If the residual plot fans outward (points spread wider along the vertical axis as increases), predictions become systematically less precise and less reliable for higher values of the explanatory variable.
The Coefficient of Determination () and Partitioning of Variation
Definition and operational interpretation:
The coefficient of determination, denoted by , represents the proportion or percentage of the total variation in the response variable () that is explained by the linear relationship with the explanatory variable ().
Standard sentence frame:
" of the variation in [response variable ] can be explained by the linear relationship with [explanatory variable ]."
Example interpretations:
Barbie bungee drop (): of the variation in the distance traveled by the doll can be explained by the linear relationship with the number of attached rubber bands.
Dinosaur bones (): of the variation in humerus bone length can be explained by the linear relationship with femur bone length.
National Spelling Bee vs. spider bites (): of the variation in spider bite fatalities can be mathematically accounted for by the linear model with the number of letters in winning spelling bee words (though this is entirely non-causal).
Poor interpretation phrasing to avoid: Stating that " of the variation is explained by the data" is mathematically incomplete. The phrasing must explicitly specify that variation is explained by the linear relationship with the changes in the explanatory variable .
Mathematical derivation and partitioning of variation:
Total Sum of Squares ():
Measures the total dispersion of the observed response values () around their unconditional mean ():
Represents the baseline error present when predicting using only the horizontal mean line without considering the explanatory variable .
Sum of Squared Residuals ( or ):
Measures the remaining unexplained vertical variation after fitting the regression line:
Because the regression line tilts through the centroid to follow the trend, the squared residual errors from the line () are smaller than the deviations from the mean line ().
Ratio of unexplained variation:
The fraction represents the proportion of total variation in that cannot be accounted for by the linear model.
Complementary relationship for :
Subtracting the unexplained fraction from the total baseline variation gives the proportion of explained variation:
As the regression line approaches the data points, approaches , driving toward and bringing closer to .
Step-by-Step Computational Procedures and Technology Guide
Manual calculation of using standard scores (-scores):
Step 1: Compute the summary statistics for both variables: sample means () and sample standard deviations ().
Step 2: In a calculation table, list paired columns for and .
Step 3: Compute the standardized score for each horizontal observation: .
Step 4: Compute the standardized score for each vertical observation: .
Step 5: Calculate the product of the standardized pairs for each row: .
Step 6: Sum all calculated products across the dataset: .
Step 7: Divide the total sum by the degrees of freedom () to obtain :
Manual derivation of the LSRL from summary statistics:
Given: , and .
Step 1: Calculate the slope :
Step 2: Calculate the intercept using the centroid :
Step 3: Write the final regression model using formal prediction notation:
Graphing calculator setup (TI-84 series):
Enabling statistical diagnostics (required once per calculator reset):
Modern OS: Press
MODE, navigate down toSTAT DIAGNOSTICS, set toON. EnsureSTAT WIZARDSis set toON.Legacy OS: Press
2ndthen0to open theCATALOG. PressD(or scroll downward) to findDiagnosticOn. PressENTERtwice until the screen displaysDone. If diagnostics remain inactive, the calculator will output slope and intercept without computing or .Entering bivariate data:
Press
STAT, select1:Edit....Enter explanatory observations () into list
L1.Enter response observations () into list
L2.Ensure ordered pairs correspond correctly row-by-row; never sort either list.
Computing the regression model:
Press
STAT, move the cursor right toCALC.Select
8:LinReg(a+bx)(avoid option4:LinReg(ax+b)to maintain standard statistical convention).Specify parameters:
Xlist: L1,Ylist: L2.To automatically store the regression line into graphing memory:
Method A: Navigate to
Store RegEQ:, pressVARS, scroll toY-VARS, select1:Function, then select1:Y1.Method B: Press
ALPHA, pressF4(orTRACE), and selectY1.
Move cursor to
Calculateand pressENTER. The output provides the intercept , slope , coefficient of determination , and correlation coefficient .Generating a scatterplot with the fitted regression line:
Press
2ndthenY=(STAT PLOT).Select
Plot1, toggle toOn.Choose the first icon under
Type(scatterplot).Set
Xlist: L1andYlist: L2.Press
ZOOM, select9:ZoomStat.The screen will display the scatterplot alongside the regression line stored in
Y1.Generating a diagnostic residual plot:
After running
LinReg(a+bx), the calculator automatically writes the residuals into a dedicated system list namedRESID.To view residuals in the table: Press
STAT, select1:Edit..., move the cursor to the header of empty listL3, pressENTER, press2ndthenSTAT(LIST), selectRESID, and pressENTER.Plotting the residuals:
Press
2ndthenY=(STAT PLOT).Ensure
Plot1(or the equation inY=) is turned off to prevent the original line from appearing.Select
Plot2, toggle toOn, choose the scatterplot icon.Set
Xlist: L1andYlist: L3(or direct selection ofRESID).Press
ZOOM, select9:ZoomStat.Examine the resulting plot relative to the horizontal axis () for signs of curvature or fanning.
Questions & Discussion
Q: What is an explanatory variable?
Response: An explanatory variable explains or accounts for the changes observed in the response variable. It represents the quantity manipulated by the investigator (such as time, pressure, or volume) to observe changes in an experimental outcome.
Q: In the initial snack preference warmup (where the -axis represents Person A and the -axis represents Person B across identical snack scales), what would the scatterplot look like if two individuals share similar snack preferences?
Response: The points trace an upward, positive linear pattern across the coordinate space. However, the points are not connected by continuous line segments because preferences represent discrete ordered entities.
Q: In the same warmup, what would the scatterplot look like if two individuals have opposite snack preferences, or if they share no common interests at all?
Response: If preferences are completely opposite (e.g., Person A ranks Cheez-It lowest while Person B ranks Funyuns highest), the points trace a negative linear downward pattern. If the two individuals share no common interests, the points scatter randomly without a discernible linear path.
Q: In the Barbie bungee experiment, what are the explanatory and response variables?
Response: The explanatory variable is the number of rubber bands tied to the doll's feet (the input quantity manipulated by adding bands from to ). The response variable is the total distance traveled by the doll's head (the resulting output measured in centimeters).
Q: How should the initial Barbie bungee scatterplot be described?
Response: There is a strong, positive, linear relationship between the number of rubber bands tied to the doll and the distance traveled. The term "correlation" must not be used in the visual description unless its numerical value is calculated. There are no apparent outliers in either the horizontal or vertical directions.
Q: What happens if an additional point is recorded for rubber bands that aligns closely with the existing pattern?
Response: Because the point conforms to the existing linear trend without deviating, the value of the correlation coefficient remains essentially unchanged or slightly strengthens.
Q: What happens if a data value is mistyped as instead of ?
Response: The mistyped point becomes an artificial outlier that deviates substantially from the overall linear trend, increasing dispersion around the line and reducing the magnitude of the correlation coefficient .
Q: In the Star Wars study, which variable is explanatory and which is response?
Response: The explanatory variable is the screen time exposure in minutes, because the research question investigates whether exposure time can predict a character's popularity rank (which serves as the response variable).
Q: Does the Star Wars screen time vs. popularity scatterplot exhibit any direction, form, or unusual points?
Response: The plot shows a weak, negative linear trend. Darth Vader represents an unusual point (ranking at number despite minimal screen time). The distribution forms two distinct clusters separated by a gap; within these clusters, the lower group shows a positive slope while the upper group displays a negative slope.
Q: In the fossil dinosaur bones dataset, which bone length is the explanatory variable and which is the response variable?
Response: Neither variable naturally causes or predicts the other. In the absence of an explicit experimental setup, either bone length may serve as the explanatory variable. For the class analysis, femur length is designated as the explanatory variable () and humerus length as the response variable ().
Q: How is the linear relationship between femur length and humerus length described?
Response: There is a strong, positive, linear relationship between femur length and humerus length, with longer femur bones corresponding to longer humerus bones. The points cluster tightly along a line with no unusual outliers.
Q: What is the physical meaning of the regression slope in the Ford F-150 pricing model?
Response: On average, the resale price of a Ford F-150 truck decreases by approximately (or about ) for each additional mile driven.
Q: What does the -intercept represent in the Ford F-150 model?
Response: The -intercept represents the modeled baseline price () for a brand-new truck with zero miles driven.
Q: Why does predicting the price of a Ford F-150 with produce a negative value ()?
Response: The calculation represents an extrapolation far beyond the domain of the sample data ( to approximately ). A linear model inevitably crosses the horizontal axis at zero dollars, whereas actual vehicle values level off nonlinearly due to scrap metal and parts value.
Q: In the lab rat growth model (), what do the parameters represent and why does predicting weight at fail?
Response: The intercept indicates that an average newborn lab rat weighs . The slope indicates that rats gain an average of per week during their first . Predicting at () produces an impossible weight of (about ) because animal growth tapers off into a logistic plateau rather than continuing along a linear trajectory indefinitely.
Q: In the dinosaur bone regression model, what does the -intercept of signify?
Response: The intercept has no realistic biological application because a dinosaur cannot have a negative humerus length when its femur length is zero. It functions solely as a mathematical anchor to position the regression line correctly across the observed data interval.
Q: How is a residual calculated, and what does its sign indicate?
Response: A residual is calculated as observed response minus predicted response: . A positive residual indicates that the observed point lies above the line (the model underestimated the value). A negative residual indicates that the observed point lies below the line (the model overestimated the value).
Q: What ideal visual pattern should appear in a diagnostic residual plot to confirm that a linear model fits well?
Response: The residual plot should show a random, uniform scatter of points centered around the reference line , with consistent vertical spread across the domain and no evidence of curvature, parabolic arcs, or fanning.