Bivariate Numeric Data, Scatter Plots, Correlation, and Determination

Fundamentals of Bivariate Numeric Data and Linear Regression

• Linear regression focuses on bivariate numeric data, which examines the statistical relationship between two continuous quantitative variables across a $1$-week unit.

• Bivariate numeric data differs from univariate quantitative data (such as single-variable histograms or box plots) and bivariate categorical data (such as two-way tables).

• Examples of paired bivariate numeric variables that share a logical relationship include:

  • Ice cream sales and outdoor temperature.

  • Shoe size and body height.

  • Academic grades and the number of hours spent studying.

• Pairing two variables together does not imply that a meaningful relationship exists between them.

  • Example of non-associated variables: The number of hours an individual sleeps and their shoe size should not be paired as bivariate data because neither variable influences the other in any capacity.

Variable Identification: Explanatory and Response Variables

• Prior mathematical frameworks (such as algebra or general science) classify variables as independent ($x$) and dependent ($y$).

• In statistics, variables are classified using specific terminology:

  • Response Variable: Represented by $y$. This is the outcome variable that is being predicted.

  • Explanatory Variable or Predictor Variable: Represented by $x$. This is the variable used to explain changes in or make predictions about the response variable.

• Terminology usage across platforms:

  • Pearson software most commonly uses the term predictor variable for $x$.

  • Standardized examinations (such as the AP Statistics exam) most commonly use the term explanatory variable for $x$.

  • Both terms are interchangeable for designating the $x$ variable.

• Determining variable roles:

  • Example (Sleep vs. Grades): Number of hours spent sleeping is the explanatory variable ($x$) used to predict academic grades ($y$), because sleep duration directly impacts grades, whereas grades do not dictate sleep duration.

  • Example (Real Estate Dataset): In a dataset containing house characteristics—such as presence of a fenced yard, acreage, and square footage—the primary goal is to predict the list price.

  • List price serves as the response variable ($y$).

  • House traits (e.g., square footage or acreage) serve as explanatory variables ($x$).

Graphical Display of Bivariate Data: Scatter Plots

• A scatter plot is the standard graphical representation for displaying bivariate numeric data.

• Scatter plots replace single-variable graphics such as histograms, box plots, and segmented bar graphs when analyzing two quantitative variables simultaneously.

• Scatter plots allow for the visual identification of overall trends, such as observing that as individual height increases, weight tends to increase as well.

• Incorporating a third categorical variable:

  • A third variable can be represented on a two-dimensional scatter plot using distinct symbols, markers, or colors for data points.

  • Example (Height, Weight, and Sex): Plotting height and weight while categorizing data points by sex (male vs. female) reveals two distinct clusters of data.

  • This categorization demonstrates that the rate of weight increase relative to height increase is less pronounced among females than among males.

Describing Scatter Plots: Form, Direction, and Strength

• Descriptive terminology restrictions:

  • Never use the terms "symmetric" or "skewed" when describing scatter plots.

  • Terms like "symmetric" and "skewed" are strictly reserved for univariate distributions displayed in histograms and box plots.

• Scatter plots are described using three specific components: Form, Direction, and Strength.

• Component $1$: Form

  • Evaluates the geometric pattern formed by the data points.

  • Categorized as Linear, Curvilinear (curved pattern, such as a shape resembling a "Nike swoosh"), or No Relationship (random point scatter, referred to as "chickenpox").

  • Scope constraint: Linear regression procedures apply exclusively to linear forms. If a scatter plot exhibits a curvilinear form or no relationship, evaluation stops, and direction and strength are not described.

• Component $2$: Direction

  • Evaluated only if the form is linear.

  • Positive Direction: Indicated by a positive, increasing slope from left to right.

  • Negative Direction: Indicated by a negative, decreasing slope from left to right.

• Component $3$: Strength

  • Evaluated only if the form is linear.

  • Measures how closely data points follow a straight line.

  • Strong: Points lie extremely close to a straight line.

  • Moderate: Points follow a linear trend but exhibit a wider spread around the line.

  • Weak: Points exhibit a vague linear trend that requires careful observation to discern.

Step-by-Step Procedure for Creating Scatter Plots in Excel

• Variable placement rule in Excel:

  • Excel automatically assigns the leftmost selected column as the explanatory variable ($x$) and the rightmost selected column as the response variable ($y$).

  • If the response variable ($y$) appears to the left of the explanatory variable ($x$) in a dataset (e.g., list price appearing in an early column), copy and paste the response variable column to the far right side of the explanatory variable column.

• Step-by-step selection procedure:

  • Adjacent columns: Click the header letter of the first column (e.g., column $D$ for SAT Total) and drag the cursor across to the adjacent column header letter (e.g., column $E$ for College GPA).

  • Non-adjacent columns: Click the header letter of the first column, hold the Command key (Mac) or Control key (PC), and click the header letter of the second column.

• Chart insertion steps:

  • Select the Chart menu.

  • Click the Scatter plot icon (represented by bubble-like point markers).

  • Select the basic unconnected Scatter Plot option.

• Chart editing and scaling verification:

  • Add axis titles to explicitly label the explanatory ($x$) and response ($y$) variables.

  • Verify scale appropriateness: Ensure the response variable scale appears on the vertical $y$-axis (e.g., College GPA scaled up to $4.0$) and the explanatory variable scale appears on the horizontal $x$-axis (e.g., SAT scores scaled between $800$ and $1600$).

The Correlation Coefficient ($r$)

• Definition: The correlation coefficient, denoted by lowercase $r$, provides an objective numeric measure of both the direction and strength of a linear relationship between two quantitative variables.

• Numerical Range: The value of $r$ strictly ranges from $-1$ to $1$ (i.e., $-1 \le r \le 1$).

• Direction interpretation:

  • Positive $r$ values indicate a positive linear slope.

  • Negative $r$ values indicate a negative linear slope.

• Strength interpretation benchmarks:

  • $r = 1$: Perfect positive linear relationship (all data points fall exactly on a straight line with a positive slope).

  • $r = -1$: Perfect negative linear relationship (all data points fall exactly on a straight line with a negative slope).

  • $r = 0.9$ or $r = -0.8$: Strong linear relationship (points cluster tightly along a linear path). Strong values generally range from $0.8$ to $1$ (or $-0.8$ to $-1$).

  • $r = 0.7$ or $r = 0.6$: Moderate linear relationship. Moderate values generally fall between $0.6$ and $0.8$ (or $-0.6$ and $-0.8$).

  • $r = 0.5$, $r = 0.4$, or $r = 0.3$: Weak linear relationship (points display significant dispersion around a linear path).

  • $r = 0.2$, $r = 0.1$, or $r = 0$: No linear relationship (points appear as a random cluster with no discernible linear pattern).

• Invariance under variable swapping:

  • Reversing or interchanging the $x$ and $y$ variables does not alter the value of the correlation coefficient $r$.

  • Standardized tests (such as the AP Statistics exam) frequently test the property that swapping explanatory and response variables leaves $r$ unchanged.

• Numerical Example (Homework Average vs. Exam Score):

  • Explanatory variable ($x$): Homework average.

  • Response variable ($y$): Exam score.

  • Visual inspection: Linear form, positive direction, moderate point spread.

  • Calculated correlation coefficient: $r \approx 0.767$.

  • Classification: Moderate positive linear relationship, as $r \approx 0.767$ falls below the $0.8$ threshold for strong linear relationships.

Correlation vs. Causation and Lurking Variables

• Fundamental Principle: Correlation does not equal causation. Observed association between two variables does not imply that changes in one variable cause changes in the other.

• Lurking Variable: An unobserved or unmeasured variable that influences the relationship between the explanatory and response variables, creating a false or misleading association.

• Real-world examples of strong correlations driven by lurking variables:

  • Example $1$: Maine Divorce Rate vs. U.S. Per Capita Margarine Consumption

  • Correlation value: $r = 0.992558$ (extremely strong positive linear correlation).

  • Analysis: Declining divorce rates in Maine do not cause decreased margarine consumption nationwide.

  • Potential lurking variable: Societal trends toward healthier living and natural foods (e.g., switching from margarine to butter) coinciding with broader social factors influencing marital stability.

  • Example $2$: Bedding Entanglement Deaths vs. Skiing Facility Revenue

  • Correlation value: $r \approx 0.97$ (extremely strong positive linear correlation).

  • Analysis: Deaths caused by bedsheet entanglement do not generate revenue for ski resorts.

  • Lurking variable: Overall population growth. As total population increases, total deaths from rare occurrences increase simultaneously with overall participation and spending at ski facilities.

The Coefficient of Determination ($R^2$)

• Definition: The coefficient of determination, denoted as uppercase $R^2$ (or lowercase $r^2$), measures the proportion of total variability in the response variable ($y$) that is explained by the linear relationship with the explanatory variable ($x$).

• Mathematical Relationship:

  • $R^2 = (r)^2$

  • $r = \sqrt{R^2}$ (where the sign of $r$ is determined by the direction of the slope).

• Value Range:

  • Because $R^2$ is a squared quantity, it cannot be negative.

  • Ranges from $0$ to $1$ in decimal form, or $0\%$ to $100\%$ in percentage form.

• Interpretation Guidelines:

  • Do not describe $R^2$ using subjective terms like "strong", "moderate", or "weak".

  • Describe $R^2$ explicitly as the percentage of variation in the response variable explained by the line of best fit (or explained by the explanatory variable).

• Numerical Calculation Example (Homework Average vs. Exam Score):

  • Given $r = 0.767$.

  • $R^2 = (0.767)^2 \approx 0.5884$ or $58.84\%$.

  • Formal Interpretation: $58.84\%$ of the variation in exam scores is described by the line of best fit (or explained by homework averages).

Excel Commands for Trendlines, $R^2$, and Correlation Coefficients

• Obtaining $R^2$ via Chart Trendline:

  1. Generate a scatter plot in Excel.

  2. Right-click or select the data series and select "Add Trendline".

  3. In the "Format Trendline" pane on the right side of the screen, ensure the regression model is set to "Linear".

  4. Check the box labeled "Display R-squared value on chart".

  5. Excel inserts a text box displaying the calculated $R^2$ value directly on the chart canvas.

• Calculating $r$ from $R^2$ in Excel:

  • Select an empty spreadsheet cell.

  • Use the square root command: =SQRT(R2_value) (e.g., =SQRT(0.1597)).

• Calculating Correlation Coefficient ($r$) directly using =CORREL:

  • Select an empty spreadsheet cell.

  • Enter the command: =CORREL(array1, array2), selecting the explanatory variable cell range and response variable cell range.

  • Advantage: =CORREL computes $r$ directly without requiring a scatter plot to be created first.

  • Order independence: Entering the response variable first and explanatory variable second produces the exact same numerical result.

• Critical Diagnostic Requirement:

  • Always construct and inspect the visual scatter plot prior to reporting or interpreting $r$ or R^2$.\n * Correlation coefficients and coefficients of determination strictly evaluate linear relationships; if the underlying data distribution is non-linear or curvilinear, r$and$R^2$provide misleading and invalid summaries.</p></li></ul></li></ul><p></p><h5 id="d7af92d7-d0ba-4306-a7bb-67e3b2efbbb4" data-toc-id="d7af92d7-d0ba-4306-a7bb-67e3b2efbbb4" collapsed="false" seolevelmigrated="true">Step 1: Variable Identification (Explanatory vs. Response)</h5><ul><li><p><strong>Identify Response Variable</strong> ($y$):</p><ul><li><p>Outcome variable being predicted (e.g., list price, exam score, college GPA).</p></li></ul></li><li><p><strong>Identify Explanatory or Predictor Variable</strong> ($x$):</p><ul><li><p>Variable used to explain changes in or predict$y$(e.g., square footage, homework average, sleep duration).</p></li></ul></li><li><p><strong>Logical Association Check</strong>:</p><ul><li><p>Confirm variables share a logical relationship prior to pairing (e.g., sleep duration and shoe size should not be paired as bivariate data).</p></li></ul></li></ul><h5 id="6949717e-74fb-4f2a-855a-ea12cbd5cef9" data-toc-id="6949717e-74fb-4f2a-855a-ea12cbd5cef9" collapsed="false" seolevelmigrated="true">Step 2: Excel Data Formatting and Preparation</h5><ol><li><p><strong>Arrange Data Columns</strong>:</p><ul><li><p>Position Column$x$(explanatory) to the left of Column$y$(response).</p></li><li><p>If Column$y$is to the left of Column$x$in raw data, copy and paste Column$y$to the right of Column$x$.\n2. **Select Data Ranges**:\n - **Adjacent columns**: Click the header letter of Column $x$ and drag the cursor across to Column y$.

  • Non-adjacent columns: Click the header letter of Column x$, hold <code>Command</code> (Mac) or <code>Control</code> (PC), and click the header letter of Column$y$.\n\n### Step 3: Excel Scatter Plot Creation Workflow\n\n1. **Insert Plot**:\n - Navigate to the **Insert** / **Chart** menu.\n - Click the **Scatter** plot icon.\n - Select the basic unconnected scatter plot.\n2. **Verify Scale and Axes**:\n - Confirm the horizontal axis displays explanatory variable $x$ values.\n - Confirm the vertical axis displays response variable $y$ values.\n3. **Label Axes**:\n - Add explicit axis titles for both variables.\n\n### Step 4: Scatter Plot Evaluation Procedure\n\n- **Step 4.1: Check Form**\n - **Linear**: Points follow a straight line pattern (proceed to evaluate direction and strength).\n - **Curvilinear**: Points follow a curved pattern (stop; linear regression procedures do not apply).\n - **No Relationship**: Random scatter ("chickenpox") (stop; linear procedures do not apply).\n- **Step 4.2: Evaluate Direction** (Linear Form Only)\n - **Positive**: Slope increases from left to right.\n - **Negative**: Slope decreases from left to right.\n- **Step 4.3: Evaluate Strength** (Linear Form Only)\n - **Strong**: Points cluster tightly along a line.\n - **Moderate**: Points follow a clear linear path with moderate spread.\n - **Weak**: Points show a vague linear trend with wide dispersion.\n\n### Step 5: Computing Correlation Coefficient ($r$) in Excel\n\n- **Direct Formula Method**:\n 1. Select an empty spreadsheet cell.\n 2. Enter `=CORREL(array1, array2)` where `array1` is the $x$ data range and `array2` is the $y$ data range.\n 3. Press Enter.\n- **Interpretation Benchmark Ranges** (-1 le r \le 1):\n - $r = 1$ or $r = -1$: Perfect linear relationship.\n - $0.8 \le |r| < 1$: Strong linear relationship.\n - $0.6 \le |r| < 0.8$: Moderate linear relationship.\n - $0.3 \le |r| < 0.6$: Weak linear relationship.\n - $|r| < 0.3$: No linear relationship.\n- **Invariance Property**:\n - Swapping $x$ and $y$ variables leaves $r$ completely unchanged.\n\n### Step 6: Displaying Trendline and $R^2$ in Excel\n\n1. Right-click any data point on the scatter plot canvas and select **Add Trendline**.\n2. In the options menu, verify the regression model is set to **Linear**.\n3. Check the box labeled **Display R-squared value on chart**.\n4. **Calculate $r$ from $R^2$ in Excel**:\n - Select an empty cell and enter `=SQRT(R2_value)` (e.g., `=SQRT(0.5884)`).\n - Assign a positive or negative sign to $r$ based on the slope direction of the scatter plot.\n\n### Step 7: Interpreting Coefficient of Determination ($R^2$)\n\n- **Mathematical Relation**:\n - $R^2 = (r)^2$\n - Ranges strictly from $0$ to $1$ ($0\%$ to $100\%$).\n- **Standard Interpretation Statement**:\n - "$R^2 \times 100\%$ of the variation in the response variable ($y$) is explained by the line of best fit (or explanatory variable $x$)."\n - **Example**: If $r = 0.767$, then $R^2 = (0.767)^2 = 0.5884$ ($58.84\%$). Interpretation: $58.84\%$ of the variation in exam scores ($y$) is explained by homework averages ($x$).\n\n### Step 8: Critical Analysis Rules\n\n- **Correlation vs. Causation**:\n - Strong correlation does not imply that changes in $x$ cause changes in $y$.\n- **Lurking Variables**:\n - Unobserved underlying factors (e.g., population growth or general social trends) can create misleading statistical associations.\n- **Visual Inspection Rule**:\n - Always inspect the scatter plot before reporting $r$ or $R^2$ to ensure non-linear data is not invalidly summarized.