Week 11 - Understanding Correlation and Regression for Business Decisions in Business

Introduction and Learning Objectives

• Presenter: Petar Zivkovic
• Date: 02.06.2026
• Theme: Understanding relationships in data to make better business decisions using correlation and regression.
• Agenda:
  • Key Concepts: Variables and Relationships.
  • Measuring Relationships: Correlation and Covariance.
  • Building regression models to quantify effects.
  • Evaluating model quality: $R^2$, residuals, and assumptions.
  • Real-world applications: Business examples.
• Core Learning Objectives:
  • Identify and interpret relationships between variables within a business context.
  • Develop and apply regression models to explain and predict specific outcomes.
  • Assess the reliability and quality of regression models to support data-driven decision-making processes.

Bivariate Analysis

• Definition: Bivariate analysis is the study of the relationship between exactly two variables.
• Primary Goal: To explore if and how two variables are related, associated, or influence one another.
• Applications in Hospitality:
  • Analyzing guest age and the likelihood to book a spa treatment.
  • Analyzing room type and its effect on satisfaction ratings.
  • Analyzing guest nationality and breakfast preference.
  • Analyzing staff experience levels and the rate of guest complaints.
• Relationship to Multivariate Analysis: Bivariate analysis serves as a foundational step. Multivariate analysis moves further by examining how multiple variables interact simultaneously (e.g., how age, nationality, and income level together influence booking behavior).

Key Roles of Variables

• Independent Variable ($X$):
  • Alternative Names: Predictor variable, Explanatory variable, Input, Feature.
  • Purpose: The variable believed to influence or explain changes in another variable.
  • Hospitality Example: Room rate.
• Dependent Variable ($Y$):
  • Alternative Names: Response variable, Outcome variable, Output, Target.
  • Purpose: The variable the researcher is attempting to understand, predict, or explain.
  • Hospitality Example: Customer satisfaction.
• Analysis Comparison:
  • Bivariate Analysis: One independent variable ($X$) and one dependent variable ($Y$). Example: $Y = \text{Customer satisfaction}$, $X = \text{Room rate}$.
  • Multivariate Analysis: One dependent variable ($Y$) and two or more independent variables ($X_1, X_2, \dots, X_n$). Example: $Y = \text{Customer satisfaction}$, $X_1 = \text{Room rate}$, $X_2 = \text{Staff friendliness}$, $X_3 = \text{Room cleanliness}$.
• Assignment of Roles:
  • Prediction: $Y$ is the target to be predicted; $X$ is the input used for prediction.
  • Causal Analysis: $X$ is explicitly assumed to affect $Y$ (e.g., more amenities leads to higher satisfaction).
  • Exploration: Roles are not always strict; tools like pairplots or correlation matrices may treat all variables equally.

Statistical Indicators for Measuring Relationships

• Covariance:
  • Formula: $\text{Cov}(x,y) = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})$
  • Excel Function: COVARIANCE.S(range1, range2)
  • Usage: Measures how two variables change together. Positive values indicate they move in the same direction; negative values indicate they move in opposite directions.
• Correlation ($r$):
  • Formula: $r = \frac{\text{Cov}(x,y)}{s_x \times s_y}$
  • Excel Function: CORREL(range1, range2)
  • Usage: Standardizes covariance to a scale of $-1$ to $1$. It shows the strength and direction of a linear relationship.
• Interpreting Correlation:
  • Range: $-1$ to $1$. Positive means same direction; negative means opposite direction; near $0$ means weak or no linear relationship.
  • Linearity: Correlation only measures linear relationships. Data must be visualized with scatterplots to confirm linearity.
  • Outliers: Unusual data points can distort correlation values, necessitating investigation through scatterplots.
  • Causation: Correlation does not imply causation.
  • Business Uses: Standardized metrics allow for comparing relationship strengths across different variables, help prioritize variables for predictive modeling, and reveal patterns for business insights.

Qualitative Mechanics of Correlation and Covariance

• Deviation Interaction:
  • Positive deviation from $x$ and positive deviation from $y$ results in a positive ($+$) overall effect.
  • Negative deviation from $x$ and negative deviation from $y$ results in a positive ($+$) overall effect.
  • Positive deviation from $x$ and negative deviation from $y$ results in a negative ($-$) overall effect.
  • Negative deviation from $x$ and positive deviation from $y$ results in a negative ($-$) overall effect.
• Strength Identification: Correlation is used specifically to determine if a relationship is strong or weak, which covariance alone cannot signify because covariance is not standardized.

Causation vs. Coincidence

• The Principle: Correlation is not causation.
• Example Case: A graph showing a high correlation between "Divorce rate in Maine" and "Per capita consumption of margarine (US)".
  • Conclusion: While variations in divorce rate and margarine consumption appear synchronized, there is no causal link; the relationship is a pure coincidence.

Regression Analysis Objectives

• Test and Quantify Relationships:
  • Existence: Use significance tests like p-values and t-tests to determine if an $X-Y$ relationship exists.
  • Impact: Use regression coefficients ($\beta$) to interpret the amount $Y$ changes when $X$ changes.
• Evaluate Model Quality:
  • Fit: Evaluate model fit using $R^2$ or adjusted $R^2$.
  • Residuals: Inspect residuals for patterns.
  • Reliability: Check for outliers and verify regression assumptions.
• Predict Outcomes:
  • Forecasting: Use the regression equation $\hat{y} = b_0 + b_1 x$ to estimate the value of $Y$ based on known inputs ($X$).

The Linear Regression Model and Equation

• Conceptual Model: $y = \beta_0 + \beta_1 x + \nu$
• Estimated Regression Equation: $\hat{y} = b_0 + b_1 x$
• Least Squares Criterion: This method finds values for $b_0$ and $b_1$ that minimize the sum of squared residuals: $\sum (y_i - \hat{y}_i)^2$.
• Calculations:
  • $b_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$
  • $b_0 = \bar{y} - b_1 \bar{x}$
• Visualization of the Equation:
  • Slope ($b_1$): $\Delta \hat{y} / \Delta x$. Represents the change in the predicted value of $Y$ for every one-unit increase in $X$.
  • Intercept ($b_0$): The value of $Y$ when $X = 0$.

Business Examples of Regression

• Battery Degradation:
  • Model: $\text{Battery Range (miles)} = \beta_0 + \beta_1 \times \text{Mileage} + \nu$
  • Interpretation: If $b_1 = -0.02$ and p-value = $0.01$, the result is significant. Battery range decreases as mileage increases.
• Hotel Room Price:
  • Model: \text{Room Price ($)} = \beta_0 + \beta_1 \times \text{Distance from beach (km)} + \nu
  • Interpretation: If $b_1 = -5$ and p-value = $0.20$, the result is not significant (Accept $H_0$). Distance from the beach does not significantly affect room price in this model.
• House Price Prediction:
  • Model: \text{House Price ($)} = \beta_0 + \beta_1 \times \text{Square Footage} + \nu
  • Interpretation: If $b_1 = 150$ and p-value = $0.005$, the result is significant. House price increases significantly by $\$150$ for every additional square foot.

Types of Regression Relationships

• Positive Linear Relationship: Upward straight line ($H_1$ holds).
• Negative Linear Relationship: Downward straight line ($H_1$ holds).
• Positive Curvilinear Relationship: Upward curve.
• Negative Curvilinear Relationship: Downward curve.
• U-shaped Curvilinear Relationship: A parabolic curve.
• No Relationship: Flat line or scattered data points with no trend ($H_0$ holds).

Hypothesis Testing and the t-Test

• Null Hypothesis ($H_0$): $\beta_1 = 0$. Suggests there is no slope and therefore no relationship between the variables.
• Alternative Hypothesis ($H_1$): $\beta_1 \neq 0$. Suggests a significant slope exists, indicating a relationship.
• Test Statistic: $t = \frac{b_1}{s_{b_1}}$
• Decision Rule: Accept $H_0$ (proving no relationship) if the p-value is greater than the Significance Level (SL). Reject $H_0$ (confirming a relationship) if the p-value is less than or equal to the SL.

ANOVA and the Coefficient of Determination ($R^2$)

• $R^2$ Definition: Measures the proportion of variation in the dependent variable that is explained by the independent variable in the model.
• Formula: $R^2 = \frac{\text{Variation explained by regression}}{\text{Total variation in } Y} = \frac{SSR}{SST} = \frac{\sum (\hat{y}_i - \bar{y})^2}{\sum (y_i - \bar{y})^2}$
• Variables:
  • $y_i$: The actual measured value.
  • $\hat{y}_i$: The predicted value from the regression line.
  • $\bar{y}$: The average value of all actual measured points.
• Sum of Squares Components:
  • Total Sum of Squares (SST): $\sum (y_i - \bar{y})^2$. Represents the total variation in the data.
  • Error Sum of Squares (SSE): $\sum (y_i - \hat{y}_i)^2$. Represents variation not explained by the model.
  • Regression Sum of Squares (SSR): $\sum (\hat{y}_i - \bar{y})^2$. Represents variation explained by the relationship between $X$ and $Y$.

Model Assumptions

The validity of the regression model $y = \beta_0 + \beta_1 x + \nu$ relies on four assumptions about the error term (nu/epsilon):

1. Expected Value: The error term is a random variable with an expected value (mean) of zero: $E(\nu) = 0$.
2. Independence: The values of error terms are independent of each other.
3. Normality: The error term is normally distributed.
4. Constant Variance: The variance of the error term, denoted by $\sigma^2$, is the same for all values of $x$ (Homoscedasticity).

Outliers

• Definition: An outlier is a data point that deviates significantly from the trend shown by the rest of the observations.
• Handling Outliers:
  • Erroneous Data: If caused by error, they should be corrected.
  • Assumption Violation: If they signal the model is incorrect, a different model should be considered.
  • Chance Occurrence: If they are valid unusual values occurring by chance, they should be retained.
• Detection: Standardized residuals are used to identify outliers. Any observation with a standardized residual less than $-3$ or greater than $+3$ is classified as an outlier.