MARK 212 - Lecture 18: Bivariate Tests of Differences

Overview of Bivariate Tests of Differences

Fundamental Application: Market research often involves comparing different groups of data to measure variations in a dependent variable across segments, markets, or experimental conditions.
Visualizing Distributions: Differences are often represented by multiple distributions. Even if they appear visually distinct, statistics are required to determine if the difference is due to the randomness of sampling or reflects a true difference in the population.
Measurement Scales and Test Selection: The choice of statistical test is driven by the type of measurement scale used: - Interval or Ratio Scales: Primarily used to compare means. - Nominal or Categorical Scales: Require different types of tests (often non-parametric).

The Statistical Foundation of Difference Testing

Hypothesis Formulation: Conclusive quantitative research begins with a hypothesis. - Research Hypothesis: Asks whether groups are the same or different. - Null Hypothesis ( $H_0$ ): Represents "business as usual" or the status quo, stating there is no difference between groups ( $\mu_1 = ext{mu}_2$ or $\mu_1 - ext{mu}_2 = 0$ ).
Bivariate vs. Multivariate Analysis: - Bivariate Analysis: Analyzing the relationship or difference between exactly two variables. - Multivariate Analysis: Analyzing relationships between more than two variables.
Causation: Experiments are the most effective way to establish the "arrow of causation" (e.g., confirming that a specific advertisement caused a change in attitude rather than an external factor).

Independent Samples t-Test

Definition: A test used to compare the means of two groups where the participants are entirely separate. A subject cannot belong to both groups (e.g., comparing gender, age groups, or different retail locations).
The t-Statistic (t-Value): A ratio calculated to determine the significance of the difference. - Numerator: The difference between the means ( $x_1 - x_2$ ). - Denominator: The standard error ( $s$ ), which measures the uncertainty or variation in the data. - Formula Concept: $t = \frac{\text{Difference Between Means}}{\text{Standard Error of the Difference}}$
Pooled Estimate: This is a single statistic derived by Combining data from both samples to calculate a standard error of the difference ( $s_{\bar{x}_1 - \bar{x}_2}$ ).
Case Study: Blood Donation Ad Experiment: - Control Group: No exposure to the ad. - Experimental Group: Exposed to a new advertisement. - Measurement: Attitude toward donating blood on a 5-point scale (high score = positive disposition). - Data Specifics: - Control group $n = 48$ . - Experimental group $n = 49$ . - Pooled standard error estimate: $0.275$ . - Calculated t-statistic: $3.45$ (Note: the sign (+/-) depends on which mean is subtracted from which, but only the magnitude matters). - Inference: Because the t-statistic exceeded the critical level and the $p$ -value was less than $0.05$ , the null hypothesis was rejected. The ad was proven effective in improving attitudes.

Statistical Output and Pre-Testing

Levene’s Test for Equality of Variances: An automatic pre-test performed before the t-test. - Goal: To determine if the variances of the two groups are equal. - Null Hypothesis for Levene's: Variances are equal. - Result Importance: If p > 0.05, variances are assumed equal (homogeneity is met), and the standard pooled variance calculation is used. If p < 0.05, a "shrinkage" adjustment is made to the t-statistic to remain conservative.
Degrees of Freedom ( $df$ ): Calculated based on the number of observations minus one for the test itself ( $N - 1$ ).
Interpretation of $p$ -values: If the $p$ -value is less than the threshold (typically $0.05$ ), there is strong evidence to support that the means are not equal in the sampled population.

Paired (Dependent) Samples t-Test

Definition: Also known as a "Related" or "Dependent" samples t-test. It is used when the same subject is measured twice, such as in a "before and after" experiment.
Application: Useful for picking up changes over time while controlling for individual respondent differences.
Case Study: Blood Donation (One Group Pre-test Post-test): - Measure 1: Intent before the ad ( $n = 97$ ). - Measure 2: Intent after the ad ( $n = 97$ ). - Social Desirability Bias: Measuring the same person twice helps eliminate certain biases (like social desirability) by focusing on the individual's change rather than the raw score.
Visual Interpretation: Using 95% Confidence Intervals. If the range of the difference between the means does not include zero (the null value), the null hypothesis is rejected visually. - Example Results: Difference range between $0.8$ and $1.33$ . Since the lower bound ( $0.8$ ) is well above zero, the change is statistically significant.

Analysis of Variance (ANOVA)

Definition: Used when comparing the means of three or more independent groups.
The F-Statistic (F-Ratio): A ratio of variability across groups compared to variability within groups. - Between-Group Variation: Calculated using differences in the means of each group ( $3$ groups = $2$ degrees of freedom). - Within-Group Variation: The individual error/variance within the groups. - Formula Concept: $F = \frac{\text{Mean Square Between Groups}}{\text{Mean Square Within Groups}}$
ANOVA Case Study: Tourism Promotion: - Independent Variable (Factor): Introductory Promotion Type. 1. Relaxing holiday (Low excitement). 2. Rally driving (Medium excitement). 3. Shark diving (High excitement). - Inverted U-Relationship ("Goldilocks Effect"): Results showed that rally driving peaked in popularity. Too little excitement was unengaging, and too much (shark diving) was intimidating. The "optimal" level was in the middle. - Small Sample Size Warning: In the provided data ( $n = 12$ total, $4$ per group), the F-statistic was $1.95$ with a $p$ -value greater than $0.05$ . Despite the means looking different (inverted U), the study failed to reject the null hypothesis due to lack of evidence, likely caused by an insufficient sample size.

Practical vs. Statistical Significance

Statistical Significance: Confirms that a result is unlikely to have occurred by chance (p < 0.05).
Practical Significance (Effect Size): Concerns whether the magnitude of the difference is large enough to matter in a real-world business context. - Commercial Relevance: A change in market share of $0.05\%$ might be statistically significant with a large enough sample but commercially irrelevant. A change of $20\%$ is practically significant. - Consulting Clients: Researchers should ask stakeholders what level of change (e.g., movement on a Likert scale) would prompt them to change their marketing strategy or launch a new product.

Assumptions and Nonparametric Tests

Parametric Assumptions: Tests like t-tests and ANOVA assume the data follows a normal distribution (bell-shaped) and has equal variances.
Nonparametric Tests: Used when data is not symmetric or distribution assumptions are violated. They do not assume a normal distribution.
Practical Application: In most modern software (like SAS Studio), parametric and nonparametric tests often yield the same inference. Nonparametric tests serve as a valuable "check" to ensure the robustness of the findings.

Summary Checklist for Test Selection

How many groups? - 2 groups: t-test. - 3+ groups: ANOVA.
Is it the same person? - Yes: Paired/Related t-test. - No: Independent samples t-test.
Software Automation (SAS Studio): - Automatically selects t-test vs. ANOVA based on the number of groups in the variable. - Automatically performs equality of variance tests. - Requires one metric (interval/ratio) variable and one categorical (nominal) grouping variable.