Advanced Research Design and Statistical Test Selection Guide

Foundational Strategies for Statistical Test Selection

• The primary goal in selecting the correct statistical test is identifying the independent variable (IV) and the dependent variable (DV).

• Determining how participants are assigned to positions (between-subjects vs. within-subjects) is crucial for selecting the appropriate model of analysis.

• The hypothesis provides the framing: it usually states or implies which variable acts as the cause or influence (independent variable) and which acts as the outcome (dependent variable).

• Experimental Language in Hypotheses: Even in non-experimental settings, the language of a hypothesis often suggests that one factor causes an effect on another.
      - Independent Variable (IV): The factor seen as causing or influencing an effect.
      - Dependent Variable (DV): The outcome showing the change.

Comparative Analysis of Single IV Designs

• Case Study: Smoking Support Group
      - Hypothesis: Craving diminishes significantly over time while participating in the support group.
      - Structure: $10$ participants measured every three to twelve weeks (specifically at $3$, $6$, $9$, and $12$ weeks).
      - Independent Variable: Time spent in the group (measured at multiple points).
      - Dependent Variable: Degree of craving evaluated via questionnaire. Higher scores indicate greater craving.
      - Scale: The scores imply a quantitative measure, likely an interval or ratio scale.
      - Statistical Test: Repeated Measures ANOVA.
          - One IV with more than two levels ($3$, $6$, $9$, and $12$ weeks).
          - Within-subjects design because the same individuals are followed throughout the timeline.

• Case Study: Mayberry Kennels (The Andy Griffith Show Reference)
      - Context: Average canine kennel resident nationwide is fed $350\,g$ of canned dog food per day. This is the population mean ($\mu$).
      - Hypothesis: The owner believes Mayberry dogs are fed more than the national average.
      - Sample Data: $22$ dogs at Mayberry; Mean consumption = $359\,g$; Standard Deviation ($s$) = $20\,g$.
      - Analysis Choice: Comparison of a single sample to a population when the population standard deviation ($\sigma$) is unknown.
      - Statistical Test: One-Sample t-test.
          - If the population standard deviation ($\sigma$) had been known, a One-Sample z-test would have been used.

• Case Study: Employee Absenteeism Programs
      - Hypothesis: Assessing the effectiveness of two programs designed to reduce absenteeism.
      - Independent Variable: Type of program (Free Time program vs. Exercise in the corporate gym).
      - Dependent Variable: Number of absences per year (Quantitative/Ratio scale).
      - Design: Between-subjects (different employees assigned to each program).
      - Statistical Test: Two Independent Samples t-test.
          - Note on Overkill: A One-Way Between-Subjects ANOVA would result in the same outcome but is considered "overkill," similar to using a chair to kill a fly instead of a flyswatter.

• Case Study: Lab Rat Birth Weights
      - Context: Average birth weight for the species is $27.6\,g$ with a population standard deviation of $3.4\,g$.
      - Experiment: Random sample of $10$ pregnant rats on a new feeding schedule.
      - Statistical Test: One-Sample z-test.
          - Since the population standard deviation ($3.4\,g$) is explicitly provided, the z-test is the appropriate match.

Factorial Designs and Multivariate Considerations

• Case Study: Memory Factors Study
      - Goal: Identifying factors involved in memory.
      - Factors (Independent Variables):
          - IV 1: Item difficulty (Difficult vs. Easy items).
          - IV 2: Recall timing (Immediate vs. One-day delay).
      - Dependent Variable: Number of items recalled (Ratio scale: magnitude, equal intervals, and absolute zero).
      - Statistical Test: Two-Way Between-Subjects ANOVA.
          - Two IVs and one quantitative DV requires a factorial ANOVA.

• Case Study: Hunger and Perception
      - Hypothesis: Does hunger affect an individual's perception of ambiguous shapes?
      - Independent Variable: Hunger levels ($3$ groups: $1$ hour, $4$ hours, or $12$ hours after a meal).
      - Dependent Variable: Number of food-related perceptions reported (e.g., seeing "wings" in inkblots/Rorschach shapes).
      - Design: Between-subjects (different participants in each of the three groups).
      - Statistical Test: One-Way Between-Subjects ANOVA.

• Case Study: Separation Anxiety/Daycare Design
      - Context: Measuring separation anxiety in children.
      - Independent Variables:
          - IV 1: Caregiver Presence (Leaves vs. Stays).
          - IV 2: Environment Activity (Party vs. No Party).
      - Dependent Variable: Time spent crying (quantitative measure).
      - Statistical Test: Two-Way Between-Subjects ANOVA.

• Case Study: Sleep Research and Performance
      - Independent Variables:
          - IV 1: Form of rest (Napping, Resting, or Normal Activity).
          - IV 2: Time of day (Afternoon vs. Evening).
      - Structure: $30$ volunteers, $5$ per condition in a $2 \times 3$ design.
      - Dependent Variable: Number of errors on a test.
      - Statistical Test: Two-Way Between-Subjects ANOVA.

• Note on MANOVA (Multivariate ANOVA)
      - Occurs when there are multiple dependent variables in a single study.
      - Example: Evaluating Vitamin C effects on both the number of colds and the severity of colds.
      - While common in real-world research, it is generally beyond the scope of introductory statistics courses.

Non-Parametric Tests: Chi-Square Applications

• Chi-Square Goodness of Fit Test
      - Used when the dependent variable is nominal/categorical.
      - Example 1: The "100 Closest Pals" Survey:
          - IV: Type of Heavy Metal concert (Guns N' Roses, Nirvana, Metallica, Def Leppard).
          - DV: Friend’s selection of which concert to attend.
          - Results: $47$ liked Guns N' Roses, $30$ liked Nirvana, $10$ Metallica, $13$ Def Leppard.
          - Note: If participants were asked to rate each band on a scale of $1$-$7$ instead of just picking one, the test would shift to a Repeated Measures ANOVA.

• Chi-Square Test for Independence
      - Used to determine if there is a relationship between two categorical variables.
      - Example: Cerebral Dominance:
          - Variable A: Dominant hemisphere (Right vs. Left).
          - Variable B: Preferred activity (Music vs. Reading).
          - Participants: $127$ randomly selected subjects.
      - Example: Health and Weight Matrix:
          - Variable A: Weight category (Underweight, Normal, Overweight).
          - Variable B: Blood Pressure category (Optimal, Normal, High).
          - Frequencies recorded for each cell (e.g., $1,839$ overweight/high BP out of $5,203$ total subjects).

Predictors, Correlation, and Regression

• Correlation Matrix
      - Used when exploring the relationships among several continuous variables simultaneously.
      - Example (Dr. Hopson's Variables): Examining if the following are related: Number of steps, Mood (scale $1$-$70$), Time spent outside (minutes), and Calories consumed.

• Linear Regression
      - Used when the primary goal is prediction.
      - Simple Correlation vs. Prediction: "Are these related?" implies Correlation. "Does X predict Y?" implies Regression.

• Multiple Regression
      - Used when multiple continuous predictors are used to predict a single outcome.
      - Example: Predicting Social Success:
          - Predictors: Shyness, Sociability, Neuroticism, Agreeableness.
          - Outcome (DV): Number of friends made over the first academic year.

Review of Probability and Contingency Tables

Using a dataset of $5,203$ individuals:

• General Probability: The chance of selecting someone of normal weight with optimal blood pressure.
      - Calculation: $\frac{374\,(\text{Normal/Optimal})}{5203\,(\text{Total})}$ results in a percentage of $7.19\%$, or a probability ($p$) of $0.0719$.

• Conditional Probability: Probability of normal blood pressure given the person has normal weight.
      - Context: Focus only on the normal weight row ($1,472$ people).
      - Calculation: $\frac{704\,(\text{Normal Weight with Normal BP})}{1472\,(\text{Total Normal Weight})}$ results in a percentage of $47.83\%$, or a probability of $0.4783$.

Questions & Discussion

• Question: What is the difference between observed and expected frequencies in a Chi-square test, specifically for the blood pressure chart?

• Answer:
      - Observed Frequencies: The actual counts found in the study (e.g., the $52$ underweight/optimal people).
      - Expected Frequencies: What we would expect to see if the null hypothesis were true.
          - In a Goodness of Fit test with 4 bands and 100 people, the "Equally Popular" expectation would be $25$ people for each band.
          - In some cases, expectations are based on known population proportions (e.g., if a college is $58\%$ women and $42\%$ men, the expected frequency for a sample of $100$ would be $58$ women and $42$ men).
          - In a Test for Independence, expected values are calculated based on the marginal totals (row and column totals) of the grid to account for the fact that some categories (like high blood pressure) are more common than others.

• Anecdotes and Context:
      - The professor mentioned teaching assistant materials from $40$ years ago, resulting in "ancient" band references like Nirvana and Metallica.
      - A joke regarding the Def Leppard drummer (Rick Allen) involving reaching a decision "hands down" was mentioned as a piece of background humor required to understand the original quiz question.
      - Ethics in research: Mention of the shift away from rat research due to budgets and ethical concerns regarding animal mistreatment (e.g., brain trauma studies on rats).
      - Real-world application: The "Norva" venue in Norfolk (stand-in only) was used as a metaphor for crowded, non-random sampling at concerts in early $2020$.