Choosing the Right Statistical Test

Overview of Choosing the Correct Statistical Test

  • Concept of statistical conclusion validity

  • Importance of choosing the right statistical test for data analysis

  • Goal: Provide a structured approach to navigate choices in statistical tests

Key Questions for Statistical Test Selection

  • Introduction to the four key questions that will guide the selection process

  • Understanding the relationship between questions and statistical tests

Question 1: Research Purpose

  • Determine if the research aims to:

    • Compare two or more groups

    • Analyze the relationship between variables

  • Distinction between comparison and relational designs:

    • Comparison Designs: Focus on cause and effect (e.g., causal relationships)

    • Relational Designs: Focus on correlation without establishing causation

    • Implication: Understanding these distinctions helps narrow down appropriate tests

    • Reminder: "Correlation does not equal causation"

Question 2: Type of Variables

  • For Comparison Designs:

    • Are there more than one independent variable?

    • If yes: Look at complex comparison designs (interaction effects)

    • If no: Proceed with basic comparison designs

  • For Relational Designs:

    • Are there more than two total variables?

    • If yes: Leads to predictive models

    • If no: Basic correlations only

Visual Charts for Statistical Tests

  • Chart Overview: Will discuss four charts that assist in choosing statistical tests

  • Charts categorized based on types of comparisons or relationships and variables

Comparison Charts

  • Chart 1: Basic Comparison

    • Used when there is one independent variable and establishes what type of dependent variable data is used:

    • Dependent variable types:

      • Interval/Ratio (with or without statistical assumptions met)

      • Ordinal

      • Nominal

    • Next steps depend on number (two vs. more) of levels in the independent variable

    • Determine design type:

    • Between-subjects vs. within-subjects

    • Applications: various statistical tests based on the intersecting cells of these criteria.

  • Chart 2: Complex Comparison

    • Used when there are multiple independent variables

    • Similar steps to basic comparison, but requires knowledge of each independent variable’s grouping (between or within)

Relational Charts

  • Chart 3: Basic Relational

    • Used when analyzing simple correlations between two variables

    • Types of data for both variables:

    • Interval/Ratio (with assumptions met or violated)

    • Ordinal

    • Nominal

    • Determining analyses based on intersection of these variable types

  • Chart 4: Complex Relational

    • Focused on prediction and multiple variable relationships

    • Outcome variable data is crucial to determine statistical analysis, whether they are:

    • Interval/Ratio data

    • Dichotomous (binary outcomes)

    • Predictive variables categorized in terms of their data types (dichotomous or continuous)

    • Applications of statistical tests:

    • Multiple regression, logistical regression, and discriminant analysis based on the data type

Data Type Considerations

  • Importance of data type:

    • Nominal/Ordinal: Non-parametric statistics applicable

    • Interval/Ratio: Parametric statistics applicable

  • Need to check for statistical assumptions before analysis:

    • Assumptions include:

    • Normality: Distribution resembling a bell curve

    • Homogeneity of Variance: Consistency in variance across groups

    • Independence Assumption: Each data point should be independent

  • What to do if assumptions are violated:

    • Treat interval or ratio data as ordinal data for analysis

Between vs. Within Subjects Design

  • Define between-subjects design:

    • Comparison of different people across groups

  • Define within-subjects design:

    • Comparison of the same individuals over multiple trials or conditions

  • Mention matched pairs design:

    • Match participants based on variables closely related to dependent variable to minimize variation

    • Important to treat this design type as within-subjects despite appearing as between-subjects

Concluding Thoughts

  • Recap of the process:

    • Four key questions lead to identifying the correct charts and statistical analyses

  • Exam expectations:

    • No need to memorize charts, but understanding and applying the four guiding questions is essential

    • Recognize importance of statistical assumptions for interval/ratio data in determining correct statistical tests