Statistical Test Selection and Research Methodology

The Fundamental Goal of Statistics Education
  • The primary purpose of a statistics course is not merely to learn how to produce data but to understand when to use which statistic.
  • Selecting the correct statistical test is the single most important skill a student can acquire.
  • Learning to use software like JASP is secondary; as the instructor notes, "We could teach a monkey to use JASP." The software facilitates point-and-click operations, but it cannot interpret output or select the appropriate test independently.
  • Students must master two main components:     1. Determining the correct test for a given research scenario.     2. Interpreting the generated output.
Study Strategy: The Thirteen Statistics
  • Students are required to know approximately 13 different statistics.
  • The recommended method for mastery is the use of flashcards:     * Side One: The name of the statistic.     * Side Two: The category of the research question and the specific conditions/assumptions under which the test is utilized.
Categorization: Parametric vs. Nonparametric Statistics

Statistics are divided based on their assumptions and the nature of the data being analyzed.

Parametric Statistics
  • Scale of Measurement: These require the dependent variable (DV) to be either interval or ratio.
  • Nature: Often referred to as inferential statistics. The goal is to draw a sample and make an inference about a larger population.
  • Focus: Researchers are interested in general determinations (e.g., the effect of a treatment drug vs. a placebo on the population) rather than individual sample peculiarities.
  • Assumptions:     1. Distribution: Data must be normally distributed.     2. Homogeneity of Variance: Each group must have an approximately equal spread of data.     3. Absence of Outliers: Excessive outliers disrupt the homogeneity of variance.     4. Sample Size: Requires a decent sample size. While historical rules of thumb (especially for ANOVA) suggested 10 subjects10\,subjects per group (55 to 1010), some modern researchers suggest 5050 to 100100 subjects per group (though this is debated and often considered absurd for undergraduate practice).
  • Higher Power: Parametric statistics are preferred because they have more "power," which refers to the ability to detect a difference or relationship if it truly exists in the population.
Nonparametric Statistics
  • Usage Conditions:     * Used when the DV is on a nominal or ordinal scale.     * Used when the data has a non-normal distribution, particularly with small sample sizes.     * Used when there are significant outliers.
  • Counterparts: Every parametric test (T-tests, ANOVAs, correlations, regressions) has a nonparametric counterpart, though these are typically not taught until graduate school.
  • Bias: There is a common bias against nonparametrics because they are less familiar and generally have lower power.
Overview of Specific Statistics
Nonparametric List
  • Chi-square
  • Spearman correlation
  • Median
  • Mode
Parametric List
  • Mean
  • Standard deviation
  • Variance
  • Z-test
  • One-sample T-test
  • Pearson product moment correlation
  • Two independent samples T-test
  • Dependent sample T-test
  • ANOVA (Analysis of Variance / F-test): Includes One-way, Multifactor, and Repeated Measures.
  • MANOVA (Multiple Analysis of Variance)
  • Regression (Linear and Multiple)
Descriptive Research Questions

Descriptive research focuses on describing a phenomenon or data set using measures of central tendency and variability.

Measures of Central Tendency (The "Central Score")
  1. Mean: The statistical name for an "average." Used for interval/ratio data.
  2. Median: The middle score in an ordered list. Used for ordinal data or skewed data.
  3. Mode: The most frequent data point. Used for nominal data.
Measures of Variability (The "Spread")
  1. Range: The distance between the highest and lowest scores. Used for ordinal, interval, or ratio data (but not nominal).
  2. Standard Deviation and Variance:     * Technically separate but mathematically linked.     * Standard Deviation=Variance\text{Standard Deviation} = \sqrt{\text{Variance}}     * Used for interval/ratio data.     * Reporting Rule: A mean should never be reported without its standard deviation (e.g., "Mean=X,SD=Y\text{Mean} = X, \text{SD} = Y").
Assessing Relationships Research Questions

These questions investigate the strength and direction of the relationship between two variables.

  • Strength: Determines how well the independent variable (IV) predicts the outcome/DV.
  • Direction: How variables change together (e.g., both increasing, or one increasing while the other decreases).
Testing Options
  1. Pearson's Product Moment Correlation (rr): Used when both variables are interval or ratio.
  2. Spearman's Rank Order Correlation (Spearman's Rho): Used when one or both variables are ordinal. The test statistic is still recorded as an rr but often referred to as "row" (ρ\rho).
Group Differences Compared to a Standard

This involves comparing a sample to a known population value or national average.

Interval or Ratio DV
  1. Z-test: Used when the population standard deviation (or variance) is known. It is also used when comparing a single individual (n=1n=1) to a standard (e.g., comparing one child's IQ to the national average, or one person's cholesterol levels).
  2. One-sample T-test: Used when comparing a sample to a known standard but the population standard deviation is unknown.
Nominal DV
  1. Chi-square Goodness of Fit: Used to see if data is equally distributed (e.g., across days of the week) or if it matches a specific population distribution (e.g., comparing a sample's racial distribution to target national percentages such as 66%66\% White or 12%12\% African American).
Group Differences (Between Groups)

Determines if performance differs significantly across two or more groups.

Interval or Ratio DV
  1. Two Independent Samples T-test: Compares two independent groups (e.g., kittens vs. puppies).
  2. Dependent T-test: Used in three specific variations:     * Pre-Post Design: One sample measured twice (before and after).     * Related Samples: Groups connected biologically or socially (twins, spouses, roommates).     * Matched Pairs: Two different people matched on variables like age, income, and education to ensure similarity.
  3. ANOVA (Analysis of Variance / F-test):     * One-way: One DV (interval/ratio) and one IV with three or more groups.     * Multifactor: One DV (interval/ratio) and more than one IV.
  4. Repeated Measures ANOVA: Used for pre-post designs when there are more than two measurements, more than two groups, or more than one IV.
  5. MANOVA: Used when there is more than one DV (must be interval or ratio).
Nominal DV
  1. Chi-square Test of Independence: Used when the DV is nominal. Although students may mistake this for "assessing relationships" (e.g., "Is there a relationship between race and using alternative medicine?"), if the DV is a "Yes/No" or category, it is a chi-square test of independence.
Regression Research Questions
  • Goal: Explaining variability in a DV using a linear combination of IVs.
  • Linear Regression: Technically describes the whole field, but in software like JASP, it is often the term for 1 IV and 1 DV.
  • Multiple Regression: Used when there are multiple IVs and one DV (interval/ratio).
Practice Application

The instructor suggests taking a screenshot of 19 practice research questions to identify:

  1. The IV and DV.
  2. The scale of measurement for each variable.
  3. The type of research question.
  4. The appropriate statistic to use.