Statistical Test Selection and Research Methodology
The Fundamental Goal of Statistics Education
- The primary purpose of a statistics course is not merely to learn how to produce data but to understand when to use which statistic.
- Selecting the correct statistical test is the single most important skill a student can acquire.
- Learning to use software like JASP is secondary; as the instructor notes, "We could teach a monkey to use JASP." The software facilitates point-and-click operations, but it cannot interpret output or select the appropriate test independently.
- Students must master two main components: 1. Determining the correct test for a given research scenario. 2. Interpreting the generated output.
Study Strategy: The Thirteen Statistics
- Students are required to know approximately 13 different statistics.
- The recommended method for mastery is the use of flashcards: * Side One: The name of the statistic. * Side Two: The category of the research question and the specific conditions/assumptions under which the test is utilized.
Categorization: Parametric vs. Nonparametric Statistics
Statistics are divided based on their assumptions and the nature of the data being analyzed.
Parametric Statistics
- Scale of Measurement: These require the dependent variable (DV) to be either interval or ratio.
- Nature: Often referred to as inferential statistics. The goal is to draw a sample and make an inference about a larger population.
- Focus: Researchers are interested in general determinations (e.g., the effect of a treatment drug vs. a placebo on the population) rather than individual sample peculiarities.
- Assumptions: 1. Distribution: Data must be normally distributed. 2. Homogeneity of Variance: Each group must have an approximately equal spread of data. 3. Absence of Outliers: Excessive outliers disrupt the homogeneity of variance. 4. Sample Size: Requires a decent sample size. While historical rules of thumb (especially for ANOVA) suggested per group ( to ), some modern researchers suggest to subjects per group (though this is debated and often considered absurd for undergraduate practice).
- Higher Power: Parametric statistics are preferred because they have more "power," which refers to the ability to detect a difference or relationship if it truly exists in the population.
Nonparametric Statistics
- Usage Conditions: * Used when the DV is on a nominal or ordinal scale. * Used when the data has a non-normal distribution, particularly with small sample sizes. * Used when there are significant outliers.
- Counterparts: Every parametric test (T-tests, ANOVAs, correlations, regressions) has a nonparametric counterpart, though these are typically not taught until graduate school.
- Bias: There is a common bias against nonparametrics because they are less familiar and generally have lower power.
Overview of Specific Statistics
Nonparametric List
- Chi-square
- Spearman correlation
- Median
- Mode
Parametric List
- Mean
- Standard deviation
- Variance
- Z-test
- One-sample T-test
- Pearson product moment correlation
- Two independent samples T-test
- Dependent sample T-test
- ANOVA (Analysis of Variance / F-test): Includes One-way, Multifactor, and Repeated Measures.
- MANOVA (Multiple Analysis of Variance)
- Regression (Linear and Multiple)
Descriptive Research Questions
Descriptive research focuses on describing a phenomenon or data set using measures of central tendency and variability.
Measures of Central Tendency (The "Central Score")
- Mean: The statistical name for an "average." Used for interval/ratio data.
- Median: The middle score in an ordered list. Used for ordinal data or skewed data.
- Mode: The most frequent data point. Used for nominal data.
Measures of Variability (The "Spread")
- Range: The distance between the highest and lowest scores. Used for ordinal, interval, or ratio data (but not nominal).
- Standard Deviation and Variance: * Technically separate but mathematically linked. * * Used for interval/ratio data. * Reporting Rule: A mean should never be reported without its standard deviation (e.g., "").
Assessing Relationships Research Questions
These questions investigate the strength and direction of the relationship between two variables.
- Strength: Determines how well the independent variable (IV) predicts the outcome/DV.
- Direction: How variables change together (e.g., both increasing, or one increasing while the other decreases).
Testing Options
- Pearson's Product Moment Correlation (): Used when both variables are interval or ratio.
- Spearman's Rank Order Correlation (Spearman's Rho): Used when one or both variables are ordinal. The test statistic is still recorded as an but often referred to as "row" ().
Group Differences Compared to a Standard
This involves comparing a sample to a known population value or national average.
Interval or Ratio DV
- Z-test: Used when the population standard deviation (or variance) is known. It is also used when comparing a single individual () to a standard (e.g., comparing one child's IQ to the national average, or one person's cholesterol levels).
- One-sample T-test: Used when comparing a sample to a known standard but the population standard deviation is unknown.
Nominal DV
- Chi-square Goodness of Fit: Used to see if data is equally distributed (e.g., across days of the week) or if it matches a specific population distribution (e.g., comparing a sample's racial distribution to target national percentages such as White or African American).
Group Differences (Between Groups)
Determines if performance differs significantly across two or more groups.
Interval or Ratio DV
- Two Independent Samples T-test: Compares two independent groups (e.g., kittens vs. puppies).
- Dependent T-test: Used in three specific variations: * Pre-Post Design: One sample measured twice (before and after). * Related Samples: Groups connected biologically or socially (twins, spouses, roommates). * Matched Pairs: Two different people matched on variables like age, income, and education to ensure similarity.
- ANOVA (Analysis of Variance / F-test): * One-way: One DV (interval/ratio) and one IV with three or more groups. * Multifactor: One DV (interval/ratio) and more than one IV.
- Repeated Measures ANOVA: Used for pre-post designs when there are more than two measurements, more than two groups, or more than one IV.
- MANOVA: Used when there is more than one DV (must be interval or ratio).
Nominal DV
- Chi-square Test of Independence: Used when the DV is nominal. Although students may mistake this for "assessing relationships" (e.g., "Is there a relationship between race and using alternative medicine?"), if the DV is a "Yes/No" or category, it is a chi-square test of independence.
Regression Research Questions
- Goal: Explaining variability in a DV using a linear combination of IVs.
- Linear Regression: Technically describes the whole field, but in software like JASP, it is often the term for 1 IV and 1 DV.
- Multiple Regression: Used when there are multiple IVs and one DV (interval/ratio).
Practice Application
The instructor suggests taking a screenshot of 19 practice research questions to identify:
- The IV and DV.
- The scale of measurement for each variable.
- The type of research question.
- The appropriate statistic to use.