Understanding Statistical Significance and Hypothesis Testing

Significance Level

Risk of not being 100% confident in results.

Statistical Significance

Degree of risk in rejecting a true null hypothesis.

Null Hypothesis

Assumption that there is no effect or difference.

Type 1 Error

Rejecting null when it is actually true.

Type 2 Error

Failing to reject null when it is false.

Descriptive Statistics

Describes characteristics of a sample.

Inferential Statistics

Infers population characteristics from sample data.

T-Test

Tests significance of differences between means.

Dependent Samples T-Test

Compares two groups tested more than once.

Independent Samples T-Test

Compares means of two different groups.

Homogeneity of Variance

Assumption of equal variability in groups.

Effect Size

Indicates significance and meaningfulness of differences.

T-Distribution

Bell-shaped distribution used in t-tests.

Degrees of Freedom

Number of independent values in calculations.

Analysis of Variance (ANOVA)

Tests differences between two or more means.

F-Test

Test statistic used in ANOVA.

Eta-Squared

Effect size measure for ANOVA results.

Post-Hoc Comparison

Analysis after ANOVA to find specific differences.

Chi-Square Test

Tests independence between categorical variables.

Parametric Assumptions

Assumptions required for parametric statistical tests.

Sample Size

Number of observations in a sample.

One-Tailed Test

Tests for effect in one direction only.

Two-Tailed Test

Tests for effect in both directions.

Interval-Ratio Level

Measurement scale for continuous data.

Non-parametric statistics

Analyze data violating parametric assumptions.

Chi-square

Determines if observed frequencies differ from expected.

One-sample chi-square

Goodness-of-fit test for one categorical dimension.

Two-sample chi-square

Test of independence for two categorical dimensions.

Observed values

Actual frequencies recorded in a study.

Expected values

Frequencies anticipated based on chance.

Statistical independence

Absence of association between two variables.

Hypothesis testing

Procedure to test assumptions about variables.

Null hypothesis (H0)

States no association exists between variables.

Alternative hypothesis (H1)

Proposes a relationship exists between variables.

Chi-square test statistic (χ2)

Summarizes differences between observed and expected frequencies.

Degrees of freedom

Determines the χ2 sampling distribution.

Proportional Reduction of Error (PRE)

Measures strength of relationship between variables.

Lambda (λ)

Asymmetrical measure for nominal variable association.

Cramer's V

Chi-square-related measure ranging from 0 to 1.

Gamma (γ)

Symmetrical measure for ordinal variable association.

Kendall's t-b

Symmetrical measure for ordinal or dichotomous variables.

Phi coefficient (Φ)

Measure for association in 2x2 tables.

Contingency coefficient (C)

Used for larger than 2x2 tables.

Bivariate tables

Display scores on two different variables simultaneously.

Limitations of Chi-square

Difficult interpretation with many categories.

Statistical significance

Indicates likelihood results are not due to chance.

Substantive significance

Practical importance of a result.