Psychological Research Methodology II: Exhaustive Tutorial Notes

Basic Statistical Concepts and Methodology

• Population: The entire group of individuals that a researcher is interested in studying.
• Sample: A subset of the population selected for study, used to make inferences about the whole population.
• Random Sample: A sample in which every member of the population has an equal chance of being selected; this is critical for the generalisability of results.
• Random Assignment: The process of assigning participants to different experimental conditions by chance, reducing the likelihood of systematic differences between groups prior to the experiment.
• Independent Variable (IV): The variable that is manipulated or categorized by the researcher to determine its effect on another variable.
• Dependent Variable (DV): The variable that is measured; it is expected to change as a result of the manipulation of the IV.
• Discrete Variables: Variables that can only take on specific, distinct values (e.g., the number of children in a family).
• Continuous Variables: Variables that can theoretically take on any value within a range (e.g., time, height).
• Measurement Scales:
  • Nominal: Data categorized by names or labels with no inherent order (e.g., gender, hair color).
  • Ordinal: Data where the order/ranking is meaningful, but the intervals between ranks are not necessarily equal (e.g., race finishing positions).
  • Interval: Numeric data where the difference between values is consistent, but there is no absolute zero (e.g., temperature in Celsius).
  • Ratio: Numeric data with equal intervals and a true absolute zero point (e.g., weight, number of items recalled).
• Data Types:
  • Measurement Data: Quantitative data resulting from measurement.
  • Categorical Data: Qualitative data representing different categories.
• Parameter vs. Statistic:
  • Parameter: A numerical value that describes a characteristic of a population (usually denoted by Greek letters).
  • Statistic: A numerical value that describes a characteristic of a sample (usually denoted by Roman letters).
• Statistics Branches:
  • Descriptive Statistics: Used to summarize and describe the features of a specific dataset.
  • Inferential Statistics: Used to make predictions or generalizations about a population based on sample data.

Measures of Central Tendency and Variability

• Measures of Central Tendency:
  • Mean: The arithmetic average of all scores.
  • Median: The middle score in a distribution when ordered.
  • Mode: The most frequently occurring score.
• Measures of Variability:
  • Range: The difference between the highest and lowest scores.
  • Inter-quartile Range (IQR): The range of the middle $50\%$ of scores.
  • Standard Deviation (SD): The square root of the variance; represents the average amount scores deviate from the mean.
  • Variance: The average of the squared deviations from the mean.
• Distributions:
  • Normal: Bell-shaped, symmetrical distribution.
  • Bimodal: A distribution with two distinct peaks.
  • Positively Skewed: The tail of the distribution extends toward the higher values (right).
  • Negatively Skewed: The tail extends toward the lower values (left).
  • Kurtosis: The "peakedness" or flatness of a distribution.
• Parametric Tests: These tests (e.g., $z$-test, $t$-test, ANOVA) assume data is drawn from normally distributed populations.

Summation Notation and Variance Formulae

• Notation:
  • $\sum X$: Sum of all scores.
  • $(\sum X)^2$: Add all scores first, then square the total.
  • $\sum X^2$: Square each individual score first, then add the results.
• Sum of Squares (SS): The sum of the squared deviations of scores around their mean.
  • Conceptual Formula: $SS = \sum(X - \bar{X})^2$
• Variance ($s^2$): An unbiased estimate of the population variance (also called Mean Square or $MS$).
  • Conceptual Formula: $s^2 = \frac{\sum(X - \bar{X})^2}{N - 1}$
• Standard Deviation ($s$ or $SD$): The square root of the unbiased variance.
  • Conceptual Formula: $s = \sqrt{\frac{\sum(X - \bar{X})^2}{N - 1}}$

Normal Distribution and Hypothesis Testing

• Standard Normal Distribution ($z$-distribution): A normal distribution transformed to have a mean of $0$ ($\mu = 0$) and a standard deviation of $1$ ($\sigma = 1$).
• $z$-score: Represents the number of standard deviations a score lies above or below the mean.
  • Formula: $z = \frac{X - \mu}{\sigma}$
• Hypothesis Testing Framework:
  • Null Hypothesis ($H_0$): States there is no effect or difference (any observed difference is due to sampling error).
  • Alternative Hypothesis ($H_1$): States there is a significant difference or effect (the difference is not due to sampling error).
• Sampling Distribution of the Mean: The distribution of means from all possible samples of a specific size $N$ drawn from a population.
• Standard Error of the Mean (SEM): The standard deviation of the sampling distribution of the mean.
  • Formula: $\sigma_M = \frac{\sigma}{\sqrt{N}}$
• Factors influencing SEM:
  1. Amount of variation in the population ($\sigma$).
  2. Sample size ($N$).
• Decision Rule: If the obtained value (e.g., $|z_{obt}|$) is greater than the critical value (e.g., $z_{crit} = 1.96$ for $\alpha = .05$, two-tailed), reject $H_0$.

The Single-Sample t-Test

• Purpose: Used to compare a sample mean against a population mean ($\mu$) when the population variance ($\sigma^2$) is unknown and must be estimated from the sample ($s^2$).
• Assumptions:
  1. Random Sampling: Scores are randomly sampled from the population.
  2. Normal Distribution: The sampling distribution of the mean should be normal (guaranteed if population is normal or $N \ge 30$ by Central Limit Theorem).
• Degrees of Freedom ($df$): $df = N - 1$.
• Standard Error of the Mean Estimate: $s_M = \frac{s}{\sqrt{N}}$
• $t$-statistic Formula: $t = \frac{\bar{X} - \mu}{s_M}$

Repeated-Measures and Independent-Groups t-Tests

• Repeated-Measures (Within-Participants) t-Test:
  • Analyses paired scores from the same participants or matched pairs (e.g., twins).
  • Advantage: More sensitive; individual differences do not contribute to error variance.
  • Hypothesis: Usually tested against $H_0: \mu_D = 0$.
  • $df$: $N - 1$ (where $N$ is the number of pairs).
• Independent-Groups (Between-Participants) t-Test:
  • Compares means of two independent samples.
  • Assumptions: Random sampling, normality, and Homogeneity of Variance (population variances are equal).
  • Pooled Variance ($s_p^2$): A weighted average of the two sample variances.
    • Formula: $s_p^2 = \frac{SS_1 + SS_2}{(n_1 - 1) + (n_2 - 1)}$
  • Standard Error of Difference: $s_{\bar{X}_1 - \bar{X}_2} = \sqrt{s_p^2 \times (\frac{1}{n_1} + \frac{1}{n_2})}$
  • $df$: $(n_1 - 1) + (n_2 - 1)$ or $N - 2$.
• Levene's Test: A statistical test for the assumption of homogeneity of variance. If $p < .05$, variances are significantly different (assumption violated).

Statistical Power and Effect Size

• Power: The probability of correctly rejecting a false null hypothesis. It depends on:
  1. Chosen $\alpha$-level.
  2. True difference between distributions ($\mu_1 - \mu_0$).
  3. Sample size ($N$) and population variance ($\sigma^2$).
• Effect Size (Cohen's $d$): A standardized measure of the magnitude of the difference between means.
  • Formula: $d = \frac{\mu_1 - \mu_0}{\sigma}$
  • Cohen's Conventions for $d$:
    • Small: $0.20$ ($85\%$ overlap).
    • Medium: $0.50$ ($67\%$ overlap).
    • Large: $0.80$ ($53\%$ overlap).
• Statistical Effect ($\delta$): Combines effect size and sample size to determine power using tables.
  • Single Sample: $\delta = d \times \sqrt{n}$
  • Independent Samples: $\delta = d \times \sqrt{\frac{n}{2}}$

One-Way Independent-Groups ANOVA

• Purpose: An extension of the $t$-test for comparing two or more group means. It is an omnibus test, meaning a significant result only shows somewhere a difference exists.
• Assumptions: Normality, Homogeneity of Variance, and Independence of Observations.
• Partitioning Variation:
  • Total Variation: Sum of squared deviations of each score around the Grand Mean ($SS_{total}$).
  • Between-Groups (Treatment) Variation: Variation attributed to group differences ($SS_{treatment}$).
  • Within-Groups (Error) Variation: Variation among scores within each group ($SS_{error}$).
• ANOVA Summary Table:
  • Treatment: $df = k - 1$; $MS_{treat} = \frac{SS_{treat}}{df_{treat}}$
  • Error: $df = N - k$; $MS_{error} = \frac{SS_{error}}{df_{error}}$
  • $F$-ratio: $F = \frac{MS_{treat}}{MS_{error}}$

Multiple Comparisons: A Priori and Post Hoc

• Type I Error Accumulation: Making multiple comparisons increases the Familywise (FW) error rate.
• A Priori (Planned) Comparisons:
  • Planned before data collection.
  • Bonferroni adjustment: Adjusts the critical value to keep FW error at $\alpha = .05$.
  • Bonferroni $t'$: $\alpha' = \frac{\alpha}{c}$ (where $c$ is the number of comparisons).
• Linear Contrasts ($L$): A weighted comparison of means where $\sum a_j = 0$.
  • $t$-test of contrast: $t = \frac{L}{\sqrt{\frac{\sum a_j^2 \times MS_{error}}{n}}}$
• Orthogonal Contrasts: Sets of contrasts that provide independent information ($\sum a_j b_j = 0$). Maximum number of orthogonal contrasts is $k - 1$.
• Post Hoc Comparisons: Performed only after a significant omnibus $F$.
  • Scheffe Test: The most conservative post hoc test; sets FW error against all possible linear contrasts.
  • Critical Value: $CV_{Scheffe} = (k - 1) \times F_{crit}(k-1, N-k)$.

One-Way Repeated-Measures ANOVA and Magnitude of Effect

• Partitioning in RM ANOVA: Removes individual differences (Variation Between Participants) from the error term, making the test more powerful.
  • **$SS_{error} = SS_{total} - SS_{treatment} - SS_{participants}$
  • **$df_{error} = (n - 1)(k - 1)$
• Effect Magnitude Measures:
  • Eta-squared ($\eta^2$): Descriptive statistic of sample effect size.
    • RM Formula: $\eta^2 = \frac{SS_{treatment}}{SS_{treatment} + SS_{error}}$
  • Omega-squared ($\omega^2$): Estimates population variability attributed to treatment.

Two-Way Factorial ANOVA and Simple Effects

• Terminology:
  • Main Effect: The effect of one independent variable ignoring the others (marginal means).
  • Interaction: Occurs when the effect of one variable depends on the level of another (lines on a graph are not parallel).
  • Simple Effect: The effect of one variable at a specific level of another variable.
• Models:
  • Additive: No interaction; effects combine consistently.
  • Interactive: Presence of an interaction requires analyzing simple effects.

Non-Parametric Statistics

• Wilcoxon's Rank-Sum Test: Non-parametric alternative to the independent $t$-test (uses ranked data).
• Wilcoxon's Matched-Pairs Signed-Ranks Test: Non-parametric alternative to the repeated-measures $t$-test.
• Decision Criterion: Unlike parametric tests, for these tests, the obtained value ($W_s$ or $T$) must be less than or equal to the critical value to be significant.

Chi-Square Tests

• Goodness-of-Fit Test: Compares observed frequencies against expected frequencies for a single categorical variable.
  • **$df = k - 1$.
• Test of Independence: Determines if two categorical variables are related.
  • Expected Frequency ($E$): $E = \frac{\text{Row Total} \times \text{Column Total}}{\text{Grand Total}}$
  • **$df = (r - 1)(c - 1)$.
• Formula: $\chi^2 = \sum \frac{(O - E)^2}{E}$
• Cramer’s phi ($\phi$): Measure of association for $\chi^2$.

Correlation and Covariance

• Covariance: Measure of linear association affected by measurement scale.
• Pearson's $r$: Standardized correlation ranging from $-1$ to $+1$.
  • Conceptual Formula: $r = \frac{\text{Covariance}}{\text{SD}_X \times \text{SD}_Y}$
  • Significance Test: $t = \frac{r \times \sqrt{N - 2}}{\sqrt{1 - r^2}}$
• Spearman's Rho ($r_s$): Correlation used for ranked or badly skewed data.
• Point-Biserial Correlation ($r_{pb}$): Correlation between one dichotomous and one continuous variable.
• Fisher’s $r'$ Transformation: Used to test the difference between two independent correlations.
  • Standard Error of difference: $\sigma_{r'_1 - r'_2} = \sqrt{\frac{1}{n_1 - 3} + \frac{1}{n_2 - 3}}$

Linear Regression

• Regression Equation: $\hat{Y} = bX + a$
  • Slope ($b$): Predicted change in $Y$ for every one-unit change in $X$.
  • Intercept ($a$): Value of $Y$ when $X = 0$.
• Least Squares Criterion: Minimizes $\sum(Y - \hat{Y})^2$.
• Residuals: The difference between actual and predicted scores ($Y - \hat{Y}$).
• Standard Error of Estimate: Accuracy of prediction; represents the average residual.
• Standardized Regression ($\beta$): Bivariate regression where $\beta = r$.