Comprehensive Study Guide for Psychological Statistics

Prologue: The Nature of Statistics

• Personal Perspective of Garett C. Foster, Ph.D.: Statistics can be perceived as difficult, esoteric, and overwhelming. Initially, Foster detested the subject during his introductory course. However, once the underlying logic was understood, it became a powerful tool for objective observation.

• Finding the Signal in the Noise: The primary function of statistics is to distinguish meaningful patterns ("signals") from random chance ("noise").

• Objective Filtering: Statistics provides an objective lens to filter the constant onslaught of information in contemporary life, ensuring sanity and clarity through logic.

• Pedagogical Goal: This guide focuses on formulae not merely as numerical manipulations, but as a system of interconnected topics and methods applicable to behavioral sciences and everyday decisions.

Chapter 1: Introduction and Terminology

• Defining Statistics: In a broad sense, statistics refers to techniques and procedures for analyzing, interpreting, displaying, and making decisions based on data. It is the universal language of science, allowing researchers across fields to articulate findings accurately.

• Statistical Misconceptions: Statistics is not merely a "math class." While it utilizes math as a tool, it is better defined as a method for organizing and communicating information objectively.

• Statistical Facts and Figures: Examples include:

  • The largest earthquake measured $9.2$ on the Richter scale.

  • Men are at least $10\times$ more likely than women to commit murder.

  • $1$ in every $8$ South Africans is HIV positive.

  • $79.48\%$ of all statistics are reportedly made up on the spot (a ironic illustration of statistical misuse).

• Flaws in Interpretation: Numbers may be accurate, but the logic applied to them can be flawed:

  • History Effect: An ice cream ad precedes a $30\%$ sales increase. However, if this occurs in June, the increase is likely due to the season (time), not the ad.

  • Third-Variable Problem: A correlation exists between the number of churches and crime rates. The third variable is population size (larger cities have more of both).

  • Missing Context: A statement that interracial marriages increased by $75\%$ in $25$ years lacks the base rate. If the rate went from $1\%$ to $1.75\%$, the social implication is very different.

• Logic for Daily Life: Studying statistics empowers individuals to evaluate claims like "$4$ out of $5$ dentists recommend Dentine" or "Women make $75$ cents to every dollar a man makes" to ensure intelligent consumer behavior.

Types of Variables and Levels of Measurement

• Variables: Characteristics or features of the item being studied (e.g., stress levels, health).

• Independent Variable (IV): The variable manipulated by the experimenter (e.g., dosage of a drug).

• Dependent Variable (DV): The outcome variable measured to see the effect of the IV (e.g., relief from symptoms).

• Levels: The number of conditions in an IV (e.g., a study with a placebo and an active drug has $2$ levels).

• Qualitative (Categorical) Variables: Variables expressing attributes without numerical ordering (e.g., hair color, religion, favorite movie).

• Quantitative Variables: Variables measured in numbers (e.g., height, weight, test scores).

• Discrete Variables: Variables with scores at discrete points (e.g., number of children; you cannot have $4.53$ children).

• Continuous Variables: Variables on a continuous scale where any value is possible (e.g., response time measured as $1.64237\,s$).

• Levels of Measurement (S.S. Stevens' Scales):

  • Nominal: Categorization or naming only (e.g., Gender, Handedness). No ordering is implied.

  • Ordinal: Categories with a specific order (e.g., satisfaction ratings like "very dissatisfied" to "very satisfied"). Intervals between values are not necessarily equal.

  • Interval: Numerical scales where intervals are consistent (e.g., Fahrenheit). No true zero point (zero degrees does not mean absence of temperature).

  • Ratio: Most informative scale. Has a true zero point representing the absence of the quantity (e.g., money, weight, Kelvin scale). Allows for ratio statements (e.g., "twice as much").

Sampling and Research Design

• Population: The entire collection of people sharing a characteristic of interest.

• Sample: A small subset of the population used to draw inferences.

• Sampling Bias: Occurs when a sample over-represents certain segments, making results non-generalizable.

• Sampling Error: The natural, expected discrepancy between a sample statistic and a population parameter.

• Sampling Strategies:

  • Simple Random Sampling (SRS): Every member has an equal and independent chance of selection.

  • Stratified Sampling: Identifying groups ("strata") and sampling proportionally to ensure the sample matches the population's demographics (e.g., $70\%$ day students and $30\%$ night students).

  • Convenience Sampling: Using easily accessible subjects; often biased and used only for preliminary data.

• Types of Research Designs:

  • Experimental: Uses random assignment and manipulation of the IV to determine causality.

  • Quasi-Experimental: Manipulates the IV but lacks random assignment (often due to ethics or logistics, e.g., using existing classrooms).

  • Non-Experimental (Correlational): Observes variables as they naturally occur. Useful for prediction but cannot establish causality.

Mathematical Notation

• Summation Notation ($\Sigma$): The Greek letter sigma indicates the sum of a set of numbers.

• General Formula: $\sum_{i=1}^{n} X_i$ means sum the variable $X$ from the first to the $n$-th observation.

• Order of Operations:

  • $\sum X^2$: Square each number first, then sum the squares.

  • $(\sum X)^2$: Sum all numbers first, then square the total.

• Sum of Cross Products: $\sum XY$ indicates multiplying each person's $X$ score by their $Y$ score, then summing those products.

Chapter 2: Visualizing Data

• Frequency Tables: Derived from the count of observations in each category. Relative frequency is the proportion (e.g., $85/500 = 0.17$).

• Qualitative Visualization:

  • Pie Charts: Effective for small numbers of categories. The area of the slice is proportional to the percentage of the whole.

  • Bar Charts: Used for frequencies. The Y-axis usually shows count, X-axis shows the category. Fanciness (e.g., $3D$ effects) should be avoided as they create a "lie factor" (distortion).

• Quantitative Visualization:

  • Stem and Leaf Displays: Best for small datasets. Stems represent higher digits (e.g., $10s$), leaves represent lower digits (e.g., $1s$). Useful for seeing distribution shape while retaining raw data.

  • Histograms: Used for large datasets. Scores are grouped into "class intervals" or "bins." The height of the bar represents frequency.

  • Frequency Polygons: Points are placed at the height of the frequency for each interval and connected by lines. Ideal for comparing distributions by overlaying them.

  • Box Plots: Primarily used to identify outliers and compare spreads. Components include:

    • Hinges: $25th$ and $75th$ percentiles (the box).

    • Median: Line inside the box ($50th$ percentile).

    • Whiskers: Lines extending to the adjacent values.

    • Outliers: Marked as circles ("outside") or asterisks ("far out").

• Distribution Shapes:

  • Symmetrical: The left and right halves are mirror images (e.g., Normal Distribution/Bell Curve).

  • Bimodal: Two distinct peaks.

  • Skew: Asymmetry where one tail is longer.

    • Positive (Right) Skew: Tail points toward positive/higher numbers.

    • Negative (Left) Skew: Tail points toward negative/lower numbers.

Chapter 3: Central Tendency and Spread

• Measures of Central Tendency:

  • Mean ($\mu$ or $\bar{X}$): The arithmetic average. It is the balance point of the distribution. It minimizes the sum of squared deviations.

    • $\mu = \frac{\sum X}{N}$

  • Median: The exact midpoint ($50th$ percentile). Minimizes the sum of absolute deviations. Resistant to outliers.

  • Mode: The most frequent score. The only measure usable for nominal data.

• Relative Positions in Skewed Distributions:

  • In a Positive Skew, $\text{Mean} > \text{Median} > \text{Mode}$.

  • In a Negative Skew, $\text{Mode} > \text{Median} > \text{Mean}$.

• Measures of Spread (Variability):

  • Range: Highest score minus lowest score. Extremely sensitive to outliers.

  • Interquartile Range (IQR): The middle $50\%$ of data ($75th - 25th$ percentiles).

  • Sum of Squares (SS): $\sum (X - \bar{X})^2$. The total squared deviation from the mean.

  • Variance ($\sigma^2$ or $s^2$): The average squared deviation.

    • Population Variance: $\sigma^2 = \frac{SS}{N}$

    • Sample Variance: $s^2 = \frac{SS}{N-1}$. Dividing by $N-1$ (degrees of freedom/df) corrects for the tendency of samples to underestimate population variability.

  • Standard Deviation ($\sigma$ or $s$): The square root of the variance. It returns the measure to the original units of the data.

Chapter 4: z-scores and the Standard Normal Distribution

• Normal Distribution Properties: Symmetrical, bell-shaped, area under curve $= 1.0$. Mean, median, and mode are equal.

• The Empirical Rule:

  • $68\%$ of data falls within $\pm 1\,SD$.

  • $95\%$ of data falls within $\pm 2\,SD$.

  • $99.7\%$ of data falls within $\pm 3\,SD$.

• Standard Normal Distribution: A normal distribution with $\mu = 0$ and $\sigma = 1$.

• Standardized Scores (z-scores): Convert raw scores into units of standard deviation to determine relative location.

  • $z = \frac{X - \mu}{\sigma}$

  • Interpretation: Sign ($+/\text{--}$) indicates direction; Magnitude indicates distance. A score with $|z| > 1.5$ might be considered relative far from the mean.

• Standardizing Scales: z-scores can be transformed into any scale ($X = z\sigma + \mu$). E.g., converting a z-score of $1.71$ to an IQ score ($\mu=100, \sigma=16$) yields an IQ of $127$.

Chapter 5: Probability

• Definition: $P(A) = \frac{\text{Number of outcomes satisfying criteria}}{\text{Total number of possible outcomes}}$.

• Probability and the Normal Curve: The area under the curve represents the probability of a random observation falling within a specific range.

• z-Table Usage: Allows for finding the area in the "body" (larger part) or "tail" (smaller part) relative to a z-score.

• Symmetry and Complements: Because the curve is symmetrical, the area of the left tail is the same as the right tail for identical z-magnitudes. Total area $= 1.0$, so $\text{Tail Area} = 1 - \text{Body Area}$.

Chapter 6: Sampling Distributions

• Definition: A theoretical distribution of a statistic (e.g., sample means) calculated from all possible samples of a specific size $n$.

• Central Limit Theorem (CLT): States that for samples of size $n$, the sampling distribution of the mean will:

  1. Have a mean $\mu_{\bar{X}} = \mu$.

  2. Have a standard deviation (Standard Error) $\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$.

  3. Approach normality as $n$ increases ($n \ge 30$ is the general threshold).

• Standard Error ($SE$): Quantifies sampling error. As sample size ($n$) increases, $SE$ decreases ($n \uparrow, SE \downarrow$).

• Law of Large Numbers: As sample size increases, the sample mean becomes a more accurate estimate of the population mean.

• Calculate z for a Mean: $z = \frac{\bar{X} - \mu}{SE}$, used to find the probability of obtaining a specific sample mean.

Chapter 7: Introduction to Hypothesis Testing

• The Logic of Fisher: Uses the probability of an outcome given a specific state of the world to cast doubt on that state. (Example: James Bond correctly identifying martinis $13_{/16}$ times; probability of guessing is $0.0106$.)

• The Null Hypothesis ($H_0$): The baseline assumption that there is no effect, no difference, or that any apparent effect is due to chance (e.g., $H_0: \mu = 8$).

• The Alternative Hypothesis ($H_A$): The research hypothesis; states that there is a significant difference or effect.

  • Directional (One-tailed): Specifies direction (e.g., $H_A: \mu > 8$).

  • Non-directional (Two-tailed): Specifies any difference (e.g., $H_A: \mu \neq 8$).

• Significance Level ($\alpha$): The threshold for rejecting $H_0$, typically set at $0.05$. It is the probability of a Type I error.

• Critical Values ($z^*$ or $t^*$): The z-scores that bound the "rejection region."

• p-value: The actual probability of the data given $H_0$. If $p < \alpha$, reject $H_0$.

• Errors in Testing:

  • Type I Error: Rejecting a true $H_0$ (False Positive).

  • Type II Error ($\beta$): Failing to reject a false $H_0$ (False Negative).

  • Power ($1 - \beta$): The ability to correctly reject a false $H_0$.

• Effect Size (Cohen's $d$): Measures practical significance. $\text{Small}=0.2, \text{Moderate}=0.5, \text{Large}=0.8$.

  • $d = \frac{\bar{X} - \mu}{\sigma}$

Chapter 8-10: t-tests and Interval Estimation

• The t-statistic: Used when the population standard deviation ($\sigma$) is unknown and must be estimated using the sample standard deviation ($s$).

  • $t = \frac{\bar{X} - \mu}{s_{|\bar{X}}}$

• Degrees of Freedom (df): For t-tests, $df = n - 1$. As $df \to \infty$, the t-distribution becomes the normal distribution.

• Confidence Intervals (CI): A range of plausible values for the population mean.

  • $CI = \bar{X} \pm (t^* \times SE)$

  • Hypothesis Testing via CI: If the CI does not bracket the null value, the result is statistically significant.

• Repeated Measures (Dependent Samples) t-test: Analyzes change within the same subjects over time or matched pairs.

  • Difference Scores ($X_D$): calculated as $\text{Time 2} - \text{Time 1}$.

  • $H_0: \mu_D = 0$

• Independent Samples t-test: Compares means of two unrelated groups (e.g., Treatment vs. Control).

  • Pooled Variance ($s_p^2$): Weighted average of variances from both groups. $s_p^2 = \frac{SS_1 + SS_2}{df_1 + df_2}$

  • Degrees of Freedom: $df = n_1 + n_2 - 2$.

Chapter 11-13: ANOVA, Correlation, and Regression

• ANOVA (Analysis of Variance): Tests for differences between three or more group means without inflating Type I Error.

  • F-statistic: Ratio of between-groups variance (systematic) to within-groups variance (random error).

  • Null Hypothesis: $H_0: \mu_1 = \mu_2 = \mu_3$

  • Effect Size ($\eta^2$): Variance explained. $\eta^2 = \frac{SS_B}{SS_T}$.

  • Post Hoc Tests: (e.g., Tukey's HSD, Bonferroni) Conducted after a significant ANOVA to find which specific means differ.

• Correlation (Pearson's $r$): Measures the strength and direction of the linear relation between two continuous variables.

  • Range: $-1.00$ to $+1.00$. $0$ = No relation.

  • Covariance: How two variables vary together. $cov_{XY} = \frac{\sum (X - \bar{X})(Y - \bar{Y})}{N - 1}$.

  • Correlation vs. Causation: Correlation does not prove causation due to potential lurking/confound variables.

• Linear Regression: Predicting a Y-value from an X-value using the Line of Best Fit ($\hat{Y} = a + bX$).

  • Slope ($b$): Change in Y for every $1$-unit change in X.

  • Standard Error of the Estimate: Average distance between predicted and actual scores.

Chapter 14: Chi-Square ($\chi^2$)

• Non-Parametric Test: Does not estimate population parameters; uses frequency of nominal categories.

• Goodness-of-Fit: Measures how well one variable's observed frequencies match expected frequencies (usually equal distribution).

• Test for Independence: Uses a contingency table to see if two categorical variables are related.

• Formula: $\chi^2 = \sum \frac{(O - E)^2}{E}$.

• Effect Size (Cramer's V): A correlation coefficient for categorical data. $\text{Small}=0.1, \text{Medium}=0.3, \text{Large}=0.5$.