Comprehensive Study Notes: Quantitative and Qualitative Research Methods in Psychology

Data Screening, Measurement Scales, and Statistical Assumptions

• Measurement Scales:

  • Categorical:

  • Nominal: Discrete categories without inherent quantitative order (e.g., biological sex, eye color, condition assignment).

  • Ordinal: Categories with a meaningful rank order, but unequal or unquantified intervals (e.g., assignment grades such as High Distinction, Distinction, Credit, Pass; birth order).

  • Continuous / Numerical:

  • Interval: Equal numerical units without an absolute true zero point (e.g., temperature in Celsius, Beck Depression Inventory [BDI], Center for Epidemiological Studies Depression [CES-D] scores ranging from $0$ to $60$).

  • Ratio: Equal numerical intervals featuring a true, meaningful zero point (e.g., response time, age, count data).

• Parametric vs. Non-Parametric Frameworks:

  • Parametric Tests: Assess group means and linear parameters; require continuous interval/ratio data, absence of severe outliers, and adherence to distributional assumptions. Supported by the Central Limit Theorem (CLT) when sample size is large ($N > 30$).

  • Non-Parametric Tests: Assess median ranks and ordinal distributions (e.g., Mann-Whitney U, Kruskal-Wallis, Spearman correlation); utilized when data are nominal/ordinal, sample sizes are small ($N < 30$), or parametric assumptions are irremediably violated.

• L.I.N.E. Assumptions for Parametric Models:

  • Linearity (and Additivity): The relationship between predictors and outcome follows a straight line ($Y_i = b_0 + b_1 X_1 + b_2 X_2 + \dots + \varepsilon_i$). Assessed via scatterplots of observed versus predicted values and residual plots (fitted values vs. standardized residuals). Curvature or bow shapes indicate non-linearity.

  • Independence of Observations: Residuals and individual observations are uncorrelated across cases ($\text{Cov}(\varepsilon_i, \varepsilon_j) = 0$). Violation inflates Type I error rates; handled via repeated-measures designs or mixed modeling.

  • Normality: Residual error terms ($\varepsilon_i$) must be normally distributed with a mean of $0$. Evaluated visually using histograms and Normal Q-Q plots (points tracking the diagonal line), or statistically via Shapiro-Wilk ($p > .05$ denotes normality).

  • Equal Variance (Homoscedasticity / Homogeneity of Variance): Constant residual variance across all levels of the predictor or across groups. Evaluated through Levene's test ($p > .05$ met) or fitted-versus-residuals plots (funnel or cone shapes denote heteroscedasticity).

• Outlier Detection and Remediation:

  • Univariate Outliers: Extreme values on a single variable, identified via boxplots ($1.5 \times \text{IQR}$ beyond quartiles) or standardized $z$-scores exceeding $\pm 3.29$.

• Bivariate & Multivariate Outliers: Evaluated using scatterplots, Mahalanobis distance (evaluating joint distance from centroids), and Cook's distance (leverage and influence on regression slopes).

• Remediation Techniques:

  • Trimming/exclusion of verified aberrant cases.

  • Winsorizing: Recoding extreme values to the next highest non-outlier value plus a unit constant, or to values corresponding to $z = \pm 3.29$.

  • Non-linear transformations:

  • Log transformation ($\log(X)$, $\ln(X)$): Stabilizes right/positive skew; defined only for $X > 0$.

  • Square root transformation ($\sqrt{X}$): Moderate compression for positive skew and count data; defined for $X \ge 0$.

  • Reciprocal transformation ($\frac{1}{X}$): Strongest non-linear compression for severe skew; reverses score orders unless adjusted ($\frac{1}{X_{\text{max}} - X}$).

Correlation and Bivariate Relationships

• Covariance vs. Correlation:

  • Covariance measures the directional co-variation of two variables:     $\text{Cov}(X, Y) = \frac{\sum_{i=1}^N (X_i - \bar{X})(Y_i - \bar{Y})}{N - 1}$

  • Pearson Product-Moment Correlation ($r$) standardizes covariance by the product of both standard deviations, yielding a scale-free metric ranging from $-1.0$ to $+1.0$:     $r = \frac{\text{Cov}(X,Y)}{s_X s_Y}$

• Correlation Coefficients:

  • Pearson Correlation ($r$): Measures linear strength and direction between two continuous variables. Assumes bivariate normality and linearity.

  • Spearman Rank Correlation ($\rho$ or $r_s$): Non-parametric coefficient evaluating monotonic associations between ordinal ranks or non-linear continuous variables.

• Shared Variance and Multivariate Extensions:

  • Coefficient of Determination ($r^2$): Quantifies the proportion of shared variance between two variables (e.g., $r = .411 \implies r^2 = .169$, or $16.9\%$ shared variance).

  • Partial Correlation: Correlation between two variables ($X_1$ and $Y$) while simultaneously partialling out (controlling for) the variance accounted for by a third variable ($X_2$) from both $X_1$ and $Y$.

  • Semi-Partial (Part) Correlation ($sr$): Controls for the effect of a third variable on only one predictor, isolating the unique variance contributed by that predictor to the total outcome variance ($sr^2$).

  • Zero-Order Correlation: A raw bivariate correlation with zero control variables partialled out.

• Causal Invariance and Missing Data Handling:

  • Causation Caveats: Correlations do not demonstrate causal inference due to the directionality problem and the third variable (confounding) problem.

  • Pairwise Deletion: Excludes participants missing data on either variable for each specific bivariate pair; sample sizes vary across cells.

  • Listwise Deletion: Excludes any participant missing data across any variable in the entire analytical matrix; maintains consistent sample size across all tests.

Linear, Multiple, and Hierarchical Regression

• Simple Linear Regression:

  • Models the continuous outcome ($Y$) via the line of best fit determined by the ordinary least squares (OLS) criterion, minimizing $\sum (Y_{\text{observed}} - Y_{\text{predicted}})^2$.

  • Regression equation:     $Y_i = b_0 + b_1 X_1 + \varepsilon_i$     where $b_0$ is the intercept ($Y$ when $X = 0$), and $b_1$ is the unstandardized slope ($\Delta Y$ per unit change in $X$).

  • Standardized Slope ($\beta$): Indicates the standard deviation change in $Y$ per one standard deviation change in $X$. In simple regression, $\beta = r$.

• Multiple Linear Regression:

  • Extends simple regression to $k$ predictors:     $Y_i = b_0 + b_1 X_1 + b_2 X_2 + \dots + b_k X_k + \varepsilon_i$

  • Each partial regression coefficient ($b_j$) indicates the unique effect of $X_j$ on $Y$, holding all other predictors constant.

  • Multicollinearity Diagnostics: High intercorrelations among predictors inflate standard errors and destabilize slopes. Measured by:

  • Tolerance ($1 - R_j^2$): Values $< 0.10$ indicate severe multicollinearity.

  • Variance Inflation Factor ($\text{VIF} = \frac{1}{\text{Tolerance}}$: Values $> 10$ indicate severe multicollinearity.

• Model Fit, Significance, and Effect Sizes:

  • Omnibus Model Fit ($F$-ratio):     $F = \frac{MS_{\text{Model}}}{MS_{\text{Residual}}} = \frac{\frac{SS_{\text{Model}}}{k}}{\frac{SS_{\text{Residual}}}{N - k - 1}}$

  • Explained Variance ($R^2$) benchmarks: Small = $0.04$ ($4\%$), Medium = $0.09$ ($9\%$), Large = $0.25$ ($25\%$).

  • Adjusted $R^2$: Corrects for sample size and number of predictors, penalizing model complexity to counter sample overfitting:     $\text{Shrinkage} = R^2 - \text{Adjusted } R^2$

  • Cohen's $f^2$: Measures local effect size of unique predictor variance relative to unexplained residual variance:     $f^2 = \frac{R^2}{1 - R^2}$     Benchmarks: Small = $0.02$, Medium = $0.15$, Large = $0.35$.

• Dummy Coding Categorical Predictors:

  • Categorical variables with $k$ levels require $k - 1$ binary dummy variables ($0$ or $1$).

  • A single un-coded baseline serves as the reference group, represented when all associated dummy variables equal $0$.

• Hierarchical Linear Regression and Model Comparisons:

  • Predictors enter sequentially in pre-specified conceptual blocks to isolate incremental variance contributions ($\Delta R^2$ and $F_{\text{change}}$).

  • Nested Models: A reduced model ($M_0$) is strictly nested within an expanded model ($M_1$) if $M_1$ contains all parameters of $M_0$ plus additions. Evaluated using $\Delta R^2$ and $F_{\text{change}}$.

  • Non-Nested Model Comparisons: Compared using information criteria that penalize log-likelihood ($LL$) for complexity:

  • Akaike Information Criterion: $\text{AIC} = 2k - 2LL$

  • Bayesian Information Criterion: $\text{BIC} = \ln(N)k - 2LL$

  • Lower AIC/BIC denotes superior parsimony and out-of-sample predictive generalizability.

Moderation and Interaction Effects in Regression

• Conceptualizing Moderation:

  • Moderation occurs when the direction or magnitude of the effect of predictor $X$ on outcome $Y$ varies as a function of a third variable, the moderator $M$.

• Forms of Moderation:

  • Enhancing: Higher levels of $M$ amplify the positive or negative association between $X$ and $Y$.

  • Buffering: Higher levels of $M$ weaken or attenuate the association between $X$ and $Y$.

  • Antagonistic: The moderator reverses the sign/direction of the association between $X$ and $Y$.

  • Statistical Formulation:

• Tested by adding an interaction product term ($X \times M$) into the regression model:     $Y = b_0 + b_1 X + b_2 M + b_3 (X \times M)$

• Factoring illustrates the conditioned slope:     $Y = b_0 + (b_1 + b_3 M)X + b_2 M$

• The composite term $(b_1 + b_3 M)$ represents the simple slope of $X$ at a specified level of $M$.

• Self-Moderation (Quadratic Effects): Entering $X^2$ ($Y = b_0 + b_1 X + b_2 X^2$) tests for non-linear, parabolic curvilinear relationships.

  • Centering and Simple Slopes Analysis:

• In uncentered moderation, $b_1$ is the effect of $X$ strictly when $M = 0$, which may be biologically or clinically meaningless.

• Mean-Centering: Subtracting variable means ($X_c = X - \bar{X}$, $M_c = M - \bar{M}$) prior to multiplying product terms makes first-order coefficients interpretable at the sample mean of the other variable.

• Probing Simple Slopes: Conventional testing computes the significance of slopes at the mean of $M$, Low $M$ ($\text{Mean} - 1\,\text{SD}$), and High $M$ ($\text{Mean} + 1\,\text{SD}$).

• Visual Interpretation: Parallel regression slopes indicate the absence of an interaction; divergent, non-parallel, or intersecting lines indicate interaction effects.

One-Way and Factorial Analysis of Variance (ANOVA)

• Overview and Partitioning of Variance:

  • ANOVA partitions total sum of squares ($SS_T$) into model/between-group variance ($SS_M$) and residual/within-group error variance ($SS_R$):     $SS_T = SS_M + SS_R$

  • The $F$-ratio tests the omnibus null hypothesis ($H_0: \mu_1 = \mu_2 = \dots = \mu_k$):     $F = \frac{MS_{\text{Between}}}{MS_{\text{Within}}} = \frac{\frac{SS_M}{k - 1}}{\frac{SS_R}{N - k}}$

  • Alpha Inflation: Conducting pairwise $t$-tests across $C$ comparisons inflates the family-wise error rate ($\alpha_{FW}$):     $\alpha_{FW} = 1 - (1 - \alpha_{DW})^C$     For 3 pairwise comparisons at $\alpha_{DW} = .05$, $\alpha_{FW} = 1 - (.95)^3 = .143$ ($14.3\%$).

• Repeated-Measures ANOVA:

  • The same participants are tested across all levels of the factor. Removes systematic between-subject variability ($SS_{\text{subjects}}$) from the error term:     $SS_{\text{Within-subjects}} = SS_{\text{subjects}} + SS_{\text{error}}$

  • Enhances statistical power by isolating true unexplained experimental noise ($SS_{\text{error}}$) for the $F$-ratio denominator.

• Factorial ANOVA Designs:

  • Evaluates two or more categorical factors simultaneously (e.g., $2 \times 2$ factorial design).

  • Generates $2^k - 1$ separate $F$-tests across main effects and interaction terms:

  • Main Effect: Difference across levels of one factor collapsed (averaged) across all levels of all other factors.

  • Interaction Effect: Occurs when the simple effects of one factor differ across levels of another factor.

• Violations of Assumptions in ANOVA:

  • Unequal Variances: When Levene's test is significant ($p < .05$) with unequal group sizes, standard $F$ is biased. Robust alternatives include Welch's $F$ and Brown-Forsythe $F$.

  • Non-normality: Handled via data transformations or the non-parametric Kruskal-Wallis test.

• ANOVA Effect Sizes:

  • Eta-squared ($\eta^2$): Biased sample estimator overestimating population variance accounted for:     $\eta^2 = \frac{SS_{\text{Effect}}}{SS_{\text{Total}}}$     Benchmarks: Small = $0.01$, Medium = $0.09$, Large = $0.25$.

  • Partial Eta-squared ($\eta_p^2$): For factorial models, isolates effect variance from residual error:     $\eta_p^2 = \frac{SS_{\text{Effect}}}{SS_{\text{Effect}} + SS_{\text{Residual}}}$

  • Omega-squared ($\omega^2$): Unbiased population variance estimator adjusting for mean square error:     $\omega^2 = \frac{SS_M - (df_M)MS_R}{SS_T + MS_R}$     Benchmarks: Small = $0.01$, Medium = $0.06$, Large = $0.14$.

  • Cohen's $d$: Standardized pairwise mean difference utilizing pooled variance ($s_p^2$):     $d = \frac{|M_1 - M_2|}{\sqrt{s_p^2}}, \quad s_p^2 = \frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}$     Benchmarks: Small = $0.20$, Medium = $0.50$, Large = $0.80$.

Planned Contrasts and Post-Hoc Testing

• Planned Contrasts (A Priori Comparisons):

  • Hypothesis-driven comparisons designed prior to data collection. Partitions $SS_M$ into independent components without inflating family-wise error.

  • Contrast formula:     $L = \sum_{i=1}^k \lambda_i \bar{X}_i$

  • Coding Rules for Contrast Weights ($\lambda$):

  1. Weights must sum to zero ($\sum \lambda_i = 0$).

  2. Groups coded with positive weights are compared directly against groups coded with negative weights.

  3. Uninvolved groups receive a weight of $0$.

  4. Weights must balance: the absolute magnitude assigned to one side equals the number of groups on the opposite side.

  • Orthogonal Contrasts: Independent, non-redundant contrasts. A design with $k$ groups allows a maximum of $k - 1$ orthogonal contrasts. Two contrasts are orthogonal if the sum of the products of their corresponding weights equals zero:     $\sum (\lambda_{1i} \times \lambda_{2i}) = 0$

• Post-Hoc Tests (A Posteriori Comparisons):

  • Unplanned pairwise tests conducted following a statistically significant omnibus ANOVA result.

  • Total possible pairwise comparisons for $k$ groups:     $C = \frac{k(k - 1)}{2}$

  • Bonferroni Correction: Controls family-wise error by adjusting the decision-wise threshold:     $\alpha_{DW} = \frac{\alpha_{FW}}{C}$

  • Tukey's HSD: Optimized pairwise comparison for equal variances and group sizes.

  • Games-Howell: Robust post-hoc test applied when the assumption of homogeneity of variance is violated ($p < .05$ on Levene's test).

Qualitative Research Methodology and Trustworthiness

• Qualitative Paradigm and Core Assumptions:

  • Operates primarily through linguistic and contextual data to investigate meaning, subjective experiences, and social processes.

  • Ontological Positions:

  • Critical Realism: A reality exists independently of human observation, but it is accessed and understood through subjective human interpretation.

  • Constructionism / Relativism: No singular fixed social reality exists; realities and knowledge are co-constructed through language, culture, and social interaction.

• Contrasting Methodological Paradigms:

  • Qualitative: Focuses on situated meaning, inductive theory-generation, smaller purposive samples, and researcher subjectivity as an active analytical tool.

  • Quantitative: Focuses on objective measurement, deductive hypothesis-testing, large representative samples, and statistical generalization.

• Sampling Strategies in Qualitative Research:

  • Purposive / Judgement Sampling: Selecting participants intentionally based on predefined criteria relevant to the research phenomenon.

  • Theoretical Sampling: Iterative sampling directed by emerging conceptual and theoretical insights during ongoing analysis (core to Grounded Theory).

  • Snowball / Network Sampling: Existing participants refer subsequent eligible individuals.

  • Convenience Sampling: Selection based on ease of accessibility and recruitment.

• Data Collection Modalities:

  • Semi-Structured Interviews: Flexible, conversational inquiry guided by open-ended prompts, probing questions, and spontaneous elaboration.

  • Focus Groups: Moderated group interactions that elicit collective perspectives and observe interactive peer dialogue.

  • Ethnographic Observation: Naturalistic fieldwork and immersion documenting cultural contexts, physical mapping, and interactions.

• Trustworthiness Framework (Lincoln & Guba, 1985):

  • Credibility: Truth-value and confidence in findings; established via prolonged engagement, triangulation, member checking, and negative case analysis.

  • Transferability: Applicability of findings to alternative settings; facilitated through thick, detailed contextual descriptions.

  • Dependability: Stability and consistency of the analytical process; verified through a documented audit trail and decision tracking.

  • Confirmability: Ensuring interpretations reflect participant accounts rather than researcher bias; supported via reflexivity journals, audit trails, and peer debriefing.

Qualitative Analytical Frameworks: RTA and IPA

• Reflexive Thematic Analysis (RTA; Braun & Clarke):

  • A method for identifying, analyzing, and interpreting patterned meaning across datasets. Prioritizes researcher subjectivity and continuous reflexivity.

• Six Phases of RTA:

  • Phase 1: Dataset Familiarization: Immersive reading and re-reading of transcripts; initial note-taking.

  • Phase 2: Coding: Generating concise, meaningful labels capturing both semantic (explicit surface content) and latent (underlying conceptual meaning) data features.

  • Phase 3: Generating Initial Themes: Clustering shared, patterned codes around central organizing concepts.

  • Phase 4: Developing and Reviewing Themes: Evaluating candidate themes against coded extracts and the entire dataset.

  • Phase 5: Refining, Defining, and Naming Themes: Finalizing conceptual boundaries, generating scope definitions, and establishing descriptive titles.

  • Phase 6: Writing Up: Synthesizing data extracts and analytic narrative into a cohesive, persuasive argument.

  • Interpretative Phenomenological Analysis (IPA; Smith, Flowers, & Larkin):

• A qualitative approach exploring how individuals make sense of major lived life experiences.

• Theoretical Pillars:

  • Phenomenology: The study of human experience as lived.

  • Hermeneutics: The theory of interpretation; operates as a double hermeneutic (the researcher makes sense of the participant making sense of their experience).

  • Idiography: Deep commitment to the particular, requiring exhaustive case-by-case analysis before cross-case aggregation.

• Analytical Hierarchy:

  • Individual transcript immersion and exploratory noting $\rightarrow$ Generating experiential statements $\rightarrow$ Clustering into Personal Experiential Themes (PETs) for single cases $\rightarrow$ Cross-case consolidation into Group Experiential Themes (GETs).