Advanced Methods in Psychological Research - Comprehensive Study Guide

General Linear Model and Foundations

The General Linear Model (GLM) represents a foundational framework in statistical analysis, adhering to a specific structural equation where the outcome of interest is defined by the sum of a linear model and an associated error term. Specifically, the relationship is expressed as: $\text{outcome} = (\text{linear model}) + \text{error}$. Within this framework, it is noted that the model fit is inversely related to the level of error; thus, a higher error does not indicate a better fit, but rather a poorer one. Additionally, the GLM allows for robust significance testing of its parameters.

Regression analysis is categorized as a method primarily used to confirm the structure of data. This stands in contrast to exploratory methods like Exploratory Factor Analysis (EFA), which is explicitly utilized to detect the structure or underlying dimensions within a dataset when the researcher does not have a prior hypothesis about the number or nature of factors.

Experimental Design and Variable Types

Research designs are broadly categorized by how participants are measured across different conditions. An experimental design where different participants are measured under each level of the independent variable (IV) is known as a Between-Subject Design. This differs from a Within-Subject Design, where the same participants are measured across all levels of the independent variable.

In terms of variable classification, a confounding variable is defined as a variable in an experiment that correlates with both the independent variables (IVs) and the dependent variable (DV). This relationship can obscure or bias the actual relationship being investigated between the IV and the DV. Other variable types include moderator variables, which influence the strength or direction of a relationship, and outcome (dependent) variables, which are the primary focus of the measurement.

Psychometric Properties: Validity and Reliability

The quality of an assessment is often evaluated through psychometric properties. Validity refers specifically to the extent to which an assessment measures what it is intended to measure. This is distinct from reliability, which refers to the consistency or stability of a measurement over time. Other concepts like objectivity (freedom from bias) and fairness (just treatment of all participants) are also relevant but serve different conceptual roles in measurement quality.

Statistical Significance and Hypothesis Testing

Statistical significance is determined by evaluating the probability of the observed test statistics under the assumption that the Null Hypothesis ($H_0$) is true. If this probability (p-value) is low (typically below a threshold like $0.05$), the result is considered significant, leading to the rejection of the Null Hypothesis.

General test statistics are often derived from a ratio comparing explained variance to unexplained variance. Specifically, the test statistic is described by the equation: $\frac{\text{variance explained by the model}}{\text{variance not explained by the model}}$.

Regarding effect sizes, there is no direct positive correlation between the effect size $d$ and the probability of an alpha-error ($\alpha$). It is false to state that a higher effect size $d$ increases the probability of committing an alpha-error, which is the error of incorrectly rejecting a true null hypothesis.

Simple and Multiple Linear Regression Analysis

In Simple Linear Regression (SLR), the standardized regression coefficient (often denoted as $\beta$) is identical to the bivariate correlation coefficient ($r$) between the predictor and the outcome. This coefficient can take both positive and negative values depending on the direction of the relationship; it does not always have a positive sign, nor does it always have a negative sign. In the context of simple linear regression, this value is bounded and cannot exceed 1.

Predictor variables in regression analysis can be continuous or nominal. It is entirely possible to include nominal predictors in a regression model. If a nominal predictor has only two levels (e.g., binary), it can be included directly. If it has more than two levels, it must be dummy coded (or otherwise contrast-coded) to be appropriately integrated into the analysis. It is important to note that the inclusion of such predictors does not imply that the relationship between the predictor and the outcome remains unchanged across all contexts.

Error, Variance, and Prediction in Regression

When conducting Ordinary Least Squares (OLS) regression, the residual variance represents the discrepancy between the observed values and the predicted values. This is represented mathematically by the sum of squared differences: $\sum_{i=1}^{n} (y_i - \hat{y}_i)^2$ This equation identifies the portion of the outcome's variance that the model fails to explain.

In a scenario where no predictors are available for a given outcome, the best possible prediction for that outcome is the mean ($\bar{y}$) of the distribution. Any prediction made using the mean will result in an error term, but it serves as the baseline model against which more complex models are compared.

The global significance test for OLS regression involves calculating an F-value. Based on instructional materials, this statistic can be expressed as the ratio of Residual Variance to Total Variance, although in broader statistical contexts, the F-test specifically evaluates Model Variance against Residual Variance to determine if the predictors collectively improve the model fit.

Assumptions and Violations: Heteroskedasticity

A critical assumption in linear regression is homoskedasticity, which requires the variance of the residuals to be constant across all levels of the predictor variable. If this assumption is violated—meaning the variance of the residuals differs at different levels of the predictor—it describes a condition known as Heteroskedasticity. This is distinct from Multicollinearity, which refers to high correlations between predictors, and dependency of observations, which occurs when data points are not independent.