Research Methods: Surveys, Observation, and Analysis

Survey/Poll

A method of asking people questions to measure a variable of interest.

Open-Ended Survey Format

Respondents answer any way they like; provides rich info but requires time-consuming coding.

Forced-Choice Survey Format

Respondents choose the best of two or more options; easy to score but may lack true representation.

Likert Scale Survey Format

Respondents use a rating scale to reflect their degree of agreement with a statement.

Semantic Differential Format

Respondents rate a target object using a numerical scale anchored by adjectives.

Leading Question

A problematic question whose wording encourages one response more than others.

Double-Barreled Question

A problematic question that asks two questions at once.

Negatively Worded Question

A question with negatively phrased statements, making it confusing and difficult to answer.

Acquiescence (Yea-Saying)

A response set where participants always agree with questions regardless of content.

Fence Sitting

A response set where participants always provide a neutral response.

Socially Desirable Responding (Faking Good/Bad)

Trying to look better or worse than they really are when self-reporting.

Combating Acquiescence

Avoid asking about levels of agreement and use reverse-scored questions instead.

Combating Fence Sitting

Use forced-choice questions or an even number of response options (e.g., 1-4 scale).

Observational Research

Watching people or animals and systematically recording how they behave or what they do.

Observer Bias

When observers' expectations influence their own interpretation of participants' behavior.

Observer Effect

When observers' expectations influence participants' behavior to match those expectations.

Reactivity

When participants change their behavior simply because they know they are being observed.

Codebooks

Clear operational definitions and training for observers to prevent observer bias and effect.

Interrater Reliability

Checking if two different observers agree on how they interpret participants' behavior.

Masked/Blind Design

Keeping observers unaware of the study's purpose, hypothesis, or condition assignments.

Preventing Reactivity in Observation

Blend in, wait it out, or measure the results/traces of behavior.

Bivariate Correlation

Measuring two variables to see if they are associated or related to each other.

Restriction of Range

A statistical validity issue that often leads to underestimating the size of a correlation.

Curvilinear Association

A relationship that cannot be described with a simple straight-line correlation (r-value close to 0).

Three Requirements for a Causal Claim

Covariance, temporal precedence, and internal validity (eliminating alternative explanations).

Moderator

A variable that changes the relationship between two other variables (affects external validity).

Longitudinal Design

Measuring the same variables in the same people at different points in time to address temporal precedence.

Cross-Sectional Correlations

Correlations between two variables measured at the same time in a longitudinal study.

Autocorrelations

Correlations of one variable with itself, measured at two different times.

Cross-Lag Correlations

Correlations between an earlier measure of one variable and a later measure of another, helping establish temporal precedence.

Multiple Regression

A statistical technique that helps address internal validity by controlling for potential third variables.

Criterion Variable

The dependent variable or outcome in a multiple regression that researchers want to predict.

Predictor Variables

The independent variables in a multiple regression thought to relate to the criterion variable.

Standardized Beta

Indicates strength and direction of a predictor's relationship with the criterion when other predictors are controlled.

Interpreting Beta's Significance

Significant if the 95% CI excludes 0 and the p-value is less than .05.

Mediation vs. Moderation

Mediation explains why variables are related; moderation asks for whom or in what situations the relationship is strongest.

Outliers and Statistical Validity

Extreme scores (outliers) that can disproportionately affect the overall correlation and association between variables.

Covariance

The first requirement for a causal claim, stating that variables A and B must be correlated.

Temporal Precedence

The second requirement for a causal claim, stating that variable A must come before variable B in time.

Internal Validity in Causal Claims (Requirement 3)

The requirement that variable A is the only possible explanation for B, eliminating all alternative explanations and third variables.

External Validity

The extent to which an association or study result generalizes to other groups, settings, and conditions.

Construct Validity in Association Claims

An evaluation of how well each variable was measured, checking for good reliability and evidence of face, criterion, convergent, and discriminant validity.

Mediation

why are two variables related?

Moderators

are the certain groups or situations for which the two variables are more strongly related?

Third-Variable Problem

two variables are correlated, but only because they are both linked to a third variable