Research Design Vocab

Falsifiability

New discoveries can overturn old findings.

Positivism

Generating and testing theory, accumulating facts, and searching for truth.

Interpretivism

All “facts” are social, there is no real truth.

Democratic Backsliding

A decline in the quality of democratic governance, often involving reduced citizen rights, weakened institutions, and less government accountability. 

Deductive Research

Process moves from theory to hypothesis to data. (top-down approach)

Inductive Research

Process moves from data to empirical patterns to theory. (bottom-up approach)

Research Design

Overall plan of a study for collecting and analyzing data. 

Units of Analysis

Cases or entities that are being studied.

Independent Variable

Variable expected to account for variation in the dependent variable. (explanatory variable)

Dependent Variable

Variable to be explained by the research. (outcome variable)

Causality

Established with association, temporal order, non-spuriousness, and a plausible mechanism.

Non-spuriousness

The association is not caused by an outside third variable.

Plausible Mechanism

Some realistic way for the independent variable to actually influence the dependent variable. 

Conceptualization

Clarify and define concepts.

Operationalization

Switch from abstract concept to something observable and measurable.

Manipulated Experiment

When variables in an experiment are treated to produce results.

Measured/observational Experiment

Data that is naturally collected rather than manipulated.

Nominal Measurement of Data

Classification

Ordinal Measurement of Data

Classification and rank order

Interval Measurement of Data

Classification, rank order, and equal intervals (ex: numbers)

Ratio Measurement of Data

Classification, rank order, equal intervals, and nonarbitrary zero (ex: percentages, rates)

Validity

Accuracy; measuring what you want to measure; getting the right answer.

Subjective Validity

Seems valid on the “face” of it.

Convergent Validity

Compares well with other well-established measures of same concept.

Construct Validity

Associated with other variables (measures of other concepts) in expected ways.

Reliability

Consistency; getting the same answer across time and contexts.

Test-retest Reliability 

Same answers from same persons (or units) at different times.

Alternative Forms

Comparable answers to different phrasing.

Inter-rater Reliability

Same answers from different observers.

Systematic Error

Error that is consistently biased in one direction; cased by factors that influence the process of measurement or concept being measured.

Random Error

Unpredictable error; without a consistent direction; caused by temporary and chance factors. 

Generalizability 

The extent to which the findings from a particular study hold true to the larger population outside of the study. 

Target Population

The exact population to which the research wants to generalize.

Probability Sampling

The probability of selection into sample is known for all members of the population.

Simple Random Samples

Number of cases selected at random, probability is the same for every case in the population. 

Stratified Random Sample

Members of the population are divided into strata (homogenous groups) based on some relevant characteristic, a sample frame is created for each strata.

Cluster Samples

Identify “clusters” of cases, such as geographic units, make a sampling frame of all clusters, randomly or proportionately select some of the clusters. 

Clusters

Naturally occurring, diverse groups of cases in the population.

Non-probability sampling

Probability of selection into sample is NOT known for members of the population.

Snowball Samples

Rare or hard to talk to individuals using social networks.

Theoretical sample

Used to develop aspects of emerging theory.

Saturation

The point at which new data stops providing new information.

Sampling Frame

List of all possible members of the target population.

Best number of samples to gather in order to produce valid results?

1500-2500

Coverage Error

Sampling error due to mismatch between target population and sampling frame (example of systematic error).

Belmont Ethical Principles

Respect for persons, beneficence, and justice.

Hawthorne Effect

People behave differently when they know they are being observed.

Institutional Review Board (IRB)

Panel of judgement that participant’s rights are protected and comply with federal regulations; exist at nearly all U.S. colleges and universities (and other institutions).

“Backfire Effect”

Corrections to misinformation can actually strengthen misperceptions among the most strongly committed subjects.

Counterfactual

If x=y, then -x=-y (what would happen if the cause didn’t happen).

Sampling

The process of defining and selecting cases.

Mills Method of Agreement

select cases that have same value of outcome variable (all positive cases)

Mills Method of Difference

compare positive and negative cases

Most Similar System

choose cases that are similar in all confounding (control) variables but differ on values of a key explanatory variable

Most Different System

choose cases that take on very different values of multiple independent variables.

Most Likely Case Design

if the theory would work anywhere, it would work here.

Least likely Case Design

if the theory works here, it should hold everywhere.

Process Tracing

look for intervening causal mechanisms linking identified variables to the event of interest

Cross-sectional Surveys

Data collected at one point in time

Retrospective Cross-sectional Surveys

Data collected at one point in time, asking about the past

Longitudinal Surveys

Data collected at more than one point in time

Trend Study

repeated cross-sectional surveys with different samples; shows change in time over a population

Panel Study

repeated surveys of the same sample; shows change over time in individuals

Metadata

additional data that can be drawn from respondent’s reactions to questions, etc.

Intensive in-depth Interviews

each interview is tailored to each respondent

Structured in-depth Interviews

each respondent is asked same basic set of questions

Interview Guide

list of topics and specific questions

“Grand Tour” Question

asks for a general description of the people, processes, or events being studied

Ethnography

research method in which social phenomena and processes are observed in their natural setting as they happen

Emic Perspective

truly understand insider’s perspective

Etic Perspective

still be the outsider who can see the big picture

Reflexivity

practice of reflecting on how a researcher’s own characteristics may shape the research

Non-reactive Measures

measures in which the subject does not know they are being measured

Ecological Fallacies

making claims at one level of analysis, but collecting data at another level of analysis

Content Analysis

set of procedures used to systematically analyze symbolic content of usually pre-existing texts, images, and communications to make inferences about the senders, messages, and audiences

Open Coding

What themes appear in a subset of data?

Focused Coding

Narrow down themes from the open coding into a set of focused codes to apply to the data.

Attribute Coding

Code content based on non-thematic characteristics.

Codebook

definitions and lists of codes

Dictionary

particular words, phrases, or other word-based indicators for AI/computers to use in order to code

Mixed Methods

using both qualitative and quantitative methods in the same project

Triangulation

using multiple methods or measures that do not share the same methodological weaknesses to address the same research question

Four purposes of multiple methods design:

triangulation, complementarity, development, expansion

External Validity

extent to which experiment findings are generalizable to the real world

Equifinality

the same outcome can come from multiple cases

Multifinality

one cause can lead to multiple outcomes

Experimental Studies

when the researcher deliberately manipulates the potential cause and then observes the effects of those manipulations

Observational Studies

when the researcher only observes what is already going on

Tendencies

mean, median, mode

Variance

standard deviation, histograms

Correlation

how related are the data within the data set

β (Beta)

the slope of the line that best fits the pattern of data we used to estimate the equation (should consider the sign, sureness, and size of each β)

Positive (direct) Relationship

increase in x = increase in y

Negative (inverse) Relationship

increase in x = decrease in y

Sureness

when p value is less than 0.5

Reverse Causation

the dependent variable influences the independent variable