Experimental Methodology

Conceptual definition

Definition of a variable at a theoretical level

operational definition

how a researcher wants to measure or manipulate a conceptual variable

categorical variables

variables with categories as levels, normally just qualitative, if they contain numbers they have no numerical meaning

quantitative variables

variables with meaningful numbers representing amount or degree

ordinal scale 

numbers represent ranked order, intervals are not necessarily equal

interval scale

numerals represent equal intervals between levels and have no true zero

ratio scale

numerals have equal intervals and a true zero (none of the variable is present)

reliability

consistency of results across time or observers

validity

whether the measurement actually assesses what it intends to meaurete

test-retest reliability

consistency of scores each time the same test is administered

interrater reliability

consistency of scores across different observers 

internal reliability

consistency of responses across multiple items measuring the same construct

cronbach’s alpha

a coefficient combining inter-item correlations; closer to 1.0 indicates higher internal reliability

average inter-item correlation

average correlation among items, 0.15-0.5 indicates good consistency

face validity

measure appears to assess the intended variable on the surface

content validity

measure captures all parts of a defined cosntruct

criterion validity

extent to which a measure is related to an outcome it should be related tosc

known groups paradigm

scores on the measure differ among groups known to vary on the variable of interest co

convergent validity

measure correlates strongly with measures of similar constructs

discriminant validity

measure does not correlate strongly with measures of different constructs 

forced choice questions 

respondents pick the best of two or more optionsl

likert scale

degree of agreement scale with labeled response values

semantic differential format

numerical scale anchored with adjectives

open-ended questions,

allow respondents to answer freely; rich but time consuming to code

leading question 

wording influences respondents toward a particular answer 

double-barreled questions

contain two questions in one, confusing interpretation

negatively worded questions

include negative phrasing that can confuse respondents

question order effects 

earlier questions can affect responses to later ones 

response sets

tendency to answer all the questions the same way

acquiescence

agreeing with every item instead of responding thoughtfully f

fence sitting 

choosing middle or neutral options to avoid extremes 

socially desirable responding

giving answers that make oneself look good

faking bad 

providing answers that make oneself look worse 

flashbulb memories

vivid but often inaccurate memories of significant events

observer bias

observers’ expectations influence their interpretation of behaviors

observer effects 

observers unintentionally affect participant behavior 

reactivity

participants alter behavior when they know they are being observed

codebook

detailed guide for observers on how to code behaviors

masked design

observers are unaware of study hypothesis or participant conditions 

interview

structured conversation to learn participants perspectives

focus group

group interview (6-10 people) discussing a shared experience or topic

external validity 

extent to which results generalize to populations beyond the sample 

population

entire group of interest

sample

subset of the population studied

census

studying an entire population 

population of interest

specific population relevant to the research question

biased sample

sample that excludes some population members systematically

unbiased sample 

sample where every member has an equal chance of inclusion 

convenience sampling

using participants who are easiest to access

self-selection

participants volunteer themselves, often leading to bias

probability sampling

random methods ensure all members have equal chances 

simple random sampling

completely random selection from the population sy

systematic sampling

selecting every nth member after a random start

cluster sampling 

randomly select clusters and include all individuals within them 

multistage sampling

randomly select clusters, then individuals within those clusters

stratified random sampling 

randomly sample within identified demographic groups 

oversampling

intentionally overrepresenting certain groups

weighting

adjusting data to correct for underrepresented groups

purposive sampling 

selecting specific individuals with certain characteristics 

snowball sampling

participants recruit acquaintances

quota sampling 

selecting a target number for each subgroup nonrandomly 

frequency claims

claims about how often something occurs; rely heavily on external validity

sample size

affects statistical validity more than external validity

association claim

describes a relationship between two measured variables 

bivariate correlation

association involving exactly two variables

scatterplot

graph used to visualize relationships between two quantitative variables

bar graph

graph used when one variable is categorical; shows mean differences 

effect size

magnitude of a relationship between variables

outlier

extreme score differing greatly from others in the sample

restriction of range

lack of full score range that can weaken correlations 

curvilinear association

nonlinear relationship between two variables

covariance

two variables are related (first condition for causation)

temporal precedence 

cause precedes effect (second condition for causation)

directionality problem

uncertainty about which variable causes which

internal validity

no alternative explanations (third condition for causation)

third-variable problem

hidden variable explaining the observed relationship between

spurious association

apparent correlation due to subgroup differences

moderator

variable that changes the strength or direction of an association