BBH Exam 3

sample weights

allows us to accurately generalize when some subgroups are under/over represented in the sample

sampling error

difference between sample and population of interest

N (sample weight equation)

total size of population of interest

n (sample weight equation)

total sample size

F (sample weight equation)

n/N
total sampling fraction

f (sample weight equation)

stratum weight

F/f (sampling weight equation)

sampling fraction
weights

F= n(total)/N(total)
f= n1/N1 & n2/N2
weight= F/f

sampling equations

Telephone surveys

dominant in 80s and 90s

Joseph Waksberg (Warren Mitofsky) method of random digit dialing

"radical idea" in the 1960s
standardized area codes & 7 digit numbers

most distance calls at the time were through operator

not expected to take over personal/mail surveys/interviews

facilitated by computers

John Zogby poll of 1966 election

1,561 adults of 82,000, 1,333 registered to vote
landlines randomly selected from 82,000 person list
random digits selected to get listed and unlisted numbers

cellphone numbers generated in similar process

samples were then combined and adjusted to have representative ratios

predicted 8% margins when others said 11-18%

Key components of Survey research

objectives/research question

Field preparation

sampling design

instrument construction (questionnaire)

Data collection

analysis

National Health Interview Survey (NHIS)

principal source of civilian health info in US
established by Congressional Act (1956)
tracks progress towards national public health goals
cross-sectional household survey
diverse populations oversampled

questionnaire construction

critical to connecting theory to empiricism
means of transforming ideas into data
an art not a science

theory of standardized questions

goal is to present uniform approach to asking questions so responses are comparable
small changes in wording or order can substantially affect responses

Likert scale

developed to identify the extent of a person's attitudes/feelings towards international affairs
similar to correlation matrix & cronbach's alpha
Rensis Likert (1932)

considerations with question wording

informed consent
potential audience
educational level
language issues
cultural sensitivity
previous experience

Cluster (area) sampling

stratum usually geographic regions (states, cities, etc.)

clusters randomly selected and may involve multiple stages of selection

decreased cost, increased efficiency

requirements of cluster sampling

clusters well-defined and mutually exclusive

number of individuals per cluster must be known or estimated prior

clusters must be small for cost

selection of clusters to minimize sampling error

homogeneity

individuals in sample have similar measures on important characteristics (bad)
needs undersampling
increased sampling error

heterogeneity

individuals in sample vary on important characteristics (good)
needs oversampling
decreased sampling error

qualitative data

any info not captured in numerical form
in-depth interviews
directed observation
written documents

methods of qualitative data

one on one interviews
case study research
record keeping
process of observation
ethnographic research
focus groups

ethnographic research

studying a cultural phenomena from the perspective of the subject of study

coding

process through which data are chunked, conceptualized, and integrated to form a theory

community-based participatory research

participants are the experts and tell researchers what to study

bottom-up

Difference between qualitative research and other research

other research requires objectivity when analyzing data

Researchers instead must use their world view to interpret and describe the data

qualitative researcher characteristics

ability to take a step back
recognizes personal bias
flexibility
open to criticism and other's ideas
sensitivity to the needs of others
creativity

qualitative techniques

naturalistic observation
key-informant interview
focus groups
photo voice

naturalistic observation

observing subjects in natural environment
environment cannot have been set up or artificial

key-informant interview

loose structured interview/conversation with experts in the interested field of interest

originally used to understand cultures

focus groups

group of people (usually from specific population) asked about their perceptions, opinions, beliefs, and attitudes on a product, service, concept, etc.,
usually 5-7 people

photo voice

participants take pics to represent their community or POV and then share and discuss together, develop narratives and conduct outreach or other action

steps in coding

1. find common themes in responses
2. naming the codes/sort responses
3. have another research check your categorization
4. selecting your quote

phenomenology

how a phenomenon is experienced by participants

qualitative research traditions

ethnography
phenomenology
grounded theory
field research

aspects affecting qualitative research

credibility
transferability
dependability
confirmability

credibility (qualitative research)

results are credible from participant POV

confirmability (qualitative research)

degree that others can confirm/corroborate results

dependability (qualitative research)

description by the researcher of changes in the context that the research that could affect conclusions

Margaret Mead (1901-1978)

Cultural anthropologist

Pioneer in qualitative research?

Reported attitudes on sex in South Pacific and Southeast Asian traditional cultures which influenced sexual revolution in 60s

Proponent of broadening sexual mores in the context of Western religious life

disadvantages of focus groups

potential strong influence from the moderator
difficulty separating individual viewpoints from collective group point
difficulty obtaining representative sample
individuals may be less willing to reveal sensitive info
may have positive/negative effects on participants

James Lind (1716-1794)

experimented with sailors and found solution to scurvy (vitamin C)

Captain cook took his advice and his crew stayed healthy on 4 year journey

one of the first experiments with classic design methodology

"Design of Experiments"

Book by RA Fisher(1935)
described features of experimental research designs that are still used today
compare experimental group with control
random assignment
requires multiple measures and testing

Internal validity question

did the treatment cause a change in behavior?

Threats to internal validity

selection bias
history
maturation
testing
instrumentation
mortality
regression

Lind's treatment groups

cider
sulfuric acid
vinegar
seawater
oranges & lemons
barley

no reason for some, used what he had

John Snow

Pioneer in studies of cholera in 1800s

stood strong that it was transmitted through contaminated water, and not miasmas as believed (he was right)

Proved that outbreak in Soho was due to water pump being contaminated with fecal matter

Reasons for double-blind studies (doctors)

When administering new treatment
- more optimistic
- evaluate health more favorably
- communicate positive expectations which causes patients to want to meet these expectations

Reasons for double-blind studies

more optimistic
minimize problems, more weight on positives
take better care of themselves
comply with experiment design
Hawthorne effect

Advantages of RCT

random assignment minimizes confounding

control of introduction of treatment clarifies direction and increases strength

pre- and post-testing controls for time-related validity

greater flexibility

more powerful statistical analysis

only research design, in theory, that can prove causal relationships

Disadvantages of RCT

biggest: lack of external validity

can't always be extrapolated

patients consenting and completing are very select

phase 1 RCT

checking for safety
10-20 healthy volunteers
There may be unexpected effects

Phase 2 RCT

checking efficacy
roughly 200 subjects
most fail in this phase

Phase 3 RCT

mainly testing effectiveness
confirm findings in large population
over 1,000 subjects
likelihood of detecting rare side effects increases

Phase 4 RCT

Mainly testing effectiveness
testing long-term safety in diverse population
testing occurring outside of clinical setting

preclinical phase RCT

Laboratory research determines if treatment is safe

COVID 19 Efficacy

results suggest high efficacy rates
Moderna- 94.5%
Pfizer and BioVTech- 95%
Johnson & Johnson- 66.3%
AstraZeneca- 1 showed 70% another showed 90%

Vaccine efficiacy

how well a vaccine performs in ideal (clinical) conditions
high on internal validity, low on external
RCT patients

Vaccine effectiveness

how well a vaccine performs in real world conditions
general patient population
routine clinical care
high on external validity, low on external

between-subjects design

each treatment is administered to a different group of subjects

between-subjects design advantages

minimizes learning and transfer across conditions
shorter sessions
easier to set up, especially when using multiple independent variables

within-subjects design

a single group of subjects receives all treatments, one at a time

within-subjects design advantages

fewer participants (cheap)
increased chance of discovering differences among conditions
randomization counteracts order effects and minimizes transfer and learning
minimizes error variance

error variance

variability in scores caused by something other than independent variable

ways to reduce error variance

rigid lab setting
increase effectiveness of independent variable
randomize error variance across groups

Hawthorne effect

theory that the behavior of an individual or group will change to meet the expectations of the observer if they are aware that their behavior is being observed

Discovery of the Hawthorne Effect

1924-1932
Harvard researchers studied effect of lighting on worker productivity, in Hawthorne plant

productivity increased with all treatments

increased productivity was due to psychological stimulus of being made to feel important, not the lighting

factorial designs

research designs with multiple treatments

interaction

An effect that occurs when differences on one factor depend on which level it is on another factor

Null outcome

a situation in which the treatment or intervention has no effect

Amazon Mechanical Turk (AMT)

crowdsourcing website where workers find variety of jobs and opportunities to participate in research

workers very different from general population

still viable alternative to traditional subject pools

nonrandom sampling and internet research

participants know how to navigate internet (differs from general population

less drawbacks than critics admit

can be problematic with sensitive issues

nonrandom sampling and animal subjects

nonrandom samples are general practice

most ordered from same supplier and are of the same age, sex, color, etc.

Importance of nonrandom samples in psychological research

not particularly important
most findings are applied indirectly anyways

field research

take all equipment and staff to participant's natural environment to conduct study

almost same controls as lab experiment but subjects are chosen based on who is available when you are present

single-subject design

similar to within-subjects design but effects are not averaged and the researcher instead focuses on one subject or a small group of subjects

ways to increase effectiveness of the independent variable

ensure that treatments are strong enough to produce results in subjects

ensure dependent variable is sensitive enough to detect effects

Randomized two-group design advantages

simple to carry out
relatively few subjects
no pretesting or categorization of subjects
fairly simple statistical analysis

randomized two-group design disadvantages

little information about the function of the independent and dependent variable

randomized multigroup design

randomized groups design with three or more groups and levels

parametric design

manipulating independent variable quantitatively

nonparametric design

manipulating independent variable qualitatively

multiple control group design

when a single control group isn't adequate to rule out alternative explanations

matched-groups design

matched sets of subjects are distributed (pairs split up) at random into different treatment groups

matched-groups design advantages

control over subject variables
may be able to demonstrate an effect with fewer individuals saving time and money

matched-groups design disadvantages

may be hard to detect effects of independent variable if matched characteristic has no effect

inferential statistics weaker than unmatched designs

pretesting and matching. make it more time consuming

may have to extend subject pool

within-subjects design disadvantages

more demanding on subjects
can take a long time
difficult to find willing subjects

carryover effect

exposure to a treatment alters the participants behavior subsequent treatment

sources of carryover effects

learning
fatigue
habituation
sensitization
contrast
adaptation

contrast (carryover source)

when subjects can compare what they receive in one treatment to what they receive in another

solution to carryover effects

counterbalancing

counterbalancing

assigning various treatments in a different order for every subject

factorial designs

designs with two or more independent variables

main (marginal) effect

effect of a singular independent variable on the dependent variable averaged across the levels of other independent variables

simple main effect

effect of an independent variable at a given level of the other independent variable

interaction

the combination effect of two or more independent variables on the dependent variable

factorial within-subjects design

each subject is exposed to every combination of all the factors

higher-order factorial design

factorial design including more than two independent variables

higher-order factorial design disadvantages

Lots of subjects required (expensive)
complexity of potential interactions (difficult to analyze)

* most investigators stick to a maximum of 3 independent variables *

randomized block design

experimental design where subjects who share relevant characteristics are grouped together and then randomly assigned to a treatment group