231 Final - Quantitative Research Designs and Methods

Causality

the relationship between cause and effect

Causes are

rarely deterministic, they only increase the likelihood that an event will occur

effect

the difference between what did happen with an exposure and what would have happened without it

research questions

drive the design of the study

experimental designs (RCT)

best possible design for showing causal relationships "gold standard"

A true RCT is characterized by

Intervention, Control, Randomization

Disadvantages to RCT

Often not feasible or ethical

Often very expensive

Quasi-experimental designs

testing an intervention but lack randomization and sometimes lack control group ex: non-equivalent control group design, one group pre-test/post-test

Non-equivalent control group design

those getting the intervention are compared with a non-randomized comparison group

One group pre-test post-test study

one group is studied before and after intervention

Advantages to quasi-experimental design

More practical: ease of implementation

More feasible: resources, subjects, time, setting

may be the only way to evaluate the effect of the independent variable of interest

disadvantages to quasi-experimental design

Difficult to make clear cause and effect relationships

Some generalizability, but more limited conclusions

Non-experimental designs

used when researchers cannot (or should not) manipulate the independent variable

correlational research

interrelationships across variables are explored with no researcher intervention

correlation is.....

not causation

types of non-experimental designs

observational cohort studies, descriptive research

observational cohort studies

researchers recruit sample of people with two distinct characteristics (ex. smokers and non smokers) and observe difference in variable (ex. lung cancer)

it is hard to draw conclusions in these types of studies as there is often confounding variables (ex. lifestyle of smokers vs non smokers)

descriptive research

researchers recruit a sample of people and observe the relationship between variables

advantages to non-experimental studies

efficient way to collect large amounts of data when intervention and/or randomization is not possible

this type of research is often needed to inform interventions

disadvantages to non-experimental studies

no persuasive evidence for causal relationships

cross-sectional studies

data are collected at a single point in time

cross sectional studies

do not provide persuasive evidence for phenomena that change over time

longitudinal designs

data collection over multiple time points

challenge to longitudinal designs

attrition- loss of participants over time

takes a lot of time and money and devoted research participants

strategies for controlling the study context

blinding, standardized communication, intervention protocols

Controlling Participant Factors

randomization, homogeneity, matching, statistical control

randomization

a process of randomly assigning subjects to different treatment groups

homogeneity

only people who are similar with respect to confounding variables are included in the study

matching

consciously forming comparable groups

statistical control

controlling of confounding variables statistically

random assignment

the most effective approach to controlling confounding variables

cross over designs

involves exposing people to one treatment first, then crossover to another treatment. Only a true experiment if people are randomly assigned to different orderings of treatment

statistical conclusion validity

The strength of the evidence that a relationship between variables exists

Statistical tests used to support inferences about relationship between IV and DV

Threats to statistical conclusion validity

poor statistical power (sample size too small)

Independent variable not powerful (poor intervention implementation) (difference between intervention and control not detectable with statistical tests)

internal validity

the strength of the evidence that the independent variable is what causes the change in the dependent variable

ALL ABOUT CONTROL (was the actual intervention the thing that affected the dependent variable)

threats to internal validity

Temporal ambiguity, selection threat, history threat, maturation threat, mortality/attrition threat

temporal ambiguity

Lack of clarity on if the IV preceded the DV (timing)

Selection Threat

bias arising from pre-existing differences between groups being compared (biggest threat to internal validity)

history threat

other events co-occurring that could influence the outcomes (observer bias, reactivity)

maturation threat

changes that occur simply as a result of time

mortality/ attrition threat

participants who are lost from the study are different from those who stay in the study

external validity

the strength of the evidence that the observed relationships are generalizable across peoples, settings and time.

Crucial for EIP (we want to generalize evidence from controlled quantitative studies to real world settings)

Very regimented and controlled studies often don't transfer to a clinical setting (not realistic)

threats to external validity

inadequate sampling, lack of replication in diverse settings

for external validity

you want samples that are representative of the general population

enhancing internal validity

can decrease external validity (generalizability)

construct validity

the likelihood that the theoretical constructs of interest are the ones being captured

Are the researchers actually manipulating and measuring what they intent to?

ex: can a pain score differentiate the facial expressions of a baby getting a vaccine vs a diaper change

threats to construct validity

if the intervention is not a good representation of the underlying construct

if awareness of the intervention led to the benefits

if the measures used are not actually measuring the construct of interest

populations

the entire group of interest. Characteristics of the population specified through eligibility criteria

eligibility criteria

inclusion and exclusion criteria

sample

a subset of the population. Goal is to have sample that is representative of the population

sampling bias

when there is a systemic over or under representation of key characteristics of a population

sampling designs in quantitative research

non-probability and probability

non-probability sampling

researchers select the participants in the study using non-random methods

less likely to produce representative samples

* most common sampling approach in nursing research

Probability sampling

random selection from the population

all participants have an equal chance of being selected

types of non-probability sampling

convenience, purposive, consecutive

convenience sampling

the most convenient and available people

weakest form of sampling

highest risk of sampling bias

consecutive sampling

selecting all people from an accessible population

typically over a specific time period or to a specific sample size

often used in retrospective studies, what happened in the past

purposive sampling

using knowledge about the population to hand-pick sample members

* can lead to bias

types of probability sampling

simple random, stratified random, systematic

simple random sampling

random selection of participants in a broad sampling frame

* low risk of sampling bias

stratified random

sampling frame divided into strata (subgroups) and participants randomly selected within the strata

* goal is to promote representative sample

systematic sampling

Selecting every nth member of the population

you can produce a similar result to simple random sampling

general sample size rule in quantitative research

larger sample size = less risk of sampling bias

* large sample cannot correct for poor research design, but a large non-probability sample is better than a small one

power analysis

statistical technique used by researchers to estimate how large the sample should be in order to adequately test a hypothesis

data sources for quantitative research

questionnaires, Likert scale, visual analogue scale, category systems, rating systems, bio- physiologic

questionnaires

good for accessing large amounts of information, and information from geographically dispersed samples. They are also low cost and offer anonymity

Likert Scale

consists of declarative statements the express viewpoint, and respondents are asked to indicate how much they agree or disagree

Visual analogue scale

a straight line, the ends are labelled as extreme limits of the sensation being measured

Advantages of scales

permit researchers to efficiently quantify subtle change in the intensity of individual characteristics. Can be used for most people

Disadvantages of scales

social desirability response bias, people may give answers that are consistent with societal views

extreme response set bias: tendency to express extreme attitudes

Acquiescence response set bias: yes-sayers and nay-sayers

Observational data sources

category systems, rating systems

category systems

Formal systems for systematically recording the incidence or frequency of prespecified behaviors or events.

categories must be explicit and explained

used to construct a checklist; how observers record observations

rating systems/rating scale

observers rate phenomena along a descriptive continuum. Can be used as an extension of checklists, in which the observer records not only the occurrence of some behavior but also some qualitative aspect of it such as intensity

Bio-physiologic measures

in-vivo and in-vitro

in vivo

measurements performed directly within or on living organism, such as BP or temperature

In vitro

measures in which data is collected by extracting bio physiologic material from them and subjecting it to analysis by laboratory technicians

advantages to bio-physiologic measures

objective data, not interpretable, cannot be distorted by participants

Reliability

the extent to which scores are free from measurement error and are consistent over time for people who have not changed

test-retest reliability

often referred to as "stability" or "reproducibility"

extent to which same scores can be obtained on repeated administration when train being measured has not changed

inter-rater reliability

how reliably the measures reflect the attribute of the person being assessed (and is not related to characteristics of the reviewer)

important for observation methods

Internal consistency

when a measure has multiple components that produce similar measurement of the same trait

correlation coefficient

a statistical index of the relationship between two things (from -1 to +1)

validity

the extent to which a test measures or predicts what it is supposed to

face validity

if the measure appears to be measuring the target

not the best measure of validity

ex: measuring depression may look like measuring sleep, mood, motivation

content validity

extent to which the instruments content adequately measures the construct

often achieved through expert consultation and consideration of the literature on a construct

beyond face value, more concrete and valad

criterion validity

extent to which scores on a measure correlate the the "gold standard" measure

not possible for all measures

ex: the "gold standard" measure for testing infant pain is the PIP score, a researchers methods should be similar to this