Module 4: Significance of Research Designs + (Causality & AGS Review)

DAGS (Directed Acyclic Graphs)

DAGS (Directed Acyclic Graphs)

• Usage:

  • Visually represent and clarify relationships between variables

    • Helps researcher visualize:

      • How different variables may be related

      • Which variables may be confounding the association of interest

• Based on theoretical and substantive knowledge about the subject matter

Why Would Researcher use a DAG?

• Can help one design a study

• Identify variables in analyses that one needs to control in order to a exclude alternative explanation for the pattern one finds

Brief Overview of Causal Modeling

Relationships between variables X, Y and Z can take different forms

Three common structures:

• (1) Mediation Structure (X → Z → Y)

  • X is the independent variable, Z is the mediator, Y is the dependent variable

  • X influences Y through Z where Z transmits some or all of the effect of X on Y

• (2) Fork Structure (X ← Z → Y)

  • Z is the common cause of both X and Y (Z is a confounder)

  • X and Y are not directly connected (don’t directly influence each other), but share a common ancestor in Z

• (3) Collider Structure (X → Z ← Y)

  • X and Y both (are not correlated) influence Z, but Z does not mediate between them

Why Does a Mediation Structure Matter?

• Not accounting for the mediator can:

  • Result in falsely attributing the entire effect of the cause directly to the outcome

  • Lead to an overestimation of the direct effect of the independent variable on the dependent variable

  • We may miss important intervention points that could be targeted for improving outcomes

Why Does a Fork Structure Matter?

• Spurious correlation: Without account for the common cause, two variables (X & Y) might appear related when, in related, their association to due to the confounder (Z)

• Overestimating Direct Effects: Forks can make us overestimate the direct effect between two variables by not recognizing the role of a 3rd factor (the common cause)

Why Does A Collider Structure Matter?

• Spurious Association: If we condition on the collider (eg. by selecting individuals who share the collider outcome)m we may incorrectly conclude that two variables are associated when they are not

  • X and Y can both independently affect Z

    • X and Y are not linked, just that that they affect separately!

• Bias in Results: When researcher mistakenly control for a collider, they introduce boas often referred as collider bias/selection bias

Research Design

Structured framework that outline how a research study will be conducted to answer a specific set of questions or hypotheses

• Provides a detailed plan specifying methods and procedures for collecting and analyzing required information

• RD because it affects:

  • (1) Reliability, (2) Validity, (3) Generalizability of study findings

How Does Research Design Help the Researcher?

• Address Research Questions

  • Clear path to answer research questions

  • Test hypotheses set out at the beginning of studies

• Validity

  • Study measures what it intends to measure

  • Minimizes biases and other errors that can invalidate the study

• Reliability

  • Study has consistent and repeatable results

    • Copyable where other researchers will yield the similar results if conducted

• Generalizability

  • Results are generalizable to broader population and different contexts

Types of Validity

• Construct Validity

• Content Validity

• Internal Validity

• External Validity

Construct Validity

Actually reflects the true theoretical meaning of concept

• eg. self-esteem, not another related construct such as confidence

Content Validity

Assures that a measure covers all aspect of a given construct

• eg. depression scale measuring all facets of depression, not just sadness

Internal & External Validity

Internal Validity

• Concerned with the results of a study and their attribution to the independent variable, and not some other factor

External Validity

• Generatability of research results

Descriptive Research Designs

• (1) Case Reports

• (2) Case Series

  • Do a great job at answering all the Ws except the why

Case Reports (Descriptive) → Interchangeable with Case Studies

Article that describes and interprets an individual case that is written like a detailed story

• First line of evidence, but also lowest level

  • New issues and idea come out of here (low on hierarchy pyramid)

• Good case report will always be clear about the importance of the observation being reported

Case Studies Advantages

• Assist in:

  • Identification of new trends and diseases

  • Detect new drug side effect and potential uses (Adverse or beneficial)

• Educational

• Identify rare manifestation in disease

Case Study Disadvantages

• Cases may not be generalizable

• Causes/associations may have other explanations

• Potentially viewed as

  • Emphasizing the bizarre

  • Focusing on misleading elements

Case Series (Descriptive)

Track cases/patients who possess a specific disease or disease-related outcome OR examines patients medical records for exposure and outcome (H. Lacks doctor did not do this shi btw 😭)

• “Series” = likely to be small sample

Case Series Advantages

• Simple + inexpensive

• Useful for rare conditions

• Generates hypotheses

• Real-world observations

• Quick publication

Case Series Disadvantages

• No control group

• Lower level of evidence

• Potential for bias

• Does not establish causation

  Susceptible to findings by plain chance

• Lack of statistical rigor

  • Commitment to best practices in research

Observational Research Designs

• (1) Cross-sectional

• (2) Case-control

• (3) Cohort

  • Observe naturally what is going on in the world without interfering in-between

Cross Sectional Study Design (Observational)

A type of research (observational) that examine a group of individuals at one point in time to gather data on various variables

• Provide a snapshot of the population at a single moment (Hook-up one and done ah)

• In kinesiology, can be:

  • Assessing PA factors, biomechanical measures, or any other related factors in a given population

Cross Sectional Study Design Advantages

• Efficient

• Good for descriptive analysis

• No dropouts

• Establishing prevalence

  • Proportion of population who possess certain characteristic(s)

Cross Sectional Study Design Disadvantages

• Does not determine causality

• Temporal sequence unclear

• Potential for bias

• Snapshot has limitations

Cohort Study Design (Observational)

Any designation group of persons who are followed over a period of time (Not in a snapshot like Cross Sectional)

• Exposure not randomized (participants are selected based on their exposure status and evaluated for occurence of the outcome of interest), but allows for sample to vary naturally as it not in the researchers control

  • In other words, observe how exposure to certain factors (this creates exposure status) affect the development of specific outcomes

  • Does exposure to this = often see this outcome???

    • eg. a cohort study examining how smoking (exposure) affects the incidence of lung cancer (outcome) by following a group of smokers and non-smokers over several years.

• Traditional epidemiology:

  • Likes to compares occurrences of disease within one or more cohorts (eg. one exposed to some exposure/treatment vs. non-exposed → Jonkler ah study)

Difference Between Randomized vs. Observational Cohorts

*Refer to photo

Experiment = Randomized

Observation = Observation

Why an Observational Cohort and not a Randomized Cohort?

• In place for exposure that can not randomized (ethically, or practically)

• Cohort studies (Secondary analysis of existing data) provide good information to guide action

• To reflect reality better

  • Experimental conditions limit this

• Can be prospective (future) or retrospective (past)

  • Note: can argue randomized trial is a prospective observational cohort with additional conditions → randomized assignment of standardized (consistent) treatment

• Fundamental study design

*Note Image to clarify prospective and retrospective cohort studies

Cohort Study Design Advantage

• Causality

• Rare exposures

• Temporal sequences

• Reduce recall bias

• Direct measures

Cohort Study Design Disadvantages

• Cost and time

• Loss to follow-up

• Changes over time

• Rare outcomes

• Confounding

  • Mixing of effects resulting in a distortion of the true relationship

Case Control Study Design (Observational)

Type of study (observational) which situates two existing groups differing in outcome and are compared on the basis of some supposed causal attribute

• Participants here selected for outcome status

• Often used to identify factors that may contribute to medical conditions

  • Compare subject:

    • (1) Who have the condition/diseases (“the “cases”)

    • (2) Those who do not, but are otherwise similar (the “controls”)

Idea Behind “Sampled Exposures”

A concept of case-control study that looks to increase efficiency of a study by sampling

Case-Control Study Design Advantages

• Good use for:

  • Rare diseases

  • Diseases with a long latency period between exposure and ideas manifestation

  • Less costly and time consuming

  • Advantageous in terms of exposure data (becomes hard or expensive to obtain)

Case-Control Study Design Disadvantages

• Selection bias may be present

• Inefficient for rare exposures

• Info on exposure subject to observation bias

• Finding appropriate control group sometimes proves to be difficult

Experimental Research Design

• (1) Lab Trials

• (2) Field Trials

  • Always implies cause and effect link

  • Experimenter controls for everything but the variable of interest

First Controlled Experiment EVERRRR! (Brainrot)

• Daniel BIBLICAL!

• King wanted some people to serve in his court

• As part of their education, they would receive royal meat and wine

• Daniel (true sigma btw) refused sat nah vro, I love God and my religion, and asked that he and his friends be given a diet of vegetables instead

  • Assured this vegetarian diet will not diminish their capacities to serve the king

• To prove this theory, Daniel proposed an experiment (wtf)

  • 10 days, 4 of us only vegetable, another grup have only king’s meat and wine

    • After 10 days is up, compare the two groups, “And as thou seest, deal with thy servants”

Principles of Daniel’s Experiment

• Groups are comparable and representative of some population → generalizability

• This was a prospective type of study

  • Groups chosen in advance

• Two things Daniel didn’t have in his study that is otherwise important today

  • Blinding

  • Selection bias

Randomized Trial

• Group of individuals who are randomly assigned to 2 or more treatment groups and the followed up until outcome/end of study

Usually there is an active arm vs. a placebo arm

Drug Trials: Classic Example of Experimental Research

• Medication we are prescribe conducted through drug trials

• Groups being assigned randomly → technical name: Randomized control trials

• In experimental research, experimenters are always manipulating the world to case a change, so in theory, everything but the effect of the drug has been controlled for

Types of Randomization

• (1) Individual Randomization

• (2) Blocked Randomization

• (3) Cluster Randomization

Individual Randomization

Example: each person flips a coin

Blocked Randomization

• Random order within a fixed block of a set size

  • Guarantees equal (or near-equal) numbers in each study arm

    • Advantageous in smaller studies where larger numbers may be less reliable

Cluster Randomization

• Community or group or classroom get the intervention

  • eg. intervention is a radio campaign to encourage exercise

Blinding (Masking)

• Some participants are unaware who is in which arm during the trial

• eg. trial of an active drug vs. a placebo

  • Patients unaware if in active or placebo

  • Study staff do not also know (Wth)

  • Individuals and/or data do not know which arm of the trial is which (is active or placebo?)

Causality with Randomized Control Trials

• Causal inference principles

  • Exchangeability

  • Positivity

  • Consistency

  • Temporality

How Are Such Principles Met?

Take placebo-controlled randomized trial

• Everyone assigned either the treatment or placebo in the same way, therefore, having no variation the treatment assignment → CONSISTENCY

• Everyone who enter the trial have a nonzero chance of being assigned to each of treatment & placebo (and getting it) → POSITIVITY

  • meaning, no participant is excluded from either possibility

• Every for example 25 year old gooner named Jelqseph who is assigned, a 25 year old gooner named Jelqseph is assigned placebo → EXCHANGEABILITY

  • That is if trial has a large number of people

Randomized Study Design Advantages

• Unambiguous temporal sequence - exposure precedes outcome

• Ensures

  • Positivity, consistency by design

  • Exchangeability in expectation - including by known and unknown factors

Randomized Study Design Disadvantages

• Expense and organization

• Many pitfalls potentially in study implementation

• Vulnerable to loss to follow-up & missing data

• Lack of generalizability

• Typically only assess outcomes in the short-term

• Many determinants of health cannot be randomized