Advanced research methods - Lecture 1

Lecture 1 + notes

What questions can be asked for assessing causal inference studies?

1. What was the explicit research question?
2. What was the implicit question?
3. What was actually estimated?

* Is the estimate biased or unbiased?
* Is this an estimate of a full or partial effect?


4. Is the estimate really an answer to the research question?
5. How was the analysis designed?
6. Which statistical methods were applied?
7. Were these methods applied correctly?
8. What is the (type of) estimate? Is it big, small, good, bad, etc.?

* How uncertain is the estimate?


9. What do the researchers conclude? Is that conclusion justified?
10. Is the conclusion supported with strong or weak evidence?
11. How does the conclusion compare to what we (already/thought we) knew?

What are potential outcomes approaches in causal inference?

Y = outcome

a = treatment

1 = yes treatment

0 = no treatment

i = individual

=/ does not equal

\
This approach is about thinking about potential outcomes. Only one potential outcome can be observed in reality, but we need information on the potential (or counterfactual) outcome “what would have happened had the women not used the powder”. This cannot be observed. To draw a causal conclusion (i.e. to say X had effect on Y) we therefore need a control group and meet the identificability conditions.

What is critical thinking about causal effect estimation?

What is causal effect?

In an individual, a treatment has a causal effect if the outcome under treatment 1 would be different from the outcome under treatmeant 2.

\
To asses, we need information on:

* What would have happened?
* What will happen?

What is consistency?

The treatment (or intervention, exposure) has to be well-defined until meaningsless vagueness is left. (Hernan 2016, does water kill?)

What is positivity?

* Each individual has to have a positive probability of being assigned to each of the treament arms. (i.e. PR(A=a)>0 for all treatment arms).

What is exchangeability?

* The individuals assigned to the different treatment arms have to be similar
* It does not matter who gets treatment A or B

What problems can there be in research?

* Too small sample size
* Is this always a problem?
* Study performed or financed by commercial company
* Is this always a problem?
* No control group
* Essential omission (weglaten)
* What would happen without treatment?
* Potential regression to the mean

What are the identificability conditions?

Average causal effect can be determined if, and only if, three identifiability conditions are met:

* consistency
* positivity
* exchangeability

If all conditions are met (and an association is found in the data) the association between exposure and outcome is an unbiased estimate of a causal effect.

What means counter the fact?

Counterfactual outcome: potential outcome that is not observed because the subject did not experiencce the treatment (counter the fact). The control group did not receive treatment.

What is a fundamental problem in causal effect?

Individual causal effect cannot be observed, except under extremely strong (and generally unreasonable) assumptions.

Average causal effect (i.e. in a population) cannot be determined based on individual estimates.

* Causal inference as a missing data problem.

How can you meet the exchangeability condition?

1. RCT
2. Matching
3. Stratification
4. Adjustment

\

What is the golden standard for identificability conditions?

RCT

* Individuals are radomly assigned to one of each treatment arms.
* Differences between individuals in the different treatment arms are cancelled out on the sample level
* Differences are independent from the treatment and outcome
* Differences are random, not systematic

What is matching?

* For each individual with characteristics x,y,z whp gets treatment A, there is an individual with chracteristics x,y,z who gets treatment B
* Statistical methods can be applied when perfect matching (i.e. use identical twins, triplets) is not possible (e.g. propensity score matching).

What is propensity score matching?

A statistical matching technique that attempts to setimate the effect of a treatment, policy or other intervention by accounting for the covriates that predict receiving the treatment.

What is stratification?

* Randomly select individuals from different subsets (i.e. strata) of the larger population
* Difficult to meet the positivity condition (i.e. individuals in all strata).
* With smoking and carrying a lighter, its hard to find groups that also have other relevant strata (e.g. based on age group, sex, education level)

Stratification becomes quickly infeasible (unworkable)

What is adjustment?

* Control for factors that influence (i.e. bias) the association between the treatment and outcome in regression analysis
* Indivudlas are assigned to all treatment arms within all levels of adjustment factors
* Can also be combined with RCT, stratification, matching
* Complee and correct adjustment leads to exchangeability (directly acyclic graphs, DAGs)

What is the sidenote on the golden standard?

RCT:

* limited external validity (e.g. controlled setting, efficacy of treatment)
* Ethical and practical considerations

In observations (non-randomised studies):

* real world outcomes (effetiveness of treatment)
* availability of data
* positivity and consistency need close attention
* Internal validity threatened by lack of exchangeability

\

Why is it important to meet the identificability conditions?

* To obtain insight in the quesiton: what would have happend, if …
* Estimate an unbiased effect of an exposure on an outcome

What are the assessing causal inference questions?

1. What was the explicit research question?
2. What was the implicit question?
3. What was actually estimated?


1. Is the estimate biased or unbiased?
2. Is this an estimate of a full or partial effect?
4. Is the estimate really an answer to the RQ?
5. How was the analysis designed?
6. Which statistical methods were applied?
7. Were these methodsapplied correctly?
8. What is the type of estimate? Is it big, small good, bad?


1. How uncertain is the estimate?
9. What do the researchers conclude?


1. Is that a justified conclusion?
2. Is the conclusion supported with strong or weak evidence? How does the conclusion compare to what we already thought?

\

How can you use graph theroy to achieve exchangeability?

What are the rules of DAGs?

\

What is a confounder?

Any variable that can be used to remove confounding

What is confounding?

Bias caused by common cause of exposure and outcome (if not adjusted)

Why do we use DAGs?

* Individual causal effect cannot be observed; no information about the counterfactual outcome
* Average causal effect can be estimated if, and only if, all three identifiability condiotons are met

Why is correlation no causal inference?

Correlation implies association

* A statistical relationship between the treatment and outcome
* Knowing the value of one variable may provide information on the value of another variable, but that does not mean that one caused the other
* Knowing that Zeus died 5 days after a heart traplant does not mean the transplant caused Zeus death (Hernan and Robins, 2020)

What is a statistical association?

It equals the difference in potential outcomes if, and only if, the identifiability conditions are ment

We need:

* Theory and subject knowledge (previous studies in literature)
* Insight into causal structure underluing the research question
* To meet the positivity, consistency and exchangeability conditions, and hence design the study and analysis accordingly.

What are problems with traditional designs?

* These methods rely on the available data, rather than on theory.subject knowledge (e.g. omitted variable bias).
* Selected strategy may increase rather than reduce bias
* Setepwise selection may lead to the uncertainty associated.
* Methods are outdated

→ solution: DAGs

What are basic rules of DAGs?

* Arrows represent causal effect
* Variables are all associated in data
* Paths connect variables in a DAG
* A connection transmits association

What happens if you adjust for a variable?

A path is blocked. It changes the association. By blocking all the connections it removes the association.

What is a path?

A route between X and Y. It does not have to follow the firection of the arrows.

What is the difference between a causal path and a backdoor path?

* Causal path: follows direction of the arrows between X and Y
* Backdoor path does not

What is the difference between an open and a closed path?

All paths are open, unless they collide.

What is adjusting for a variable?

The influence of L is removed on the association between X and Y **by including L in the regression analysis**

What happens if you adjust for a collider?

* When you adjust for a collider (W) you open the blocked path between X and Y that goes through W (we then allow W to influence the association    between X and Y by including W in the regression analysis).

How can you remove counding?

Remove confouding by adjusting for L or M, not for both.

What are the causal inference assessment questions?

1. What was the explicit research question?
2. What was the implicit research question?
3. What was actually estimated?


1. Is the estimate biased or unbiased?
2. Is this an estimate of a full or partial effect?
4. Is the estimate really an answer to the RQ?
5. How was the analysis designed?
6. Which statistical methods were applied?
7. Were these methods applied correctly?
8. What is the type of estimate?


1. Is it big, small, good bad?
2. How uncertain is the estimate?
9. What do the researchers conclude?
10. Is the conclusion supported by data?
11. How does the …

What are colliders?

A variable which naturally blocks a backdoor path.

This path does not contribute to association between exposure and outcome. Adjusting for a collider opens a path.

What is confounding bias?

What is selection bias?

Sometimes means sample not representative.

Confounding is sometimes called selection bias.

What is the role of theory in causal inference; designing the analysis?

What happens is you adjust for a collider?

* It opens the path.
* Adjusting for a collider transmits association between X and Y.
* The association between diet and disease consists of the combination of all open paths between them.
* Assume you have information about a collider. This implies association, but no causation.

What happens if you adjust for a confounder?

There is no bias.

What happens if you adjust for a collider (selection/collider)

There is bias.

What are disadvantages of RCT?

* Limited generalizability (external validity) due to treatment protocol and patient selection
* Practical, ethical considerations

What are risks of RCT?

* Statistical errors
* Table 2 fallacy
* Chance findings

What are (dis)advantages of observational data?

* Overall good generalizability
* Availability of data is high
* Internal validity threatened by lack of exchangeability
* Positivity and consistency need explicit attention.

What are risks with observational studies?

* Statistical errors
* Table 2 fallacy
* Chance findings
* Confounding bias
* Selection bias
* Lack of consistency

What is moderation?

* Not speciafically part of DAG concept
* Coefficient additional effect in subgroup
* interpretation is easier when you fill out the regression equatiations for subgroeps

What is mediation?

* Occurs when there is a causal path betwen exposure and outcome which runs through another variable (the mediator)
* When adjusting for a mediator, part of the association is removed. The effect of exposure on outcome would be underestimated (partial causal effect, direct effect).

What is a partial effect?

The solely effect of exposure on outcome. Adjusted for intermediate/mediator.

When there is no association after adjustment, the mediator completely explains the effect.

What is a full effect?

Full causal effect of exposure on outcome, not adjusst for the mediator.

What is mediator outcome confounding?

Occurs when a confounder affects outcome and intermediate. This is a problem when you want to estimate the partial effect of the explanatory variable.

In that case, if you adjust for the mediator, the backdoor path from the confounder to X is opened. This is a biased effect. It is the direct causal effect + something non causal.

You have to adjust for the mediator and the confounder. When not having information on the confounder, you cannot do this as the results will be biased. Thats a problem.

→ We have a biased estimate of the partial effect.