Evaluating health behaviour change interventions via experiments and randomised controlled trials (RTC)

Qualitative data

• non numerical data

• focus on why and how

• generates theories based on data

• 'data are analysed via categorisation

quantitative data

• numerical data, to quantify variables and for statistical estimation or inference

• focus on what and how much

• test theories and hypothesis

• data are analysed using statistical methods

quantitative behaviour is used if you;…

• want to quantify behaviours

• establish general patterns

• test specific theories or hypothesis

qualtitative behaviour is used if you;…

• explore new research areas or phenomenon

• understand deep psychological processes

• study individual experiences

Types of research design

Extraneous variables are

any variables other than the IV that may affect the DV

extraneous variables become confounding variables when…

their values change systematically with the level of the IV

The impact of extraneous and confounding variables is assumed to be …

minimised through ‘true’ randomisation

What are RTCs

• RTCs are a subset of possible experimental design

• an RTC is a planned experiment designed to assess the efficacy of an intervention by comparing the intervention group to a control group

• the allocation to intervention or control group is determined purely by chance (true randomisation)

True (proper) randomisation in experiments/ RTC’s

1. eliminates bias in treatment/ intervention assignment

2. ensures that differences between experimental groups e.g. intervention vs control group can be attributed to the treatment/intervention

3. Permits the use of probability theory to express the likelihood that any differences in outcome between treatment/ intervention groups merely indicates choice

Why conduct RCTs?

• observational and quasi experimental deign are subject to potential bias and confounding due to: self selection, observer bias, secular trends (before and after the study)

• RCT provides the gold standard for proof of concept

Why conduct RCTs?

1. Eligibility criteria
   ii. Specified hypotheses
   iii. Predefined intervention and control groups (proper control group)
   iv. Primary and secondary outcomes/endpoints (e.g., behavioural change, HIV incidence) to address hypotheses
   v. Methods for enrolment and follow up
   vi. Rigorous monitoring
   vii. Analysis plans and stopping rules
   viii. Comprehensive reporting of methods and data analysis

Why conduct RCTs? hierarchy of evidence

The history of RCT’s -

• First ever RCT conducted in 1747 by James Lind examining the impact of citrus in treating scurvy

• First published RCT in medicine “streptomycin treatment of pulmonary tuberculosis”, which described a medical research council (MRC) Investigation conducted by Austin Bradford Hill

Types of RCT’s - Individually randomised trials

• Eligible individuals are randomised (conventional medical RCTs and
  many behavioural RCTs)

• Self-selection of persons volunteering for enrolment

Types of RCT’s - cluster randomised trials

• Clusters (e.g., communities, hospitals), or other aggregates of people
  (e.g., workplaces, bars) are randomised, and all consenting persons
  enrolled

• Less individual-level self-selection -> increasing generalisability

• Nature of the intervention (e.g., mass media campaign, population-level
  interventions)

• Acceptability and reduced stigma (everyone gets the same treatment within a cluster)
  

Types of RCT’s - cluster randomised trial - BUT

• Cluster randomisation more vulnerable to lack of comparability between
  study groups than individual randomisation (fewer units of
  randomisation, more correlated characteristics within members of
  clusters)

• Cluster RCTs increase sample size requirements and are less efficient
  than individual RCTs due to intra-cluster correlation

The different steps taken when designing and implementing RCTs - pic

Steps taken when designing and implementing RCTs - 1- conduct a sample size calculation

• a sample size calculation is an a priori statistical analysis to determine the appropriate number of participants needed in the RCT in order to detect a meaningful statistical effect

• an effect size is the magnitude of the differences between 2 groups (arms/conditions within the RCT

Steps taken when designing and implementing RCTs - 1- conduct a sample size calculation - 1a - sample size calculation for individually randomised RCTs

• specific type 1 and 2 error (e.g. power less than 80%) to detect a difference significant at p - 0.05

• specific an expected or meaningful difference in outcome rates between intervention and control groups

• Estimate losses to follow up on the primary outcome

• Estimate required sample size at enrolment using conventional formula

Steps taken when designing and implementing RCTs - 2 - register the protocol for the RCT - protocol contains precise information regarding

• intervention vs. control groups [includes information on what exactly the
  different groups are exposed to]

• Outcome/endpoint measures (what we know as DVs in experiments)

• Hypotheses

• Participant eligibility criteria

• Analysis plan

Steps taken when designing and implementing RCTs - 2 - register the protocol for the RCT - overviews

• fulfil ethical oblicagtiosn to participants and researcher

• provide information to potential participants and referring clinicians

• reduce publication bias

• help editors and others understand the context of study results

• promote more efficient allocation of research funds

• Help institutional review boards determine the appropriateness of a research study

Steps taken when designing and implementing RCTs - 2 - register the protocol for the RCT - control groups

• Controls may receive no treatment/intervention (e.g., placebo, or
  nothing in behavioural RCTs) if there is no standard of care (no
  previously accepted intervention that works)

• If there is an established standard of care it would be unethical to
  withhold this from controls, so standard of care becomes the reference
  control

Steps taken when designing and implementing RCTs - 2 - register the protocol for the RCT - Primary outcome

• Primary outcome is the outcome that an investigator considers to be
  the most important among the many outcomes that are to be examined
  in the study.

• The primary outcome needs to be defined at the time the study is
  designed.

• The primary outcome reduces the risk of a Type I error resulting from
  the statistical testing of many outcomes.

• It reduces the risk of a Type II error by providing the basis for the
  estimation of the sample size necessary for an adequately powered
  study.

Steps taken when designing and implementing RCTs - 2 - register the protocol for the RCT - Secondary outcome

• Secondary outcome measures, also known as secondary endpoints,
  may provide information on therapeutic effects of secondary
  importance, side effects, or tolerability.

• An operational definition (operationalisation) is how a variable is
  measured [and, this forms an important part of the registration].

Steps taken when designing and implementing RCTs - 3 - enrolment - select and define study sample - elibility criteria

Eligibility criteria establish the parameters for determining who is able to
participate (inclusion) and who is not able to participate in the study
(exclusion

Steps taken when designing and implementing RCTs - 3 - enrolment - select and define study sample - eligibility is predefined to

• Ensure that participants meet the criteria for the intervention (e.g., have
  a specific disease for a therapeutic trial, are free of disease for a
  preventive trial etc.)

• Usually eligibility is also defined by age, gender, race, state of health
  (absence of contraindications etc.)

• The narrower the eligibility criteria, the less generalisable the results

• Participants must consent to screening for eligibility

Steps taken when designing and implementing RCTs - 3 - enrolment - select and define study sample - enrolment occurs only after

edibility is established, participants only enrolled after providing informed consent

Steps taken when designing and implementing RCTs - 4 - Baseline assessment

• assess all participant enrolled in the study on measure pertaining to primary and secondary outcomes and mediating variables prior to introduction or intervention/ treatment

• Create a logic model depicting the hypothesised pathway that you are testing

Steps taken when designing and implementing RCTs - 5 - random allocation (randomisation) - There are two main features to consider when randomising participants to study conditions:

• implementing a valid randomisation procedure;

• establishing procedures to safeguard the integrity of the randomisation
  procedure so that unintentional or intentional biases do not influence the
  participant allocation process.

Steps taken when designing and implementing RCTs - 5 - random allocation (randomisation) - 5a - implementing a valid randomisation procedure

• Simple randomisation: Analogous to a repeated fair coin tossing

• Restricted randomisation or Blocking: Done to ensure equal balance of
  participants across groups throughout all portions of the study
  ▪ For example, blocks of six would have 3 intervention/3 control pps

• Stratified randomisation: Individuals are identified based on important
  covariates (sex, age, etc.) and then randomisation occurs within the
  strata

• Dynamic or adaptive methods (not common): Not pre-defined, only first
  participant truly randomly assigned

Steps taken when designing and implementing RCTs - 5 - random allocation (randomisation) - 5b - safeguarding integrity of randomisation

• Concealment of allocation strategies are designed to mask participants’
  knowledge about their group assignment (e.g., sealed opaque
  envelopes).

• Prior to beginning study enrolment, the main investigator or RCT
  statistician generates the allocation sequence. This pre-generated
  sequence must be adhered to when randomising all participants. This
  sequence is not shared with other members of the RCT team.

Steps taken when designing and implementing RCTs - 6 - implementing the study groups/ conditions as per protocol - binding is done to minimise participant or researcher bias

• Single blinding (researcher but not participant knows the
  randomisation group, e.g., some cluster-level RCTs)

• Double blinding (neither researcher nor participant know the group of
  randomisation)

• Triple-blinding (researcher, participant, and statistician analysing data
  from the study all do not know the group randomisation)Unblinded/open (cannot conceal randomisation, e.g., surgical
  interventions)

Steps taken when designing and implementing RCTs - 6a - concealment of allocation vs blinding

Concealment of allocation: Procedure to protect the randomisation
process before the subject enters the RCT (Concealment of allocation is ALWAYS feasible)

• Blinding: Masking of the treatments after randomisation (once trial
  begins)
  ▪ Blinding is not always feasible

Steps taken when designing and implementing RCTs - 6b - ensuring fidelity to protocol

• Throughout the RCT the research team needs to measure and record
  the fidelity to protocol, i.e., the extent to which the outcome measures
  and intervention are administered in accord with the registered protocol.
  

• Any violations to protocol need to be recorded and reported in the RCT
  publication (with sensitivity analyses conducted for deviations to
  protocol).

Steps taken when designing and implementing RCTs - 7 - follow up

• Follow up is conducted at predetermined intervals needed to detect the
  occurrence of RCT outcomes/endpoints

• The frequency and duration of follow up will depend on:

• Type of outcome/endpoint (e.g., response to treatment, development of
  new disease, progression of disease, behavioural change, sustainability
  of change)

• The level of risk (e.g., higher the risk, more frequent the follow up)

Steps taken when designing and implementing RCTs - 7 - follow up - Losses to follow up must be minimised because:

• Losses are often selective (e.g., high risk persons, low socio-economic
  status participants drop out of trials) and this introduces bias

• Losses to follow up should be comparable in the intervention and
  control groups to avoid biased comparisons, if not this leads to attrition
  bias

• Losses to follow up reduce study power by reducing the person-time of
  observation

Steps taken when designing and implementing RCTs - 8 - Analysis - intention to treat

• Analyse all persons randomised, even if some do not receive the
  intervention/control treatment or drop out before completion of RCT

• Analysis based on the group participants were initially (and randomly)
  allocated to

• Least biased and most conservative

Steps taken when designing and implementing RCTs - 8 - Analysis - as treated (per protocol)

• Analyse only those who actually complete the RCT

• Analysis includes only those participants who completed the
  intervention/control group they were originally allocated to

• Potentially biased by selection of the most compliant and often lowest
  risk population

Steps taken when designing and implementing RCTs - Reporting RTC findings

To assist in reporting of RCTs, a well-articulated, structured format (a
checklist) has been developed by the CONSORT (Consolidated
Standards of Reporting Trials) group.

Steps taken when designing and implementing RCTs - Reporting RTC findings - A CONSORT flow diagram -

graphically presents the progress of participants through the different phases of a randomised controlled trial (RCT).

Are results of RCTs always valid?

• RCTs can provide conflicting results (so, important to carry out
  systematic reviews and meta-analyses)

• RCT design, execution, analyses, and reporting can be flawed

• Intervention vs. control comparisons are internally valid, but restrictions
  on participant eligibility can reduce external validity (e.g., specific age or
  sex groups omitted)

• RCTs could suffer from conflicts of interest (e.g., industry funding)