BM3101

BM3101 Research Methods

Importance of experimental design

save time and money, address ethical issues, generate useful data, minimize random variation, and account for confounding factors.

Goals in designing experiments

reduce inter-dependent samples, minimize random variation, and account for confounding factors.

Independent data points

When the measured value of one individual has no effect on the possible values another individual will have.

Sample

The subjects in an experiment.

Population

The wider set of individuals the sample represents.

Names for random variation

Noise, inter-individual variation, between-individual variation, within treatment variation.

Random variation

The differences between measured values of the same variables taken from different individuals.

Confounding factor

An unknown variable that influences the response variable and creates the appearance of a relationship between the studied explanatory and response variable.

Hypothesis

A clear statement articulating a plausible explanation for observations and designed so that it can be supported or refuted by the gathering of data.

Order of scientific method

Observation → question → hypothesis → prediction.

Pilot study

An exploration of the study system, conducted before the main body of data collection, in order to refine research aims and data-collection techniques.

Correlative/observational study

A study that examines whether two or more processes tend to co-occur or are mutually exclusive without altering the experimental system.

Mechanistic/manipulative/intervention studies

Studies that provide evidence for interaction between pathways and their involvement in a biological process by altering the experimental system.

Advantages/disadvantages of correlative study

the inability to prove causative relationships or process interaction, less work and money, less intervention when animals are involved, and the potential for confounding factors and reverse causation.

Advantages/disadvantages of mechanistic study

the ability to avoid reverse causation or confounding factors, but requiring more work, money, and time, and the potential for unintended consequences of manipulations.

in Vitro

Performing a given procedure in a controlled environment outside of a living organism.

Issue with in vitro models

in vitro models do not replicate the precise cellular conditions of an organism, so results may not correspond to the circumstances occurring in a living organism.

in Vivo

Experimentation using a whole, living organism as opposed to a partial or dead organism.

in Silico

Experimentation carried out on a computer or via computer simulation.

Features of good experimental design

clear objectives, avoidance of systematic error, avoids bias, sufficient power, precision, reproducibility, randomised comparisons, error estimation, statistical analysis, and broad validity.

Systematic error

Error that is not determined by chance but is introduced by an inaccuracy inherent in the system.

Precision

How close repeated measurements are to each other, a measure of random error.

Factors in precision

experimental design, sample size, and size of the random errors.

Experimental factors

Aspects of the experiment that change or influence the experiment's outcome.

Continuous

Ordered numerical data, such as body weight in kg.

Discrete

Unordered numerical data, such as the number of cells.

Nominal categorical

Unordered phrases, such as gender.

Ordinal categorical

Ordered phrases, such as tumor grade.

Sources of variance

Biological variance (noise) and technical noise.

Batch effect

When a subset of the sample behaves differently due to measurement or technique variance.

Replication

Making the same manipulations and measurements on a number of different experimental units.

Biological replicates

Increased sample size, more of the same measurement on different subjects.

Technical replicates

Same sample size, more of the same measurement on the same subject (mean of technical replicates = 1 biological replicate).

Pseudoreplication

Taking the incorrect level of replication by artificially inflating the number of replicates to create spurious results.

Randomisation

Ensuring that any individual in the population of interest has the same chance as any other individual in that population of finding itself in each experimental group.

Power

The probability that a statistical test will reject a false null hypothesis or detect an effect if it is really there.

Factors in power

effect size, standard deviation, amount of random variation, biological differences, experimental design, sample size, number of replicates, and one vs two-sided test.

Effect size

The difference between the mean of the experimental group and the mean of the control group divided by the standard deviation.

Type I error

False positive; the results show an effect but really there is no effect (2 = 0. 95)

Type II error

False negative; the results show no effect but really there is an effect (B = 0. 2) (power = 1-B)

Critical value

a point on the test distribution that is compared to the test statistic to determine whether or not to reject the null hypothesis (test statistic > critical value = statistical significance)

Specificity

returning a negative result when it is negative (low false positive, high false negative

Sensitivity

returning a positive result when it is positive (low false negative, high false positive)

Control

a reference against which the results of an experimental manipulation can be measured

Purpose of control

establish a baseline for comparison, & reduce effect of confounding variables

Negative control

no manipulation

Positive control

provides data on the outcome of an alternative procedure

Concurrent control

running the controls at the same time as the treatment group

Historic control

using old data as a control (can introduce confounding factors)

Blind procedure

Scientists have no knowledge of which experimental group each subject belongs to, removes concern of scientist/assessor bias

Double blind procedure

subjects also have no knowledge of which experimental group they belong to, avoids subject bias

Placebo effect

if people believe they are receiving treatment, they show improvements in health

Placebo

a vehicle control designed to appear exactly like the real treatment without the experimental variable

Purpose of placebo

subjects continue to participate in experiment, to control every experimental factor except for the parameter under investigation

Main effect

direct effect of explanatory on response variable

Interaction effect

the effect of the explanatory variable is affected by another independent variable

Blocking

splitting experimental subjects into blocks based on another variable that may interfere with the experimental variable (ex: age, gender) before randomly distributing the individuals in each block in turn

Imprecision

adding error to measurements in an uncorrelated manner (problem with reproducibility)

Inaccuracy/bias

adding error to measurements in a correlated manner (systematic error)

Bias

any deviation from the truth in data collection/analysis/interpretation which can cause false conclusions

Intra-observer variability

Inaccuracy introduced by human error

Observer drift

systematic change In a measurement instrument (ex: human observer) such that the measurement taken is affected by when in a sequence of measurements it was taken

Avoiding intra-observer bias

repeatability study, create objective categorising criteria

Repeatability study

a procedure of measuring random samples several times& comparing their values

High repeatability

same score each time something is measured, suggests low imprecision but tells nothing about bias or inaccuracy

Checking bias/inaccuracy

score samples& check against the known true measurement, recruit more observers

Inter-observer variability

error introduced because different observers score the same sample differently

Avoiding inter-observer variability

clearly define rules& methods of scoring, normalise the data, discussion to find resolution

Observer effect

observing a biological system may change the way it behaves

Floor effect

the majority of measurements taken are at the lowest possible value of the range

Ceiling effect

the majority of measurements taken are at the highest possible value of the range

Accuracy

a test's ability to correctly Identify each sample as positive or negative (number of correct assessments/number of all assessments)

Specificity calculation

TP/TP+FN

Sensitivity calculation

TN/TN+FP

Longitudinal study

A correlative study that involved repeated measurements of the same variables over a long period of time.

Panel study

A longitudinal study that takes repeated measurements from the same individuals to track individual changes.

Cohort study

A longitudinal study that samples a specific population that share a common trait (ex: married)

Prospective study

A longitudinal cohort study that follows a sample of similar individuals who differ in certain specific factors to measure how those differing factors affect certain outcomes.

Retrospective study

A longitudinal cohort study that examines historical data to study an event or outcome that has already happened.

Cross-sectional study

A correlative study that takes data from many different individuals at one specific point in time

Case-control study

A correlative study that groups individuals based on the outcome of interest and examines data for causal factors differing between the groups.

Advantages/disadvantages of cross-sectional study

Inexpensive and quick, can estimate prevalence, can assess many outcomes, no loss to follow-ups, difficult to make causal inference, only a snapshot, prevalence-incidence bias

Neyman (prevalence-incidence) bias

Selection bias where prevalence and incidence are used interchangeably which can bias results to make the risk look more/less severe than the reality.

Incidence

Rate at which new cases occur in a population during a specific period

Prevalence

Proportion of the population that are cases at a point in time

Mortality

Incidence of death from a disease

Pearson’s Chi-square test

test comparing the observations in your data to what you would expect if the null hypothesis was true and if there was no association between the explanatory and response variable.

Odds ratio

measures effect size: odds of exposure to risk factor in cases divided by odds of exposure to risk factor in controls

Ethics

Working out the right thing to do.

Bioethics

Working out the right ways to use biology or medicine

Research Ethics

values/principles that help researchers work out what they can and can’t do.

Research Ethics Committee

Group of people from different expertises in a local community that discuss the ethics of new studies before they are allowed to go ahead

7 ethical requirements

Have social/scientific value, have scientific validity, fair subject selection, favourable risk-benefit ratio, independent review, informed consent, and respect for potential and enrolled subjects

Refinement

Reduce pain, suffering, and distress of animals without comprising the quality of the evidence collected

Reduction

Reduce the number of animals used to the minimal necessary while still gathering quality data

Replacement

Replacing animal testing with other methods, ex: in SIlico models