surveys, experiments and observational studies

aim of a survey

estimate the proportion of a population who have a certain trait or opinion

margin of error

the measure of accuracy of a survey

simple random sampling

every member of the population of interest has the same chance of being in the sample

one way of picking a random sample

table of random numbers

stratified random sampling

• Divide the population into the different groups or strata

• Draw a simple random sample within each stratum

strata

natural groups of the population

cluster sampling

the population consists of a number of groups called clusters. Rather than sampling within each cluster however, a random sample of clusters are selected and measurements are made only on those clusters

care must be taken when using cluster sampling

members of the same cluster may be similar

systematic sampling

the population is ordered into a list and every 50th (example) member of the population is selected

problem with systematic sampling

can lead to biased samples

not using the correct sampling frame

Your sampling frame may contain unwanted units

not reaching the intended individuals

The units desired may not be reached

having a low response rate

the lower the response rate the less the results can be generalised to the population as a whole

volunteer samples

not a good idea

convenience sampling

not a good idea

when are experiments or observational studies performed?

when interested in examining the relationship between variables

explanatory variable

one which attempts to explain the differences in the response variable

a treatment

a category or a combination of categories of the explanatory variable which is assigned by the experimenter

randomised experiments

the explanatory variable is manipulated and we examine the effect of this on the response

an observational study

observe differences in the explanatory variable and examine whether they are related to differences in the response variable

2 reasons for using an observational study over a randomised experiment

1. It is unethical/impossible to assign subjects to receive a specific treatment

2. Inherent traits cannot be randomly assigned

confounding variable properties

• It is related to the explanatory variable

• It affects the response variable

interactions between variables

The effect of one explanatory variable on the response, depends on the status of another explanatory variable

randomisation of treatment type

ach experimental unit should have the same probability of being assigned any of the treatments

a control group

treated in exactly the same manner as the treatment group(s) with the exception that the members of the control group do not receive the active treatment

medical context of placebos

the placebo looks exactly like the drug being tested but does not contain the active ingredient

double blind experiment

neither the researcher or the participant knows who had which treatment

single blind experiment

either the participant or the researcher knows which treatment type the participant was assigned

matched pairs

experimental designs that use either the same experimental unit or two matched experimental units to receive each of two treatments

randomised block design

The extension of the matched-pair design to more than two treatments

random sampling

• Used to get a representative sample from the population

• Results can (mostly) be extended to the full population

random assignment

• Used to control for confounding variables

• Cause-and-effect conclusions can be drawn

confounding variables problem

variables other than the explanatory variable under consideration may be responsible for changes in the response variable

confounding variable solution

randomisation of experimental units to treatments

interacting variables problem

a second variable interacts with the explanatory variable causing an effect on the response

interacting variables solution

researcher should measure and report all other variables which they deem as possible interacting variables

placebo, Hawthorne and experimenter effects problem

placebo effect as defined previously. The Hawthorne effect claims that participants respond differently just because they are in an experiment. The experimenter can bias results.

placebo, Hawthorne and experimenter effects solution

use of double-blind experimental designs and the inclusion of a placebo or control group

ecological validity and generalisability problem

 an ecologically invalid experiment is one where the variables of interest have been removed from their natural setting. Results will therefore not accurately reflect the impact of the variables in the real world

ecological validity and generalisability solution

no ideal solution. Try and perform the experiment in a natural setting. Measure variables in which the participants may differ from the general population

clinical trials

a series of experiments increasing in size and complexity in order to verify the safety and efficacy of a new treatment for a given condition

preclinical

initial experiments and studies carried out on biological models, animals, etc

phase I

dose-finding experiments to establish safe dosage (5-10 patients)

phase II

patients with condition are treated to establish the therapeutic effect and possible side-effect (20-50 patients)

phase III

multi-centre randomised control trials across different regions and demographic groups (100s-1000s patients)

phase IV

monitoring of long-term side effects

classifications of observational studies

retrospective or prospective studies

retrospective

participants are requested to recall past events

prospective studies

participants are followed into the future and events are recorded

case control studies

Cases who have a particular variable of interest are compared with controls who do not. Sometimes cases are matched with controls on an individual basis - this is similar to the matched-pair designed experiment

method of case control

Case-control studies begin by identifying a suitable number of individuals who have the characteristic of interest: the cases.

A group of controls are then identified where they have as similar as possible characteristics as the cases, but do not have the characteristic.

efficiency

more efficient than a randomised experiment in terms of time, money and finding a large enough sample with the characteristic of interest

reduction of potential confounding variables

the controls in case-control studies can be selected in such a way that the occurrence of confounding variables is reduced

confounding variables and causation problem

a link between two variables established by an observational study does not necessarily imply that one causes the other. Without it is impossible to separate out all potential confounding factors.

confounding variables and causation solution

try to measure the effect of all potential confounding variables on the response. Choose controls that are as similar as possible to the cases

generalising incorrectly problem

generally in observational studies it is hard to get a representative sample - convenience samples are often used

generalising incorrectly solution

get a representative sample

Using the past as a source of data problem

etrospective observational studies can be particularly unreliable as they are based on memory. Also, confounding factors that existed in the past may not exist now and the researcher may not consider them as confounding variable

using the past as a source of data solution

try and use a prospective study but if this is not possible try and use scientific records rather than memory as your source of data

using both

large observational studies uncover interesting relationships between variables on a given population. These results are more trustworthy than anecdotal evidence, but they do not prove any causal links