Uncertainty

Uncertainty

- Limitations of knowledge in face of variability.

- An inherent part of science and research.

- No need to do research if we have certainty.

- Science does not provide absolute truth only probability.

- Uncertainty is perceived as a problem mostly by those who think certainty is an ideal state of knowledge.

Statistics

- Branch of applied mathematics that involves collection description, analysis, and inference of conclusions from quantitative data.

- Practice or science of collecting and analyzing numerical data in large quantities, especially for the purpose of inferring proportions in a while from those in a representative sample.

Probability

- Mathematical study of probability began in 17th century in France.

- Games of chance

- Famous correspondence between mathematicians Blaise Pascal (1623 - 1662) and Pierre Fermat (1607 - 1665).

Evolution of Probability

- Shift in the 19th century toward scientific investigation of statistical patterns.

- Probability begins to replace emphasis on certainty and determinism.

- Chance and uncertainty become 'tamed' by quantification.

Jacob Bernoulli

Law of Large Numbers: if you repeat an experiment enough times, the average of random variables should approach the expected value.

Carl Gauss

- Theory of errors—foundation of normal distribution.

- Small errors more likely than big errors.

- In the presence of several measurements of same quantity the most likely value of the quantity being measured is their average.

- The average is the best estimate of the quantity being measured.

Ecological Fallacy

- Taking the mathematical average of a population as a guide to understanding individuals. Attributes the properties of the population to individuals.

- John Venn

Ergodic Switch

Inferences about individuals made erroneously on data collected at group level.

John Venn

- Conflicting accounts of best cannonball.

- The average does not give us the best answer to act upon.

Frequentist Statistics

- Developed in first half of 20th century by the English School of Statistics

- Karl Pearson, Ronald Fisher, Jerzy Neyman, Egon Pearson.

- Assumption: as the sample size gets larger, we get closer to the true frequency in the population.

- Key concepts: P value, experimental design, null hypothesis testing.

Ronald Fisher

- 'Father' of modern statistics and genetics.

- Experimental Design

- P-Values

- Analysis of Variance (ANOVA)

- Statistical Methods for Research Workers

- Genetic Theory of Natural Selection

- Design of Experiments

- Statistical Methods and Scientific Inference

Experimental Design

Null hypothesis to reduce bias.

P-Value

- Formalization of statistical testing

- Central concept for modern science

- Used to make determinations

- Widely misunderstood

Analysis of Variance (ANOVA)

Test to see if different groups have different means.

Null hypothesis (N_0)

Claim about probability. Can't be proven but can only be disproven.

P-Value — Defined

- Probability that the null hypothesis its tru given the data.

- If p-value is low, then suggests that null hypothesis can be rejected (reject the null).

- Fisher randomly picked 5% as a threshold (p =

Neyman-Pearson Approach

- Jerzy Neyman and Egon Pearson transformed statistical analysis from single hypothesis to decision-making framework that compares competing options.

- Introduced an alternative hypothesis (H1) that contradicts the null.

- Approach became universally accepted—standard tool in empirical research in social and natural sciences.

Type 1 Error: False Positive (Alpha)

- Probability of falsely rejecting a null hypothesis (incorrectly concluding that an effect exists when it does not).

- Also known as significance level.

- Conventionally set at 5%. This means that you would accept a 5% chance of committing a type 1 error.

- P-Value must be less than or equivalent to chosen alpha threshold for results to be considered statistically significant.

Type 2 Error: False Negative (Beta)

- Probability of failing to reject a false null hypothesis.

Problems with P-Values

- Often misinterpreted as certainty (e.g. probability that results are true)

- Often taken as rigid rules rather than arbitrary threshold.

- Confuse statistical significance with social significance.

- Doesn't tell you much without further information about the study (e.g. sample size, study design).

- Can lead to 'p-hacking': journals favour significant results so data tweaked until crosses 0,05 threshold.

Bayesian Statistics

- Precursor to frequentist statistics

- First proposed by amateur mathematician and pastor Thomas Bayes.

- Formulated by Dr. Richard Price

- Assumption: Knowledge is conditioned by what is already known (the prior)

- Prior knowledge influences estimations of probabilities.

- Probabilities are subjective estimates based on prior beliefs and experiences.

Non-Quantifiable Types of Uncertainty

- Ambiguity

- Vagueness

- Ignorance

Ambiguity

Differences in meaning.

Vagueness

Differences in boundaries and border.

Ignorance

Unknown unknowns—don't know what we don't know.

Scientific Consensus

The position generally agreed upon at a given time by most scientists specialized in a field.

Risk Analysis

- Not an effect on the physical environment.

- Unrealized in the physical world.

- Justice Rehnquist

Potential f

Realization of unwanted, adverse consequences to human life, health, property, or the environment.

Risk Management and Actuarial Literature

A loss calculated by probability of its occurrence over magnitude.

Biomedical

Relationship between exposure and outcome.

Economics

The variance in possible (positive or negative) returns.

Public Discourse

Multifaceted, included danger, fear, hazard, dread (perceived loss of control), exposure—can be perceived or actual.

Risk Assessment As Scientific Field

- Developed over past 4 decades—through journals, papers, conferences covering fundamental ides and principles on how to assess, manage, and communicate risk.

- Foundational to contemporary policy

- Probability (quantitative) of harm expressed as likelihood (qualitative).

'Understanding Risk'

- Influential report — "Red Book".

- Challenges linear notions of risk assessment.

- Risk assessment involves both facts and values.

- Analytical—deliberative model

- Science conducts risk assessment

- Values, interests and practicalities taken into consideration by risk management.

Values and Judgements

- What to include and exclude?

- What is 'threshold of harm'?

- How 'safe' is safe enough?

- When are large risks acceptable?

- When are small risks unacceptable?

Limitations of Risk-Based View of Uncertainty

- Bounded uncertainty: Certain aspects are highlighted, particularly those that are manageable through calculation.

- Leave invisible other uncertainties (ignorance, ambiguity, vagueness)

- Assumptions, values, and biases typically not acknowledged.

The Precautionary Principle

- General philosophy to managing threats of serious or irreversible harm where there is scientific uncertainty.

- Can be applied to all kinds of situations.

- The application of precaution requires increased risk avoidance where there is risk of serious harm and uncertainty is great.

Elements of a Precautionary Approach

1. Uncertainty should not delay regulations. Where ambiguous scientific proof of cause and effect is not available, it is necessary to act with a duty of care.

2. Where the benefits of early action are judged to be greater than the likely costs of delay, it is appropriate to take the lead and to inform society why such action is being taken.

3. Where there is the possibility of irreversible damage to natural life support functions, precautionary action should be taken irrespective of benefits.

4. Always listen for calls for change of course, incorporate representatives of such calls into deliberative forums and maintain transparency throughout.

5. Never shy away from publicity and never try to suppress information.

6. Where there is public unease, act decisively to respond to that unease by introducing extensive discussions and deliberative techniques.

7. Burden of proof should shift to those proposing development and away from those opposing it.

Risk

Outcomes and probabilities are known.

Indeterminacy

Causal chains and networks are open.

Critiques of Precautionary Principle

- Applied in an overreaching manner

- Decisions not based on existing data

- Limits innovation: eliminates the ability to take reasonable risks that are needed for individual or societal gains.

Manufacturing Uncertainty 'Playbook'

1. Shift focus of the debate away from the health, safety and/or the environmental damage caused by a product and onto the economic contribution that the product makes.

2. Cherry pick data and fund research showing the product is not dangerous or is less dangerous or problematic than thought.

3. Discredit scientists whose research challenges powerful interests by calling their work 'junk science' and promoting 'sound science' created by industry.

Sound Science

Ideological statement about stringent criteria for the use of science in policy making. Invoked mostly to call into question the validity of a science supporting regulation.

Requirement for 'Sound Science'

- Requires a higher burden of proof before action can be taken to protect public health and the environment: evidence can be disregarded if it does not meet impossibly high standards.

- Appeals to sound science rely on scientific outliers to sow doubt about mainstream findings.

- Provides a "window of opportunity" challenge regulatory science.