Data Skills for Scientists: Navigating Uncertainty and Study Design

Comprehensive vocabulary flashcards covering introductory statistical concepts, data types, uncertainty, lurking variables, paradoxes, and experimental design principles from Forum 1 of Data Skills for Scientists.

Data

Measurements or observations directly collected from subjects without any added interpretation or context.

Information

Measurements or observations that have been processed, calculated from data, or contextualised with interpretation to inform understanding or decision-making.

Subject

The individual, case, or object upon which measurements or observations are made during data collection.

Variable (Statistical)

A characteristic or attribute of a subject that is measured or observed and can take on different values.

Categorical Variable

A qualitative variable recorded as words or categories, where the numerical distance between different values has no meaning.

Nominal Variable

A categorical variable whose categories have no natural ordering (e.g., colours of cars, flavours of ice cream).

Ordinal Variable

A categorical variable whose categories follow a natural, hierarchical ordering (e.g., Olympic medals, course grades).

Numerical Variable

A quantitative, interval, or scale variable recorded as numbers, where the distance between values carries meaningful quantitative significance.

Discrete Variable

A numerical variable that takes values from a distinct, countable set of numbers, typically representing counts (e.g., number of coin toss heads).

Continuous Variable

A numerical variable that can take any value within a continuous interval (e.g., temperature, time, weight, length).

Rectangular Data

A structured data format arranged in a two-dimensional grid where each row represents a subject and each column represents a variable.

One-way Table

A tabular summary of a single categorical variable showing the frequency or count of subjects falling into each category.

Two-way Table (Contingency Table)

A table summarising two categorical variables simultaneously by displaying counts across all possible combinations of their categories.

Aggregate Data

Data that summarises and combines individual-level observations, often functioning as information unless subjects and variables are formally redefined.

Accuracy

How closely measurements represent the true value they intend to measure on average, closely linked to validity and serving as the opposite of bias.

Precision

How consistently measurements reproduce similar values under the same conditions, closely linked to reliability and serving as the opposite of variability.

Accuracy and Precision (Target Analogy)

A conceptual model illustrating measurement uncertainty: accuracy reflects how close hits are to the center bullseye on average, while precision reflects how tightly clustered the hits are to one another.

Lurking Variable

An unmeasured or unobserved variable not included in a study that, if accounted for, can substantially alter the apparent relationship between studied variables.

Simpson's Paradox

A statistical phenomenon in which an association consistent across all subgroups reverses or disappears when the data from all subgroups are combined.

Common Response

A situation where an observed relationship between two variables $X$ and $Y$ is explained by both variables responding simultaneously to changes in a third lurking variable $Z$.

Confounding

A condition occurring when the effects of two or more variables on a response variable cannot be distinguished from one another.

Descriptive Statistics

Statistical procedures focused on summarising, organizing, and visualising data to highlight and clarify its primary features.

Inferential Statistics

Statistical methods used to draw conclusions, test hypotheses, or make predictions about a population from sample data while explicitly quantifying uncertainty.

Observational Study

A study design where researchers observe subjects and measure variables without intervening or manipulating treatment assignments.

Designed Experiment

A study design where researchers deliberately impose specific treatments or conditions on subjects to measure the resulting responses and establish causality.

Treatment

The specific controlled experimental condition or intervention deliberately applied to subjects by researchers in an experiment.

Response Variable

The outcome variable measured on subjects after receiving a treatment to assess the treatment's effect.

Selection Bias

A systemic bias occurring when some members of a population are more likely to be selected for a study than others, making the sample unrepresentative.

Survivorship Bias (Abraham Wald Bomber Analysis)

A form of selection bias where conclusions are drawn solely from surviving cases; Abraham Wald showed that returning aircraft with bullet holes indicated where planes could survive damage, meaning armor was needed on the undamaged engines instead.

Survey

A type of observational study where participants are asked to self-report information.

Response Rate

The percentage of individuals invited or sent a survey who ultimately choose to respond and participate.

Control (Principle of Experimental Design)

The experimental practice of managing and holding constant unwanted sources of variation affecting the response variable.

Method of Comparison

The experimental strategy of comparing outcomes between an active treatment group and a control group receiving a standard treatment or placebo.

Placebo

An inactive substance or procedure designed to look, smell, and feel like the experimental treatment without containing active therapeutic components.

Placebo Effect

A measurable change in a subject's response variable caused by the psychological perception or expectation of receiving a treatment.

Blinding

The practice of withholding knowledge of treatment assignment from subjects to prevent bias and neutralize placebo effects.

Double-Blind Study

An experimental trial in which neither the participants nor the experimenters administering treatments know which group each participant belongs to.

Triple-Blind Study

An experimental trial in which participants, treating experimenters, and the data analysts assessing results are all kept unaware of treatment allocations.

Randomisation

The random allocation of subjects to treatment and control groups to eliminate selection bias and balance out lurking variables across groups.

Blocking

The grouping of subjects with known similar characteristics expected to influence response prior to an experiment, followed by random assignment of treatments within each block.

Matched Pairs Design

A randomized block experimental design where each block consists of two closely matched units (e.g., identical twins) or a single unit receiving both treatments.

Replication

The inclusion of a sufficient number of experimental units in each treatment group to reduce chance variation and improve reliability.

Balanced Experiment

An experimental design characterized by having an equal number of subjects allocated to each treatment and control group.