NS Stats and Exp Design 1

What is the definition of science?

The concerted human effort to increase our understanding of the natural world and how it works, using observable physical evidence as the basis of that understanding.

What are the 5 phases of the scientific process?

I) Identify the research question/problem and review literature; II) Design the study; III) Conduct the experiment(s); IV) Data analysis; V) Communication of findings.

What role does statistics play in science?

It provides a method for analyzing and interpreting data objectively, separating real patterns from random noise and quantifying uncertainty.

What role does experimental design play in science?

It provides the logical structure that makes scientific inference possible, ensuring data can answer a specific question, support causal conclusions, and be analyzed statistically.

What are the key elements of experimental design?

Manipulation of independent variables, selection of appropriate controls, and randomization.

How does good experimental design handle natural variation between subjects?

It doesn't eliminate variation, it manages it - minimizing unwanted 'noise' from the experiment so the real biological 'signal' can be examined, and identifying confounds before they bias results.

What did Karl Popper argue about scientific hypotheses and theories?

That they should be based on the logic of falsification: you can never prove a hypothesis true, only reject (falsify) it by finding contradictory evidence.

Why can't 'All swans are white' be proven true by finding more white swans, but can be disproven by one black swan?

Confirmatory evidence can never rule out an unobserved exception, but a single disconfirming case definitively falsifies a universal claim.

What is inductive reasoning?

Accumulating multiple corroborating observations to propose a general theory or hypothesis (e.g., observing many metals expand when heated leads to inferring all metals expand when heated).

What is deductive reasoning?

Critically testing logical, falsifiable predictions that follow from a general theory or hypothesis, in an attempt to falsify it.

What is the hypothetico-deductive method?

Step 1: Postulate a hypothesis or theory. Step 2: Search for evidence that contradicts it by testing specific falsifiable predictions using controlled experiments.

Why is deduction a more effective way than induction to demonstrate causal relationships?

Induction can only increase confidence through accumulating confirmations and never yields certainty, while deduction actively tests falsifiable predictions that can definitively rule a hypothesis out.

What is the difference between an alternative hypothesis (HA) and a null hypothesis (H0)?

HA proposes the manipulation has an effect; H0 proposes it has no effect. Popper argued the goal of an experiment should be to obtain evidence that rejects H0.

How does modern science use statistics with hypothesis testing?

Statistics is used to reject the null hypothesis, not to confirm the alternate hypothesis - e.g., if an effect can't be explained by chance (P < 0.05), the scientist rejects H0 and accepts HA.

Why must a null hypothesis be falsifiable?

An unfalsifiable H0 (e.g., 'there are no pink elephants') can never be definitively rejected, since you can never rule out every possibility - leaving permanent uncertainty.

What is anecdotal induction?

Drawing a broad generalization from one or a few personal observations or stories, using a small, unsystematic sample (e.g., inferring the medial temporal lobe is essential for memory from the single case of Patient H.M.).

What is confirmation-based induction?

Using repeated confirmatory (not falsifying) observations to support a hypothesis; confidence increases with replication and consistency but remains always provisional, never definitive.

What is post hoc induction?

Inferring a general causal claim after the fact from an observed outcome, based only on temporal sequence, without prior prediction or experimental control (e.g., inferring the frontal lobe controls personality from Phineas Gage's accident).

What is the core flaw of post hoc induction?

It confuses temporal order with causation - just because A happened before B doesn't mean A caused B - and it lacks controls or alternative-explanation testing.

What is quack science?

The promotion of unsubstantiated, fraudulent, or false health practices, products, or beliefs presented as scientifically valid, often relying on weak/anecdotal evidence, exaggerated claims, and a refusal to acknowledge scientific doubt.

Why is the field of neuroscience especially vulnerable to quack science?

The CNS is extremely complex and only partially observable, and research on the human brain is ethically constrained, forcing heavy reliance on inductive reasoning - which never yields certainty and can be exploited (e.g., phrenology, the left/right-brain myth, brain-training claims).

What is a dependent variable?

The variable that changes as a result of the independent variable manipulation - the outcome variable that is hypothesized to 'depend' on the IV.

Why do investigators often use more than one dependent variable to characterize mental constructs like stress or hunger?

Because such constructs can't be captured by any single ideal measure; using multiple complementary measures (e.g., self-report, behavior frequency, hormone levels) provides converging evidence and captures different aspects of the construct.

What is an operational definition and why is it important?

A precise description of exactly how a variable was measured or manipulated (e.g., the exact scale used and its precision); it increases the odds that investigators in different labs can replicate the same measurement.

What is noise in measurement?

Random error - the tendency of measurements to scatter unsystematically around the true value due to uncontrollable random factors.

What is bias in measurement?

A consistent deviation from the true value in a specific, directional way.

Why are there true values for some but not all dependent measures?

Physical quantities (e.g., weight) have a definable true value ('bull's-eye'), but constructs based on subjective human judgment (e.g., wine quality ratings, medical diagnoses) have no objectively 'true' or 'correct' value.

Why is it often easier to estimate noise than bias?

Noise (the spread of measurements) can be measured without knowing the true value, whereas measuring bias requires knowing where the true 'bull's-eye' is, which is often unknown.

What are methods for reducing noise in a measurement?

Use the best measurement equipment available; take repeated measurements and average them; standardize test conditions across subjects; and select a homogeneous population of subjects.

What are the three main sources of bias in measurement?

1) Problems with the measurement device, 2) Observer bias, 3) Subject bias.

What is observer bias?

When a researcher's expectations, personal prejudices, or subjective opinions unintentionally influence how they record or interpret data.

What is subject bias?

When participants in a study consciously or unconsciously alter their responses to match what they believe the researcher expects.

How can you minimize observer bias?

Automate data collection/entry, simplify the observer's task (use objective measures, record frequency rather than intensity, video-record sessions), limit data collection to observable events, and make the observer 'blind' to the subject's treatment condition.

How can you minimize subject bias?

Make measurements nonverbal, use physiological recordings as the response measure, and make the subject 'blind' to their treatment condition and to the goals of the experiment.

How can you minimize noise arising from a measurement device?

Ensure the device is functioning normally and recently calibrated, ensure the operator is well-trained and rested, and use a device appropriate to what is being measured.

What is a nominal scale of measurement?

The simplest scale; subjects are assigned to mutually exclusive categories with no inherent order (e.g., gender, diagnosis, favorite juice) - it is qualitative.

What is an ordinal scale of measurement?

Numbers represent rankings (bigger means more), but the actual difference between ranks can't be measured, and there is no true zero point (e.g., rankings of a professor).

What is an interval scale of measurement?

A rank-order scale with a constant interval size but no true zero point, so ratio statements ('twice as much') are not meaningful (e.g., Fahrenheit/Celsius temperature, time of day).

What is a ratio scale of measurement?

An interval scale with an absolute/true zero point representing the total absence of the property being measured, which allows valid ratio statements (e.g., weight, height, Kelvin temperature).

Why is Kelvin a ratio scale for temperature while Fahrenheit and Celsius are only interval scales?

Fahrenheit and Celsius have arbitrary zero points, so you can't say one temperature is 'twice as hot' as another; Kelvin's zero represents an absolute absence of thermal energy, so 200K really is twice as hot as 100K.

What is a population (in the context of sampling)?

The entire group of subjects about which you want to draw inferences (e.g., all US citizens, all mice of a particular inbred strain).

What is a sample?

A small fraction of the subjects in a study population, used to infer truths about the entire population; it should accurately reflect the population.

What two factors strongly influence how well a sample represents the population?

Sample size (larger samples are usually more representative) and sampling procedure (the method used to select subjects).

What is non-probability sampling, and when is it typically used?

Investigators select subjects based on personal judgment or accessibility/willingness, so not all population members have an equal chance of selection and the sample can be biased; it's commonly used in the exploratory stage of a study since it's quick and cheap.

What is probability sampling?

Investigators randomly select subjects so that every member of the population has an equal chance of being selected; it is considered the best way to obtain a representative sample.

What is simple random sampling, and what are its advantage/disadvantages?

Every individual is assigned a number and subjects are selected at random (e.g., via a random number table). Advantage: equal selection probability for everyone. Disadvantages: cumbersome for large populations, and may not represent a heterogeneous population well by chance.

What is stratified random sampling, and what are its advantage/disadvantages?

The population is divided into subgroups ('strata'), and subjects are randomly selected within each stratum proportional to its abundance. Advantage: ensures each stratum is represented. Disadvantage: requires classifying every member into one stratum, which can be difficult for ambiguous categories.

What is systematic random sampling, and what are its advantage/disadvantages?

Assign numbers to all members, compute k = N/n, randomly pick a start between 1 and k, then select every kth person. Advantage: simple, cost-effective, works even with diverse populations. Disadvantage: requires an accurate population size, and can produce biased samples if the list has a hidden pattern matching k.

What is an experimental unit?

The smallest division of an experiment that could receive a different treatment (e.g., an individual subject); also called a 'replicate,' denoted 'n'.

What is the difference between true replicates and pseudoreplicates?

True replicates are independent measurements from different experimental units (e.g., one blood sample from each of 3 different mice); pseudoreplicates are non-independent, repeated measurements from the same experimental unit (e.g., 3 blood samples from 1 mouse).

Why is replication important in experimental design?

It provides an estimate of variability (noise) in the sample population, increases the accuracy of statistical estimates, and provides insurance against random/stochastic events.

What is simple pseudoreplication?

When each treatment level contains only one experimental unit (e.g., one tank), and multiple concurrent measurements taken from within it (e.g., individual fish in that tank) are treated as if they were independent replicates.

What is temporal pseudoreplication?

When multiple measurements are taken from a single experimental unit over time and are treated as independent replicates (e.g., 10 repeated recordings from the same animal treated as n=10, when the true n is 1).

Are pseudoreplicates always a bad thing?

No - pseudoreplicates can improve the accuracy of the estimate for a given experimental unit (e.g., measuring 10 fish in one tank gives a better mean estimate than measuring 1), but only true replication reveals variation across experimental units.

Why is treating pseudoreplicates as true replicates problematic?

It makes the investigator believe the study has more experimental units than it really does, artificially inflating the sample size and producing inaccurate measures of variation and invalid statistical results - making the results look more successful than they really are.

What is the independent variable (IV)?

The variable the researcher manipulates or varies; it's called 'independent' because it is not influenced by other variables in the study.

What is the dependent variable (DV)?

The variable that changes as a result of the independent variable manipulation - the outcome being measured.

What are the two types of independent variables?

Subject variables (cannot be manipulated, only used to group subjects, e.g., sex, age, species) and experimental variables (directly manipulated by the researcher, e.g., drug dose).

What is a negative control group?

A group that ensures there is no effect when there should not be one; it receives no manipulation, a sham manipulation, or a placebo ('sugar pill').

What is a positive control group?

A group that ensures there is an effect when there should be one; it establishes that the researcher can produce a positive result and increases confidence in negative results elsewhere in the study.

What are the three types of extraneous variables?

Situational (from the testing environment), Subject (from individual differences among subjects), and Experimenter (from unintentional experimenter influence).

How can situational extraneous variables (e.g., noise, lighting, time of day) be controlled?

By implementing standardized procedures and instructions so that testing conditions are as similar as possible for all participants.

What are order effects, and how are they controlled?

Changes in scores over repeated trials due to fatigue or learning; controlled through counterbalancing (varying the order treatments are presented across subjects) or randomized block designs.

What is a randomized block design (for controlling order effects)?

Each presentation sequence is treated as a 'block,' and the sequence within each block is randomized independently for every subject, minimizing potential order effects.

What are the ways to control subject-related extraneous variables?

Select a homogeneous population, use subjects as their own control (within-subject design), use blocking, use matching, or randomly assign subjects to treatment levels.

What is the main weakness of selecting a homogeneous population of subjects to control extraneous variables?

Results can only be generalized to that specific subset of the population.

What is a within-subject design ('using subjects as their own control')?

The same subjects receive all treatment levels, increasing statistical power and reducing the impact of individual differences, but risking carry-over, fatigue, and practice effects.

What is blocking?

Explicitly incorporating a known source of individual variation (e.g., age, gender) into the design by assigning an equal number of each subgroup to each treatment group.

What is matching?

Pairing subjects across treatment levels based on a characteristic that may confound the dependent measure (e.g., genetics, anxiety level), then randomly assigning each member of the matched pair to a different treatment group.

Why is matching difficult to implement in practice?

Perfect matches don't exist (even identical twins aren't identical), there may be too many variables that need matching, researchers don't always know which variables matter most, and regression to the mean can distort matching over long-term studies.

What does randomly assigning subjects to treatment levels accomplish?

It distributes both known and unknown confounding variables roughly equally across treatment groups, increasing confidence that any group differences after treatment are due to the treatment itself.

What is a single-blind experimental procedure?

The experimenter withholds treatment information from the subjects, controlling for expectation effects the subjects may have.

What is a double-blind experimental procedure?

Neither the subjects nor the experimenters who interact with them know the treatment assignment, which prevents both subject expectation effects and unconscious experimenter influence.

What are the 4 threats to the validity of an experimental design?

Statistical conclusion validity, construct validity, internal validity, and external validity.

What is statistical conclusion validity?

Whether there is a true relationship between the IV and DV; threatened by low statistical power, violated statistical test assumptions, or noisy data.

What is construct validity?

Whether the constructs used to represent the IV/DV are valid; improved through triangulation (multiple measures), agreement with established measures, and operational definitions.

What is internal validity?

The extent to which a study's results reflect a true cause-and-effect relationship within that specific group of subjects; maximized by minimizing extraneous variables.

What is external validity?

The extent to which a study's results can be generalized to other people, settings, times, and real-world contexts.

What is an experimental design?

A plan for assigning experimental units to treatment conditions that allows causal inference about the IV-DV relationship, minimizes extraneous variables, and reduces variability within treatment levels.

What are the three classes of experiment?

True experiments, quasi-experiments, and descriptive (observational) experiments.

What is the defining feature of a true experiment, and why does it provide the most robust causal evidence?

Random assignment of experimental units to treatment conditions combined with manipulation of an IV; randomization distributes both known and unknown confounds roughly equally across groups, making the treatment the most plausible explanation for any resulting group differences.

What is a quasi-experiment, and what does the investigator control vs. not control?

It resembles a true experiment (may manipulate a variable or exploit a naturally occurring intervention) but lacks random assignment. The investigator controls the IV levels and extraneous factors, but not group assignment; major weaknesses are selection bias and confounding.

What is a descriptive (observational) experiment?

The investigator observes and measures variables without manipulating them; there is no independent variable, so there is no opportunity to test causal relationships - only correlations.

Why can't descriptive studies establish causation?

Without manipulation of an IV or controlled random assignment to groups, any observed relationship could reflect reverse causation or an unmeasured confounding variable rather than a true causal link.

What are the strengths and weaknesses of descriptive studies?

Strengths: allow studying the world as it naturally exists, are feasible for ethically/practically unmanipulable variables, and are valuable for discovering patterns and generating hypotheses. Weaknesses: vulnerable to confounding and reverse causation, and cannot establish causality.

What is a confounding (lurking) variable? Explain using the shoe-size/reading-score example.

A variable that influences both the supposed cause and effect, creating a spurious correlation. In children, age increases both shoe size and reading test scores, so age (not shoe size) is the true cause of the apparent shoe-size/reading correlation.

What is the difference between correlation and causation?

Correlation means two variables statistically covary; causation means changes in one variable directly produce changes in the other. Correlation alone cannot establish which variable causes the other, or rule out a confounding variable.

What is the difference between a prospective and a retrospective design?

A prospective design follows subjects forward in time from a potential cause toward an outcome; a retrospective design looks backward from an existing outcome to identify potential prior causes.

What is the difference between a between-subjects and a within-subjects design?

In a between-subjects design, each subject receives only one treatment and comparisons are made between different groups (individual differences contribute to error). In a within-subjects design, the same subjects receive all treatments and serve as their own control (reduces individual-difference error, but risks order, carryover, and learning effects).

What is a single-factor design, and what is its main limitation?

A design where one independent variable is manipulated and compared; its limitation is that many phenomena cannot be explained by just one factor and may require multiple factors to explain.

What is a two-factor (factorial) design, and what added benefit does it provide over a single-factor design?

A design that manipulates two independent variables simultaneously (e.g., a 2x2 design); besides testing each factor's main effect, it allows testing for an interaction between the two factors.

What is an interaction between two independent variables?

When the effect of one independent variable on the dependent variable depends on (covaries with) the level of the other independent variable - e.g., two drugs that are each safe alone but lethal in combination.

What is the difference between a 2-factor between-subjects design and a mixed-model design?

In a 2-factor between-subjects design, both factors are between-subjects (each subject experiences only one treatment combination). In a mixed-model design, at least one factor is between-subjects and at least one is within-subjects (e.g., drug group is between-subjects, but each person is measured both before and after treatment).

What is the difference between a test for association and a test for a causal relationship?

A test for association determines whether one variable covaries with another (e.g., smoking history and lung cancer incidence); a test for causal relationship shows that one variable (e.g., smoking) actually causes the other (e.g., lung cancer).

Before the 19th century, how were diseases explained in Europe, and why?

Through metaphor rather than experimental science - e.g., the four humors (disease from an excess of one of 4 bodily fluids) and miasma theory (disease caused by bad odors) - largely because microorganisms were unknown until the late 1600s.

What is an "ecological study," and what is its experimental unit?

A study used to identify associations where the experimental unit is a group (e.g., a nationality) rather than an individual.

What are the interpretive limitations of an ecological study?

Because the results are correlational and based on group-level averages, one can only speculate about causal relationships; you cannot be certain that the specific individuals exposed (e.g., smokers) are the same individuals who experienced the outcome (e.g., lung cancer deaths) at a higher rate.

In the alcohol/lung cancer example, why is smoking considered a confounding variable rather than alcohol use being a true cause?

People who drink heavily also tend to smoke, and smoking is a major, biologically plausible risk factor for lung cancer, whereas there is no obvious physiological mechanism linking alcohol directly to lung cancer.

In the exercise/heart health example, what is the confounding variable?

Age - younger people tend to both exercise more and have naturally better heart health, which can create a spurious link between exercise and heart health.

In the child IQ/reading ability example, what is the confounding variable?

Family socioeconomic status - wealthier households can provide more books and educational opportunities, boosting both measured IQ-related outcomes and reading ability.