Statistical Thinking Notes

Planning the study: Formulate a testable research question and establish a clear data collection design.
Examining the data: Utilize graphs and descriptive statistics to identify patterns, distributions, and potential deviations in observed data.
Inferring from the data: Apply valid statistical methods to extend conclusions beyond the collected sample and assess if findings could be due to chance.
Drawing conclusions: Determine group applicability, population generalizability, and whether cause-and-effect claims are supported.

Variation in Data: The fundamental principle of statistics is that data values vary; analyzing the pattern of variation (the distribution) yields essential research insights.
Comparing Distributions: Evaluating full distributions provides significantly more insight than comparing single summary measures like medians or means.
Short, Moriarty, & Cooley (1995) demonstrated that comparing patient reading abilities against pamphlet readability distributions reveals critical discrepancies, such as $27\%$ ( $17$ out of $63$ ) of patients reading below the simplest pamphlet level.

Statistical Significance: A result is statistically significant when it is highly unlikely to arise purely by random chance.
P-Value: The probability of observing an outcome at least as extreme as the study result, assuming random chance is the only operating factor.
Significance Level Threshold: Commonly set at $\alpha = 0.05$ ; a $p\text{-value}$ smaller than this threshold leads to rejecting the pure-chance hypothesis.
Hamlin, Wynn, & Bloom (2007) tested pre-verbal infant choices between helper and hinderer toys, finding $14$ out of $16$ infants selected the helper toy ( $p\text{-value} = 0.0021$ ), providing strong evidence of genuine preference.

Population vs. Sample: A sample is a subset selected from a broader target population to infer overall population parameters.
Random Sampling: Gives every member of a population an equal probability of selection, ensuring representative results free from selection bias.
Margin of Error: Quantifies expected variation due to random sampling alone (approximately $\frac{1}{\sqrt{n}}$ , where $n$ is sample size).
The 2004 General Social Survey (GSS) found $83.6\%$ ( $817$ of $977$ ) of respondents felt rushed, yielding a $95\%$ confidence interval of $80.6\%$ to $86.6\%$ with a margin of error of $\pm 3\%$ for all U.S. adults.

Random Assignment: A probability-based method to divide subjects into experimental treatment groups, balancing potential confounding variables (e.g., age, gender, background).
Establishing Causality: Essential for establishing cause-and-effect conclusions because it isolates the primary treatment variable as the explanation for observed group differences.
Ramsey & Schafer (2002) and Amabile (1985) examined intrinsic versus extrinsic motivation in creative writing; random assignment yielded balanced groups where the intrinsic group scored higher (mean $19.88$ , standard deviation $4.40$ ) than the extrinsic group (mean $15.74$ , standard deviation $5.25$ ) with $p\text{-value} = 0.002$ , confirming a causal relationship.

What was the primary research question in a recent study in your field, how were participants selected, were random assignment and statistical controls used, and do you agree with the drawn conclusions regarding significance, confidence, generalizability, and causality?
Is it reasonable to use a random sample of $1{,}000$ individuals to draw conclusions about all U.S. adults? Explain why or why not.