Lecture Notes on Data Interpretation, Research Justification, and Causality (comm research Sep 18, 2025)
iClicker troubleshooting and class logistics
Instructor notes that some students still have eye clicker issues; TA will coordinate with instructor to troubleshoot
Instructor can check on the back end for: registration status, whether the class is registered with iClicker
TA will contact iClicker tech rep if problems persist; rep is described as very helpful
Students should first report issues to the TA, who will collect information and pass it along
The IT support role is limited; the team will troubleshoot as a RA/TA collective, but students should go to the TA first to document details
If a student misses sessions, that will be considered in grading distribution; the instructor looks at patterns rather than penalizing isolated absences
Example given: a student (Nick) skipped the first three weeks, then started attending; the professor acknowledges life happens and will address it rather than punishing pattern alone
Current AVP (audio-visual and technology) metrics show about 86 ext{%} engagement, with time spent on technology; a rep was consulted on showing full results, but the provider doesn’t support displaying full results
Instrumental vs symbolic value: data interpretation exercise
The instructor introduces the concept with a data excerpt from a Newsweek article about AI in learning
Report details: 85 ext{ ext{%}} of teachers and students aged 14–22 use AI in some capacity; 66 ext{ ext{%}} in 2024; among students, 89 ext{ ext{%}} use AI for school work vs 77 ext{ ext{%}} the previous year
Instrumental value: numbers used to make concrete physical observations about the real world to guide decisions
Symbolic value: numbers used to reinforce or explain sociocultural beliefs, frame policy decisions, or represent broader attitudes
Example question: classify the given data as instrumental or symbolic; a student (Ella) answers ‘instrumental’ because the data show a real-world observation
Follow-up question: what value do we place in these data? This is trickier and invites scrutiny of data provenance and interpretation
Data sources and credibility: evaluating sources and language
Data source example: 2025 How America Learns report; data labeled as coming from Newsweek-like reporting
The instructor asks: what is the dataset, who produced it, and what is its trustworthiness?
Student (Gabby) admits not knowing much about the 2025 report but recognizes it as a data point; encourages asking for more information about the source and methodology
Additional data cited: as of 2024, 31% of public schools have a written AI-use policy; data from the “Post Panel,” a nationally representative survey of public K–12 schools; 18% of schools have certain policies (exact phrasing in transcript is imperfect)
The lack of a clear paradigm makes schools vulnerable to AI misuse; emphasis on asking about data origin, representativeness, and language used in reporting
The source mentions Child Trends (an advocacy organization translating education research into policy) and notes that the source’s language (e.g., “Post Panel, a nationally representative survey of public K–12 schools”) requires careful interpretation
The instructor encourages reading data critically, asking: Where do data come from? Who collected them? How were they analyzed? Is there peer review?
Peer review is introduced as a “gold standard” in publishing; overview of the process: author submits to editor, editor assigns to reviewers, reviewers provide comments, papers may be revised and re-submitted; timelines can be long (months)
The instructor notes limitations: peer review is not perfect; there are cases of false or flawed studies; Retraction Watch tracks such cases; some research may be produced without peer review (e.g., certain government panels or survey work)
Examples given of data sources lacking peer review or undergoing different vetting processes; emphasis on understanding data provenance and the vetting process
This discussion leads into a planned activity: students will read data critically and be prepared for a quiz the next day
Why do we do research? purposes, novelty, and impact
Research aims to provide answers to questions with a certain degree of certainty; justification often ties to predictive capacity and understanding causal trends in society
The instructor shares a personal anecdote from advertising: questions at work about consumer behavior that could not be answered in the workplace; pursued research to satisfy curiosity and inform practice
Core justification questions in research: Is the question worth asking? Can the study be novel or contribute to theory? Is the work feasible given resources and constraints?
Key criteria for evaluating research impact:
Novelty: originality and creativity; contributes to ongoing discussions or theories
Theory contribution: builds on or extends existing theories; helps integrate findings into broader knowledge; identifies gaps for future work
Feasibility: resources, time, tools, expertise available; research is often incremental due to practical limits
The speaker introduces the idea of theory-building and relation to broader knowledge: theory helps identify variables, connect findings, and explain relationships
There is a tension between novelty and feasibility; researchers may be overly ambitious, so feasibility often guides study design
The role of politics and ego in research: being mindful of how ambition and politics can influence research relevance and presentation
Three purposes of research (as reiterated from prior class): scripted (descriptive), exploratory, and explanatory
Explanatory research and causality: homophetic vs geographic explanations; nomothetic explanations
Distinction between explanatory and descriptive (and exploratory) research; explanatory power aims to answer the why behind observed relationships
Two kinds of causality discussed:
Nomothetic explanations: aims for generalizable patterns and statistical inference across populations
Idiographic/Geographic explanations: focus on deep understanding of specific cases or contexts; often qualitative; tries to identify all possible explanations for a particular event
The instructor uses two cases to illustrate expository approaches:
Gabrielle studies a specific manufacturer link to civil war (ideographic/idiographic approach); aims to explain a particular instance
Gabrielle’s roommate studies wars across history (nomothetic approach); looks for underlying generalizable factors across multiple cases
Geographic (idiographic) factors: micro-level attributes such as political orientation, age, region, religion, parental attitudes, personal experiences
Nomothetic factors: major predictors that apply across cases; emphasis on patterns that can be generalized
Normative vs statistical explanations: nomothetic explanations rely on statistical tests; idiographic explanations rely on qualitative observations and richer case detail
Correlation vs causality: correlation indicates a relationship but does not prove causation; must consider potential confounding variables or reverse causality
Spurious correlation: two variables related by coincidence or due to a hidden third variable
Post hoc reasoning: post hoc ergo propter hoc; the idea that because one event followed another, the first caused the second
Time ordering: establishing that the cause occurs before the effect is necessary to argue causality
The role of statistics in establishing causality: often require tests to demonstrate relationships and rule out alternative explanations
Early distinction between deep case studies (idiographic) and broader datasets (nomothetic) and the value of replication across studies for robustness
Missing variables in studies are common; subsequent research often adds variables (e.g., parental wealth) to explain outcomes; this highlights the need to consider omitted variable bias
Conceptual bridge: ways of knowing (epistemology) and how research approaches shape what we can claim to know
Ways of knowing, fallacies, and critical thinking in research
Humans rely on probabilistic reasoning and cause-and-effect reasoning; this is reinforced through everyday experiences and education
Fallacies and biases to watch for:
Authority: accepting a claim because it comes from an authority; can be flawed if authority is mistaken or biased
Personal experience: overgeneralization from a single or small set of experiences
Heuristics: mental shortcuts that can lead to systematic errors if not checked against data
The Mars “face on Mars” example: initial interpretation of a natural feature as a face; subsequent data (more imagery) debunks the illusion
The Uber example (careful: discussed in class context with a caveat): initial data implying a causal link between Uber availability and fewer events; later studies show the relationship may be explained by other factors
The value of the scientific approach: emphasizes systematic data collection, reproducibility, peer review, and skepticism toward simple cause-effect narratives
The plan for next week: unpack scientific approach further, including research ethics, true vs false claims, and the broader implications for knowledge and policy
Group activity and closing directions
Instruction to find a group of four students for an activity
The instructor emphasizes preparing for upcoming discussions and quizzes by applying the concepts discussed
Key terms and concepts to review
Instrumental value vs symbolic value
Data provenance, representativeness, and policy language
Peer review process and its limitations
Descriptive ( scripted ) vs exploratory vs explanatory research
Nomothetic vs idiographic (geographic) explanations
Correlation vs causation; spurious correlations; post hoc reasoning; time ordering
Ways of knowing: authority, personal experience, heuristics, scientific inquiry
Ethical considerations in research and the role of research in policy and society
Notable numbers to memorize (for quick recall)
85 ext{ ext{%}} of teachers and students age use AI in some capacity
66 ext{ ext{%}} in 2024
Among students, 89 ext{ ext{%}} use AI for school work; 77 ext{ ext{%}} previous year
US Department of Education data: 31 ext{ ext{%}} of public schools have a written AI-use policy (as of 2024)
18 ext{ ext{%}} of schools have a specified policy or related metric (as cited)
Time references: a half hour spent with a rep; research review timelines often range for peer review cycles
Mention of 2025 and 2024 data points: and as calendar years for the reports discussed
Quick recap of the big ideas
Research aims to provide reliable answers with some certainty, using a mix of descriptive, exploratory, and explanatory approaches
Data can be instrumental (observable, practical impact) or symbolic (interpretive, attitudinal/policy framing), and discerning which is at play requires careful consideration of data sources and context
Data provenance, representativeness, and peer review matter for trust and policy relevance
Causality requires careful reasoning beyond correlation, including time ordering and ruling out confounding factors
Humans rely on prior knowledge and heuristics, which can bias interpretation; the scientific method offers a disciplined path to minimize these biases while acknowledging fallibility
Ethics, transparency, and replication are central to credible research; be wary of claims that overstate causal findings without robust evidence
Practical next steps: practice data reading, identify data sources, assess whether findings are instrumental or symbolic, and consider how to ask better questions for future research