conclusion
The Historical Context and Rising Power of Data
Evolution of Data Accessibility: * In the recent past, information was significantly harder to gather and more expensive to analyze. * Analyzing one million credit card transactions a few decades ago involved physical paper receipts and a lack of personal computers for data processing.
Data Scarcity During the Great Depression: * The U.S. government lacked official statistics on unemployment and Gross Domestic Product (). * In , Herbert Hoover inaccurately declared the Great Depression over based on outdated information. * Hoover claimed in a State of the Union address that Americans were out of work; the reality was jobless citizens, with unemployment increasing by every week. * James Surowiecki observed in The New Yorker that "Washington was making policy in the dark."
Modern Data Proliferation: * Society is now "awash in data," allowing statistical tools to address significant social challenges across various sectors.
The Future of Football and Head Trauma
Malcolm Gladwell’s Thesis (): * Posed the question in The New Yorker: "How different are dog fighting and football?" * The comparison arose as quarterback Michael Vick was reinstated in the NFL following a prison sentence for dog fighting, coinciding with emerging data on football-related brain trauma.
Evidence of Neurological Damage: * Head trauma in football is associated with depression, memory loss, dementia, and other neurological disorders. * Case Study: Dave Duerson: A former safety and Super Bowl winner for the Chicago Bears who committed suicide by shooting himself in the chest, specifically requesting his brain be studied post-mortem.
Statistical Findings: * A phone survey of randomly selected former NFL players (with at least years in the league) found: * For players over , the rate of diagnosis for dementia or Alzheimer’s was , which is times the national average. * For younger players, the diagnosis rate was times the national average.
Scientific Research: * Ann McKee (Veterans Hospital, Bedford, MA): Documented the buildup of abnormal proteins called "tau" in the brains of athletes, leading to Chronic Traumatic Encephalopathy (), a progressive neurological disorder. * Kevin Guskiewicz (Sports Concussion Research Program, UNC): Used sensors in helmets to record impact force. Data shows players receive blows equivalent to a car crash windshield impact at .
Statistical Uncertainties: * Questioning if current findings represent a "cluster" or statistical aberration. * Investigating causality: Is damage caused by the sport itself, or other factors like steroid use or the predispositions of men who choose contact sports?
Ethical Implications: * Gladwell argues that dog fighting became unacceptable because the owner submits an animal to suffering for entertainment and profit; statistics are currently being used to determine if football faces a similar moral or regulatory reckoning.
The Rising Incidence of Autism
Current Statistics (CDC ): * in American children was diagnosed with an Autism Spectrum Disorder () based on data. * The rate has climbed from in in and in in , nearly doubling in less than a decade. * Boys are times as likely to be diagnosed as girls.
Epidemic of Autism vs. Epidemic of Diagnosis: * Increased awareness among parents, doctors, and teachers may lead to more diagnoses of cases previously labeled as "learning disabilities."
The Burden of ASD: * The average lifetime cost of managing an ASD for a single individual is .
Statistical Inquiries into Causality: * Environmental/Demographic Factors: A UC Davis study identified locations in California with autism rates double that of surrounding areas; these clusters were neighborhoods with white, highly educated parents. * Genetic Factors: Research on twins shows the likelihood of both children having ASD is higher among identical twins than fraternal twins.
Debunking Misconceptions: * Vaccines and Thimerosal: Despite claims by individuals like Congressman Dan Burton (whose grandson became autistic after receiving multiple shots containing mercury-based thimerosal), statistics show no link. * Autism rates did not decline when thimerosal was removed from the MMR vaccine, nor are rates lower in countries that never used it. * Confusion between correlation and causation persists because ASD symptoms often appear between a child's first and second birthdays, coinciding with vaccination schedules.
Identifying and Rewarding Teacher Quality
Value-Added Assessments: * Evaluating teachers by measuring student progress over time, rather than absolute test scores. * Statistical models can refine this by accounting for student demographics (race, income, prior performance/aptitude).
Implementation and Controversy: * New York City (): Published ratings for teachers based on value-added data. * Los Angeles (): Similar rankings published by the Los Angeles Times. * U.S. Secretary of Education Arne Duncan supports these measures, stating, "Silence is not an option." * Teachers' unions oppose this, with the NYC union spending on ads claiming "This Is No Way to Rate a Teacher."
The "Noise" in Educational Data: * Doug Staiger (Dartmouth College): Notes that results for a single teacher are "noisy" due to random factors (e.g., a broken air conditioner on test day). * The year-to-year correlation in performance for a single teacher is approximately . This is oddly identical to the year-to-year correlation for Major League Baseball players' batting averages or .
Air Force Academy Study (Scott Carrell and James West): * Cadets were randomly assigned to standardized core courses (e.g., introductory calculus) to eliminate selection effects. * Findings: Less experienced professors with fewer advanced degrees produced students with higher scores in intro courses and better student evaluations. * The Twist: Students of experienced, highly credentialed (often "crusty") professors performed better in follow-on courses. * Interpretation: Less experienced teachers may "teach to the test," whereas experienced professors focus on core concepts that matter long-term.
Statistical Tools in Fighting Global Poverty
Esther Duflo’s Approach: * Duflo (MIT), winner of the John Bates Clark Medal (), uses randomized controlled experiments () to test interventions in developing countries.
Addressing Teacher Absenteeism in India (Rajasthan): * A sample of one-teacher schools was given cameras with tamperproof date/time stamps; teachers took photos with students to prove attendance for a bonus. * Absenteeism dropped by compared to a control group of schools; test scores and graduation rates increased.
Fertilizer Use in Kenya: * Farmers often fall into a "poverty trap" where they lack cash for fertilizer at planting time. * Duflo found that offering free fertilizer delivery immediately after harvest (when farmers had cash) increased use by to percentage points.
Gender and Financial Responsibility (Côte d'Ivoire): * Different crops respond to rainfall differently: Men grow cocoa and coffee; women grow plantains and coconuts. * Duflo found that when women’s crops flourished, they spent extra cash on family food; when men's crops flourished, they did not.
Data Privacy and Ethics
Predictive Analytics at Target: * Target uses a "pregnancy prediction score" based on products (e.g., unscented lotion, vitamin supplements, large bags of cotton balls) to identify women in their second trimester. * Case Study: A Minneapolis father complained to Target about his high school daughter receiving baby coupons, only to discover later she was indeed pregnant and due in August. * Masking Tactics: Companies often mix "spooky" targeted coupons (like diapers) with random items (like lawn mowers or bowling socks) to hide how much they know.
The Value of Facebook: * Investors value Facebook for its data on user behavior, locations, and spending habits, rather than its social utility. * Chris Cox (Facebook VP): "The challenge of the information age is what to do with it."
Government and Law Enforcement: * Surveillance: GPS tracking and facial recognition technology. * Legal Precedent: In United States v. Jones (), the U.S. Supreme Court ruled unanimously that police cannot attach GPS tracking devices to private vehicles without a warrant. * DNA Databases: Governments debate whether to include DNA from all citizens, all arrestees, or only convicted criminals.
Final Conclusion: * Statistics is a powerful tool, similar to fire, knives, or automobiles; it enhances life when used wisely but causes problems if abused. * Math cannot replace human judgment.