Comprehensive Study Notes: Data Visualization Analysis and Course Overview

Administrative Announcements and Course Assignments

• Team Contract and Statement of Individual Work:

  • Submission is required by the end of the day.

  • A total of $20$ students have not yet submitted their forms.

  • Ensure the communication section is fully completed, as several submissions missed this section.

  • Students looking to form or join a team should contact the instructor directly to be assigned to partners before finalizing the form.

• Problem Set 1 Details:

  • Posted under the Week 3 module in Canvas.

  • Official due date is next Friday.

  • Format: Provided as an HTML preview, but must be executed and submitted as a Jupyter Notebook.

  • Team Submissions: Teams only submit one single file. Canvas team groups automatically apply the grade to all listed group members. All contributing team members must have their names explicitly listed inside the notebook file.

  • Artificial Intelligence Policy: AI tools (e.g., ChatGPT) are permitted for assistance, provided students document which tool was used for each specific problem (e.g., describing scatter plots versus hexbin plots). Students must not let AI perform all the work.

• Apache Spark Google Colab Assignment:

  • Location: Located at the bottom of the module assignments.

  • Environment Setup: Set up Spark directly inside Google Colab and save a copy of the notebook to a personal Google Drive.

  • Dataset: Complete works of William Shakespeare text file.

  • Primary Task: Program a Spark application to compute word counts aggregated by their initial starting character to identify which letter begins the highest frequency of words across the text corpus.

  • Error Documentation: Save the final code as a Jupyter Notebook. If execution bugs or technical errors occur, document and explain the specific failure within the notebook to review during next Wednesday's session.

• Schedule and Reading:

  • Next week follows a shortened schedule; the physical campus building will be closed.

  • Class discussion will occur asynchronously entirely on Canvas (no in-person meeting).

  • Required Reading: Case study article regarding Meta and its core operations in data management.

Data Visualization Analysis: Most Common 4-Digit PINs

• Dataset & Visual Structure:

  • Sample Size: Contains $3,400,000$ data points extracted from security data breaches (analyzed by Gail and Sam).

  • Scope: Includes chip-and-PIN cards, phone passcodes, and standard 4-digit security codes.

  • Heatmap Grid Encoding: First two digits represent the vertical/horizontal axis, mapping against the final two digits.

  • White Cells: Represent the most frequently chosen PINs.

  • Black Cells: Represent the least frequently chosen or unobserved PINs.

• Observed User Behavioral Patterns:

  • Birth Dates & Years: A massive concentration of codes utilizes birth dates ($DD/MM$ or $MM/DD$ formats) or birth years ($19XX$).

  • Sequential & Repeating Patterns: 1234 is among the most common overall. Distinct visual diagonal lines across the matrix represent repeated two-digit combinations or identical repeating digits (e.g., 1111, 2222).

  • Physical Keypad Trajectories: Codes such as 2580 or 0852 represent straight vertical lines drawn down the middle column of standard phone/ATM keypads.

  • Linguistic Puns: 1004 appears with high frequency due to being a linguistic pun for "Angel" ($Cheonsa$) in North and South Korea.

  • Cultural/Historical Years: Years such as 2000 and 2001 spike due to historical memory markers (e.g., Y2K, I Love You virus era).

• Critical Visualization Evaluation:

  • Interactive Limitations: Lacks hover capabilities, making it impossible to identify exact numeric values corresponding to dark or unselected cells at the top-right margins.

  • Color Scale Inversion: Because black cells draw immediate visual focus, reversing the color scale would better highlight the safest (least common) PIN combinations if the visual goal is security selection.

Data Visualization Analysis: Russian-Ukraine War Casualties and Metrics

• Graphic Scope and Design:

  • Comprehensive multi-year tracking visualization comparing military spending, occupation area, human casualties, economic impact, and international military aid.

  • Casualties Representation: Uses concentric/nested proportional square boxes where the outer, larger box depicts total wounded personnel and the inner, bright box depicts soldiers killed.

• Key Observations & Omissions:

  • International Aid Perception: United States military aid appears small when measured as a percentage of national GDP, but represents the single largest total contribution when evaluated in absolute currency units.

  • Missing Contextual Factors: The graphic implies Ukraine is performing better based strictly on absolute casualty counts, but fails to incorporate relative baseline population scales (e.g., $90,000$ soldiers represents a significantly higher percentage of Ukraine's total population compared to Russia's active military reserve size).

  • Data Sources: Combines official direct estimates from both countries alongside energy export/gas production graphs dating back to $2022$.

Data Visualization Analysis: Vaccines and Autism Research

• Historical Scientific Literature Mapping:

  • Plots historical research publication timelines starting from early flawed papers (e.g., Wakefield) to massive population studies (e.g., Dale et al., studying sample populations exceeding $17,000$ individuals).

  • Text Term Frequency: Utilizes a word cloud/frequency chart displaying key terms across published pediatric and epidemiological literature.

• Critical Analysis of Conceptual Comparisons:

  • Flawed Equivalencies: The graphic attempts to compare aluminum concentration levels in dietary foods against aluminum in vaccines. This comparison is scientifically misleading because gastrointestinal ingestion and direct vascular/intramuscular injection carry entirely different physiological absorption pathways.

  • Genetic Factors: Highlights twin studies, which demonstrate high concordance rates for autism among identical twins, proving strong genetic heritability.

Data Visualization Analysis: Renewable Electricity Usage by Nation

• Data Structure & Geographic Distribution:

  • Maps global renewable energy adoption percentages per country, with a toggle filter to include or exclude hydroelectric power.

  • Dominant Renewable Nations: Canada, Brazil, Greenland, and Norway exhibit extremely high total renewable percentages heavily driven by natural water resources.

• Regional Highlights and Outliers:

  • Kenya: Hosts the largest single wind farm on the African continent.

  • Denmark: Reaches high renewable reliance ($30$ to $50$ range) primarily through wind generation and district heat infrastructure, even when completely excluding hydropower.

  • United States Regional Disparities: Washington state relies almost entirely on hydroelectric power (ranking poorly if hydro is removed). California leads in solar capacity, while New Hampshire derives nearly all its electrical power from non-hydro renewable sources.

Data Visualization Analysis: World Population Cell Diagram

• Visual Encoding Mechanics:

  • Replaces traditional pie charts with a circular Voronoi cell diagram/tessellation partitioned into individual country polygons.

  • Unit Scale: Raw cell numbers represent population measured in millions (e.g., India and China are mapped at approximately $1,500$ million or $1.5 \times 10^9$ individuals).

• Perceptual Advantages vs. Design Flaws:

  • Visual Perception Advantage: Human visual processing is inherently inaccurate at comparing angular slices in pie charts (a limitation documented in standard computational software like MATLAB). Irregular polygonal cell partitions allow clearer comparative judgments of relative area.

  • Inconsistent Hierarchy: The diagram mixes geographical scale levels, grouping some areas by individual nations and others by entire continental landmasses.

  • Text Density Issues: Small demographic regions (e.g., North Korea, which has a population of approximately $25,000,000$, similar to Australia) are rendered as tiny narrow slivers where overlay text becomes illegible and interaction controls prove clunky.

Data Visualization Analysis: Interactive Cocktail Mixology Guide

• Visualization Scope:

  • Entitled "$77$ Drinks That Every Bartender Should Know."

  • Data Matrix: Integrates glassware type, primary/secondary alcohol bases, mix methods, ingredient ratios, cost categories, and specific extra garnishes.

  • Functionality: Features interactive filter controls allowing users to dynamically isolate cocktails based on specific liquor ingredients, serving styles, or glass types.

Data Visualization Analysis: Evolution of Love Songs in Pop Music (1958–2023)

• Dataset & Classification Methodology:

  • Sample Corpus: $5,100$ global top-$10$ Billboard hits spanning from $1958$ to $2023$

  • Visual Scaling: Each song is depicted as a circular bubble where bubble area corresponds to total weeks spent inside the top-$10$ ranking.

  • Category Definitions:

  • Serenades: Defined strictly as explicit expressions of romantic devotion and unconditional love from one individual to another.

  • Broader Love Songs: Category encompassing heartache, breakup tracks, jealousy, and complex relational themes.

• Key Historical Trends:

  • Traditional Serenades: Experienced a sharp historical decline, dropping from $231\%$ down to $12\%$ of overall top-$10$ hits across the studied timeframe.

  • Overall Love Song Presence: When factoring in songs about heartache, infidelity, and relational angst, love-themed music continues to dominate pop charts, maintaining between $62\%$ and $69\%$ of all top-$10$ songs.

  • Structural Shift: Demonstrates that romantic music is not disappearing, but rather evolving from traditional ideal romantic serenades into nuanced emotional themes.

Data Visualization Analysis: New York Times 3D Economic Yield Curve and Visual Storytelling

• Graphic Features:

  • Renders a 3D topographic surface plot displaying shifts in Treasury yield curves over time to predict broader macroeconomic trends.

  • Storytelling Standard: Demonstrates self-contained visual storytelling. The graphic incorporates inline annotations (such as marking a $16\%$ increase in T-wave metrics) to convey clear analytical conclusions directly without requiring secondary explanatory prose.

Technical Troubleshooting: Document Sharing and Permissions

• Google Drive and Document Access Protocols:

  • Issue: Users encountering access restrictions ("can't edit" or locked view-only states) when attempting to access shared web documents.

  • Standard Resolution Procedure: Adjust visibility settings under document properties from restricted access to "Anyone with the link can view/edit," or download the base document locally to re-host permissions.