Exhaustive Study Notes on Exploratory Data Analysis, Graphical Displays, Software Debugging, and Contingency Tables

Categorical Data Analysis and Measures of Dispersion

  • Nominal Variables and Category Ordering:

    • Nominal variables possess no inherent logical ordering. Any arrangement of categories is completely arbitrary.
    • Changing the display sequence of nominal categories (e.g., listing car colors as black then white versus white then black, or reordering black, blue, green, yellow, orange, and navy) alters the table visual layout but does not change the underlying mathematical meaning.
    • Neither sequence is more correct than another for nominal data.

  • Frequency Tables as Measures of Dispersion:

    • For categorical variables, data is summarized by constructing frequency tables and relative frequency tables.
    • The frequency table itself serves as the measure of dispersion for categorical data. It illustrates how spread out or dispersive the data is by revealing the total number of distinct options and the exact count or rate at which each option occurs.

  • Measures of Center for Categorical Data:

    • The primary measure of center for a categorical variable is the mode, defined as the category that occurs most frequently.

Histograms and Quantitative Data Visualizations

  • Core Functions of a Histogram:

    • A histogram is the premier graphical display for a single quantitative variable.
    • It conveys three fundamental characteristics of a data distribution:
    • The number of major peaks (modality).
    • The presence and degree of visible skewness.
    • The existence of anomalies, such as individual outliers, or the explicit absence thereof.

  • Evaluating Modality and Shape:

    • Histograms must be evaluated holistically by examining the overall shape, rather than inspecting minor individual bins or small localized bumps.
    • Modality is determined by counting major ascents and descents across the distribution.
    • A peak spanning across two adjacent bins (e.g., a high plateau at value 33) constitutes a single major peak, making the graph unimodal.

  • Symmetry and Outliers:

    • Symmetry is assessed by comparing the number of bins on either side of the central peak.
    • Example: A distribution with 55 bins to the left and 55 bins to the right of the central peak is equidistant.
    • Even if visual imperfection prevents exact folding, such distributions are classified as approximately symmetrical.
    • If an outlier is present (e.g., an observation at 4040), all intermediate empty bins with zero frequency between the main body and the outlier must remain fully represented on the axis.

  • Bin Geometry and Midpoints:

    • Every bin (bar) in a histogram represents a continuous interval of quantitative values.
    • All bins within a single histogram must maintain identical width.
    • If a histogram begins at 00 and has its second bin boundary at 33, the intermediate boundary dividing the two bins is 1.51.5, creating uniform intervals of 00 to 1.51.5 and 1.51.5 to 33.

Graphical Selection, Binning Rules, and Software Syntax

  • Sources of Variation in Student Visualizations:

    • Multiple histograms constructed from the exact same dataset can be equally correct while appearing visually distinct.
    • Valid variations between student graphs include:
    • Chosen bin width (bandwidth).
    • Graphical style and color schemes.
    • Main titles and axis labels.
    • Axis scale increments.

  • Guidelines for Selecting Bins:

    • Avoid extreme bin selections: using only 22 bins or over 5050 bins provides poor structural detail.
    • Target an optimal range of 55 to 1010 bins per histogram.
    • Procedure: Find the minimum value and maximum value, calculate the total data range, and divide into equal intervals.
    • Example: For human height measured in inches with a minimum value of 5555\,\text{inches} and a maximum value of 7575\,\text{inches} (range = 2020\,\text{inches}):
    • Selecting a bin width of 1212\,\text{inches} is inappropriate because it yields only 22 (or at most 33) bins.
    • Selecting bin widths of 22\,\text{inches}, 2.52.5\,\text{inches}, or 33\,\text{inches} provides an informative display.\n\n- **Code Syntax for Bins**:\n - Bins are not restricted to whole integers; decimal values are completely valid (e.g., setting bin width to 2.5).\n - Software code implementation specifies parameter values directly (e.g., `binwidth = 2.5`), followed by required language syntax such as closing parentheses `)` or semicolons `;`.\n\n\n# Programming Practices and Debugging in R, Python, and SAS\n\n- **Code Writing vs. Troubleshooting Difficulty**:\n - Writing code for introductory statistical plots rates around 4toto5onaon a10−pointdifficultyscale(rangingfrom-point difficulty scale (ranging from2forexperiencedprogrammerstofor experienced programmers to7 for beginners).\n - Advanced algorithmic coding is reserved for 4000-level specialized Python or R courses.\n - Debugging syntax errors can reach an annoyance difficulty level of 26outofout of10.\n\n- **Common Programming Traps and Syntax Requirements**:\n - Semicolon omission: Omitting a single required semicolon can prevent script execution entirely.\n - Case sensitivity: Environment processing in R is 100\% case-sensitive. Using an uppercase letter instead of a lowercase letter triggers error messages such as `object not found`.\n\n- **Debugging Protocols**:\n - When code fails, pause and capture a screenshot containing both the written code and the exact error message generated.\n - Support options include asynchronous email reviews or direct screen-sharing sessions via Microsoft Teams.\n - More than 90\% of syntax errors are common, easily recognizable mistakes.\n - Uncommon edge cases account for less than 10\% of issues. Example: Importing data from Excel where a variable header contained an unexpected accented character (an "i" with an umlaut: `ï`) causing reference failures until import settings were modified.\n\n\n# Qualitative Visualizations: Pie Charts, Bar Charts, and Box Plots\n\n- **Pie Charts**:\n - Pie charts display relative frequencies as slices of a circle.\n - Limitation: They become unreadable when applied to variables with high counts of unique categories.\n - Example: Surveying moviegoers at a screening of the movie *Obsession* for exact age in years yields over 30distinctresponsecategories,producingacluttereddistinct response categories, producing a cluttered30-slice pie chart.\n - Optimal Use: Suitable only for categorical variables with few responses (e.g., 8toto9categories,oracategories, or a1−to−-to-10 rating scale).\n\n- **Bar Charts and Ordinal Data**:\n - Bar charts are preferred over pie charts for qualitative and especially **ordinal variables**.\n - Pie charts lose sequential ordering because a circle can begin at any arbitrary angle.\n - Bar charts preserve order because they possess a clear, defined starting point at the origin (0,0)onthecoordinateaxis(e.g.,displayinggradecategorieson the coordinate axis (e.g., displaying grade categoriesA, B, C, D, F in order).\n - Bar charts can be oriented vertically or horizontally.\n\n- **Box Plots (Box-and-Whisker Plots)**:\n - Box plots display summary quartiles and can be oriented horizontally or vertically.\n - Software implementations can overlay explicit symbols for both the mean and the median within the box.\n - **Proper Skewness Reasoning**:\n - Incorrect statement: "The data is skewed because the mean is greater than the median."\n - Correct statement: "The data is skewed because the visual display exhibits a skewed shape, which consequently causes the mean to be pulled beyond the median."\n\n- **Software Execution Capabilities**:\n - Modern statistical packages process datasets containing 1,000observationsinroughlyobservations in roughly23 ms(\,\text{ms} (0.023\,\text{seconds}).\n - Advanced statistical computing environments handle large-scale files containing up to 10,000,000observationsandobservations and300 variables.\n\n\n# Contingency Tables and Multi-Dimensional Relative Frequencies\n\n- **Contingency Table Definition**:\n - While a standard frequency table evaluates a single variable, a **contingency table** (also termed a **two-way table** or **cross-tabulation**) evaluates two categorical variables simultaneously.\n - It calculates cross-tabulations across intersecting variable levels.\n - Example: Evaluating Gender (Female, Male) against Work Location (Dallas, Norbrough) across a total sample size of N = 40 employees:\n - Count of Females working in Dallas = 13\n - Total Females = 20\n - Total Dallas Workers = 23\n\n- **The Four Contingency Tables**:\n 1. **Frequency Contingency Table**: Displays raw observation counts for each cell intersection.\n 2. **Percentage of Total Table**: Displays relative frequencies calculated against the grand total sample size (N = 40).Examplecellcalculation:). Example cell calculation:\frac{13}{40} = 32.5\%.\n 3. **Row Frequency Table (Percentage of Row)**:\n - Relative frequencies calculated using row marginal totals as denominators.\n - Example cell calculation: \frac{13}{20} = 65\%\n - Identified by row sums equaling 100\% at the end of every row.\n 4. **Column Frequency Table (Percentage of Column)**:\n - Relative frequencies calculated using column marginal totals as denominators.\n - Example cell calculation: \frac{13}{23} \approx 56.52\%\n - Identified by column sums equaling 100\%$$ at the bottom of every column.

  • Software Behavior and Table Title Requirements:

    • SAS automatically outputs all four contingency tables simultaneously in a single procedure execution.
    • R and Python require explicit, separate command instructions to generate each distinct table.
    • Every contingency table included in analytical reports must be explicitly titled and labeled (e.g., Table 1: Percentage of Total) with matching analytical interpretations.